POPULARITY
Michael and Nikolay are joined by Gülçin Yıldırım Jelínek and Robert Haas to discuss both the technical question of whether or not pg_dump is a backup tool, as well as the tone and intent behind the statement "pg_dump is not a backup tool". Here are some links to things they mentioned:Gülçin Yıldırım Jelínek https://postgres.fm/people/gulcin-yildirim-jelinekRobert Haas https://postgres.fm/people/robert-haasWhy you should upgrade PostgreSQL today (blog post by Gülçin) https://xata.io/blog/cve-2024-7348-postgres-upgradeIf pg_dump is not a backup tool, what is? (blog post by Gülçin) https://xata.io/blog/pgdump-is-not-a-backup-toolIs pg_dump a backup tool? (blog post by Robert) https://rhaas.blogspot.com/2024/10/is-pgdump-backup-tool.html?m=1Why pg_dump is amazing (blog post by Robert) https://rhaas.blogspot.com/2024/11/why-pgdump-is-amazing.htmlAvoid too prominent use of "backup" on pg_dump man page (commit by Peter Eisentraut) https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=4f29394ea941f688fd4faf7260d2c198931ca797Is pg_dump a backup tool? (poll by Nikolay with options Yes / No / Define backup) https://x.com/samokhvalov/status/1847015453056786771What's the best way to make a backup (a recent example discussion on Reddit) https://www.reddit.com/r/PostgreSQL/comments/1gu4r05/whats_the_best_way_to_make_a_backup/Hackers mailing list https://www.postgresql.org/list/pgsql-hackers/ Praise, Criticism, and Dialogue (blog post by Robert) https://rhaas.blogspot.com/2023/12/praise-criticism-and-dialogue.html Out-of-cycle release scheduled for November 21, 2024 https://www.postgresql.org/about/news/out-of-cycle-release-scheduled-for-november-21-2024-2958/ pgBackRest https://github.com/pgbackrest/pgbackrest Barman https://github.com/EnterpriseDB/barman Our previous episode on backups https://postgres.fm/episodes/backups ~~~What did you like or not like? What should we discuss next time? Let us know via a YouTube comment, on social media, or by commenting on our Google doc!~~~Postgres FM is produced by:Michael Christofides, founder of pgMustardNikolay Samokhvalov, founder of Postgres.aiWith special thanks to:Jessie Draws for the elephant artwork
Bharath Rupireddy has carved a niche for himself in the Postgres community since he began using the database system back in 2020. From his start at EnterpriseDB to making strides at Microsoft, and now contributing to the AWS open-source project, Bharath is entrenched in the inner workings of Postgres development. He has worked in many areas of Postgres such as WAL, Replication, pg_walinspect extension, bug fixes, performance improvements, new SQL functions etc.In this episode we explore:Starting with Postgres in 2020 and learning the basics while contributing to Parallel COPY featurePostgres recent features such as pg_walinspect extension, WAL source switch from archive to streaming, WAL insertion lock improvements, WAL read from buffers, more replication slot invalidation mechanisms like XID age based and inactive_timeout based ones, new table access methods for multi inserts.The future of native active-active replicationThe complexities of conflict resolutionUsing PostgreSQL 16 in production scenarios for active-active replicationThe role of databases evolving within different industriesLinks mentioned:¡Databases! – A Database Seminar SeriesHacking Postgres, Ep. 9: Bertrand DrouvotPGConf IndiaHyderabad PostgreSQL User GroupPGConf.devBharath on LinkedInBharath on X (@BRupireddy)Bharath on Github
Irish-based sales performance, efficiency and planning start-up Lative has raised $3 million in seed funding led by Elkstone Ventures to automate the B2B sales planning and execution process in real-time. Lative has developed powerful and easy-to-use cloud-based sales performance, efficiency and planning software that helps B2B revenue teams monitor and improve sales performance. It allows users to understand their sales performance, cost efficiency and return on investment (ROI) with real-time risk and opportunity assessment, as well as easily simulate future capacity and investment plans with far higher accuracy than how it's currently done within organisations today. The software connects top-down revenue plans and targets to bottoms-up performance and costs, empowering companies to boost sales productivity, maximise every dollar spent, and plan with precision and purpose. Lative recognises the need for combining planning with execution within sales organisations and has built the platform to solve this need. Historically, this has been a top-down approach which Lative acknowledges is an inaccurate and inefficient method and leads to expensive mistakes. Instead, the company takes the customers' actual bottom-up data in real-time in order to build accurate capacity plans and see the impact of strategic initiatives on sales productivity and return on investment. This is not a once-off solution, but can be used consistently to monitor sales performance and efficiency as the company grows and changes. Currently, there are a growing number of companies on the platform including EnterpriseDB, Oneflow and Saleshood. Lative will continue to improve and deepen their integrations with strategic technology partners Salesforce and Snowflake, as well as continue to expand their broader integration partner ecosystem with a range of CRMs, ERPs, Finance and HR systems. With the funding, Lative intends to invest in its go-to-market strategy and further development of the platform. This funding will also enable Lative to build an integral insights and benchmarking module which will improve decision-making by sales teams. Lative was co-founded in 2022 by former colleagues Werner Schmidt (led revenue operations and sales enablement for high-growth SaaS businesses like Sage and Citrix), and Laura Tortosa Sancho (led multiple sales operations functions at Sage and was a senior consultant at Deloitte). Both founders have extensive revenue operations backgrounds and consistently struggled firsthand with the issues that Lative now solves. Werner Schmidt, CEO & co-founder of Lative said, "As economic conditions have changed in recent years, for B2B sales teams sales productivity and efficiency has gone from an afterthought to a critical success factor. Lative connects revenue plans to performance and costs in real-time so that you can focus on making the right decisions or course correct at the right times to achieve your revenue and profitability goals." Elkstone Ventures (Flipdish, LetsGetChecked, Manna) led the funding round, with participation from Enterprise Ireland, new investor Handshake Ventures (Beautiful.ai, 1Qbit, SteamElements), and existing investors WestWave Capital, Future Five and various Scouts and Angels from Sequoia, EQT, Sage, GoTo, Contentful and others were also involved in the funding round, including Kasper Hulthin (founder of Peakon, Podio and Kost) and Vernon Bubb (Stage2 Capital, Managing Director EMEA at Clari) who are both on the board of Lative, and Warren Weiss (founder, GP at WestWave Capital). Niall McEvoy, Partner at Elkstone Ventures said, "Lative is making sales teams more efficient, by replacing spreadsheets with software that can drastically improve sales results. The platform is potentially category-defining for sales team management. Werner and Laura are real domain experts with decades of relevant sector experience. Customers speak highly of the transformational impact of the solution " Jenny Melia, Execut...
Nikolay and Michael discuss Postgres backups — why we need them, what the options are, whether a dump is a backup or not, and some considerations for lowering RPO and RTO at scale. Here are some links to some extra things they mentioned:pg_dump https://www.postgresql.org/docs/current/app-pgdump.html pg_basebackup https://www.postgresql.org/docs/current/app-pgbasebackup.htmlpgBackRest https://github.com/pgbackrest/pgbackrest WAL-G https://github.com/wal-g/wal-g Barman https://github.com/EnterpriseDB/barman Data loss at GitLab (2017) https://about.gitlab.com/blog/2017/02/01/gitlab-dot-com-database-incident/ Dev Deletes Entire Production Database, Chaos Ensues (YouTube video) https://www.youtube.com/watch?v=tLdRBsuvVKc Our episode on corruption https://postgres.fm/episodes/corruption DBLab Engine https://github.com/postgres-ai/database-lab-engine ~~~What did you like or not like? What should we discuss next time? Let us know via a YouTube comment, on social media, or by commenting on our Google doc!~~~Postgres FM is brought to you by:Nikolay Samokhvalov, founder of Postgres.aiMichael Christofides, founder of pgMustardWith special thanks to:Jessie Draws for the amazing artwork
Nikolay and Michael discuss self-managing Postgres — both the practicalities of doing so, as well as some managed-service style tooling. Here are some links to some things they mentioned:Our episode on Managed services vs. DIY https://postgres.fm/episodes/managed-services-vs-diy WAL-G https://github.com/wal-g/wal-g pgBackRest https://pgbackrest.org/ Barman https://github.com/EnterpriseDB/barman Dead Man's Snitch https://deadmanssnitch.com/ Netdata https://www.netdata.cloud/ Upgrades https://postgres.fm/episodes/upgrades High availability https://postgres.fm/episodes/high-availability Configuration https://postgres.fm/episodes/default-configuration Corruption https://postgres.fm/episodes/corruption Connection poolers https://postgres.fm/episodes/connection-poolers Index maintenance https://postgres.fm/episodes/index-maintenance StackGres supported extensions (Michael was wrong, it also has a timescale_tls extension!) https://stackgres.io/extensions/ postgresql_cluster https://github.com/vitabaks/postgresql_cluster Supabase self-hosting https://supabase.com/docs/guides/self-hostingTembo https://github.com/tembo-io/tembo Open source licenses, clouds, Postgres (Postgres TV discussion) https://www.youtube.com/watch?v=1rcbyIjA4gI&t=149s ~~~What did you like or not like? What should we discuss next time? Let us know via a YouTube comment, on social media, or by commenting on our Google doc!~~~Postgres FM is brought to you by:Nikolay Samokhvalov, founder of Postgres.aiMichael Christofides, founder of pgMustardWith special thanks to:Jessie Draws for the amazing artwork
About AnaïsAnaïs is a Developer Advocate at Aqua Security, where she contributes to Aqua's cloud native open source projects. When she is not advocating DevOps best practices, she runs her own YouTube Channel centered around cloud native technologies. Before joining Aqua, Anais worked as SRE at Civo, a cloud native service provider, where she helped enhance the infrastructure for hundreds of tenant clusters. As CNCF ambassador of the year 2021, her passion lies in making tools and platforms more accessible to developers and community members.Links Referenced: Aqua Security: https://www.aquasec.com/ Aqua Open Source YouTube channel: https://www.youtube.com/c/AquaSecurityOpenSource Personal blog: https://anaisurl.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at AWS AppConfig. Engineers love to solve, and occasionally create, problems. But not when it's an on-call fire-drill at 4 in the morning. Software problems should drive innovation and collaboration, NOT stress, and sleeplessness, and threats of violence. That's why so many developers are realizing the value of AWS AppConfig Feature Flags. Feature Flags let developers push code to production, but hide that that feature from customers so that the developers can release their feature when it's ready. This practice allows for safe, fast, and convenient software development. You can seamlessly incorporate AppConfig Feature Flags into your AWS or cloud environment and ship your Features with excitement, not trepidation and fear. To get started, go to snark.cloud/appconfig That's snark.cloud/appconfig.Corey: This episode is sponsored in part by Honeycomb. When production is running slow, it's hard to know where problems originate. Is it your application code, users, or the underlying systems? I've got five bucks on DNS, personally. Why scroll through endless dashboards while dealing with alert floods, going from tool to tool to tool that you employ, guessing at which puzzle pieces matter? Context switching and tool sprawl are slowly killing both your team and your business. You should care more about one of those than the other; which one is up to you. Drop the separate pillars and enter a world of getting one unified understanding of the one thing driving your business: production. With Honeycomb, you guess less and know more. Try it for free at honeycomb.io/screaminginthecloud. Observability: it's more than just hipster monitoring.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Every once in a while, when I start trying to find guests to chat with me and basically suffer my various slings and arrows on this show, I encounter something that I've never really had the opportunity to explore further. And today's guest leads me in just such a direction. Anaïs is an open-source developer advocate at Aqua Security, and when I was asking her whether or not she wanted to talk about various topics, one of the first thing she said was, “Don't ask me much about AWS because I've never used it,” which, oh my God. Anaïs, thank you for joining me. You must be so very happy never to have dealt with the morass of AWS.Anaïs: [laugh]. Yes, I'm trying my best to stay away from it. [laugh].Corey: Back when I got into the cloud space, for lack of a better term, AWS was sort of really the only game in town unless you wanted to start really squinting hard at what you define cloud as. I mean yes, I could have gone into Salesforce or something, but I was already sad and angry all the time. These days, you can very much go all in-on cloud. In fact, you were a CNCF ambassador, if I'm not mistaken. So, you absolutely are in the infrastructure cloud space, but you haven't dealt with AWS. That is just an interesting path. Have you found others who have gone down that same road, or are you sort of the first of a new breed?Anaïs: I think to find others who are in a similar position or have a similar experience, as you do, you first have to talk about your experience, and this is the first time, or maybe the second, that I'm openly [laugh] saying it on something that will be posted live, like, to the internet. Before I, like, I tried to stay away from mentioning it at all, do the best that I can because I'm at this point where I'm so far into my cloud-native Kubernetes journey that I feel like I should have had to deal with AWS by now, and I just didn't. And I'm doing my best and I'm very successful in avoiding it. [laugh]. So, that's where I am. Yeah.Corey: We're sort of on opposite sides of a particular fence because I spend entirely too much time being angry at AWS, but I've never really touched Kubernetes and anger. I mean, I see it in a lot of my customer accounts and I get annoyed at its data transfer bills and other things that it causes in an economic sense, but as far as the care and feeding of a production cluster, back in my SRE days, I had very old-school architectures. It's, “Oh, this is an ancient system, just like grandma used to make,” where we had the entire web tier, then a job applic—or application server tier, and then a database at the end, and everyone knew where everything was. And then containers came out of nowhere, and it seemed like okay, this solves a bunch of problems and introduces a whole bunch more. How do I orchestrate them? How do I ensure that they're healthy?And then ah, Kubernetes was the answer. And for a while, it seemed like no matter what the problem was, Kubernetes was going to be the answer because people were evangelizing it pretty hard. And now I see it almost everywhere that I turn. What's your journey been like? How did you get into the weeds of, “You know what I want to do when I grow up? That's right. I want to work on container orchestration systems.” I have a five-year-old. She has never once said that because I don't abuse my children by making them learn how clouds work. How did you wind up doing what you do?Anaïs: It's funny that you mention that. So, I'm actually of the generation of engineers who doesn't know anything else but Kubernetes. So, when you mentioned that you used to use something before, I don't really know what that looks like. I know that you can still deploy systems without Kubernetes, but I have no idea how. My journey into the cloud-native space started out of frustration from the previous industry that I was working at.So, I was working for several years as developer advocate in the open-source blockchain cryptocurrency space and it's highly similar to all of the cliches that you hear online and across the news. And out of this frustration, [laugh] I was looking at alternatives. One of them was either going into game development, into the gaming industry, or the cloud-native space and infrastructure development and deployment. And yeah, that's where I ended up. So, at the end of 2020, I joined a startup in the cloud-native space and started my social media journey.Corey: One of the things that I found that Kubernetes solved for—and to be clear, Kubernetes really came into its own after I was doing a lot more advisory work and a lot more consulting style activity rather than running my own environments, but there's an entire universe of problems that the modern day engineer never has to think about due to, partially cloud and also Kubernetes as well, which is the idea of hardware or node failure. I've had middle of the night driving across Los Angeles in a panic getting to the data center because the disk array on the primary database had degraded because the drive failed. That doesn't happen anymore. And clouds have mostly solved that. It's okay, drives fail, but yeah, that's the problem for some people who live in Virginia or Oregon. I don't have to think about it myself.But you do have to worry about instances failing; what if the primary database instance dies? Well, when everything lives in a container then that container gets moved around in the stateless way between things, well great, you really only have to care instead about okay, what if all of my instances die? Or, what if my code is really crappy? To which my question is generally, what do you mean, ‘if?' All of us write crappy code.That's the nature of the universe. We open-source only the small subset that we are not actively humiliated by, which is, in a lot of ways, what you're focusing on now, over at Aqua Sec, you are an advocate for open-source. One of the most notable projects that come out of that is Trivy, if I'm pronouncing that correctly.Anaïs: Yeah, that's correct. Yeah. So, Trivy is our main open-source project. It's an all-in-one cloud-native security scanner. And it's actually—it's focused on misconfiguration issues, so it can help you to build more robust infrastructure definitions and configurations.So ideally, a lot of the things that you just mentioned won't happen, but it obviously, highly depends on so many different factors in the cloud-native space. But definitely misconfigurations of one of those areas that can easily go wrong. And also, not just that you have data might cease to exist, but the worst thing or, like, as bad might be that it's completely exposed online. And they are databases of different exposures where you can see all the kinds of data of information from just health data to dating apps, just being online available because the IP address is not protected, right? Things like that. [laugh].Corey: We all get those emails that start with, “Your security is very important to us,” and I know just based on that opening to an email, that the rest of that email is going to explain how security was not very important to you folks. And it's the apology, “Oops, we have messed up,” email. Now, the whole world of automated security scanners is… well, it's crowded. There are a number of different services out there that the cloud providers themselves offer a bunch of these, a whole bunch of scareware vendors at the security conferences do as well. Taking a quick glance at Trivy, one of the problems I see with it, from a cloud provider perspective, is that I see nothing that it does that winds up costing extra money on your cloud bill that you then have to pay to the cloud provider, so maybe they'll put a pull request in for that one of these days. But my sarcasm aside, what is it that differentiates Trivy from a bunch of other offerings in various spaces?Anaïs: So, there are multiple factors. If we're looking from an enterprise perspective, you could be using one of the in-house scanners from any of the cloud providers available, depending which you're using. The thing is, they are not generally going to be the ones who have a dedicated research team that provides the updates based on the vulnerabilities they find across the space. So, with an open-source security scanner or from a dedicated company, you will likely have more up-to-date information in your scans. Also, lots of different companies, they're using Trivy under the hood ultimately, or for their own scans.I can link a few where you can also find them in a Trivy repository. But ultimately, a lot of companies rely on Trivy and other open-source security scanners under the hood because they are from dedicated companies. Now, the other part to Trivy and why you might want to consider using Trivy is that in larger teams, you will have different people dealing with different components of your infrastructure, of your deployments, and you could end up having to use multiple different security scanners for all your different components from your container images that you're using, whether or not they are secure, whether or not they're following best practices that you defined to your infrastructure-as-code configurations, to you're running deployments inside of your cluster, for instance. So, each of those different stages across your lifecycle, from development to runtime, will maybe either need different security scanners, or you could use one security scanner that does it all. So, you could have in a team more knowledge sharing, you could have dedicated people who know how to use the tool and who can help out across a team across the lifecycle, and similar. So, that's one of the components that you might want to consider.Another thing is how mature is a tool, right? A lot of cloud providers, what they end up doing is they provide you with a solution, but it's nice to decoupled from anything else that you're using. And especially in the cloud-native space, you're heavily reliant on open-source tools, such as for your observability stack, right? Coming from Site Reliability Engineering also myself, I love using metrics and Grafana. And for me, if anything open-source from Loki to accessing my logs, to Grafana to dashboards, and all their integrations.I love that and I want to use the same tools that I'm using for everything else, also for my security tools. I don't want to have the metrics for my security tools visualized in a different solution to my reliability metrics for my application, right? Because that ultimately makes it more difficult to correlate metrics. So, those are, like, some of the factors that you might want to consider when you're choosing a security scanner.Corey: When you talk about thinking about this, from the perspective of an SRE is—I mean, this is definitely an artifact of where you come from and how you approach this space. Because in my world, when you have ten web servers, five application servers, and two database servers and you wind up with a problem in production, how do you fix this? Oh, it's easy. You log into one of those nodes and poke around and start doing diagnostics in production. In a containerized world, you generally can't do that, or there's a problem on a container, and by the time you're aware of that, that container hasn't existed for 20 minutes.So, how do you wind up figuring out what happens? And instrumenting for telemetry and metrics and observability, particularly at scale becomes way more important than it ever was, for me. I mean, my version of monitoring was always Nagios, which was the original Call of Duty that wakes you up at two in the morning when the hard drive fails. The world has thankfully moved beyond that and a bunch of ways. But it's not first nature for me. It's always, “Oh, yeah, that's right. We have a whole telemetry solution where I can go digging into.” My first attempt is always, oh, how do I get into this thing and poke it with a stick? Sometimes that's helpful, but for modern applications, it really feels like it's not.Anaïs: Totally. When we're moving to an infrastructure to an environment where we can deploy multiple times a day, right, and update our application multiple times a day, multiple times a day, we can introduce new security issues or other things can go wrong, right? So, I want to see—as much as I want to see all of the other failures, I want to see any security-related issues that might be deployed alongside those updates at the same frequency, right?Corey: The problem that I see across all this stuff, though, is there are a bunch of tools out there that people install, but then don't configure because, “Oh, well, I bought the tool. The end.” I mean, I think it was reported almost ten years ago or so on the big Target breach that they wound up installing some tool—I want to say FireEye, but please don't quote me on that—and it wound up firing off a whole bunch of alerts, and they figured was just noise, so they turned it all off. And it turned out no, no, this was an actual breach in progress. But people are so used to all the alarms screaming at them, that they don't dig into this.I mean, one of the original security scanners was Nessus. And I seen a lot of Nessus reports because for a long time, what a lot of crappy consultancies would do is they would white-label the output of whatever it was that Nessus said and deliver that in as the report. So, you'd wind up with 700 pages of quote-unquote, “Security issues.” And you'd have to flip through to figure out that, ah, this supports a somewhat old SSL negotiation protocol, and you're focusing on that instead of the oh, and by the way, the primary database doesn't have a password set. Like, it winds up just obscuring it because there is so much. How does Trivy approach avoiding the information overload problem?Anaïs: That's a great question because everybody's complaining about vulnerability fatigue, of them, for the first time, scanning their container images and workloads and seeing maybe even hundreds of vulnerabilities. And one of the things that can be done to counteract that right from the beginning is investing your time into looking at the different flags and configurations that you can do before actually deploying Trivy to, for example, your cluster. That's one part of it. The other part is I mentioned earlier, you would use a security scan at different parts of your deployment. So, it's really about integrating scanning not just once you—like, in your production environment, once you've deployed everything, but using it already before and empowering engineers to actually use it on their machines.Now, they can either decide to do it or not; it's not part of most people's job to do security scanning, but as you move along, the more you do, the more you can reduce the noise and then ultimately, when you deploy Trivy, for example, inside of your cluster, you can do a lot of configuration such as scanning just for critical vulnerabilities, only scanning for vulnerabilities that already have a fix available, and everything else should be ignored. Those are all factors and flags that you can place into Trivy, for instance, and make it easier. Now, with Trivy, you won't have automated PRs and everything out of the box; you would have to set up the actions or, like, the ways to mitigate those vulnerabilities manually by yourself with tools, as well as integrating Trivy with your existing stack, and similar. But then obviously, if you want to have something more automated, if you want to have something that does more for you in the background, that's when you want to use to an enterprise solution and shift to something like Aqua Security Enterprise Platform that actually provides you with the automated way of mitigating vulnerabilities where you don't have to know much about it and it just gives you the solution and provides you with a PR with the updates that you need in your infrastructure-as-code configurations to mitigate the vulnerability [unintelligible 00:15:52]?Corey: I think that's probably a very fair answer because let's be serious when you're running a bank or someone for whom security matters—and yes, yes, I know, security should matter for everyone, but let's be serious, I care a little bit less about the security impact of, for example, I don't know, my Twitter for Pets nonsense, than I do a dating site where people are not out about their orientation or whatnot. Like, there is a world of difference between the security concerns there. “Oh, no, you might be able to shitpost as me if you compromise my lasttweetinaws.com Twitter client that I put out there for folks to use.” Okay, great. That is not the end of the world compared to other stuff.By the time you're talking about things that are critically important, yeah, you want to spend money on this, and you want to have an actual full-on security team. But open-source tools like this are terrific for folks who are just getting started or they're building something for fun themselves and as it turns out, don't have a full security budget for their weird late-night project. I think that there's a beautiful, I guess, spectrum, as far as what level of investment you can make into security. And it's nice to see the innovation continued happening in the space.Anaïs: And you just mentioned that dedicated security companies, they likely have a research team that's deploying honeypots and seeing what happens to them, right? Like, how are attackers using different vulnerabilities and misconfigurations and what can be done to mitigate them. And that ultimately translates into the configurations of the open-source tool as well. So, if you're using, for instance, a security scanner that doesn't have an enterprise company with a research team behind it, then you might have different input into the data of that security scanner than if you do, right? So, these are, like, additional considerations that you might want to take when choosing a scanner. And also that obviously depends on what scanning you want to do, on the size of your company, and similar, right?Corey: This episode is sponsored in part by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on-premises, private cloud, and they just announced a fully-managed service on AWS and Azure called BigAnimal, all one word. Don't leave managing your database to your cloud vendor because they're too busy launching another half-dozen managed databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications—including Oracle—to the cloud. To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: Something that I do find fairly interesting is that you started off, as you say, doing DevRel in the open-source blockchain world, then you went to work as an SRE, and then went back to doing DevRel-style work. What got you into SRE and what got you out of SRE, other than the obvious having worked in SRE myself and being unhappy all the time? I kid, but what was it that got you into that space and then out of it?Anaïs: Yeah. Yeah, but no, it's a great question. And it's, I guess, also was shaped my perspective on different tools and, like, the user experience of different tools. But ultimately, I first worked in the cloud-native space for an enterprise tool as developer advocate. And I did not like the experience of working for a paid solution. Doing developer advocacy for it, it felt wrong in a lot of ways. A lot of times you were required to do marketing work in those situations.And that kind of got me out of developer advocacy into SRE work. And now I was working partially or mainly as SRE, and then on the side, I was doing some presentations in developer advocacy. However, that split didn't quite work, either. And I realized that the value that I add to a project is really the way I convey information, which I can't do if I'm busy fixing the infrastructure, right? I can't convey the information of as much of how the infrastructure has been fixed as I can if I'm working with an engineering team and then doing developer advocacy, solely developer advocacy within the engineering team.So, how I ultimately got back into developer advocacy was just simply by being reached out to by my manager at Aqua Security, and Itay telling me, him telling me that he has a role available and if I want to join his team. And it was open-source-focused. Given that I started my career for several years working in the open-source space and working with engineers, contributing to open-source tools, it was kind of what I wanted to go back to, what I really enjoy doing. And yeah, that's how that came about [laugh].Corey: For me, I found that I enjoy aspects of the technology part, but I find I enjoy talking to people way more. And for me, the gratifying moment that keeps me going, believe it or not, is not necessarily helping giant companies spend slightly less money on another giant company. It's watching people suddenly understand something they didn't before, it's watching the light go on in their eyes. And that's been addictive to me for a long time. I've also found that the best way for me to learn something is to teach someone else.I mean, the way I learned Git was that I foolishly wound up proposing a talk, “Terrible Ideas in Git”—we'll teach it by counterexample—four months before the talk. And they accepted it, and crap, I'd better learn enough get to give this talk effectively. I don't recommend this because if you miss the deadline, I checked, they will not move the conference for you. But there really is something to be said for watching someone learn something by way of teaching it to them.Anaïs: It's actually a common strategy for a lot of developer advocates of making up a talk and then waiting whether or not it will get accepted. [laugh] and once it gets accepted, that's when you start learning the tool and trying to figure it out. Now, it's not a good strategy, obviously, to do that because people can easily tell that you just did that for a conference. And—Corey: Sounds to me, like, you need to get better at bluffing. I kid.Anaïs: [laugh].Corey: I kid. Don't bluff your way through conference talks as a general rule. It tends not to go well. [laugh].Anaïs: No. It's a bad idea. It's a really bad idea. And so, I ultimately started learning the technologies or, like, the different tools and projects in the cloud-native space. And there are lots, if you look at the CNCF landscape, right? But just trying to talk myself through them on my YouTube channel. So, my early videos on my channel, it's just very much on the go of me looking for the first time at somebody's documentation and not making any sense out of them.Corey: It's surprising to me how far that gets you. I mean, I guess I'm always reminded of that Tom Hanks movie from my childhood Big where he wakes up—the kid wakes up as an adult one day, goes to work, and bluffs his way into working at a toy company. He's in a management meeting and just they're showing their new toy they're going to put out there and he's, “I don't get it.” Everyone looks at him like how dare you say it? And, “I don't get it. What's fun about this?” Because he's a kid.And he wants to getting promoted to vice president because wow, someone pointed out the obvious thing. And so often, it feels like using a tool or a product, be it open-source or enterprise, it is clearly something different in my experience of it when I try to use this thing than the person who developed it. And very often it's that I don't see the same things or think of the problem space the same way that the developers did, but also very often—and I don't mean to call anyone in particular out here—it's a symptom of a terrible user interface or user experience.Anaïs: What you've just said, a lot of times, it's just about saying the thing that nobody that dares to say or nobody has thought of before, and that gets you obviously, easier, further [laugh] then repeating what other people have already mentioned, right? And a lot of what you see a lot of times in these—also an open-source projects, but I think more even in closed-source enterprise organizations is that people just repeat whatever everybody else is saying in the room, right? You don't have that as much in the open-source world because you have more input or easier input in public than you do otherwise, but it still happens that I mean, people are highly similar to each other. If you're contributing to the same project, you probably have a similar background, similar expertise, similar interests, and that will get you to think in a similar way. So, if there's somebody like, like a high school student maybe, somebody just graduated, somebody from a completely different industry who's looking at those tools for the first time, it's like, “Okay, I know what I'm supposed to do, but I don't understand why I should use this tool for that.” And just pointing that out, gets you a response, most of the time. [laugh].Corey: I use Twitter and use YouTube. And obviously, I bias more for short, pithy comments that are dripping in sarcasm, whereas in a long-form video, you can talk a lot more about what you're seeing. But the problem I have with bad user experience, particularly bad developer experience, is that when it happens to me—and I know at a baseline level, that I am reasonably competent in technical spaces, but when I encounter a bad interface, my immediate instinctive reaction is, “Oh, I'm dumb. And this thing is for smart people.” And that is never, ever true, except maybe with quantum computing. Great, awesome. The Hello World tutorial for that stuff is a PhD from Berkeley. Good luck if you can get into that. But here in the real world where the rest of us play, it's just a bad developer experience, but my instinctive reaction is that there's stuff I don't know, and I'm not good enough to use this thing. And I get very upset about that.Anaïs: That's one of the things that you want to do with any technical documentation is that the first experience that anybody has, no matter the background, with your tool should be a success experience, right? Like people should look at it, use maybe one command, do one thing, one simple thing, and be like, “Yeah, this makes sense,” or, like, this was fun to do, right? Like, this first positive interaction. And it doesn't have to be complex. And that's what many people I think get wrong, that they try to show off how powerful a tool is, of like, oh, “My God, you can do all those things. It's so exciting, right?” But [laugh] ultimately, if nobody can use it or if most of the people, 99% of the people who try it for the first time have a bad experience, it makes them feel uncomfortable or any negative emotion, then it's really you're approaching it from the wrong perspective, right?Corey: That's very apt. I think it's so much of whether people stick with something long enough to learn it and find the sharp edges has to do with what their experience looks like. I mean, back when I was more or less useless when it comes to anything that looked like programming—because I was a sysadmin type—I started contributing to SaltStack. And what was amazing about that was Tom Hatch, the creator of the project had this pattern that he kept up for way too long, where whenever anyone submitted an issue, he said, “Great, well, how about you fix it?” And because we had a patch, like, “Well, I'm not good at programming.” He's like, “That's okay. No one is. Try it and we'll see.”And he accepted every patch and then immediately, you'd see another patch come in ten minutes later that fixed the problems in your patch. But it was the most welcoming and encouraging experience, and I'm not saying that's a good workflow for an open-source maintainer, but he still remains one of the best humans I know, just from that perspective alone.Anaïs: That's amazing. I think it's really about pointing out that there are different ways of doing open-source [laugh] and there is no one way to go about it. So, it's really about—I mean, it's about the community, ultimately. That's what it boils down to, of you are dependent, as an open-source project, on the community, so what is the best experience that you can give them? If that's something that you want to and can invest in, then yeah [laugh] that's probably the best outcome for everybody.Corey: I do have one more question, specifically around things that are more timely. Now, taking a quick look at Trivy and recent features, it seems like you've just now—now-ish—started supporting cloud scanning as well. Previously, it was effectively, “Oh, this scans configuration and containers. Okay, great.” Now, you're targeting actually scanning cloud providers themselves. What does this change and what brought you to this place, as someone who very happily does not deal with AWS?Anaïs: Yeah, totally. So, I just started using AWS, specifically to showcase this feature. So, if you look at the Aqua Open Source YouTube channel, you will find several tutorials that show you how to use that feature, among others.Now, what I mentioned earlier in the podcast already is that Trivy is really versatile, it allows you to scan different aspects of your stack at different stages of your development lifecycle. And that's made possible because Trivy is ultimately using different open-source projects under the hood. For example, if you want to scan your infrastructure-as-code misconfigurations, it's using a tool called tfsec, specifically for Terraform. And then other tools for other scanning, for other security scanning. Now, we have—or had; it's going to be probably deprecated—a tool called CloudSploit in the Aqua open-source project suite.Now, that's going to, kind of like, the functionality that CloudSploit was providing is going to get converted to become part of Trivy, so everything scanning-related is going to become part of Trivy that really, like, once you understand how Trivy works and all of the CLI commands in Trivy have exactly the same structure, it's really easy to scan from container images to infrastructure-as-code, to generating s-bombs to scanning also now, your cloud infrastructure and Trivy can scan any of your AWS services for misconfigurations, and it's using basically the AWS client under the hood to connect with the services of everything you have set up there, and then give you the list of misconfigurations. And once it has done the scan, you can then drill down further into the different aspects of your misconfigurations without performing the entire scan again, since you likely have lots and lots of resources, so you wouldn't want to scan them every time again, right, when you perform the scan. So, once something has been scanned, Trivy will know whether the resource changed or not, it won't scan it again. That's the same way that in-classes scanning works right now. Once a container image has been scanned for vulnerabilities, it won't scan the same container image again because that would just waste time. [laugh]. So yeah, do check it out. It's our most recent feature, and it's going to come out also to the other cloud providers out there. But we're starting with AWS and this kind of forced me to finally [laugh] look at it for the sake of it. But I'm not going to be happy. [laugh].Corey: No, I don't think anyone is. It's every time I see on a resume that someone says, “Oh, I'm an expert in AWS,” it's, “No you're not.” They have 400-some-odd services now. We have crossed the point long ago, where I can very convincingly talk about AWS services that do not exist to Amazonians and not get called out for it because who in the world knows what they run? And half of their services sound like something I made up to be funny, but they're very real. It's wild to me that it is a sprawling as it is and apparently continues to work as a viable business.But no one knows all of it and everyone feels confused, lost, and overwhelmed every time they look at the AWS console. This has been my entire career in life for the last six years, and I still feel that way. So, I'm sure everyone else does, too.Anaïs: And this is how misconfigurations happen, right? You're confused about what you're actually supposed to do and how you're supposed to do it. And that's, for example, with all the access rights in Google Cloud, something that I'm very familiar with, that completely overwhelms you and you get super frustrated by, and you don't even know what you give access to. It's like, if you've ever had to configure Discord user roles, it's a similar disaster. You will not know which user has access to which. They kind of changed it and try to improve it over the past year, but it's a similar issue that you face in cloud providers, just on a much larger-scale, not just on one chat channel. [laugh]. So.Corey: I think that is probably a fair place to leave it. I really want to thank you for spending as much time with me as you have talking about the trials and travails of, well, this industry, for lack of a better term. If people want to learn more, where's the best place to find you?Anaïs: So, I have a weekly DevOps newsletter on my blog, which is anaisurl—like, how you spell U-R-L—and then dot com. anaisurl.com. That's where I have all the links to my different channels, to all of the resources that are published where you can find out more as well. So, that's probably the best place. Yeah.Corey: And we will, of course, put a link to that in the show notes. I really want to thank you for being as generous with your time as you have been. Thank you.Anaïs: Thank you for having me. It was great.Corey: Anaïs, open-source developer advocate at Aqua Security. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry, insulting comment that I will never see because it's buried under a whole bunch of minor or false-positive vulnerability reports.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About AlexAlex is the Chief Product Officer of Twingate, which he cofounded in 2019. Alex has held a range of product leadership roles in the enterprise software market over the last 16 years, including at Dropbox, where he was the first enterprise hire in the company's transformation from consumer to enterprise business. A focus of his product career has been using the power of design thinking to make technically complex products intuitive and easy to use. Alex graduated from Stanford University with a degree in Electrical Engineering.Links Referenced:twingate.com: https://twingate.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Sysdig. Sysdig secures your cloud from source to run. They believe, as do I, that DevOps and security are inextricably linked. If you wanna learn more about how they view this, check out their blog, it's definitely worth the read. To learn more about how they are absolutely getting it right from where I sit, visit Sysdig.com and tell them that I sent you. That's S Y S D I G.com. And my thanks to them for their continued support of this ridiculous nonsense.Corey: This episode is sponsored in part by Honeycomb. When production is running slow, it's hard to know where problems originate. Is it your application code, users, or the underlying systems? I've got five bucks on DNS, personally. Why scroll through endless dashboards while dealing with alert floods, going from tool to tool to tool that you employ, guessing at which puzzle pieces matter? Context switching and tool sprawl are slowly killing both your team and your business. You should care more about one of those than the other; which one is up to you. Drop the separate pillars and enter a world of getting one unified understanding of the one thing driving your business: production. With Honeycomb, you guess less and know more. Try it for free at honeycomb.io/screaminginthecloud. Observability: it's more than just hipster monitoring.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. This promoted episode is brought to us by our friends at Twingate, and in addition to bringing you this episode, they also brought me a guest. Alex Marshall is the Chief Product Officer at Twingate. Alex, thank you for joining me, and what is a Twingate?Alex: Yeah, well, thanks. Well, it's great to be here. What is Twingate? Well, the way to think about Twingate is we're really a network overlay layer. And so, the experience you have when you're running Twingate as a user is that network resources or network destinations that wouldn't otherwise be accessible to you or magically accessible to you and you're properly authenticated and authorized to access them.Corey: When you say it's a network overlay, what I tend to hear and the context I usually see that in, in the real world is, “Well, we're running some things in AWS and some things in Google Cloud, and I don't know because of a sudden sharp blow to the head, maybe Azure as well, and how do you get all of the various security network models of security groups on one side to talk to their equivalent on the other side?” And the correct answer is generally that you don't and you use something else that more or less makes the rest of that irrelevant. Is that the direction you're coming at this from, or do you view it differently?Alex: Yeah, so I think the way that we view this in terms of, like, why we decide to build a product in the first place is that if you look at, sort of like, the internet in 2022, like, there's one thing that's missing from the network routing table, which is authentication and authorization on each row [laugh]. And so, the way that we designed the product is we said, “Okay, we're not going to worry about everything, basically, above the network layer and we're going to focus on making sure that what we're controlling with the client is looking at outbound network connections and making sure that when someone accesses something and only when they access it, that we check to make sure that they're allowed access.” We're basically holding those network connections until someone's proven that they're allowed to access to, then we let it go. And so, from the standpoint of, like, figuring out, like, security groups and all that kind of stuff, we're basically saying, like, “Yeah, if you're allowed to access the database in AWS, or your home assistant on your home network, fine, we'll let you do that, but we'll only let you go there once you've proven you're allowed to. And then once you're there, then you know, we'll let you figure out how you want to authenticate into the destination system.” So, our view is, like, let's start at the network layer, and then that solves a lot of problems.Corey: When I call this a VPN, I know a couple of things are going to be true. One, you're almost certainly going to correct me on that because this is all about Zero Trust. This is the Year of our Lord 2022, after all. But also what I round to what basically becomes a VPN to my mind, there are usually two implementations or implementation patterns that I think about. One of them is the idea of client access, where I have a laptop; I'm in a Starbucks; I want to connect to a thing. And the other has historically been considered, site to site, or I have a data center that I want to have constantly connected to my cloud environment. Which side of that mental model do you tend to fall in? Or is that the wrong way to frame it?Alex: Mm-hm. The way we look at it and sort of the vision that we have for what the product should be, the problem that we should be solving for customers is what we want to solve for customers is that Twingate is a product that lets you be certain that your employees can work securely from anywhere. And so, you need a little bit of a different model to do that. And the two examples you gave are actually both entirely valid, especially given the fact that people just work from everywhere now. Like, resources everywhere, they use a lot of different devices, people work from lots of different networks, and so it's a really hard problem to solve.And so, the way that we look at it is that you really want to be running something or have a system in place that's always taking into account the context that user is in. So, in your example of someone's at a Starbucks, you know, in the public WiFi, last time I checked, Starbucks WiFi was unencrypted, so it's pretty bad for security. So, what we should do is you should take that context into account and then make sure that all that traffic is encrypted. But at the same time, like, you might be in the corporate office, network is perfectly safe, but you still want to make sure that you're authorizing people at the point in time they try to access something to make sure that they actually are entitled to access that database in the AWS network. And so, we're trying to get people away from thinking about this, like, point-to-point connection with a VPN, where you know, the usual experience we've all had as employees is, “Great. Now, I need to fire up the VPN. My internet traffic is going to be horrible. My battery's probably going to die. My—”Corey: Pull out the manual token that rotates with an RSA—Alex: Exactly.Corey: —token that spits out a different digital code every 30 seconds if the battery hasn't died or they haven't gotten their seeds leaked again, and then log in and the rest; in some horrible implementations type that code after your password for some Godforsaken reason. Yeah, we've all been down that path and it's like, “Yeah, just sign into the corporate VPN.” It's like, “Did you just tell me to go screw myself because that's what I heard.”Alex: [laugh]. Exactly. And that is exactly the situation that we're in. And the fact is, like, VPNs were invented a long time ago and they were designed to connect to networks, right? They were designed to connect a branch office to a corporate office, and they're just to join all the devices on the network.So, we're really, like—everybody has had this experience of VPN is suffering from the fact that it's the wrong tool for the job. Going back to, sort of like, this idea of, like, us being the network overlay, we don't want to touch any traffic that isn't intended to go to something that the company or the organization or the team wants to protect. And so, we're only going to gate traffic that goes to those network destinations that you actually want to protect. And we're going to make sure that when that happens, it's painless. So, for example, like, you know, I don't know, again, like, use your example again; you've been at Starbucks, you've been working your email, you don't really need to access anything that's private, and all of a sudden, like, you need to as part of your work that you're doing on the Starbucks WiFi is access something that's in AWS.Well, then the moment you do that, then maybe you're actually fine to access it because you've been authenticated, you know, and you're within the window, it's just going to work, right, so you don't have to go through this painful process of firing up the VPN like you're just talking about.Corey: There are a number of companies out there that, first, self-described as being, “Oh, we do Zero Trust.” And when I hear that, what I immediately hear in my own mind is, “I have something to sell you,” which, fair enough, we live in an industry. We're trying to have a society here. I get it. The next part that I wind up getting confused by then is, it seems like one of those deeply overloaded terms that exists to, more or less—in some cases to be very direct—well, we've been selling this thing for 15 years and that's the buzzword, so now we're going to describe it as the thing we do with a fresh coat of paint on it.Other times it seems to be something radically different. And, on some level, I feel like I could wind up building an entire security suite out of nothing other than things self-billing themselves as Zero Trust. What is it that makes Twingate different compared to a wide variety of other offerings, ranging from Seam to whatever the hell an XDR might be to, apparently according to RSA, a breakfast cereal?Alex: So, you're right. Like, Zero Trust is completely, like, overused word. And so, what's different about Twingate is that really, I think goes back to, like, why we started the company in the first place, which is that we started looking at the remote workspace. And this is, of course, before the pandemic, before everybody was actually working remotely and it became a really urgent problem.Corey: During the pandemic, of course, a lot of the traditional VPN companies are, “Huh. Why is the VPN concentrator glowing white in the rack and melting? And it sounds like screaming. What's going on?” Yeah, it turns out capacity provisioning and bottlenecking of an entire company tends to be a thing at scale.Alex: And so, you're right, like, that is exactly the conversation. We've had a bunch of customers over the last couple years, it's like their VPN gateway is, like, blowing up because it used to be that 10% of the workforce used it on average, and all of a sudden everybody had to use it. What's different about our approach in terms of what we observed when we started the company, is that what we noticed is that this term Zero Trust is kind of floating out there, but the only company that actually implemented Zero Trust was Google. So, if you think about the situations that you look at, Zero Trust is like, obvious. It's like, it's what you would want to do if you redesigned the internet, which is you'd want to say every network connection has to be authorized every single time it's made.But the internet isn't actually designed that way. It's designed default open instead of default closed. And so, we looked at the industry are, like, “Great. Like, Google's done it. Google has, like, tons and tons of resources. Why hasn't anyone else done it?”And the example that I like to talk about when we talk about inception of the business is we went to some products that are out there that were implementing the right technological approach, and one of these products is still in use today, believe it or not, but I went to the documentation page, and I hit print, and it was almost 50 pages of documentation to implement it. And so, when you look at that, you're, like, okay, like, maybe there's a usability problem here [laugh]. And so, what we really, really focus on is, how do we make this product as easy as possible to deploy? And that gets into, like, this area of change management. And so, if you're in IT or DevOps or engineering or security and you're listening to this, I'm sure you've been through this process where it's taken months to deploy something because it was just really technically difficult and because you had to change user behavior. So, the thing that we focus on is making sure that you didn't have to change user behavior.Corey: Every time you expect people to start doing things completely differently, congratulations, you've already lost before you've started.Alex: Yes, exactly. And so, the difference with our product is that you can switch off the VPN one day, have people install a Twingate client, and then tomorrow, they still access things with exactly the same addresses they used before. And this seems like such a minor point, but the fact that I don't have to rewrite scripts, I don't have to change my SSH proxy configuration, I don't have to do anything, all of those private DNS addresses or those private IP address, they'll still work because of the way that our client works on the device.Corey: So, what you're saying is fundamental; you could even do a slow rollout. It doesn't need to be a knife-switch cutover at two in the morning where you're scrambling around and, “Oh, my God, we forgot the entire accounting department.”Alex: Yep, that's exactly right. And that is, like, an attraction of deploying this is that you can actually deploy it department by department and not have to change all your infrastructure at the same time. So again, it's like pretty fundamental point here. It's like, if you're going to get adoption technology, it's not just about how cool the technology is under the hood and how advanced it is; it's actually thinking about from a customer and a business standpoint, like, how much is actually going to cost time-wise and effort-wise to move over to the new solution. So, we've really, really focused on that.Corey: Yeah. That is generally one of those things, that seems to be the hardest approach. I mean, let's back up a little bit here because I will challenge—likely—something that you said a few minutes ago, which is Google was the first and only company for a little while doing Zero Trust. Back in 2012, it turned out that we weren't calling it that then, but that is fundamentally what I built out of the ten-person startup that I was at, where I was the first ops hire, which generally comes in right around Series B when developers realize, okay, we can no longer lie to ourselves that we know what we're doing on an ops side. Everything's on fire and no one can sleep through the night. Help, help, help. Which is fine.I've never had tolerance or patience for ops people who insult people in those situations. It's, “Well, they got far enough along to hire you, didn't they? So, maybe show some respect.” But one of the things that I did was, being on the corporate network got you access to the printer in the corner and that was it. There was no special treatment of that network.And I didn't think much of it at the time, but I got some very strange looks and had some—uh, will call it interesting a decade later; most of the pain has faded—discussions with our auditor when we were going through some PCI work, and they showed up and said, “Great. Okay, where are the credentials for your directory?” And my response was, “Our what now?” And that's when I realized there's a certain point of scale. Back when I started as an independent consultant, everything I did for single-sign-on, for example, was my 1Password vault. Easy enough.Now, that we've scaled up beyond that, I'm starting to see the value of things like single-sign-on in a way that I never did before, and in hindsight, I'd like to go back and do things very differently as a result. Scale matters. What is the point of scale that you find is your sweet spot? Is it one person trying to connect to a whole bunch of nonsense? Is it small to midsize companies—and we should probably bound that because to me, a big company is still one that has 200 people there?Alex: To your original interesting point, which is that yeah, kudos to you for, like, implementing that, like, back then because we've had probably—Corey: I was just being lazy and it was what was there. It's like, “Why do I want to maintain a server in the closet? Honestly, I'm not sure that the office is that secure. And all it's going to do—what I'm I going to put on that? A SharePoint server? Please. We're using Macs.”Alex: Yeah, exactly. Yeah. So it's, we've had, like, I don't know at this point, thousands of customer conversations. The number of people have actually gone down that route implementing things themselves as a very small number. And I think that just shows how hard it is. So again, like, kudos.And I think the scale point is, I think, really critical. So, I think it's changed over time, but actually, the point at which a customer gets to a scale where I think a solution has, like, leveraged high value is when you get to maybe only 50, 75 people, which is a pretty small business. And the reason is that that's the point at which a bunch of tools start getting implemented a company, right? When you're five people, you're not going to install, like, an MDM or something on people's devices, right? When you get to 50, 75, 100, you start hiring your first IT team members. That's the point where them being able to, like, centralize management of things at the company becomes really critical.And so, one of the other aspects that makes this a little bit different terms of approach is that what we see is that there's a huge number of tools that have to be managed, and they have different configuration settings. You can't even get consistency on MDM is across different platforms, necessarily, right? Like, Linux, Windows, and Mac are all going to have slight differences, and so what we've been working with the platform towards is actually being the centralization point where we integrate with these different systems and then pull together, like, a consistent way to create those authentication authorization policies I was talking about before. And the last thing on SSO, just to sort of reiterate that, I think that you're talking about you're seeing the value of that, the other thing that we've, like, made a deliberate decision on is that we're not going to try to, like, re-solve, like, a bunch of these problems. Like, some of the things that we do on the user authentication point is that we rely on there being an SSO, like, user directory, that handles authentication, that handles, like, creating user groups. And we want to reuse that when people are using Twingate to control access to network destinations.So, for us, like, it's actually, you know, that point of scale comes fairly early. It only gets harder from there, and it's especially when that IT team is, like, a relatively small number of people compared to number of employees where it becomes really critical to be able to leverage all the technology they have to deploy.Corey: I guess this might be one of those areas where I'm not deep enough in your space to really see it the same way that you do, which is the whole reason I have people like you on the show: so I can ask these questions directly. What is the painful position that I find myself in that I should say, “Ah, I should bring Twingate in to solve this obnoxious, painful problem so I never have to think about it again.” What is it that you solve?Alex: Yeah, I mean, I think for what our customers tell us, it's providing a, like, consistent way to get access into, like, a wide variety of internal resources, and generally in multi-cloud environments. That's where it gets, like, really tricky. And the consistency is, like, really important because you're trying to provide access to your team—often like it's DevOps teams, but all kinds of people can access these things—trying to write access is a multiple different environments, again, there's a consistency problem where there are multiple different ways to provide that, and there isn't a single place to manage all that. And so, it gets really challenging to understand who has access to what, makes sure that credentials expire when they're supposed to expire, make sure that all the routing inside those remote destinations is set up correctly. And it just becomes, like, a real hassle to manage those things.So, that's the big one. And usually where people are coming from is that they've been using VPN to do that because they didn't know anything better exists, or they haven't found anything that's easy enough to deploy, right? So, that's really the problem that they're running into.Corey: There's also a lot of tribal knowledge that gets passed down. The oral tradition of, “I have this problem. What should I do? I know, I will consult the wise old sage.” “Well, where can you find the wise old sage?” “Under the rack of servers, swearing at them.” “Great, cool. Well, use a VPN. That's what we've used since time immemorial.” And then the sins are visited onto yet another generation.There's a sense that I have that companies that are started now are going to have a radically different security posture and a different way of thinking about these things than the quote-unquote, “Legacy companies.”—legacy, of course, being that condescending engineering term for ‘it makes money—who are migrating their way into a brave new world because they had the temerity to found themselves as companies before 2012.Alex: Absolutely. When we're working with customers, there is a sort of a sweet spot, both in terms of, like, the size and role that we were talking about before, but also just in terms of, like, where they are, in, sort of like, the sort of lifecycle of their company. And I think one of the most exciting things for us is that we get to work with companies that are kind of figuring this stuff out for the first time and they're taking a fresh look at, like, what the capabilities are out there in the landscape. And that's, I think, what makes this whole space, like, super, super interesting.There's some really, really fantastic things you can do. Just give you an example, again, that I think might resonate with your audience quite a bit is this whole topic of automation, right? Your time at the tribal knowledge of, like, “Oh, of course. You know, we set up a VPN and so on.” One of the things that I don't think is necessarily obvious in this space is that for the teams that—at companies that are deploying, configuring, managing internal network infrastructure, is that in the past, you've had to make compromises on infrastructure in order to accommodate access, right?Because it's kind of a pain to deploy a bunch of, like, VPN gateways, mostly for the end-user because they got to, like, choose which one they're connecting to. You potentially had to open up traffic routes to accommodate a VPN gateway that you wouldn't otherwise want to open up. And so, one of the things that's, like, really sort of fascinating about, like, a new way of looking at things is that what we allow with Twingate—and part of this is because we've really made sure that the product is, like, API-first in the very beginning, which allows us to very easily integrate in with things, like, Terraform and Pulumi for deployment automation, is that now you have a new way of looking at things, which is that you can build a network infrastructure that you want with the data flow rules that you want, and very easily provide access into, like, points of that infrastructure, whether that's an entire subnet or just a single host somewhere. I think these are the ways, like, the capabilities have been realized are possible until they, sort of like, understand some of these new technologies.Corey: This episode is sponsored in part by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on-premises, private cloud, and they just announced a fully-managed service on AWS and Azure called BigAnimal, all one word. Don't leave managing your database to your cloud vendor because they're too busy launching another half-dozen managed databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications—including Oracle—to the cloud. To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: This feels like one of those technologies where the place that a customer starts from and where they wind up going are very far apart. Because I can see the metaphorical camel's nose under the tent flap being, “Ah, this is a VPN except it doesn't suck. Great.” But once you wind up with effectively an overlay network connecting all the things that you care about within an organization, it feels like that unlocks a whole universe of possibility.Alex: Mm-hm. Yeah, definitely. I mean, I think you hit the nail on the head there. Like, a lot of people approach us because they're having a lot of pain with VPN and all the operational difficulties they were talking about earlier, but I think what sort of starts to open up is there's some, sort of like, not obvious things that happen. And one of them is that all of a sudden, when you can limit access at a network connection level, you start to think about, like, credentials and access management a little differently, right?So, one of the problems that well-known is people set a bastion host. And they set bastion host so that there's, like, a limited way into the network and all the, you know, keys are stored in that bastion host and so on. So, you basically have a system where fine, we had bastion host set up because, A, we want limited ingress, and B, we want to make sure that we know exactly who has access to our internal resources. You could do away with that and with a simple, like, configuration change, you can basically say, “Even if this employee for whatever reason, we've forgotten to remove—revoke their SSH keys, even if they still have those keys, they can't access the destination because we're blocking network access at their actual device,” then you have a very different way to restrict access. So, it's still important to manage credentials, but you now have a way to actually block things out at a network level. And I think it's like when people start to realize that these capabilities are possible that they definitely start thinking about things a little bit differently. VPNs just don't allow this, like, level of granularity.Corey: I am a firm believer in the idea that any product with any kind of longevity gets an awful lot of its use case and product-market fit not from the people building it, but from the things that those folks learn from their customers. What did you learn from customers rolling out Twingate that reshaped how you thought about the space, or surprised you as far as use cases go?Alex: Yeah, so I think it's a really interesting question because one of the benefits of having a small business and being early on is that you have very close relationships with all your customers and they're really passionate about your product. And what that leads to is just a lot of, sort of like, knowledge sharing around, like, how they're using your product, which then helps inform the types of things that we build. So, one of the things that we've done internally to help us learn, but then also help us respond more quickly to customers, is we have this group called Twingate Labs. And it's really just a group of folks that are outside the engineering org that are just allowed to build whatever they want to try to prove out, like, interesting concepts. And a lot of those—I say a lot; honestly, probably all of those concepts have come from our customers, and so we've been able to, like, push the boundaries on that.And so, it just gave you an example, I mean, AWS can be sometimes a challenging product to manage and interact with, and so that team has, for example, built capabilities, again, using that just the regular Twingate API to show that it's possible to automatically configure resources in AWS based on tags. Now, that's not something that's in our product, but it's us showing our customers that, you know, we can respond quickly to them and then they actually, like, try to accommodate some, like, these special use cases they have. And if that works out, then great, we'll pull it into the product, right? So, I think that's, like, the nice thing about serving a smaller businesses is that you get a lot of that back and forth to your customers and they help us generate ideas, too.Corey: One thing that stands out to me from the testimonials from customers you have on your website has been a recurring theme that crops up that speaks to I guess, once I spend more than ten seconds thinking about it, one of the most obvious reasons that I would say, “Oh, Twingate? That sounds great for somebody else. We're never rolling it out here.” And that is the ease of adoption into environments that are not greenfield because I don't believe that something like this product will ever get deployed to something greenfield because this is exactly the kind of problem that you don't realize exists and don't have to solve for until it's too late because you already have that painful problem. It's an early optimization until suddenly, it's something you should have done six months ago. What is the rolling it out process for a company that presumably already is built out, has hired a bunch of people, and they already have something that, quote-unquote, “Works,” for granting access to things?Alex: Mm-hm. Yeah, so the beauty is that you can really deploy this side-by-side with an existing solution, so—whatever it happens to be; I mean, whether it's a VPN or something else—is you can put the side-by-side and the deployment process, just to talk a little bit about the architecture; we've talked a lot about this client that runs on the user's device, but on the remote network side, just to be really clear on this, there's a component called a connector that gets deployed inside the remote network, and it does not have to be installed on every single destination host. You're sort of thinking about it, sort of like this routing point inside that network, and that connector controls what traffic is allowed to go to internal locations based on the rules. So, from a deployment standpoint, it's really just put a connector in place and put it in place in whatever subnet you want to provide access to.And so you're—unlikely, but if your entire company has one subnet, great. You're done with one connector. But it does mean you can sort of gradually roll it out as it goes. And the connector can be deployed in a bunch of different environments, so we're just talking with AWS. Maybe it's inside a VPC, but we have a lot of people that actually just want to control access to specific services inside a Kubernetes cluster, and so you can deploy it as a container, right inside Kubernetes. And so, you can be, like, really specific about how you do that and then gradually roll it out to teams as they need it and without having to necessarily on that day actually shut off the old solution.So, just to your comment, by the way, on the greenfield versus, sort of like, brownfield, I think the greenfield story, I think, is changing a little bit, I think, especially to your comment earlier around younger companies. I think younger companies are realizing that this type of capability is an option and that they want to get in earlier. But the reality is that, you know, 98% of people are really in the established network situation, and so that's where that rollout process is really important.Corey: As you take a look throughout what you're seeing customers doing, what you see the industry doing as a result of that—because customers are, in fact, the industry, let's be clear here—what do you think is, I guess, the next wave of security offerings? I guess what I'm trying to do here is read the tea leaves and predict what the buzzwords will be all over the place that next RSA. But on a slightly more serious note, what do you see this is building towards? What are the trends that you're identifying in the space?Alex: There's a couple of things that we see. So one, sort of, way to look at this is that we're sort of in this, like, Third Wave. And I think these things change more slowly than—with all due respect to marketers—than marketers would [laugh] have you believe. And so, thinking about where we are, there's, like, Wave One is, like, good old happy days, we're all in the office, like, your computer can't move, like, all the data is in the office, like, everything is in one place, right?Corey: What if someone steals your desktop? Well, they're probably going to give themselves a hernia because that thing's heavy. Yeah.Alex: Exactly. And is it really worth stealing, right? But the Wave One was really, like, network security was actually just physical security, to that point; that's all it was, just, like, physically secure the premises.Wave Two—and arguably you could say we're kind of still in this—is actually the transition to cloud. So, let's convert all CapEx to OpEx, but that also introduces a different problem, which is that everything is off-network. So, you have to, like, figure out, you know, what you do about that.But Wave Three is really I think—and again, just to be clear, I think Wave Two, there are, like, multi-decade things that happen—and I'd say we're in the middle of, like, Wave Three. And I think that everyone is still, like, gradually adapting to this, which is what we describe it as sort of people everywhere, applications are everywhere, people are using a whole bunch of different devices, right? There is no such thing as BYOD in the early-2000s, late-90s, and people are accessing things from all kinds of different networks. And this presents a really, really challenging problem. So, I would argue, to your question, I think we're still in the middle of that Wave Three and it's going to take a long time to see that play through the industry. Just, things change slowly. That tribal knowledge takes time to change.The other thing that I think we very strongly believe in is that—and again, this is, sort of like, coming from our customers, too—is that people basically with security industry have had a tough time trying things out and adopting them because a lot of vendors have put a lot of blockers in place of doing that. There's no public documentation; you can't just go use the product. You got to talk to a salesperson who then filters you through—Corey: We have our fifth call with the sales team. We're hoping this is the one where they'll tell us how much it costs.Alex: Exactly. Or like, you know, now you get to the sales engineer, so you gradually adopt this knowledge. But ultimately, people just want to try the darn thing [laugh], right? So, I think we're big believers that I think hopefully, what we'll see in the security industry is that—we're trying to set an example here—is really that there's an old way of doing things, but a new way of doing things is make the product available for people to use, document the heck out of it, explain all the different use cases that exist for how to be successful your product, and then have these users actually then reach out to you when they want to have more in-depth conversation about things. So, those are the two big things, I'd say. I don't know if those are translated buzzwords at RSA, but those are two big trends we see.Corey: I look forward to having you back in a year or two and seeing how close we get to the reality. “Well, I guess we didn't see that acronym coming, but don't worry. They've been doing it for the last 15 years under different names, so it works out.” I really want to thank you for being as generous with your time as you have been. If people want to learn more, where should they go?Alex: Well, as we're just talking about, you try the product at twingate.com. So, that should be your first stop.Corey: And we will of course put links to that in the show notes. Thank you so much for being as forthcoming as you have been about all this stuff. I really appreciate your time.Alex: Yeah, thank you, Corey. I really appreciate it. Thanks.Corey: Alex Marshall, Chief Product Officer at Twingate. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with a long angry ranty comment about what you hated about the episode, which will inevitably get lost when it fails to submit because your crappy VPN concentrator just dropped it on the floor.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About MartinMartin Casado is a general partner at the venture capital firm Andreessen Horowitz where he focuses on enterprise investing. He was previously the cofounder and chief technology officer at Nicira, which was acquired by VMware for $1.26 billion in 2012. While at VMware, Martin was a fellow, and served as senior vice president and general manager of the Networking and Security Business Unit, which he scaled to a $600 million run-rate business by the time he left VMware in 2016.Martin started his career at Lawrence Livermore National Laboratory where he worked on large-scale simulations for the Department of Defense before moving over to work with the intelligence community on networking and cybersecurity. These experiences inspired his work at Stanford where he created the software-defined networking (SDN) movement, leading to a new paradigm of network virtualization. While at Stanford he also cofounded Illuminics Systems, an IP analytics company, which was acquired by Quova Inc. in 2006.For his work, Martin was awarded both the ACM Grace Murray Hopper award and the NEC C&C award, and he's an inductee of the Lawrence Livermore Lab's Entrepreneur's Hall of Fame. He holds both a PhD and Masters degree in Computer Science from Stanford University.Martin serves on the board of ActionIQ, Ambient.ai, Astranis, dbt Labs, Fivetran, Imply, Isovalent, Kong, Material Security, Netlify, Orbit, Pindrop Security, Preset, RapidAPI, Rasa, Tackle, Tecton, and Yubico.Links: Yet Another Infra Group Discord Server: https://discord.gg/f3xnJzwbeQ “The Cost of Cloud, a Trillion Dollar Paradox” - https://a16z.com/2021/05/27/cost-of-cloud-paradox-market-cap-cloud-lifecycle-scale-growth-repatriation-optimization/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by Honeycomb. When production is running slow, it's hard to know where problems originate. Is it your application code, users, or the underlying systems? I've got five bucks on DNS, personally. Why scroll through endless dashboards while dealing with alert floods, going from tool to tool to tool that you employ, guessing at which puzzle pieces matter? Context switching and tool sprawl are slowly killing both your team and your business. You should care more about one of those than the other; which one is up to you. Drop the separate pillars and enter a world of getting one unified understanding of the one thing driving your business: production. With Honeycomb, you guess less and know more. Try it for free at honeycomb.io/screaminginthecloud. Observability: it's more than just hipster monitoring.Corey: This episode is sponsored in part by our friends at Sysdig. Sysdig secures your cloud from source to run. They believe, as do I, that DevOps and security are inextricably linked. If you wanna learn more about how they view this, check out their blog, it's definitely worth the read. To learn more about how they are absolutely getting it right from where I sit, visit Sysdig.com and tell them that I sent you. That's S Y S D I G.com. And my thanks to them for their continued support of this ridiculous nonsense.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I'm joined today by someone who has taken a slightly different approach to being—well, we'll call it cloud skepticism here. Martin Casado is a general partner at Andreessen Horowitz and has been on my radar starting a while back, based upon a piece that he wrote focusing on the costs of cloud and how repatriation is going to grow. You wrote that in conjunction with your colleague, Sarah Wang. Martin, thank you so much for joining me. What got you onto that path?Martin: So, I want to be very clear, just to start with is, I think cloud is the biggest innovation that we've seen in infrastructure, probably ever. It's a core part of the industry. I think it's very important, I think every company's going to be using cloud, so I'm very pro-cloud. I just think the nature of how you use clouds is shifting. And that was the focus.Corey: When you first put out your article in conjunction with your colleague as well, like, I saw it and I have to say that this was the first time I'd really come across any of your work previously. And I have my own biases that I started from, so my opening position on reading it was this is just some jerk who's trying to say something controversial and edgy to get attention. That's my frickin job. Excuse me, sir. And who is this clown?So, I started digging, and what I found really changed my perspective because as mentioned at the start of the show, you are a general partner at Andreessen Horowitz, which means you are a VC. You are definitionally almost the archetype of a VC in that sense. And to me, being a venture capitalist means the most interesting thing about you is that you write a large check consisting of someone else's money. And that's never been particularly interesting.Martin: [laugh].Corey: You kind of cut against that grain and that narrative. You have a master's and a PhD in computer science from Stanford; you started your career at one of the national labs—Laurence Livermore, if memory serves—you wound up starting a business, Nicira, if I'm pronouncing that correctly—Martin: Yeah, yeah, yeah.Corey: That you then sold to VMware in 2012, back at a time when that was a noble outcome, rather than a state of failure because VMware is not exactly what it once was. You ran a $600 million a year business while you were there. Basically, the list of boards that you're on is lengthy enough and notable enough that it sounds almost like you're professionally bored, so I don't—Martin: [laugh].Corey: So, looking at this, it's okay, this is someone who actually knows what he is talking about, not just, “Well, I talked to three people in pitch meetings and I now think I know what is going on in this broader industry.” You pay attention, and you're connected, disturbingly well, to what's going on, to the point where if you see something, it is almost certainly rooted in something that is happening. And it's a big enough market that I don't think any one person can keep their finger on the pulse of everything. So, that's when I started really digging into it, paying attention, and more or less took a lot of what you wrote as there are some theses in here that I want to prove or disprove. And I spent a fair bit of time basically threatening, swindling, and bribing people with infinite cups of coffee in order to start figuring out what is going on.And I am begrudgingly left with no better conclusion than you have a series of points in here that are very challenging to disprove. So, where do you stand today, now that, I guess, the whole rise and fall of the hype around your article on cloud repatriation—which yes, yes, we'll put a link to it in the show notes if people want to go there—but you've talked about this in a lot of different contexts. Having had the conversations that you've had, and I'm sure some very salty arguments with people who have a certain vested interest in you being wrong, do you wind up continuing to stand by the baseline positions that you've laid out, or have they evolved into something more nuanced?Martin: So yeah, I definitely want to point out, so this was work done with Sarah Wang was also at Andreessen Horowitz; she's also a GP. She actually did the majority of the analysis and she's way smarter than I am. [laugh]. And so, I'm just very—feel very lucky to work with her on this. And I want to make sure she gets due credit on this.So, let's talk about the furor. So like, I actually thought that this was kind of interesting and it started a good discussion, but instead, like, [laugh] the amount of, like, response pieces and, like, angry emails I got, and [laugh] like, I mean it just—and I kind of thought to myself, like, “Why are people so upset?” I think there's three reasons. I'm going to go through them very quickly because they're interesting.So, the first one is, like, you're right, like, I'm a VC. I think people see a VC and they're like, oh, lack of credibility, lack of accountability, [laugh], you know, doesn't know what they're doing, broad pattern matcher. And, like, I will say, like, I did not necessarily write this as a VC; I wrote this as somebody that's, like, listen, my PhD is an infrastructure; my company was an infrastructure. It's all data center stuff. I had a $600 million a year data center business that sold infrastructure into data centers. I've worked with all of the above. Like, I've worked with Amazon, I've—Corey: So, you sold three Cisco switches?Martin: [laugh]. That's right.Corey: I remember those days. Those were awesome, but not inexpensive.Martin: [laugh]. That's right. Yeah, so like, you know, I had 15 years. It's kind of a culmination of that experience. So, that was one; I just think that people see VC and they have a reaction.The second one is, I think people still have the first cloud wars fresh in their memories and so they just don't know how to think outside of that. So, a lot of the rebuttals were first cloud war rebuttals. Like, “Well, but internal IT is slow and you can't have the expertise.” But like, they just don't apply to the new world, right? Like, listen, if you're Cloudflare, to say that you can't run, like, a large operation is just silly. If you went to Cloudflare and you're like, “Listen, you can't run your own infrastructure,” like, they'd take out your sucker and pat you on the head. [laugh].Corey: And not for nothing, if you try to run what they're doing on other cloud providers from a pure bandwidth perspective, you don't have a company anymore, regardless of how well funded you are. It's a never-full money pit that just sucks all of the money. And I've talked to a number of very early idea stage companies that aren't really founded yet about trying to do things like CDN-style work or streaming video, and a lot of those questions start off with well, we did some back-of-the-envelope math around AWS data transfer pricing, and if our numbers are right, when we scale, we'll be spending $65,000 on data transfer every minute. What did we get wrong?And it's like, “Oh, yeah, you realize that one thing is per hour not per minute, so slight difference there. But no, you're basically correct. Don't do it.” And yeah, no one pays retail price at that volume, but they're not going to give you a 99.999% discount on these things, so come up with a better plan. Cloudflare's business will not work on AWS, full stop.Martin: Yep, yep. So, I legitimately know, basically, household name public companies that are software companies that anybody listening to this knows the name of these companies, who have product lines who have 0% margins because they're [laugh] basically, like, for every dollar they make, they pay a dollar to Amazon. Like, this is a very real thing, right? And if you go to these companies, these are software infrastructure companies; they've got very talented teams, they know how to build, like, infrastructure. To tell them that like, “Well, you know, you can't build your own infrastructure,” or something is, I mean, it's like telling, like, an expert in the business, they can't do what they do; this is what they do. So, I just think that part of the furor, part of the uproar, was like, I just think people were stuck in this cloud war 1.0 mindset.I think the third thing is, listen, we've got an oligopoly, and they employ a bunch of people, and they've convinced a bunch of people they're right, and it's always hard to change that. And I also think there's just a knee-jerk reaction to these big macro shifts. And it was the same thing we did to software-defined networking. You know, like, my grad school work was trying to change networking to go from hardware to software. I remember giving a talk at Cisco, and I was, like, this kind of like a naive grad student, and they literally yelled at me out of the room. They're like, it'll never work.Corey: They tried to burn you as a witch, as I recall.Martin: [laugh]. And so, your specific question is, like, have our views evolved? But the first one is, I think that this macro downturn really kind of makes the problem more acute. And so, I think the problem is very, very real. And so, I think the question is, “Okay, so what happens?”So, let's say if you're building a new software company, and you have a choice of using, like, one of the Big Three public clouds, but it impacts your margins so much that it depresses your share price, what do you do? And I think that we thought a lot more about what the answers there are. And the ones that I think that we're seeing is, some actually are; companies are building their own infrastructure. Like, very famously MosaicML is building their own infrastructure. Fly.io, -building their own infrastructure.Mighty—you know, Suhail's company—building his own infrastructure. Cloudflare has their own infrastructure. So, I think if you're an infrastructure provider, a very reasonable thing to do is to build your own infrastructure. If you're not a core infrastructure provider, you're not; you can still use somebody's infrastructure that's built at a better cost point.So, for example, if I'm looking at a CDN tier, I'm going to use Fly.io, right? I mean, it's like, it's way cheaper, the multi-region is way better, and so, like, I do think that we're seeing, like, almost verticalized clouds getting built out that address this price point and, like, these new use cases. And I think this is going to start happening more and more now. And we're going to see basically almost the delamination of the cloud into these verticalized clouds.Corey: I think there's also a question of scale, where if you're starting out in the evening tonight, to—I want to build, I don't know Excel as a service or something. Great. You're pretty silly if you're not going to start off with a cloud provider, just because you can get instant access to resources, and if your product catches on, you scale out without having to ever go back and build it as quote-unquote “Enterprise grade,” as opposed to having building it on cheap servers or Raspberry Pis or something floating around. By the time that costs hit a certain point—and what that point is going to depend on your stage of company and lifecycle—you're remiss if you don't at least do an analysis on is this the path we want to continue on for the service that we're offering?And to be clear, the answer to this is almost entirely going to be bounded by the context of your business. I don't believe that companies as a general rule, make ill-reasoned decisions. I think that when we see a decision a company makes, by and large, there's context or constraints that we don't see that inform that. I know, it's fun to dunk on some of the large companies' seemingly inscrutable decisions, but I will say, having had the privilege to talk to an awful lot of execs in an awful lot of places—particularly on this show—I don't find myself encountering a whole lot of people in those roles who I come away with thinking that they're a few fries short of a Happy Meal. They generally are very well reasoned in why they do what they do. It's just a question of where we think the future is going on some level.Martin: Yep. So, I think that's absolutely right. So, to be a little bit more clear on what I think is happening with the cloud, which is I think every company that gets created in tech is going to use the cloud for something, right? They'll use it for development, the website, test, et cetera. And many will have everything in the cloud, right?So, the cloud is here to stay, it's going to continue to grow, it's a very important piece of the ecosystem, it's very important piece of IT. I'm very, very pro cloud; there's a lot of value. But the one area that's under pressure is if your product is SaaS if your product is selling Software as a Service, so then your product is basically infrastructure, now you've got a product cost model that includes the infrastructure itself, right? And if you reduce that, that's going to increase your margin. And so, every company that's doing that should ask the question, like, A, is the Big Three the right one for me?Maybe a verticalized cloud—like for example, whatever Fly or Mosaic or whatever is better because the cost is better. And I know how to, you know, write software and run these things, so I'll use that. They'll make that decision or maybe they'll build their own infrastructure. And I think we're going to see that decision happening more and more, exactly because now software is being offered as a service and they can do that. And I just want to make the point, just because I think it's so important, that the clouds did exactly this to the hardware providers. So, I just want to tell a quick story, just because for me, it's just so interesting. So—Corey: No, please, I was only really paying attention to this market from 2016 or so. There was a lot of the early days that I was using as a customer, but I wasn't paying attention to the overall industry trends. Please, storytime. This is how I learned things. I hang out with smart people and I come away a little bit smarter than when I started.Martin: [laugh]. This is, like, literally my fa—this is why this is one of my favorite topics is what I'm about to tell you, which is, so the clouds have always had this argument, right? The big clouds, three clouds, they're like, “Listen, why would you build your own cloud? Because, like, you don't have the expertise, and it's hard and you don't have economies of scale.” Right?And the answer is you wouldn't unless it impacts your share price, right? If it impacts your share price, then of course you would because it makes economic sense. So, the clouds had that exact same dilemma in 2005, right? So, in 2005, Google and Amazon and Microsoft, they looked at their COGS, they looked like, “Okay, I'm offering a cloud. If I look at the COGS, who am I paying?”And it turns out, there was a bunch of hardware providers that had 30% margins or 70% margins. They're like, “Why am I paying Cisco these big margins? Why am I paying Dell these big margins?” Right? So, they had the exact same dilemma.And all of the arguments that they use now applied then, right? So, the exact same arguments, for example, “AWS, you know nothing about hardware. Why would you build hardware? You don't have the expertise. These guys sell to everybody in the world, you don't have the economies of scale.”So, all of the same arguments applied to them. And yet… and yes because it was part of COGS] that it impacted the share price, they can make the economic argument to actually build hardware teams and build chips. And so, they verticalized, right? And so, it just turns out if the infrastructure becomes parts of COGS, it makes sense to optimize that infrastructure. And I would say, the Big Three's foray into OEMs and hardware is a much, much, much bigger leap than an infrastructure company foraying into building their own infrastructure.Corey: There's a certain startup cost inherent to all these things. And the small version of that we had in every company that we started in a pre-cloud era: renting virtual computers from vendors was a thing, but it was still fraught and challenging and things that we use, then, like, GoGrid no longer exist, for good reason. But the alternative was, “Great, I'm going to start building and seeing if this thing has any traction.” Well, you need to go lease a rack somewhere and buy servers from Dell, and they're going to do the fast expedited option, which means only six short weeks until they show up in the data center and then gets sent away because they weren't expecting to receive them. And you wind up with this entire universe of hell between cross-connects and all the rest.And that's before you can ever get anything in front of customers or users to see what happens. Now, it's a swipe of a credit card away and your evening's experiments round up to 25 cents. That was significant. Having to make these significant tens of thousands of dollars of investment just to launch is no longer true. And I feel like that was a great equalizer in some respects.Martin: Yeah, I think that—Corey: And that cost has been borne by the astonishing level of investment that the cloud providers themselves have made. And that basically means that we don't have to. But it does come at a cost.Martin: I think it's also worth pointing out that it's much easier to stand up your own infrastructure now than it has been in the past, too. And so, I think that there's a gradient here, right? So, if you're building a SaaS app, [laugh] you would be crazy not to use the cloud, you just be absolutely insane, right? Like, what do you know about core infrastructure? You know, what do you know about building a back-end? Like, what do you know about operating these things? Go focus on your SaaS app.Corey: The calluses I used to have from crimping my own Ethernet patch cables in data centers have faded by now. I don't want them to come back. Yeah, we used to know how to do these things. Now, most people in most companies do not have that baseline of experience, for excellent reasons. And I wouldn't wish that on the current generation of engineers, except for the ones I dislike.Martin: However, that is if you're building an application. Almost all of my investments are people that are building infrastructure. [laugh]. They're already doing these hardcore backend things; that's what they do: they sell infrastructure. Would you think, like, someone, like, at Databricks doesn't understand how to run infr—of course it does. I mean, like, or Snowflake or whatever, right?And so, this is a gradient. On the extreme app end, you shouldn't be thinking about infrastructure; just use the cloud. Somewhere in the middle, maybe you start on the cloud, maybe you don't. As you get closer to being a cloud service, of course you're going to build your own infrastructure.Like, for example—listen, I mean, I've been mentioning Fly; I just think it's a great example. I mean, Fly is a next-generation CDN, that you can run compute on, where they build their own infrastructure—it's a great developer experience—and they would just be silly. Like, they couldn't even make the cost model work if they did it on the cloud. So clearly, there's a gradient here, and I just think that you would be remiss and probably negligent if you're selling software not to have this conversation, or at least do the analysis.Corey: This episode is sponsored in part by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on-premises, private cloud, and they just announced a fully-managed service on AWS and Azure called BigAnimal, all one word. Don't leave managing your database to your cloud vendor because they're too busy launching another half-dozen managed databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications—including Oracle—to the cloud. To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: I think there's also a philosophical shift, where a lot of the customers that I talk to about their AWS bills want to believe something that is often not true. And what they want to believe is that their AWS bill is a function of how many customers they have.Martin: Oh yeah.Corey: In practice, it is much more closely correlated with how many engineers they've hired. And it sounds like a joke, except that it's not. The challenge that you have when you choose to build in a data center is that you have bounds around your growth because there are capacity concerns. You are going to run out of power, cooling, and space to wind up having additional servers installed. In cloud, you have an unbounded growth problem.S3 is infinite storage, and the reason I'm comfortable saying that is that they can add hard drives faster than you can fill them. For all effective purposes, it is infinite amounts of storage. There is no forcing function that forces you to get rid of things. You spin up an instance, the natural state of it in a data center as a virtual machine or a virtual instance, is that it's going to stop working two to three years left on maintain when a raccoon hauls it off into the woods to make a nest or whatever the hell raccoons do. In cloud, you will retire before that instance does is it gets migrated to different underlying hosts, continuing to cost you however many cents per hour every hour until the earth crashes into the sun, or Amazon goes bankrupt.That is the trade-off you're making. There is no forcing function. And it's only money, which is a weird thing to say, but the failure mode of turning something off mistakenly that takes things down, well that's disastrous to your brand and your company. Just leaving it up, well, it's only money. It's never a top-of-mind priority, so it continues to build and continues to build and continues to build until you're really forced to reckon with a much larger problem.It is a form of technical debt, where you've kicked the can down the road until you can no longer kick that can. Then your options are either go ahead and fix it or go back and talk to you folks, and it's time for more money.Martin: Yeah. Or talk to you. [laugh].Corey: There is that.Martin: No seriously, I think everybody should, honestly. I think this is a board-level concern for every compa—I sit on a lot of boards; I see this. And this has organically become a board-level concern. I think it should become a conscious board-level concern of, you know, cloud costs, impact COGS. Any software company has it; it always becomes an issue, and so it should be treated as a first-class problem.And if you're not thinking through your options—and I think by the way, your company is a great option—but if you're not thinking to the options, then you're almost fiduciarily negligent. I think the vast, vast majority of people and vast majority of companies are going to stay on the cloud and just do some basic cost controls and some just basic hygiene and they're fine and, like, this doesn't touch them. But there are a set of companies, particularly those that sell infrastructure, where they may have to get more aggressive. And that ecosystem is now very vibrant, and there's a lot of shifts in it, and I think it's the most exciting place [laugh] in all of IT, like, personally in the industry.Corey: One question I have for you is where do you draw the line around infrastructure companies. I tend to have an evolving view of it myself, where things that are hard and difficult do not become harder with time. It used to require a deep-level engineer with a week to kill to wind up compiling and building a web server. Now, it is evolved and evolved and evolved; it is check a box on a webpage somewhere and you're serving a static website. Managed databases, I used to think, were something that were higher up the stack and not infrastructure. Today, I'd call them pretty clearly infrastructure.Things seem to be continually, I guess, a slipping beneath the waves to borrow an iceberg analogy. And it's only the stuff that you can see that is interesting and differentiated, on some level. I don't know where the industry is going at all, but I continue to think of infrastructure companies as being increasingly broad.Martin: Yeah, yeah, yeah. This is my favorite question. [laugh]. I'm so glad you asked. [laugh].Corey: This was not planned to be clear.Martin: No, no, no. Listen, I am such an infrastructure maximalist. And I've changed my opinion on this so much in the last three years. So, it used to be the case—and infrastructure has a long history of, like, calling the end of infrastructure. Like, every decade has been the end of infrastructure. It's like, you build the primitives and then everything else becomes an app problem, you know?Like, you build a cloud, and then we're done, you know? You build the PC and then we're done. And so, they are even very famous talks where people talk about the end of systems when we've be built everything right then. And I've totally changed my view. So, here's my current view.My current view is, infrastructure is the only, really, differentiation in systems, in all IT, in all software. It's just infrastructure. And the app layer is very important for the business, but the app layer always sits on infrastructure. And the differentiations in app is provided by the infrastructure. And so, the start of value is basically infrastructure.And the design space is so huge, so huge, right? I mean, we've moved from, like, PCs to cloud to data. Now, the cloud is decoupling and moving to the CDN tier. I mean, like, the front-end developers are building stuff in the browser. Like, there's just so much stuff to do that I think the value is always going to accrue to infrastructure.So, in my view, anybody that's improving the app accuracy or performance or correctness with technology is an infrastructure company, right? And the more of that you do, [laugh] the more infrastructure you are. And I think, you know, in 30 years, you and I are going to be old, and we're going to go back on this podcast. We're going to talk and there's going to be a whole bunch of infrastructure companies that are being created that have accrued a lot of value. I'm going to say one more thing, which is so—okay, this is a sneak preview for the people listening to this that nobody else has heard before.So Sarah, and I are back at it again, and—the brilliant Sarah, who did the first piece—and we're doing another study. And the study is if you look at public companies and you look at ones that are app companies versus infrastructure companies, where does the value accrue? And there's way, way more app companies; there's a ton of app companies, but it turns out that infrastructure companies have higher multiples and accrue more value. And that's actually a counter-narrative because people think that the business is the apps, but it just turns out that's where the differentiation is. So, I'm just an infra maximalist. I think you could be an infra person your entire career and it's the place to be. [laugh].Corey: And this is the real value that I see of looking at AWS bills. And our narrative is oh, we come in and we fix the horrifying AWS bill. And the naive pass is, “Oh, you cut the bill and make it lower?” Not always. Our primary focus has been on understanding it because you get a phone-number-looking bill from AWS. Great, you look at it, what's driving the cost? Storage.Okay, great. That doesn't mean anything to the company. They want to know what teams are doing this. What's it going to cost for them to add another thousand monthly active users? What is the increase in cost? How do they wind up identifying their bottlenecks? How do they track and assign portions of their COGS to different aspects of their service? How do they trace the flow of capital for their organization as they're serving their customers?And understanding the bill and knowing what to optimize and what not to becomes increasingly strategic business concern.Martin: Yeah.Corey: That's the fun part. That's the stuff I don't see that software has a good way of answering, just because there's no way to use an API to gain that kind of business context. When I started this place, I thought I was going to be building software. It turns out, there's so many conversations that have to happen as a part of this that cannot be replicated by software. I mean, honestly, my biggest competitor for all this stuff is Microsoft Excel because people want to try and do it themselves internally. And sometimes they do a great job, sometimes they don't, but it's understanding their drivers behind their cost. And I think that is what was often getting lost because the cloud obscures an awful lot of that.Martin: Yeah. I think even just summarize this whole thing pretty quickly, which is, like, I do think that organically, like, cloud cost has become a board-level issue. And I think that the shift that founders and execs should make is to just, like, treat it like a first-class problem upfront. So, what does that mean? Minimally, it means understanding how these things break down—A, to your point—B, there's a number of tools that actually help with onboarding of this stuff. Like, Vantage is one that I'm a fan of; it just provides some visibility.And then the third one is if you're selling Software as a Service, that's your core product or software, and particularly it's a infrastructure, if you don't actually do the analysis on, like, how this impacts your share price for different cloud costs, if you don't do that analysis, I would say your fiduciarily negligent, just because the impact would be so high, especially in this market. And so, I think, listen, these three things are pretty straightforward and I think anybody listening to this should consider them if you're running a company, or you're an executive company.Corey: Let's be clear, this is also the kind of problem that when you're sitting there trying to come up with an idea for a business that you can put on slide decks and then present to people like you, these sounds like the paradise of problems to have. Like, “Wow, we're successful and our business is so complex and scaled out that we don't know where exactly a lot of these cost drivers are coming from.” It's, “Yeah, that sounds amazing.” Like, I remember those early days, back when all I was able to do and spend time on and energy on was just down to the idea of, ohh, I'm getting business cards. That's awesome. That means I've made it as a business person.Spoiler: it did not. Having an aggressive Twitter presence, that's what made me as a business person. But then there's this next step and this next step and this next step and this next step, and eventually, you look around and realize just how overwrought everything you've built is and how untangling it just becomes a bit of a challenge and a hell of a mess. Now, the good part is at that point of success, you can bring people in, like, a CFO and a finance team who can do some deep-level analysis to help identify what COGS is—or in some cases, have some founders, explain what COGS is to you—and understand those structures and how you think about that. But it always feels like it's a trailing problem, not an early problem that people focus on.Martin: I'll tell you the reason. The reason is because this is a very new phenomenon that it's part of COGS. It's literally five years new. And so, we're just catching up. Even now, this discussion isn't what it was when we first wrote the post.Like, now people are pretty educated on, like, “Oh yeah, like, this is really an issue. Oh, yeah. It contributes to COGS. Oh, yeah. Like, our stock price gets hit.” Like, it's so funny to watch, like, the industry mature in real-time. And I think, like, going forward, it's just going to be obvious that this is a board-level issue; it's going to be obvious this is, like, a first-class consideration. But I agree with you. It's like, listen, like, the industry wasn't ready for it because we didn't have public companies. A lot of public companies, like, this is a real issue. I mean really we're talking about the last five, seven years.Corey: It really is neat, just in real time watching how you come up with something that sounds borderline heretical, and in a relatively short period of time, becomes accepted as a large-scale problem, and now it's now it is fallen off of the hype train into, “Yeah, this is something to be aware of.” And people's attention spans have already jumped to the next level and next generation of problem. It feels like this used to take way longer for these cycles, and now everything is so rapid that I almost worry that between the time we're recording this and the time that it publishes in a few weeks, what is going to have happened that makes this conversation irrelevant? I didn't used to have to think like that. Now, I do.Martin: Yeah, yeah, yeah, for sure. Well, just a couple of things. I want to talk about, like, one of the reasons that accelerated this, and then when I think is going forward. So, one of the reasons this was accelerated was just the macro downturn. Like, when we wrote the post, you could make the argument that nobody cares about margins because it's all about growth, right?And so, like—and even then, it still saved a bunch of money, but like, a lot of people were like, “Listen, the only thing that matters is growth.” Now, that's absolutely not the case if you look at public market valuations. I mean, people really care about free cash flow, they really care about profitability, and they really care about margins. And so, it's just really forced the issue. And it also, like, you know, made kind of what we were saying very, very clear.I would say, you know, as far as shifts that are going, I think one of the biggest shifts is for every back-end developer, there's, like, a hundred front-end developers. It's just crazy. And those front-end developers—Corey: A third of a DevOps engineer.Martin: [laugh]. True. I think those front-end developers are getting, like, better tools to build complete apps, right? Like, totally complete apps, right? Like they've got great JavaScript frameworks that coming out all the time.And so, you could argue that actually a secular technology change—which is that developers are now rebuilding apps as kind of front-end applications—is going to pull compute away from the clouds anyways, right? Like if instead of, like, the app being some back-end thing running in AWS, but instead is a front-end thing, you know, running in a browser at the CDN tier, while you're still using the Big Three clouds, it's being used in a very different way. And we may have to think about it again differently. Now, this, again, is a five-year going forward problem, but I do feel like there are big shifts that are even changing the way that we currently think about cloud now. And we'll see.Corey: And if those providers don't keep up and start matching those paradigms, there's going to be an intermediary shim layer of companies that wind up converting their resources and infrastructure into things that suit this new dynamic, and effectively, they're going to become the next version of, I don't know, Level 3, one of those big underlying infrastructure companies that most people have never heard of or have to think about because they're not doing anything that's perceived as interesting.Martin: Yeah, I agree. And I honestly think this is why Cloudflare and Cloudflare work is very interesting. This is why Fly is very interesting. It's a set of companies that are, like, “Hey, listen, like, workloads are moving to the front-end and, you know, you need compute closer to the user and multi-region is really important, et cetera.” So, even as we speak, we're seeing kind of shifts to the way the cloud is moving, which is just exciting. This is why it's, like, listen, infrastructure is everything. And, like, you and I like if we live to be 200, we can do [laugh] a great infrastructure work every year.Corey: I'm terrified, on some level, that I'll still be doing the exact same type of thing in 20 years.Martin: [laugh].Corey: I like solving different problems as we go. I really want to thank you for spending so much time talking to me today. If people want to learn more about what you're up to, slash beg you for other people's money or whatnot, where's the best place for them to find you?Martin: You know, we've got this amazing infrastructure Discord channel. [laugh].Corey: Really? I did not know that.Martin: I love it. It's, like, the best. Yeah, my favorite thing to do is drink coffee and talk about infrastructure. And like, I posted this on Twitter and we've got, like, 600 people. And it's just the best thing. So, that's honestly the best way to have these discussions. Maybe can you put, like, the link in, like, the show notes?Corey: Oh, absolutely. It is already there in the show notes. Check the show notes. Feel free to join the infrastructure Discord. I will be there waiting for you.Martin: Yeah, yeah, yeah. That'll be fantastic.Corey: Thank you so much for being so generous with your time. I appreciate it.Martin: This was great. Likewise, Corey. You're always a class act and I really appreciate that about you.Corey: I do my best. Martin Casado, general partner at Andreessen Horowitz. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment telling me that I got it completely wrong and what check you wrote makes you the most interesting.Announcer: The content here is for informational purposes only and should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security and is not directed at any investors or potential investors in any a16z fund. For more details, please see a16z.com/disclosures.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About MattMatt is a Sr. Architect in Belfast, an AWS DevTools Hero, Serverless Architect, Author and conference speaker. He is focused on creating the right environment for empowered teams to rapidly deliver business value in a well-architected, sustainable and serverless-first way.You can usually find him sharing reusable, well architected, serverless patterns over at cdkpatterns.com or behind the scenes bringing CDK Day to life.Links Referenced: Previous guest appearance: https://www.lastweekinaws.com/podcast/screaming-in-the-cloud/slinging-cdk-knowledge-with-matt-coulter/ The CDK Book: https://thecdkbook.com/ Twitter: https://twitter.com/NIDeveloper TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. One of the best parts about, well I guess being me, is that I can hold opinions that are… well, I'm going to be polite and call them incendiary, and that's great because I usually like to back them in data. But what happens when things change? What happens when I learn new things?Well, do I hold on to that original opinion with two hands at a death grip or do I admit that I was wrong in my initial opinion about something? Let's find out. My guest today returns from earlier this year. Matt Coulter is a senior architect since he has been promoted at Liberty Mutual. Welcome back, and thanks for joining me.Matt: Yeah, thanks for inviting me back, especially to talk about this topic.Corey: Well, we spoke about it a fair bit at the beginning of the year. And if you're listening to this, and you haven't heard that show, it's not that necessary to go into; mostly it was me spouting uninformed opinions about the CDK—the Cloud Development Kit, for those who are unfamiliar—I think of it more or less as what if you could just structure your cloud resources using a programming language you claim to already know, but in practice, copy and paste from Stack Overflow like the rest of us? Matt, you probably have a better description of what the CDK is in practice.Matt: Yeah, so we like to say it's imperative code written in a declarative way, or declarative code written in an imperative way. Either way, it lets you write code that produces CloudFormation. So, it doesn't really matter what you write in your script; the point is, at the end of the day, you still have the CloudFormation template that comes out of it. So, the whole piece of it is that it's a developer experience, developer speed play, that if you're from a background that you're more used to writing a programming language than a YAML, you might actually enjoy using the CDK over writing straight CloudFormation or SAM.Corey: When I first kicked the tires on the CDK, my first initial obstacle—which I've struggled with in this industry for a bit—is that I'm just good enough of a programmer to get myself in trouble. Whenever I wind up having a problem that StackOverflow doesn't immediately shine a light on, my default solution is to resort to my weapon of choice, which is brute force. That sometimes works out, sometimes doesn't. And as I went through the CDK, a couple of times in service to a project that I'll explain shortly, I made a bunch of missteps with it. The first and most obvious one is that AWS claims publicly that it has support in a bunch of languages: .NET, Python, there's obviously TypeScript, there's Go support for it—I believe that went generally available—and I'm sure I'm missing one or two, I think? Aren't I?Matt: Yeah, it's: TypeScript, JavaScript, Python Java.Net, and Go. I think those are the currently supported languages.Corey: Java. That's the one that I keep forgetting. It's the block printing to the script that is basically Java cursive. The problem I run into, and this is true of most things in my experience, when a company says that we have deployed an SDK for all of the following languages, there is very clearly a first-class citizen language and then the rest that more or less drift along behind with varying degrees of fidelity. In my experience, when I tried it for the first time in Python, it was not a great experience for me.When I learned just enough JavaScript, and by extension TypeScript, to be dangerous, it worked a lot better. Or at least I could blame all the problems I ran into on my complete novice status when it comes to JavaScript and TypeScript at the time. Is that directionally aligned with what you've experienced, given that you work in a large company that uses this, and presumably, once you have more than, I don't know, two developers, you start to take on aspects of a polyglot shop no matter where you are, on some level?Matt: Yeah. So personally, I jump between Java, Python, and TypeScript whenever I'm writing projects. So, when it comes to the CDK, you'd assume I'd be using all three. I typically stick to TypeScript and that's just because personally, I've had the best experience using it. For anybody who doesn't know the way CDK works for all the languages, it's not that they have written a custom, like, SDK for each of these languages; it's a case of it uses a Node process underneath them and the language actually interacts with—it's like the compiled JavaScript version is basically what they all interact with.So, it means there are some limitations on what you can do in that language. I can't remember the full list, but it just means that it is native in all those languages, but there are certain features that you might be like, “Ah,” whereas, in TypeScript, you can just use all of TypeScript. And my first inclination was actually, I was using the Python one and I was having issues with some compiler errors and things that are just caused by that process. And it's something that talking in the cdk.dev Slack community—there is actually a very active—Corey: Which is wonderful, I will point out.Matt: [laugh]. Thank you. There is actually, like, an awesome Python community in there, but if you ask them, they would all ask for improvements to the language. So, personally if someone's new, I always recommend they start with TypeScript and then branch out as they learn the CDK so they can understand is this a me problem, or is this a problem caused by the implementation?Corey: From my perspective, I didn't do anything approaching that level of deep dive. I took a shortcut that I find has served me reasonably well in the course of my career, when I'm trying to do something in Python, and you pull up a tutorial—which I'm a big fan of reading experience reports, and blog posts, and here's how to get started—and they all have the same problem, which is step one, “Run npm install.” And that's “Hmm, you know, I don't recall that being a standard part of the Python tooling.” It's clearly designed and interpreted and contextualized through a lens of JavaScript. Let's remove that translation layer, let's remove any weird issues I'm going to have in that transpilation process, and just talk in the language it written in. Will this solve my problems? Oh, absolutely not, but it will remove a subset of them that I am certain to go blundering into like a small lost child trying to cross an eight-lane freeway.Matt: Yeah. I've heard a lot of people say the same thing. Because the CDK CLI is a Node process, you need it no matter what language you use. So, if they were distributing some kind of universal binary that just integrated with the languages, it would definitely solve a lot of people's issues with trying to combine languages at deploy time.Corey: One of the challenges that I've had as I go through the process of iterating on the project—but I guess I should probably describe it for those who have not been following along with my misadventures; I write blog posts about it from time to time because I need a toy problem to kick around sometimes because my consulting work is all advisory and I don't want to be a talking head-I have a Twitter client called lasttweetinaws.com. It's free; go and use it. It does all kinds of interesting things for authoring Twitter threads.And I wanted to deploy that to a bunch of different AWS regions, as it turns out, 20 or so at the moment. And that led to a lot of interesting projects and having to learn how to think about these things differently because no one sensible deploys an application simultaneously to what amounts to every AWS region, without canary testing, and having a phased rollout in the rest. But I'm reckless, and honestly, as said earlier, a bad programmer. So, that works out. And trying to find ways to make this all work and fit together led iteratively towards me discovering that the CDK was really kind of awesome for a lot of this.That said, there were definitely some fairly gnarly things I learned as I went through it, due in no small part to help I received from generous randos in the cdk.dev Slack team. And it's gotten to a point where it's working, and as an added bonus, I even mostly understand what he's doing, which is just kind of wild to me.Matt: It's one of those interesting things where because it's a programming language, you can use it out of the box the way it's designed to be used where you can just write your simple logic which generates your CloudFormation, or you can do whatever crazy logic you want to do on top of that to make your app work the way you want it to work. And providing you're not in a company like Liberty, where I'm going to do a code review, if no one's stopping you, you can do your crazy experiments. And if you understand that, it's good. But I do think something like the multi-region deploy, I mean, with CDK, if you'd have a construct, it takes in a variable that you can just say what the region is, so you can actually just write a for loop and pass it in, which does make things a lot easier than, I don't know, try to do it with a YAML, which you can pass in parameters, but you're going to get a lot more complicated a lot quicker.Corey: The approach that I took philosophically was I wrote everything in a region-agnostic way. And it would be instantiated and be told what region to run it in as an environment variable that CDK deploy was called. And then I just deploy 20 simultaneous stacks through GitHub Actions, which invoke custom runners that runs inside of a Lambda function. And that's just a relatively basic YAML file, thanks to the magic of GitHub Actions matrix jobs. So, it fires off 20 simultaneous processes and on every commit to the main branch, and then after about two-and-a-half minutes, it has been deployed globally everywhere and I get notified on anything that fails, which is always fun and exciting to learn those things.That has been, overall, just a really useful experiment and an experience because you're right, you could theoretically run this as a single CDK deploy and then wind up having an iterate through a list of regions. The challenge I have there is that unless I start getting into really convoluted asynchronous concurrency stuff, it feels like it'll just take forever. At two-and-a-half minutes a region times 20 regions, that's the better part of an hour on every deploy and no one's got that kind of patience. So, I wound up just parallelizing it a bit further up the stack. That said, I bet they are relatively straightforward ways, given the async is a big part of JavaScript, to do this simultaneously.Matt: One of the pieces of feedback I've seen about CDK is if you have multiple stacks in the same project, it'll deploy them one at a time. And that's just because it tries to understand the dependencies between the stacks and then it works out which one should go first. But a lot of people have said, “Well, I don't want that. If I have 20 stacks, I want all 20 to go at once the way you're saying.” And I have seen that people have been writing plugins to enable concurrent deploys with CDK out of the box. So, it may be something that it's not an out-of-the-box feature, but it might be something that you can pull in a community plug-in to actually make work.Corey: Most of my problems with it at this point are really problems with CloudFormation. CloudFormation does not support well, if at all, secure string parameters from the AWS Systems Manager parameter store, which is my default go-to for secret storage, and Secrets Manager is supported, but that also cost 40 cents a month per secret. And not for nothing, I don't really want to have all five secrets deployed to Secrets Manager in every region this thing is in. I don't really want to pay $20 a month for this basically free application, just to hold some secrets. So, I wound up talking to some folks in the Slack channel and what we came up with was, I have a centralized S3 bucket that has a JSON object that lives in there.It's only accessible from the deployment role, and it grabs that at deploy time and stuffs it into environment variables when it pushes these things out. That's the only stateful part of all of this. And it felt like that is, on some level, a pattern that a lot of people would benefit from if it had better native support. But the counterargument that if you're only deploying to one or two regions, then Secrets Manager is the right answer for a lot of this and it's not that big of a deal.Matt: Yeah. And it's another one of those things, if you're deploying in Liberty, we'll say, “Well, your secret is unencrypted at runtime, so you probably need a KMS key involved in that,” which as you know, the costs of KMS, it depends on if it's a personal solution or if it's something for, like, a Fortune 100 company. And if it's personal solution, I mean, what you're saying sounds great that it's IAM restricted in S3, and then that way only at deploy time can be read; it actually could be a custom construct that someone can build and publish out there to the construct library—or the construct hub, I should say.Corey: To be clear, the reason I'm okay with this, from a security perspective is one, this is in a dedicated AWS account. This is the only thing that lives in that account. And two, the only API credentials we're talking about are the application-specific credentials for this Twitter client when it winds up talking to the Twitter API. Basically, if you get access to these and are able to steal them and deploy somewhere else, you get no access to customer data, you get—or user data because this is not charge for anything—you get no access to things that have been sent out; all you get to do is submit tweets to Twitter and it'll have the string ‘Last Tweet in AWS' as your client, rather than whatever normal client you would use. It's not exactly what we'd call a high-value target because all the sensitive to a user data lives in local storage in their browser. It is fully stateless.Matt: Yeah, so this is what I mean. Like, it's the difference in what you're using your app for. Perfect case of, you can just go into the Twitter app and just withdraw those credentials and do it again if something happens, whereas as I say, if you're building it for Liberty, that it will not pass a lot of our Well-Architected reviews, just for that reason.Corey: If I were going to go and deploy this at a more, I guess, locked down environment, I would be tempted to find alternative approaches such as having it stored encrypted at rest via KMS in S3 is one option. So, is having global DynamoDB tables that wind up grabbing those things, even grabbing it at runtime if necessary. There are ways to make that credential more secure at rest. It's just, I look at this from a real-world perspective of what is the actual attack surface on this, and I have a really hard time just identifying anything that is going to be meaningful with regard to an exploit. If you're listening to this and have a lot of thoughts on that matter, please reach out I'm willing to learn and change my opinion on things.Matt: One thing I will say about the Dynamo approach you mentioned, I'm not sure everybody knows this, but inside the same Dynamo table, you can scope down a row. You can be, like, “This row and this field in this row can only be accessed from this one Lambda function.” So, there's a lot of really awesome security features inside DynamoDB that I don't think most people take advantage of, but they open up a lot of options for simplicity.Corey: Is that tied to the very recent announcement about Lambda getting SourceArn as a condition key? In other words, you can say, “This specific Lambda function,” as opposed to, “A Lambda in this account?” Like that was a relatively recent Advent that I haven't fully explored the nuances of.Matt: Yeah, like, that has opened a lot of doors. I mean, the Dynamo being able to be locked out in your row has been around for a while, but the new Lambda from SourceArn is awesome because, yeah, as you say, you can literally say this thing, as opposed to, you have to start going into tags, or you have to start going into something else to find it.Corey: So, I want to talk about something you just alluded to, which is the Well-Architected Framework. And initially, when it launched, it was a whole framework, and AWS made a lot of noise about it on keynote stages, as they are want to do. And then later, they created a quote-unquote, “Well-Architected Tool,” which let's be very direct, it's the checkbox survey form, at least the last time I looked at it. And they now have the six pillars of the Well-Architected Framework where they talk about things like security, cost, sustainability is the new pillar, I don't know, absorbency, or whatever the remainders are. I can't think of them off the top of my head. How does that map to your experience with the CDK?Matt: Yeah, so out of the box, the CDK from day one was designed to have sensible defaults. And that's why a lot of the things you deploy have opinions. I talked to a couple of the Heroes and they were like, “I wish it had less opinions.” But that's why whenever you deploy something, it's got a bunch of configuration already in there. For me, in the CDK, whenever I use constructs, or stacks, or deploying anything in the CDK, I always build it in a well-architected way.And that's such a loaded sentence whenever you say the word ‘well-architected,' that people go, “What do you mean?” And that's where I go through the six pillars. And in Liberty, we have a process, it used to be called SCORP because it was five pillars, but not SCORPS [laugh] because they added sustainability. But that's where for every stack, we'll go through it and we'll be like, “Okay, let's have the discussion.” And we will use the tool that you mentioned, I mean, the tool, as you say, it's a bunch of tick boxes with a text box, but the idea is we'll get in a room and as we build the starter patterns or these pieces of infrastructure that people are going to reuse, we'll run the well-architected review against the framework before anybody gets to generate it.And then we can say, out of the box, if you generate this thing, these are the pros and cons against the Well-Architected Framework of what you're getting. Because we can't make it a hundred percent bulletproof for your use case because we don't know it, but we can tell you out of the box, what it does. And then that way, you can keep building so they start off with something that is well documented how well architected it is, and then you can start having—it makes it a lot easier to have those conversations as they go forward. Because you just have to talk about the delta as they start adding their own code. Then you can and you go, “Okay, you've added these 20 lines. Let's talk about what they do.” And that's why I always think you can do a strong connection between infrastructure-as-code and well architected.Corey: As I look through the actual six pillars of the Well-Architected Framework: sustainability, cost optimization, performance, efficiency, reliability, security, and operational excellence, as I think through the nature of what this shitpost thread Twitter client is, I am reasonably confident across all of those pillars. I mean, first off, when it comes to the cost optimization pillar, please, don't come to my house and tell me how that works. Yeah, obnoxiously the security pillar is sort of the thing that winds up causing a problem for this because this is an account deployed by Control Tower. And when I was getting this all set up, my monthly cost for this thing was something like a dollar in charges and then another sixteen dollars for the AWS config rule evaluations on all of the deploys, which is… it just feels like a tax on going about your business, but fine, whatever. Cost and sustainability, from my perspective, also tend to be hand-in-glove when it comes to this stuff.When no one is using the client, it is not taking up any compute resources, it has no carbon footprint of which to speak, by my understanding, it's very hard to optimize this down further from a sustainability perspective without barging my way into the middle of an AWS negotiation with one of its power companies.Matt: So, for everyone listening, watch as we do a live well-architected review because—Corey: Oh yeah, I expect—Matt: —this is what they are. [laugh].Corey: You joke; we should do this on Twitter one of these days. I think would be a fantastic conversation. Or Twitch, or whatever the kids are using these days. Yeah.Matt: Yeah.Corey: And again, if so much of it, too, is thinking about the context. Security, you work for one of the world's largest insurance companies. I shitpost for a living. The relative access and consequences of screwing up the security on this are nowhere near equivalent. And I think that's something that often gets lost, per the perfect be the enemy of the good.Matt: Yeah that's why, unfortunately, the Well-Architected Tool is quite loose. So, that's why they have the Well-Architected Framework, which is, there's a white paper that just covers anything which is quite big, and then they wrote specific lenses for, like, serverless or other use cases that are shorter. And then when you do a well-architected review, it's like loose on, sort of like, how are you applying the principles of well-architected. And the conversation that we just had about security, so you would write that down in the box and be, like, “Okay, so I understand if anybody gets this credential, it means they can post this Last Tweet in AWS, and that's okay.”Corey: The client, not the Twitter account, to be clear.Matt: Yeah. So, that's okay. That's what you just mark down in the well-architected review. And then if we go to day one on the future, you can compare it and we can go, “Oh. Okay, so last time, you said this,” and you can go, “Well, actually, I decided to—” or you just keep it as a note.Corey: “We pivoted. We're a bank now.” Yeah.Matt: [laugh]. So, that's where—we do more than tweets now. We decided to do microtransactions through cryptocurrency over Twitter. I don't know but if you—Corey: And that ends this conversation. No no. [laugh].Matt: [laugh]. But yeah, so if something changes, that's what the well-architected reviews for. It's about facilitating the conversation between the architect and the engineer. That's all it is.Corey: This episode is sponsored in part by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on-premises, private cloud, and they just announced a fully-managed service on AWS and Azure called BigAnimal, all one word. Don't leave managing your database to your cloud vendor because they're too busy launching another half-dozen managed databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications—including Oracle—to the cloud. To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: And the lens is also helpful in that this is a serverless application. So, we're going to view it through that lens, which is great because the original version of the Well-Architected Tool is, “Oh, you built this thing entirely in Lambda? Have you bought some reserved instances for it?” And it's, yeah, why do I feel like I have to explain to AWS how their own systems work? This makes it a lot more streamlined and talks about this, though, it still does struggle with the concept of—in my case—a stateless app. That is still something that I think is not the common path. Imagine that: my code is also non-traditional. Who knew?Matt: Who knew? The one thing that's good about it, if anybody doesn't know, they just updated the serverless lens about, I don't know, a week or two ago. So, they added in a bunch of more use cases. So, if you've read it six months ago, or even three months ago, go back and reread it because they spent a good year updating it.Corey: Thank you for telling me that. That will of course wind up in next week's issue of Last Week in AWS. You can go back and look at the archives and figure out what week record of this then. Good work. One thing that I have learned as well as of yesterday, as it turns out, before we wound up having this recording—obviously because yesterday generally tends to come before today, that is a universal truism—is it I had to do a bit of refactoring.Because what I learned when I was in New York live-tweeting the AWS Summit, is that the Route 53 latency record works based upon where your DNS server is. Yeah, that makes sense. I use Tailscale and wind up using my Pi-hole, which lives back in my house in San Francisco. Yeah, I was always getting us-west-1 from across the country. Cool.For those weird edge cases like me—because this is not the common case—how do I force a local region? Ah, I'll give it its own individual region prepend as a subdomain. Getting that to work with both the global lasttweetinaws.com domain as well as the subdomain on API Gateway through the CDK was not obvious on how to do it.Randall Hunt over at Caylent was awfully generous and came up with a proof-of-concept in about three minutes because he's Randall, and that was extraordinarily helpful. But a challenge I ran into was that the CDK deploy would fail because the way that CloudFormation was rendered in the way it was trying to do stuff, “Oh, that already has that domain affiliated in a different way.” I had to do a CDK destroy then a CDK deploy for each one. Now, not the end of the world, but it got me thinking, everything that I see around the CDK more or less distills down to either greenfield or a day one experience. That's great, but throw it all away and start over is often not what you get to do.And even though Amazon says it's always day one, those of us in, you know, real companies don't get to just treat everything as brand new and throw away everything older than 18 months. What is the day two experience looking like for you? Because you clearly have a legacy business. By legacy, I of course, use it in the condescending engineering term that means it makes actual money, rather than just telling really good stories to venture capitalists for 20 years.Matt: Yeah. We still have mainframes running that make a lot of money. So, I don't mock legacy at all.Corey: “What's that piece of crap do?” “Well, about $4 billion a year in revenue. Perhaps show some respect.” It's a common refrain.Matt: Yeah, exactly. So yeah, anyone listening, don't mock legacy because as Corey says, it is running the business. But for us when it comes to day two, it's something that I'm actually really passionate about this in general because it is really easy. Like I did it with CDK patterns, it's really easy to come out and be like, “Okay, we're going to create a bunch of starter patterns, or quickstarts”—or whatever flavor that you came up with—“And then you're going to deploy this thing, and we're going to have you in production and 30 seconds.” But even day one later that day—not even necessarily day two—it depends on who it was that deployed it and how long they've been using AWS.So, you hear these stories of people who deployed something to experiment, and they either forget to delete, it cost them a lot of money or they tried to change it and it breaks because they didn't understand what was in it. And this is where the community starts to diverge in their opinions on what AWS CDK should be. There's a lot of people who think that at the minute CDK, even if you create an abstraction in a construct, even if I create a construct and put it in the construct library that you get to use, it still unravels and deploys as part of your deploy. So, everything that's associated with it, you don't own and you technically need to understand that at some point because it might, in theory, break. Whereas there's a lot of people who think, “Okay, the CDK needs to go server side and an abstraction needs to stay an abstraction in the cloud. And then that way, if somebody is looking at a 20-line CDK construct or stack, then it stays 20 lines. It never unravels to something crazy underneath.”I mean, that's one pro tip thing. It'd be awesome if that could work. I'm not sure how the support for that would work from a—if you've got something running on the cloud, I'm pretty sure AWS [laugh] aren't going to jump on a call to support some construct that I deployed, so I'm not sure how that will work in the open-source sense. But what we're doing at Liberty is the other way. So, I mean, we famously have things like the software accelerator that lets you pick a pattern or create your pipelines and you're deployed, but now what we're doing is we're building a lot of telemetry and automated information around what you deployed so that way—and it's all based on Well-Architected, common theme. So, that way, what you can do is you can go into [crosstalk 00:26:07]—Corey: It's partially [unintelligible 00:26:07], and partially at a glance, figure out okay, are there some things that can be easily remediated as we basically shift that whole thing left?Matt: Yeah, so if you deploy something, and it should be good the second you deploy it, but then you start making changes. Because you're Corey, you just start adding some stuff and you deploy it. And if it's really bad, it won't deploy. Like, that's the Liberty setup. There's a bunch of rules that all go, “Okay, that's really bad. That'll cause damage to customers.”But there's a large gap between bad and good that people don't really understand the difference that can cost a lot of money or can cause a lot of grief for developers because they go down the wrong path. So, that's why what we're now building is, after you deploy, there's a dashboard that'll just come up and be like, “Hey, we've noticed that your Lambda function has too little memory. It's going to be slow. You're going to have bad cold starts.” Or you know, things like that.The knowledge that I have had the gain through hard fighting over the past couple of years putting it into automation, and that way, combined with the well-architected reviews, you actually get me sitting in a call going, “Okay, let's talk about what you're building,” that hopefully guides people the right way. But I still think there's so much more we can do for day two because even if you deploy the best solution today, six months from now, AWS are releasing ten new services that make it easier to do what you just did. So, someone also needs to build something that shows you the delta to get to the best. And that would involve AWS or somebody thinking cohesively, like, these are how we use our products. And I don't think there's a market for it as a third-party company, unfortunately, but I do think that's where we need to get to, that at day two somebody can give—the way we're trying to do for Liberty—advice, automated that says, “I see what you're doing, but it would be better if you did this instead.”Corey: Yeah, I definitely want to spend more time thinking about these things and analyzing how we wind up addressing them and how we think about them going forward. I learned a lot of these lessons over a decade ago. I was fairly deep into using Puppet, and came to the fair and balanced conclusion that Puppet was a steaming piece of crap. So, the solution was that I was one of the very early developers behind SaltStack, which was going to do everything right. And it was and it was awesome and it was glorious, right up until I saw an environment deployed by someone else who was not as familiar with the tool as I was, at which point I realized hell is other people's use cases.And the way that they contextualize these things, you craft a finely balanced torque wrench, it's a thing of beauty, and people complain about the crappy hammer. “You're holding it wrong. No, don't do it that way.” So, I have an awful lot of sympathy for people building platform-level tooling like this, where it works super well for the use case that they're in, but not necessarily… they're not necessarily aligned in other ways. It's a very hard nut to crack.Matt: Yeah. And like, even as you mentioned earlier, if you take one piece of AWS, for example, API Gateway—and I love the API Gateway team; if you're listening, don't hate on me—but there's, like, 47,000 different ways you can deploy an API Gateway. And the CDK has to cover all of those, it would be a lot easier if there was less ways that you could deploy the thing and then you can start crafting user experiences on a platform. But whenever you start thinking that every AWS component is kind of the same, like think of the amount of ways you're can deploy a Lambda function now, or think of the, like, containers. I'll not even go into [laugh] the different ways to run containers.If you're building a platform, either you support it all and then it sort of gets quite generic-y, or you're going to do, like, what serverless cloud are doing though, like Jeremy Daly is building this unique experience that's like, “Okay, the code is going to build the infrastructure, so just build a website, and we'll do it all behind it.” And I think they're really interesting because they're sort of opposites, in that one doesn't want to support everything, but should theoretically, for their slice of customers, be awesome, and then the other ones, like, “Well, let's see what you're going to do. Let's have a go at it and I should hopefully support it.”Corey: I think that there's so much that can be done on this. But before we wind up calling it an episode, I had one further question that I wanted to explore around the recent results of the community CDK survey that I believe is a quarterly event. And I read the analysis on this, and I talked about it briefly in the newsletter, but it talks about adoption and a few other aspects of it. And one of the big things it looks at is the number of people who are contributing to the CDK in an open-source context. Am I just thinking about this the wrong way when I think that, well, this is a tool that helps me build out cloud infrastructure; me having to contribute code to this thing at all is something of a bug, whereas yeah, I want this thing to work out super well—Docker is open-source, but you'll never see me contributing things to Docker ever, as a pull request, because it does, as it says on the tin; I don't have any problems that I'm aware of that, ooh, it should do this instead. I mean, I have opinions on that, but those aren't pull requests; those are complete, you know, shifts in product strategy, which it turns out is not quite done on GitHub.Matt: So, it's funny I, a while ago, was talking to a lad who was the person who came up with the idea for the CDK. And CDK is pretty much the open-source project for AWS if you look at what they have. And the thought behind it, it's meant to evolve into what people want and need. So yes, there is a product manager in AWS, and there's a team fully dedicated to building it, but the ultimate aspiration was always it should be bigger than AWS and it should be community-driven. Now personally, I'm not sure—like you just said it—what the incentive is, given that right now CDK only works with CloudFormation, which means that you are directly helping with an AWS tool, but it does give me hope for, like, their CDK for Terraform, and their CDK for Kubernetes, and there's other flavors based on the same technology as AWS CDK that potentially could have a thriving open-source community because they work across all the clouds. So, it might make more sense for people to jump in there.Corey: Yeah, I don't necessarily think that there's a strong value proposition as it stands today for the idea of the CDK becoming something that works across other cloud providers. I know it technically has the capability, but if I think that Python isn't quite a first-class experience, I don't even want to imagine what other providers are going to look like from that particular context.Matt: Yeah, and that's from what I understand, I haven't personally jumped into the CDK for Terraform and we didn't talk about it here, but in CDK, you get your different levels of construct. And is, like, a CloudFormation-level construct, so everything that's in there directly maps to a property in CloudFormation, and then L2 is AWS's opinion on safe defaults, and then L3 is when someone like me comes along and turns it into something that you may find useful. So, it's a pattern. As far as I know, CDK for Terraform is still on L1. They haven't got the rich collection—Corey: And L4 is just hiring you as a consultant—Matt: [laugh].Corey: —to come in fix my nonsense for me?Matt: [laugh]. That's it. L4 could be Pulumi recently announced that you can use AWS CDK constructs inside it. But I think it's one of those things where the constructs, if they can move across these different tools the way AWS CDK constructs now work inside Pulumi, and there's a beta version that works inside CDK for Terraform, then it may or may not make sense for people to contribute to this stuff because we're not building at a higher level. It's just the vision is hard for most people to get clear in their head because it needs articulated and told as a clear strategy.And then, you know, as you said, it is an AWS product strategy, so I'm not sure what you get back by contributing to the project, other than, like, Thorsten—I should say, so Thorsten who wrote the book with me, he is the number three contributor, I think, to the CDK. And that's just because he is such a big user of it that if he sees something that annoys him, he just comes in and tries to fix it. So, the benefit is, he gets to use the tool. But he is a super user, so I'm not sure, outside of super users, what the use case is.Corey: I really want to thank you for, I want to say spending as much time talking to me about this stuff as you have, but that doesn't really go far enough. Because so much of how I think about this invariably winds up linking back to things that you have done and have been advocating for in that community for such a long time. If it's not you personally, just, like, your fingerprints are all over this thing. So, it's one of those areas where the entire software developer ecosystem is really built on the shoulders of others who have done a lot of work that came before. Often you don't get any visibility of who those people are, so it's interesting whenever I get to talk to someone whose work I have directly built upon that I get to say thank you. So, thank you for this. I really do appreciate how much more straightforward a lot of this is than my previous approach of clicking in the console and then lying about it to provision infrastructure.Matt: Oh, no worries. Thank you for the thank you. I mean, at the end of the day, all of this stuff is just—it helps me as much as it helps everybody else, and we're all trying to do make everything quicker for ourselves, at the end of the day.Corey: If people want to learn more about what you're up to, where's the best place to find you these days? They can always take a job at Liberty; I hear good things about it.Matt: Yeah, we're always looking for people at Liberty, so come look up our careers. But Twitter is always the best place. So, I'm @NIDeveloper on Twitter. You should find me pretty quickly, or just type Matt Coulter into Google, you'll get me.Corey: I like it. It's always good when it's like, “Oh, I'm the top Google result for my own name.” On some level, that becomes an interesting thing. Some folks into it super well, John Smith has some challenges, but you know, most people are somewhere in the middle of that.Matt: I didn't used to be number one, but there's a guy called the Kangaroo Kid in Australia, who is, like, a stunt driver, who was number one, and [laugh] I always thought it was funny if people googled and got him and thought it was me. So, it's not anymore.Corey: Thank you again for, I guess, all that you do. And of course, taking the time to suffer my slings and arrows as I continue to revise my opinion of the CDK upward.Matt: No worries. Thank you for having me.Corey: Matt Coulter, senior architect at Liberty Mutual. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice and leave an angry comment as well that will not actually work because it has to be transpiled through a JavaScript engine first.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About AlexAlex holds a Ph.D. in Computer Science and Engineering from UC San Diego, and has spent over a decade building high-performance, robust data management and processing systems. As an early member of a couple fast-growing startups, he's had the opportunity to wear a lot of different hats, serving at various times as an individual contributor, tech lead, manager, and executive. He also had a brief stint as a Cloud Economist with the Duckbill Group, helping AWS customers save money on their AWS bills. He's currently a freelance data engineering consultant, helping his clients build, manage, and maintain their data infrastructure. He lives in Los Angeles, CA.Links Referenced: Company website: https://bitsondisk.com Twitter: https://twitter.com/alexras LinkedIn: https://www.linkedin.com/in/alexras/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: I come bearing ill tidings. Developers are responsible for more than ever these days. Not just the code that they write, but also the containers and the cloud infrastructure that their apps run on. Because serverless means it's still somebody's problem. And a big part of that responsibility is app security from code to cloud. And that's where our friend Snyk comes in. Snyk is a frictionless security platform that meets developers where they are - Finding and fixing vulnerabilities right from the CLI, IDEs, Repos, and Pipelines. Snyk integrates seamlessly with AWS offerings like code pipeline, EKS, ECR, and more! As well as things you're actually likely to be using. Deploy on AWS, secure with Snyk. Learn more at Snyk.co/scream That's S-N-Y-K.co/screamCorey: DoorDash had a problem. As their cloud-native environment scaled and developers delivered new features, their monitoring system kept breaking down. In an organization where data is used to make better decisions about technology and about the business, losing observability means the entire company loses their competitive edge. With Chronosphere, DoorDash is no longer losing visibility into their applications suite. The key? Chronosphere is an open-source compatible, scalable, and reliable observability solution that gives the observability lead at DoorDash business, confidence, and peace of mind. Read the full success story at snark.cloud/chronosphere. That's snark.cloud slash C-H-R-O-N-O-S-P-H-E-R-E.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I am joined this week by a returning guest, who… well, it's a little bit complicated and more than a little bittersweet. Alex Rasmussen was a principal cloud economist here at The Duckbill Group until he committed an unforgivable sin. That's right. He gave his notice. Alex, thank you for joining me here, and what have you been up to, traitor?Alex: [laugh]. Thank you for having me back, Corey.Corey: Of course.Alex: At time of recording, I am restarting my freelance data engineering business, which was dormant for the sadly brief time that I worked with you all at The Duckbill Group. And yeah, so that's really what I've been up to for the last few days. [laugh].Corey: I want to be very clear that I am being completely facetious when I say this. When someone is considering, “Well, am I doing what I really want to be doing?” And if the answer is no, too many days in a row, yeah, you should find something that aligns more with what you want to do. And anyone who's like, “Oh, you're leaving? Traitor, how could you do that?” Yeah, those people are trash. You don't want to work with trash.I feel I should clarify that this is entirely in jest and I could not be happier that you are finding things that are more aligned with aspects of what you want to be doing. I am serious when I say that, as a company, we are poorer for your loss. You have been transformative here across a number of different axes that we will be going into over the course of this episode.Alex: Well, thank you very much, I really appreciate that. And I came to a point where I realized, you know, the old saying, “You don't know what you got till it's gone?” I realized, after about six months of working with Duckbill Group that I missed building stuff, I missed building data systems, I missed being a full-time data person. And I'm really excited to get back to that work, even though I'll definitely miss working with everybody on the team. So yeah.Corey: There are a couple of things that I found really notable about your time working with us. One of them was that even when you wound up applying to work here, you were radically different than—well, let's be direct here—than me. We are almost polar opposites in a whole bunch of ways. I have an eighth-grade education; you have a PhD in computer science and engineering from UCSD. And you are super-deep into the world of data, start to finish, whereas I have spent my entire career on things that are stateless because I am accident prone, and when you accidentally have a problem with the database, you might not have a company anymore, but we can all laugh as we reprovision the web server fleet.We just went in very different directions as far as what we found interesting throughout our career, more or less. And we were not quite sure how it was going to manifest in the context of cloud economics. And I can say now that we have concluded the experiment, that from my perspective, it went phenomenally well. Because the exact areas that I am weak at are where you excel. And, on some level, I would say that you're not necessarily as weak in your weak areas as I am in mine, but we want to reinforce it and complementing each other rather than, “Well, we now have a roomful of four people who are all going to yell at you about the exact same thing.” We all went in different directions, which I thought was really neat.Alex: I did too. And honestly, I learned a tremendous, tremendous amount in my time at Duckbill Group. I think the window into just how complex and just how vast the ecosystem of services within AWS is, and kind of how they all ping off of each other in these very complicated ways was really fascinating, fascinating stuff. But also just an insight into just what it takes to get stuff done when you're talking with—you know, so most of my clientele to date have been small to medium-sized businesses, you know, small as two people; as big as a few hundred people. But I wasn't working with Fortune 1000 companies like Duckbill Group regularly does, and an insight into just, number one, what it takes to get things done inside of those organizations, but also what it takes to get things done with AWS when you're talking about, you know, for instance, contracts that are tens, or hundreds of millions of dollars in total contract value. And just what that involves was just completely eye-opening for me.Corey: From my perspective, what I found—I guess, in hindsight, it should have been more predictable than it was—but you talk about having a background and an abiding passion for the world of data, and I'm sitting here thinking, that's great. We have all this data in the form of the Cost and Usage Reports and the bills, and I forgot the old saw that yeah, if it fits in RAM, it's not a big data problem. And yeah, in most cases, what we have tends to fit in RAM. I guess you don't tend to find things interesting until Microsoft Excel gives up and calls uncle.Alex: I don't necessarily know that that's true. I think that there are plenty of problems to be had in the it fits in RAM space, precisely because so much of it fits in RAM. And I think that, you know, particularly now that, you know—I think there's it's a very different world that we live in from the world that we lived in ten years ago, where ten years ago—Corey: And right now I'm talking to you on a computer with 128 gigs of RAM, and it—Alex: Well, yeah.Corey: —that starts to look kind of big data-y.Alex: Well, not only that, but I think on the kind of big data side, right? When you had to provision your own Hadoop cluster, and after six months of weeping tears of blood, you managed to get it going, right, at the end of that process, you went, “Okay, I've got this big, expensive thing and I need this group of specialists to maintain it all. Now, what the hell do I do?” Right? In the intervening decade, largely due to the just crushing dominance of the public clouds, that problem—I wouldn't call that problem solved, but for all practical purposes, at all reasonable scales, there's a solution that you can just plug in a credit card and buy.And so, now the problem, I think, becomes much more high level, right, than it used to be. Used to be talking about how well you know, how do I make this MapReduce job as efficient as it possibly can be made? Nobody really cares about that anymore. You've got a query planner; it executes a query; it'll probably do better than you can. Now, I think the big challenges are starting to be more in the area of, again, “How do I know what I have? How do I know who's touched it recently? How do I fix it when it breaks? How do I even organize an organization that can work effectively with data at petabyte scale and say anything meaningful about it?”And so, you know, I think that the landscape is shifting. One of the reasons why I love this field so much is that the landscape is shifting very rapidly and as soon as we think, “Ah yes. We have solved all of the problems.” Then immediately, there are a hundred new problems to solve.Corey: For me, what I found, I guess, one of the most eye-opening things about having you here is your actual computer science background. Historically, we have biased for folks who have come up from the ops side of the world. And that lends itself to a certain understanding. And, yes, I've worked with developers before; believe it or not, I do understand how folks tend to think in that space. I have not a complete naive fool when it comes to these things.But what I wasn't prepared for was the nature of our internal, relatively casual conversations about a bunch of different things, where we'll be on a Zoom chat or something, and you will just very casually start sharing your screen, fire up a Jupyter Notebook and start writing code as you're talking to explain what it is you're talking about and watching it render in real time. And I'm sitting here going, “Huh, I can't figure out whether we should, like, wind up giving him a raise or try to burn him as a witch.” I could really see it going either way. Because it was magic and transformative from my perspective.Alex: Well, thank you. I mean, I think that part of what I am very grateful for is that I've had an opportunity to spend a considerable period of time in kind of both the academic and industrial spaces. I got a PhD, basically kept going to school until somebody told me that I had to stop, and then spent a lot of time at startups and had to do a lot of different kinds of work just to keep the wheels attached to the bus. And so, you know, when I arrived at Duckbill Group, I kind of looked around and said, “Okay, cool. There's all the stuff that's already here. That's awesome. What can I do to make that better?” And taking my lens so to speak, and applying it to those problems, and trying to figure out, like, “Okay, well as a cloud economist, what do I need to do right now that sucks? And how do I make it not suck?”Corey: It probably involves a Managed NAT Gateway.Alex: Whoa, God. And honestly, like, I spent a lot of time developing a bunch of different tools that were really just there in the service of that. Like, take my job, make it easier. And I'm really glad that you liked what you saw there.Corey: It was interesting watching how we wound up working together on things. Like, there's a blog post that I believe is out by the time this winds up getting published—but if not, congratulations on listening to this, you get a sneak preview—where I was looking at the intelligent tiering changes in pricing, where any object below 128 kilobytes does not have a monitoring charge attached to it, and above it, it does. And it occurred to me on a baseline gut level that, well wait a minute, it feels like there is some object sizes, where regardless of how long it lives in storage and transition to something cheaper, it will never quite offset that fee. So, instead of having intelligent tiering for everything, that there's some cut-off point below which you should not enable intelligent tiering because it will always cost you more than it can possibly save you.And I mentioned that to you and I had to do a lot of articulating with my hands because it's all gut feelings stuff and this stuff is complicated at the best of times. And your response was, “Huh.” Then it felt like ten minutes later you came back with a multi-page blog post written—again—in a Python notebook that has a dynamic interactive graph that shows the breakeven and cut-off points, a deep dive math showing exactly where in certain scenarios it is. And I believe the final takeaway was somewhere between 148 to 161 kilobytes, somewhere in that range is where you want to draw the cut-off. And I'm just looking at this and marveling, on some level.Alex: Oh, thanks. To be fair, it took a little bit more than ten minutes. I think it was something where it kind of went through a couple of stages where at first I was like, “Well, I bet I could model that.” And then I'm like, “Well, wait a minute. There's actually, like—if you can kind of put the compute side of this all the way to the side and just remove all API calls, it's a closed form thing. Like, you can just—this is math. I can just describe this with math.”And cue the, like, Beautiful Mind montage where I'm, like, going onto the whiteboard and writing a bunch of stuff down trying to remember the point intercept form of a line from my high school algebra days. And at the end, we had that blog post. And the reason why I kind of dove into that headfirst was just this, I have this fascination for understanding how all this stuff fits together, right? I think so often, what you see is a bunch of little point things, and somebody says, “You should use this at this point, for this reason.” And there's not a lot in the way of synthesis, relatively speaking, right?Like, nobody's telling you what the kind of underlying thing is that makes it so that this thing is better in these circumstances than this other thing is. And without that, it's a bunch of, kind of, anecdotes and a bunch of kind of finger-in-the-air guesses. And there's a part of that, that just makes me sad, fundamentally, I guess, that humans built all of this stuff; we should know how all of it fits together. And—Corey: You would think, wouldn't you?Alex: Well, but the thing is, it's so enormously complicated and it's been developed over such an enormously long period of time, that—or at least, you know, relatively speaking—it's really, really hard to kind of get that and extract it out. But I think when you do, it's very satisfying when you can actually say like, “Oh no, no, we've actually done—we've done the analysis here. Like, this is exactly what you ought to be doing.” And being able to give that clear answer and backing it up with something substantial is, I think, really valuable from the customer's point of view, right, because they don't have to rely on us kind of just doing the finger-in-the-air guess. But also, like, it's valuable overall. It extends the kind of domain where you don't have to think about whether or not you've got the right answer there. Or at least you don't have to think about it as much.Corey: My philosophy has always been that when I have those hunches, they're useful, and it's an indication that there's something to look into here. Where I think it goes completely off the rails is when people, like, “Well, I have a hunch and I have this belief, and I'm not going to evaluate whether or not that belief is still one that is reasonable to hold, or there has been perhaps some new information that it would behoove me to figure out. Nope, I've just decided that I know—I have a hunch now and that's enough and I've done learning.” That is where people get into trouble.And I see aspects of it all the time when talking to clients, for example. People who believe things about their bill that at one point were absolutely true, but now no longer are. And that's one of those things that, to be clear, I see myself doing this. This is not something—Alex: Oh, everybody does, yeah.Corey: —I'm blaming other people for it all. Every once in a while I have to go on a deep dive into our own AWS bill just to reacquaint myself with an understanding of what's going on over there.Alex: Right.Corey: And I will say that one thing that I was firmly convinced was going to happen during your tenure here was that you're a data person; hiring someone like you is the absolute most expensive thing you can ever do with respect to your AWS bill because hey, you're into the data space. During your tenure here, you cut the bill in half. And that surprises me significantly. I want to further be clear that did not get replaced by, “Oh, yeah. How do you cut your AWS bill by so much?” “We moved everything to Snowflake.” No, we did not wind up—Alex: [laugh].Corey: Just moving the data somewhere else. It's like, at some level, “Great. How do I cut the AWS bill by a hundred percent? We migrate it to GCP.” Technically correct; not what the customer is asking for.Alex: Right? Exactly, exactly. I think part of that, too—and this is something that happens in the data part of the space more than anywhere else—it's easy to succumb to shiny object syndrome, right? “Oh, we need a cloud data warehouse because cloud data warehouse, you know? Snowflake, most expensive IPO in the history of time. We got to get on that train.”And, you know, I think one of the things that I know you and I talked about was, you know, where should all this data that we're amassing go? And what should we be optimizing for? And I think one of the things that, you know, the kind of conclusions that we came to there was, well, we're doing some stuff here, that's kind of designed to accelerate queries that don't really need to be accelerated all that much, right? The difference between a query taking 500 milliseconds and 15 seconds, from our point of view, doesn't really matter all that much, right? And that realization alone, kind of collapsed a lot of technical complexity, and that, I will say we at Duckbill Group still espouse, right, is that cloud cost is an architectural problem, it's not a right-sizing your instances problem. And once we kind of got past that architectural problem, then the cost just sort of cratered. And honestly, that was a great feeling, to see the estimate in the billing console go down 47% from last month, and it's like, “Ah, still got it.” [laugh].Corey: It's neat to watch that happen, first off—Alex: For sure.Corey: But it also happened as well, with increasing amounts of utility. There was a new AWS billing page that came out, and I'm sure it meets someone's needs somewhere, somehow, but the things that I always wanted to look at when I want someone to pull up their last month's bill is great, hit the print button—on the old page—and it spits out an exploded pdf of every type of usage across their entire AWS estate. And I can skim through that thing and figure out what the hell's going on at a high level. And this new thing did not let me do that. And that's a concern, not just for the consulting story because with our clients, we have better access than printing a PDF and reading it by hand, but even talking to randos on the internet who were freaking out about an AWS bill, they shouldn't have to trust me enough to give me access into their account. They should be able to get a PDF and send it to me.Well, I was talking with you about this, and again, in what felt like ten minutes, you wound up with a command line tool, run it on an exported CSV of a monthly bill and it spits it out as an HTML page that automatically collapses in and allocates things based upon different groups and service type and usage. And congratulations, you spent ten minutes to create a better billing experience than AWS did. Which feels like it was probably, in fairness to AWS, about seven-and-a-half minutes more time than they spent on it.Alex: Well, I mean, I think that comes back to what we were saying about, you know, not all the interesting problems in data are in data that doesn't fit in RAM, right? I think, in this case, that came from two places. I looked at those PDFs for a number of clients, and there were a few things that just made my brain hurt. And you and Mike and the rest of the folks at Duckbill could stare at the PDF, like, reading the matrix because you've seen so many of them before and go, ah, yes, “Bill spikes here, here, here.” I'm looking at this and it's just a giant grid of numbers.And what I wanted was I wanted to be able to say, like, don't show me the services in alphabetical order; show me the service is organized in descending order by spend. And within that, don't show me the operations in alphabetical order; show me the operations in decreasing order by spend. And while you're at it, group them into a usage type group so that I know what usage type group is the biggest hitter, right? The second reason, frankly, was I had just learned that DuckDB was a thing that existed, and—Corey: Based on the name alone, I was interested.Alex: Oh, it was an incredible stroke of luck that it was named that. And I went, “This thing lets me run SQL queries against CSV files. I bet I can write something really fast that does this without having to bash my head against the syntactic wall that is Pandas.” And at the end of the day, we had something that I was pretty pleased with. But it's one of those examples of, like, again, just orienting the problem toward, “Well, this is awful.”Because I remember when we first heard about the new billing experience, you kind of had pinged me and went, “We might need something to fix this because this is a problem.” And I went, “Oh, yeah, I can build that.” Which is kind of how a lot of what I've done over the last 15 years has been. It's like, “Oh. Yeah, I bet I could build that.” So, that's kind of how that went.Corey: This episode is sponsored in part by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on-premises, private cloud, and they just announced a fully-managed service on AWS and Azure called BigAnimal, all one word. Don't leave managing your database to your cloud vendor because they're too busy launching another half-dozen managed databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications—including Oracle—to the cloud. To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: The problem that I keep seeing with all this stuff is I think of it in terms of having to work with the tools I'm given. And yeah, I can spin up infrastructure super easily, but the idea of, I'm going to build something that manipulates data and recombines it in a bunch of different ways, that's not something that I have a lot of experience with, so it's not my instinctive, “Oh, I bet there's an easier way to spit this thing out.” And you think in that mode. You effectively wind up automatically just doing those things, almost casually. Which does make a fair bit of sense, when you understand the context behind it, but for those of us who don't live in that space, it's magic.Alex: I've worked in infrastructure in one form or another my entire career, data infrastructure mostly. And one of the things—I heard this from someone and I can't remember who it was, but they said, “When infrastructure works, it's invisible.” When you walk in the room and flip the light switch, the lights come on. And the fact that the lights come on is a minor miracle. I mean, the electrical grid is one of the most sophisticated, globally-distributed engineering systems ever devised, but we don't think about it that way, right?And the flip side of that, unfortunately, is that people really pay attention to infrastructure most when it breaks. But they are two edges of the same proverbial sword. It's like, I know, when I've done a good job, if the thing got built and it stayed built and it silently runs in the background and people forget it exists. That's how I know that I've done a good job. And that's what I aim to do really, everywhere, including with Duckbill Group, and I'm hoping that the stuff that I built hasn't caught on fire quite yet.Corey: The smoke is just the arising of the piles of money it wound up spinning up.Alex: [laugh].Corey: It's like, “Oh yeah, turns out that maybe we shouldn't have built a database out of pure Managed NAT Gateways. Yeah, who knew?”Alex: Right, right. Maybe I shouldn't have filled my S3 bucket with pure unobtainium. That was a bad idea.Corey: One other thing that we do here that I admit I don't talk about very often because people get the wrong idea, but we do analyst projects for vendors from time to time. And the reason I don't say that is, when people hear about analysts, they think about something radically different, and I do not self-identify as an analyst. It's, “Oh, I'm not an analyst.” “Really? Because we have analyst budget.” “Oh, you said analyst. I thought you said something completely different. Yes, insert coin to continue.”And that was fine, but unlike the vast majority of analysts out there, we don't form our opinions based upon talking to clients and doing deeper dive explorations as our primary focus. We're a team of engineers. All right, you have a product. Let's instrument something with it, or use your product for something and we'll see how it goes along the way. And that is something that's hard for folks to contextualize.What was really fun was bringing you into a few of those engagements just because it was interesting; at the start of those calls. “It was all great, Corey is here and—oh, someone else's here. Is this a security problem?” “It's no, no, Alex is with me.” And you start off those calls doing what everyone should do on those calls is, “How can we help?” And then we shut up and listen. Step one, be a good consultant.And then you ask some probing questions and it goes a little bit deeper and a little bit deeper, and by the end of that call, it's like, “Wow, Alex is amazing. I don't know what that Corey clown is doing here, but yeah, having Alex was amazing.” And every single time, it was phenomenal to watch as you, more or less, got right to the heart of their generally data-oriented problems. It was really fun to be able to think about what customers are trying to achieve through the lens that you see the world through.Alex: Well, that's very flattering, first of all. Thank you. I had a lot of fun on those engagements, honestly because it's really interesting to talk to folks who are building these systems that are targeting mass audiences of very deep-pocketed organizations, right? Because a lot of those organizations, the companies doing the building are themselves massive. And they can talk to their customers, but it's not quite the same as it would be if you or I were talking to the customers because, you know, you don't want to tell someone that their baby is ugly.And note, now, to be fair, we under no circumstances were telling people that their baby was ugly, but I think that the thing that is really fun for me is to kind of be able to wear the academic database nerd hat and the practitioner hat simultaneously, and say, like, “I see why you think this thing is really impressive because of this whiz-bang, technical thing that it does, but I don't know that your customers actually care about that. But what they do care about is this other thing that you've done as an ancillary side effect that actually turns out is a much more compelling thing for someone who has to deal with this stuff every day. So like, you should probably be focusing attention on that.” And the thing that I think was really gratifying was when you know that you're meeting someone on their level and you're giving them honest feedback and you're not just telling them, you know, “The Gartner Magic Quadrant says that in order to move up and to the right, you must do the following five features.” But instead saying, like, “I've built these things before, I've deployed them before, I've managed them before. Here's what sucks that you're solving.” And seeing the kind of gears turn in their head is a very gratifying thing for me.Corey: My favorite part of consulting—and I consider analyst style engagements to be a form of consulting as well—is watching someone get it, watching that light go on, and they suddenly see the answer to a problem that's been vexing them I love that.Alex: Absolutely. I mean, especially when you can tell that this is a thing that has been keeping them up at night and you can say, “Okay. I see your problem. I think I understand it. I think I might know how to help you solve it. Let's go solve it together. I think I have a way out.”And you know, that relief, the sense of like, “Oh, thank God somebody knows what they're doing and can help me with this, and I don't have to think about this anymore.” That's the most gratifying part of the job, in my opinion.Corey: For me, it has always been twofold. One, you've got people figuring out how to solve their problem and you've made their situation better for it. But selfishly, the thing I like the most personally has been the thrill you get from solving a puzzle that you've been toying with and finally it clicks. That is the endorphin hit that keeps me going.Alex: Absolutely.Corey: And I didn't expect when I started this place is that every client engagement is different enough that it isn't boring. It's not the same thing 15 times. Which it would be if it were, “Hi, thanks for having us. You haven't bought some RIs. You should buy some RIs. And I'm off.” It… yeah, software can do that. That's not interesting.Alex: Right. Right. But I think that's the other thing about both cloud economics and data engineering, they kind of both fit into that same mold. You know, what is it? “All happy families are alike, but each unhappy family is unhappy in its own way.” I'm butchering Chekhov, I'm sure. But like—if it's even Chekhov.But the general kind of shape of it is this: everybody's infrastructure is different. Everybody's organization is different. Everybody's optimizing for a different point in the space. And being able to come in and say, “I know that you could just buy a thing that tells you to buy some RIs, but it's not going to know who you are; it's not going to know what your business is; it's not going to know what your challenges are; it's not going to know what your roadmap is. Tell me all those things and then I'll tell you what you shouldn't pay attention to and what you should.”And that's incredibly, incredibly valuable. It's why, you know, it's why they pay us. And that's something that you can never really automate away. I mean, you hear this in data all the time, right? “Oh, well, once all the infrastructure is managed, then we won't need data infrastructure people anymore.”Well, it turns out all the infrastructure is managed now, and we need them more than we ever did. And it's not because this managed stuff is harder to run; it's that the capabilities have increased to the point that they're getting used more. And the more that they're getting used, the more complicated that use becomes, and the more you need somebody who can think at the level of what does the business need, but also, what the heck is this thing doing when I hit the run key? You know? And that I think, is something, particularly in AWS where I mean, my God, the amount and variety and complexity of stuff that can be deployed in service of an organization's use case is—it can't be contained in a single brain.And being able to make sense of that, being able to untangle that and figure out, as you say, the kind of the aha moment, the, “Oh, we can take all of this and just reduce it down to nothing,” is hugely, hugely gratifying and valuable to the customer, I'd like to think.Corey: I think you're right. And again, having been doing this in varying capacities for over five years—almost six now; my God—the one thing has been constant throughout all of that is, our number one source for new business has always been word of mouth. And there have been things that obviously contribute to that, and there are other vectors we have as well, but by and large, when someone winds up asking a colleague or a friend or an acquaintance about the problem of their AWS bill, and the response almost universally, is, “Yeah, you should go talk to The Duckbill Group,” that says something that validates that we aren't going too far wrong with what we're approaching. Now that you're back on the freelance data side, I'm looking forward to continuing to work with you, if through no other means and being your customer, just because you solve very interesting and occasionally very specific problems that we periodically see. There's no reason that we can't bring specialists in—and we do from time to time—to look at very specific aspects of a customer problem or a customer constraint, or, in your case for example, a customer data set, which, “Hmm, I have some thoughts on here, but just optimizing what storage class that three petabytes of data lives within seems like it's maybe step two, after figuring what the heck is in it.” Baseline stuff. You know, the place that you live in that I hand-wave over because I'm scared of the complexity.Alex: I am very much looking forward to continuing to work with you on this. There's a whole bunch of really, really exciting opportunities there. And in terms of word of mouth, right, same here. Most of my inbound clientele came to me through word of mouth, especially in the first couple years. And I feel like that's how you know that you're doing it right.If someone hires you, that's one thing, and if someone refers you, to their friends, that's validation that they feel comfortable enough with you and with the work that you can do that they're not going to—you know, they're not going to pass their friends off to someone who's a chump, right? And that makes me feel good. Every time I go, “Oh, I heard from such and such that you're good at this. You want to help me with this?” Like, “Yes, absolutely.”Corey: I've really appreciated the opportunity to work with you and I'm super glad I got the chance to get to know you, including as a person, not just as the person who knows the data, but there's a human being there, too, believe it or not.Alex: Weird. [laugh].Corey: And that's the important part. If people want to learn more about what you're up to, how you think about these things, potentially have you looked at a gnarly data problem they've got, where's the best place to find you now?Alex: So, my business is called Bits on Disk. The website is bitsondisk.com. I do write occasionally there. I'm also on Twitter at @alexras. That's Alex-R-A-S, and I'm on LinkedIn as well. So, if your lovely listeners would like to reach me through any of those means, please don't hesitate to reach out. I would love to talk to them more about the challenges that they're facing in data and how I might be able to help them solve them.Corey: Wonderful. And we will of course, put links to that in the show notes. Thank you again for taking the time to speak with me, spending as much time working here as you did, and honestly, for a lot of the things that you've taught me along the way.Alex: My absolute pleasure. Thank you very much for having me.Corey: Alex Rasmussen, data engineering consultant at Bits on Disk. I'm Cloud Economist Corey Quinn. This is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment that is so large it no longer fits in RAM.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About SterenSteren is a Group Product Manager at Google Cloud. He is part of the serverless team, leading Cloud Run. He is also working on sustainability, leading the Google Cloud Carbon Footprint product.Steren is an engineer from École Centrale (France). Before joining Google, he was CTO of a startup building connected objects and multi device solutions.Links Referenced: previous episode: https://www.lastweekinaws.com/podcast/screaming-in-the-cloud/google-cloud-run-satisfaction-and-scalability-with-steren-giannini/ Google Cloud Region Picker: https://cloud.withgoogle.com/region-picker/ Google Cloud regions: https://cloud.google.com/sustainability/region-carbon TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: DoorDash had a problem. As their cloud-native environment scaled and developers delivered new features, their monitoring system kept breaking down. In an organization where data is used to make better decisions about technology and about the business, losing observability means the entire company loses their competitive edge. With Chronosphere, DoorDash is no longer losing visibility into their applications suite. The key? Chronosphere is an open-source compatible, scalable, and reliable observability solution that gives the observability lead at DoorDash business, confidence, and peace of mind. Read the full success story at snark.cloud/chronosphere. That's snark.cloud slash C-H-R-O-N-O-S-P-H-E-R-E.Corey: This episode is sponsored in part by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on-premises, private cloud, and they just announced a fully-managed service on AWS and Azure called BigAnimal, all one word. Don't leave managing your database to your cloud vendor because they're too busy launching another half-dozen managed databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications—including Oracle—to the cloud. To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. My guest today was recently on the show. Steren Giannini is the product lead for Google Cloud Run, and we talked about that in a previous episode. If you haven't listened to it, you might wish to go back and listen to it, but it's not a prerequisite for what we're about to talk about today. Because apparently Google still does it's 20% time, and one of the things that Steren decided to do—because, you know, everyone needs a hobby—you decided to go ahead and start the Google Cloud Carbon Footprint, which is—well, Steren, thanks for coming back. What the hell is that?Steren: Thanks for having me back on the show. So yes, we started with Cloud Carbon Footprint, and this is a product that now has launched publicly, available to every Google Cloud customer right out of the box of the Google Cloud Console.Corey: I should also point out, because people always wonder and it's the first thing I always check, yes, this one is free. I'm trying to imagine a scenario which you charge for this and I wasn't incensed by it, and I can't. So, good work, you aren't charging anything for it. Good job. Please continue.Steren: So, Google Cloud Carbon Footprint helps a Google Cloud customer understand and reduce their gross carbon emissions linked to their Google cloud usage. So yeah, what do we mean by carbon emission? Just so that we are all on the same page, these are the greenhouse gases that are emitted due to the activity of using Google Cloud that are notably responsible for climate change. And we report them in equivalent of carbon dioxide—CO2—and you know, the shortcut is just to say ‘carbon.' Corey: Now, I'm going to start with something relatively controversial. It's an opinion I have around this sort of thing. And I should also disclaim, I am not in any way, shape, or form, disputing the climate change as caused by humans is real. It is. If you don't believe that, please go listen to something else, maybe Infowars. I don't know and I don't care. I just don't want you around.Now, the problem that I have with this is, on some level, it feels like a cloud provider talking to its customers about their carbon footprint is, on some level, shifting the onus of responsibilities in some way away from the cloud provider and onto the customer. Now, I freely admit that this is a nuanced topic, but how do you view this?Steren: What I mentioned is that we are exposing to customer their gross carbon emissions, but what about their net carbon emissions? Well, Google Cloud customers, net operational carbon emissions are simply zero. Why? Because if you open Google's environmental report, you will see that Google is purchasing as much renewable energy globally for the year as it is using. So, that means that on a yearly basis worldwide, every kilowatt hour of electricity has been matched with renewable energy.And you know, this Google has been doing since 2017. Since 2007, Google was already matching its carbon footprint with carbon offsets. But 2017, Google went beyond and is matching the purchase of the electricity with renewable energy. So, in a sense, your net operational emissions are zero.Now, that's not sufficient for our customers. They have some reporting obligations; they need to know before this renewable matching, what were their gross emissions? And they also need to know what are their emissions coming from, not only the electricity usage, but maybe the data center or manufacturing. And this is all of what we expose in Google Cloud Carbon Footprint. They are before offset, before renewable energy matching.And you're right also to say that this is not only the customer's problem, and indeed, Google itself has set a goal to get to a hundred percent carbon-free electricity for every hour in every location. The big goal for 2030 is that at every hour, every location, the electricity comes from carbon-free sources. This is very ambitious and never done before, of course, at the scale of Google, but this is the next goal for Google.Corey: The challenge that I have—in the abstract—with cloud providers, more or less, shaming customers—not to say that's what you're doing here—about their carbon usage and their carbon footprint is, okay, I appreciate that this is everyone's problem, and yes, it is something that we should be focusing on extensively. The counterargument is that I don't recall ever getting a meeting invite to a Google or Amazon or Microsoft or Oracle negotiation with any of your power bills or power companies or power sourcing. I have no input whatsoever as a customer on those things. And, on some level, it's “Ooh, you're causing a particular amount of carbon to be used by your usage of these services.” Like, well, at some level, it feels like that is more of a you thing than a me thing.And I want to be clear, I'm speaking more in the abstract to the industry rather than the specifics of Google Cloud, not to unfairly put you in the position of having to speak for everyone.Steren: No, but you're right. If you were to do nothing, Google is constantly working hard to sign more power purchase agreements with some renewable energy sources or optimizing its data centers. Google Cloud data centers are one of the most optimized data centers in the industry with a power usage effectiveness of 1.1, which is basically saying that the energy that is used to power the facility over the energy used to actually power the server is 1.1. So, not that much loss in between.So, all of that to say, Google Cloud and Google are working very hard anyway to reduce Google Cloud's carbon footprints and the carbon footprint of Google Cloud customers. So, if you were to do nothing, the charts that you're seeing on Google Cloud Carbon Footprint should trend to zero. But in the meantime, you know, that's not the case, so that's why we show the data. And, like, many customers want to know or have the obligation to report on this data.Corey: One of the challenges that I see—and I believe this might even be related to the carbon footprint tool you have built out on top of Google Cloud—is when I am looking at… at where to place something—first, let me just say the region experience is wildly different than I'm used to with AWS. In the AWS universe, every region is basically its own island; it is very difficult to get a holistic view between regions. Google Cloud does not have that approach. There are advantages and disadvantages to both. I'm not passing any particular value judgment—for once—on this topic in this context. But where do I want to spin something up? And I have a dropdown of the regions that I can put it in. And some of these now have a green leaf next to them and others do not. I'm going to go out on a limb and assume you had a hand in this somewhere.Steren: Exactly. That's something I worked on with the team. So, you will see on the Google Cloud Console location selectors on the Google Cloud location page, on the Google Cloud documentation, you will see a small low CO2 indicator next to some regions. And this indicator is basically saying that this region meets some criteria of high carbon-free energy percentage or low grid carbon intensity. So, you don't need to go into the details; you just need to know that if you see this small leaf, that means that for a given workload, the emissions in that particular region will be way lower than on another region which doesn't have the leaf.Often at Google, when we do a change we A/B test it. We A/B tested those small low CO2 indicators because, you know, that's a console-wide change so we want to make sure that it's worth is. And well, it turns out that for people who were in the experiment—so people will be seeing the leaf—among new Google Cloud users, they were 50% more likely to pick a low-carbon region when the leaf was displayed. And among all users, it was 19%. So, you see how just by surfacing the information, we were able to significantly influence customers' behavior towards reducing their carbon emissions.And, you know, if you ask me, I think picking the cleanest region is probably one of the simplest action you can take—if possible, of course—to reduce your gross carbon emissions because, you know, they don't require to change your architecture or your infrastructure; it just requires you to make the right choice in the first place. And just by letting people know that some regions are emitting much less carbon than others we basically allow them to reduce their footprint.Corey: A question I have is that as you continue to move up the stack, one of the things that Google has done extraordinarily well is the global network. And we talked previously about how I run the snark.cloud URL shortener in Google Cloud. That is homed out of us-central1 as far as regions go. But given that thing is effectively stateless, it just talks to Google Sheets for its source of truth, but then just runs a Docker invocation on every request, cool, I can see a scenario in which that becomes much more of a global service.In other words, if you can run that in pops in every region around the world on some level, there is no downside, from my perspective, on doing that. What I'm wondering then, as a result of that, is as you start seeing the evolution of services becoming more and more global, instead of highly region-specific, does that change the way that we should be thinking potentially about carbon footprint and regional selection? Or is that too much of a niche edge case to really be top of radar right now?Steren: Oh, there are many things to talk about here. The first one is that you might be hinting at something that Google is already doing, which is location shifting of workloads in order to optimize power usage, and, you know of course, carbon emissions. So, Google itself is already doing that. For example, I guess, to process YouTube videos, that can be done, not necessarily right away and that can be done in the location in which, for example, the sun is shining. So, there are some very interesting things that can be done if you allow the workloads to be run in not necessarily a specific region.Now, that being said, I think there are many other things that people consider when they pick a region. First, well, maybe they have some data locality constraints, right? This is very much the case in European countries where the data must stay in a given region, by law. Second, well, maybe they care about the price. And as you probably know, [laugh] the price of cloud providers is not the same in every region.Corey: I've noticed that and in fact, I was going to get into that as our next transition, but you've just opened Pandora's Box early. It's great to have the carbon-friendly indicator next to the region, but I also want number of dollar signs next to it as well. Like in AWS-land, do you have the tier one regions where everything is the lowest price: us-east-1, us-west-2, and a few others escaped me from time to time, where Managed NAT Gateways are really expensive. And then you go under some others and they get even more expensive, somehow. Like, talk about pushing the bounds of cloud economics. It's astonishing to me.Steren: Yes. And so—Corey: Because I want that display, on some level—Steren: Exactly.Corey: —as a customer, in many cases.Steren: So, there is price, there is carbon, but of course, you know, if you are serving web requests, there is probably also latency that you care about, right? Even if—for example, Finland is very low carbon. You might not host your workloads in Finland if you want to serve US customers. So, in a sense, there are many dimensions to optimize when you pick a region. And I just sent you a link to something that I built, which is called Google Cloud Region Picker.It's basically a tool with three sliders. First one is carbon footprint; you tell us how much you care about that. Hopefully, you put it to the right. The second one is lower price. So, how much do you want the tool to optimize to lower your bill? And third one is latency, and then you tell us where your users are coming from and if you care about latency.Because some workloads are not subject to latency requirements. Like, if you do batch jobs, well, that doesn't serve a user request, so that can be done asynchronously at a later time or in a different place. And what this tool does is that it takes your inputs and it basically tells you which Google Cloud region is the best fit for you. And if you use it, you will see it has very small symbols like three dollars for the most expensive regions, one dollar for the least expensive ones, three leaves for the greenest regions, and zero leaves for the non-green one.Corey: This is awesome. I'm a little bit disappointed that I hadn't seen this before. This is a thing of beauty.Steren: Yeah. Again, done by me as a 20%. [laugh]. And, you know, the goal is to educate, like, of course, it's way more complex. Like, you know that price optimization is way more complex than a slider, but the goal of this tool is to educate and to give a first result. Like, okay, if you are in France and care about carbon, then go here. If you are in Oregon, go here. And so, many parameters that this tool help you optimize in a very simple way.Corey: One of the challenges I think I get into when I look at this across the board, is that you have a couple of very different ends on a confusing spectrum, by which I mean that one of the things I would care about from a region picker, for example, is there sufficient capacity in that region for the things I want to run. At my scale of things where right now on Google Cloud I run a persistent VM that hangs out all the time, and I run some Google Cloud Run stuff. Great. If you have capacity problems with either one of those, are you really a cloud?But then we have other folks who are spinning up tens or hundreds of thousands of a very particular instance type in a very specific region. That's the sort of thing that requires a bit more in the way of capacity planning and the rest. So, I have to imagine for those types of use cases, this tool is insufficient. The obvious reason, of course, if you're spinning up that much of anything, for God's sake, reach out and talk to your account manager before trying to do it willy-nilly but yes.Steren: That's exactly right. So, as I said, this tool is simplified tool to give, like, the vast majority of users a sense of where to put their workloads. But of course, if you're a very big enterprise customer who is going to sign a very big deal with Google Cloud, talk to your account manager because if you do need a lot of capacity, Google Cloud might need to plan for it. And not every regions have the same capacity and we are always working with our customers to make sure we direct them in the right place and have enough capacity. A real-life example of a very high profile Google Cloud customer was that they were selecting a region without knowing its carbon impact, and when we started to disclose the carbon characteristics of Google Cloud regions—which is another link we can send to the audience—this customer realized that the region they selected—you know, maybe because it was close to their user base—was really not the most carbon friendly.So, they decided to switch to another one. And if we take an example, if you take Las Vegas, it has a carbon-free energy percentage of 20%. So, that basically means that on average, 20% of the time, the electricity comes from carbon-free sources. If you are to move to Oregon, this same workload, Oregon has a carbon-free energy percentage of 90%. So, you can see how just by staying on the West Coast, moving from Las Vegas to Oregon, you have drastically reduced your carbon emissions. And your bill, by the way because it turns out Oregon is one of the cheapest Google Cloud Data Center. So, you see how just being aware of those numbers led some very important customers who care about sustainability to make some fundamental choices when it comes to the regions they select.Corey: I guess that leads to my big obvious question, where I wind up pulling up my own footprint in Google Cloud—again, I don't run much there—and apparently over the last year, I've had something on the order of two kilograms of carbon. Great. It feels like for this scale, I almost certainly burn more carbon than that just searching Google for carbon-related questions about where to place things. So, for my use case, this is entirely academic. You can almost run my workloads based upon, I don't know, burning baby seals or something, and the ecological footprint does not materially change.Then we go to the other extreme end of the spectrum with the hundreds of thousands of instances, where this stuff absolutely makes a significant and massive difference. My question is, when should people begin thinking about the carbon footprint of their cloud workload at what point of scale?Steren: So, as you said, a good order of magnitude is one transatlantic flight is a thousand kilogram of equivalent CO2. So, you see how just by flying once, you're already, like, completely overshadowing your Google Cloud carbon footprint. But that's because you are not using a lot of Google Cloud resources. Overall, you know, I think your question is basically the same as when should individuals try to optimize reducing their carbon footprint? And here I always recommend there are tons of things you can optimize.Start by the most impactful ones. And impactful means an action will have a lot of impact in reducing the footprint, but also the footprint reduction will be significant by itself. And two kilograms of CO2, yes indeed, it is very low, but if you start reaching out into the thousands of kilograms of CO2 that starts to represent, like, one flight, for example. So, you should definitely care about it. And as I said, some actions might be rather easy, like picking the right region might be something you can do pretty easily for your business and then you will see your carbon emissions being divided by, you know, sometimes five.This episode is sponsored in part by our friends at Lambda Cloud. They offer GPU instances with pricing that's not only scads better than other cloud providers, but is also accessible and transparent. Also, check this out, they get a lot more granular in terms of what's available. AWS offers NVIDIA A100 GPUs on instances that only come in one size and cost $32/hour. Lambda offers instances that offer those GPUs as single card instances for $1.10/hour. That's 73% less per GPU. That doesn't require any long term commitments or predicting what your usage is gonna look like years down the road. So if you need GPUs, check out Lambda. In beta, they're offering 10TB of free storage and, this is key, data ingress and egress are both free. Check them out at lambdalabs.com/cloud. That's l-a-m-b-d-a-l-a-b-s.com/cloud.Corey: I want to challenge your assertion, incidentally. You say that I'm not using a whole lot of Google Cloud resources. I disagree. I use roughly a dozen different Google Cloud resources tied together for some of these things, but they're built on serverless design patterns, which means that they scale to nothing. I'm not sitting there with an idle VM—except that one—that is existing on a persistent basis.For example, I look at the things that show up on the top five list. Compute Engine is number one, Cloud Run, Cloud Logging, Cloud Storage, and App Engine are the rest that are currently being used. I think there's a significant untold story around the idea of building in a serverless way for climate purposes.Steren: Yes. So, maybe for those who are not aware of what you are seeing on the dashboard, so when you open this Google Cloud Carbon Footprint tool on the Cloud Console, you saw a breakdown of your yearly carbon footprint and monthly carbon footprint across a few dimensions. The first one is the regions because as we said, this matters a lot; like, the regions have a lot of impact. The second one are the month; of course, you can see over time, how you're trending. The third one is a concept called Google Cloud Project, which is, for those who are not aware, it's a way to group Google Cloud resources into buckets.And the third one is Google Cloud services. So, what you described here is, which of your services emits the most and therefore which ones should you optimize first? Like, again, to go back to impactful actions. And to your point, yes, it is very interesting that if you use products which auto-scale, basically, the carbon attributed to you, the customer, will really follow this auto-scaling behavior. Compare that to a virtual machine that is always on, burning some CPU for almost nothing because you have a server that doesn't process requests. That is wasting, in a sense, resources.So, what you describe here is very interesting, which is basically the most optimized products you're going to pick, the less waste you're going to have. Now, I also want to be careful because comparing one CPU hour of Cloud Run and one CPU hour of Compute Engine is not comparing apples to apples. Why? Because when you use Cloud Run, I'm not sure if you know, but you are using a regional product. So, a product which has built-in redundancy, which is safe in case of one zone going down in a region.But that means the Cloud Run infrastructure has to provision a little bit more machines than if it was a zonal product. While Compute Engine, your virtual machine lives in one zone and there is only one machine for you. So, you see how we should also be careful comparing products with other products because fundamentally, they are not offering the same value and they are not running on the same infrastructure. But overall, I think you are correct to say that, you know, avoiding waste, using auto-scaling products, is a good way to reduce your footprint.Corey: I do want to ask—and this is always a delicate topic because you're talking about cultural things—how much headwind did you have internally at Google when you had the idea to start exposing this? How difficult was it to bring this to fruition?Steren: I think we are lucky that our leadership cares about reducing carbon emissions and understood that our customers needed our help to understand their cloud emissions. Like, many customers before we had this tool, we're trying to some kind of estimate their cloud emissions. And it was—you know, Google Cloud was a black box for them. They did not have access to what you said, to some data that only Google has access to.And you know, to build that tool, we are using energy measurement of every machine in every data center. We are using, you know, customer-wide resource usage. And that is something that we use to divide the footprint among customers. So, there is some data used to compute those numbers that only Google Cloud has access to. And indeed, you're correct; it required some executive approval which we received because many of our leaders believe that, you know, this is the right thing to do, and this is helping customers towards the same goal as Google, which is being net-zero and carbon-free.Many of our customers have made some sustainability commitments, and they need our help to meet those goals. So yeah, we did receive approval, first to share the per-region characteristics. This was already, you know, a first in the industry where a cloud provider disclosed that not every region is equal and some are emitting more carbon than others. And second, another approval which was to disclose a per customer carbon footprint, which is broken down by service, project, region, using some, you know, if you touch a little bit on the methodology, you know, it uses energy consumption, resource usage, and carbon intensity coming from a partner of ours to compute, basically, a per customer footprint.Corey: My question for you is, on some level, given that Google is already committed to being net-zero across the board for all of its usage, why do customers care? Why should they care? Effectively, haven't you made that entirely something that is outside of their purview? You've solved the problem, either way.Steren: This is where we should explore it a bit more the kinds of carbon emissions that exist. For a customer, their emissions linked to the cloud usage is all considered the indirect emissions. This, in the Greenhouse Gas Protocol Standard, this is called Scope 3. So, our Google Cloud emissions are the customers' Scope 3 emissions; they are all indirect for them. But those indirect emissions, what I mentioned as being net-zero are the emissions coming from electricity usage.So, to power those data centers, those data centers are located in certain electricity grids. Those electricity grids might be using energy sources that emit more or less carbon, right? Simply put, if in a given place, the electricity comes from coal, it will be emitting a lot of carbon compared to when electricity comes from solar, for example. So, you see how the location itself determines the carbon intensity. And these are the emissions coming from electricity usage, right?So, these are neutralized by Google purchasing as much renewable energy. But there are also types of emissions. For example, when a data center loses connection to the grid, we startup diesel generators. Those diesel generators will directly emit carbon. This is called Scope 1 emissions.And finally, there is the carbon emissions that are coming from the manufacturing of those servers, of those data centers. And this is called Scope 3 emissions. And the goal of Google is for the emissions coming from electricity to be always coming from carbon-free sources. So, this is a change that we've recently released to Google Cloud Carbon Footprint, which is now we also break down your emissions by scope. So, they are all Scope 3 for you, the customer, they are all indirect emissions for you, the customer, but now those indirect emissions, you can see how much is coming from diesel generators, how much is coming from electricity consumption, and how much is coming from manufacturing of the data center, and other, like, upstream, downstream activities. And yeah, overall, this is something that customers do need to report on.Corey: I think that's very fair. I do want to thank you for taking so much time to speak with me. And instead of the usual question I'd like to ask here of where can people go to find out more because we have a bunch of links for that, instead, I want to ask something a little bit different here, which is, what are the takeaways that customers or prospective customers should really have around their carbon footprint when it comes to cloud?Steren: So, I would recommend our audience to consider carbon emissions in your cloud infrastructure decisions. And my advice is, first, move to the cloud. Like, we've talked that Google Cloud has very well-optimized data centers. Like, your cloud gross carbon emissions are anyway going to be much lower than any on-premise carbon emissions. And by the way, if you use Google Cloud, your net operational emissions are zero.Second action is pick the region with the lowest carbon impact. Like we discussed that this is probably a low-effort action, if possible, that will have a lot of impact on your gross carbon emissions. And you know, if you want to go further, try to schedule those workloads when the electricity is the greenest, you know, when the sun is shining, the wind is blowing, for example, or try to schedule those workloads in regions which have the lowest impact. And yeah, Google Cloud gives you all the tools to do that, the tools to optimize your region selection, and the tools to report and reduce your gross carbon emissions. We haven't talked about it, but Google Cloud Carbon Footprint will even send you some proactive recommendations of things to do to reduce your emissions.For example, if you have a project, a machine that you forgotten, Google Cloud Carbon Footprint, will recommend you to delete it and we'll tell you how much carbon you would save by deleting it, as well as dollar, of course.Corey: It's funny because I feel like there's a definite alignment between my view of cloud economics and the carbon perspective on this, which is step one, everyone wins if you turn things off when you're not using them. What a concept. I sometimes try and take it too far of, ‘turn off all of production because your company's terrible.' Yeah, it turns out, that doesn't work super well. But the idea of step one, turn it off, especially when you're not using it. And if you're never using it, why would you want to pay for it? That becomes a very clear win for everyone involved. I think that in the fullness of time, economics are what are going to move the needle on driving further adoption about this. I have to guess that you see the same thing from where you are?Steren: Yes, very often working to reduce your carbon footprint is also working to reduce your bill. And we've also observed—not always—but some correlation between regions that have the lowest carbon impact and regions that are the cheapest. So, in a sense, this region selection, optimizing for price and carbon is often optimizing for the same thing. It's not always true, but it is often true.Corey: I really want to thank you for spending so much time to talk with me about this. This has definitely giving me a lot of food for thought, and I have to imagine that this will not be our last conversation around the topic.Steren: Well, thanks for having me. And I'm very happy to talk to you in the podcast, of course.Corey: Steren Giannini, product lead for Google Cloud Carbon Footprint and Google Cloud Run. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry screed about how climate change isn't real as you sit there wondering why it's 120 degrees in March.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About AlexAlex Su is a lawyer who's currently the Head of Community Development at Ironclad, the #1 contract lifecycle management technology company that's backed by Accel, Sequoia, Y Combinator, and other leading investors. Prior to joining Ironclad, Alex sold cloud software to legal departments and law firms on behalf of early stage startups. Alex maintains an active presence on social media, with over 180,000 followers across Twitter, LinkedIn, Instagram, and TikTok. Links Referenced: Ironclad: https://ironcladapp.com/ LinkedIn: https://www.linkedin.com/in/alexander-su/ Twitter: https://twitter.com/heyitsalexsu Instagram: https://www.instagram.com/heyitsalexsu/ TikTok: https://www.tiktok.com/@legaltechbro TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by Honeycomb. When production is running slow, it's hard to know where problems originate. Is it your application code, users, or the underlying systems? I've got five bucks on DNS, personally. Why scroll through endless dashboards while dealing with alert floods, going from tool to tool to tool that you employ, guessing at which puzzle pieces matter? Context switching and tool sprawl are slowly killing both your team and your business. You should care more about one of those than the other; which one is up to you. Drop the separate pillars and enter a world of getting one unified understanding of the one thing driving your business: production. With Honeycomb, you guess less and know more. Try it for free at honeycomb.io/screaminginthecloud. Observability: it's more than just hipster monitoring.Corey: I come bearing ill tidings. Developers are responsible for more than ever these days. Not just the code that they write, but also the containers and the cloud infrastructure that their apps run on. Because serverless means it's still somebody's problem. And a big part of that responsibility is app security from code to cloud. And that's where our friend Snyk comes in. Snyk is a frictionless security platform that meets developers where they are - Finding and fixing vulnerabilities right from the CLI, IDEs, Repos, and Pipelines. Snyk integrates seamlessly with AWS offerings like code pipeline, EKS, ECR, and more! As well as things you're actually likely to be using. Deploy on AWS, secure with Snyk. Learn more at Snyk.co/scream That's S-N-Y-K.co/screamCorey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I've been off the beaten path from the traditional people building things in cloud by the sweat of their brow and the snark on their Twitters. I'm joined today by Alex Su, who's the Head of Community Development at Ironclad, and also relatively well-renowned on the TikToks, as the kids say. Alex, thank you for joining me.Alex: Thank you so much for having me on the show.Corey: It's always been an interesting experience because I joined TikTok about six months or so ago, due to an escalatingly poor series of life choices that continue to fail me, and I have never felt older in my life. But your videos consistently tend to show up there. You are @legaltechbro, which sounds like wow, I hate all of those things, and yet your content is on fire.How long have you been doing the public dance thing, for lack of a better term? I don't even know what they call it. I know how to talk about Twitter. I know how to talk about LinkedIn—sad. LinkedIn is sad—but TikTok is still something I'm trying to wrap my ancient brain around.Alex: Yeah, I felt out of place when I first made my first TikTok. And by the way, I'm known for making funny skits. I have actually never danced. I've always wanted to, but I don't think I have that… that talent. I started posting TikToks in, I will call it—let's call it the fall of 2020. So, after the pandemic.Before that, I had been posting consistently on LinkedIn for, gosh, ever since 2016, when I got into legal tech. And during the pandemic, I tried a bunch of different things including making funny skits. I'd seen something somewhere online if somebody's making fun of the doctor life. And so, I thought, hey, I could do that for legal too. And so, I made one with iMovie. You know, I recorded it on Zoom.And then people started telling me, “Hey, you should get on this thing called TikTok.” And so, I resisted it for a while because I was like, “This is not for me.” But at some point, I said, “I'll try this out. The editing seems pretty easy.” So, I made a couple of videos poking fun at the life of a law firm lawyer or a lawyer working for a corporate legal department.And on my fourth video, I went massively viral. Like, unexpected went viral, like, millions of—I think two million or so views. And I found myself with a following. So, I thought, “Hey, I guess this is what I'm doing now.” And so, it's been, I don't know, a year-and-a-half since then, and I've been continuously posting these skits.Corey: It's like they say the worst thing can happen when you go into a casino and play for the first time is you win.Alex: [laugh].Corey: You get that dopamine hit, and suddenly, well now, guess what you're doing for the rest of your life? There you go. It sounds like it worked out for you in a lot of fun ways. Your skits about big law of life definitely track. My wife used to work in that space, and we didn't meet till she was leaving that job because who has time to date in those environments?But I distinctly remember one of our early dates, we went out to meet a bunch of her soon-to-be-former coworkers at something like eight or nine o'clock in Los Angeles on a Friday night. And at the end of it, we went back to one of our places, and they went back to work. Because that is the lifestyle, apparently, of being in big law. I don't have the baseline prerequisites to get into law school, to let alone get the JD and then go to work in big law, and looking at that lifestyle, it's, “Yeah, you know, I don't think that's for me.” Of course, I say that, and then three days later, I was doing a middle of the night wake up because the pager went off.Like, “Oh, are you a doctor?” And the pager is like, “Holy shit. This SSL certificate expires in 30 days.” It's, yeah. Again, life has been fun, but it's always been one of those things that was sort of, I guess, held in awe. And you're putting a very human face on it.Alex: Yeah. You know, I never expected to be in big law either, Corey. Like, I was never good at school, but as I got older, I found a way to talk my way into, like, a good school. I hustled my way into a job at a firm that I never imagined I could get a job at. But once I got in, that's when I was like, “Okay, I don't feel like I fit in.”And so, I struggled but I still you know grinded it out. I stayed at the job for a couple of years. And I left because I was like, “This is not right for me.” But I never imagined that all of those experiences in big law ended up being the source material for my content, like, eight years after I'd left. So, I'm very thankful that I had that experience even if it wasn't a good fit for me. [laugh].Corey: And on some level, it feels like, “Where do you get your material from?” It's, “Oh, the terrible things that happened to me. Why do you ask?”Alex: That's basically it. And people ask me, they say, you know, “You haven't worked in that environment for eight years. It's probably different now, right?” Well, no. You know, the legal industry is not like the tech industry. Like, things move very slowly there.The jokes that made people laugh back then, you know, 10 years ago, even 20 years ago, people still laugh at today because it's the same way things have always worked. So, again, I'm very thankful that that's been the case. And, you know, I feel like, the reason why my content is popular is because a lot of people can resonate with it. Things that a lot of people don't really talk about publicly, about the lifestyle, the culture, how things work in a large firm, but I make jokes about it, so people feel comfortable laughing about it, or commenting and sharing.Corey: I want to get into that a little bit because when you start seeing someone pop up again and again and again on TikTok, you're one of those, “Okay, I should stalk this person and figure out what the hell their story is.” And I didn't have to look very far in your case because you're very transparent about it. You're the head of community development at a company called Ironclad, and that one threw me for a little bit of a loop. So, let's start with the easy question, I suppose. What is Ironclad?Alex: We're a digital contracting technology that helps accelerate business contracts. Companies deal with contracts of all types; a lot of times it gets bogged down in legal review. We just help with that process to make that process move faster. And I never expected I'd be in this space. You know, I always thought I was going to be a trial lawyer.But I left that world, you know, maybe six years ago to go into the legal technology space, and I quickly saw that contracts was kind of a growing challenge, contracting, whether it's for sales or for procurement. So, I found myself as a salesperson in legal tech selling, first e-discovery software, and then contracting software. And then I found my way to Ironclad as part of the community team, really to talk about how we can help, but also speaking up about the challenges of the legal profession, of working at a law firm or at a legal department. So, I feel like it's all been the culmination of all my experiences, both in law and technology.Corey: In the world in which I've worked, half of my consulting work has been helping our clients negotiate their large-scale AWS contracts and the other half is architectural nonsense of, “Hey, if you make these small changes, that cuts your bill in half. Maybe consider doing them.” But something that I've learned that is almost an industry-wide and universal truism, is that you want to keep the salespeople and the lawyers relatively separate just due to the absolute polar opposites of incentives. Salespeople are incentivized to sell anything that holds still long enough or they can outrun, whereas lawyers are incentivized to protect the company from risk. No, is the easy answer and everything else is risk that has to be managed. You are one of those very rare folks who has operated successfully and well by blending the two. How the hell did that happen?Alex: I'm not sure to this day how it happened. But I think part of the reason why I left law in the first place was because I don't think I fit in. I think there's a lot of good about having a law degree and being part of the legal profession, but I just wanted to be around people, I wanted to work with people, I didn't want to always worry about things. And so, that led me to technology sales, which took me to the other extreme. And so, you know, I carried a sales quota for five years and that was such an interesting experience to see where—to both sell technology, but also to see where legal fit into that process.And so, I think by having the legal training, but also having been part of a sales team, that's given me appreciation for what both teams do. And I think they're often at tension with one another, but they're both there to serve the greater goals of the company, whether it's to generate revenue or protect against risk.Corey: I think that there's also a certain affinity that you may have—I'm just spitballing wildly—one of the things that sales folks and attorneys tend to have in common is that in the public imagination, as those roles are not, shall we call it, universally beloved. There tend to be a fair number of well, jokes, in which case, both sides of that tend to be on the receiving end. I mean, at some level, all you have to do is become an IRS auditor and you've got the holy trifecta working for you.Alex: [laugh]. I don't know why I gravitated to these professions, but I do think that it's partly because both of these roles hold a significant amount of power. And if you look at just contracting in general, a salesperson at a company, they're really the driver of the sales process. Like, if there's no sale to be made, there's no contract. On the flip side, the law person, the lawyer, knows everything about what's inside of the contract.They understand the legal terms, the jargon, and so they hold an immense amount of power over advising people on what's going to happen. And so, I think sometimes, salespeople and legal people take it too far and either spend too much time reviewing a contract and lording it over the business folks, or maybe the salesperson is too blase about getting a deal done and maybe bypasses legal and doesn't go through the right processes. By the way, Corey, these are jokes that I make in my TikToks all the time and they always go viral because it's so relatable to people. But yeah, that's probably why people always make jokes about lawyers and salespeople. There's probably some element of ridiculing people with a significant amount of power within a company to determine these transactions.Corey: Do you find that you have a better affinity for the folks doing contract work on the seller side or the buyer side? Something they don't tell you when you run companies is, yeah, you're going to spend a lot of time working on contracts, not just when selling things, but also when buying things and going back and forth. Aspects of what you're talking about so far in this conversation have resonated, I guess, with both sides of that for me. What do you have the affinity for?Alex: I think on the sales side, just because of my experience, you know, I think when you go through a transaction and you're trying to convince someone to doing something, and this is probably why I wanted to go to law school in the first place. Like I watched those movies, right? I watched A Few Good Men and I thought I'd be standing up in court convincing a jury of something. Little did I know that that sort of interest [crosstalk 00:10:55]—Corey: Like, Perry Mason breakthrough moment.Alex: That moment where—the gotcha moment, right? I found that in sales. And so, it was really a thrill to be able to, like, talk to someone, listen to them, and then kind of convince them that, based on what challenges they're facing, for them to buy some technology. I love that. And I think that was again, tied to why I went to law school in the first place.I didn't even know sales was a possible profession because I grew up in an immigrant community that was like, you just go to school, and that'll lead to your career. But there's a lot of different careers that are super interesting that don't require formal schooling, or at least the seven years of schooling you need for law. So, I always identify with the sales side. And maybe that's just how I am, but obviously, the folks who deal with the buy side, it's a pretty important job, too.Corey: There's a lot of surprise when I start talking to folks in the engineering world. First, they're in for a rough awakening at times when they learn exactly how much qualified enterprise salespeople can make. But also because being a lawyer without, you know, the appropriate credentials to tie into that, you're going to have a bad time. There are regulatory requirements imposed on lawyers, whereas to be a salesperson, forget the law degree, forget the bachelor's, forget the high school diploma, all you really need to be able to do from an academic credential standpoint is show up.The rest of it is, can you actually sell? Can you have the conversations that convince people to see the outcome that benefits everyone? And I don't know what that it's possible, or advised necessarily, to be able to find a way to teach that in some formalized way. It almost feels like folks either have that spark or they don't. Do you think it's one of those things that can be taught? Do you think it's something that people have to have a pre-existing affinity for?Alex: It's both, right, because part of it is some people will just—they don't have the personality to really sell. It's also like their interest; they don't want to do that. But what I found that's interesting is that what I thought would make a good salesperson didn't end up being true when I looked at the most effective sellers. Like, in my head, I thought, “Oh, this is somebody who's very boisterous, very extroverted,” but I found that in my experience in B2B SaaS that the most effective sellers are very, very much active listeners. They're not the people showing up and talking at you. They are asking you about your day-to-day asking about processes, understanding the context of your situation, before making a small suggestion about what you might want to do.I was very impressed the first time I saw one of these enterprise sellers who was just so good at that. Like, I saw him, and he looked nothing like what I imagined an effective sales guy to look like. And he was really kind and he just, kind of, just talked to me, like, I was a human being, and listened to my answers. So, I do think that there is some element of nature, your talent when it comes to that, but it can also be trained because I think a lot of folks who have sales talent, they don't realize that they could be good at it. They think that they've got to be this extroverted, happy hour, partying, storyteller, where —Corey: The Type A personality that interrupts people as they're having the conversation.Alex: Yeah, yeah.Corey: Yeah.Alex: So anyways, I think that's why it's a mix of both.Corey: The conversations that I've learned the most from when I'm talking to prospects and clients have been when I asked the quote-unquote, dumb question that I already know the answer to, and then I shut up and I listen. And wow, I did not expect that answer. And when you dig a little further, you realize there's nuance that—at least in my case—that I've completely missed to the entire problem space. I think that is really one of the key differentiators to my mind, that separate people who are good at this role from folks who just misunderstand what the role is based upon mass media, or in other cases—same problem with lawyers—the worst examples, in some cases, of the profession. The pushy used car salesperson or the lawyer they see advertising on the back of a bus for personal injury cases. The world is far more nuanced than that.Alex: Absolutely. And I think you hit the nail on the head when you said, you know, you ask those questions and let them talk. Because that's an entire process within the sales process. It's called discovery, and you're really asking questions to understand the person's situation. More broadly, though, I think pitching at people doesn't seem to work as well as understanding the situation.And you know, I've kind of done that with my content, my TikToks because, you know, if you look at LinkedIn, a lot of people in our space, they're always prescribing solutions, giving advice, posting content about teaching people things. I don't do that. As a marketer, what I do is I talk about the problems and create discussions. So, I'll create a funny video—Corey: I think you're teaching a whole generation that maybe law school isn't what they want to be doing, after all there is that.Alex: There is that. There is that. It's a mix of things. But one of the things I think I focus on is talking about the challenges of working with a sales team if you're an in-house lawyer. And I don't prescribe technology, I don't prescribe Ironclad, I don't say this is what you need to do, but by having people talk about it, they realize, right—and I think this is why the videos are popular—as opposed to me coming out and saying, “I think you need technology because of XYZ.” I think, like, facilitating the conversation of the problem space, that leads people to naturally say, “Hey, I might need something. What do you guys do, by the way?”Corey: This episode is sponsored in part by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on-premises, private cloud, and they just announced a fully-managed service on AWS and Azure called BigAnimal, all one word. Don't leave managing your database to your cloud vendor because they're too busy launching another half-dozen managed databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications—including Oracle—to the cloud. To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: It sounds ridiculous for me to say that, “Oh, here's my entire business strategy: step one, I shitpost on the internet about cloud computing; step two, magic happens here; and step three people reach out to talk about their AWS bills.” But it's also true. Is that the pattern that you go through: step one, shitpost on TikTok; step two, magic happens here; and step three people reach out asking to learn more about what your company does? Or is there more nuance to do it?Alex: I'm still figuring out this whole thing myself, but I will say shitposting is incredibly effective. Because I'm active on Twitter. Twitter is where I start my shitposts. TikTok, I also shitpost, but in video format, I think the number one thing to do is figure out what resonates with people, whether it's the whole contracting thing or if it's frustrations about law school. Once you create something that's compelling, the conversation gets going and you start learning about what people are thinking.And I think that what I'm trying to figure out is how that can lead to a deeper conversation that can lead to a business transaction or lead to a sale. I haven't figured it out, right, but I didn't know that when I started creating content that spoke to people when I was a quota-carrying salesperson, people reached out to me for demo requests, for sales conversations. There is something that is happening in this quote-unquote, “Dark funnel,” that I'm sure you're very familiar with. There's something that's happening that I'm trying to understand, and I'm starting to see.Corey: This is probably a good thing to the zero in on a bit because to most people's understanding of the sales process, it would seem that you going out and making something of a sensation out of yourself on the internet, well what are you doing that for? That's not sales work? How is that sales? That's just basically getting distracted and going to do something fun. Shouldn't you be picking up the phone and cold calling people or mass-emailing folks who don't want to hear from you because you trick them into having a badge scanned somewhere? I don't necessarily think that is accurate. How do you see the interplay of what you do and sales?Alex: When you're selling something like makeup or clothing, it's a pretty transactional process. You create a video; people will buy, right? That's B2C. In B2B, it's a much more complex processes. There's so many touchpoints. The start of a sales conversation and when they actually buy may take six months, 12 months, years. And so, there's got to be a lot of touch points in between.I remember when I was starting out in my content journey, I had this veteran enterprise sales leader, like, your classic, like, CRO. He said to me, “Hey, Alex, your content's very funny, but shouldn't you be making cold calls and emails? Like, why are you spending your time doing this?” And I said, “Hey, listen, do you notice that I'm actually sourcing more outbound sales calls than any other sales rep? Like, have you noticed that?”And he's like, “Actually, yeah, I did notice that. You know, how are you doing it?” And I was like, “Do you not see that these two are tied? These are not people I just started calling. They are people who have seen my content over time. And this is how it works.”And so, I think that the B2B world is starting to wise up to this. I think, for example, Ironclad is leading the way on creating a community team to create those conversations, but plenty of B2B companies are doing the same thing. And so, I think by inserting themselves in a conversation—a two-way conversation—during that process, that's become incredibly effective, far more so than, like, cold-calling a lawyer or a developer who doesn't want to be bothered by some pushy salesperson.Corey: Busy, expensive professionals generally don't want to spend all their time doing that. The cold outreach emails that drive me nuts are, “Hey, can we talk for half an hour?” Yeah, I don't tend to think in terms of billable hours because that's not how I do anything that I do, but there is an internal rate that I used to benchmark and it's what you want me just reach into my pocket and give you how much money for a random opportunity to pitch me on something that you haven't even qualified whether I need or not? It's like, asking people for time is worse, in some ways, than asking for money because they can always make more money, but no one can make more time.Alex: Right, right. That's absolutely right.Corey: It's the lack of awareness of understanding the needs and motivations of your target market. One thing that I found that really aided me back when I was working for other folks was trying to find a company or a management structure that understood and appreciated this. Easy example, when I was setting out as an independent consultant after a few months I'd been doing this and people started to hear about me. But you know, it turns out that there are challenges to running a business that are not recommended for most people. And I debated, do I take a job somewhere else?So, I interviewed at a few places, and I was talking to one company that's active in the cloud costing space at the time and they wanted me to come aboard. But discussions broke down because they thought I was, quote, “More interested in thought leadership than I was and actually fixing the bills themselves.” And looking at this now, four years later or so, yeah, they were right. And amazing how that whole thing played out, but that the lack of vision around, there's an opportunity here, if we can chase it, at least in the places I was at, was relatively hard to come by. Did you luck out in finding a role that works for you in this way or did you basically have to forge it for yourself from the sweat of your brow and the strength of your TikTok account?Alex: It was uphill at first, but eventually, I got lucky. And you know, part of it was engineered luck. And I'll explain what I mean. When I first started out doing this, I didn't expect this to lead to any jobs. I just thought it would support my sales career.Over time, as the content got more popular, I never wanted to do anything else because I was like, I don't want to be a marketer. I'm not a—I don't know anything about demand gen. All I know is how to make funny videos that get people talking. The interesting that happened was that these videos created this awareness, this energy in our space, in the legal space. And it wasn't long before Ironclad found me.And you know, Ironclad has always been big on community, has always done things like—like, our CEO, our founder, he said that he used to host these dinners, never talking about Ironclad, but just kind of talking about law school and law with potential clients. And it would lead to business. Like, it's almost the same concept of, like, not pushing sales on people. And so, Ironclad has always had that in its DNA. And one of our investors, our board members, Jessica Lee from Sequoia, she is a huge believer in community.I mean, she was the CEO of another company that leveraged community, and so there's this community element all throughout the DNA of Ironclad. Now, had I not put myself out there with this content, I may not have been discovered by Ironclad. But they saw me, they found me, and they said, “We don't think about these things like many other companies. We really want to invest in this function.” And so, it's almost like when you put yourself out there, yes, sometimes some people will say, “What are you doing? Like, this makes no sense. Like, stop doing that.” But there's going to be some true believers who come out and seek you out and find you.And that's been my experience here, like, at Ironclad. Like, people were like, “When you go there, are they going to censor you? Is your content going to be less edgy?” No. Like, they pulled me aside multiple times and said, “Keep being yourself. This is what we want.” And I think that is so special and unique. And part of it is very much lucky, but it's also when you put yourself out there kind of in a big way, like-minded people will seek you out as well.Corey: I take the position that part of marketing, part of the core of marketing, is you've got to have an opinion. But as soon as you have an opinion, people are going to disagree with you. They're going to, effectively, forget the human on the other side of it and start taking you for a drag on social media and whatnot. So, the default reaction a lot of people have is oh, I shouldn't venture opinions forward.No. People are always going to dislike you for something and you may as well have it be for who you are and what you want to be doing rather than who you're pretending to be. That's always been my approach. For me, the failure mode was not someone on Twitter is going to get mad about what I wrote. No one's going to read it. That's the failure mode. And the way to avoid that is make it interesting.Alex: That is a hundred percent relatable to me because I think when I was younger, I was scared. I did worry that I would get in trouble for what I posted. But I realized these people I was worried about, they weren't going to help me anyways. These are not people who are going to seek me out and help me but then say, “Oh, I saw your content, so now I can't help you.” They were not going to help me anyways.But by being authentic to myself and putting things out there, I attracted my own tribe of people who have helped me, right? A lot of my early results from content came not because I reached my target customers; it was because somebody resonated with what I put out there and they carried my message and said, “Hey, you should talk to Alex.” Something special happens when you kind of put yourself out there and say an opinion or share a perspective that not everyone agrees with because that tribe you build ends up helping you a lot. And meanwhile, these other people that might not like it, they probably weren't going to help you either.Corey: I maintain that one of the most valuable commodities in the universe is attention. And so, often there's so much information overload that's competing for our attention every minute of every day that trying to blend in with the rest of it feels like the exact wrong approach. I'm not a large company here. I don't have a full marketing department to wind up doing ad buys, and complicated campaigns, and train a team of attacking interns to wind up tackling people to scan their badges at conferences. I've got to work with what I've got.So, the goal I've always had is trigger the Rolodex moment where someone hears about a problem in the AWS billing space—ideally—and, “Oh, my God, you need to talk to Corey about that.” And it worked, for better or worse. And a lot of it was getting lucky, let's be very clear here, and people doing me favors that they had no reason to do and I'll never be able to repay. But being able to be in that space really is what made the difference. Now, the downside, of course, when you start doing that is, how do you go back to what happened before?If you decide okay, well, it's been a fun run for you and Ironclad. And yeah, TikTok. Turns out that is, in fact, for kids; time to go somewhere else. Like, I don't know that you would fit into your old type of job.Alex: Yeah. No, I wouldn't. But very early on, I realized, I said, “If I'm going to find meaningful work, it's okay to be wrong.” And when I went to big law, I realized this is not right for me. That's okay. I'm just not going to get another big law job.And so, when people ask me, “Hey, now that you've put yourself out there, you probably can't get a job at a big firm anymore.” And that's okay to me because I wasn't going to go back anyways. But what I have found, Corey, is that there's this other universe of people, whether it's a entrepreneur, smaller businesses, technology companies, they would be interested in working with me. And so, by being myself, I may have blocked out a certain level of opportunities or a safety net, but now I'm kind of in this other world where I feel very confident that I won't have trouble finding a job. So, I feel very lucky to have that, but that's why I also don't worry about the possibility of not going back.Corey: Yeah, I've never had to think about the idea of, well, what if I go have to get a job again? Because at that point, it means well, it's time to let every one at the company who is depending on the go, and that's the bigger obstacle because, let's be honest, I'm a white guy in tech, and I look like it. My failure mode is basically a board seat and a book deal because of inherent bias in the system.Alex: [laugh]. Oh, my god.Corey: That's the outcome that, for me personally, I will be just fine. It's the other people took a chance on me. I'm terrified of letting them down. So far, knock on wood, I haven't said anything too offensive in public is going to wind up there. That's also not generally my style.But it is the… it is something that has weighed on me that has kept me from I guess, thinking about what would my next job be? I'm convinced this is the last job I'll ever have, if for no other reason that I've made myself utterly unemployable.Alex: [laugh]. Well, I think many of us aspire to find that perfect intersection of what you love doing and what pays the bills. Sounds like you've found it, I really do feel like I found it, too. I never imagined I'd be doing what I do now. Which is also sometimes hard to describe.I'm not making TikToks for a living; I'm just on the community team, doing events—I'm getting to work with people. I'm basically doing the things that I wanted to do that led me to quit that job many years ago, that big law job many years ago. So, I feel very blessed and for anybody who's, like, looking for that type of path, I do think that at some point, you do need to kind of shed the safety nets because if you always hang on to the safety nets, whether it's a big tech job or a big law job, there's going to be elements of that that don't fit in with your personality, and you're never going to be able to find that if you kind of stay there. But if you venture out—and, you know, I admire you for what you've done; it sounds like you're very successful at what you do and get to do what you love every day—I think great things can happen.Corey: Yeah, I get to insult Amazon for a living. It's what I love. It's what I would do if I weren't being paid. So, here we are. Yeah—Alex: [laugh].Corey: I have no sense of self-preservation. It's kind of awesome.Alex: I love it.Corey: But you're right. It's… there's something to be said for finding the thing that winds up resonating with you and what you want to be doing.Alex: It really does. And you know, I think when I first made the move to technology, to sales, there was no career path. I thought I would—maybe I thought I might be a VP of Sales. But the thing is, when you put yourself out there, the opportunities that show up might not be the ones that you had always seen from the beginning. Like if you ask a lawyer, like, “What can I do if I don't practice law?” They're going to give you these generic answers. “Work here. Work there. Work for that company. I've seen a lot of people do this.”But once you put yourself out there in the wilderness, these opportunities arise. And I've been very lucky. I mean, I never imagined I'd be a TikTokker. And by the way, I also make memes on Twitter. Couldn't imagine I'd be doing that either. I learned, like, Mematic, these tools. Like, you know, like, I'm immersed in this internet culture now.Corey: It is bizarre to me and I never saw it coming either. For better or worse, though, here we are, stuck at it.Alex: [laugh].Corey: I really want to thank you for taking so much time to speak with me today. If people want to learn more about what you're up to and follow along for the laughs, if nothing else, where's the best place for them to find you?Alex: The best way to find me is on LinkedIn; just look up Alex Su. But I'm around and on lots of social media platforms. You can find me on Twitter, on Instagram, and on TikTok, although I might be a little bit embarrassed of what I put on TikTok. I put some crazy gnarly stuff out there. But yeah, LinkedIn is probably the best place to find me.Corey: And we will put links to all of it in the show notes, and let people wind up making their own decisions. Thanks so much for your time, Alex. I really appreciate it.Alex: Corey, thank you so much for having me. This was so much fun.Corey: Alex Su, Head of Community Development at Ironclad. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry insipid comment talking about how unprofessional everything we talked about is that you will not be able to post for the next six months because it'll be hung up in legal review.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About JonA husband, father of 3 wonderful kids who turned Podcaster during the pandemic. If you told me in early 2020 I would be making content or doing a podcast, I probably would have said "Nah, I couldn't see myself making YouTube videos". In fact, I told my kids, no way am I going to make videos for YouTube. Well, a year later I'm over 100 uploads and my subscriber count is growing.Links Referenced: LinkedIn: https://www.linkedin.com/in/jon-myer/ Twitter: https://twitter.com/_JonMyer jonmyer.com: https://jonmyer.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by Honeycomb. When production is running slow, it's hard to know where problems originate. Is it your application code, users, or the underlying systems? I've got five bucks on DNS, personally. Why scroll through endless dashboards while dealing with alert floods, going from tool to tool to tool that you employ, guessing at which puzzle pieces matter? Context switching and tool sprawl are slowly killing both your team and your business. You should care more about one of those than the other; which one is up to you. Drop the separate pillars and enter a world of getting one unified understanding of the one thing driving your business: production. With Honeycomb, you guess less and know more. Try it for free at honeycomb.io/screaminginthecloud. Observability: it's more than just hipster monitoring.Corey: DoorDash had a problem. As their cloud-native environment scaled and developers delivered new features, their monitoring system kept breaking down. In an organization where data is used to make better decisions about technology and about the business, losing observability means the entire company loses their competitive edge. With Chronosphere, DoorDash is no longer losing visibility into their applications suite. The key? Chronosphere is an open-source compatible, scalable, and reliable observability solution that gives the observability lead at DoorDash business, confidence, and peace of mind. Read the full success story at snark.cloud/chronosphere. That's snark.cloud slash C-H-R-O-N-O-S-P-H-E-R-E.Corey: Welcome to Screaming in the Cloud, I'm Corey Quinn. Every once in a while I get to talk to a guest who has the same problem that I do. Now, not that they're a loud, obnoxious jerk, but rather that describing what they do succinctly is something of a challenge. It's not really an elevator pitch anymore if you have to sabotage the elevator before you start giving it. I'm joined by Jon Myer. Jon, thank you for joining me. What the hell do you do?Jon: Corey, thanks for that awesome introduction. What do I do? I get to talk into a microphone. And sometimes I get to stare at myself on camera, whether it makes a recording or not. And either I talk to myself or I talk to awesome people like you. And I get to interview and tell other people's stories on my show; I pull out the interesting parts and we have a lot of freaking fun doing it.Corey: I suddenly feel like I've tumbled down the rabbit hole and I'm in the wrong side of the conversation. Are we both trying to stand in the same part of the universe? My goodness.Jon: Is this your podcast or mine? Maybe I should do an introduction right now to introduce you onto it and we'll see how this works.Corey: The dueling podcast banjo. I liked the approach quite a bit. So, you have done a lot of very interesting things. For example, once upon a time, you worked at AWS. But you have to go digging to figure that out because everything I'm seeing about you in your professional bio and the rest is forward-looking, as opposed to Former Company A, Former Company B, and this one time I was an early investor in Company C, which means, that's right, one of the most interesting things about me is that I wrote a check once upon a time, which is never something I ever want to say about myself, ever. You're very forward-looking, and I strive to do the same. How do you wind up coming at it from that position?Jon: When I first left AWS—it's been a year ago, so I served my time—and I actually used to have ex-Amazonian on it and listed on it. But as I continuously look at it, I used to have a podcast called The AWS Blogger. And it was all about AWS and everything, and there's nothing wrong with them. And what I would hear—Corey: Oh, there's plenty wrong with them, but please continue.Jon: [laugh]. We won't go there. But anyway, you know, kind of talking about it and thinking about it ex-Amazonian, yeah, that's great, you put it on your resume, put it on your stuff, and it, you know, allows you that foot in the door. But I want to look at and separate myself from AWS, in that I am my own independent voice. Yes, I worked for them; great company, I've learned so much from them, worked with some awesome people there, but my voice in the community has become very engaging and trustworthy. I don't want to say I'm no longer an Amazonian; I still have some of the guidelines, some of the stuff that's instilled in me, but I'm independent. And I want that to speak for itself when I come into a room.Corey: It's easy as hell, by the way, for me to sit here and cast stones at folks who, “Oh, you're going to talk about this big company you worked for, even though you don't work there anymore.” Yeah, I really haven't worked anywhere that most people would recognize unless they're, you know, professionally sad all the time. So, I don't have that luxury; I had to wind up telling a story that was forward-looking just because I didn't really have much of a better option. You have that option and decided to go in a direction where it presents, honestly as your viewpoint is that your best days are yet to come. And I want to be clear that for folks who are constantly challenged in our space to justify their existence there, usually because they don't look like our wildly over-represented selves, Jon, they need that credibility.And when they say that it's necessary for them, I am not besmirching that. I'm speaking from my own incredibly privileged position that you share. That is where I'm coming from on this, so I don't want people to hear this as shaming folks who are not themselves wildly over-represented. I'm not talking about you fine folks, I assure you.Jon: You can have ex-Amazonian on your resume and be very proud of it. You can remove it and still be very proud of the company. There's nothing wrong with either approach. There are some conversations that I'll be in, and I'll be on with AWS folks and I'll say, “I completely understand where you're coming from. I'm an ex-Amazonian.” And they're like, “Oh, you get us. You get the process. You get the everything.”I just want to look forward that I will be that voice in the community and that I have an understanding of what AWS is and will continuously be. And I have so much that I'm working towards that I'm very proud of where I've come from, but I do want to look forward.Corey: One of these days, I really feel like I should hang out with some Amazonians or ex-Amazonians who don't know who I am—which is easier to find than you think—and pretend that I used to work there and wonder how long I can keep the ruse going. Just because I've been told a few times that I am suspiciously Amazonian for someone who's never worked there.Jon: You have a lot of insights on the AWS processes and understanding. I think you could probably keep it going for quite a while. You will have to get that orange lanyard though, when you go to, like—Corey: I got one once when I was at a New York Summit a couple years ago. My affiliation then, before I started The Duckbill Group, was Last Week in AWS, and apparently, someone saw that and thought that I was the director of Take-this-Job-and-Shove-it, but I'll serve out my notice until Friday. So, cool; employee lanyard, it was. And I thought this is going to be awesome because I'll be able to walk around and I'll get the inside track if people think I work there. And they treated me like crap until I put the customer lanyard back on. It's, “Oh, it's better to be a customer at an AWS event than it is to be an employee.” I learned that when the fun way.Jon: There is one day that I hope to get the press or analyst lanyard. I think it would be an accomplishment for me. But you get to experience that firsthand, and I hate to switch the tables because I know it's your podcast recording, not mine, but—Corey: Having the press analyst lanyard is interesting because a lot of people are not allowed to speak to you unless they've gone through training. Which, okay, great. I will say that it is a lot nicer walking the expo floor because most of the people working the booths know that means that person is press, generally—they're not quite as familiar with analysts—but they know that regardless that they're not going to sell you a damn thing, so they basically give you a little bit of breathing room, which is awesome, especially in these pandemic times. But the challenge I have with it is that very often I want to talk to folks who are AWS employees who may not have gone through press training. And I've never gotten anyone in trouble or taken advantage of things that I hear in those conversations and write about them.Everything I write about is what I've experienced in public or as a customer, not based upon privileged inside information. I have so many NDAs at this point, I can't keep track, so I just make sure everything I talked about publicly cited I have that already.Jon: Corey, I got to flip the script real quick. I got to give you a shout-out because everybody sees you on Twitter and sees, like, “Oh, my God, he's saying this negative, that negative towards AWS.” You and I had, I don't know, it was a 30, 45 minute at the San Francisco Summit, and I think every Summit, we try to connect for a little bit. But that was really the premise I kicked off a lot of our conversations when you joined my podcast. No, this is not my podcast, this is Corey's, but anyway—Corey: And just you remember that. Please continue.Jon: [laugh]. But you know, kind of going off it you have so much insight, so much value, and you kind of really understand the entire processes and all the behind the scenes and everything that's going on that I was like, “Corey, I got to get your voice out there and show the other side of you, that you're not there trying to get people in trouble, you never poke fun of an AWS employee. I heard there was some guy named Larry that you do, but we won't jump into that.”Corey: One of the things that I think happened is, first and foremost, there is an algorithmic bias towards outrage. When I say nice things about AWS or other providers, which I do periodically, they get basically no engagement. When I say something ridiculous, inflammatory, and insulting about a company, oh, goes around the internet three times. One of the things that I'm slowly waking up to is that when I went into my Covid hibernation, my audience was a quarter of the size it is now. People don't have the context of knowing what I've been up to for the last five or six years. All they see is a handful of tweets.And yeah, of course, you wind up taking some of my more aggravated moment tweets and put a few of those on a board, and yeah, I start to look a fair bit like a jerk if you're not aware of what's going on inside-track-wise. That's not anyone else's fault, except my own, and I guess understanding and managing that perception does become something of a challenge. I mean, it's weird; Amazon is a company that famously prides itself on being misunderstood for long periods of time. I guess I never thought that would apply to me.Jon: Well, it does. Maybe that's why most people think you're an Amazonian.Corey: You know, honestly, I've got to say, there are a lot of worse things people can and do call me. Amazon has a lot to recommend it in different ways. What I find interesting now is that you've gone from large companies to sort of large companies. You were at Spot for a hot minute, then you were doing the nOps thing. But one thing that you've been focusing on a fair bit has been getting your own voice and brand out there—and we talked about this a bit at the Summit when we encountered each other which is part of what sparked this conversation—you're approaching what you're doing next in a way that I don't ever do myself. I will not do it justice, but what are you working on?Jon: All right. So Corey, when we talked at the New York Summit, things are actually moving pretty good. And some of the things that I am doing, and I've actually had a couple of really nice engagements kind of kick off is, that I'm creating highly engageable, trustworthy content for the community. Now, folks, you're asking, like, what is that? What is that really about? You do podcasts?Well, just think about some of the videos that you're seeing on customer sites right now. How are they doing? How's the views? How's the engagement? Can you actually track those back to, like, even a sales engagement in utilizing those videos?Well, as Jon Myer—and yes, this is highly scalable because guess what I am in talks with other folks to join the crew and to create these from a brand awareness portion, right? So, think about it. You have customers that you want to get engaged with: you have products, you have demos, you have reviews that you want to do, but you can't get them turned around in a quick amount of time. We take the time to actually dive into your product and pull out the value prop of the exact product, a demo, maybe a review, all right? We do sponsors as well; I have a number of them that I can talk about, so Veeam on AWS, Diabolical Coffee, there's a couple of other I cannot release just yet, but don't worry, they will be hitting out there on social pretty soon.But we take that and we make it an engaging kind of two to three-minute videos. And we say, “Listen, here's the value of it. We're going to turn this around, we're going to make this pop.” And putting this stuff, right, so we'll take the podcast and I'll put it on to my YouTube channel, you will get all my syndication, you'll get all my viewers, you'll get all my views, you'll get my outreach. Now, the kicker with that is I don't just pick any brand; I pick a trusted brand to work with because obviously, I don't want to tarnish mine or your brand. And we create these podcasts and we create these videos and we turn them around in days, not weeks, not months. And we focus on those who really need to actually present the value of their product in the environment.Corey: It sounds like you're sort of the complement to the way that I tend to approach these things. I'll periodically do analyst engagements where I'll kick the tires on a product in the space—that's usually tied to a sponsorship scenario, but not always—where, “Oh, great. You want me to explain your product to people. Great, could I actually kick the tires on it so I understand at first? Otherwise, I'm just parroting what may as well be nonsense. Maybe it's true, maybe it's not.”Very often small companies, especially early stage, do a relatively poor job of explaining the value of their product because everyone who works there knows the product intimately and they're too close to the problem. If you're going to explain what this does in a context where you have to work there and with that level of intensity on the problem space, you're really only pitching to the already converted as opposed to folks who have the expensive problem that gets in the way of them doing their actual job. And having those endless style engagements is great; they periodically then ask me, “Hey, do you want to build a bunch of custom content for us?” And the answer is, “No, because I'm bad at deadlines in that context.”And finding intelligent and fun and creative ways to tell stories takes up a tremendous amount of time and is something that I find just gets repetitive in a bunch of ways. So, I like doing the typical sponsorships that most people who listen to this are used to: “This episode is sponsored by our friends at Chex Mix.” And that's fine because I know how to handle that and I have that down to a set of study workflows. Every time I've done custom content, I find it's way more work than I anticipated, and honestly, I get myself in trouble with it.Jon: Well, when you come across it, you send them our way because guess what, we are actually taking those and we're diving deep with them. And yes, I used an Amazon term. But if you take their product—yeah [laugh]. I love the reaction I got from you. But we dive into the product. And you said it exactly: those people who are there at the facility, they understand it, they can say, “Yeah, it does this.”Well, that's not going to have somebody engaged. That's not going to get somebody excited. Let me give you an example. Yesterday, I had a call with an awesome company that I want to use their product. And I was like, “Listen, I want to know about your product a little bit more.”We demoed it for my current company, and I was like, “But how do you work for people like me: podcasters who do a lot of the work themselves? Or a social media expert?” You know, how do I get my content out there? How does that work? What's your pricing?And they're, like, “You know, we thought about getting it and see if there was a need in that space, and you're validating that there's a need.” I actually turned it around and I pitched them. I was like, “Listen, I'd love for you guys to be a sponsor on my show. I'd love for you to—let me do this. Let me do some demos. Let's get together.”And I pitched them this idea that I can be a spokesperson for their product because I actually believed in it that much just from two calls, 30 minutes. And I said, “This is going to be great for people like me out there and getting the voice, getting the volume out there, how to use it.” I said, “I can show some quick integration setups. You don't have to have the full-blown product that you sell out the businesses, us as individuals or small groupings, we're only going to use certain features because, one, is going to be overwhelming, and two, it's going to be costly. So, give us these features in a nice package and let's do this.” And they're like, “Let's set something up. I think we got to do this.”Corey: How do you avoid the problem where if you do a few pieces of content around a particular brand, you start to become indelibly linked to that brand? And I found that in my early days when I was doing a lot of advisory work and almost DevRel-for-hire as part of the sponsorship story thing that I was doing, and I found that that did not really benefit the larger thing I was trying to build, which is part of the reason that I got out of it. Because it makes sense for the first one; yeah, it's a slam dunk. And the second one, sure, but sooner or later, it feels like wow, I have five different sponsors in various ways that want me to be building stories and talking about their stuff as I travel the world. And now I feel like I'm not able to do any of them a decent service, while also confusing the living hell out of the audience of, “Who is it you work for again anyway?” It was the brand confusion, for lack of a better term.Jon: Okay, so you have two questions there. One of them is, how do you do this without being associated with the brand? I don't actually see a problem with that. Think of a race car; NASCAR drivers are walking around with all their stuff on their jackets, you know, sponsored by this person, this group, that group. Yeah, it's kind of overwhelming at times, but what's wrong with being tied to a couple of brands as long as the brands are trustworthy, like yourself? Or you believing those, right? So, there's nothing wrong with that.Second is the scalability that you're talking about where you're traveling all over the world and doing this and that. And that's where I'm looking for other leaders and trustworthy community members that are doing this type of thing to join a highly visible team, right? So, now you have a multitude and a diverse group of individuals who can get the same message out that's ultimately tied to—and I'm actually going to call it out here, I have it already as Myer Media, right? So, it's going to be under the Jon Myer Podcast; everything's going to be grouped in together under Myer Media, and then we're going to have a group of highly engaging individuals that enjoy doing this for a living, but also trust what they're talking about.Corey: If you can find a realistic way to scale that, that sounds like it's going to have some potential significant downstream consequences just as far as building almost a, I guess, a DevRel workshop, for lack of a better term. And I mean, that in the sense of an Andy Warhol workshop style approach, not just a training course. But you wind up with people in your orbit who become associated, affiliated with a variety of different brands. I mean, last time I did the numbers, I had something like 110 sponsors over the last five years. If I become deeply linked to those brands, no one knows what the hell I do because every company in the space, more or less, has at some level done a sponsorship with me at some point.Jon: I guess I'll cross that when it happens, or keep that in the top-of-mind as it moves forward. I mean, it's a good point of view, but I think if we keep our individualism, that's what's going to separate us as associated. So, think of advertising, you have a, you know, actor, actress that actually gets on there, and they're associated with a certain brand. Did they do it forever? I am looking at long-term relationships because that will help me understand the product in-depth and I'll be able to jump in there and provide them value in a expedited version.So, think about it. Like, they are launching a new version of their product or they're talking about something different. And they're, like, “Jon, we need to get this out ASAP.” I've had this long-term relationship with them that I'm able to actually turn it around rather quickly, but create highly engaging out of it. I guess, to really kind of signify that the question that you're asking is, I'm not worried about it yet.Corey: What stage or scale of company do you find is, I guess, the sweet spot for what you're trying to build out?Jon: I like the small to medium. And looking at it, the small to medium—Corey: Define your terms because to my mind, I'm still stuck in this ancient paradigm that I was in as an employee, where a big company is anything that has more than 200 people, which is basically everyone these days.Jon: So, think about startups. Startups, they are usually relatively 100 or less; medium, 200 or less. The reason I like that type of—is because we're able to move fast. As you get bigger, you're stuck in processes and you have to go through so many steps. If you want speed and you want scalability, you got to pay attention to some of the stuff that you're doing and the processes that are slowing it down.Granted, I will evaluate, you know, the enterprise companies, but the individuals who know the value of doing this will ultimately seek me and say, “Hey, listen, we need this because we're just kicking this off and we need highly visible content, and we want to engage with our current community, and we don't know how.”Corey: This episode is sponsored in part by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on-premises, private cloud, and they just announced a fully-managed service on AWS and Azure called BigAnimal, all one word. Don't leave managing your database to your cloud vendor because they're too busy launching another half-dozen managed databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications—including Oracle—to the cloud. To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: I think that there's a fair bit of challenge somewhere in there. I'm not quite sure how to find it, that you're going to, I think, find folks that are both too small and too big, that are going to think that they're ready for this. I feel like this doesn't, for example, have a whole lot of value until a company has found product-market fit unless what you're proposing to do helps get them to that point. Conversely, at some point, you have some of the behemoth companies out there, it's, “Yeah, we can't hire DevRel people fast enough. We've hired 500 of them. Cool, can you come do some independent work for us?” At which point, it's… great, good luck standing out from the crowd in any meaningful way at that point.Jon: Well, even a high enterprise as hired X number of DevRels, the way you stand out is your personality and everything that you built behind your personal brand, and your value brand, and what you're trying to do, and the voice that you're trying to achieve out there. So, think about it—and this is very difficult for me to, kind of, boost and say, “Hey, listen, if I were to go to a DevRel of, like, say, 50 people, I will stand out. I might be one of the top five, or I might be two at the top five.” It doesn't matter. But for me why and what I do, the value that I am actually driving across is what will stand out, the engaging conversations.Every interview, every podcast that I do, at the end, everybody's like, “Oh, my God, you're, like, really good at it; you kind of keep us engaging, you know when to ask a question; you jump in there and you dive even deeper.” I literally have five bullet points on any conversation, and these are just, like, two or three sentences, maybe. And they're not exact questions. They're just topics that we need to talk about, just like we did going into this conversation. There is nothing that scripted. Everything that's coming across the questions that you're pulling out from me giving an answer to one of your questions and then you're diving deep on it.Corey: I think that that's probably a fair approach. And it's certainly going to lead to a better narrative than the organic storytelling that tends to arise internally. I mean, there's no better view to see a lot of these things than working on bills. One of my favorite aspects of what I do is I get to see the lies that clients tell to themselves, where it's—like, they believe these things, but it no longer matches the reality. Like developer environments being far too expensive as a proportion of the rest of their environment. It's miniscule just because production has scaled since you last really thought about it.Or the idea that a certain service is incredibly expensive. Well, sure. The way that it was originally configured and priced, it was and that has changed. Once people learn something, they tend to stop keeping current on that thing because now they know it. And that's a bit of a tricky thing.Jon: That's why we keep doing podcasts, you keep doing interviews, you keep talking with folks is because if you look at when you and I actually started doing these podcasts—and aka, like, webinars, and I hate to say webinars because it's always negative and—you know because they're not as highly engaging, but taking that story and that narrative and creating a conversation out of it and clicking record. There are so many times that when I go to a summit or an event, I will tell people, they're like, “So, what am I supposed to do for your podcast?” And we were talking for, like, ten minutes, I said, “You know, I would have clicked record and we would have ten minutes of conversation.” And they're like, “What?” I was like, “That's exactly what it is.”My podcast is all about the person that I'm interviewing, what they're doing, what they're trying to achieve, what's their message that they're trying to get across? Same thing, Corey. When you kick this off, you asked me a bunch of questions and then that's why we took it. And that's where this conversation went because it's—I mean, yeah, I'm spinning it around and making it about you, sometimes because obviously, it's fun to do that, and that's normally—I'm on the other side.Corey: No, it's always fun to wind up talking to people who have their own shows just because it's fun watching the narrative flow back and forth. It's kind of a blast.Jon: It's almost like commentators, though. You think about it at a sporting event. There's two in the booth.Corey: Do a team-up at some point, yeah.Jon: Yeah.Corey: In fact, doing the—what is it like the two old gentlemen in the Sesame Street box up in the corner? I forget their names… someone's going to yell at me for that one. But yeah, the idea of basically kibitzing back and forth. I feel like at some level, we should do a team up and start doing a play-by-play of the re:Invent keynotes.Jon: Oh… you know what, Corey, maybe we should talk about this offline. Having a huge event there, VIP receptions, a podcasting booth is set up at a villa that we have ready to go. We're going to be hosting social media influencers, live-tweeting happening for keynotes. Now, you don't have to go to the keynotes personally. You can come to this room, you can click record, we'll record a live session right there, totally unscripted, like everything else we do, right? We'll have a VIP reception, come in chat, do introductions. So, Corey, love to have you come into that and we can do a live one right there.Corey: Unfortunately, I'm going to be spending most of re:Invent this year dressed in my platypus costume, but you know how it works.Jon: [laugh]. Oh man, you definitely got to go for that because oh, I have a love to put that on the show. I'm actually doing something not similar, but in true style that I've been going to the last couple of re:Invents I will be doing something unique and standing out.Corey: I'm looking forward to it. It's always fun seeing how people continue to successfully exceed what they were able to do previously. That's the best part, on some level, is just watching it continually iterate until you're at a point where it just becomes, well frankly, either ridiculous or you flame out or it hits critical mass and suddenly you launch an entire TV network or something.Jon: Stay tuned. Maybe I will.Corey: You know, it's always interesting to see how that entire thing plays out. Last question before we call it a show. Talk to me about your process for building content, if you don't mind. What is your process when you sit down and stare at—at least from my perspective—that most accursed of all enemies, a blank screen? “All right time to create some content, Jackwagon, better be funny. And by the way, you're on a deadline.” That is the worst part of my job.Jon: All right, so the worst part of your job is the best part of my job. I have to tell you, I actually don't—and I'm going to have to knock on wood because I don't get content block. I don't sit at a screen when I'm doing it. I actually will go for a walk or, you know, I'll have my weirdest ideas at the weirdest time, like at the gym, I might have a quick idea of something like that and I'll have a backlog of these ideas that I write down. The thing that I do is I come down, I open up a document and I'll just drop this idea.And I'll write it out as almost as it seems like a script. And I'll never read it verbatim because I look at it and be like, “I know what I'm going to say right now.” An example, if you take a look at my intros that I do for my podcast, they are done after the recording because I recap what we do on a recording.So, let's take this back. Corey will talk about the one you and I just did. And you and I we hopped on, we did a recording. Afterwards, I put together the intro. And what I'm going to say the intro, I have no freaking clue until I actually get to it, and then all of a sudden, I think of something—not at my desk, but away from my desk—what I'm going to say about you or the guest.An example, there was a gentleman I did his name's called Mat Batterbee, and he's from the UK. And he's a Social Media Finalist. And he has this beard and he always wears, like, this hat or something. And I saw somebody on Twitter make a comment about, you know, following in his footsteps or looking like him. So, they spoofed him with a hat and everything—glasses.I actually bought a beard off of Amazon, put it on, glasses, hat, and I spoofed him for the intro. I had this idea, like, the day before. So, thank goodness for Prime delivery, that I was able to get this beard ASAP, put it on. One take; I only tried to do one take. I don't think I've ever recorded any more.Corey: I have a couple of times sometimes because the audio didn't capture—Jon: Yeah.Corey: —but that's neither here nor there. But yeah, I agree with you, I find that the back-and-forth with someone else is way easier from a content perspective for me. Because when you and I started talking, on this episode, for example, I had, like, three or four bullet points I wanted to cover and that's about it. The rest of it becomes this organic freewheeling conversation and that just tends to work when it's just me free-associating in front of the camera, it doesn't work super well. I need something that's a bit more structured in that sense. So apparently, my answer is just never be alone, ever.Jon: [laugh]. The content that I create, like how-to tutorials, demos, reviews, I'll take a lot more time on them and I'll put them together in the flow. And I record those in certain sections. I'll actually record the demo of walking through and clicking on everything and going through the process, and then I will actually put that in my recording software, and then I will record against it like a voiceover.But I don't record a script. I actually follow the flow that I did and in order to do that, I understand the product, so I'll dive deep on it, I'll figure out some of the things using keywords along the way to highlight the value of utilizing it. And I like to create these in, like, two to three minutes. So, my entire process of creating content—podcast—you know what we hop on, I give everybody the spiel, I click record and I say, “Welcome.” And I do the introduction. I cut that out later. We talk. I'll tell you what, I never edited anything throughout the entire length of it because whatever happens happens in his natural and comes across.And then I slap on an ending. And I try to make it as quick and as efficiently as possible because if I start doing cuts, people are going to be, like, “Oh, there's a cut there. What did he cut out?” Oh, there's this. It's a full-on free flow. And so, if I mess up and flub or whatever it is, I poke fun of myself and we move on.Corey: Oh, I have my own favorite punching bag. And I honestly think about that for a second. If I didn't mock myself the way that I do, I would be insufferable. The entire idea of being that kind of a blowhard just doesn't work. From my perspective, I am always willing to ask the quote-unquote dumb question.It just happens to turn out but I'm never the only person wondering about that thing and by asking it out loud, suddenly I'm giving a whole bunch of other folks air cover to say, “Yeah, I don't know the answer to that either.” I have no problem whatsoever doing that. I don't have any technical credibility to worry about burning.Jon: When you start off asking and say, “Hey, dumb question or dumb question,” you start being unsure of yourself. Start off and just ask the question. Never say it's a dumb question because I'll tell you what, like you said, there's probably 20 other people in that room that have the same question and they're afraid to ask it. You can be the one that just jumps up there and says it and then you're well-respected for it. I have no problem asking questions.Corey: Honestly, the problem I've got is I wish people would ask more questions. I think that it leads to such a better outcome. But people are always afraid to either admit ignorance. Or worse, when they do ask questions just for the joy they get from hearing themselves talk. We've all been conference talks where you there's someone who's just asking the question because they love the sound of their own voice. I say, they, but let's be serious; it's always a dude.Jon: That is very true.Corey: So, if people want to learn more about what you're up to, where's the best place to go?Jon: All right, so the best place to go is to follow me on LinkedIn. LinkedIn is my primary one, right? Jon Myer; can't miss me. At all. Twitter, I am active on Twitter. Not as well as Corey; I would love to get there one day, but my audience right now is LinkedIn.Else you can go to jonmyer.com. Yes, that's right, jonmyer.com. Because why not? I found I have to talk about this just a little bit. And the reason that I changed it—I actually do own the domain awsblogger, by the way and I still have it—is that when I was awsblogger, I had to chan—I didn't have to change anything' nobody required me to, but I changed it to, like, thedailytechshow. And that was pretty cool but then I just wanted to associated with me, and I felt that going with jonmyer, it allowed me not having to change the name ever again because, let's face it, I'm not changing my name. And I want to stick with it so I don't have to do a whole transition and when this thing takes off really huge, like it is doing right now, I don't have to change the name.Corey: Yeah. I would have named it slightly differently had I known was coming. But again, this far in—400 some-odd episodes in last I checked recorded—though I don't know what episode this will be when it airs—I really get the distinct impression that I am going to learn as I go and, you know, you can't change that this far in anymore.Jon: I am actually rounding so I'm not as far as you are with the episodes, but I'm happy to say that I did cross number 76—actually 77; I recorded yesterday, so it's pretty good. And 78 tomorrow, so I am very busy with all the episodes and I love it. I love everybody reaching out and enjoying the conversations that I have. And just the naturalness and the organicness of the podcast. It really puts people at ease and comfortable to start sharing more and more of their stories and what they want to talk about.Corey: I really want to thank you for being so generous with your time and speak with me today. Thanks. It's always a pleasure to talk with you and I look forward to seeing what you wind up building next.Jon: Thanks, Corey. I really appreciate you having me on. This is very entertaining, informative. I had a lot of fun just having a conversation with you. Thanks for having me on, man.Corey: Always a pleasure. Jon Myer, podcaster extraordinaire and content producer slash creator. The best folks really have no idea what to refer to themselves and I am no exception, so I made up my own job title. I am Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry, insulting comment telling me that I'm completely wrong and that you are a very interesting person. And then tell me what company you wrote a check to once upon a time.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About AntonDr. Anton Chuvakin is now involved with security solution strategy at Google Cloud, where he arrived via Chronicle Security (an Alphabet company) acquisition in July 2019.Anton was, until recently, a Research Vice President and Distinguished Analyst at Gartner for Technical Professionals (GTP) Security and Risk Management Strategies team. (see chuvakin.org for more)Links Referenced: Google Cloud: https://cloud.google.com/ Cloud Security Podcast: https://cloud.withgoogle.com/cloudsecurity/podcast/ Twitter: https://twitter.com/anton_chuvakin Medium blog: https://medium.com/anton.chuvakin TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on-premises, private cloud, and they just announced a fully-managed service on AWS and Azure called BigAnimal, all one word. Don't leave managing your database to your cloud vendor because they're too busy launching another half-dozen managed databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications—including Oracle—to the cloud. To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: Let's face it, on-call firefighting at 2am is stressful! So there's good news and there's bad news. The bad news is that you probably can't prevent incidents from happening, but the good news is that incident.io makes incidents less stressful and a lot more valuable. incident.io is a Slack-native incident management platform that allows you to automate incident processes, focus on fixing the issues and learn from incident insights to improve site reliability and fix your vulnerabilities. Try incident.io, recover faster and sleep more.Corey: Welcome to Screaming in the Cloud, I'm Corey Quinn. My guest today is Anton Chuvakin, who is a Security Strategy Something at Google Cloud. And I absolutely love the title, given, honestly, how anti-corporate it is in so many different ways. Anton, first, thank you for joining me.Anton: Sure. Thanks for inviting me.Corey: So, you wound up working somewhere else—according to LinkedIn—for two months, which in LinkedIn time is about 20 minutes because their date math is always weird. And then you wound up going—according to LinkedIn, of course—leaving and going to Google. Now, that was an acquisition if I'm not mistaken, correct?Anton: That's correct, yes. And it kind of explains that timing in a little bit of a title story because my original title was Head of Security Solution Strategy, and it was for a startup called Chronicle. And within actually three weeks, if I recall correctly, I was acquired into Google. So, title really made little sense of Google, so I kind of go with, like, random titles that include the word security, and occasionally strategy if I feel generous.Corey: It's pretty clear, the fastest way to get hired at Google, given their famous interview process is to just get acquired. Like, “I'm going to start a company and raise it to, like, a little bit of providence, and then do an acquihire because that will be faster than going through the loop, and ideally, there will be less algorithm solving on whiteboards.” But I have to ask, did you have to solve algorithms on whiteboards for your role?Anton: Actually, no, but it did come close to that for some other people who were seen as non-technical and had to join technical roles. I think they were forced to solve coding questions and stuff, but I was somehow grandfathered into a technical role. I don't know exactly how it happened.Corey: Yeah, how you wound up in a technical role. Let's be clear, you are Doctor Anton Chuvakin, and you have written multiple books, you were a research VP at Gartner for many years, and once upon a time, that was sort of a punchline in the circles I hung out with, and then I figured out what Gartner actually does. And okay, that actually is something fairly impressive, let's be clear here. Even as someone who categorically defines himself as not an analyst, I find myself increasingly having a lot of respect for the folks who are actually analysts and the laborious amount of work that they do that remarkably few people understand.Anton: That's correct. And I don't want to boost my ego too much. It's kind of big enough already, obviously, but I actually made it all the way to Distinguished Analyst, which is the next rank after VP.Corey: Ah, my apologies. I did not realize it. This [challenges 00:02:53] the internal structure.Anton: [laugh]. Yeah.Corey: It's like, “Oh, I went from Senior to Staff,” or Staff to Senior because I'm external; I don't know the direction these things go in. It almost feels like a half-step away from oh, I went from [SDE3 to SDE4 00:03:02]. It's like, what do those things mean? Nobody knows. Great.Anton: And what's the top? Is it 17 or is it 113? [laugh].Corey: Exactly. It's like, oh okay, so you're Research VP—or various kinds of VPs—the real question is, how many people have to die before you're the president? And it turns out that that's not how companies think. Who knew?Anton: That's correct. And I think Gartner was a lot of hard work. And it's the type of work that a lot of people actually don't understand. Some people understand it wrong, and some people understand it wrong, kind of, for corrupt reasons. So, for example, a lot of Gartner machinery involves soaking insight from the outside world, organizing it, packaging it, writing it, and then giving it as advice to other people.So, there's nothing offensive about that because there is a lot of insight in the outside world, and somebody needs to be a sponge slash filter slash enrichment facility for that insight. And that, to me, is a good analyst firm, like Gartner.Corey: Yeah. It's a very interesting world. But you historically have been doing a lot of, well, let's I don't even know how to properly describe it because Gardner's clientele historically has not been startups because let's face it, Gartner is relatively expensive. And let's be clear, you're at Google Cloud now, which is a different kind of expensive, but in a way that works for startups, so good for you; gold star. But what was interesting there is that the majority of the Gartner clientele that I've spoken to tend to be big-E Enterprise, which runs legacy businesses, which is a condescending engineering term for ‘it makes money.'And they had the temerity to start their company before 15 years ago, so they built data centers and did things in a data center environment, and now they're moving in a cloudy direction. Your emphasis has always been on security, so my question for you to start with all this is where do you see security vendors fitting in? Because when I walk the RSA expo hall and find myself growing increasingly depressed, it seems like an awful lot of what vendors are selling looks very little removed from, “We took a box, now we shoved in a virtual machine and here you go; it's in your cloud environment. Please pay us money.” The end. And it feels, if I'm looking at this from a pure cloud-native, how I would build things in the cloud from scratch perspective, to be the wrong design. Where do you stand on it?Anton: So, this has been one of the agonizing questions. So, I'm going to kind of ignore some of the context. Of course, I'll come back to it later, but want to kind of frame it—Corey: I love ignoring context. My favorite thing; it's what makes me a decent engineer some days.Anton: So, the frame was this. One of the more agonizing questions for me as an analyst was, a client calls me and says, “We want to do X.” Deep in my heart, I know that X is absolutely wrong, however given their circumstances and how they got to decided to do X, X is perhaps the only thing they can logically do. So, do you tell them, “Don't do X; X is bad,” or you tell them, “Here's how you do X in a manner that aligns with your goals, that's possible, that's whatever.”So, cloud comes up a lot in this case. Somebody comes and says, I want to put my on-premise security information management tool or SIM in the cloud. And I say, deep in my heart, I say, “No, get cloud-native tool.” But I tell them, “Okay, actually, here's how you do it in a less painful manner.” So, this is always hard. Do you tell them they're on their own path, but you help them tread their own path with least pain? So, as an analyst, I agonized over that. This was almost like a moral decision. What do I tell them?Corey: It makes sense. It's a microcosm of the architect's dilemma, on some level, because if you ask a typical Google-style interview whiteboard question, one of my favorites in years past was ‘build a URL shortener.' Great. And you can scale it out and turn it into different things and design things on the whiteboard, and that's great. Most mid-level people can wind up building a passable designed for most things in a cloud sense, when you're starting from scratch.That's not hard. The problem is that the real world is messy and doesn't fit on a whiteboard. And when you're talking about taking a thing that exists in a certain state—for whatever reason, that's the state that it's in—and migrating it to a new environment or a new way of operating, there are so many assumptions that have to break, and in most cases, you don't get the luxury of just taking the thing down for 18 months so you can rework it. And even that it's never as easy as people think it is, so it's going to be 36. Great.You have to wind up meeting people where they are as they're contextualizing these things. And I always feel like the first step of the cloud migration has been to improve your data center environment at the cost of worsening your cloud environment. And that's okay. We don't all need to be the absolute vanguard of how everything should be built and pushing the bleeding edge. You're an insurance company, for God's sake. Maybe that's not where you want to spend your innovation energies.Anton: Yeah. And that's why I tend to lean towards helping them get out of this situation, or maybe build a five-step roadmap of how to become a little bit more cloud-native, rather than tell them, “You're wrong. You should just rewrite the app in a cloud-native way.” That advice almost never actually works in real world. So, I see a lot of the security people move their security stacks to the cloud.And if I see this, I deepen my heart and say, “Holy cow. What do you mean, you want to IDS every packet between Cloud instances? You want to capture every packet in cloud instances? Why? It's all encrypted anyway.” But I don't say that. I say, “Okay, I see how this is the first step for you. Let's describe the next seven steps.”Corey: The problem I keep smacking into is that very often folks who are pushing a lot of these solutions are, yes, they're meeting customers where they are, and that makes an awful lot of sense; I'm not saying that there's anything inherently wrong about that. The challenge is it also feels on the high end, when those customers start to evolve and transform, that those vendors act as a drag. Because if you wind up going in a full-on cloud-native approach, in the fullness of time, there's an entire swath of security vendors that do not have anything left to sell you.Anton: Yes, that is correct. And I think that—I had a fight with an EDR vendor, Endpoint Detection Response, vendor one day when they said, “Oh, we're going to be XDR and we'll do cloud.” And I told them, “You do realize that in a true cloud-native environment, there's no E? There is no endpoint the way you understand it? There is no OS. There is no server. And 99% of your IP isn't working on the clients and servers. How are you going to secure a cloud again?”And I get some kind of rambling answer from them, but the point is that you're right, I do see a lot of vendors that meet clients where they are during their first step in the cloud, and then they may become a drag, or the customer has to show switch to a cloud-native vendor, or to both sometimes, and pay into two mouths. Well, shove money into two pockets.Corey: Well, first, I just want to interject for a second here because when I was walking the RSA expo floor, there were something like 15 different vendors that were trying to sell me XDR. Not a single one of them bothered to expand the acronym—Anton: Just 15? You missed half of them.Corey: Well, yeah—Anton: Holy cow.Corey: As far as I know XDR cable. It's an audio thing right? I already have a bunch of those for my microphone. What's the deal here? Like, “I believe that's XLR.” It's like, “I believe you should expand your acronyms.” What is XDR?Anton: So, this is where I'm going to be very self-serving and point to a blog that I've written that says we don't know what's XDR. And I'm going to—Corey: Well, but rather than a spiritual meaning, I'm going to ask, what does the acronym stands for? I don't actually know the answer to that.Anton: Extended Detection and Response.Corey: Ah.Anton: Extended Detection and Response. But the word ‘extended' is extended by everybody in different directions. There are multiple camps of opinion. Gartner argues with Forrester. If they ever had a pillow fight, it would look really ugly because they just don't agree on what XDR is.Many vendors don't agree with many other vendors, so at this point, if you corner me and say, “Anton, commit to a definition of XDR,” I would not. I will just say, “TBD. Wait two years.” We don't have a consensus definition of XDR at this point. And RSA notwithstanding, 30 booths with XDRs on their big signs… still, sorry, I don't have it.Corey: The problem that I keep running into again and again and again, has been pretty consistently that there are vendors willing to help customers in a very certain position, and for those customers, those vendors are spot on the right thing to do.Anton: Mmm, yep.Corey: But then they tried to expand and instead of realizing that the market has moved on and the market that they're serving is inherently limited and long-term is going to be in decline, they instead start trying to fight the tide and saying, “Oh, no, no, no, no. Those new cloud things, can't trust them.” And they start out with the FU, the Fear, Uncertainty, and Doubt marketing model where, “You can't trust those newfangled cloud things. You should have everything on-prem,” ignoring entirely the fact that in their existing data centers, half the time the security team forgets to lock the door.Anton: Yeah, yeah.Corey: It just feels like there is so much conflict of interest about in the space. I mean, that's the reason I started my Thursday Last Week in AWS newsletter that does security round-ups, just because everything else I found was largely either community-driven where it understood that it was an InfoSec community thing—and InfoSec community is generally toxic—or it was vendor-captured. And I wanted a round-up of things that I had to care about running an infrastructure, but security is not in my job title, even if the word something is or is not there. It's—I have a job to do that isn't security full time; what do I need to know? And that felt like an underserved market, and I feel like there's no equivalent of that in the world of the emerging cloud security space.Anton: Yes, I think so. But it has a high chance of being also kind of captured by legacy vendors. So, when I was at Gartner, there was a lot of acronyms being made with that started with a C: Cloud. There was CSPM, there was CWBP, and after I left the coined SNAPP with double p at the end. Cloud-Native Application Protection Platform. And you know, in my time at Gartner, five-letter acronyms are definitely not very popular. Like, you shouldn't have done a five-letter acronym if you can help yourself.So, my point is that a lot of these vendors are more from legacy vendors. They are not born in the cloud. They are born in the 1990s. Some are born in the cloud, but it's a mix. So, the same acronym may apply to a vendor that's 2019, or—wait for it—1989.Corey: That is… well, I'd say on the one hand, it's terrifying, but on the other, it's not that far removed from the founding of Google.Anton: True, true. Well, '89, kind of, it's another ten years. I think that if you're from the '90s, maybe you're okay, but if you're from the '80s… you really need to have superpowers of adaptation. Again, it's possible. Funny aside: at Gartner, I met somebody who was an analyst for 32 years.So, he was I think, at Gartner for 32 years. And how do you keep your knowledge current if you are always in an ivory tower? The point is that this person did do that because he had a unique ability to absorb knowledge from the outside world. You can adapt; it's just hard.Corey: It always is. I'm going to pivot a bit and put you in a little bit of a hot seat here. Not intentionally so. But it is something that I've been really kicking around for a while. And I'm going to basically focus on Google because that's where you work.I yeah, I want you to go and mouth off about other cloud companies. Yeah, that's—Anton: [laugh]. No.Corey: Going to go super well and no one will have a problem with that. No, it's… we'll pick on Google for a minute because Google Cloud offers a whole bunch of services. I think it's directionally the right number of services because there are areas that you folks do not view as a core competency, and you actually—imagine that—partner with third parties to wind up delivering something great rather than building this shitty knockoff version that no one actually wants. Ehem, I might be some subtweeting someone here with this, only out loud.Anton: [laugh].Corey: The thing that resonates with me though, is that you do charge for a variety of security services. My perspective, by and large, is that the cloud vendors should not be viewing security as a profit center but rather is something that comes baked into the platform that winds up being amortized into the cost of everything else, just because otherwise you wind up with such a perverse set of incentives.Anton: Mm-hm.Corey: Does that sound ridiculous or is that something that aligns with your way of thinking. I'm willing to take criticism that I'm wrong on this, too.Anton: Yeah. It's not that. It's I almost start to see some kind of a magic quadrant in my mind that kind of categorizes some things—Corey: Careful, that's trademarked.Anton: Uhh, okay. So, some kind of vis—Corey: It's a mystical quadrilateral.Anton: Some kind of visual depiction, perhaps including four parts—not quadrants, mind you—that is focused on things that should be paid and aren't, things that should be paid and are paid, and whatever else. So, the point is that if you're charging for encryption, like basic encryption, you're probably making a mistake. And we don't, and other people, I think, don't as well. If you're charging for logging, then it's probably also wrong—because charging for log retention, keeping logs perhaps is okay because ultimately you're spending resources on this—charging for logging to me is kind of in the vile territory. But how about charging for a tool that helps you secure your on-premise environment? That's fair game, right?Corey: Right. If it's something you're taking to another provider, I think that's absolutely fair. But the idea—and again, I'm okay with the reality of, “Okay, here's our object storage costs for things, and by the way, when you wind up logging things, yeah, we'll charge you directionally what it costs to store that an object store,” that's great, but I don't have the Google Cloud price list shoved into my head, but I know over an AWS land that CloudWatch logs charge 50 cents per gigabyte, for ingress. And the defense is, “Well, that's a lot less expensive than most other logging vendors out there.” It's, yeah, but it's still horrifying, and at scale, it makes me want to do some terrifying things like I used to, which is build out a cluster of Rsyslog boxes and wind up having everything logged to those because I don't have an unbounded growth problem.This gets worse with audit logs because there's no alternative available for this. And when companies start charging for that, either on a data plane or a management plane level, that starts to get really, really murky because you can get visibility into what happened and reconstruct things after the fact, but only if you pay. And that bugs me.Anton: That would bug me as well. And I think these are things that I would very clearly push into the box of this is security that you should not charge for. But authentication is free. But, like, deeper analysis of authentication patterns, perhaps costs money. This to me is in the fair game territory because you may have logs, you may have reports, but what if you want some kind of fancy ML that analyzes the logs and gives you some insights? I don't think that's offensive to charge for that.Corey: I come bearing ill tidings. Developers are responsible for more than ever these days. Not just the code that they write, but also the containers and the cloud infrastructure that their apps run on. Because serverless means it's still somebody's problem. And a big part of that responsibility is app security from code to cloud. And that's where our friend Snyk comes in. Snyk is a frictionless security platform that meets developers where they are - Finding and fixing vulnerabilities right from the CLI, IDEs, Repos, and Pipelines. Snyk integrates seamlessly with AWS offerings like code pipeline, EKS, ECR, and more! As well as things you're actually likely to be using. Deploy on AWS, secure with Snyk. Learn more at Snyk.co/scream That's S-N-Y-K.co/screamCorey: I think it comes down to what you're doing with it. Like, the baseline primitives, the things that no one else is going to be in a position to do because honestly, if I can get logging and audit data out of your control plane, you have a different kind of security problem, and—Anton: [laugh].Corey: That is a giant screaming fire in the building, as it should be. The other side of it, though, is that if we take a look at how much all of this stuff can cost, and if you start charging for things that are competitive to other log analytics tools, great because at that point, we're talking about options. I mean, I'd like to see, in an ideal world, that you don't charge massive amounts of money for egress but ingress is free. I'd like to see that normalized a bit.But yeah, okay, great. Here's the data; now I can run whatever analytics tools I want on it and then you're effectively competing on a level playing field, as opposed to, like, okay, this other analytics tool is better, but it'll cost me over ten times as much to migrate to it, so is it ten times better? Probably not; few things are, so I guess I'm sticking with the stuff that you're offering. It feels like the cloud provider security tools never quite hit the same sweet spot that third-party vendors tend to as far as usability, being able to display things in a way that aligns with various stakeholders at those companies. But it still feels like a cash grab and I have to imagine without having insight into internal costing structures, that the security services themselves are not a significant revenue driver for any of the cloud companies. And the rare times where they are is almost certainly some horrifying misconfiguration that should be fixed.Anton: That's fair, but so to me, it still fits into the bucket of some things you shouldn't charge for and most people don't. There is a bucket of things that you should not charge for, but some people do. And there's a bucket of things where it's absolutely fair to charge for I don't know the amount I'm not a pricing person, but I also seen things that are very clearly have cost to a provider, have value to a client, have margins, so it's very clear it's a product; it's not just a feature of the cloud to be more secure. But you're right if somebody positions as, “I got cloud. Hey, give me secure cloud. It costs double.” I'd be really offended because, like, what is your first cloud is, like, broken and insecure? Yeah. Replace insecure with broken. Why are you selling broken to me?Corey: Right. You tried to spin up a service in Google Cloud, it's like, “Great. Do you want the secure version or the shitty one?”Anton: Yeah, exactly.Corey: Guess which one of those costs more. It's… yeah, in the fullness of time, of course, the shitty one cost more because you find out about security breaches on the front page of The New York Times, and no one's happy, except maybe The Times. But the problem that you hit is that I don't know how to fix that. I think there's an opportunity there for some provider—any provider, please—to be a trendsetter, and, “Yeah, we don't charge for security services on our own stuff just because it'd be believed that should be something that is baked in.” Like, that becomes the narrative of the secure cloud.Anton: What about tiers? What about some kind of a good, better, best, or bronze, gold, platinum, where you have reasonable security, but if you want superior security, you pay money? How do you feel, what's your gut feel on this approach? Like, I can't think of example—log analysis. You're going to get some analytics and you're going to get fancy ML. Fancy ML costs money; yay, nay?Corey: You're bringing up an actually really interesting point because I think I'm conflating too many personas at once. Right now, just pulling up last months bill on Google Cloud, it fits in the free tier, but my Cloud Run bill was 13 cents for the month because that's what runs my snark.cloud URL shortener. And it's great. And I wound up with—I think my virtual machine costs dozen times that much. I don't care.Over in AWS-land, I was building out a serverless nonsense thing, my Last Tweet In AWS client, and that cost a few pennies a month all told, plus a whopping 50 cents for a DNS zone. Whatever. But because I was deploying it to all regions and the way that configural evaluations work, my config bill for that was 16 bucks. Now, I don't actually care about the dollar figures on this. I assure you, you could put zeros on the end of that for days and it doesn't really move the needle on my business until you get to a very certain number there, and then suddenly, I care a lot.Anton: [laugh]. Yeah.Corey: And large enterprises, this is expected because even the sheer cost of people's time to go through these things is valuable. What I'm thinking of is almost a hobby-level side project instead, where I'm a student, and I'm learning this in a dorm room or in a bootcamp or in my off hours, or I'm a career switcher and I'm doing this on my own dime out of hours. And I wind up getting smacked with the bill for security services that, for a company, don't even slightly matter. But for me, they matter, so I'm not going to enable them. And when I transition into the workforce and go somewhere, I'm going to continue to work the same way that I did when I was an independent learner, like, having a wildly generous free tier for small-scale accounts, like, even taking a perspective until you wind up costing, I don't know, five, ten—whatever it is—thousand dollars a month, none of the security stuff is going to be billable for you because it's it is not aimed at you and we want you comfortable with and using these things.This is a whole deep dive into the weeds of economics and price-driven behavior and all kinds of other nonsense, but every time I wind up seeing that, like, in my actual production account over at AWS land for The Duckbill Group, all things wrapped up, it's something like 1100 bucks a month. And over a third of it is monitoring, audit, and observability services, and a few security things as well. And on the one hand, I'm sitting here going, “I don't see that kind of value coming from it.” Now, the day there's an incident and I have to look into this, yeah, it's absolutely going to be worth having, but it's insurance. But it feels like a disproportionate percentage of it. And maybe I'm just sitting here whining and grousing and I sound like a freeloader who doesn't want to pay for things, but it's one of those areas where I would gladly pay more for a just having this be part of the cost and not complain at all about it.Anton: Well, if somebody sells me a thing that costs $1, and then they say, “Want to make it secure?” I say yes, but I'm already suspicious, and they say, “Then it's going to be 16 bucks.” I'd really freak out because, like, there are certain percentages, certain ratios of the actual thing plus security or a secure version of it; 16x is not the answer expect. 30%, probably still not the answer I expect, frankly. I don't know. This is, like, an ROI question [crosstalk 00:23:46]—Corey: Let's also be clear; my usage pattern is really weird. You take a look at most large companies at significant scale, their cloud environments from a billing perspective look an awful lot like a crap ton of instances—or possibly containers running—and smattering of other things. Yeah, you also database and storage being the other two tiers and because of… reasons data transfer loves to show up too, but by and large, everything else was more or less a rounding error. I have remarkably few of those things, just given the weird way that I use services inappropriately, but that is the nature of me, so don't necessarily take that as being gospel. Like, “Oh, you'll spend a third of your bill.”Like, I've talked to analyst types previously—not you, of course—who will hear a story like this and that suddenly winds up as a headline in some report somewhere. And it's, “Yeah, if your entire compute is based on Lambda functions and you get no traffic, yeah, you're going to see some weird distortions in your bill. Welcome to the conversation.” But it's a problem that I think is going to have to be addressed at some point, especially we talked about earlier, those vendors who are catering to customers who are not born in the cloud, and they start to see their business erode as the cloud-native way of doing things continues to accelerate, I feel like we're in for a time where they're going to be coming at the cloud providers and smacking them for this way harder than I am with my, “As a customer, wouldn't it be nice to have this?” They're going to turn this into something monstrous. And that's what it takes, that's what it takes. But… yeah.Anton: It will take more time than than we think, I think because again, back in the Gartner days, I loved to make predictions. And sometimes—I've learned that predictions end up coming true if you're good, but much later.Corey: I'm learning that myself. I'm about two years away from the end of it because three years ago, I said five years from now, nobody will care about Kubernetes. And I didn't mean it was going to go away, but I meant that it would slip below the surface level of awareness to point where most people didn't have to think about it in the same way. And I know it's going to happen because it's too complex now and it's going to be something that just gets handled in the same way that Linux kernels do today, but I think I was aggressive on the timeline. And to be clear, I've been misquoted as, “Oh, I don't think Kubernetes is going to be relevant.”It is, it's just going to not be something that you need to spend the quarter million bucks an engineer on to run in production safely.Anton: Yeah.Corey: So, we'll see. I'm curious. One other question I had for you while I've got you here is you run a podcast of your own: the Cloud Security Podcast if I'm not mistaken, which is—Anton: Sadly, you are not. [laugh].Corey: —the Cloud Se—yeah. Interesting name on that one, yeah. It's like what the Cloud Podcast was taken?Anton: Essentially, we had a really cool name [Weather Insecurity 00:26:14]. But the naming team here said, you must be descriptive as everybody else at Google, and we ended up with the name, Cloud Security Podcast. Very, very original.Corey: Naming is challenging. I still maintain that the company is renamed Alphabet, just so it could appear before Amazon in the yellow pages, but I don't know how accurate that one actually is. Yeah, to be clear, I'm not dunking on your personal fun podcast, for those without context. This is a corporate Google Cloud podcast and if you want to make the argument that I'm punching down by making fun of Google, please, I welcome that debate.Anton: [laugh]. Yes.Corey: I can't acquire companies as a shortcut to hire people. Yet. I'm sure it'll happen someday, but I can aspire to that level of budgetary control. So, what are you up to these days? You spent seven years at Gartner and now you're doing a lot of cloud security… I'll call it storytelling, and I want to be clear that I mean that as a compliment, not the, “Oh, you just tell stories rather than build things?”Anton: [laugh].Corey: Yeah, it turns out that you have to give people a reason to care about what you've built or you don't have your job for very long. What are you talking about these days? What narratives are you looking at going forward?Anton: So, one of the things that I've been obsessed with lately is a lot of people from more traditional companies come in in the cloud with their traditional on-premise knowledge, and they're trying to do cloud the on-premise way. On our podcast, we do dedicate quite some airtime to people who do cloud as if it were a rented data center, and sometimes we say, the opposite is called—we don't say cloud-native, I think; we say you're doing the cloud the cloudy way. So, if you do cloud, the cloudy way, you're probably doing it right. But if you're doing the cloud is rented data center, when you copy a security stack, you lift and shift your IDS, and your network capture devices, and your firewalls, and your SIM, you maybe are okay, as a first step. People here used to be a little bit more enraged about it, but to me, we meet customers where they are, but we need to journey with them.Because if all you do is copy your stack—security stack—from a data center to the cloud, you are losing effectiveness, you're spending money, and you're making other mistakes. I sometimes joke that you copy mistakes, not just practices. Why copy on-prem mistakes to the cloud? So, that's been bugging me quite a bit and I'm trying to tell stories to guide people out of a situation. Not away, but out.Corey: A lot of people don't go for the idea of the lift and shift migration and they say that it's a terrible pattern and it causes all kinds of problems. And they're right. The counterpoint is that it's basically the second-worst approach and everything else seems to tie itself for first place. I don't mean to sound like I'm trying to pick a fight on these things, but we're going to rebuild an application while we move it. Great.Then it doesn't work or worse works intermittently and you have no idea whether it's the rewrite, the cloud provider, or something else you haven't considered. It just sounds like a recipe for disaster.Anton: For sure. And so, imagine that you're moving the app, you're doing cut-and-paste to the cloud of the application, and then you cut-and-paste security, and then you end up with sizeable storage costs, possibly egress costs, possibly mistakes you used to make beyond five firewalls, now you make this mistake straight on the edge. Well, not on the edge edge, but on the edge of the public internet. So, some of the mistakes do become worse when you copy them from the data center to the cloud. So, we do need to, kind of, help people to get out of the situation but not by telling them don't do it because they will do it. We need to tell them what step B; what's step 1.5 out of this?Corey: And cost doesn't drive it and security doesn't drive it. Those are trailing functions. It has to be a capability story. It has to be about improving feature velocity or it does not get done. I have learned this the painful way.Anton: Whatever 10x cost if you do something in the data center-ish way in the cloud, and you're ten times more expensive, cost will drive it.Corey: To an extent, yes. However, the problem is that companies are looking at this from the perspective of okay, we can cut our costs by 90% if we make these changes. Okay, great. It cuts the cloud infrastructure cost that way. What is the engineering time, what is the opportunity cost that they gets baked into that, and what are the other strategic priorities that team has been tasked with this year? It has to go along for the ride with a redesign that unlocks additional capability because a pure cost savings play is something I have almost never found to be an argument that carries the day.There are always exceptions, to be clear, but the general case I found is that when companies get really focused on cost-cutting, rather than expanding into new markets, on some level, it feels like they are not in the best of health, corporately speaking. I mean, there's a reason I'm talking about cost optimization for what I do and not cost-cutting.It's not about lowering the bill to zero at all cost. “Cool. Turn everything off. Your bill drops to zero.” “Oh, you don't have a company anymore? Okay, so there's a constraint. Let's talk more about that.” Companies are optimized to increase revenue as opposed to reduce costs. And engineers are always more expensive than the cloud provider resources they're using, unless you've done something horrifying.Anton: And some people did, by replicating their mistakes for their inefficient data centers straight into the cloud, occasionally, yeah. But you're right, yeah. It costs the—we had the same pattern of Gartner. It's like, it's not about doing cheaper in the cloud.Corey: I really want to thank you for spending so much time talking to me. If people want to learn more about what you're up to, how you view the world, and what you're up to next, where's the best place for them to find you?Anton: At this point, it's probably easiest to find me on Twitter. I was about to say Podcast, I was about to say my Medium blog, but frankly, all of it kind of goes into Twitter at some point. And so, I think I am twitter.com/anton_chuvakin, if I recall correctly. Sorry, I haven't really—Corey: You are indeed. It's always great; it's one of those that you have a sizable audience, and you're like, “What is my Twitter handle, again? That's a good question. I don't know.” And it's your name. Great. Cool. “So, you're going to spell that for you, too, while you're at it?” We will, of course, put a link to that in the [show notes 00:32:09]. I really want to thank you for being so generous with your time. I appreciate it.Anton: Perfect. Thank you. It was fun.Corey: Anton Chuvakin, Security Strategy Something at Google Cloud. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment because people are doing it wrong, but also tell me which legacy vendor you work for.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About BenjaminBenjamin Anderson is CTO, Cloud at EDB, where he is responsible for developing and driving strategy for the company's Postgres-based cloud offerings. Ben brings over ten years' experience building and running distributed database systems in the cloud for multiple startups and large enterprises. Prior to EDB, he served as chief architect of IBM's Cloud Databases organization, built an SRE practice at database startup Cloudant, and founded a Y Combinator-funded hardware startup.Links Referenced: EDB: https://www.enterprisedb.com/ BigAnimal: biganimal.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.I come bearing ill tidings. Developers are responsible for more than ever these days. Not just the code that they write, but also the containers and the cloud infrastructure that their apps run on. Because serverless means it's still somebody's problem. And a big part of that responsibility is app security from code to cloud. And that's where our friend Snyk comes in. Snyk is a frictionless security platform that meets developers where they are - Finding and fixing vulnerabilities right from the CLI, IDEs, Repos, and Pipelines. Snyk integrates seamlessly with AWS offerings like code pipeline, EKS, ECR, and more! As well as things you're actually likely to be using. Deploy on AWS, secure with Snyk. Learn more at Snyk.co/scream That's S-N-Y-K.co/screamCorey: This episode is sponsored by our friends at Fortinet. Fortinet's partnership with AWS is a better-together combination that ensures your workloads on AWS are protected by best-in-class security solutions powered by comprehensive threat intelligence and more than 20 years of cybersecurity experience. Integrations with key AWS services simplify security management, ensure full visibility across environments, and provide broad protection across your workloads and applications. Visit them at AWS re:Inforce to see the latest trends in cybersecurity on July 25-26 at the Boston Convention Center. Just go over to the Fortinet booth and tell them Corey Quinn sent you and watch for the flinch. My thanks again to my friends at Fortinet.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. This promoted guest episode is brought to us by our friends at EDB. And not only do they bring us this promoted episode, they bring me their CTO for Cloud, Benjamin Anderson. Benjamin, thank you so much for agreeing to suffer the slings and arrows that I will no doubt throw at you in a professional context, because EDB is a database company, and I suck at those things.Benjamin: [laugh]. Thanks, Corey. Nice to be here.Corey: Of course. So, databases are an interesting and varied space. I think we can all agree—or agree to disagree—that the best database is, of course, Route 53, when you misuse TXT records as a database. Everything else is generally vying for number two. EDB was—back in the days that I was your customer—was EnterpriseDB, now rebranded as EDB, which is way faster to say, and I approve of that.But you were always the escalation point of last resort. When you're stuck with a really weird and interesting Postgres problem, EDB was where you went because if you folks couldn't solve the problem, it was likely not going to get solved. I always contextualized you folks as a consulting shop. That's not really what you do. You are the CTO for Cloud.And, ah, interesting. Do databases behave differently in cloud environments? Well, they do when you host them as a managed service, which is an area you folks have somewhat recently branched into. How'd you get there?Benjamin: Ah, that's interesting. So, there's a bunch of stuff to unpack there. I think EDB has been around for a long time. It's something like 13, 14, 15 years, something like that, and really it's just been kind of slowly growing, right? We did start very much as a product company. We built some technology to help customers get from Oracle database on to Postgres, way back in 2007, 2008.That business has just slowly been growing. It's been going quite well. Frankly, I only joined about 18 months ago, and it's really cool tech, right? We natively understand some things that Oracle is doing. Customers don't have to change their schemas to migrate from Oracle to Postgres. There's some cool technology in there.But as you point out, I think a lot of our position in the market has not been that product focused. There's been a lot of people seeing us as the Postgres experts, and as people who can solve Postgres problems, in general. We have, for a long time, employed a lot of really sharp Postgres people. We still employ a lot of really sharp Postgres people. That's very much, in a lot of ways, our bread and butter. That we're going to fix Postgres problems as they come up.Now, over the past few years, we've definitely tried to shift quite a bit into being more of a product company. We've brought on a bunch of people who've been doing more enterprise software product type development over the past few years, and really focusing ourselves more and more on building products and investing in ourselves as a product company. We're not a services company. We're not a consulting company. We do, I think, provide the best Postgres support in the market. But it's been a journey. The cloud has been a significant part of that as well, right? You can't get away.Corey: Oh, yeah. These days, when someone's spinning up a new workload, it's unlikely—in most cases—they're going to wind up spinning up a new data center, if they don't already have one. Yes, there's still a whole bunch of on-prem workloads. But increasingly, the default has become cloud. Instead of, “Why cloud?” The question's become, “Why not?”Benjamin: Right, exactly. Then, as people are more and more accepting of managed services, you have to be a product company. You have to be building products in order to support your database customers because what they want his managed services. I was working in managed databases and service, something like, ten years ago, and it was like pulling teeth. This is after RDS launched. This was still pulling teeth trying to get people to think about, oh, I'm going to let you run my database. Whereas, now obviously, it's just completely different. We have to build great products in order to succeed in the database business, in general.Corey: One thing that jumped out at me when you first announced this was the URL is enterprisedb.com. That doesn't exactly speak to, you know, non-large companies, and EDB is what you do. You have a very corporate logo, but your managed service is called BigAnimal, which I absolutely love. It actually expresses a sense of whimsy and personality that I can no doubt guess that a whole bunch of people argued against, but BigAnimal, it is. It won through. I love that. Was that as contentious as I'm painting it to be, or people actually have a sense of humor sometimes?Benjamin: [laugh]. Both, it was extremely contentious. I, frankly, was one of the people who was not in favor of it at first. I was in favor of something that was whimsical, but maybe not quite that whimsical.Corey: Well, I call it Postgres-squeal, so let's be very clear here that we're probably not going to see eye-to-eye on most anything in pronunciation things. But we can set those differences aside and have a conversation.Benjamin: Absolutely, no consider that. It was deliberate, though, to try to step away a little bit from the blue-suit-and-tie, enterprise, DB-type branding. Obviously, a lot of our customers are big enterprises. We're good at that. We're not trying to be the hip, young startup targeting business in a lot of ways. We have a wide range of customers, but we want to branch out a little bit.Corey: One of the challenges right now is if I spin up an environment inside of AWS, as one does, and I decide I certainly don't want to take the traditional approach of running a database on top of an EC2 instance—the way that we did in the olden days—because RDS was crappy. Now that it's slightly less crappy, that becomes a not ideal path. I start looking at their managed database offerings, and there are something like 15 distinct managed databases that they offer, and they never turn anything off. And they continue to launch things into the far future. And it really feels, on some level, like 20 years from now—what we call a DBA today—their primary role is going to look a lot more like helping a company figure out which of Amazon's 40 managed databases is the appropriate fit for this given workload. Yet, when I look around at what the industry has done, it seems that when we're talking about relational databases. Postgres has emerged back when I was, more or less, abusing servers in person in my data center days, it was always MySQL. These days, Postgres is the de facto standard, full stop. I admit that I was mostly keeping my aura away from any data that was irreplaceable at that time. What happened? What did I miss?Benjamin: It's a really good question. And I certainly am not a hundred percent on all the trends that went on there. I know there's a lot of folks that are not happy about the MySQL acquisition by Oracle. I think there's a lot of energy that was adopted by the NoSQL movement, as well. You have people who didn't really care about transactional semantics that were using MySQL because they needed a place to store their data. And then, things like MongoDB and that type of system comes along where it's significantly easier than MySQL, and that subset of the population just sort of drifts away from MySQL.Corey: And in turn, those NoSQL projects eventually turn into something where, okay, now we're trying to build a banking system on top of it, and it's, you know, I guess you can use a torque wrench as a hammer if you're really creative about it, but it seems like there's a better approach.Benjamin: Yeah, exactly. And those folks are coming back around to the relational databases, exactly. At the same time, the advancements in Postgres from the early eight series to today are significant, right? We shouldn't underestimate how much Postgres has really moved forward. It wasn't that long ago that replication was hardly a thing and Postgres, right? It's been a journey.Corey: One thing that your website talks about is that you accelerate your open-sourced database transformation. And this is a bit of a hobby horse I get on from time to time. I think that there are a lot of misunderstandings when people talk about this. You have the open-source purists—of which I shamefully admit I used to be one—saying that, “Oh, it's about the idea of purity and open and free as in software.” Great. Okay, awesome. But when I find that corporate customers are talking about when they say open-source database, they don't particularly care if they have access to the source code because they're not going to go in and patch a database engine, we hope. But what they do care about is regardless of where they are today—even if they're perfectly happy there—they don't want to wind up beholden to a commercial database provider, and/or they don't want to wind up beholden to the environment that is running within. There's a strategic Exodus that's available in theory, which on some level serves to make people feel better about not actually Exodus-ing, but it also means if they're doing a migration at some point, they don't also have to completely redo their entire data plan.Benjamin: Yeah, I think that's a really good point. I mean, I like to talk—there's a big rat's nest of questions and problems in here—but I generally like talk to about open APIs, talk about standards, talk about how much is going to have to change if you eliminate this vendor. We're definitely not open-source purists. Well, we employ a lot of open-source purists. I also used to be an open—Corey: Don't let them hear you say that, then. Fair enough. Fair enough.Benjamin: [laugh] we have proprietary software at EDB, as well. There's a kind of wide range of businesses that we participate in. Glad to hear you also mention this where-it's-hosted angle, as well. I think there's some degree to which people are—they figured out that having at least open APIs or an open-source-ish database is a good idea rather than being beholden to proprietary database. But then, immediately forget that when they're picking a cloud vendor, right? And realizing that putting their data in Cloud Vendor A versus Cloud Vendor B is also putting them in a similar difficult situation. They need to be really wary of when they're doing that. Now, obviously, I work at an independent software company, and I have some incentive to say this, but I do think it's true. And you know, there's meaningful data gravity risk.Corey: I assure you, I have no incentive. I don't care what cloud provider you're on. My guidance has been, for years, to—as a general rule—pick a provider, I care about which one, and go all in until there's a significant reason to switch. Trying to build an optionality, “Oh, everything we do should be fully portable at an instance notice.” Great. Unless you're actually doing it, you're more or less, giving up a whole bunch of shortcuts and feature velocity you could otherwise have, in the hopes of one day you'll do a thing, but all the assumptions you're surrounded by baked themselves in regardless. So, you're more or less just creating extra work for yourself for no defined benefit. This is not popular in some circles, where people try to sell something that requires someone to go multi-cloud, but here we are.Benjamin: No, I think you're right. I think people underestimate the degree to which the abstractions are just not very good, right, and the degree to which those cloud-specific details are going to leak in if you're going to try to get anything done, you end up in kind of a difficult place. What I see more frequently is situations where we have a big enterprise—not even big, even medium-sized companies where maybe they've done an acquisition or two, they've got business units that are trying to do things on their own. And they end up in two or three clouds, sort of by happenstance. It's not like they're trying to do replication live between two clouds, but they've got one business unit in AWS and one business unit and Azure, and somebody in the corporate—say enterprise architect or something like that—really would like to make things consistent between the two so they get a consistent security posture and things like that. So, there are situations where the multi-cloud is a reality at a certain level, but maybe not at a concrete technical level. But I think it's still really useful for a lot of customers.Corey: You position your cloud offering in two different ways. One of them is the idea of BigAnimal, and the other—well, it sort of harkens back to when I was in sixth grade going through the American public school system. They had a cop come in and talk to us and paint to this imaginary story of people trying to push drugs. “Hey, kid. You want to try some of this?” And I'm reading this and it says EDB, Postgres for Kubernetes. And I'm sent back there, where it's like, “Hey, kid. You want to run your stateful databases on top of Kubernetes?” And my default answer to that is good lord, no. What am I missing?Benjamin: That's a good question. Kubernetes has come a long way—I think is part of that.Corey: Oh, truly. I used to think of containers as a pure story for stateless things. And then, of course, I put state into them, and then, everything exploded everywhere because it turns out, I'm bad at computers. Great. And it has come a long way. I have been tracking a lot of that. But it still feels like the idea being that you'd want to have your database endpoints somewhere a lot less, I guess I'll call it fickle, if that makes sense.Benjamin: It's an interesting problem because we are seeing a lot of people who are interested in our Kubernetes-based products. It's actually based on—we recently open-sourced the core of it under a project called cloud-native PG. It's a cool piece of technology. If you think about sort of two by two. In one corner, you've got self-managed on-premise databases. So, you're very, very slow-moving, big-iron type, old-school database deployments. And on the opposite corner, you've got fully-managed, in the cloud, BigAnimal, Amazon RDS, that type of thing. There's a place on that map where you've got customers that want a self-service type experience. Whether that's for production, or maybe it's even for dev tests, something like that. But you don't want to be giving the management capability off to a third party.For folks that want that type of experience, trying to build that themselves by, like, wiring up EC2 instances, or doing something in their own data center with VMware, or something like that, can be extremely difficult. Whereas if you've go to a Kubernetes-based product, you can get that type of self-service experience really easily, right? And customers can get a lot more flexibility out of how they run their databases and operate their databases. And what sort of control they give to, say application developers who want to spin up a new database for a test or for some sort of small microservice, that type of thing. Those types of workloads tend to work really well with this first-party Kubernetes-based offering. I've been doing databases on Kubernetes in managed services for a long time as well. And I don't, frankly, have any concerns about doing it. There are definitely some sharp edges. And if you wanted to do to-scale, you need to really know what you're doing with Kubernetes because the naive thing will shoot you in the foot.Corey: Oh, yes. So, some it feels almost like people want to cosplay working for Google, but they don't want to pass the technical interview along the way. It's a bit of a weird moment for it.Benjamin: Yeah, I would agree.Corey: I have to go back to my own experiences with using RDS back at my last real job before I went down this path. We were migrating from EC2-Classic to VPC. So, you could imagine what dates me reasonably effectively. And the big problem was the database. And the joy that we had was, “Okay, we have to quiesce the application.” So, the database is now quiet, stop writes, take a snapshot, restore that snapshot into the environment. And whenever we talk to AWS folks, it's like, “So, how long is this going to take?” And the answer was, “Guess.” And that was not exactly reassuring. It went off without a hitch because every migration has one problem. We were sideswiped in an Uber on the way home. But that's neither here nor there. This was two o'clock in the morning, and we finished in half the maintenance time we had allotted. But it was the fact that, well, guess we're going to have to take the database down for many hours with no real visibility, and we hope it'll be up by morning. That wasn't great. But that was the big one going on, on an ongoing basis, there were maintenance windows with a database. We just stopped databasing for a period of time during a fairly broad maintenance window. And that led to a whole lot of unfortunate associations in my mind with using relational databases for an awful lot of stuff. How do you handle maintenance windows and upgrading and not tearing down someone's application? Because I have to assume, “Oh, we just never patch anything. It turns out that's way easier,” is in fact, the wrong answer.Benjamin: Yeah, definitely. As you point out, there's a bunch of fundamental limitations here, if we start to talk about how Postgres actually fits together, right? Pretty much everybody in RDS is a little bit weird. The older RDS offerings are a little bit weird in terms of how they do replication. But most folks are using Postgres streaming replication, to do high availability, Postgres in managed services. And honestly, of course—Corey: That winds up failing over, or the application's aware of both endpoints and switches to the other one?Benjamin: Yeah—Corey: Sort of a database pooling connection or some sort of proxy?Benjamin: Right. There's a bunch of subtleties that get into their way. You say, well, did the [vit 00:16:16] failover too early, did the application try to connect and start making requests before the secondaries available? That sort of thing.Corey: Or you misconfigure it and point to the secondary, suddenly, when there's a switchover of some database, suddenly, nothing can write, it can only read, then you cause a massive outage on the weekend?Benjamin: Yeah. Yeah.Corey: That may have been of an actual story I made up.Benjamin: [laugh] yeah, you should use a managed service.Corey: Yeah.Benjamin: So, it's complicated, but even with managed services, you end up in situations where you have downtime, you have maintenance windows. And with Postgres, especially—and other databases as well—especially with Postgres, one of the biggest concerns you have is major version upgrades, right? So, if I want to go from Postgres 12 to 13, 13 to 14, I can't do that live. I can't have a single cluster that is streaming one Postgres version to another Postgres version, right?So, every year, people want to put things off for two years, three years sometimes—which is obviously not to their benefit—you have this maintenance, you have some sort of downtime, where you perform a Postgres upgrade. At EDB, we've got—so this is a big problem, this is a problem for us. We're involved in the Postgres community. We know this is challenging. That's just a well-known thing. Some of the folks that are working EDB are folks who worked on the Postgres logical replication tech, which arrived in Postgres 10. Logical replication is really a nice tool for doing things like change data capture, you can do Walter JSON, all these types of things are based on logical replication tech.It's not really a thing, at least, the code that's in Postgres itself doesn't really support high availability, though. It's not really something that you can use to build a leader-follower type cluster on top of. We have some techs, some proprietary tech within EDB that used to be called bi-directional replication. There used to be an open-source project called bi-directional replication. This is a kind of a descendant of that. It's now called Postgres Distributed, or EDB Postgres Distributed is the product name. And that tech actually allows us—because it's based on logical replication—allows us to do multiple major versions at the same time, right? So, we can upgrade one node in a cluster to Postgres 14, while the other nodes in the clusters are at Postgres 13. We can then upgrade the next node. We can support these types of operations in a kind of wide range of maintenance operations without taking a cluster down from maintenance.So, there's a lot of interesting opportunities here when we start to say, well, let's step back from what your typical assumptions are for Postgres streaming replication. Give ourselves a little bit more freedom by using logical replication instead of physical streaming replication. And then, what type of services, and what type of patterns can we build on top of that, that ultimately help customers build, whether it's faster databases, more highly available databases, so on and so forth.Corey: Let's face it, on-call firefighting at 2am is stressful! So there's good news and there's bad news. The bad news is that you probably can't prevent incidents from happening, but the good news is that incident.io makes incidents less stressful and a lot more valuable. incident.io is a Slack-native incident management platform that allows you to automate incident processes, focus on fixing the issues and learn from incident insights to improve site reliability and fix your vulnerabilities. Try incident.io, recover faster and sleep more.Corey: One approach that I took for, I guess you could call it backup sort of, was intentionally staggering replication between the primary and the replica about 15 minutes or so. So, if I drop a production table or something like that, I have 15 short minutes to realize what has happened and sever the replication before it is now committed to the replica and now I'm living in hell. It felt like this was not, like, option A, B, or C, or the right way to do things. But given that meeting customers where they are as important, is that the sort of thing that you support with BigAnimal, or do you try to talk customers into not being ridiculous?Benjamin: That's not something we support now. It's not actually something that I hear that many asks for these days. It's kind of interesting, that's a pattern that I've run into a lot in the past.Corey: I was an ancient, grumpy sysadmin. Again, I'm dating myself here. These days, I just store everything at DNS text records, and it's way easier. But I digress.Benjamin: [laugh] yeah, it's something that we see a lot for and we had support for a point-in-time restore, like pretty much anybody else in the business at this point. And that's usually the, “I fat-fingered something,” type response. Honestly, I think there's room to be a bit more flexible and room to do some more interesting things. I think RDS is setting a bar and a lot of database services out there and kind of just meeting that bar. And we all kind of need to be pushing a little bit more into more interesting spaces and figuring out how to get customers more value, get customers to get more out of their money for the database, honestly.Corey: One of the problems we tend to see, in the database ecosystem at large, without naming names or companies or anything like that, is that it's a pretty thin and blurry line between database advocate, database evangelist, and database zealot. Where it feels like instead, we're arguing about religion more than actual technical constraints and concerns. So, here's a fun question that hopefully isn't too much of a gotcha. But what sort of workloads would you actively advise someone not to use BigAnimal for in the database world? But yes, again, if you try to run a DNS server, it's probably not fit for purpose without at least a shim in the way there. But what sort of workloads are you not targeting that a customer is likely to have a relatively unfortunate time with?Benjamin: Large-scale analytical workloads is the easy answer to that, right? If you've got a problem where you're choosing between Postgres and Snowflake, you're seriously considering—you actually have as much data that you seriously be considering Snowflake? You probably don't want to be using Postgres, right? You want to be using something that's column, or you want to be using a query planner that really understands a columnar layout that's going to get you the sorts of performance that you need for those analytical workloads. We don't try to touch that space.Corey: Yeah, we're doing some of that right now with just the sheer volume of client AWS bills we have. We don't really need a relational model for a lot of it. And Athena is basically fallen down on the job in some cases, and, “Oh, do you want to use Redshift, that's basically Postgres.” It's like, “Yeah, it's Postgres, if it decided to run on bars of gold.” No, thank you. It just becomes this ridiculously overwrought solution for what feels like it should be a lot similar. So, it's weird, six months ago or so I wouldn't have had much of an idea what you're talking about. I see it a lot better now. Generally, by virtue of trying to do something the precise wrong way that someone should.Benjamin: Right. Yeah, exactly. I think there's interesting room for Postgres to expand here. It's not something that we're actively working on. I'm not aware of a lot happening in the community that Postgres is, for better or worse, extremely extensible, right? And if you see the JSON-supported Postgres, it didn't exist, I don't know, five, six years ago. And now it's incredibly powerful. It's incredibly flexible. And you can do a lot of quote-unquote, schemaless stuff straight in Postgres. Or you look at PostGIS, right, for doing GIS geographical data, right? That's really a fantastic integration directly in the database.Corey: Yeah, before that people start doing ridiculous things almost looks similar to a graph database or a columnar store somehow, and yeah.Benjamin: Yeah, exactly. I think sometimes somebody will do a good column store that's an open-source deeply integrated into Postgres, rather than—Corey: I've seen someone build one on top of S3 bucket with that head, a quarter of a trillion objects in it. Professional advice, don't do that.Benjamin: [laugh]. Unless you're Snowflake. So, I mean, it's something that I'd like to see Postgres expand into. I think that's an interesting space, but not something that, at least especially for BigAnimal, and frankly, for a lot of EDB customers. It's not something we're trying to push people toward.Corey: One thing that I think we are seeing a schism around is the idea that some vendors are one side of it, some are on the other, where on the one side, you have, oh, every workload should have a bespoke, purpose-built database that is exactly for this type of workload. And the other school of thought is you should generally buy us for a general-purpose database until you have a workload that is scaled and significant to a point where running that on its own purpose-built database begins to make sense. I don't necessarily think that is a binary choice, where do you tend to fall on that spectrum?Benjamin: I think everybody should use Postgres. And I say not just because I work in a Postgres company.Corey: Well, let's be clear. Before this, you were at IBM for five years working on a whole bunch of database stuff over there, not just Postgres. And you, so far, have not struck me as the kind of person who's like, “Oh, so what's your favorite database?” “The one that pays me.” We've met people like that, let's be very clear. But you seem very even-handed in those conversations.Benjamin: Yeah, I got my start in databases, actually, with Apache CouchDB. I am a committer on CouchDB. I worked on a managed at CouchDB service ten years ago. At IBM, I worked on something in nine different open-source databases and managed services. But I love having conversations about, like, well, I've got this workload, should I use Postgres, rr should I use Mongo, should I use Cassandra, all of those types of discussions. Frankly, though, I think in a lot of cases people are—they don't understand how much power they're missing out on if they don't choose a relational database. If they don't understand the relational model well enough to understand that they really actually want that. In a lot of cases, people are also just over-optimizing too early, right? It's just going to be much faster for them to get off the ground, get product in customers hands, if they start with something that they don't have to think twice about. And they don't end up with this architecture with 45 different databases, and there's only one guy in the company that knows how to manage the whole thing.Corey: Oh, the same story of picking a cloud provider. It's, “Okay, you hire a team, you're going to build a thing. Which cloud provider do you pick?” Every cloud provider has a whole matrix and sales deck, and the rest. The right answer, of course, is the one your team's already familiar with because learning a new cloud provider while trying not to run out of money at your startup, can't really doesn't work super well.Benjamin: Exactly. Yeah.Corey: One thing that I think has been sort of interesting, and when I saw it, it was one of those, “Oh, I sort of like them.” Because I had that instinctive reaction and I don't think I'm alone in this. As of this recording a couple of weeks ago, you folks received a sizable investment from private equity. And default reaction to that is, “Oh, well, I guess I put a fork in the company, they're done.” Because the narrative is that once private equity takes an investment, well, that company's best days are probably not in front of it. Now, the counterpoint is that this is not the first time private equity has invested in EDB, and you folks from what I can tell are significantly better than you were when I was your customer a decade ago. So clearly, there is something wrong with that mental model. What am I missing?Benjamin: Yeah. Frankly, I don't know. I'm no expert in funding models and all of those sorts of things. I will say that my experience has been what I've seen at EDB, has definitely been that maybe there's private equity, and then there's private equity. We're in this to build better products and become a better product company. We were previously owned by a private equity firm for the past four years or so. And during the course of those four years, we brought on a bunch of folks who were very product-focused, new leadership. We made a significant acquisition of a company called 2ndQuadrant, which they employed a lot of the European best Postgres company. Now, they're part of EDB and most of them have stayed with us. And we built the managed cloud service, right? So, this is a pretty significant—private equity company buying us to invest in the company. I'm optimistic that that's what we're looking at going forward.Corey: I want to be clear as well, I'm not worried about what I normally would be in a private equity story about this, where they're there to save money and cut costs, and, “Do we really need all these database replicas floating around,” and, “These backups, seems like that's something we don't need.” You have, at last count, 32 Postgres contributors, 7 Postgres committers, and 3 core members. All of whom would run away screaming loudly and publicly, in the event that such a thing were taking place. Of all the challenges and concerns I might have about someone running a cloud service in the modern day. I do not have any fear that you folks are not doing what will very clearly be shown to be the right thing by your customers for the technology that you're building on top of. That is not a concern. There are companies I do not have that confidence in, to be clear.Benjamin: Yeah, I'm glad to hear that. I'm a hundred percent on board as well. I work here, but I think we're doing the right thing, and we're going to be doing great stuff going forward.Corey: One last topic I do want to get into a little bit is, on some level, launching in this decade, a cloud-hosted database offering at a time when Amazon—whose product strategy of yes is in full display—it seems like something ridiculous, that is not necessarily well thought out that why would you ever try to do this? Now, I will temper that by the fact that you are clearly succeeding in this direction. You have customers who say nice things about you, and the reviews have been almost universally positive anywhere I can see things. The negative ones are largely complaining about databases, which I admit might be coming from me.Benjamin: Right, it is a crowded space. There's a lot of things happening. Obviously, Amazon, Microsoft, Google are doing great things, both—Corey: Terrible things, but great, yes. Yes.Benjamin: [laugh] right, there's good products coming in. I think AlloyDB is not necessarily a great product. I haven't used it myself yet, but it's an interesting step in the direction. I'm excited to see development happening. But at the end of the day, we're a database company. Our focus is on building great databases and supporting great databases. We're not entering this business to try to take on Amazon from an infrastructure point of view. In fact, the way that we're structuring the product is really to try to get the strengths of both worlds. We want to give customers the ability to get the most out of the AWS or Azure infrastructure that they can, but come to us for their database.Frankly, we know Postgres better than anybody else. We have a greater ability to get bugs fixed in Postgres than anybody else. We've got folks working on the database in the open. We got folks working on the database proprietary for us. So, we give customers things like break/fix support on that database. If there is a bug in Postgres, there's a bug in the tech that sits around Postgres. Because obviously, Postgres is not a batteries-included system, really. We're going to fix that for you. That's part of the contract that we're giving to our customers. And I know a lot of smaller companies maybe haven't been burned by this sort of thing very much. We start to talk about enterprise customers and medium, larger-scale customers, this starts to get really valuable. The ability to have assurance on top of your open-source product. So, I think there's a lot of interesting things there, a lot of value that we can provide there.I think also that I talked a little bit about this earlier, but like the box, this sort of RDS-shaped box, I think is a bit too small. There's an opportunity for smaller players to come in and try to push the boundaries of that. For example, giving customers more support by default to do a good job using their database. We have folks on board that can help consult with customers to say, “No, you shouldn't be designing your schemas that way. You should be designing your schemas this way. You should be using indexes here,” that sort of stuff. That's been part of our business for a long time. Now, with a managed service, we can bake that right into the managed service. And that gives us the ability to kind of make that—you talk about shared responsibility between the service writer and the customer—we can change the boundaries of that shared responsibility a little bit, so that customers can get more value out of the managed database service than they might expect otherwise.Corey: There aren't these harsh separations and clearly defined lines across which nothing shall pass, when it makes sense to do that in a controlled responsible way.Benjamin: Right, exactly. Some of that is because we're a database company, and some of that is because, frankly, we're much smaller.Corey: I'll take it a step further beyond that, as well, that I have seen this pattern evolve a number of times where you have a customer running databases on EC2, and their AWS account managers suggests move to RDS. So, they do. Then, move to Aurora. So, they do. Then, I move this to DynamoDB. At which point, it's like, what do you think your job is here, exactly? Because it seems like every time we move databases, you show up in a nicer car. So, what exactly is the story here, and what are the incentives? Where it just feels like there is a, “Whatever you're doing is not the way that it should be done. So, it's time to do, yet, another migration.”There's something to be said for companies who are focused around a specific aspect of things. Then once that is up and working and running, great. Keep on going. This is fine. As opposed to trying to chase the latest shiny, on some level. I have a big sense of, I guess, affinity for companies that wind up knowing where they start, and most notably, where they stop.Benjamin: Yeah, I think that's a really good point. I don't think that we will be building an application platform anytime soon.Corey: “We're going to run Lambda functions on top of a database.” It's like, “Congratulations. That is the weirdest stored procedure I can imagine this week, but I'm sure we can come up with a worse one soon.”Benjamin: Exactly.Corey: I really want to thank you for taking the time to speak with me so much about how you're thinking about this, and what you've been building over there. If people want to learn more, where's the best place to go to find you?Benjamin: biganimal.com.Corey: Excellent. We will throw a link to that in the show notes and it only just occurred to me that the Postgres mascot is an elephant, and now I understand why it's called BigAnimal. Yeah, that's right. He who laughs last, thinks slowest, and today, that's me. I really want to thank you for being so generous with your time. I appreciate it.Benjamin: Thank you. I really appreciate it.Corey: Benjamin Anderson, CTO for Cloud at EDB. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment that you then wind up stuffing into a SQLite database, converting to Base64, and somehow stuffing into the comment field.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About ChrisChris is the Co-founder and Chief Product Officer at incident.io, where they're building incident management products that people actually want to use. A software engineer by trade, Chris is no stranger to gnarly incidents, having participated (and caused!) them at everything from early stage startups through to enormous IT organizations.Links Referenced: incident.io: https://incident.io Practical Guide to Incident Management: https://incident.io/guide/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: DoorDash had a problem. As their cloud-native environment scaled and developers delivered new features, their monitoring system kept breaking down. In an organization where data is used to make better decisions about technology and about the business, losing observability means the entire company loses their competitive edge. With Chronosphere, DoorDash is no longer losing visibility into their applications suite. The key? Chronosphere is an open-source compatible, scalable, and reliable observability solution that gives the observability lead at DoorDash business, confidence, and peace of mind. Read the full success story at snark.cloud/chronosphere. That's snark.cloud slash C-H-R-O-N-O-S-P-H-E-R-E.Corey: Let's face it, on-call firefighting at 2am is stressful! So there's good news and there's bad news. The bad news is that you probably can't prevent incidents from happening, but the good news is that incident.io makes incidents less stressful and a lot more valuable. incident.io is a Slack-native incident management platform that allows you to automate incident processes, focus on fixing the issues and learn from incident insights to improve site reliability and fix your vulnerabilities. Try incident.io, recover faster and sleep more.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Today's promoted guest is Chris Evans, who's the CPO and co-founder of incident.io. Chris, first, thank you very much for joining me. And I'm going to start with an easy question—well, easy question, hard answer, I think—what is an incident.io exactly?Chris: Incident.io is a software platform that helps entire organizations to respond to recover from and learn from incidents.Corey: When you say incident, that means an awful lot of things. And depending on where you are in the ecosystem in the world, that means different things to different people. For example, oh, incident. Like, “Are you talking about the noodle incident because we had an agreement that we would never speak about that thing again,” style, versus folks who are steeped in DevOps or SRE culture, which is, of course, a fancy way to say those who are sad all the time, usually about computers. What is an incident in the context of what you folks do?Chris: That, I think, is the killer question. I think if you look at organizations in the past, I think incidents were those things that happened once a quarter, maybe once a year, and they were the thing that brought the entirety of your site down because your big central database that was in a data center sort of disappeared. The way that modern companies run means that the definition has to be very, very different. So, most places now rely on distributed systems and there is no, sort of, binary sense of up or down these days. And essentially, in the general case, like, most companies are continually in a sort of state of things being broken all of the time.And so, for us, when we look at what an incident is, it is essentially anything that takes you away from your planned work with a sense of urgency. And that's the sort of the pithy definition that we use there. Generally, that can mean anything—it means different things to different folks, and, like, when we talk to folks, we encourage them to think carefully about what that threshold is, but generally, for us at incident.io, that means basically a single error that is worthwhile investigating that you would stop doing your backlog work for is an incident. And also an entire app being down, that is an incident.So, there's quite a wide range there. But essentially, by sort of having more incidents and lowering that threshold, you suddenly have a heap of benefits, which I can go very deep into and talk for hours about.Corey: It's a deceptively complex question. When I talk to folks about backups, one of the biggest problems in the world of backup and building a DR plan, it's not building the DR plan—though that's no picnic either—it's okay. In the time of cloud, all your planning figures out, okay. Suddenly the site is down, how do we fix it? There are different levels of down and that means different things to different people where, especially the way we build apps today, it's not is the service or site up or down, but with distributed systems, it's how down is it?And oh, we're seeing elevated error rates in us-tire-fire-1 region of AWS. At what point do we begin executing on our disaster plan? Because the worst answer, in some respects is, every time you think you see a problem, you start failing over to other regions and other providers and the rest, and three minutes in, you've irrevocably made the cutover and it's going to take 15 minutes to come back up. And oh, yeah, then your primary site comes back up because whoever unplugged something, plugged it back in and now you've made the wrong choice. Figuring out all the things around the incident, it's not what it once was.When you were running your own blog on a single web server and it's broken, it's pretty easy to say, “Is it up or is it down?” As you scale out, it seems like that gets more and more diffuse. But it feels to me that it's also less of a question of how the technology has scaled, but also how the culture and the people have scaled. When you're the only engineer somewhere, you pretty much have no choice but to have the entire state of your stack shoved into your head. When that becomes 15 or 20 different teams of people, in some cases, it feels like it's almost less than a technology problem than it is a problem of how you communicate and how you get people involved. And the issues in front of the people who are empowered and insightful in a certain area that needs fixing.Chris: A hundred percent. This is, like, a really, really key point, which is that organizations themselves are very complex. And so, you've got this combination of systems getting more and more complicated, more and more sort of things going wrong and perpetually breaking but you've got very, very complicated information structures and communication throughout the whole organization to keep things up and running. The very best orgs are the ones where they can engage the entire, sort of, every corner of the organization when things do go wrong. And lived and breathed this firsthand when various different previous companies, but most recently at Monzo—which is a bank here in the UK—when an incident happened there, like, one of our two physical data center locations went down, the bank wasn't offline. Everything was resilient to that, but that required an immediate response.And that meant that engineers were deployed to go and fix things. But it also meant the customer support folks might be required to get involved because we might be slightly slower processing payments. And it means that risk and compliance folks might need to get involved because they need to be reporting things to regulators. And the list goes on. There's, like, this need for a bunch of different people who almost certainly have never worked together or rarely worked together to come together, land in this sort of like empty space of this incident room or virtual incident room, and figure out how they're going to coordinate their response and get things back on track in the sort of most streamlined way and as quick as possible.Corey: Yeah, when your bank is suddenly offline, that seems like a really inopportune time to be introduced to the database team. It's, “Oh, we have one of those. Wonderful. I feel like you folks are going to come in handy later today.” You want to have those pathways of communication open well in advance of these issues.Chris: A hundred percent. And I think the thing that makes incidents unique is that fact. And I think the solution to that is this sort of consistent, level playing field that you can put everybody on. So, if everybody understands that the way that incidents are dealt with is consistent, we declare it like this, and under these conditions, these things happen. And, you know, if I flag this kind of level of impact, we have to pull in someone else to come and help make a decision.At the core of it, there's this weird kind of duality to incidents where they are both kind of semi-formulaic and that you can basically encode a lot of the processes that happen, but equally, they are incredibly chaotic and require a lot of human impact to be resilient and figure these things out because stuff that you have never seen happen before is happening and failing in ways that you never predicted. And so, this is where incident.io plays into this is that we try to take the first half of that off of your hands, which is, we will help you run your process so that all of the brain capacity you have, it goes on to the bit that humans are uniquely placed to be able to do, which is responding to these very, very chaotic, sort of, surprise events that have happened.Corey: I feel as well—because I played around in this space a bit before I used to run ops teams—and, more or less I really should have had a t-shirt then that said, “I am the root cause,” because yeah, I basically did a lot of self-inflicted outages in various environments because it turns out, I'm not always the best with computers. Imagine that. There are a number of different companies that play in the space that look at some part of the incident lifecycle. And from the outside, first, they all look alike because it's, “Oh, so you're incident.io. I assume you're PagerDuty. You're the thing that calls me at two in the morning to make sure I wake up.”Conversely, for folks who haven't worked deeply in that space, as well, of setting things on fire, what you do sounds like it's highly susceptible to the Hacker News problem. Where, “Wait, so what you do is effectively just getting people to coordinate and talk during an incident? Well, that doesn't sound hard. I could do that in a weekend.” And no, no, you can't.If this were easy, you would not have been in business as long as you have, have the team the size that you do, the customers that you do. But it's one of those things that until you've been in a very specific set of a problem, it doesn't sound like it's a real problem that needs solving.Chris: Yeah, I think that's true. And I think that the Hacker News point is a particularly pertinent one and that someone else, sort of, in an adjacent area launched on Hacker News recently, and the amount of feedback they got around, you know, “You're a Slack bot. How is this a company?” Was kind of staggering. And I think generally where that comes from is—well, first of all that bias that engineers have, which is just everything you look at as an engineer is like, “Yeah, I can build that in a weekend.” I think there's often infinite complexity under the hood that just gets kind of brushed over. But yeah, I think at the core of it, you probably could build a Slack bot in a weekend that creates a channel for you in Slack and allows you to post somewhere that some—Corey: Oh, good. More channels in Slack. Just when everyone wants.Chris: Well, there you go. I mean, that's a particular pertinent one because, like, our tool does do that. And one of the things—so I built at Monzo, a version of incident.io that we used at the company there, and that was something that I built evenings and weekends. And among the many, many things I never got around to building, archiving and cleaning up channels was one of the ones that was always on that list.And so, Monzo did have this problem of littered channels everywhere, I think that sort of like, part of the problem here is, like, it is easy to look at a product like ours and sort of assume it is this sort of friendly Slack bot that helps you orchestrate some very basic commands. And I think when you actually dig into the problems that organizations above a certain size have, they're not solved by Slack bots. They're solved by platforms that help you to encode your processes that otherwise have to live on a Google Doc somewhere which is five pages long and when it's 2 a.m. and everything's on fire, I guarantee you not a single person reads that Google Doc, so your process is as good as not in place at all. That's the beauty of a tool like ours. We have a powerful engine that helps you basically to encode that and take some load off of you.Corey: To be clear, I'm also not coming at this from a position of judging other people. I just look right now at the Slack workspace that we have The Duckbill Group, and we have something like a ten-to-one channel-to-human ratio. And the proliferation of channels is a very real thing. And the problem that I've seen across the board with other things that try to address incident management has always been fanciful at best about what really happens when something breaks. Like, you talk about, oh, here's what happens. Step one: you will pull up the Google Doc, or you will pull up the wiki or the rest, or in some aspirational places, ah, something seems weird, I will go open a ticket in Jira.Meanwhile, here in reality, anyone who's ever worked in these environments knows that step one, “Oh shit, oh shit, oh shit, oh shit, oh shit. What are we going to do?” And all the practices and procedures that often exist, especially in orgs that aren't very practiced at these sorts of things, tend to fly out the window and people are going to do what they're going to do. So, any tool or any platform that winds up addressing that has to accept the reality of meeting people where they are not trying to educate people into different patterns of behavior as such. One of the things I like about your approach is, yeah, it's going to be a lot of conversation in Slack that is a given we can pretend otherwise, but here in reality, that is how work gets communicated, particularly in extremis. And I really appreciate the fact that you are not trying to, like, fight what feels almost like a law of nature at this point.Chris: Yeah, I think there's a few things in that. The first point around the document approach or the clearly defined steps of how an incident works. In my experience, those things have always gone wrong because—Corey: The data center is down, so we're going to the wiki to follow our incident management procedure, which is in the data center just lost power.Chris: Yeah.Corey: There's a dependency problem there, too. [laugh].Chris: Yeah, a hundred percent. [laugh]. A hundred percent. And I think part of the problem that I see there is that very, very often, you've got this situation where the people designing the process are not the people following the process. And so, there's this classic, I've heard it through John Allspaw, but it's a bunch of other folks who talk about the difference between people, you know, at the sharp end or the blunt end of the work.And I think the problem that people are facing the past is you have these people who sit in the, sort of, metaphorical upstairs of the office and think that they make a company safe by defining a process on paper. And they ship the piece of paper and go, “That is a good job for me done. I'm going to leave and know that I've made the bank—the other whatever your organization does—much, much safer.” And I think this is where things fall down because—Corey: I want to ambush some of those people in their performance reviews with, “Cool. Just for fun, all the documentation here, we're going to pull up the analytics to see how often that stuff gets viewed. Oh, nobody ever sees it. Hmm.”Chris: It's frustrating. It's frustrating because that never ever happens, clearly. But the point you made around, like, meeting people where you are, I think that is a huge one, which is incidents are founded on great communication. Like, as I said earlier, this is, like, a form of team with someone you've never ever worked with before and the last thing you want to do is be, like, “Hey, Corey, I've never met you before, but let's jump out onto this other platform somewhere that I've never been or haven't been for weeks and we'll try and figure stuff out over there.” It's like, no, you're going to be communicating—Corey: We use Slack internally, but we have a WhatsApp chat that we wind up using for incident stuff, so go ahead and log into WhatsApp, which you haven't done in 18 months, and join the chat. Yeah, in the dawn of time, in the mists of antiquity, you vaguely remember hearing something about that your first week and then never again. This stuff has to be practiced and it's important to get it right. How do you approach the inherent and often unfortunate reality that incident response and management inherently becomes very different depending upon the specifics of your company or your culture or something like that? In other words, how cookie-cutter is what you have built versus adaptable to different environments it finds itself operating in?Chris: Man, the amount of time we spent as a founding team in the early days deliberating over how opinionated we should be versus how flexible we should be was staggering. The way we like to describe it as we are quite opinionated about how we think incidents should be run, however we let you imprint your own process into that, so putting some color onto that. We expect incidents to have a lead. That is something you cannot get away from. However, you can call the lead whatever makes sense for you at your organization. So, some folks call them an incident commander or a manager or whatever else.Corey: There's overwhelming militarization of these things. Like, oh, yes, we're going to wind up taking a bunch of terms from the military here. It's like, you realize that your entire giant screaming fire is that the lights on the screen are in the wrong pattern. You're trying to make them in the right pattern. No one dies here in most cases, so it feels a little grandiose for some of those terms being tossed around in some cases, but I get it. You've got to make something that is unpleasant and tedious in many respects, a little bit more gripping. I don't envy people. Messaging is hard.Chris: Yeah, it is. And I think if you're overly virtuoustic and inflexible, you're sort of fighting an uphill battle here, right? So, folks are going to want to call things what they want to call things. And you've got people who want to import [ITIL 00:15:04] definitions for severity ease into the platform because that's what they're familiar with. That's fine.What we are opinionated about is that you have some severity levels because absent academic criticism of severity levels, they are a useful mechanism to very coarsely and very quickly assess how bad something is and to take some actions off of it. So yeah, we basically have various points in the product where you can customize and put your own sort of flavor on it, but generally, we have a relatively opinionated end-to-end expectation of how you will run that process.Corey: The thing that I find that annoys me—in some cases—the most is how heavyweight the process is, and it's clearly built by people in an ivory tower somewhere where there's effectively a two-day long postmortem analysis of the incident, and so on and so forth. And okay, great. Your entire site has been blown off the internet, yeah, that probably makes sense. But as soon as you start broadening that to things like okay, an increase in 500 errors on this service for 30 minutes, “Great. Well, we're going to have a two-day postmortem on that.” It's, “Yeah, sure would be nice if we could go two full days without having another incident of that caliber.” So, in other words, whose foot—are we going to hire a new team whose full-time job it is, is to just go ahead and triage and learn from all these incidents? Seems to me like that's sort of throwing wood behind the wrong arrows.Chris: Yeah, I think it's very reductive to suggest that learning only happens in a postmortem process. So, I wrote a blog, actually, not so long ago that is about running postmortems and when it makes sense to do it. And as part of that, I had a sort of a statement that was [laugh] that we haven't run a single postmortem when I wrote this blog at incident.io. Which is probably shocking to many people because we're an incident company, and we talk about this stuff, but we were also a company of five people and when something went wrong, the learning was happening and these things were sort of—we were carving out the time, whether it was called a postmortem, or not to learn and figure out these things. Extrapolating that to bigger companies, there is little value in following processes for the sake of following processes. And so, you could have—Corey: Someone in compliance just wound up spitting their coffee over their desktop as soon as you said that. But I hear you.Chris: Yeah. And it's those same folks who are the ones who care about the document being written, not the process and the learning happening. And I think that's deeply frustrating to me as—Corey: All the plans, of course, assume that people will prioritize the company over their own family for certain kinds of disasters. I love that, too. It's divorced from reality; that's ridiculous, on some level. Speaking of ridiculous things, as you continue to grow and scale, I imagine you integrate with things beyond just Slack. You grab other data sources and over in the fullness of time.For example, I imagine one of your most popular requests from some of your larger customers is to integrate with their HR system in order to figure out who's the last engineer who left, therefore everything immediately their fault because lord knows the best practice is to pillory whoever was the last left because then they're not there to defend themselves anymore and no one's going to get dinged for that irresponsible jackass's decisions, even if they never touched the system at all. I'm being slightly hyperbolic, but only slightly.Chris: Yeah. I think [laugh] that's an interesting point. I am definitely going to raise that feature request for a prefilled root cause category, which is, you know, the value is just that last person who left the organization. That it's a wonderful scapegoat situation there. I like it.To the point around what we do integrate with, I think the thing is actually with incidents that's quite interesting is there is a lot of tooling that exists in this space that does little pockets of useful, valuable things in the shape of incidents. So, you have PagerDuty is this system that does a great job of making people's phone making noise, but that happens, and then you're dropped into this sort of empty void of nothingness and you've got to go and figure out what to do. And then you've got things like Jira where clearly you want to be able to track actions that are coming out of things going wrong in some cases, and that's a great tool for that. And various other things in the middle there. And yeah, our value proposition, if you want to call it that, is to bring those things together in a way that is massively ergonomic during an incident.So, when you're in the middle of an incident, it is really handy to be able to go, “Oh, I have shipped this horrible fix to this thing. It works, but I must remember to undo that.” And we put that at your fingertips in an incident channel from Slack, that you can just log that action, lose that cognitive load that would otherwise be there, move on with fixing the thing. And you have this sort of—I think it's, like, that multiplied by 1000 in incidents that is just what makes it feel delightful. And I cringe a little bit saying that because it's an incident at the end of the day, but genuinely, it feels magical when some things happen that are just like, “Oh, my gosh, you've automatically hooked into my GitHub thing and someone else merged that PR and you've posted that back into the channel for me so I know that that happens. That would otherwise have been a thing where I jump out of the incident to go and figure out what was happening.”Corey: This episode is sponsored in part by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on-premises, private cloud, and they just announced a fully-managed service on AWS and Azure called BigAnimal, all one word. Don't leave managing your database to your cloud vendor because they're too busy launching another half-dozen managed databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications—including Oracle—to the cloud. To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: The problem with the cloud, too, is the first thing that, when there starts to be an incident happening is the number one decision—almost the number one decision point is this my shitty code, something we have just pushed in our stuff, or is it the underlying provider itself? Which is why the AWS status page being slow to update is so maddening. Because those are two completely different paths to go down and you are having to pursue both of them equally at the same time until one can be ruled out. And that is why time to identify at least what side of the universe it's on is so important. That has always been a bit of a tricky challenge.I want to talk a bit about circular dependencies. You target a certain persona of customer, but I'm going to go out on a limb and assume that one explicit company that you are not going to want to do business with in your current iteration is Slack itself because a tool to manage—okay, so our service is down, so we're going to go to Slack to fix it doesn't work when the service is Slack itself. So, that becomes a significant challenge. As you look at this across the board, are you seeing customers having problems where you have circular dependency issues with this? Easy example: Slack is built on top of AWS.When there's an underlying degradation of, huh, suddenly us-east-1 is not doing what it's supposed to be doing, now, Slack is degraded as well, as well as the customer site, it seems like at that point, you're sort of in a bit of tricky positioning as a customer. Counterpoint, when neither Slack nor your site are working, figuring out what caused that issue doesn't seem like it's the biggest stretch of the imagination at that point.Chris: I've spent a lot of my career working in infrastructure, platform-type teams, and I think you can end up tying yourself in knots if you try and over-optimize for, like, avoiding these dependencies. I think it's one of those, sort of, turtles all the way down situations. So yes, Slack are unlikely to become a customer because they are clearly going to want to use our product when they are down.Corey: They reach out, “We'd like to be your customer.” Your response is, “Please don't be.” None of us are going to be happy with this outcome.Chris: Yeah, I mean, the interesting thing that is that we're friends with some folks at Slack, and they believe it or not, they do use Slack to navigate their incidents. They have an internal tool that they have written. And I think this sort of speaks to the point we made earlier, which is that incidents and things failing or not these sort of big binary events. And so—Corey: All of Slack is down is not the only kind of incident that a company like Slack can experience.Chris: I'd go as far as that it's most commonly not that. It's most commonly that you're navigating incidents where it is a degradation, or some edge case, or something else that's happened. And so, like, the pragmatic solution here is not to avoid the circular dependencies, in my view; it's to accept that they exist and make sure you have sensible escape hatches so that when something does go wrong—so a good example, we use incident.io at incident.io to manage incidents that we're having with incident.io. And 99% of the time, that is absolutely fine because we are having some error in some corner of the product or a particular customer is doing something that is a bit curious.And I could count literally on one hand the number of times that we have not been able to use our products to fix our product. And in those cases, we have a fallback which is jump into—Corey: I assume you put a little thought into what happened. “Well, what if our product is down?” “Oh well, I guess we'll never be able to fix it or communicate about it.” It seems like that's the sort of thing that, given what you do, you might have put more than ten seconds of thought into.Chris: We've put a fair amount of thought into it. But at the end of the day, [laugh] it's like if stuff is down, like, what do you need to do? You need to communicate with people. So, jump on a Google Chat, jump on a Slack huddle, whatever else it is we have various different, like, fallbacks in different order. And at the core of it, I think this is the thing is, like, you cannot be prepared for every single thing going wrong, and so what you can be prepared for is to be unprepared and just accept that humans are incredibly good at being resilient, and therefore, all manner of things are going to happen that you've never seen before and I guarantee you will figure them out and fix them, basically.But yeah, I say this; if my SOC 2 auditor is listening, we also do have a very well-defined, like, backup plan in our SOC 2 [laugh] in our policies and processes that is the thing that we will follow that. But yeah.Corey: The fact that you're saying the magic words of SOC 2, yes, exactly. Being in a responsible adult and living up to some baseline compliance obligations is really the sign of a company that's put a little thought into these things. So, as I pull up incident.io—the website, not the company to be clear—and look through what you've written and how you talk about what you're doing, you've avoided what I would almost certainly have not because your tagline front and center on your landing page is, “Manage incidents at scale without leaving Slack.” If someone were to reach out and say, well, we're down all the time, but we're using Microsoft Teams, so I don't know that we can use you, like, the immediate instinctive response that I would have for that to the point where I would put it in the copy is, “Okay, this piece of advice is free. I would posit that you're down all the time because you're the kind of company to use Microsoft Teams.” But that doesn't tend to win a whole lot of friends in various places. In a slightly less sarcastic bent, do you see people reaching out with, “Well, we want to use you because we love what you're doing, but we don't use Slack.”Chris: Yeah. We do. A lot of folks actually. And we will support Teams one day, I think. There is nothing especially unique about the product that means that we are tied to Slack.It is a great way to distribute our product and it sort of aligns with the companies that think in the way that we do in the general case but, like, at the core of what we're building, it's a platform that augments a communication platform to make it much easier to deal with a high-stress, high-pressure situation. And so, in the future, we will support ways for you to connect Microsoft Teams or if Zoom sought out getting rich app experiences, talk on a Zoom and be able to do various things like logging actions and communicating with other systems and things like that. But yeah, for the time being very, very deliberate focus mechanism for us. We're a small company with, like, 30 people now, and so yeah, focusing on that sort of very slim vertical is working well for us.Corey: And it certainly seems to be working to your benefit. Every person I've talked to who is encountered you folks has nothing but good things to say. We have a bunch of folks in common listed on the wall of logos, the social proof eye chart thing of here's people who are using us. And these are serious companies. I mean, your last job before starting incident.io was at Monzo, as you mentioned.You know what you're doing in a regulated, serious sense. I would be, quite honestly, extraordinarily skeptical if your background were significantly different from this because, “Well, yeah, we worked at Twitter for Pets in our three-person SRE team, we can tell you exactly how to go ahead and handle your incidents.” Yeah, there's a certain level of operational maturity that I kind of just based upon the name of the company there; don't think that Twitter for Pets is going to nail. Monzo is a bank. Guess you know what you're talking about, given that you have not, basically, been shut down by an army of regulators. It really does breed an awful lot of confidence.But what's interesting to me is the number of people that we talk to in common are not themselves banks. Some are and they do very serious things, but others are not these highly regulated, command-and-control, top-down companies. You are nimble enough that you can get embedded at those startup-y of startup companies once they hit a certain point of scale and wind up helping them arrive at a better outcome. It's interesting in that you don't normally see a whole lot of tools that wind up being able to speak to both sides of that very broad spectrum—and most things in between—very effectively. But you've somehow managed to thread that needle. Good work.Chris: Thank you. Yeah. What else can I say other than thank you? I think, like, it's a deliberate product positioning that we've gone down to try and be able to support those different use cases. So, I think, at the core of it, we have always tried to maintain the incident.io should be installable and usable in your very first incident without you having to have a very steep learning curve, but there is depth behind it that allows you to support a much more sophisticated incident setup.So, like, I mean, you mentioned Monzo. Like, I just feel incredibly fortunate to have worked at that company. I joined back in 2017 when they were, I don't know, like, 150,000 customers and it was just getting its banking license. And I was there for four years and was able to then see it scale up to 6 million customers and all of the challenges and pain that goes along with that both from building infrastructure on the technical side of things, but from an organizational side of things. And was, like, front-row seat to being able to work with some incredibly smart people and sort of see all these various different pain points.And honestly, it feels a little bit like being in sort of a cheat mode where we get to this import a lot of that knowledge and pain that we felt at Monzo into the product. And that happens to resonate with a bunch of folks. So yeah, I feel like things are sort of coming out quite well at the moment for folks.Corey: The one thing I will say before we wind up calling this an episode is just how grateful I am that I don't have to think about things like this anymore. There's a reason that the problem that I chose to work on of expensive AWS bills being very much a business-hours only style of problem. We're a services company. We don't have production infrastructure that is externally facing. “Oh, no, one of our data analysis tools isn't working internally.”That's an interesting curiosity, but it's not an emergency in the same way that, “Oh, we're an ad network and people are looking at ads right now because we're broken,” is. So, I am grateful that I don't have to think about these things anymore. And also a little wistful because there's so much that you do it would have made dealing with expensive and dangerous outages back in my production years a lot nicer.Chris: Yep. I think that's what a lot of folks are telling us essentially. There's this curious thing with, like, this product didn't exist however many years ago and I think it's sort of been quite emergent in a lot of companies that, you know, as sort of things have moved on, that something needs to exist in this little pocket of space, dealing with incidents in modern companies. So, I'm very pleased that what we're able to build here is sort of working and filling that for folks.Corey: Yeah. I really want to thank you for taking so much time to go through the ethos of what you do, why you do it, and how you do it. If people want to learn more, where's the best place for them to go? Ideally, not during an incident.Chris: Not during an incident, obviously. Handily, the website is the company name. So, incident.io is a great place to go and find out more. We've literally—literally just today, actually—launched our Practical Guide to Incident Management, which is, like, a really full piece of content which, hopefully, will be useful to a bunch of different folks.Corey: Excellent. We will, of course, put a link to that in the [show notes 00:29:52]. I really want to thank you for being so generous with your time. Really appreciate it.Chris: Thanks so much. It's been an absolute pleasure.Corey: Chris Evans, Chief Product Officer and co-founder of incident.io. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this episode, please leave a five-star review on your podcast platform of choice along with an angry comment telling me why your latest incident is all the intern's fault.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About MaishMaish Saidel-Keesing is a Senior Enterprise Developer Advocate @AWS working on containers and has been working in IT for the past 20 years and with a stronger focus on cloud and automation for the past 7.He has extensive experience with AWS Cloud technologies, DevOps and Agile practices and implementations, containers, Kubernetes, virtualization and, and a number of fun things he has done along the wayHe is constantly trying to bridge the gap between Developers and Operators to allow all of us provide a better service for our customers (and not wake up from pages in the middle of the night). He is an avid practitioner of dissolving silos - educating Ops how to code and explaining to Devs what the hell is OperationsLinks Referenced: @maishsk: https://twitter.com/maishsk duckbillgroup.com: https://duckbillgroup.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored by our friends at Fortinet. Fortinet's partnership with AWS is a better-together combination that ensures your workloads on AWS are protected by best-in-class security solutions powered by comprehensive threat intelligence and more than 20 years of cybersecurity experience. Integrations with key AWS services simplify security management, ensure full visibility across environments, and provide broad protection across your workloads and applications. Visit them at AWS re:Inforce to see the latest trends in cybersecurity on July 25-26 at the Boston Convention Center. Just go over to the Fortinet booth and tell them Corey Quinn sent you and watch for the flinch. My thanks again to my friends at Fortinet.Corey: Let's face it, on-call firefighting at 2am is stressful! So there's good news and there's bad news. The bad news is that you probably can't prevent incidents from happening, but the good news is that incident.io makes incidents less stressful and a lot more valuable. incident.io is a Slack-native incident management platform that allows you to automate incident processes, focus on fixing the issues and learn from incident insights to improve site reliability and fix your vulnerabilities. Try incident.io, recover faster and sleep more.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I'm a cloud economist at The Duckbill Group, and that was a fun thing for me to become because when you're starting to set out to solve a problem, well, what do you call yourself? I find that if you create a job title for yourself, well, no one knows quite how to categorize you and it leads to really interesting outcomes as a result. My guest today did something very similar. Maish Saidel-Keesing is an EntReloper, or Enterprise Developer Advocate, specifically for container services at AWS. Maish, thank you for joining me.Maish: Thank you for having me on the show, Corey. It's great to be here.Corey: So, how did you wind up taking a whole bunch of words such as enterprise, developer, advocate because I feel like the way you really express seniority at big companies, almost as a display of dominance, is to have additional words in your job title, which all those words are very enterprise-y, very business-y, and very serious. And in container services to boot, which is a somewhat interesting culture, just looking at the enterprise adoption of the pattern. And then at AWS, whose entire sense of humor can be distilled down into, “That's not funny.” You have the flexibility to refer to yourself as an EntReloper in public. I love it. Is it just something you started doing? Was there, like, 18 forms of approval you had to go through to do it? How did this happen? I love it.Maish: So, no. I didn't have to go through approval, of course. Same way, you didn't call yourself a cloud economist with anybody else's approval. But I got the idea mostly from you because I love your term of coining everybody who's in developer advocacy or developer relations as a DevReloper. And specifically, the reason that I coined the term of an EntReloper—and actually looked it up on Google to see if anybody had actually used that term before, and no they haven't—it's the fact of I came into the position on the premise of trying to bring the enterprise voice of the customer into developer advocacy.When we speak about developer advocates today, most of them are the people who are the small startups, developers who write the code, and we kind of forget that there is a whole big world outside of, besides small startups, which are these big, massive, behemoth sort of enterprise companies who kind of do things differently because they've been around for many, many years; they have many, many silos inside their organizations. And it's not the most simple thing to open up your laptop, and install whatever software you want on, because some of these people don't even give you admin rights on your laptop, or you're allowed to ssh out to a computer in the cloud because also the same thing: everything is blocked by corporate firewalls where you have to put in a ticket in order to get access to the outside world. I worked in companies like that when I was—before I moved to Amazon. So, I want to bring that perspective to the table on behalf of our customers.Corey: Bias is a very funny thing. I spent the overwhelming majority of my career in small environments like you describe. To me a big company is one that has 200 people there, and it turns out that there's a whole ‘nother sense of scale that goes beyond that. And there's, like, 18 different tiers beyond. But I still bias based upon my own experiences when I talk about how I do things and how I think about things to a certain persona that closely resembles my own experiences where, “Just install this thing as a tool and it'll be great,” ignoring entirely, the very realistic fact that you've got an entire universe out there of people who are not empowered to install things on their own laptop, for example.How is developer advocacy different within enterprises than in the common case of, “We're a startup. We're going to change the world with our amazing SaaS.” Great, maybe you will. Statistically, you won't. But enterprises have different concerns, different challenges, and absolutely a different sense of scale. How is the practice of advocacy different in those environments?Maish: So, I think the fact is, mostly working on standardization from the get-go that these big enterprises want things to work in a standard way where they can control it, they can monitor it, they can log everything, they can secure it mostly, of course, the most important thing. But it's also the fact that as a developer advocate, you don't always talk to developers within the enterprise. You also have to talk to the security team and to the network team and to the business itself or the C-level to understand. And as you also probably have found out as well in your job, you connect the people with inside the business one to another, these different groups, and get them talking to each other to make these decisions together. So, we act as kind of a bridge in between the people with inside their own company where they don't really talk to each other, or don't have the right connections, or the right conduit in order for them to start that conversation and make things better for themselves.Corey: On some level, my line about developer relations, developer advocacy, has generally taken the track of, “What does that mean? Well, it means you work in marketing, but they're scared to tell you that.” Do you view what you do as being within marketing, aligned with marketing, subtly different and I'm completely wrong, et cetera, et cetera? All positions are legitimate, by the way.Maish: So, I think, at the position that I'm currently in, which is a developer advocate but for the service team, is slightly different than a marketing developer advocate. The marketing developer advocate—and we have many of them which are amazing people and doing amazing work within AWS—their job is to teach everybody about the services and the capabilities available within AWS. That is also part of my job, but I would think that is the 40% of my job. I also go on stage, I go on podcasts like this, I present at conferences, I write blog posts. I also do the kind of marketing work as well.But the other 60% of my job as a service developer advocate is to seek out the feedback, or the signals, or the sentiment from our customers, and bring that back into the service teams, into the product management, into the engineering teams. And, as I said, sit as the enterprise customer in the chair in those meetings, to voice their concerns… their opinions, how they would like the products to go, how we can make the products better. So, the 60% is mostly what we call inbound, which is taking feedback from our customers back into the service teams directly in order to have some influence on the roadmap. And 40% is the outbound work, which we do, as I said, conferences, blog posts, and things like this.Corey: I have a perception. And I am thrilled to be corrected on this because it's not backed by data; it's backed by my own biases—and some people tend to conflate the two; I strive not to—that there's a—I think the term that I heard bandied around at one point was ‘the dark matter developers.' These are folks that primarily work in .NET or Java. They work for companies that are not themselves tech companies, but rather tech is a supporting function, usually in a central IT-style organization, that supports what the business actually does, and they generally are not visible to a lot of traditional developer advocacy approaches.They, by and large, don't go to conferences, they don't go on Twitter to yell at people about things, they commit the terrible sin—according to many startup folks—of daring to view the craft of writing software as this artistic thing, and they just view it as a job and a thing to make money for—filthy casuals—as opposed to this higher calling that's changing the world. Which I think is wild take. But there are a tremendous number of people out there who do fit the profile of they show up, they do their jobs working on this stuff, they don't go to conferences, they don't go out into the community, and they just do their job and go home. The end. Is that an accurate perception? Are there large swaths of folks like that in the industry, and if so, do they centralize or congregate more around enterprises than they do around smaller companies?Maish: I think that your perception is correct. Specifically, for my experience, when I worked, for example, my first two years before I was a developer advocate, I was an enterprise solutions architect which I worked with financial institutions, which are banks, which usually have software which are older than me, which are written in languages, which are older than I am. So, there are people which, as they say, they come there to—they do their job. They're not interested in looking at Twitter, or writing blog posts, or participating in any kind of thing which is outgoing. And they just, they're there to write the code. They go home at the end of the day.They also usually don't have pagers that page them in middle of the night because that's what you have operations teams for, not developers because they're completely different entities. So, I do think your perception might be correct, yes. There are people like that when you say, these dark matter people, dark matter developers.Corey: And I don't have any particular problem. I'm not here to cast shade on anything that they're doing, to be very clear.Maish: Not at all.Corey: Everyone makes different choices and that's great. I don't think necessarily everyone should have a job that is all-consuming, that eats them alive. I wish I didn't, some days. [laugh]. The challenge I have for you then is, as an EntReloper, how do you reach folks in positions like that? Or don't you?Maish: I think the way to reach those people is to firstly, expose them to technology, expose them to the capabilities that they can use in AWS in the cloud, specifically with my position in container services, and gain their trust because that's one of the LPs in Amazon itself: customer obsession. And we work consistently in order to—with our customers to gain their trust and help them along their journey, whatever it may be. If it might be the fact, okay, I only want to write software for nine to five and go home and do everything afterwards, which most normal people do without having to worry about work, or they still want to continue working and adopt the full model of you build it, you own it; manage everything in production on their own and go into the new world of modern software, which many enterprises, unfortunately, are not all the way there yet, but hopefully, they will get there sooner than later.Corey: There's a misguided perception in many corners that you have to be able to reach everyone at all times; wherever they are, you have to be able to go there. I don't think that's true. I think that showing up and badgering people who are just trying to get a job done into, “Hey, have you heard the good word of cloud?” It's like, evangelists knocking on your door at seven o'clock in the morning on a weekend and you're trying to sleep in because the kids are somewhere else for the week. Yeah, I might be projecting a little bit on that.I think that is the wrong direction to go. And I find that being able and willing to meet people where they are is key to success on this. I'm also a big believer in the idea that in any kind of developer advocacy role, regardless whether their targets are large, small, or in my case, patently ridiculous because my company is in fact ridiculous in some ways, you have to meet them where they are. There's no choice around that. Do you find that there are very different concerns that you have to wind up addressing with your audience versus a more, “Mainstream,” quote-unquote, developer advocacy role?Maish: For the enterprise audience, they need to, I would say, relate to what we're talking to. For an example, I gave a talk a couple of weeks ago on the AWS Summit here in Tel Aviv, of how to use App Runner. So, instead of explaining to the audiences how you use the console, this is what it does, you can deploy here, this is how the deployments work, blue, green, et cetera, et cetera, I made up an imaginary company and told the story of how the three people in the startup of this company would start working using App Runner in order to make the thing more relatable, something which people can hopefully remember and understand, okay, this is something which I would do as a startup, or this is what my project, which I'm doing or starting to work on, something I can use. So, to answer your question, in two words, tell stories instead of demo products.Corey: It feels like that's a… heavy lift, in many cases, because I guess it's also partially a perception issue on my part, where I'm looking at this across the board, where I see a company that has 5000 developers working there and, like, how do you wind up getting them to adopt cloud, or adopt new practices, or change anything? It feels like it's a Herculean, impossible task. But in practice, I feel like you don't try and do all of that at once. You start with small teams, you start with specific projects, and move on. Is that directionally accurate?Maish: Completely accurate. There's no way to move a huge mothership in one direction at one time. You have to do, as you say, start small, find the projects, which are going to bring value to the company or the business, and start small with those projects and those small teams, and continue that education within the organization and help the people with your teaching or introducing them to the cloud, to help others within inside their own organization. Make them, or enable them, or empower them to become leaders within their own organization. That's what I tried to do, at least.Corey: You and I have a somewhat similar background, which is weird given that we've just spent a fair bit of time talking about how different our upbringings were in tech at scales of companies and whatnot, but we're alike in that we are both fairly crusty, old operations-side folks, sysadmins—Maish: [laugh]. Yep.Corey: —grumpy people.Maish: Grumpy old sysadmins. Yeah, exactly.Corey: Exactly. Because do you ever notice there's never a happy one? Imagine that. And DevOps was always a meeting of the development and operations, meaning everyone's unhappy. And there's a school of thought that—like, I used to think that, “Oh, this is just what we call sysadmins once they want a better title and more money, but it's still the same job.”But then I started meeting a bunch of DevOps types who had come from the exact opposite of our background, where they were software developers and then they wound up having to learn not so much how the code stuff works the way that we did, but rather how systems work, how infrastructure works. Compare and contrast those for me. Who makes, I guess, the more successful DevOps engineer when you look at it through that lens?Maish: So, I might be crucified for this on the social media from a number of people from the other side of the fence, but I have the firm belief that the people who make the best DevOps engineers—and I hate that term—but people who move DevOps initiatives or changes or transformations with organization is actually the operations people because they usually have a broader perspective of what is going on around them besides writing code. Too many times in my career, I've been burned by DNS, by a network cable, by a power outage, by somebody making a misconfiguration in the Puppet module, or whatever it might have been, somebody wrote it to deploy to 15,000 machines, whatever it may be. These are things where developers, at least my perception of what developers have been doing up until now, don't really do that. In a previous organization I used to work for, the fact was, there was a very, very clear delineation about between the operations people, and the developers who wrote the software. We had very hard times getting them into rotations for on-call, we had very hard times educating them about the fact that not every single log line has to be written to the log because it doesn't interest anybody.But from developer perspective, of course, we need that log because we need to know what's happening in the end. But there are 15, different thousand… turtles all the way down, which have implications about the number of log lines which are written into a piece of software. So, I am very much of the belief that the people that make the best DevOps engineers—if we can use that term still today—are actually people which come from an operations background because it's easier to teach them how to write code or become a programmer than the other way round of teaching a developer how to become an operations person. So, the change or the move from one direction from operations to adding the additional toil of writing software is much easier to accomplish than the other way around, from a developer learning how to run infrastructure at scale.Corey: This episode is sponsored in part by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on-premises, private cloud, and they just announced a fully-managed service on AWS and Azure called BigAnimal, all one word. Don't leave managing your database to your cloud vendor because they're too busy launching another half-dozen managed databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications—including Oracle—to the cloud. To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: I once believed much the same because—and it made sense coming from the background that I was in. Everyone intellectually knows that if you're having trouble with a piece of equipment, have you made sure that it's plugged in? Yes, everyone knows that intellectually. But there's something about having worked on a thing for three hours that wasn't working and only discover it wasn't plugged in, that really sears that lesson into your bones. The most confidence-inspiring thing you can ever hear from someone an operations role is, “Oh, I've seen this problem before. Here's how we fixed it.”It feels like there are no junior DevOps engineers, for lack of a better term. And for a long time, I believed that the upcoming and operational side of the world were in fact, the better DevOps types. And in the fullness of time, I think a lot of that—at least my position on it—was rooted in some level of insecurity because I didn't know how to write code and the thing that I saw happening was my job that I had done historically was eroding. Today, I don't know that it's possible to be in the operation space and not be at least basically conversant with how code works. There's a reason most of these job interviews turn into algorithm hazing.And my articulation of it was rooted, for me at least—at least in a small way—in a sense of defensiveness and wanting to validate the thing that I had done with my career that I defined myself by, I was under threat. And obviously, the thing that I do is the best thing because otherwise it's almost a tacit admission that I made poor career choices at some point. And I don't think that's true, either. But for me, at least psychologically, it was very much centered in that. And honestly, I found that the right answer for me was, in fact, neither of those two things because I have met a couple of people in my life that I would consider to be full-stack engineers.And there's a colloquialism these days, that means oh, you do front-end and back-end. Yeah. The people I'm thinking of did front-end, they did back-end, they did mobile software, they did C-level programming, they wrote their own freaking device drivers at one point. Like, they have done basically everything. And they were the sort of person you could throw any technical issue whatsoever at and get out of their way because it was going to get solved. Those people are, as it turns out, the best. Like, who does a better job developer or operations, folks? Yes. Specifically, both of those things together.Maish: Exactly.Corey: And I think that is a hard thing to talk about. I think that it's a hard—it's certainly a hard thing to find. It turns out that there's a reason that I only know two or three of those folks in the course of my entire career. They're out there, but they're really, really hard to track down.Maish: I completely agree.Corey: A challenge that I hear articulated in some cases—and while we're saying things that are going to get us yelled out on social media, let's go for the fences on this one—a concern that comes up when talking about enterprises moving to cloud is that they have a bunch of existing sysadmin types—while we're on the topic—and well, those people need to learn to work within cloud. And the reality is, in many cases that first, that's a whole new skill set that not everyone is going to be willing or able to pick up. For those who can they have just found that their market rate has effectively doubled. And that seems, on some level, to pose a significant challenge to companies undergoing this, and the larger the company, the more significant the challenge.Because it's my belief that you pay market rate for the talent you have whether you want to or not. And if companies don't increase compensation, these people will leave for things that double their income. And if they raise compensation internally, good for them, but that does have a massive drag on their budget that may not have been accounted for in a lot of the TCO analyses. How do you find that the companies you talk to wind up squaring that circle?Maish: I don't think I have a correct answer for that. I do completely agree—Corey: Oh, I'm not convinced there's a correct answer at all. I'm just trying [laugh] to figure out how to even think about it.Maish: I… have seen this as well in companies which I used to work for and companies that were customers that I have also worked with as part of my tenure in AWS. It's the fact of, when companies are trying to move to the cloud and they start upskilling their people, there's always the concern in the back of their mind of the fact, “Okay, I'm now training this person with new technology. I'm investing time, I'm investing money. And why would I do this if I know that, for example, as soon as I finish this, I'm going to have to just say, I have to pay them more because they can go somewhere else and get the same job with a better pay? So, why would we invest amount of time and resources into upscaling the people?”And these are questions which I have received and conversations which I've had with customers many times over the last two, three years. And the answer, from my perspective always, is the fact is because, number one, you're making the world a better place. Number two, you're making your employees feel more appreciated, giving them better knowledge. And if you're afraid of the fact of teaching somebody to become better is going to have negative effects on your organization then, unfortunately, you deserve to have that person leave and let them find a better job because you're not taking good care of your people. And it's sometimes hard for companies to hear that.Sometimes we get, “You know what? You're completely right.” Sometimes I don't agree with you because I need to compete there, get to the bottom line, and make sure that I stay within my budget or my TCO. But the most important thing is to have the conversation, let people hear different ideas, see how it can benefit them, not only by giving people more options to maybe leave the company, but it can actually make their whole organization a lot better in the long run.Corey: I think that you have to do right by people because reputations last a long time. Even at big companies it becomes a very slow thing to change and almost impossible to do in the short term. So, people tell stories when they feel wronged. That becomes a problem. I do want to pivot a little bit because you're not merely an EntReloper; you are an EntReloper specifically focusing on container services.Maish: Correct.Corey: Increasingly, I am viewing containers as what amounts to effectively a packaging format. That is the framework through which I am increasingly seeing. How are you seeing customers use containers? Is that directionally correct? Is it completely moonbat stuff compared to what you're seeing in the wild, or something else?Maish: I don't think it's a packaging format; I think it's more as an accelerator to enable the customers to develop in a more modern way with using twelve-factor apps with modern technology and not necessarily have their own huge, sticky, big monolith of whatever it might be, written in C# or whatever, or C++ whatever it may be, as they've been using up until now, but they now have the option and the technology and the background in order to split it up into smaller services and develop in the way that most of the modern world—or at least, the what we perceive as the modern world—is developing and creating applications today.Corey: I feel like on some level, containers were a radical change to how companies envisioned software. They definitely provide a path of modernizing things that were very tied to hardware previously. It let some companies even just leapfrog the virtualization migration that they'd been considering doing. But, on some level, I also feel like it runs counter to the ideas of DevOps, where you have development and operations working in partnership, where now it's like, welp, inside the container is a development thing and outside the container, ops problem now. It feels almost, on some level, like, it reinforces a wall. But in a lot of cultures and a lot of companies, that wall is there and there's no getting rid of it anytime soon. So, I confess that I'm conflicted on that.Maish: I think you might be right, and it depends, of course, on the company and the company culture, but what I think that companies need to do is understand that there will never be one hundred percent of people writing software that want to know one hundred percent of how the underlying infrastructure works. And the opposite direction as well: that there will never be people which maintain infrastructure and understand how computers and CPUs and memory buses and NUMA works on motherboards, that they don't need to know how to write the most beautiful enticing and wonderful software for programs, for the world. There's always going to have to be a compromise of who's going to be doing this or who's going to be doing that, and how comfortable they are with taking at least part of the responsibility of the other side into their own realm of what they should be doing. So, there's going to be a compromise on both sides, but there is some kind of divide today of separating, okay, you just write the Helm chart for your Kubernetes Pod spec, or your ECS task, or whatever task definition, whatever you would like. And don't worry about the things in the background because they're just going to magically happen in the end. But they do have to understand exactly what is happening at the background in the end because if something goes wrong, and of course, something will go wrong, eventually, one day somewhere, somehow, they're going to have to know how to take care of it.Corey: I really want to thank you for taking the time to speak with me today about, well, I guess a wide ranging variety of topics, some of which will absolutely inspire people to take to their feet—or at least their Twitter accounts—and tell us, “You know what your problem is?” And I honestly live for that. If you don't evoke that kind of reaction on some level, have you ever really had an opinion in the first place? So, I'm looking forward to that. If people want to learn more about you, your beliefs, call set beliefs misguided, et cetera, et cetera, where's the best place to find you?Maish: So, I'm on Twitter under @maishsk. I assume that will be in the [show notes 00:26:31]. I pontificate some time on technology, on cooking every now and again, on Friday before the end of the weekend, a little bit of politics, but you can find me @maishsk on Twitter. Or maishsk everywhere else social that's possible.Corey: Excellent. We will toss links to that, of course, in the [show notes 00:26:50]. Thank you so much for being so generous with your time. I appreciate it.Maish: Thank you very much, Corey. It was fun.Corey: Maish Saidel-Keesing, EntReloper of container services at AWS. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated it, please leave a five-star review on your podcast platform of choice along with an angry comment that your 5000 enterprise developer colleagues can all pile on.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
Full Description / Show Notes Marie talks about Oki Doki's primary product, Notion Mastery (2:38) Corey and Marie talk ADHD diagnosis and how it has impacted their lives and work (4:26) Marie and Corey discuss techniques they've developed for coping with ADHD (11:22) Corey and Marie talk about workarounds for people with ADHD who want to adopt something like Notion (16:13) Marie discusses the importance of being excited about the tools you're employing (18:54) Corey and Marie talk about finding tools that work for you (26:43) Marie and Corey discuss the unique challenge of teaching skills versus dumping knowledge (30:35) About Marie PoulinMarie teaches business owners to level up their digital systems, workflow, and knowledge management processes using Notion.She's the co-founder of Oki Doki and creator of Notion Mastery, an online program and community that helps creators, entrepreneurs and small teams tame their work + life chaos by building life and business management systems with Notion.Diagnosed with ADHD at age 37, Marie is especially passionate about helping folks customize their workflows and workspaces to meet their unique needs and preferences.She believes that Notion is especially powerful for neurodivergent folks who have long struggled to adhere to traditional or rigid project management processes, and may need a little extra customization and flexibility.When she's not tinkering in Notion or doing live trainings, you can find her in the garden, playing video games, or cooking up some delicious vegetarian tacos.Links Referenced: Oki Doki: https://weareokidoki.com/ Personal website: https://mariepoulin.com Notion Mastery: https://notionmastery.com Twitter: https://twitter.com/mariepoulin TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by Honeycomb. When production is running slow, it's hard to know where problems originate. Is it your application code, users, or the underlying systems? I've got five bucks on DNS, personally. Why scroll through endless dashboards while dealing with alert floods, going from tool to tool to tool that you employ, guessing at which puzzle pieces matter? Context switching and tool sprawl are slowly killing both your team and your business. You should care more about one of those than the other; which one is up to you. Drop the separate pillars and enter a world of getting one unified understanding of the one thing driving your business: production. With Honeycomb, you guess less and know more. Try it for free at honeycomb.io/screaminginthecloud. Observability: it's more than just hipster monitoring.Corey: DoorDash had a problem. As their cloud-native environment scaled and developers delivered new features, their monitoring system kept breaking down. In an organization where data is used to make better decisions about technology and about the business, losing observability means the entire company loses their competitive edge. With Chronosphere, DoorDash is no longer losing visibility into their applications suite. The key? Chronosphere is an open-source compatible, scalable, and reliable observability solution that gives the observability lead at DoorDash business, confidence, and peace of mind. Read the full success story at snark.cloud/chronosphere. that's snark.cloud slash C-H-R-O-N-O-S-P-H-E-R-E.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Today I'm joined by Marie Poulin, the CEO of Oki Doki. Marie, thank you for joining me.Marie: Thank you for having me. I'm excited.Corey: So, let's start at the very beginning. What does Oki Doki do? And for folks listening that is O-K-I D-O-K-I, so you might want to have to think about that if you're doing the Google approach of, “What is this thing?”Marie: Well, at the moment, the majority of our products and services are surrounded by helping people learn how to use Notion to manage their life and business. So, it's only a pivot that we took in the last couple of years, and so our signature program is a course called Notion Mastery. So, there's four full-time employees now and that's what we do. We design live trainings, we have a forum, we have a curriculum. It's all products and services related to Notion.Corey: That is an interesting pivot that you can wind up going through. Please tell me I'm not the first person to make the observation that you called it Oki Doki and you've turned yourself around.Marie: [laugh]. You are the first, Corey? [laugh].Corey: Oh, good. I am broken like that, so that's kind of awesome. So, you've been more or less doing—I don't know the best way to frame this, so my apologies if I'm getting it wrong—but the idea of well, what are you selling? Knowledge. You're selling an understanding of how to improve things, you're selling a better outcome.And it's easy to look at that and say, “Oh, you're selling education.” No, you're selling understanding. Education is the way that you get there because at least at the moment, you can't just jack gigabytes of data directly into people's head without going to prison for it. Or raising a whole boatload of VC money.Marie: [laugh]. I mean, you can also say you're kind of selling an outcome, right? You're selling this future version of who someone wants to be. And so, we talk a lot about—you know, on our sales page, we get a lot of compliments on our sales page, but just speaking to the scattered mind, you know, feeling like a shitshow, feeling like you don't really have all your data in one place. You know, it's learning how to improve your workflow at work but also in life as well.And so, a lot of our language speaks to the sort of future version of yourself. Like, stop feeling scattered, stop feeling stretched thin. Let's actually get it so that you turn things into a well-oiled machine. So, you could say we're selling a dream. [laugh].Corey: This is an interesting direction to take this conversation in because I don't normally talk about this. But why not; we'll give it a shot. It's been sufficiently long since the last time. Last year—you've been very public about this—you were diagnosed with ADHD. I periodically talk about the fact that I was diagnosed with it myself—back when it was called ADD—when I was five years old.So, growing up I always knew that there was something neurodivergent about me. And the lesson I took away from this, as someone growing up with a lot of the limitations—yes, there are advantages but at the time, all I saw were limitations—about, “Well, what is ADHD?” It's like, oh, okay. They sat down and explained it to me. And it's not what they said, but it was, “See, this is the medical reason why you suck.”And that was not the most constructive way of framing it. In adulthood, talking to other people who have been diagnosed with this, especially later in life. There's a—it's a spectrum disorder. It winds up impacting an awful lot of people differently, but the universal experience that I hear is, wait, you mean there's a reason that I am the way that I am? It's not that I'm lazy. It's not that I'm shitty at things. It's not that I'm—Marie: Yeah.Corey: —careless. And that is one of those things that just is transformative. I didn't realize at the time how fortunate I was to be diagnosed that early on because trying to try to figure out why am I getting fired all the time? Why do I get bored doing the same thing too many days in a row, so I start causing problems for other people? What is going on with this? Why do I have this incredible opposition to anything that remotely resembles authority, et cetera, et cetera?Not all of this might be ADHD traits, but here I am. And my only solution after, you know, deciding that I didn't really want to set a world record for number of times getting fired was, well, I guess I'll start my own company because that at least to get fired, it's going to take some work. You figured this out while you were already self-employed.Marie: Yes.Corey: What was that like?Marie: What was it like to find out that I finally had an answer or reason for, maybe, past behaviors? [laugh].Corey: Right. Because it's the simultaneous, “Oh, my God, there's a reason that I am like I am,” and then followed immediately by, “I still am the way that I am. Huh. Okay.” It feels like it helps things, but it also doesn't help things. But it does, and it comes back around. What was your experience with it?Marie: Yeah, it started because I was doing research to understand my sister better because she had been diagnosed with ADHD for a couple years. It made so much sense once I kind of understood and started researching a little bit more about it. And then, of course, doing my deep-dive research. I'm hearing all these traits that I'm like, “Oh. Wait, that does really sound like me.” The not being able to wake—Corey: What do you [mean 00:07:01]—Marie: —up in the morning—Corey: ADHD trait? Everyone does that. Wait.Marie: [laugh]. Yeah. When you said that enough times, you're like, “Oh, wait. Maybe this is not normal.” Or you don't really know what is—what is normal anyway, right? So, in doing that research, trying to connect with her, trying to understand her experience better, I just started learning about more and more of these traits.I also knew a shit ton of people in our course, had mentioned that they had ADHD in their intake form, and I was like, what is it about people that ADHD that are actually drawn to my YouTube videos or my way of explaining things? And I started to learn a little bit more; it's quite common for folks with ADHD to be drawn to one another, probably because of our communication styles, even the sort of mild interrupting, or kind of the way we banter together. There's different styles of communicating that I think often folks with ADHD are maybe drawn to one another or have an easier time understanding one another. So, listening to some of these symptoms, I was like, “Wait a second.” Because my sister and I are so different in the way our symptoms present.I thought, “Well, that's what ADHD looks like.” It's pure unbridled chaos and unfiltered. And I just had this idea of what it looked like because she was one of the few examples that I had. Meanwhile, I'm skipping grades, I'm in the gifted program, I'm off, you know, doing my own thing. It looked very different.I thought, “Oh, people with ADHD don't thrive in university,” or whatnot. So, I had a lot of assumptions that I had to unpack. And I think the one, sort of, I don't know, symptom that kind of twinged something in my brain was extreme difficulty getting up in the morning and even sort of waking up your brain in the morning. This has been a problem with jobs, it's been a problem was school, getting to school on time, getting to work on time. Similar to you, it has caused job loss, it has caused tension with partners. They don't understand, like, why can't you get out of bed and seize the day?And I just thought, “There's something weird going on there with my body.” But I can be, you know, wide awake at 7 p.m. and I'm, like, ready to go. And I can hyperfocus for days on end. So, just noticing some of these symptoms and kind of unpacking it a bit, I thought, “Okay, there's something to go a little deeper in here.”Corey: I have trouble getting up, but I'm almost never late. That one does not hit me in quite the same way. In fact—Marie: Well—Corey: —my first consulting clients, and I'd been building—I was independent for two weeks at that point, and I was in an in-person meeting in San Francisco and one day, I showed up 20 minutes late, and he just stared at me. “You're never late. What's the deal here?” And it's like, “Yeah, I had trouble getting up this morning.” That was a lie.I was able to tell him about three or four months later, that morning, I found out I was going to be a father. And that was an—you know, it turns out that I was going to be okay being late, but it was so early, you didn't want to tell anyone, yet. But it was—yeah, it's one of those things where that was more important than—Marie: Absolutely.Corey: —doing the work thing. But I still remember, yeah, I feel like I'm always about to be late but apparently my reputation is, I never am, so okay. I'll take it. That is a—again, it is a spectrum disorder. I also—Marie: Absolutely.Corey: —further there want to call out for viewers, listeners, et cetera, a couple of things. One, this is not mental health advice. If any of the stories we're telling resonate, talk to a qualified mental health professional. Secondly, I want to be clear as well here, Marie, that you and I both have significant advantages when it comes to dealing with these things. We both run our own companies, we can effectively restructure the way that we work in ways that are more accommodating for what we do.It turns out that in my employment days, that was never really a solution where, “Yeah, I decided I'm not going to wind up doing the on-call checklist every day. It doesn't resonate with me.”Marie: “Just not feeling like it.”Corey: “It's doing the same thing too many days in a row. And yeah, I'm not going to check the backups, either. What do you mean ‘I'm fired?'” yeah, it turns out, you're not able to—you're empowered to make those kinds of sweeping changes in the same way.Marie: Exactly.Corey: So, this is not advice for people. This is simply a pair of experience reports, the way I view it.Marie: Absolutely. I sort of feel like self-employment wasn't necessarily a choice, in a way. It just felt like that's the only way I'm going to be able to operate in this world. I need some more sense of control and say in how I structure my days, how I structure my work, being able to switch things up, being able to pivot quickly. I knew that I was going to need more control over that. So yeah, pretty unemployable over here. [laugh].Corey: So, once you wound up with the diagnosis, what happened next? What changes did you make that wound up resonating for you, things that were actionable? And, yeah, you've been very public about it as well. I want to highlight that. I'm not, for the most part.And part of that is because I internalized growing up that it was somehow a shameful thing that we don't talk about. And the other part of it, too, on some level, was I didn't want to turn it into a part of my brand identity, where, “Oh, yeah, Corey is very hard to describe.” So, people thrash around and look for labels to slap on me. ‘Shitposter' seems to have stuck rather well. Because as soon as people feel that they have a label for something, it becomes easier to classify and then dismiss it.It's aspects of my personality. It's who I am. I don't think of it as a disorder so much as it is part and parcel of who and what I am. And it turns out that being me is not—yet—a medically recognized diagnosis. So, I'm cautious to avoid the labeling aspect of it.But you have very publicly not, if not going for the label, you at least embraced it as an aspect of who you are, and you've been very vocal about your experiences and telling people how you have overcome aspects of this. It's admirable. I wish I did more of it, honestly.Marie: I think it's kind of essential, I think, in the nature of what we're teaching. Like, when we're teaching people to become more organized and we know that executive dysfunction is one of the signs or, you know, issues with ADHD, to me it sort of recontextualized why I became so freakin' obsessed with systems and organization: because I never felt organized. I always felt the sense of what is the stuff come so easy to other people? Why is it taking me so much longer? Why am I spending nights, evenings, taking courses about systems like I'm trying to understand how to give my life structure?And so, in a way, the way I have become organized was trial by fire, just teaching myself, learning, you know, getting coaches. Like, I literally had a systems coach to teach myself how to get my business organized. So, I had kind of obsessed over it, like a hyperfocus. And so, realizing that other people are struggling with this and there's a reason that people with ADHD are coming to the course seeking that sense of control. And so, learning that I had it, I was like, oh, this actually [laugh] does explain, in a way, my obsession with this or my curiosity about this, of, like, why does this come easy to some other people? Why do some people need to study this and learn this? Like, what is it about that?And so, I sort of felt like it would be doing a disservice if I didn't kind of name it and talk about it and say, well, this actually colors a lot of my opinions. This actually influences the way I approach organization or even productivity, not from a timing perspective, but from an energy management perspective. I didn't realize that was something that I'm doing. I'm not managing time, we're managing Marie's energy. And even my team is learning how to do that, too.So, I was like, “Oh, that actually makes a ton of sense.” And it also makes sense why some people won't resonate with this energy management thing or might think I'm going way too far down a rabbit hole on something and they're like, “Why can't people just do what they say?” Like, you don't understand, some of us need to trick ourselves into being productive. And this is how I've learned to do that. So, it was just kind of a funny recontextualizing or uncovering, oh, our brains operate very differently. And even within ADHD, people's brains operate differently, so how do we get people moving toward progress, but knowing that we kind of need different ways of doing that. So, it's just been kind of an interesting process.Corey: There's a fairly common experience report from folks who have ADHD that when they're kids, their memory is generally very good with a number of expressions of it, so we form our self-image in a lot of those times. And then for the rest of our lives, we tell ourselves the same lie, regardless of how many times it has proven to be a lie. And that lie is, “I don't need to write this down. I'll remember it.”Marie: Oh yes.Corey: “No, Corey, you will not remember it. You need to write it down. I promise.” And, for example, right now—I finally gave in and technology leapt ahead to the point where my entire life is run by Google Calendar—specifically three or four of them—that all route through Fantastical—which is the app I use—but it winds up grabbing my attention at the right time. It tells me what I need to do, when, and how, and it's wonderful.Because if it's not on my calendar, it does not happen.Marie: Yes.Corey: Like, I will forget our anniversary, my kids' birthdays, to pick my children up from school. We are talking about, if it is not on my calendar, it does not happen. That is the one system that has been forced on me that worked. Then we—let's talk about Notion for a minute because I looked at it briefly a few years ago, and it is one in the long, long, long list of tools or approaches or systems that I have played with and then discarded to act as basically an auxiliary brain pack. I used Evernote for a while and that sort of worked because I just would do different notes all the time and I'd wind up with 3000 of those things, and then the app gets bloaty and I move on to something else.For the last five years or so I've been using Drafts, a Mac slash iOS app, that only does text, which makes image management and attaching things kind of hard, but okay. And that's great, and now I have 5000 of those in my [back 00:16:25] folder, not categorized or organized anyway, so I focus instead on well, search for terms and hope I use the term I thought I did at the time. And so, every time I've tried to use something like Notion, it's yeah, this requires a way of thinking that I know I will get excited about if I look at it, and in a month, I'll be right back to where I am now. So, there's only so many times you go on the same ride before you know how it ends. How do y—like, that feels like a very common experience. How did you fix it?Marie: I think at the core though, you kind of have to be excited about the tool that you're using. And so, I don't think—Notion is not going to be an exciting fun tool for everyone. Some people are going to be like, “I don't want to frickin' build my productivity system. Are you kidding me? Like, just give me something that works out of the box.” Absolutely.But I think there's something about the visual components of Notion. Like, I am a designer; I went to design school. I think I'm—it's almost like something doesn't click until I see it in the way that I need to see it. And that's something I've learned about my brain is just, sometimes the same information can be presented to me, but if it's not in a visual way, or whether it's not spaced in the right way, my brain just kind of ignores it or it gets overwhelmed by it. And so, for me that visual aspect actually helps me learn.I'm priming my brain, I'm making my goals front and center. The fact that I can design it the way I need my brain to see it is part of its appeal to me. But I also recognize that's not something everyone gets excited about. They're not drawn to it. I'm all for using the tool that works the way that your brain is going to work.I get excited about making databases. I get excited about building glossaries of information to help me learn things. Like, for me, that's part of my learning and part of my process and it's just kind of what I'm used to, but I fully acknowledge, like, that stuff does not get everybody excited.[midroll 00:18:03]Corey: This episode is sponsored in part by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on-premises, private cloud, and they just announced a fully-managed service on AWS and Azure called BigAnimal, all one word. Don't leave managing your database to your cloud vendor because they're too busy launching another half-dozen managed databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications—including Oracle—to the cloud. To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: There's something very key you're talking about here, which is the idea of having to be excited about what it is that you do. I look at the things that I do professionally, and if I didn't deeply enjoy them, they would not get done, and I would have pivoted long ago to something else. People wonder why—Marie: Absolutely.Corey: —I make fun of so many things in the tech ecosystem. The honest answer is because if I just tell the dry, boring version of it, I will get bored because it's a fairly boring field. Whereas instead, okay, someone releases a new thing. Great. How do I keep it interesting for me? How do I find a way to tell that story?How do I find a way to, in turn, build that into something that, in turn, I can start dragging in different directions and opening up to new ways of talking without going too far? It's always a razor's edge, it's always a bit of a mind puzzle, and it's always different. I love that. That's why I do it. It's not for the audience so much as it is for myself. Because if I'm not engaged, no one else is going to care what I have to say.Marie: Absolutely. And I think that's a huge part of ADHD as well which is that interest-based nervous system, right? It's like we have to [laugh] trick ourselves into finding the excitement in it or whatever that looks like for each of us. But just if I'm not motivated, if I'm not excited about it—writing email newsletters doesn't get me excited; I'm like, “Okay, do I need to hire someone to do this?” Or how can I find a way to do it, whether it's—if making a video is more fun or easy, great.How can I, you know, make content do double-duty in that way? So yeah, I'm always trying to find ways to incentivize myself to do the things that need to get done, even though they may not be the most exciting. But step one is actually run a business that is based on something that you love doing. Which not everyone, maybe, has the privilege to do, but I think everything about the way I've designed my business model and the services that we offer is, don't offer services you don't really want to offer. Don't make products that you don't want to maintain, you're not excited about. So, it's definitely a core part of kind of how we design our whole business model.Corey: For me, a big part of it has always been just trying to make sure that I'm doing the things that engage me. And this is where that whole idea of being in a very privileged position enters into it. Take this podcast slash video right now, as a terrific example. I'm having this conversation, I have an entire system when I wind up sending a link to someone, it fires off Calendly, that hides webhooks and gets a whole bunch of other things set up. I show up, we have a conversation before the show to figure out just this is the general ebb and flow of the show. Here's the generalized topics we want to talk about. Let's dive in.And we finish the recording session. Great, I wind up closing the window and that's the last time I generally think about it. Because everything else has been automated. If anything other than me having this conversation with you does not need to be me, I there is no differentiated value in me being the person that does the audio engineering. It turns out, I can pay people who are world's better than I am at that, who actually enjoy it as opposed to viewing it as unnecessary chore, and I can do things that I find more appealing, like shitposting about a $1.108 trillion—Marie: Exactly.Corey: Company. It comes down to find the thing, the differentiation point, and find ways to make sure you don't have to do the other parts of it. But that is not a path that's available to everyone in every context. And again, I'm talking about this in a professional sense. I still have to do a whole bunch of stuff as I go through the course of my life that is not differentiated, but I can't very well hire someone to get me dressed in the morning. Well, I can but I feel like that becomes a little bit out of the scope of the lived human experience most of the [crosstalk 00:22:29].Marie: [laugh]. Absolutely. I feel like that's one thing I sort of regret not doing earlier is hiring someone to work with. So, the very first hire that I made was my chief of operations, and oh my gosh, the things that she took on that I used to do that I'm like, how on earth did I do that before? Because now that you do that, and you do it way faster, I just got to wonder, like, how the heck did I ever convince myself to do those activities?I don't want to do touch spreadsheets, I don't want to [laugh] deal with that stuff. I don't want to, you know, email reminders, or whatever it is. There's so many activities that she handles that I just… I would be happy to never touch again. And so, I sort of wish I had explored that earlier, but I was in that lone wolf, like, I got this. I'm going to run my own business solo forever.And, you know, I just sort of thought it's difficult to work with me or because of the way that I work, I don't know how to delegate. Like, it's all in your head. I just didn't really know how to do that. So, that process, I think, takes a while. That first hire when you're going from solo person to okay, now we're two; how do we work together? Okay, who else can we hire? What other activities can I get other people to do? So, that's been a process, for sure.Corey: Mike Julian, my business partner who you know, is a very process-driven person. He is very organized. His love language is Microsoft Excel, as I frequently tease him with. And one of the—not the only factor by a landslide, but one of the big early factors of what would—okay, I know what I'd do. What would Mike do here?Part of it is the never-ending litany of mail I get from the state around things like taxes, business registration, the rest. And normally my response when I get those, is I look at it, and it's like, “Welp, I'm going to fucking prison. That's the end of it. The end.” Because it's not that I don't have the money to pay my taxes, I assure you. What, I don't have it—because I—financial planning is kind of part and parcel of how we think about cloud economics.But no, it's the fact that I'm not going to sit there, fill out the form, put a stamp on it—or God forbid, fax it somewhere—and the rest. It's not the paying of the taxes that bothers me it is the paperwork and the process and the heavy lift associated with getting the executive function necessary to do it. So, it never gets done and deadlines slide by. And Mike was good at that for a time, and then he took the more reasonable approach about this of, “Huh. Seems to me like a lot of this stuff is not differentiated value that I need to be doing either.”So, we have a CFO who handles a lot of that stuff now and other operational folks. And it turns out that yeah, wow, there's a lot—I can—the quality of what I put out is a lot better because I get to focus on things instead of having to deal with the ebb and flow minutia of running payroll myself every week.Marie: Oh, yeah. All of that is very relatable. And this is why I can't do paper in the office. I think this is why I just moved my entire brain online. It's like if there's paper, stamps, anything related to having to go [laugh] to a post office to mail something. I think I still have the stack of thank you cards from our wedding from, you know, five years ago. So, yeah. [laugh].Corey: That you haven't sent out yet. Of course.Marie: Yes, exactly.Corey: Exact same—sorry, people 13—11 years ago, whenever it was.Marie: I'm so sorry.Corey: Yeah, one of these years. Yeah, and see, that's exactly how I treat things like Drafts or Notion, if I were to use it, or something else is great, it's still going to be the digital equivalent of a giant pile of paper. The thing is that computers can search through the contents of that paper a hell of a lot faster than I can, even with my own, at times, uncanny reading speed. There's some value to that. So, understanding how the systems work and having them bend to accommodate you, rather than trying to fool yourself in half to work within the confines of an existing system, that seems to be the direction that you're taking Notion in, specifically in the context of it is not prescriptive.And, on some level, that's kind of the problem I have with it. Whenever I try the getting started for us, it's, “Great, you can build your own system.” It's like, “Isn't that your job? What am I missing here?” Because the scariest thing I ever see when it's time for you to write a blog post or whatnot is an empty editor. It's, where do I get started? Where's the rest?I even built a template that I wind up sometimes using text expander to autofill, that gets me started. And it's just get—once I get started, it's great. It's hard to get me started; it's hard to get me to stop, in case no one has been aware of that. But it's been understanding how I work and how that integrates with it. I'm curious, given that you do talk to people who are trying to build these systems for a living for themselves? How common is my perspective on this? Am I out there completely, this unique, beautiful Snowflake? Is it yeah, that's basically everyone? Or somewhere in between?Marie: Oh, I definitely don't think you're alone with that. And again, I often will dissuade people from taking on Notion. I'm like, “Oh, if you're just looking for a note-taker, or you're just looking for something else,” or, “Your tools are already working for you, great. Keep using them.” So, I think it's quite common. I don't think Notion is the right tool for everyone.I think it's great for very visual people like myself, people that it matters how you are seeing your information, and how much information you're seeing, and you want more control over that, that's great. For me, I like the integration. I know that as soon as I'm bouncing around to different tools, like, I just already feel kind of scattered, so I was like, how can I pull everything that I need into these, sort of, singular dashboards. So, my approach is very dashboard-focused. Okay, Marie is going into content mode, it's time to write a blog. Go to the content hub. On the content hub is your list of most recent ideas, your templates for how to write a blog post. There's resources for creating video. It's already there for me; I'm not having to start from scratch like you said.But again, it took time to build that up for myself. So, I think you're not alone, and I think some people get excited about that building process; other people get irritated by it, and I don't think there's a right or wrong answer. It's just how do our brains work? Know thyself. And, yeah, I've sort of—I think also in a way, something that's a little different, maybe, about the way that I use Notion is I think of it as a personal development tool.It is a tool for making me better in different ways. It's for exploring my interests, it's for feeding my curiosity, it's for looking at change over time. I track my feelings every day. I've been journaling for 1300 days in a row, which is probably the only thing I've done consistently in my life [laugh] in the last couple of years. But now I can look and I can see trends over time in a really beautiful and visual way. And I just, to me, it's like a curiosity tool, to see, like, where am I going? Where have I been? What do I want more of?Corey: I need to look into this a bit more because my idea of a well-designed user interface is—I'm very opinionated on this—but it comes down to the idea of where do you use nouns versus verbs in command-line arguments to things you're running in the terminal. Because I was a grumpy Unix sysadmin for the first part of my career—because there's no other kind of Unix sysadmin—and going down that path was great. Okay, everything I'm interacting with is basically a text file piped together to do different things. And it took a while for me to realize, you know, maybe—just spitballing here—there's a better way to convey information than a wall of text, sometimes. Blasphemy.And no, no, it turns out that just because it's hard using the tools I'm used to doesn't mean that's the best way to convey information. And even now, these days, I'm spending more time getting the color theme and the font choices and typeface choices of what I'm doing in the terminal to represent something that's a bit more aesthetically pleasing. Does it actually account for anything? I don't know, but it feels better and there's almost a Feng Shui element of it. Similar to work in a—Marie: Yes.Corey: Clean office versus a messy one.Marie: A hundred percent. I think that's kind of how I think of an approach. I am much more likely to get the things done. If, when I come in and I open Notion, it's like, “Here's what's on today, Marie.” And it's like speaking nicely to me, there's little positive messages, there's beautiful imagery.It just makes me feel good when I'm starting my day. And knowing that how I feel is going to very much influence what I'm likely to accomplish in the day, again, I'm constantly tricking myself into getting [laugh] more excited and amped up about what's on the schedule for the day. So, I really liked that about it. It feels beautiful to me.Corey: I'm going to have to take another look at it at some point. I think that there's a lot of interesting directions to go into on this. I also have the privilege of having known you for a little while, back when you were more or less just getting started. One of the things that you said at the time that absolutely resonated with me was the idea of, wait, you mean build a business around teaching people how to use Notion? Like an info product or a training approach?And a lot of your concerns are the ones that I've harbored for a while, too, which is the idea of there's a proliferation of info products in technical and other spaces, and an awful lot of them—without naming any names or talking in any particular direction—are not the highest quality. People are building these courses while learning the thing themselves. And when they tell stories about it, it's all about, “And this is how I'm making money quickly.” I don't find that admirable; I don't necessarily want to learn how to do a thing from someone who does not have themselves at least a decent understanding themselves of what they're working on so they can address questions that go a bit off into the weeds. And so mu—again, knowing how to do a thing and knowing how to teach a thing are orthogonal concepts. And very often a lot of these info products are being created by people who don't really know how to do either, as best I can tell.Marie: Yes. So, I think you've nailed a point to that, knowing a thing deeply and then knowing how to teach that thing really well are two totally different skills. And I definitely bumped up against that myself. I'm like, I know, Notion inside and out. Like, you know, name something, I can make it, I can optimize it, I can, you know, build a system out of thin air really fast, no problem. I'm a problem solver that way.But to teach someone else how to do that requires very different skills. And I knew [laugh] as I was starting to teach people stuff, I'm like, “You could do this. You could do that.” And I'm like kind of bouncing around and I'm all over the place because I'm so excited about the possibilities. But wait a second.Beginners that are just learning how to use Notion don't need to know every frickin' possible way that you could use it. So, knowing that instructional design, curriculum design is a whole other skill, and I care about student results, it's like, this is a gap that I have, and I want to be an excellent teacher. It matters to me. I actually do want to become a better teacher. I want to have higher quality YouTube videos, I want to make sure that I'm not losing people along the way.I don't just care about making a shit ton of money with an info product; I care about peoples' experience and kind of having that, I don't know, that prestige element. Like, that's something that does matter in terms of producing quality products. So, I hired experts to help me do that because again, it's a not necessarily a strength of mine. So, I think I hired three different people in the course of six months to various consultants and people who understand learning design and that sort of thing. And I think that's something a lot of info product creators. They think of it as just packaging a blog and selling it, right?It's different. When you're teaching a course, for example, your formatting matters, how you display information matters, how you design activities matters. What separates a course from a passive income product or blog, right? We need to think about those things, and I think a lot of people are just like, what's the quickest, you know, buck that I can make on these products and just kind of turn them out. And I don't think every course creator has maybe done the extra legwork to really understand what makes students actually follow through and complete a course. It's hard. It's really hard.Corey: And these are also very different products. There's what you are teaching, which is here's how to contextualize these things and how to build a system around it. There's another offering out there that would be something that would also be very compelling from my perspective where, cool, I appreciate the understanding and the deep systems design approach that goes into this. Can I just give you a brain dump of all the problems that I have with this? You go away and build a system that accounts for all of that.And again, it's the outcome that I care about. There's this belief that oh we want consultants to build by the hour and work hard. No. I don't care. If you listen to this, nod and do the great customer service thing, the Zoom call, and just like, “Okay, that's template number three with three one-line changes. Done. Now, we're going to sit on it for a week so it looks hard.”Which we've all got that as consultants in the early days. And then you turn that around because it's the outcome that I really care about. But that's a different business, that is a different revenue model, that is different—Marie: Yes.Corey: That is not nearly so much a one-to-many, like an info product. That is a one-to-one or one-to-few.Marie: And I did that for the whole first year that the course was being developed and was out there. I was simultaneously consulting with people one-on-one all the time, with teams, with individuals. So, I'm learning about what are all those common challenges that keep popping up over and over again? What are the unique challenges? What are the common ones?And in my experience, what I bumped up against is people think they want to just pay someone to solve that, but then when you give someone a very fleshed out, organized system that they didn't participate in the building, it's a lot harder to get somebody to use it, to plug into a ready-made system. So, in our experience, there's a sort of back and forth. It has to happen in tandem; we do it over time. And you know, in my partner's case, Ben does consulting with companies as well, so he'll meet with them on a weekly basis and working with the different members of the team. So, there is some element of we built you a thing. Let's have you use it, notice where there's gaps, friction, whatever, because it's not a one-and-done process.It's not like, “You gave me all the info. We're good to go.” It's not until people are using it that you're like, “Oh, okay, that's close, but I'm finding myself doing this, or avoiding this, or clicking around too much.” And so, to me, it's a really organic process. But that's not something that I'm as keen to do. And maybe it's because I did it for, like, two years and kind of burnt out on it. I'm like, “I'm done. Like, I'd rather teach folks to do it themselves.” But so a partner does the consulting; I'm doing more of the teaching.Corey: That's what happened to an awful lot of our consulting work here at The Duckbill Group where it was exciting and fun for me for years, and at some point it turned into, I am interested in teaching how to do this a little bit more and systematizing it because I'm starting to get bored with aspects of it. And I was thinking, “Well, do I build a course?” It's, “Well, no. As it turns out that if you have the right starting point, I can hire people who I can teach how to do AWS bill analysis if they have the right starting point.” And it turns out that a lot of those people—read as all of them—are going to be way better at doing the systemic deep-dive across the board, rather than just finding the things that they find personally interesting and significant, and then, “Well, there you go. I did a consulting engagement.” And the output is basically three bullet points scrawled on the back of an envelope.Yeah, turns out that that's not quite the level of professionalism clients expect. Great, so our product is better, we're getting better insight into it, and I get to scratch my itch of teaching people how to do things internally without becoming a critical path blocker.Marie: Yeah, absolutely.Corey: I mean, I have shitposting to get back to. Come on.Marie: Yeah exactly. [laugh]. The important things. Love it.Corey: I really want to thank you for taking so much time to speak with me about all of these things. If people want to learn more—Marie: Absolutely.Corey: —where's the best place to find you?Marie: Yeah, you can find me at mariepoulin.com is where my personal blog, or weareokidoki.com, or notionmastery.com. You can also catch me on Twitter.Corey: And we will put links to—Marie: That's where I am most active. Yeah.Corey: Oh, of course. And all the links wind up going into the [show notes 00:37:42], as always. Thank you so much for your time. I appreciate it.Marie: Thanks for having me, Corey. It was awesome.Corey: Marie Poulin, CEO of Oki Doki. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice—and if it's on the YouTubes smash the like and subscribe buttons—whereas if you've hated this podcast episode, great, same thing, five-star review on whatever platform, smash the two buttons, but also leave an insulting comment and then turn that comment into an info product that you wind up selling to a whole bunch of people primarily to boost your own Twitter threads about how successful you are as a creator.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About swyxswyx has worked on React and serverless JavaScript at Two Sigma, Netlify and AWS, and now serves as Head of Developer Experience at Airbyte. He has started and run communities for hundreds of thousands of developers, like Svelte Society, /r/reactjs, and the React TypeScript Cheatsheet. His nontechnical writing was recently published in the Coding Career Handbook for Junior to Senior developers.Links Referenced: “Learning Gears” blog post: https://www.swyx.io/learning-gears The Coding Career Handbook: https://learninpublic.org Personal Website: https://swyx.io Twitter: https://twitter.com/swyx TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on-premises, private cloud, and they just announced a fully-managed service on AWS and Azure called BigAnimal, all one word. Don't leave managing your database to your cloud vendor because they're too busy launching another half-dozen managed databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications—including Oracle—to the cloud. To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: Let's face it, on-call firefighting at 2am is stressful! So there's good news and there's bad news. The bad news is that you probably can't prevent incidents from happening, but the good news is that incident.io makes incidents less stressful and a lot more valuable. incident.io is a Slack-native incident management platform that allows you to automate incident processes, focus on fixing the issues and learn from incident insights to improve site reliability and fix your vulnerabilities. Try incident.io, recover faster and sleep more.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Some folks are really easy to introduce when I have them on the show because, “My name is, insert name here. I built thing X, and my job is Y at company Z.” Then we have people like today's guest.swyx is currently—and recently—the head of developer experience at Airbyte, but he's also been so much more than that in so many different capacities that you're very difficult to describe. First off, thank you for joining me. And secondly, what's the deal with you?swyx: [laugh]. I have professional ADD, just like you. Thanks for having me, Corey. I'm a—Corey: It works out.swyx: a big fan. Longtime listener, first time caller. Love saying that. [laugh].Corey: You have done a lot of stuff. You have a business and finance background, which… okay, guilty; it's probably why I feel some sense of affinity for a lot of your work. And then you went into some interesting directions. You were working on React and serverless YahvehScript—which is, of course, how I insist on pronouncing it—at Two Sigma, Netlify, AWS—a subject near and dear to my heart—and most recently temporal.io.And now you're at Airbyte. So, you've been focusing on a lot of, I won't say the same things, but your area of emphasis has definitely consistently rhymed with itself. What is it that drives you?swyx: So, I have been recently asking myself a lot of this question because I had to interview to get my new role. And when you have multiple offers—because the job market is very hot for DevRel managers—you have to really think about it. And so, what I like to say is: number one, working with great people; number two, working on great products; number three, making a lot of money.Corey: There's entire school of thought that, “Oh, that's gauche. You shouldn't mention trying to make money.” Like, “Why do you want to work here because I want to make money.” It's always true—swyx: [crosstalk 00:03:46]—Corey: —and for some reason, we're supposed to pretend otherwise. I have a lot of respect for people who can cut to the chase on that. It's always been something that has driven me nuts about the advice that we give a new folks to the industry and peop—and even students figuring out their career path of, “Oh, do something you love and the money will follow.” Well, that's not necessarily true. There are ways to pivot something you'd love into something lucrative and there are ways to wind up more or less borderline starving to death. And again, I'm not saying money is everything, but for a number of us, it's hard to get to where we want to be without it.swyx: Yeah, yeah. I think I've been cast with the kind of judgmental label of being very financially motivated—that's what people have called me—for simply talking about it. And I'm like, “No. You know, it's number three on my priority list.” Like, I will leave positions where I have a lot of money on the table because I don't enjoy the people or the products, but having it up there and talking openly about it somehow makes you [laugh] makes you sort of greedy or something. And I don't think that's right. I tried to set an example for the people that I talk to or people who follow me.Corey: One of the things I've always appreciated about, I guess, your online presence, which has remained remarkably consistent as you've been working through a bunch of different, I guess, stages of life and your career, is you have always talked in significant depth about an area of tech that I am relatively… well, relatively crap at, let's be perfectly honest. And that is the wide world of most things front-end. Every time I see a take about someone saying, “Oh, front-end is junior or front-end is somehow less than,” I'd like to know what the hell it is they know because every time I try and work with it, I wind up more confused than I was when I started. And what I really appreciate is that you have always normalized the fact that this stuff is hard. As of the time that we're recording this a day or so ago, you had a fantastic tweet thread about a friend of yours spun up a Create React App and imported the library to fetch from an endpoint and immediately got stuck. And then you pasted this ridiculous error message.He's a senior staff engineer, ex-Google, ex-Twitter; he can solve complex distributed systems problems and unable to fetch from a REST endpoint without JavaScript specialist help. And I talk about this a lot in other contexts, where the reason I care so much about developer experience is that a bad developer experience does not lead people to the conclusion of, “Oh, this is a bad interface.” It leads people to the conclusion, “Oh, I'm bad at this and I didn't realize it.” No. I still fall into that trap myself.I was under the impression that there was just this magic stuff that JS people know. And your tweet did so much to help normalize from my perspective, the fact that no, no, this is very challenging. I recently went on a Go exploration. Now, I'm starting to get into JavaScript slash TypeScript, which I think are the same thing but I'm not entirely certain of that. Like, oh, well, one of them is statically typed, or strongly typed. It's like, “Well, I have a loud mechanical keyboard. Everything I do is typing strongly, so what's your point?”And even then we're talking past each other in these things. I don't understand a lot of the ecosystem that you live your career in, but I have always had a tremendous and abiding respect for your ability to make it accessible, understandable, and I guess for lack of a better term, to send the elevator back down.swyx: Oh, I definitely think about that strongly, especially that last bit. I think it's a form of personal growth. So, I think a lot of people, when they talk about this sending the elevator back down, they do it as a form of charity, like I'm giving back to the community. But honestly, you actually learn a lot by trying to explain it to others because that's the only way that you truly know if you've learned something. And if you ever get anything wrong, you'll—people will never let you forget it because it is the internet and people will crawl over broken glass to remind you that you're wrong.And once you've got it wrong, you will—you know, you've been so embarrassed that you'll never forget it. So, I think it's just a really good way to learn in public. And that's kind of the motto that I'm kind of known for. Yeah, we can take the direction anywhere you want to go in JavaScript land. Happy to talk about it all day. [laugh].Corey: Well, I want to start by something you just said where you're doing the learning in public thing. And something I've noticed is that there are really two positions you can take—in the general sense—when you set out to make a bit of a reputation for yourself in a particular technical space. You can either do the, “I'm a beginner here, same as the rest of you, and I'm learning in public,” or you can position yourself as something of an expert. And there are drawbacks and advantages to both. I think that if you don't look as wildly over-represented as I do, both of them are more fraught in different ways, where it's, “Oh, you're learning in public. Ah, look at the new person, she's dumb.”Or if you're presenting yourself as an expert, you get nibbled to death by ducks on a lot of the deep technical nuances and well, actually'ed to death. And my position has always been and this is going to be a radical concept for some folks, is that I'm genuinely honest. I tend to learn in public about the things that I don't know, but the things that I am something of a subject matter expert in—like, I don't know, cloud billing—I don't think that false modesty necessarily serves me particularly well. It's yeah, I know exactly what I'm talking about here. Pretending otherwise it's just being disingenuous.swyx: I try to think of it as having different gears of learning in public. So, I've called this “Learning Gears” in a previous blog post of mine, where you try to fit your mode of learning to the terrain that you're on, your domain expertise, and you should never over-represent the amount that you know because I think people are very rightly upset when there are a lot of people—let's say on Twitter, or YouTube, or Udemy even—who present themselves as experts who are actually—they just read the docs the previous night. So, you should try not to over-represent your expertise.But at the same time, don't let your imposter syndrome stop you from sharing what you are currently learning and taking corrections when you're wrong. And I think that's the tricky balance to get which is constantly trying to put yourself out there while accepting that you might be wrong and not getting offended when or personally attacked when someone corrects you, inevitably. And sometimes people will—especially if you have a lot of followers, people will try to say—you know, someone of your following—you know, it's—I kind of call this follower shaming, like, you should act, uh—invulnerable, or run every tweet through committee before you tweet after a certain sort of following size. So, I try to not do that and try to balance responsibility with authenticity.Corey: I think that there's something incredibly important about that, where there's this idea that you either become invulnerable and get defensive and you yell at people, and down that path lies disaster because, believe it or not, we all get it wrong from time to time, and doubling down and doubling down and doubling down again, suddenly, you're on an island all by yourself and no one respectable is going to be able to get there to help you. And the other side of it is going too far in the other direction, where you implicitly take any form of criticism whatsoever as being de facto correct. And I think that both paths don't lead to super great places. I think it's a matter of finding our own voices and doing a little bit of work as far as the validity of accepting a given piece of feedback goes. But other than that, I'm a big fan of being able to just more or less be as authentic as possible.And I get that I live in a very privileged position where I have paths open to me that are not open to most folks. But in many respects so to you are one of the—easily—first five people I would think of if someone said, “Hey if I need to learn JavaScript for someone, who should I talk to first?” You're on that list. And you've done a lot of things in this area, but you've never—you alluded to it a few minutes ago, but I'm going to call it out a little more pointedly—without naming names, let's be clear—and that you're never presented as a grifter, which is sort of the best way I can think of it of, “Well, I just learned this new technology stack yesterday and now I'm writing a book that I'm going to sell to people on how to be an expert at this thing.” And I want to be clear, this is very distinct from gatekeeping because I think that, “Oh, well, you have to be at least this much of an expert—” No, but I think that holding yourself out as I'm going to write a book on how to be proud of how to become a software engineer.Okay, you were a software engineer for six months, and more to the point, knowing how to do a thing and knowing how to teach a thing are orthogonal skill sets, and I think that is not well understood. If I ever write a book or put something—or some sort of info product out there, I'm going to have to be very careful not to fall into that trap because I don't want to pretend to be an expert in things that I'm not. I barely think I'm an expert in things that I provable am.swyx: there are many ways to answer that. So, I have been accused a couple of times of that. And it's never fun, but also, if you defend yourself well, you can actually turn a critic into a fan, which I love doing.Corey: Mm-hm.swyx: [laugh].Corey: Oh yes.swyx: what I fall back to, so I have a side interest in philosophy, based on one of my high school teachers giving us, like, a lecture in philosophy. I love him, he changed my life. [Lino Barnard 00:13:20], in case—in the off chance that he's listening. So, there's a theory of knowledge of, like, how do you know what you know, right? And if you can base your knowledge on truth—facts and not opinions, then people are arguing with the facts and not the opinions.And so, getting as close to ground truth as possible and having certainty in your collection of facts, I think is the basis of not arguing based on identity of, like, “Okay, I have ten years experience; you have two years experience. I am more correct than you in every single opinion.” That's also not, like, the best way to engage in the battlefield of ideas. It's more about, do you have the right amount of evidence to support the conclusions that you're trying to make? And oftentimes, I think, you know, that is the basis, if you don't have that ability.Another thing that I've also done is to collect the opinions of others who have more expertise and present them and curate them in a way that I think adds value without taking away from the individual original sources. So, I think there's a very academic way [laugh] you can kind of approach this, but that defends your intellectual integrity while helping you learn faster than the typical learning rate. Which is kind of something I do think about a lot, which is, you know, why do we judge people by the number of years experience? It's because that's usually the only metric that we have available that is quantifiable. Everything else is kind of fuzzy.But I definitely think that, you know, better algorithms for learning let you progress much faster than the median rate, and I think people who apply themselves can really get up there in terms of the speed of learning with that. So, I spend a lot of time thinking about this stuff. [laugh].Corey: It's a hard thing to solve for. There's no way around it. It's, what is it that people should be focusing on? How should they be internalizing these things? I think a lot of it starts to with an awareness, even if not in public, just to yourself of, “I would like advice on some random topic.” Do you really? Are you actually looking for advice or are you looking—swyx: right.Corey: For validation? Because those are not the same thing, and you are likely to respond very differently when you receive advice, depending on which side of that you're coming from.swyx: Yeah. And so, one way to do that is to lay out both sides, to actually demonstrate what you're split on, and ask for feedback on specific tiebreakers that would help your decision swing one way or another. Yeah, I mean, there are definitely people who ask questions that are just engagement bait or just looking for validation. And while you can't really fix that, I think it's futile to try to change others' behavior online. You just have to be the best version of yourself you can be. [laugh].Corey: DoorDash had a problem. As their cloud-native environment scaled and developers delivered new features, their monitoring system kept breaking down. In an organization where data is used to make better decisions about technology and about the business, losing observability means the entire company loses their competitive edge. With Chronosphere, DoorDash is no longer losing visibility into their applications suite. The key? Chronosphere is an open source compatible, scalable, and reliable observability solution that gives the observability lead at DoorDash business, confidence, and peace of mind. Read the full success story at snark.cloud/chronosphere. That's snark.cloud slash C-H-R-O-N-O-S-P-H-E-R-E.Corey: So, you wrote a book that is available at learninpublic.org, called The Coding Career Handbook. And to be clear, I have not read this myself because at this point, if I start reading a book like that, and you know, the employees that I have see me reading a book like that, they're going to have some serious questions about where this company is going to be going soon. But scrolling through the site and the social proof, the testimonials from various people who have read it, more or less read like a who's-who of people that I respect, who have been on this show themselves.Emma Bostian is fantastic at explaining a lot of these things. Forrest Brazeal is consistently a source to me of professional envy. I wish I had half his musical talent; my God. And your going down—it explains, more or less, the things that a lot of folks people are all expected to know but no one teaches them about every career stage, ranging from newcomer to the industry to senior. And there's a lot that—there's a lot of gatekeeping around this and I don't even know that it's intentional, but it has to do with the idea that people assume that folks, quote-unquote, “Just know” the answer to some things.Oh, people should just know how to handle a technical interview, despite the fact that the skill set is completely orthogonal to the day-to-day work you'll be doing. People should just know how to handle a performance review, or should just know how to negotiate for a raise, or should just know how to figure out is this technology that I'm working on no longer the direction the industry is going in, and eventually I'm going to wind up, more or less, waiting for the phone to ring because there's only three companies in the world left who use it. Like, how do you keep—how do you pay attention to what's going on around you? And it's the missing manual that I really wish that people would have pointed out to me back when I was getting started. Would have made life a lot easier.swyx: Oh, wow. That's high praise. I actually didn't know we're going to be talking about the book that much. What I will say is—Corey: That's the problem with doing too much. You never know what people have found out about you and what they're going to say when they drag you on to a podcast.swyx: got you, got you. Okay. I know, I know, I know where this is going. Okay. So, one thing that I really definitely believe is that—and this happened to me in my first job as well, which is most people get the mentors that they're assigned at work, and sometimes you have a bad roll the dice. [laugh].And you're supposed to pick up all the stuff they don't teach you in school at work or among your friend group, and sometimes you just don't have the right network at work or among your friend group to tell you the right things to help you progress your career. And I think a lot of this advice is written down in maybe some Hacker News posts, some Reddit posts, some Twitter posts, and there's not really a place you to send people to point to, that consolidates that advice, particularly focused at the junior to senior stage, which is the stage that I went through before writing the book. And so, I think that basically what I was going for is targeting the biggest gap that I saw, which is, there a lot of interview prep type books like Crack the Coding Career, which is kind of—Crack the Coding Interview, which is kind of the book title that I was going after. But once you got the job, no one really tells you what to do after you got that first job. And how do you level up to the senior that everyone wants to hire, right? There's—Corey: “Well, I've mastered cracking the coding interview. Now, I'm really trying to wrap my head around the problem of cracking the showing up at work on time in the morning.” Like, the baseline stuff. And I had so many challenges with that early in my career. Not specifically punctuality, but just the baseline expectation that it's just assumed that by the time you're in the workplace earning a certain amount of money, it's just assumed that you have—because in any other field, you would—you have several years of experience in the workplace and know how these things should play out.No, the reason that I'm sometimes considered useful as far as giving great advice on career advancement and the rest is not because I'm some wizard from the future, it's because I screwed it all up myself and got censured and fired and rejected for all of it. And it's, yeah, I'm not smart enough to learn from other people's mistakes; I got to make them myself. So, there's something to be said for turning your own missteps into guidance so that the next person coming up has an easier time than you did. And that is a theme that, from what I have seen, runs through basically everything that you do.swyx: I tried to do a lot of research, for sure. And so, one way to—you know, I—hopefully, I try not to make mistakes that others have learned, have made, so I tried to pick from, I think I include 1500 quotes and sources and blog posts and tweets to build up that level of expertise all in one place. So hopefully, it gives people something to bootstrap your experience off of. So, you're obviously going to make some mistakes on your own, but at least you have the ability to learn from others, and I think this is my—you know, I'm very proud of the work that I did. And I think people have really appreciated it.Because it's a very long book, and nobody reads books these days, so what am I doing [laugh] writing a book? I think it's only the people that really need this kind of advice, that they find themselves not having the right mentorship that reach out to me. And, you know, it's good enough to support a steady stream of sales. But more importantly, like, you know, I am able to mentor them at various levels from read my book, to read my free tweets, to read the free chapters, or join the pay community where we have weekly sessions going through every chapter and I give feedback on what people are doing. Sometimes I've helped people negotiate their jobs and get that bump up to senior staff—senior engineer, and I think more than doubled their salary, which was very personal proud moment for me.But yeah, anyway, I think basically, it's kind of like a third place between the family and work that you could go to the talk about career stuff. And I feel like, you know, maybe people are not that open on Twitter, but maybe they can be open in a small community like ours.Corey: There's a lot to be said for a sense of professional safety and personal safety around being—having those communities. I mean, mine, when I was coming up was the freenode IRC network. And that was great; it's pseudo-anonymous, but again, I was Corey and network staff at the time, which was odd, but it was great to be able to reach out and figure out am I thinking about this the wrong way, just getting guidance. And sure, there are some channels that basically thrived on insulting people. I admittedly was really into that back in the early-two-thousand-nothings.And, like, it was always fun to go to the Debian channel. It's like, “Yeah, can you explain to me how to do this or should I just go screw myself in advance?” Yeah, it's always the second one. Like, community is a hard thing to get right and it took me a while to realize this isn't the energy I want in the world. I like being able to help people come up and learn different things.I'm curious, given your focus on learning in public and effectively teaching folks as well as becoming a better engineer yourself along the way, you've been focusing for a while now on management. Tell me more about that.swyx: I wouldn't say it's been, actually, a while. Started dabbling in it with the Temporal job, and then now fully in it with Airbyte.Corey: You have to know, it has been pandemic time; it has stood still. Anything is—swyx: exactly.Corey: —a while it given that these are the interminable—this is the decade of Zoom meetings.swyx: [laugh]. I'll say I have about a year-and-a-half of it. And I'm interested in it partially because I've really been enjoying the mentoring side with the coding career community. And also, I think, some of the more effective parts of what I do have to be achieved in the planning stages with getting the right resources rather than doing the individual contributor work. And so, I'm interested in that.I'm very wary of the fact that I don't love meetings myself. Meetings are a means to an end for me and meetings are most of the job in management time. So, I think for what's important to me there, it is that we get stuff done. And we do whatever it takes to own the outcomes that we want to achieve and try to manage people's—try to not screw up people's careers along the way. [laugh]. Better put, I want people to be proud of what they get done with me by the time they're done with me. [laugh].Corey: So, I know you've talked to me about this very briefly, but I don't know that as of the time of this recording, you've made any significant public statements about it. You are now over at Airbytes, which I confess is a company I had not heard of before. What do y'all do over there?swyx: [laugh]. “What is it we do here?” So Airbyte—Corey: Exactly. Consultants want to know.swyx: Airbyte's a data integration company, which means different things based on your background. So, a lot of the data engineering patterns in, sort of, the modern data stack is extracting from multiple sources and loading everything into a data warehouse like a Snowflake or a Redshift, and then performing analysis with tools like dbt or business intelligence tools out there. We like to use MetaBase, but there's a whole there's a whole bunch of these stacks and they're all sort of advancing at different rates of progress. And what Airbyte would really like to own is the data integration part, the part where you load a bunch of sources, every data source in the world.What really drew me to this was two things. One, I really liked the vision of data freedom, which is, you have—you know, as—when you run a company, like, a typical company, I think at Temporal, we had, like, 100, different, like, you know, small little SaaS vendors, all of them vying to be the sources of truth for their thing, or a system of record for the thing. Like, you know, Salesforce wants to be a source of truth for customers, and Google Analytics want to be source of truth for website traffic, and so on and so forth. Like, and it's really hard to do analysis across all of them unless you dump all of them in one place.So one, is the mission of data freedom really resonates with me. Like, your data should be put in put somewhere where you can actually make something out of it, and step one is getting it into a format in a place that is amenable for analysis. And data warehouse pattern has really taken hold of the data engineering discipline. And I find, I think that's a multi-decade trend that I can really get behind. That's the first thing.Corey: I will say that historically, I'm bad at data. All jokes about using DNS as a database aside, one of the reasons behind that is when you work on stateless things like web servers and you blow trunks and one of them, oops. We all laugh, we take an outage, so maybe we're not laughing that hard, but we can reprovision web servers and things are mostly fine. With data and that going away, there are serious problems that could theoretically pose existential risk to the business. Now, I was a sysadmin and a, at least mediocre one, which means that after the first time I lost data, I was diligent about doing backups.Even now, the data work that we do have deep analysis on our customers' AWS bills, which doesn't sound like a big data problem, but I assure you it is, becomes something where, “Okay, step one. We don't operate on it in place.” We copy it into our own secured environment and then we begin the manipulations. We also have backups installed on these things so that in the event that I accidentally the data, it doesn't wind up causing horrifying problems for our customers. And lastly, I wind up also—this is going to surprise people—I might have securing the access to that data by not permitting writes.Turns out it's really hard—though apparently not impossible—to delete data with read-only calls.swyx: [crosstalk 00:28:12].Corey: It tends to be something of just building guardrails against myself. But the data structures, the understanding the analysis of certain things, I would have gotten into Go way sooner than I did if the introduction to Go tutorial on how to use it wasn't just a bunch of math problems talking about this is how you do it. And great, but here in the year of our lord 2022, I mostly want a programming language to smack a couple of JSON objects together and ideally come out with something resembling an answer. I'm not doing a whole lot of, you know, calculating prime numbers in the course of my week. And that is something that took a while for me to realize that, no, no, it's just another example of not being a great way of explaining something that otherwise could be incredibly accessible to folks who have real problems like this.I think the entire field right now of machine learning and the big data side of the universe struggles with this. It's, “Oh, yeah. If you have all your data, that's going to absolutely change the world for you.” “Cool. Can you explain how?” “No. Not effectively anyway.” Like, “Well, thanks for wasting everyone's time. It's appreciated.”swyx: Yeah, startup is sitting on a mountain of data that they don't use and I think everyone kind of feels guilty about it because everyone who is, like, a speaker, they're always talking about, like, “Oh, we used our data to inform this presidential campaign and look at how amazing we are.” And then you listen to the podcasts where the data scientists, you know, talk amongst themselves and they're like, “Yeah, it's bullshit.” Like, [laugh], “We're making it up as we go along, just like everyone else.” But, you know, I definitely think, like, some of the better engineering practices are arising under this. And it's professionalizing just like front-end professionalized maybe ten years ago, DevOps professionalized also, roughly in that timeframe, I think data is emerging as a field that is just a standalone discipline with its own tooling and potentially a lot of money running through it, especially if you look at the Snowflake ecosystem.So, that's why I'm interested in it. You know, I will say there's also—I talked to you about the sort of API replication use case, but also there's database replication, which is kind of like the big use case, which, for example, if you have a transactional sort of SQL database and you want to replicate that to an analytical database for queries, that's a very common one. So, I think basically data mobility from place to place, reshaping it and transferring it in as flexible manner as possible, I think, is the mission, and I think there's a lot of tooling that starts from there and builds up with it. So, Airbyte integrates pretty well with Airflow, Dexter, and all the other orchestration tools, and then, you know, you can use dbt, and everything else in that data stack to run with it. So, I just really liked that composition of tools because basically when I was a hedge fund analyst, we were doing the ETL job without knowing the name for it or having any tooling for it.I just ran a Python script manually on a cron job and whenever it failed, I would have to get up in the middle of night to go kick it again. It's, [laugh] it was that bad in 2014, '15. So, I really feel the pain. And, you know, the more data that we have to play around with, the more analysis we can do.Corey: I'm looking forward to seeing what becomes of this field as folks like you get further and further into it. And by, “Well, what do you mean, folks like me?” Well, I'm glad you asked, or we're about to as I put words in your mouth. I will tell you. People who have a demonstrated ability not just to understand the technology—which is hard—but then have this almost unicorn gift of being able to articulate and explain it to folks who do not have that level of technical depth in a way that is both accessible and inviting. And that is no small thing.If you were to ask me to draw a big circle around all the stuff that you've done in your career and define it, that's how I would do it. You are a storyteller who is conversant with the relevant elements of the story in a first-person perspective. Which is probably a really wordy way to put it. We should get a storyteller to workshop that, but you see the point.swyx: I try to call it, like, accessibly smart. So, it's a balance that you want to make, where you don't want to talk down to your audience because I think there are a lot of educators out there who very much stay at the basics and never leave that. You want to be slightly aspirational and slightly—like, push people to the bounds of their knowledge, but then not to go too far and be inaccessible. And that's my sort of polite way of saying that I dumb things down as service. [laugh].Corey: But I like that approach. The term dumbing it down is never a phrase to use, as it turns out, when you're explaining it to someone. It's like, “Let me dumb that down for you.” It's like, yeah, I always find the best way to teach someone is to first reach them and get their attention. I use humor, but instead we're going to just insult them. That'll get their attention all right.swyx: No. Yeah. It does offend some people who insist on precision and jargon. And I'm quite against that, but it's a constant fight because obviously there is a place at time for jargon.Corey: “Can you explain it to me using completely different words?” If the answer is, “No,” the question then is, “Do you actually understand it or are you just repeating it by rote?”swyx: right.Corey: There's—people learn in different ways and reaching them is important. [sigh].swyx: Exactly.Corey: Yeah. I really want to thank you for being so generous with your time. If people want to learn more about all the various things you're up to, where's the best place to find you?swyx: Sure, they can find me at my website swyx.io, or I'm mostly on Twitter at @swyx.Corey: And we will include links to both of those in the [show notes 00:33:37]. Thank you so much for your time. I really appreciate it.swyx: Thanks so much for having me, Corey. It was a blast.Corey: swyx, head of developer experience at Airbyte, and oh, so much more. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice or if it's on the YouTubes thumbs up and subscribe, whereas if you've hated this podcast, same thing, five-star review wherever you want, hit the buttons on the YouTubes, but also leaving insulting comment that is hawking your book: Why this Episode was Terrible that you're now selling as a legitimate subject matter expert in this space.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About AlyssaAlyssa Miller, Business Information Security Officer (BISO) for S&P Global, is the global executive leader for cyber security across the Ratings division, connecting corporate security objectives to business initiatives. She blends a unique mix of technical expertise and executive presence to bridge the gap that can often form between security practitioners and business leaders. Her goal is to change how security professionals of all levels work with our non-security partners throughout the business.A life-long hacker, Alyssa has a passion for technology and security. She bought her first computer herself at age 12 and quickly learned techniques for hacking modem communications and software. Her serendipitous career journey began as a software developer which enabled her to pivot into security roles. Beginning as a penetration tester, her last 16 years have seen her grow as a security leader with experience across a variety of organizations. She regularly advocates for improved security practices and shares her research with business leaders and industry audiences through her international public speaking engagements, online content, and other media appearances.Links Referenced: Cybersecurity Career Guide: https://alyssa.link/book A-L-Y-S-S-A dot link—L-I-N-K slash book: https://alyssa.link/book Twitter: https://twitter.com/AlyssaM_InfoSec alyssasec.com: https://alyssasec.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Vultr. Optimized cloud compute plans have landed at Vultr to deliver lightning-fast processing power, courtesy of third-gen AMD EPYC processors without the IO or hardware limitations of a traditional multi-tenant cloud server. Starting at just 28 bucks a month, users can deploy general-purpose, CPU, memory, or storage optimized cloud instances in more than 20 locations across five continents. Without looking, I know that once again, Antarctica has gotten the short end of the stick. Launch your Vultr optimized compute instance in 60 seconds or less on your choice of included operating systems, or bring your own. It's time to ditch convoluted and unpredictable giant tech company billing practices and say goodbye to noisy neighbors and egregious egress forever. Vultr delivers the power of the cloud with none of the bloat. Screaming in the Cloud listeners can try Vultr for free today with a $150 in credit when they visit getvultr.com/screaming. That's G-E-T-V-U-L-T-R dot com slash screaming. My thanks to them for sponsoring this ridiculous podcast.Corey: This episode is sponsored in part by Honeycomb. When production is running slow, it's hard to know where problems originate. Is it your application code, users, or the underlying systems? I've got five bucks on DNS, personally. Why scroll through endless dashboards while dealing with alert floods, going from tool to tool to tool that you employ, guessing at which puzzle pieces matter? Context switching and tool sprawl are slowly killing both your team and your business. You should care more about one of those than the other; which one is up to you. Drop the separate pillars and enter a world of getting one unified understanding of the one thing driving your business: production. With Honeycomb, you guess less and know more. Try it for free at honeycomb.io/screaminginthecloud. Observability: it's more than just hipster monitoring.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. One of the problems that many folks experience in the course of their career, regardless of what direction they're in, is the curse of high expectations. And there's no escaping for that. Think about CISOs for example, the C-I-S-O, the Chief Information Security Officer.It's generally a C-level role. Well, what's better than a C in the academic world? That's right, a B. My guest today is breaking that mold. Alyssa Miller is the BISO—B-I-S-O—at S&P Global. Alyssa, thank you for joining me to suffer my slings and arrows—Alyssa: [laugh].Corey: —as we go through a conversation that is certain to be no less ridiculous than it has begun to be already.Alyssa: I mean, I'm good with ridiculous, but thanks for having me on. This is awesome. I'm really excited to be here.Corey: Great. What the heck's BISO?Alyssa: [laugh]. I never get that question. So, this is—Corey: “No one's ever asked me that before.” [crosstalk 00:03:38]—Alyssa: Right?Corey: —the same thing as, “Do you know you're really tall?” “No, you're kidding.” Same type of story. But I wasn't clear. That means I'm really the only person left wondering.Alyssa: Exactly. I mean, I wrote a whole blog on it the day I got the job, right? So, Business Information Security Officer, Basically what it means is I am like the CISO but for my division, the Ratings Division at S&P Global. So, I lead our cyber security efforts within that division, work closely with our information security teams, our corporate IT teams, whatever, but I don't report to them; I report into the business line.I'm in the divisional CTO's org structure. And so, I'm the one bridging that gap between that business side where hey, we make all the money and that corporate InfoSec side where hey, we're trying to protect all the things, and there's usually that little bit of a gap where they don't always connect. That's me building the bridge across that.Corey: Someone who speaks both security and business is honestly in a bit of rare supply these days. I mean, when I started my Thursday newsletter podcast nonsense Last Week in AWS: Security, the problem I kept smacking into was everything I saw was on one side of that divide or the other. There was the folks who have the word security in their job title, and there tends to be this hidden language of corporate speak. It's a dialect I don't fully understand. And then you have the community side of actual security practitioners who are doing amazing work, but also have a cultural problem that more or less distills down to being an awful lot of shitheads in them there waters.And I wanted something that was neither of those and also wasn't vendor captured, which is why I decided to start storytelling in that space. But increasingly, I'm seeing that there's a significant problem with people who are able to contextualize security in the context of business. Because if you're secure enough, you can stop all work from ever happening, whereas if you're pure business side and only care about feature velocity and the rest, like, “Well, what happens if we get breached?” It's, “Oh, don't worry, I have my resume up to date.” Not the most reassuring answer to give people. You have to be able to figure out where that line lies. And it seems like that figuring out where that line is, is more or less your entire stock-in-trade.Alyssa: Oh absolutely, yeah. I mean, I can remember my earliest days as a developer, my cynical attitude towards security myself was, you know, their Utopia would be an impenetrable room full of servers that have no connections to anything, right? Like that would be wildly secure, yet completely useless. And so yeah, then I got into security and now I was one of them. And, you know, it's one of those things, you sit in, say a board meeting sometime and you listen to a CISO, a typical CISO talk to the board, and they just don't get it.Like, there's so much, “Hey, we're implementing this technology and we're doing this thing, and here's our vulnerability counts, and here's how many are overdue.” And none of that means anything. I mean, I actually had a board member ask me once, “What is a CISO?” I kid you not. Like, that's where they're at.Like, so don't tell them what you're doing, but tell them why connected back to, like, “Hey, the business needs this and this, and in order to do it, we've got to make sure it's secure, so we're going to implement these couple of things. And here's the roadmap of how we get from where we are right now to where we need to be so they can launch that new service or product,” or whatever the hell it is that they're going to do.Corey: It feels like security is right up there with accounting, in the sense of fields of endeavor where you don't want someone with too much personality involved. Because if the CISO's sitting there talking to the board, it's like, “So, what do you do here, exactly?” And the answer is the honest, “Hey, remember last month how we were in The New York Times for that giant data breach?” And they do a split take, “No, no, I don't.” “Exactly. You're welcome.” On some level, it is kind of honest, but it also does not instill confidence when you're that cavalier with the description of what it is you do here.Alyssa: Oh there's—Corey: At least there's some corners. I prefer—Alyssa: —there's so much—Corey: —places where that goes over well, but that's me.Alyssa: Yeah. But there's so much of that too, right? Like, here's the one I love. “Well, you know, it's not if you get breached, it's when. Oh, by the way, give me millions and millions of dollars, so I can make sure we don't get breached.”But wait, you just told me we're going to get breached no matter what we do. [laugh]. We do that in security. Like, and then you wonder why they don't give you funding for the initiative. Like, “Hello?” You know?And that's the thing that gets me it's like, can we just sit back and understand, like, how do you message to these people? Yeah I mean, you bring up the accounting thing; the funny thing is, at least all of them understand some level of accounting because most of them have MBAs and business degrees where they had to do some accounting. They didn't go through cyber security in their MBA program.So, one of my favorite questions on Twitter once was somebody asked me, you know, if I want to get into cyber security leadership, what is the one thing that I should focus on or what skills should I study? I said, “Go study MBA concepts.” Like, forget all the cyber security stuff. You probably have plenty of that technolog—go understand what they learn in MBA programs. And if you can start to speak that language, that's going to pay dividends for bridging that gap.Corey: So, you don't look like the traditional slovenly computer geek showing up at those meetings who does not know how to sound as if they belong in the room. Like, it's unfair, on some level, and I used to have bitter angst about that. Like, “Why should how I dress matter how people perceive me?” Yeah, in an absolute sense you're absolutely right, however, I can talk about the way the world is or the way I wish it were and there has to be a bit of a divide there.Alyssa: Oh, for sure. Yeah. I mean, you can't deny that you have to be prepared for the audience you're walking into. Now, I work in big conservative financial services on Wall Street. You know, and I had this conversation with a prominent member of our community when I started the job.I'm like, “Boy, I guess I can't really put stickers on my laptop. I'm going to have to get, you know, a protector or something to put stickers on.” Because the last thing I want to do is go into a boardroom with my laptop and whip out a bunch of hacker stickers on the backside of my laptop. Like, in a lot of spaces that will work, but you can't really do that when you're, you know, at, you know, the executive level and you're in a conservative, financial [unintelligible 00:10:16]. It just, I would love to say they should deal with that, I should be able to have pink hair, and you know, face tattoos and everything else, but the reality is, yeah, I can do all that, but these are still human beings who are going to react to that.And it's the same when talking about cyber security, then. Like, I have to understand as a security practitioner that all they know about cyber security is it's big and scary. It's the thing that keeps them up at night. I've had board members tell me exactly that. And so, how do I make it a little less scary, or at least get them to have some confidence in me that I'll, like, carry the shield in front of them and protect them. Like, that's my job. That's why I'm there.Corey: When I was starting my consultancy five years ago, I was trying to make a choice between something in the security cloud direction or the cost cloud direction. And one of the things that absolutely tipped the balance for me was the fact that the AWS bill is very much a business-hours-only problem. No one calls me at two in the morning screaming their head off. Usually. But there's a lot of alignment between those two directions in that you can spend all your time and energy fixing security issues and/or reducing the bill, but past a certain point, knock it off and go do the thing that your company is actually there to do.And you want to be responsible to a point on those things, but you don't want it to be the end-all-be-all because the logical outcome of all of that, if you keep going, is your company runs out of money and dies because you're not going to either cost optimize or security optimize your business to its next milestone. And weighing those things is challenging. Now, too many people hear that and think, “See, I don't have to worry about those things at all.” It's, “Oh, you will sooner or later. I promise.”Alyssa: So, here's the fallacy in that. There is this assumption that everything we do in security is going to hamper the business in some way and so we have to temper that, right? Like, you're not wrong. And we talked about before, right? You know, security in a traditional sense, like, we could do all of the puristic things and end up just, like, screeching the world to a halt.But the reality is, we can do security in a way that actually grows the business, that actually creates revenue, or I should say enables the creation of revenue in that, you know, we can empower the business to do more things and to be more innovative by how we approach security in the organization. And that's the big thing that we miss in security is, like, look, yes, we will always be a quote-unquote, “Cost center,” right? I mean, we in security don't—unless you work for a security organization—we're not getting revenue attributed to us, we're not creating revenue. But we are enabling those people who can if we approach it right.Corey: Well, the Red Team might if they go a little off-script, but that's neither here nor there.Alyssa: I—yeah, I mean, I've had that question. “Like, couldn't we just sell resell our Red Team services?” No. No. That's not our core [crosstalk 00:13:14]Corey: Oh, I was going the other direction. Like, oh, we're just going to start extorting other businesses because we got bored this week. I'm kidding. I'm kidding. Please don't do an investigation, any law enforcement—Alyssa: I was going to say, I think my [crosstalk 00:13:22]—Corey: —folks that happen to be listening to this.Alyssa: [crosstalk 00:13:24] is calling me right now. They're want to know what I'm [laugh] talking about. But no—Corey: They have some inquiries they would like you to assist them with and they're not really asking.Alyssa: Yeah, yeah, they're good at that. No, I love them, though. They're great. [laugh]. But no, seriously, like, I mean, we always think about it that way because—and then we wonder why do we have the reputation of, you know, the Department of No.Well, because we kind of look at it that way ourselves; we don't really look at, like how can we be a part of the answer? Like, when we look at, like, DevSecOps, for instance. Okay, I want to bring security into my pipeline. So, what do we say? “Oh, shared responsibility. That's a DevOps thing.” So, that means security is everybody's responsibility. Full stop.Corey: Right. It's a—Alyssa: Well—Corey: And there, I agree with you wholeheartedly. Cost is—Alyssa: But—Corey: —aligned with this. It has to be easier to do it the right way than to just go off half-baked and do it yourself off the blessed path. And that—Alyssa: So there—Corey: —means there's that you cannot make it harder to do the right thing; you have to make it easier because you will not win against human psychology. Depending on someone when they're done with an experiment to manually go in and turn things off. It will not happen. And my argument has been that security and cost are aligned constantly because the best way to secure something and save money on at the same time is to turn that shit off. You wouldn't think it would be that simple, but yet here we are.Alyssa: But see, here's the thing. This is what kills me. It's so arrogant of security people to look at it and say that right? Because shared responsibility means shared. Okay, that means we have responsibilities we're going to share. Everybody is responsible for security, yes.Our developers have responsibilities now that we have to take a share in as well, which is get that shit to production fast. Period. That is their goal. How fast can I pop user stories off the backlog and get them to deployment? My SRE is on the ops side. They're, like, “We just got to keep that stuff running. That's all we that's our primary focus.”So, the whole point of DevOps and DevSecOps was everybody's responsible for every part of that, so if I'm bringing security into that message, I, as security, have to be responsible for site's stability; I, in security, have to be responsible for efficient deployment and the speed of that pipeline. And that's the part that we miss.Corey: This episode is sponsored in parts by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on-premises, private cloud, and they just announced a fully-managed service on AWS and Azure called BigAnimal, all one word. Don't leave managing your database to your cloud vendor because they're too busy launching another half-dozen managed databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications—including Oracle—to the cloud. To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: I think you might be the first person I've ever spoken to that has that particular take on the shared responsibility model. Normally, when I hear it, it's on stage from an AWS employee doing a 45-minute song-and-dance about what the secured responsibility model is, and generally, that is interpreted as, “If you get breached, it's your fault, not ours.”Alyssa: [laugh].Corey: Now, you can't necessarily say it that directly to someone who has just suffered a security incident, which is why it takes 45 minutes and slides and diagrams and excel sheets and the rest. But that is what it fundamentally distills down to, and then you wind up pointing out security things that they've had that [unintelligible 00:17:11] security researchers have pointed out and they are very tight-lipped about those things. And it's, “Oh, it's not that you're otherworldly good at security; it's that you're great at getting people to shut up.” You know, not me, for whatever reason because I'm noisy and obnoxious, but most people who actually care about not getting fired from their jobs, generally don't want to go out there making big cloud companies look bad. Meanwhile, that's kind of my entire brand.Alyssa: I mean, it's all about lines of liability, right?Corey: Oh yeah.Alyssa: I mean, where am I liable, where am I not? And yeah, well, if I tell you you're responsible for security on all these things, and I can point to any part of that was part of the breach, well, hey, then it's out of my hands. I'm not liable. I did what I said I would; you didn't secure your stuff. Yeah, it's—and I mean, and some of that is to be fair.Like, I mean, okay, I'm going to host my stuff on your computer—the whole cloud is just somebody else's computer model is still ultimately true—but, yeah, I mean, I'm expecting you to provide me a stable and secure environment and then I'm going to deploy stuff on it, and you are expecting me to deploy things that are stable and secure as well. And so, when they say shared model or shared responsibility model, but it—really if you listen to that message, it's the exact opposite. They're telling you why it's a separate responsibility model. Here's our responsibilities; here's yours. Boom. It's not about shared; it's about separated.Corey: One of the most formative, I guess, contributors to my worldview was 13 years ago, I went on a date and met someone lovely. We got married. We've been together ever since, and she's an attorney. And it is been life-changing to understand a lot of that perspective, where it turns out when you're dealing with legal, they are not—and everyone says, “Oh, and the lawyers insisted on these things.”No, they didn't. A lawyer's entire role in a company is to identify risk, and then it is up to the business to make a decision around what is acceptable and what is not. If your lawyers ever insist on something, what that actually means in my experience is, you have said something profoundly ignorant that is one of those, like—that is—they're doing the legal equivalent of slapping the gun out of the toddler's hand of, “No, you cannot go and tweet that because you'll go to prison,” level of ridiculous nonsense where it is, “That will violate the law.” Everything else is different shades of the same answer: it depends. Here's what to consider.Alyssa: Yes.Corey: And then you choose—and the business chooses its own direction. So, when you have companies doing what appeared to be ridiculous things, like Oracle, for example, loves to begin every keynote with a disclaimer about how nothing they're about to say is true, the lawyers didn't insist on that—though they are the world's largest law firm, Kirkland Ellison. But instead, it's this entire story of given the risk and everything that we know about how we say things onstage and people gunning for us, yeah, we are going to [unintelligible 00:20:16] this disclaimer first. Most other tech companies do not do that exact thing, which I've got to say when you're sitting in the audience ready to see the new hotness that's about to get rolled out and it starts with a disclaimer, that is more or less corporate-speak for, “You are about to hear some bullshit,” in my experience.Alyssa: [laugh]. Yes. I mean and that's the thing, like, [clear throat], you know, we do deride legal teams a lot. And you know, I can find you plenty of security people who hate the fact that when you're breached, who's the first call you make? Well, it's your legal team.Why? Because they're the ones who are going to do everything in their power to limit the amount that you can get sued on the back-end for anything that got exposed, that you know, didn't meet service levels, whatever the heck else. And that all starts with legal privilege.Corey: They're reporting responsibilities. Guess who keeps up on what those regulatory requirements are? Spoiler, it's probably not you, whoever's listening to this, unless you're an attorney because that is their entire job.Alyssa: Yes, exactly. And, you know, work in a highly regulated environment—like mine—and you realize just how critical that is. Like, how do I know—I mean, there are times there's this whole discussion of how do you determine if something is a material impact or not? I don't want to be the one making that, and I'm glad I don't have to make that decision. Like, I'll tell you all the information, but yes, you lawyers, you compliance people, I want you to make the decision of if it's a material impact or not because as much as I understand about the business, y'all know way more about that stuff than I do.I can't say. I can only say, “Look, this is what it impacted. This is the data that was impacted. These are the potential exposures that occurred here. Please take that information now and figure out what that means, and is there any materiality to that that now we have to report that to the street.”Corey: Right, right. You can take my guesses on this or you can get it take an attorney's. I am a loud, confident-sounding white guy. Attorneys are regulated professionals who carry malpractice insurance. If they give wrong advice that is wrong enough in these scenarios, they can be sanctioned for it; they can lose their license to practice law.And there are challenges with the legal profession and how much of a gatekeeper the Bar Association is and the rest, but this is what it is [done 00:22:49] for itself. That is a regulated industry where they have continuing education requirements they need to certify in a test that certain things are true when they say it, whereas it turns out that I don't usually get people even following up on a tweet that didn't come true very often. There's a different level of scrutiny, there's a different level of professional bar it raises to, and it turns out that if you're going to be legally held to account for things you say, yeah, turns out a lot of your answers to are going to be flavors of, “It depends.”Alyssa: [laugh].Corey: Imagine that.Alyssa: Don't we do that all the time? I mean, “How critical is this?” “Well, you know, it depends on what kind of data, it depends on who the attacker is. It depends.” Yeah, I mean, that's our favorite word because no one wants to commit to an absolute, and nor should we, I mean, if we're speaking in hyperbole and absolutes, boy, we're doing all the things wrong in cyber.We got to understand, like, hey, there is nuance here. That's how you run—no business runs on absolutes and hyperbole. Well, maybe marketing sometimes, but that's a whole other story.Corey: Depends on if it's done well or terribly.Alyssa: [laugh]. Right. Exactly. “Hey, you can be unhackable. You can be breached-proof.” Oh, God.Corey: Like, what's your market strategy? We're going to paint a big freaking target in the front of the building. Like, I still don't know how Target the company was ever surprised by a data breach that they had when they have a frickin' bullseye as their logo.Alyssa: “Come get us.”Corey: It's, like, talk about poking the bear. But there we are.Alyssa: [unintelligible 00:24:21] no. I mean, hey, [unintelligible 00:24:23] like that was so long ago.Corey: It still casts a shadow.Alyssa: I know.Corey: People point to that as a great example of, like, “Well, what's going to happen if we get breached?” It's like, well look at Target because they wound up—like, their stock price a year later was above where it had been before and it seemed to have no lasting impact. Yeah, but they effectively replaced all of the execs, so you know, let's have some self-interest going on here by named officers of the company. It's, “Yeah, the company will be fine. Would you like to still be here what it is?”Alyssa: And how many lawsuits do you think happened that you never heard about because they got settled before they were filed?Corey: Oh, yes. There's a whole world of that.Alyssa: That's what's really interesting when people talk about, like, the cost of breach and stuff, it's like, we don't even know. We can't know because there is so much of that. I mean, think about it, any organization that gets breached, the first thing they're trying to do is keep as much of it out of the news as they can, and that includes the lawsuits. And so, you know, it's like, all right, well, “Hey, let's settle this before you ever file.”Okay, good. No one will ever know about that. That will never show up anywhere. It is going to show up on a balance sheet anywhere, right? I mean, it's there, but it's buried in big categories of lots of other things, and how are you ever going to track that back without, you know, like, a full-on audit of all of their accounting for that year? Yeah, it's—so I always kind of laugh when people start talking about that and they want to know, what's the average cost of a breach. I'm like, “There's no way to measure that. There is none.”Corey: It's not cheap, and the reputational damage gets annoying. I still give companies grief for these things all the time because it's—again, the breach is often about information of mine that I did not consciously choose to give to you and the, “Oh, I'm going to blame a third-party process.” No, no, you can outsource work, but not responsibility. You can't share that one.Alyssa: Ah, third-party diligence, uh, that seems to be a thing. You know, I think we're supposed to make sure our third parties are trustworthy and doing the right things too, right? I mean, it's—Corey: Best example I ever saw that was an article in the Wall Street Journal about the Pokemon company where they didn't name the vendor, but they said they declined to do business with them in part based upon their lax security policy around S3 buckets. That is the first and so far only time I have had an S3 Bucket Responsibility Award engraved and sent to their security director. Usually, it's the ignoble prize of the S3 Bucket Negligence Award, and there are oh so many of those.Alyssa: Oh, and it's hard, right? Because you're standing—I mean, I'm in that position a lot, right? You know, you're looking at a vendor and you've got the business saying, “God, we want to use this vendor. All their product is great.” And I'm sitting there saying, but, “Oh, my God, look at what they're doing. It's a mess. It's horrible. How do I how do we get around this?”And that's where, you know, you just have to kind of—I wish I could say no more, but at the end of the day, I know what that does. That just—okay, well, we'll go file an exception and we'll use it anyway. So, maybe instead, we sit and work on how to do this, or maybe there is an alternative vendor, but let's sort it out together. So yeah, I mean, I do applaud them. Like that's great to, like, be able to look at a vendor and say, “No, we ain't touching you because what you're doing over there is nuts.” And I think we're learning more and more how important that is, with a lot of the supply chain attacks.Corey: Actually, I'm worried about having emailed you, you're going to leak my email address when your inbox inevitably gets popped. Come on. It's awful stuff.Alyssa: Yeah, exactly. So, I mean, it's we there's—but like everything, it's a balance again, right? Like, how can we keep that business going and also make sure that their vendors—so that's where it just comes down to, like, okay, let's talk contracts now. So, now we're back to legal.Corey: We are. And if you talk to a lawyer and say, “I'm thinking about going to law school,” the answer is always the same. “No… don't do it.” Making it clear that is apparently a terrible life and professional decision, which of course, brings us to your most recent terrible life and professional decision. As we record this, we are reportedly weeks away from you having a physical copy in your hands of a book.And the segue there is because no one wants to write a book. Everyone wants to have written a book, but apparently—unless you start doing dodgy things and ghost-writing and exploiting people in the rest—one is a necessary prerequisite for the other. So, you've written a book. Tell me about it.Alyssa: Oof, well, first of all, spot on. I mean, I think there are people who really do, like, enjoy the act of writing a book—Corey: Oh, I don't have the attention span to write a tweet. People say, “Oh, you should write a book, Corey,” which I think is code for them saying, “You should shut up and go away for 18 months.” Like, yeah, I wish.Alyssa: Writing a book has been the most eye-opening experience of my life. And yeah, I'm not a hundred percent sure it's one I'll ever—I've joked with people already, like, I'll probably—if I ever want another book, I'll probably hire a ghostwriter. But no, I do have a book coming out: Cybersecurity Career Guide. You know, I looked at this cyber skills gap, blah, blah, blah, blah, blah, we hear about it, 4 million jobs are going to be left open.Whatever, great. Well, then how come none of these college grads can get hired? Why is there this glut of people who are trying to start careers in cyber security and we can't get them in?Corey: We don't have six months to train you, so we're going to spend nine months trying to fill the role with someone experienced?Alyssa: Exactly. So, 2020 I did a bunch of research into that because I'm like, I got to figure this out. Like, this is bizarre. How is this disconnect happening? I did some surveys. I did some interviews. I did some open-source research. Ended up doing a TED Talk based off of that—or TEDx Talk based off of that—and ultimately that led into this book. And so yeah, I mean, I just heard from the publisher yesterday, in fact that we're, like, in that last stage before they kick it out to the printers, and then it's like three weeks and I should have physical copies in my hands.Corey: I will be getting one when it finally comes out. I have an almost, I believe, perfect track record of having bought every book that a guest on this show has written.Alyssa: Well, I appreciate that.Corey: Although, God help me if I ever have someone, like, “So, what have you done?” “I've written 80 books.” Like, “Well, thank you, Stephen King. I'm about to go to have a big—you're going to see this number of the company revenue from orbit at this point with that many.” But yeah, it's impressive having written a book. It's—Alyssa: I mean, for me, it's the reward is already because there are a lot of people have—so my publisher does really cool thing they call it early acc—or electronic access program, and where there are people who bought the book almost a year ago now—which is kind of, I feel bad about that, but that's as much my publisher as it is me—but where they bought it a year ago and they've been able to read the draft copy of the book as I've been finishing the book. And I'm already hearing from them, like, you know, I'm hearing from people who really found some value from it and who, you know, have been recommending it other people who are trying to start careers and whatever. And it's like, that's where the reward is, right?Like, it was, it's hell writing a book. It was ten times worse during Covid. You know, my publisher even confirmed that for me that, like, look, yeah, you know, authors around the globe are having problems right now because this is not a good environment conducive to writing. But, yeah, I mean, it's rewarding to know that, like, all right, there's going to be this thing out there, that, you know, these pages that I wrote that are helping people get started in their careers, that are helping bring to light some of the real challenges of how we hire in cyber security and in tech in general. And so, that's the thing that's going to make it worthwhile. And so yeah, I'm super excited that it's looking like we're mere weeks now from this thing being shipped to people who have bought it.Corey: So, now it's racing, whether this gets published before the book does. So, we'll see. There is a bit of a production lag here because, you know, we have to make me look pretty and that takes a tremendous amount of effort.Alyssa: Oh, stop. Come on now. But it will be interesting to see. Like, that would actually be really cool if they came out at about the same time. Like, you know, I'm just saying.Corey: Yeah. We'll see how it goes. Where's the best place for people to find you if they want to learn more?Alyssa: About the book or in general?Corey: Both.Alyssa: So—Corey: Links will of course be in the [show notes 00:32:49]. Let's not kid ourselves here.Alyssa: The book is real easy. Go to Alyssa—A-L-Y-S-S-A, back here behind me for those of you seeing the video. Um—I can't point the right direction. There we go. That one. A-L-Y-S-S-A dot link—L-I-N-K slash book. It's that simple. It'll take you right to Manning's site, you can get in.Still in that early access program, so if you bought it today, you would still be able to start reading the draft versions of it. If you want to know more about me, honestly, the easiest way is to find me on Twitter. You can hear all the ridiculousness of flight school and barbecue and some security topics, too, once in a while. But at @alyssam_infosec. Or if you want to check out the website where I blog, every rare occasion, it's alyssasec.com.Corey: And all of that will be in the [show notes 00:33:41]. Thank you—Alyssa: There's a lot. [laugh].Corey: I'm looking forward to seeing it, too. Thank you so much for taking the time to deal with my nonsense today. I really appreciate it.Alyssa: Oh, that was nonsense? Are you kidding me? This was a great discussion. I really appreciate it.Corey: As have I. Thanks again for your time. It is always great to talk to people smarter than I am—which is, let's be clear, most people—Alyssa Miller, BISO at S&P Global. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice—or smash the like and subscribe button if this is on the YouTubes—whereas if you've hated the podcast, same thing, five-star review, platform of choice, smash both of the buttons, but also leave an angry comment, either on the YouTube video or on the podcast platform, saying that this was a waste of your time and what you didn't like about it because you don't need to read Alyssa's book; you're going to get a job the tried and true way, by printing out a copy of your resume and leaving it on the hiring manager's pillow in their home.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About SharoneI'm Sharone Zitzman, a marketing technologist and open source community builder, who likes to work with engineering teams that are building products that developers love. Having built both the DevOps Israel and Cloud Native Israel communities from the ground up, today I spend my time finding the places where technology and people intersect and ensuring that this is an excellent experience. You can find my talks, articles, and employment experience at rtfmplease.dev. Find me on Twitter or Github as @shar1z.Links Referenced: Personal Twitter: https://twitter.com/shar1z Website: https://rtfmplease.dev LinkedIn: https://www.linkedin.com/in/sharonez/ @TLVCommunity: https://twitter.com/TLVcommunity @DevOpsDaysTLV: https://twitter.com/devopsdaystlv TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: DoorDash had a problem as their cloud native environment scaled and developers delivered new features, their monitoring system kept breaking down. In an organization where data is used to make better decisions about technology and about the business, losing observability means the entire company loses their competitive edge. With Chronosphere, DoorDash is no longer losing visibility into their applications suite. The key? Chronosphere is an open source compatible, scalable, and reliable observability solution that gives the observability lead at DoorDash business, competence, and peace of mind. Read the full success story at snark.cloud/chronosphere. That's snark.cloud/C-H-R-O-N-O-S-P-H-E-R-E.Corey: The company 0x4447 builds products to increase standardization and security in AWS organizations. They do this with automated pipelines that use well-structured projects to create secure, easy-to-maintain and fail-tolerant solutions, one of which is their VPN product built on top of the popular OpenVPN project which has no license restrictions; you are only limited by the network card in the instance. To learn more visit: snark.cloud/deployandgoCorey: Welcome to Screaming in the Cloud. I'm Corey Quinn and I have been remiss by not having today's guest on years ago because back before I started this ridiculous nonsense that, well, whatever it is you'd call what I do for a living, I did other things instead. I did the DevOps, which means I was sad all the time. And the thing that I enjoyed was the chance to go and speak on conference stages. One of those stages, early on in my speaking career, was at DevOpsDays Tel Aviv.My guest today is Sharone Zitzman, who was an organizer of DevOpsDays Tel Aviv, who started convincing me to come back. And today is in fact, in the strong tradition here of making up your own job titles in ways that make people smile, she is the Chief Manual Reader at RTFM Please Ltd. Sharone, thank you for joining me.Sharone: Thank you for having me, Corey. Israelis love the name of my company, but Americans think it has a lot of moxie and chutzpah. [laugh].Corey: It seems a little direct and aggressive. It's like, oh, good, you are familiar with how this is going to go. There's something to be said for telling people what you do on the tin upfront. I've never been a big fan of trying to hide that. I mean, the first iteration of my company was the Quinn Advisory Group because I thought, you know, let's make it look boring and sedate and like I can talk to finance people. And yeah, that didn't last more than ten seconds of people talking to me.Also, in hindsight, the logo of a big stylized Q. Yeah, I would have had to change that anyway, for the whole QAnon nonsense because I don't want to be mistaken for that particular brand of nuts.Sharone: Yeah, I decided to do away with the whole formalities and upfront, just go straight [laugh]. For the core of who we are, Corey; you are very similar in that. So, yes. Being a dev first company, I thought the developers would appreciate such a title and name for my company. And I have to give a shout out here to Avishai Ish-Shalom, who's my friend from the community who you also know from the DevOpsDays community.Corey: Oh, yeah @nukemberg on Twitter—Sharone: Yes exactly.Corey: For those who are not familiar.Sharone: [laugh]. Yep. He coined the name.Corey: The problem that I found is that people when they start companies or they manage their careers, they don't bias for the things that they're really good at. And it took me a long time to realize this, I finally discovered, “Ah, what am I the best at? That's right, getting myself fired for my personality, so why don't I build a business where that stops being a liability?” So, I started my own company. And I can tell this heroic retcon of what happened, but no, it's because I had nowhere else to go at that point.And would you hire me? Think about this for a minute. You, on the other hand, had options. You are someone with a storied history in community building, in marketing to developers without that either coming across as insincere or that marked condescending accent that so many companies love to have of, “Oh, you're a developer. Let me look at you and get down on my hands and knees like we're going camping and tell a story in ways that actively and passively insult you.”No, you have always gotten that pitch-perfect. The world was your oyster. And for some godforsaken reason, you looked around and decided, “Ah, I'm going to go out independently because you know what I love? Worrying.” Because let's face it, running your own company is an exercise in finding new and exciting things to worry about that 20 minutes ago, you didn't know existed. I say this from my own personal experience. Why would you ever do such a thing?Sharone: [laugh]. That's a great question. It was a long one, but a good one. And I do a thing where I hit the mic a lot because I also have. I can't control my hand motions.Corey: I too speak with my hands. It's fine.Sharone: [laugh]. Yeah, so it's interesting because I wanted to be independent for a really long time. And I wasn't sure, you know, if it was something that I could do if I was a responsible enough adult to even run my own company, if I could make it work, if I could find the business, et cetera. And I left the job in December 2020, and it was the first time that I hadn't figured out what I was doing next yet. And I wanted to take some time off.And then immediately, like, maybe a week after I started to get a lot of, like, kind of people reaching out. And I started to interview places and I started to look into possibly being a co-founder at places and I started to look at all these different options. And then just, I was like, “Well…. This is an opportunity, right? Maybe I should finally—that thing that's gnawing at the back of my head to see if, like, you know if I should go for this dream that I've always wanted, maybe now I can just POC it and see if, you know, it'll work.”And it just, like, kind of exploded on me. It was like there was so much demand, like, I just put a little, like, signal out to the world that this is something that I'm interested in doing, and everyone was like, “Ahh, I need that.” [laugh]. I wanted to take a quarter off and I signed my first clients already on February 1st, which was, like, a month after. I left in December and that—it was crazy. And since then, I've been in business. So, yeah. So, and since then, it's also been a really crazy ride; I got to discover some really exciting companies. So.Corey: How did you get into this? I found myself doing marketing-adjacent work almost entirely by accident. I started the newsletter and this podcast, and I was talking to sponsors periodically and they'd come back with, “Here's the thing we want you to talk about in the sponsor read.” And it's, “Okay, you want to give people a URL to go to that has four sub-directories and entire UTM code… okay, have you considered, I don't know, not?” And because so much of what they were talking about did not resonate.Because I have the engineering background, and it was, I don't understand what your company does and you're spending all your time talking about you instead of my painful problem. Because as your target market, I don't give the slightest of shits about you, I care about my problem, so tell me how you're going to solve my problem and suddenly I'm all ears. Spend the whole time talking about you, and I could not possibly care less and I'll fast-forward through the nonsense. That was my path to it. How did you get into it?Sharone: How did I get into it? It's interesting. So, I started my journey in typical marketing, enterprise B2B marketing. And then at GigaSpaces, we kickstarted the open-source project Cloudify, and that's when I found myself leading this project as the open-source community team leader, building, kind of, the community from the ground floor. And I discovered a whole new world of, like, how to build experience into your marketing, kind of making it really experiential and making sure that everyone has a really, really easy and frictionless way of using your product, and that the product—putting the product at the center and letting it speak for itself. And then you discover this whole new world of marketing where it's—and today, you know, it has more of a name and a title, PLG, and people—it has a whole methodology and practice, but then it was like we were—Corey: PLG? I'm unfamiliar with the acronym. I thought tech was bad for acronyms.Sharone: Right? [laugh]. So, product-led growth. But then, you know, like, kind of wasn't solidified yet. And so, a lot of what we were doing was making sure that developers had a really great experience with the product then it kind of sold itself and marketed itself.And then you understood what they wanted to hear and how they wanted to consume the product and how they wanted it to be and to learn about it and to kind of educate themselves and get into it. And so, a lot of the things that I learned in the context of marketing was very guerilla, right, from the ground up and kind of getting in front of people and in the way they wanted to consume it. And that taught me a lot about how developers consume technology, the different channels that they're involved in, and the different tools that they need in order to succeed, and the different, you know, all the peripheral experience, that makes marketing really, really great. And it's not about what you're selling to somebody; it's making your product shine and making the experience shine, making them ensure that it's a really, really easy and frictionless experience. You know, I like how [Donald Bacon 00:08:00] says it; he calls it, like, mean time to hello world, and that to me is the best kind of marketing, right? When you enable people to succeed very, very quickly.Corey: Yeah, there's something to be said for the ring of authenticity and the rest. Periodically I'll promote guest episodes on this, where it's a sponsored episode where people get up and they talk about what they're working on. And they're like, “Great. So, here's the sales pitch I want to give,” and it's no you won't because first, it won't work. And secondly, I'm sorry, whether it's a promoted episode or not, I will not publish something that isn't good because I have a reputation to uphold here.And people run into challenges an awful lot when they're trying to effectively tell their story. If you have a startup that was founded by an engineer, for example, as so many of these technical startups were, the engineer is often so deeply and profoundly in love with this problem space and the solution and the rest, but if they talk about that, no one cares about the how. I mean, I fix AWS bills, and people don't care—as a general rule—how I do that at all if they're in my target market. They don't care if it's through clever optimization, amazing tooling, doing it on-site, or taking hostages in Seattle. They care about their outcome much more than they ever do about the how.The only people who care about the how are engineers who very often are going to want to build it themselves, or work for you, or start a competitor. And it doesn't resonate in quite the same way. It's weird because all these companies are in slightly different spaces; all of them tend to do slightly different things—or very different things—but so many of the challenges that I see in the way that they're articulating what they do to customers rhymes with one another.Sharone: Yeah. So, I agree completely that developers will talk often about how it works. How it works. How does it work under the hood? What are the bits and bytes, you know?Like, nobody cares about how it works. People care about how will this make my life better, right? How will this improve my life? How will this change my life? [laugh]. As an operations engineer, if I'm, you know, crunching through logs, how will this tool change that? What my days look like? What will my on-call rotation look like? What will—you know, how are you changing my life for the better?So, I think that that's the question. When you learn how to crystallize the answer to that question and you hit it right on the mark—you know, and it takes a long time to understand the market, and to understand the buying persona, and t—and there's so much that you have to do in the background, and so much research you have to do to understand who is that person that needs to have that question answered? But once you do and you crystallize that answer, it lands. And that's the fun part about marketing, really trying to understand the person who's going to consume your product and how you can help them understand that you will make their life better.Corey: Back when I was starting out as a consultant myself, I would tell stories that I had seen in the AWS billing environment, and I occasionally had clients reach out to me, “Hey, why don't you tell our story in public?” It's, “Because that wasn't your story. That was something I saw on six different accounts in the same month. It is something that everyone is feeling.” It's, people think that you're talking about them.So, with that particular mindset on this, without naming specific companies, what themes are you seeing emerging? What are companies getting wrong when they are attempting and failing to market effectively to developers?Sharone: So, exactly what we're talking about in terms of the product pitch, in that they're talking at developers from this kind of marketing speak and this business language that, you know, developers often—you know, unless a company does a really, really good job of translating, kind of, the business value—which they should do, by the way—to engineers, but oftentimes, it's a little bit far from them in the chain, and so it's very hard for them to understand the business fluff. If you talk to them in bits and bytes of this is what my day-to-day developer workflow looks like and if we do these things, it'll cut down the time that I'm working on these things, it'll make these things easier, it'll help streamline whatever processes that are difficult, remove these bottlenecks, and help them understand, like I said, how it improves their life.But the things that I've seen breakdown is also in the authenticity, right? So obviously, the world is built on a lot of the same gimmicks and it's just a matter of whether you're doing it right or not, right? So, there's so much content out there and webcasts and webinars, and I don't know what and podcasts and whatever it is, but a lot of the time, people, their most valuable asset is their time. And if you end up wasting their time, without it being, like, really deeply valuable—if you're going to write content, make sure that there is a valuable takeaway; if you're going to create a webinar, make sure that somebody learned something. That if they're investing their time to join your marketing activities, make sure that they come away with something meaningful and then they'll really appreciate you.And it's the same idea behind the whole DevOpsDays movement with the law of mobility and open spaces that people if they find value, they'll join this open space and they'll participate meaningfully and they'll be a part of your event, and they'll come back to your event from year to year. But if you're not going to provide that tangible value that somebody takes away, and it's like, okay, well, I can practically apply this in my specific tech stack without using your tool, without having to have this very deterministic or specific kind of tech stack that they're talking about. You want to give people something—or even if it is, but even how to do it with or without, or giving them, like, kind of practical tools to try it. Or if there's an open-source project that they can check out first, or some kind of lean utility that gives them a good indication of the value that this will give them, that's a lot more valuable, I think. And practically understandable to somebody who wants to eventually consume your product or use your products.Corey: The way that I see things, at least in the past couple of years, the pandemic has sharpened an awful lot of the messaging that needs to happen. Because in most environments, you're sitting at a DevOpsDays in the front row or whatnot, and it's time for the sponsor talks and someone gets up and starts babbling and wasting your time, most people are not going to get up and leave. Okay, they will in Israel, but in most places, they're not going to get up and leave, whereas in pandemic land, it's you are one tab away from something I actually want—Sharone: Exactly.Corey: To be doing, so if you become even slightly boring, it's not going to go well. So, you have to be on message, you have to be on point or no one cares. People are like, “Oh, well what if we say the wrong thing and people wind up yelling about us on Twitter?” It's like unless it is for something horrifying, you should be so lucky because people are then talking about you. The failure mode isn't that people don't like your product, it's no one talks about it.Sharone: Yeah. No such thing as bad publicity [crosstalk 00:14:32] [laugh]—Corey: Oh, there very much is such a thing is bad publicity. Like, “I could be tweeting about your product most days,” is apparently a version of that, according to some folks. But it's a hard problem to solve for. And one of the things that continually surprises me is the things I'm still learning about this entire industry. The reason that people sponsor this show—and the rates they pay, to be direct—have little bearing to the actual size of the audience—as best we can tell; lies, damn lies, and podcast statistics; if you're listening to this, let me know. I'd love to know if anyone listens to this nonsense—but when you see all of that coming out, why are we able to charge the rates that we do?It's because the long-term value of someone who is going to buy a long-term subscription or wind up rolling out something like ChaosSearch or whatnot that is going to be a fundamental tenet of their product, one prospect becoming a customer pays for anything, I can sell a company, it will sponsor—they can pay me to sponsor for the next ten years, as opposed to the typical mass-market audience where well, I'm here to sling Casper mattresses today or something. It's a different audience and there's a different perception there. People are starting to figure out the value of—in an age where tracking is getting harder and harder to do and attribution will drive you nuts, instead of go where your audience is. Go where the people who care about the problem that you have and will experience that problem are going to hang out. And it always is wild to me to see companies missing out on that.It's, “Okay, so you're going to do a $25 million billboard ad in spotted in airports around the world talking about your company… but looking at your billboard, it makes no sense. I don't understand what it's there for.” Even as a brand awareness play, it fails because your logo is tiny in the corner or something. It's you spent that much money on ads, and maybe a buck on messaging because it seems like with all that attention you just bought, you had nothing worthwhile to say. That's the cardinal sin to me at least.Sharone: Yeah. One thing that I found—and back to our community circuit and things that we've done historically—but that's one thing that, you know, as a person comes from community, I've seen so much value, even from the smaller events. I mean, today, like with Covid and the pandemic and everything has changed all the equilibrium and the way things are happening. But some meetups are getting smaller, face-to-face events are getting smaller, but I've had people telling me that even from small, 30 to 40 people events, they'll go up and they'll do a talk and great, okay, a talk; everybody does talks, but it's like, kind of, the hallway track or the networking that you do after the talk and you actually talk to real users and hear their real problems and you tap into the real community. And some people will tell me like, I had four concrete leads from a 30-person meet up just because they didn't even know that this was a real challenge, or they didn't know that there was a tool that solves this problem, or they didn't understand that this can actually be achieved today.Or there's so many interesting technologies and emerging technologies. I'm privileged to be able to be at the forefront of that and discover it all, and I if I could, I would drop names of all of the awesome companies that work for me, that I work with, and just give them a shout out. But really, there's so many amazing companies doing, like, developer metrics, and all kinds of troubleshooting and failure analysis that's, like, deeply intelligent—and you're going to love this one: I have a Git replacement client apropos to your closing keynote of DevOpsDays 2015—and tapping into the communities and tapping into the real users.And sometimes, you know, it's just a matter of really understanding how developers are working, what processes look like, what workflows look like, what teams look like, and being able to architect your products and things around real use cases. And that you can only discover by really getting in front of actual users, or potential users, and learning from them and feedback loops, and that's the little core behind DevRel and developer advocacy is really understanding your actual users and your consumers, and encouraging them to you know, give you feedback and try things, and beta programs and a million things that are a lot more experiential today that help you understand what your users need, eventually, and how to actually architect that into your products. And that's the important part in terms of marketing. And it's a whole different marketing set. It's a whole different skill set. It's not talking at people, it's actually… ingesting and understanding and hearing and implementing and bringing it into your products.Corey: And it takes time. And you have to make yourself synonymous with a painful problem. And those problems are invariably very point-in-time specific. I don't give a crap about log aggregation today, but in two weeks from now, when I'm trying to chase down 18 different Lambdas function trying to figure out what the hell's broken this week, I suddenly will care very much about log aggregation. Who was that company that's in that space that's doing interesting things? And maybe it's Cribl, for example; they do a lot of stuff in that space and they've been a good sponsor. Great.I start thinking about those things in that light because it is—when I started having these problems, it sticks in your head and it resonates. And there's value and validity to that, but you're never going to be able to attribute that either, which is where people often lose their minds. Because for anything even slightly complicated—you're going to be selling things to big bank—great, good on you. Most of those customers are not going to go and spin up a trial in the dead of night. They're going to hear about you somewhere and think, “Ohh, this is interesting.”They're going to talk about a meeting, they're going to get approval, and at that point, you have long since lost any tracking opportunity there. So, the problem is that by saying it like this, as someone who is a publisher, let's be very clear here, it sounds like you're trying to justify your entire business model. I feel like that half the time, but I've been reassured by people who are experts in doing these things, like, oh, yeah, we have data on this; it's working. So, the alternative is either I accept that they're right or I sit here and arrogantly presume I know more about marketing than people who've devoted their entire careers to it. I'm not that bold. I am a white guy in tech, but not that much.Sharone: Yeah, I mean, the DevRel measurement problem is a known problem. We have people like [unintelligible 00:20:21] who have written about it. We have [Sarah Drasner 00:20:23], we have a million people that have written really, really great content about how do you really measure DevRel and the quality. And one of the things that I liked, Philipp Krenn, the dev advocate at Elastic once said in one of his talks that, you know, “If you're measuring your developer advocates on leads, you're a marketing organization. If you're measuring them on revenue, you're a sales organization. It's about reach, engagement, and awareness, and a lot of things that it's much, much harder to measure.”And I can say that, like, once upon a time, I used to try and attribute it at Cloudify. Like, I remember thinking, like, “Okay, maybe I could really track this back to, you know, the first touch that I actually had with this user.” It's really, really difficult, but I do remember, like, when we used to go out into the events and we were really active in the OpenStack community, in the DevOps community, and many other things, and I remember, like, even after events, like, you get all those lead gen emails. All I would say now is, like, “Hey, if you missed us at the booth, you know, and you want still want a t-shirt, you know, reach out and I'll ship it to you.” And some of those eventually, after we continued the relationship, and we, you know, when we were friends and community friends, six months later, when they moved to their next role at their next job, they were like, “Oh, now I have an opportunity to use Cloudify and I'm going to check it out.”And it's very long relationship that you have to cultivate. It has to be, you know, mutual. You have to be, you have to give be giving something and eventually is going to come back to you. Good deeds come back to you. So, I—that's my credo, by the way, good deeds come back to you. I believe in that and I try to live by that.Corey: This episode is sponsored in parts by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on-premises, private cloud, and they just announced a fully-managed service on AWS and Azure called BigAnimal, all one word. Don't leave managing your database to your cloud vendor because they're too busy launching another half-dozen managed databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications—including Oracle—to the cloud. To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: So, I have one last question for you and it is pointed and the reason I buried it this deep in the episode is so that if I open with it, I will get letters and I'm hoping to get fewer of them. But I met you, again, at DevOpsDays Tel Aviv, and it was glorious. And then you said, “This is fun. Come help me organize it next year.”And I, like an idiot said, “Sure, that sounds awesome because I love going to conferences and it's great. So, what's involved?” “Oh, a whole bunch of meetings.” “Okay, great.” “And planning”—things I'm terrible at—“Okay.” And then the big day finally arrives where, “Great, when do we get to get on stage and tell a story?” Like, “That's the neat part. We don't.” So, I have to ask, given that it is all behind-the-scenes work that is fairly thankless unless you really screw it up because then it's very visible, what is the point of being so involved in the community?Sharone: Wow, that's a big question, Corey.Corey: It really is.Sharone: [laugh].Corey: Because you've been involved in community for a long time and you're very good at it.Sharone: It's true. It's true. Appreciate it, thank you. So, for me, first of all, I enjoy, kind of, the people aspect of it, absolutely. And that people aspect of it actually has played out in so many different ways.Corey: Oh, you mean great people, and also me.Sharone: [laugh]. Particularly you, Corey, and we will bring you back. [laugh]. And we will make sure you chop wood and carry water because eventually it'll fill your soul, you'll see. [laugh] one of the things that really I have had the privilege and honor, and having come out of, like, kind of all my community work is really the network I've built and the people that I've met.And I've learned so much and I've grown so much, but I've also had the opportunity to connect people, connect things that you wouldn't imagine, un—seemingly-related things. So, there are so many friends of mine that have grown up with me in this community, it's been already ten years now, and a lot of folks have now been going on to new adventures and are looking to kickstart their new startup and I can connect them to this investor, I can connect them to this other person who is maybe a good, you know, partner for their startup, and hiring opportunities, and something—I've had this, like, privilege of kind of being able to connect Israel to the outer world and other things and the global kind of community, and also bring really intelligent folks into the community. And this has just created this amazing flywheel of opportunity that I'm really happy to be at the center of. And I think I've grown as a person, I think our community has grown, has learned, and there's a lot of value in that, I think, yeah. We got to meet wonderful folks like you, Corey. [laugh].Corey: It has its moments. Again, you're one of those rarities in that it's almost become a trope in VC land where VCs always like, “How may I be useful?” And it's this self-serving transparent thing. Every single time you have deigned to introduce me to someone, it's been a productive conversation and I'm always glad I took the meeting. That is no small thing.A lot of people say, “I'm good at community,” which is sort of cover for, “I'm not good at anything,” but in your case, it—Sharone: [laugh]. [I'm an entrepreneur 00:24:48].—Corey: Is very much not true. Oh, yeah. I'm a big believer that ‘entrepreneur' and ‘hero' and other terms like that are things people call you; you don't call yourself that. It always feels weird for, “Oh, he's an entrepreneur.” It's like, that's a pretty lofty word for shitposting, but okay, we'll roll with it.It doesn't work that way. You've clearly invested long-term in a building reputation for yourself by building a name for yourself in the space, and I know that whenever you reach out to me as a result, you are not there to waste my time or shill some bullshit. It is always something that is going to, even if I don't love every aspect of it or agree with the core of the message you're sending, great, it is never not going to be worth my time, which is why I'm so glad I got the chance to talk to you this show.Sharone: I appreciate that. It's something that I really believe in, I don't want to waste people's time and I really only will connect folks or only really will reach out to someone if I do think that there's something meaningful for both sides. It's never only what's in it for me, also. I also want to make sure that there's something in it for the other person and it's something that makes sense and it's meaningful for both sides. I've had the opportunity of meeting such interesting folks, and sometimes it's just like, “You must meet. [laugh]. You will love each other.” You will have so much to do together or it's so much collaboration opportunity.And so yeah, I really am that type of person. And I'll even say from a personal perspective, you know, I know a lot of people, and I've even been asked from the flip side, “Okay, is this a toxic manager? Or is this a, you know, a good hire? Is this”—and I tried to provide really authentic input so people make the right decisions, or make, you know, the right contacts, or make—and that's something I really value. And I managed to build trust with a lot of really great folks—Corey: And also me—Sharone: —and it's come back to me, also. And—[laugh] and particularly you, again. [laugh].Corey: If people want to learn more about how you see the world and the space and otherwise bask in your wisdom, where's the best place to find you?Sharone: So, I'm on Twitter as @shar1z, which is SharoneZ. Basically, everyone thinks it's such a smart, or I don't know what, like, or an esoteric screen name. And I'm like, no, it's just my name, I just—the O-N-E is… the one. [laugh].So yes, shar1z on Twitter, but also my website, rtfmplease.dev, you can reach out, there's a contact form there. You can find me on the web anywhere—LinkedIn. Reach out, I answer almost all my DMs when I can. It's very rare that I don't answer DMs. Maybe there'll be a slight lag, but I do. And I really do like when folks reach out to me. I do like it when people try and make contact.Corey: And you can also be found, of course, wherever find DevOps products are sold, on stage apparently.Sharone: [laugh]. The DevOps community, that's right. @TLVCommunity, @DevOpsDaysTLV—don't out me. All those are—yes, those are also handles that I run on Twitter, it's true.Corey: Excellent.Sharone: So, when you see them all retweeting the same tweet, yes, it's happening within same five minutes, it's me.Corey: Oh, that would have made it way easier to go viral. My God, I should have just thought of that earlier.Sharone: [laugh].Corey: Thank you so much for your time. I appreciate it.Sharone: Thank you, Corey, for having me. It's been a privilege and honor being on your show and I really do think that you are doing wonderful things in the cloud space. You're teaching us, and we're all learning, and you—keep up the good work.Corey: Well, thank you. I appreciate that.Sharone: I also want to add that on proposed marketing and whatever, I do actually listen to all of your openings of all of your shows because they're not fluffy and I like that you do, like, kind of a deep explanation, a deep technical explanation of what your sponsoring product does, and it gives a lot more insight into why is this important. So, I think you're doing that right. So, anybody who's sponsoring this show, listen. Corey knows what he's doing.Corey: Well, thank you. I appreciate that. Yay, “I know what I'm doing.” That one's going in the testimonial kit. My God.Sharone: [laugh]. That's the name of this episode, “Corey knows what he's doing.”Corey: We're going to roll with it, you know. No take-backsies. Sharone Zitzman, Chief Manual Reader at RTFM Please. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review of your podcast platform of choice, or if it's on the YouTubes smash the like and subscribe buttons, whereas if you've hated this show, exact same thing—five-star review wherever you happen to find it, smash both the buttons—but also leave an insulting comment telling me that I'm completely wrong which then devolves into an 18-page diatribe about exactly how your nonsense, bullshit product is built and works.Sharone: [laugh].Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About RafalRafal is Serverless Engineer at Stedi by day, and Dynobase founder by night - a modern DynamoDB UI client. When he is not coding or answering support tickets, he loves climbing and tasting whiskey (not simultaneously).Links Referenced:Company Website: https://dynobase.dev TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored by our friends at Revelo. Revelo is the Spanish word of the day, and its spelled R-E-V-E-L-O. It means “I reveal.” Now, have you tried to hire an engineer lately? I assure you it is significantly harder than it sounds. One of the things that Revelo has recognized is something I've been talking about for a while, specifically that while talent is evenly distributed, opportunity is absolutely not. They're exposing a new talent pool to, basically, those of us without a presence in Latin America via their platform. It's the largest tech talent marketplace in Latin America with over a million engineers in their network, which includes—but isn't limited to—talent in Mexico, Costa Rica, Brazil, and Argentina. Now, not only do they wind up spreading all of their talent on English ability, as well as you know, their engineering skills, but they go significantly beyond that. Some of the folks on their platform are hands down the most talented engineers that I've ever spoken to. Let's also not forget that Latin America has high time zone overlap with what we have here in the United States, so you can hire full-time remote engineers who share most of the workday as your team. It's an end-to-end talent service, so you can find and hire engineers in Central and South America without having to worry about, frankly, the colossal pain of cross-border payroll and benefits and compliance because Revelo handles all of it. If you're hiring engineers, check out revelo.io/screaming to get 20% off your first three months. That's R-E-V-E-L-O dot I-O slash screaming.Corey: The company 0x4447 builds products to increase standardization and security in AWS organizations. They do this with automated pipelines that use well-structured projects to create secure, easy-to-maintain and fail-tolerant solutions, one of which is their VPN product built on top of the popular OpenVPN project which has no license restrictions; you are only limited by the network card in the instance. To learn more visit: snark.cloud/deployandgoCorey: Welcome to Screaming in the Cloud. I'm Corey Quinn. It's not too often that I wind up building an episode here out of a desktop application. I've done it once or twice, and I'm sure that the folks at Microsoft Excel are continually hoping for an invite to talk about things. But we're going in a bit of a different direction today. Rafal Wilinski is a serverless engineer at Stedi and, in apparently what is the job requirement at Stedi, he also has a side project that manifests itself as a desktop app. Rafal, thank you for joining me today. I appreciate it.Rafal: Yeah. Hi, everyone. Thanks for having me, Corey.Corey: I first heard about you when you launched Dynobase, which is awesome. It sounds evocative of dinosaurs unless you read it, then it's D-Y-N-O, and it's, “Ah, this sounds a lot like DynamoDB. Let me see what it is.” And sure enough, it was. As much as I love misusing things as databases, DynamoDB is actually a database that is decent and good at what it does.And please correct me if I get any of this wrong, but Dynobase is effectively an Electron app that you install, at least on a Mac, in my case; I don't generally use other desktops, that's other people's problems. And it provides a user-friendly interface to DynamoDB that is not actively hostile to the customer.Rafal: Yeah, exactly. That was the goal. That's how I envisioned it, and I hope I executed correctly.Corey: It was almost prescient in some ways because they recently redid the DynamoDB console in AWS to actively make it worse, to wind up working with individual items, to modify things. It feels like they are validating your market for you by, “Oh, we really like Dynobase. How do we drive more traffic to it? We're going to make this thing worse.” But back then when you first created this, the console was his previous version. What was it that inspired you to say, “You know what I'm going to build? A desktop application for a cloud service.” Because on the surface, it seems relatively close to psychotic, but it's brilliant.Rafal: [laugh]. Yeah, sure. So, a few years ago, I was freelancing on AWS. I was jumping between clients and my side projects. That also involved jumping between regions, and AWS doesn't have a good out-of-the-box solution for switching your accounts and switching your regions, so when you want it to work on your client table in Australia and simultaneously on my side project in Europe, there was no other solution than to have two browser windows open or to, even, browsers open.And it was super frustrating. So, I was like, hey, “DynamoDB has SDK. Electron is this thing that allows you to make a desktop application using HTML and JS and some CSS, so maybe I can do something with it.” And I was so naive to think that it's going to be a trivial task because it's going to be—come on, it's like, a couple of SDK calls, displaying some lists and tables, and that's pretty much it, right?Corey: Right. I use Retool as my system to build my newsletter every week, and that is the front-end I use to interact with DynamoDB. And it's great. It has a table component that just—I run a query that, believe it or not, is a query, not a scan—I know, imagine that, I did something slightly right this one time—and it populates things for the current issue into it, and then I basically built a CRUD API around it and have components that let me update, delete, remove, the usual stuff. And it's great, it works for my purposes, and it's fine.And that's what I use most of the time until I, you know, hit an edge case or a corner case—because it turns out, surprise everyone, I'm bad at programming—and I need to go in and tweak the table myself manually. And that's where Dynobase, at least for my use case, really comes into its own.Rafal: Good to hear. Good to hear. Yeah, that was exactly same case why I built it because yeah, I was also, a few years ago, I started working on some project which was really crazy. It was before AppSync times. We wanted to have GraphQL serverless API using single table design and testing principles [unintelligible 00:04:38] there.So, we've been verifying many things by just looking at the contents of the table, and sometimes fixing them manually. So, that was also the thing that motivated me to make the editing experience a little bit better.Corey: One thing I appreciate about the application is that it does things right. I mean, there's no real other way to frame that. When I fire up the application myself and I go to the account that I've been using it with—because in this case, there's really only one account that I have that contains the data that I spent that my time working with—and I get access to it on my machine via Granted, which because it's a federated SSO login. And it says, “Ah, this is an SSL account. Click here to open the browser tab and do the thing.”I didn't have to configure Dynobase. It is automatically reading my AWS config file in my user directory. It does a lot of things right. There's no duplication of work. From my perspective. It doesn't freak out because it doesn't know how SSO works. It doesn't have run into these obnoxious edge case problems that so many early generation desktop interfaces for AWS things seem to.Rafal: Wow, it seems like it works for you even better than for me. [laugh].Corey: Oh, well again, how I get into accounts has always been a little weird. I've ranted before about Granted, which is something that Common Fate puts out. It is a binary utility that winds up logging into different federated SSO accounts, opens them in Firefox containers so you could have you know, two accounts open, side-by-side. It's some nice affordances like that. But it still uses the standard AWS profile syntax which Dynobase does as well.There are a bunch of different ways I've logged into things, and I've never experienced friction [unintelligible 00:06:23] using Dynobase for this. To be clear, you haven't paid me a dime. In fact, just the opposite. I wind up paying my monthly Dynobase subscription with a smile on my face. It is worth every penny, just because on those rare moments when I have to work with something odd in DynamoDB, it's great having the tool.I want to be very clear here. I don't recall what the current cost on this is, but I know for a fact it is more than I spend every month on DynamoDB itself, which is fine. You pay for utility, not for the actual raw cost of the underlying resources on it. Some people tend to have issues with that and I think it's the wrong direction to go in.Rafal: Yeah, exactly. So, my logic was that it's a productivity improvement. And a lot of programmers are simply obsessed with productivity, right? We tend to write those obnoxious nasty Bash and Python scripts to automate boring tasks in our day jobs. So, if you can eliminate this chore of logging to different AWS accounts and trying to find them, and even if it takes, like, five or ten seconds, if I can shave that five or ten seconds every time you try to do something, that over time accumulates into a big number and it's a huge time investment. So, even if you save, like, I don't know, maybe one hour a month or one hour a quarter, I think it's still a fair price.Corey: Your pricing is very interesting, and the reason I say that is you do not have a free tier as such, you have a free seven-day trial, which is great. That is the way to do it. You can sign up with no credit card, grab the thing, and it's awesome. Dynobase.dev for folks who are wondering.And you have a solo yearly plan, which is what I'm on, which is $9 a month. Which means that you end up, I think, charging me $108 a year billed annually. You have a solo lifetime option for 200 bucks—and I'm going to fight with you about that one in a second; we're going to come back to it—then you have a team plan that is for I think for ten licenses at 79 bucks a month, and for 20 licenses it's 150 bucks a month. Great. And then you have an enterprise option for 250 a month, the end. Billed annually. And I have problems with that, too.So, I like arguing with pricing, I [unintelligible 00:08:43] about pricing with people just because I find that is one of those underappreciated aspects of things. Let's start with my own decisions on this, if I may. The reason that I go for the solo yearly plan instead of a lifetime subscription of I buy this and I get to use it forever in perpetuity. I like the tool but, like, the AWS service that underlies it, it's going to have to evolve in the fullness of time. It is going to have to continue to support new DynamoDB functionality, like the fact that they have infrequent access storage classes now, for tables, as an example. I'm sure they're coming up with other things as well, like, I don't know, maybe a sane query syntax someday. That might be nice if they ever built one of those.Some people don't like the idea of a subscription software. I do just because I like the fact that it is a continual source of revenue. It's not the, “Well, five years ago, you paid me that one-off thing and now you expect feature enhancements for the rest of time.” How do you think about that?Rafal: So, there are a couple of things here. First thing is that the lifetime support, it doesn't mean that I will be always implementing to my death all the features that are going to appear in DynamoDB. Maybe there is going to be a some feature and I'm not going to implement it. For instance, it's not possible to create the global tables via Dynobase right now, and it won't be possible because we think that majority of people dealing with cloud are using infrastructure as a code, and creating tables via Dynobase is not a super useful feature. And we also believe that it's not going to break even without support. [laugh]. I know it sounds bad; it sounds like I'm not going to support it at some point, but don't worry, there are no plans to discontinue support [crosstalk 00:10:28]—Corey: We all get hit by buses from time to time, let's be clear.Rafal: [laugh].Corey: And I want to also point out as well that this is a graphical tool that is a front-end for an underlying AWS service. It is extremely convenient, there is tremendous value in it, but it is not critical path as if suddenly I cannot use Dynobase, my production app is down. It doesn't work that way, in the sense—Rafal: Yes.Corey: Of a SaaS product. It is a desktop application. And huge fan of that as well. So, please continue.Rafal: Yeah, exactly—Corey: I just want to make sure that I'm not misleading people into thinking it's something it's not here. It's, “Oh, that sounds dangerous if that's critical pa”—yeah, it's not designed to be. I imagine, at least. If so it seems like a very strange use case.Rafal: Yeah. Also, you have to keep in mind that AWS isn't basically introducing breaking changes, especially in a service that is so popular as DynamoDB. I cannot imagine them, like, announcing, like, “Hey, in a month, we are going to deprecate this API, so you'd better start, you know, using this new API because this one is going to be removed.” I think that's not going to happen because of the millions of clients using DynamoDB actively. So, I think that makes Dynobase safe. It's built on a rock-solid foundation that is going to change only additively. No features are going to be just being removed.Corey: I think that there's a direction in a number of at least consumer offerings where people are upset at the idea of software subscriptions, the idea of why should I pay in perpetuity for a thing? And I want to call out my own bias here. For something like this, where you're charging $9 a month, I do not care about the price, truly I don't. I am a price inflexible customer. It could go and probably as high as 50 bucks a month and I would neither notice nor care.That is probably not the common case customer, and it's certainly not over in consumer-land. I understand that I am significantly in a privileged position when it comes to being able to acquire the tools that I need. It turns out compared to the AWS bill I have to deal with, I don't have to worry about the small stuff, comparatively. Not everyone is in that position, so I am very sympathetic to that. Which is why I want to deviate here a little bit because somewhat recently, Dynobase showed up on the AWS Marketplace.And I can go into the Marketplace now and get a yearly subscription for a single seat for $129. It is slightly more than buying it directly through your website, but there are some advantages for many folks in getting it on the Marketplace. AWS is an approved vendor, for example, so there's no procurement dance. It counts toward your committed spend on contracts if someone is trying to wind up hitting certain levels of spend on their EDP. It provides a centralized place to manage things, as far as those licenses go when people are purchasing it. What was it that made you decide to put this on the Marketplace?Rafal: So, this decision was pretty straightforward. It's just, you know, yet another distribution channel for us. So, imagine you're a software engineer that works for a really, really big company and it's super hard to approve some kind of expense using traditional credit card. You basically cannot go to my site and check out with a company credit card because of the processes, or maybe it takes two years. But maybe it's super easy to click this subscribe on your AWS account. So yeah, we thought that, hey, maybe it's going to unlock some engineers working at those big corporations, and maybe this is the way that they are going to start using Dynobase.Corey: Are you seeing significant adoption yet? Or is it more or less a—it's something that's still too early to say? And beyond that, are you finding that people are discovering the product via the AWS Marketplace, or is it strictly just a means of purchasing it?Rafal: So, when it comes to discovering, I think we don't have any data about it yet, which is supported by the fact that we also have zero subscriptions from the Marketplace yet. But it's also our fault because we haven't actually actively promoted the fact, apart from me sending just a tweet on Twitter, which is in [crosstalk 00:14:51]—Corey: Which did not include a link to it as well, which means that Google was our friend for this because let's face it, AWS Marketplace search is bad.Rafal: Well, maybe. I didn't know. [laugh]. I was just, you know, super relieved to see—Corey: No, I—you don't need to agree with that statement. I'm stating it as a fact. I am not a fan of Marketplace search. It irks me because for whatever reason whenever I'm in there looking for something, it does not show me the things I'm looking for, it shows me the biggest partners first that AWS has and it seems like the incentives are misaligned. I'm sure someone is going to come on the show to yell about me. I'm waiting for your call.Rafal: [laugh].Corey: Do you find that if someone is going to purchase it, do you have a preference that they go directly, that they go through the Marketplace? Is there any direction for you that makes more sense than another?Rafal: So ideally, would like to continue all the customers to purchase the software using the classical way, using the subscriptions for our website because it's just one flow, one system, it's simpler, it's cleaner, but we want it to give that option and to have more adoption. We'll see if that's going to work.Corey: I was going to say there were two issues I had with the pricing. That was one of them. The other is at the high end, the enterprise pricing being $250 a month for unlimited licenses, that doesn't feel like it is the right direction, and the reason I say that is a 50-person company would wind up being able to spend 250 bucks a month to get this for their entire team, and that's great and they're happy. So, could AWS or Coca-Cola, and at that very high level, it becomes something that you are signing up for significant amount of support work, in theory, or a bunch of other directions.I've always found that from where I stand, especially dealing with those very large companies with very specific SLA requirements and the rest, the pricing for enterprise that I always look for as the right answer for my mind is ‘click here to contact us.' Because procurement departments, for example, we want this, this, this, this, and this around data guarantees and indemnities and all the rest. And well, yeah, that's going to be expensive. And well, yeah. We're a procurement company at a Fortune 50. We don't sign contracts that don't have two commas in them.So, it feels like there's a dialing it in with some custom optionality that feels like it is signaling to the quote-unquote, ‘sophisticated buyer,' as patio11 likes to say on Twitter from time to time, that might be the right direction.Rafal: That's really good feedback. I haven't thought about it this way, but you really opened my eyes on this issue.Corey: I'm glad it was helpful. The reason I think about it this way is that more and more I'm realizing that pricing is one of the most key parts of marketing and messaging around something, and that is not really well understood, even by larger companies with significant staff and full marketing teams. I still see the pricing often feels like an afterthought, but personally, when I'm trying to figure out is this tool for me, the first thing I do is—I don't even read the marketing copy of the landing page; I look for the pricing tab and click because if the only prices ‘call for details,' I know, A, it's going to be expensive, be it's going to be a pain in the neck to get to use it because it's two in the morning; I'm trying to get something done. I want to use it right now. If I had to have a conversation with your sales team first, that's not going to be cheap and it's not going to be something I'm going to be able to solve my problem this week. And that is the other end of it. I yell at people on both sides on that one.Rafal: Okay.Corey: Again, none of this stuff is intuitive; all of this stuff is complicated, and the way that I tend to see the world is, granted, a little bit different than the way that most folks who are kicking around databases and whatnots tend to view the world. Do you have plans in the future to extend Dynobase beyond strictly DynamoDB, looking to explore other fine database options like Redis, or MongoDB, or my personal favorite Route 53 TXT records?Rafal: [laugh]. Yeah. So, we had plans. Oh, we had really big plans. We felt that we are going to create a second JetBrains company. We started analyzing the market when it comes to MongoDB, when it comes to Cassandra, when it comes to Redis. And our first pick was Cassandra because it seemed, like, to have really, really similar structure of the table.I mean, it's also no secret it also has a primary index, secondary global indexes, and things like that. But as always, reality surprises us over the amount of detail that we cannot see from the very top. And it isn't as simple as just an install AWS SDK and install Cassandra Connector on—or Cassandra SDK and just roll with that. It requires a really big and significant investment. And we decided to focus just on one thing and nail this one thing and do this properly.It's like, if you go into the cloud, you can try to build a service that is agnostic, it's not using the best features of the cloud. And you can move your containers, for instance, across the clouds and say, “Hey, I'm cloud-agnostic,” but at the same time, you're missing out all the best features. And this is the same way we thought about Dynabase. Hey, we can provide an agnostic core, but then the agnostic application isn't going to be as good and as sophisticated as something tailored specifically for the needs of this database and user using this exact database.Corey: This episode is sponsored in parts by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on premises, private cloud, and they just announced a fully managed service on AWS and Azure called BigAnimal, all one word.Don't leave managing your database to your cloud vendor because they're too busy launching another half dozen manage databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications, including Oracle, to the cloud.To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: Some of the things that you do just make so much sense that I get actively annoyed that there aren't better ways to do it and other places for other things. For example, when I fire up a table in a particular region within Dynobase, first it does a scan, which, okay, that's not terrible. But on some big tables, that can get really expensive. But you cap it automatically to a thousand items. And okay, great.Then it tells me, how long did it take? In this case because, you know, I am using on-demand and the rest and it's a little bit of a pokey table, that scan took about a second-and-a-half. Okay. You scanned a thousand items. Well, there's a lot more than a thousand items in this table. Ah, you limited it, so you didn't wind up taking all that time.It also says that it took 51-and-a-half RCUs—or Read Credit Units—because you know, why use normal numbers when you're AWS and doing pricing dimensions on this stuff.Rafal: [laugh].Corey: And to be clear, I forget the exact numbers for reads, but it's something like a million read RCUs cost me a dollar or something like that. It is trivial; it does not matter, but because it is consumption-based pricing, I always live in a little bit of a concern that, okay, if I screw up and just, like, scan the entire 10-megabyte table every time I want to make an operation here, and I make a lot of operations in the course of a week, that's going to start showing up in the bill in some really unfortunate ways. This sort of tells me as an ongoing basis of what it is that I'm going to wind up encountering.And these things are all configurable, too. The initial stream limit that you have configured as a thousand. I can set that to any number I want if I think that's too many or too few. You have a bunch of pagination options around it. And you also help people build out intelligent queries, [unintelligible 00:22:11] can export that to code. It's not just about the graphical interface clickety and done—because I do love my ClickOps but there are limits to it—it helps formulate what kind of queries I want to build and then wind up implementing in code. And that is no small thing.Rafal: Yeah, exactly. This is how we also envision that. The language syntax in DynamoDB is really… hard.Corey: Awful. The term is awful.Rafal: [laugh]. Yeah, especially for people—Corey: I know, people are going to be mad at me, but they're wrong. It is not intuitive, it took a fair bit of wrapping my head around. And more than once, what I found myself doing is basically just writing a thin CRUD API in Lambda in front of it just so I can query it in a way that I think about it as opposed to—now I'm not even talking changing the query modeling; I just want better syntax. That's all it is.Rafal: Yeah. You also touch on modeling; that's also very important thing, especially—or maybe even scan or query. Suppose I'm an engineer with tens years of experience. I come to the DynamoDB, I jump straight into the action without reading any of the documentation—at least that's my way of working—and I have no idea what's the difference between a scan and query. So, in Dynobase, when I'm going to enter all those filtering parameters into the UI, I'm going to hit scan, Dynobase is automatically going to figure out for you what's the best way to query—or to scan if query is not possible—and also give you the code that actually was behind that operation so you can just, like, copy and paste that straight to your code or service or API and have exactly the same result.So yeah, we want to abstract away some of the weird things about DynamoDB. Like, you know, scan versus query, expression attribute names, expression attribute values, filter, filtering conditions, all sorts of that stuff. Also the DynamoDB JSON, that's also, like, a bizarre thing. This JSON-type thing we should get out of the box, we also take care of that. So, yeah. Yeah, that's also our mission to make the DynamoDB as approachable as possible. Because it's a great database, but to truly embrace it and to truly use it, it's hard.Corey: I want to be clear, just for folks who are not seeing some of the benefits of it the way that I've described it thus far. Yes, on some level, it basically just provides a attractive, usable interface to wind up looking at items in a DynamoDB table. You can also use it to wind up refining queries to look at very specific things. You can export either a selection or an entire table either to a local file—or to S3, which is convenient—but it goes beyond on that because once you have the query dialed in and you're seeing the things you want to see, there's a generate code button that spits it out in—for Python, for JavaScript, for Golang.And there are a few things that the AWS CLI is coming soon, according to the drop-down itself. Java; ooh, you do like pain. And Golang for example, it effectively exports the thing you have done by clicking around as code, which is, for some godforsaken reason, anathema to most AWS services. “Oh, you clicked around to the console to do a thing. Good job. Now, throw it all away and figure out how to do it in code.” As opposed to, “Here's how to do what you just did programmatically.” My God, the console could be the best IDE in the world, except that they don't do it for some reason.Rafal: Yeah, yeah.Corey: And I love the fact that Dynobase does.Rafal: Thank you.Corey: I'm a big fan of this. You can also import data from a variety of formats, export data, as well. And one of the more obnoxious—you talk about weird problems I have with DynamoDB that I wish to fix: I would love to move this table to a table in a different AWS account. Great, to do that, I effectively have to pause the service that is in front of this because I need to stop all writes—great—export the table, take the table to the new account, import the table, repoint the code to talk to that thing, and then get started again. Now, there are ways to do it without that, and they all suck because you have to either write a shim for it or you have to wind up doing a stream that winds up feeding from one to the other.And in many cases, well okay, I want to take the table here, I do a knife-edge cutover so that new rights go to the new thing, and then I just want to backfill this old table data into it. How do I do that? The official answer is not what you would expect it to be, the DynamoDB console of ‘import this data.' Instead, it's, “Oh, use AWS Glue to wind up writing an ETL function to do all of this.” And it's… what? How is that the way to do these things?There are import and export buttons in Dynobase that solve this problem beautifully without having to do all of that. It really is such a different approach to thinking about this, and I am stunned that this had to be done as a third party. It feels like you were using the native tooling and the native console the same way the rest of us do, grousing about it the same way the rest of us do, and then set out to fix it like none of us do. What was it that finally made you say, “You know, I think there's a better way and I'm going to prove it.” What pushed you over the edge?Rafal: Oh, I think I was spending, just, hours in the console, and I didn't have a really sophisticated suite of tests, which forced me [unintelligible 00:27:43] time to look at the data a lot and import data a lot and edit it a lot. And it was just too much. I don't know, at some point I realized, like, hey, there's got to be a better way. I browsed for the solutions on the internet; I realized that there is nothing on the market, so I asked a couple of my friends saying like, “Hey, do you also have this problem? Is this also a problem for you? Do you see the same challenges?”And basically, every engineer I talked to said, “Yeah. I mean, this really sucks. You should do something about it.” And that was the moment I realized that I'm really onto something and this is a pain that I'm not alone. And so… yeah, that gave me a lot of motivation. So, there was a lot of frustration, but there was also a lot of motivation to push me to create a first product in my life.Corey: It's your first product, but it does follow an interesting pattern that seems to be emerging, Cloudash—Tomasz and Maciej—wound up doing that as well. They're also working at Stedi and they have their side project which is an Electron-based desktop application that winds up, we're interfacing with AWS services. And it's. What are your job requirements over at Stedi, exactly?People could be forgiven for seeing these things and not knowing what the hell EDI is—which guilty—and figure, “Ah, it's just a very fancy term for a DevRels company because they're doing serverless DevRel as a company.” It increasingly feels an awful lot like that.j, what's going on over there where that culture just seems to be an emergent property?Rafal: So, I feel like Stedi just attracts a lot of people that like challenges and the people that have a really strong sense of ownership and like to just create things. And this is also how it feels inside. There is plenty of individuals that basically have tons of energy and motivation to solve so many problems not only in Stedi, but as you can see also outside of Stedi, which is a result—Cloudash is a result, the mapping tool from Zack Charles is also a result, and Michael Barr created a scheduling service. So, yeah, I think the principles that we have at Stedi basically attract top-notch builders.Corey: It certainly seems so. I'm going to have to do a little more digging and see what some of those projects are because they're new to me. I really want to thank you for taking so much time to speak with me about what you're building. If people want to learn more or try to kick the tires on Dynobase which I heartily recommend, where should they go?Rafal: Go to dynobase.dev, and there's a big download button that you cannot miss. You download the software, you start it. No email, no credit card required. You just run it. It scans your credentials, profiles, SSOs, whatever, and you can play with it. And that's pretty much it.Corey: Excellent. And we will put a link to that in the [show notes 00:30:48]. Thank you so much for your time. I really appreciate it.Rafal: Yeah. Thanks for having me.Corey: Rafal Wilinski, serverless engineer at Stedi and creator of Dynobase. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice—or a thumbs up and like and subscribe buttons on the YouTubes if that's where you're watching it—whereas if you've hated this podcast, same thing—five-star review, hit the buttons and such—but also leave an angry, bitter comment that you're not going to be able to find once you write it because no one knows how to put it into DynamoDB by hand.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About CarlaCarla Stickler is a professional multi-hyphenate advocating for the inclusion of artists in STEM. Currently, she works as a software engineer at G2 in Chicago. She loves chatting with folks interested in shifting gears from the arts to programming and especially hopes to get more women into the field. Carla spent over 10 years performing in Broadway musicals, most notably, “Wicked,” “Mamma Mia!” and “The Sound of Music.” She recently made headlines for stepping back into the role of Elphaba on Broadway for a limited time to help out during the covid surge after not having performed the role for 7 years. Carla is passionate about reframing the narrative of the “starving artist” and states, “When we choose to walk away from a full-time pursuit of the arts, it does not make us failed artists. The possibilities for what we can do and who we can be are unlimited.”Links Referenced: G2: https://www.g2.com/ Personal website: https://carlastickler.com Instagram: https://www.instagram.com/sticklercarla/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by Honeycomb. When production is running slow, it's hard to know where problems originate. Is it your application code, users, or the underlying systems? I've got five bucks on DNS, personally. Why scroll through endless dashboards while dealing with alert floods, going from tool to tool to tool that you employ, guessing at which puzzle pieces matter? Context switching and tool sprawl are slowly killing both your team and your business. You should care more about one of those than the other; which one is up to you. Drop the separate pillars and enter a world of getting one unified understanding of the one thing driving your business: production. With Honeycomb, you guess less and know more. Try it for free at honeycomb.io/screaminginthecloud. Observability: it's more than just hipster monitoring.Corey: What if there were a single place to get an inventory of what you're running in the cloud that wasn't "the monthly bill?" Further, what if there were a way to compare that inventory to what you were already managing via Terraform, Pulumi, or CloudFormation, but then automatically add the missing unmanaged or drifted parts to it? And what if there were a policy engine to immediately flag and remediate a wide variety of misconfigurations? Well, stop dreaming and start doing; visit snark.cloud/firefly to learn more.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn, there seems to be a trope in our industry that the real engineers all follow what more or less looks like the exact same pattern, where it's you wind up playing around with computers as a small child and then you wind up going to any college you want—as long as it's Stanford—and getting a degree in anything under the sun—as long as it's computer science—and then all of your next jobs are based upon how well you can re-implement algorithms on the whiteboard. A lot of us didn't go through that path. We wound up finding our own ways to tech. My guest today has one of the more remarkable stories that I've come across. Carla Stickler is a software engineer at G2. Carla, thank you for agreeing to suffer my slings and arrows today. It's appreciated.Carla: Thanks so much for having me, Corey.Corey: So, before you entered tech—I believe this is your first job as an engineer and as of the time we're recording this, it's been just shy of a year that you've done in the role. What were you doing before now?Carla: Oh, boy, Corey. What was I doing? I definitely was not doing software engineering. I was a Broadway actress. So, I spent about 15 years in New York doing musical theater, touring around the country and Asia in big Broadway shows. And that was pretty much all I did.I guess, I also was a teacher. I was a voice teacher and I taught voice lessons, and I had a studio and I taught it a couple of faculties in New York. But I was one hundred percent ride-or-die, like, all the way to the end musical theater or bust, from a very, very early age. So, it's been kind of a crazy time changing careers. [laugh].Corey: What inspired that? I mean, it doesn't seem like it's a common pattern of someone who had an established career as a Broadway actress to wake up one day and say, “You know what I don't like anymore. That's right being on stage, doing the thing that I spent 15 years doing. You know what I want to do instead? That's right, be mad at computers all the time and angry because some of the stuff is freaking maddening.” What was the catalyst that—Carla: Yeah, sounds crazy. [laugh].Corey: —inspired you to move?Carla: It sounds crazy. It was kind of a long time coming. I love performing; I do, and it's like, my heart and soul is with performing. Nothing else in my life really can kind of replace that feeling I get when I'm on stage. But the one thing they don't really talk about when you are growing up and dreaming of being a performer is how physically and emotionally taxing it is.I think there's, like, this narrative around, like, “Being an actor is really hard, and you should only do it if you can't see yourself doing anything else,” but they don't actually ever explain to you what hard means. You know, you expect that, oh, there's going to be a lot of other people doing it in, I'm going to be auditioning all the time, and I'm going to have a lot of competition, but you never quite grasp the physical and emotional toll that it takes on your body and your—you know, just ongoing in auditions and getting rejections all the time. And then when you're working in a show eight times a week and you're wearing four-inch heels on a stage that is on a giant angle, and you're wearing wigs that are, like, really, really massive, you don't really—no one ever tells you how hard that is on your body. So, for me, I just hit a point where I was performing nonstop and I was so tired. I was, like, living at my physical therapist's office, I was living at, like, my head therapist's office.I was just trying to, like, figure out why I was so miserable. And so, I actually left in 2015, performing full time. So, I went to get my Master's in Education at NYU thinking that teaching was my way out of performing full-time.Corey: It does seem that there's some congruities—there's some congruities there between your—instead of performing in front of a giant audience, you're performing in front of a bunch of students. And whether it's performing slash educating, well that comes down to almost stylistic differences. But I have a hard time imagining you just reading from your slides.Carla: Yeah, no, I loved it because it allowed me to create connections with my students, and I found I like to help inspire them on their journeys, and I really like to help influence them in a positive way. And so yeah, it came really natural to me. And my family—or I have a bunch of teachers in my family so, you know, teaching was kind of a thing I just assumed I would be good at, and I think I fell naturally into. But the thing that was really hard for me was while I was teaching, I was still… kind of—I had, like, one foot in performing. I was still, like, going in and out of the show that I've been working on, which I didn't mention.So, I was in Wicked for, like, ten years, that's kind of like my claim to fame. And I had been with that show for a really long time, and that was why—when I left to go teach, that was kind of my way out of that big show because it was hard for me to explain to people why it was leaving such a giant show. And teaching was just, like, a natural thing to go into. I felt like it was like a justifiable action, [laugh] you know, that I could explain to, like, my parents for why I was quitting Broadway.So, you know, I love teaching and—but I—and so I kept that one foot kind of in Broadway, and I was still going in and out of the show. It's like a vacation cover, filling in whenever they needed me, and I was still auditioning. But I was like, I was still so burned out, you know? Like, I still had those feelings of, like—and I wasn't booking work; I think my heart just wasn't really in it. Like, every time I'd go into audition, I would just feel awful about myself every time I left.And I was starting to really reject that feeling in my life because I was also starting to find there were other things in my life that made me really happy. Like, just having a life. Like, I had—for the first time in a very long time, I had friends that I could hang out with on the weekends because I wasn't working on the weekend. And I was able to, like, go to, you know, birthdays and weddings and I was having, like, this social life. And then every time I would go on an audition—Corey: And they did other things with their lives, and it wasn't—Carla: Yeah.Corey: All shop talk all the time—Carla: Right.Corey: Which speaking as someone who lives in San Francisco and worked in normal companies before starting this ridiculous one, it seems that your entire social circle can come out of your workplace. And congratulations, it's now all shop talk, all the time. And anyone you know or might be married to who's not deeply in tech just gets this long-suffering attitude on all of it. It's nice to be able to have varied conversations about different things.Carla: Yes. And so, I was like having all these, like—I was, like, having these life moments that felt really good, and then I would go to an audition and I would leave being, like, “Why do I do that to myself? Why do I need to feel like that?” Because I just feel awful every time I go. And so, then I was having trouble teaching my students because I was feeling really negative about it, and I was like, “I don't know how to encourage you to go into a business that's just going to, like, tear you down and make you feel awful about yourself all the time.”Corey: And then you got into tech?Carla: [laugh]. And then I was just, like, “Tech. That's great.” No, I—do you know what—Corey: Like, “I'm sad all the time and I feel like less than constantly. You know what I'm going to use to fix that? I'm going to learn JavaScript.” Oh, my God.Carla: Yeah. I'm going to just challenge myself and do the hardest thing I can think of because that's fun. But ki—I mean, sort of I [laugh] I, I was not ever—like, being an engineer was never, like, on my radar. My dad was an engineer for a long time, and he kind of always would be, like, “You're good at math. You should do engineering.”And I was like, “No, I'm an actor. [laugh]. I don't want to do that.” And so, I kind of always just, like, shooed it away. And when a friend of mine came to my birthday party in the summer of 2018, who had been a songwriter and I had done some readings of a musical of his, and he was like, “I'm an engineer now at Forbes. Isn't that great?”And I was like, “What? How does that happen? I need you to back up, explain to me what's going on.” And I just, like—but I went home and I could not stop thinking about it. I don't know if it was like my dad's voice in the back of my head, or there was like the stars aligned.My misery that I was feeling in my life, and, like, this new thing that just got thrown in my face was just such an exciting, interesting idea. I was like, “That sounds—I don't know what—I don't even know what that looks like or I don't even know what's involved in that, but I need to figure out how to do it.” And I went home when I first started teaching myself how to do it. And I would just sit on my couch and I would do, like, little coding challenges, and before I knew it, like, hours would have passed by, I forgot to eat, I forget to go to the bathroom. Like, I would just be, like, groove on the couch from where I was sitting for too long.And I was like, oh, I guess I really liked this. [laugh]. It's interesting, it's creative. Maybe I should do something with it.Corey: And then from there, did you decide at some point to pursue—like, a lot of paths into tech these days. There's a whole sea of boot camps, for example, that depending on how you look at them are either inspirational stories of how people can transform their lives, slash money-grabbing scams. And it really depends on the boot camp in particular, is that the path you took? Did you—Carla: Yes.Corey: Remain self-taught? How did you proceed from—there's a whole Couch-to-5k running program; what is about—I guess we'll call getting to tech—but what was your Couch-to-100k path?Carla: Yeah, I was just going to say, Couch-to-100k tech gig.Corey: Yeah.Carla: So, my friend to had gone to Flatiron School, which is a boot camp. I think they have a few locations around the country, and so I initially started looking at their program just because he had gone there, and it sounded great. And I was like, “Cool, great.” And they had a lot of free resources online. They have, like, this whole free, like, boot camp prep program that you can do that teaches Rails and JavaScript.And so, I started doing that online. And then I—at the time, they had, like, a part-time class. I like learning in person, which is funny because now I just work remote and I do everything on Google… it's like, Google and Stack Overflow. So—but I knew at the time—Corey: I have bad news about the people who are senior. It doesn't exactly change that much.Carla: Yeah, that's what I've heard, so I don't feel bad about telling people that I do it. [laugh].Corey: We're all Full Stack Overflow developers. It happens.Carla: Exactly. So yeah, I just. They had, like, a part-time front-end class that was, like, in person two nights a week for a couple months. And I was like, “Okay, that'll be a really good way to kind of get my feet wet with, like, a different kind of learning environment.”And I loved it. I fell in love with it. I loved being in a room of people trying to figure out how to do something hard. I liked talking about it with other people. I liked talking about it with my teachers.So, I was like, “Okay, I guess I'm going to invest in a boot camp.” And I did their, like, immersive, in-person boot camps. This was 2019 before everything shut down, so I was able to actually do it in person. And it was great. It was like, nine to six, five days a week, and it was really intense.Did I remember everything I learned when it was over? No. And did I have to, like, spend a lot of time relearning a lot of things just so I could have, like, a deeper understanding of it. Yes. But, like, I also knew that was part of it, you know? It's like, you throw a lot of information out you, hope some of it sticks, and then it's your job to make sure that you actually remember it and then know how to use it when you have to.Corey: One of the challenges that I've always found is that when I have a hobby that I'm into, similar to the way that you were doing this just for fun on your couch, and then it becomes your full-time focus, first as a boot camp and later as a job, that it has a tendency in some cases to turn a thing that you love into a thing that you view is this obligation or burden. Do you still love it? Is it still something that you find that's fun and challenging and exciting? Or is it more a means to an end for you? And there is no wrong answer there.Carla: Yeah, I think it's a little bit of both, right? Like, I found it was a creative thing I could do that I enjoy doing. Am I the most passionate software engineer that ever lived? No. Do I have aspirations to be, like, an architect one day? Absolutely not. I really, like, the small tickets that I do that are just, like, refactoring a button or, you know, like, I find that stuff creative and I think it's fun. Do I necessarily want to—Corey: You can see—Carla: —no.Corey: The results immediately as [crosstalk 00:15:15]—Carla: Yeah.Corey: More abstract stuff. It's like, “Well, when this 18 months migration finishes, and everything is 10% faster, oh, then I'll be vindicated.”Carla: Yeah. No.Corey: It's a little more attenuated from the immediate feedback.Carla: Yeah. I'm not that kind of developer, I'm learning. But I'm totally fine with that. I have no issue. Like, I am a very humble person about it. I don't have aspirations to be amazing.Don't ask me to do algorithm challenges. I'm terrible at them. I know that I'm terrible at them. But I also know that you can be a good developer and be terrible algorithm, like, challenges. So, I don't feel bad about it.Corey: The algorithm challenge is inherently biased for people who not only have a formal computer science education but have one relatively recently. I look back at some of the technical challenges I used to give candidates and take myself for jobs ten years ago, and I don't remember half of it because it's not my day-to-day anymore. It turns out that most of us don't have a job implementing quicksort. We just use the one built into the library and we move on with our lives to do something interesting and much more valuable, like, moving that button three pixels left, but because of CSS, that's now a two-week project.Carla: Yeah. Add a little border-radius, changes the su—you know. There are some database things I like. You know, I'm trying to get better at SQL. Rails is really nice because we use Active Record, and I don't really have to know SQL.But I find there are some things that you can do in Rails that are really cool, and I enjoyed working in their console. And that's exciting. You know when you write, like, a whole controller and then you make something but you can only see it in the console? That's cool. I think to me, that's fun. Being able to, like, generate things is fun. I don't have to always see them, like, on the page in a visual, pretty way, even though I tend to be more visual.Corey: This episode is sponsored in parts by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on premises, private cloud, and they just announced a fully managed service on AWS and Azure called BigAnimal, all one word.Don't leave managing your database to your cloud vendor because they're too busy launching another half dozen manage databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications, including Oracle, to the cloud.To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: One of the big fictions that we tend to have as an industry is when people sit down and say, “Oh, so why did you get into tech?” And everyone expects it to be this aspirational story of the challenge, and I've been interested in this stuff since I was a kid. And we're all supposed to just completely ignore the very present reality of well, looking at all of my different opportunities, this is the one that pays three times what the others do. Like, we're supposed to pretend that money doesn't matter and we're all following our passion. That is actively ridiculous from where I sit.Carla: Mm-hm.Corey: Do you find that effectively going from the Broadway actress side of the world to—where, let's be clear, in the world of entertaining and arts—to my understanding—90% of people in that space are not able to do that as their only gig without side projects to basically afford to eat, whereas in tech, the median developer makes an extremely comfortable living that significantly outpaces the average median income for a family of four in the United States. Do you find that it has changed your philosophy on life in any meaningful way?Carla: Oh, my God, yeah. I love talking about on all of my social platforms the idea that you can learn tech skills and you can—like, there are so many different jobs that exist for an engineer, right? There are full-time jobs. There are full-time job that are flexible and they're remote, and nobody cares what time you're working as long as you get the work done. And because of that and because of the nature of how performing and being an artist works, where you also have a lot of downtime in between jobs or even when you are working, that I feel like the two go very, very well together, and that it allows—if an artist can spend a little bit of time learning the skill, they now have the ability to feel stable in their lives, also be creative how they want to, and decide what the art looks like for them without struggling and freaking out all the time about where's my next meal going to come from, or can I pay my rent?And, like, I sometimes think back to when I was on tour—I was on tour for three years with Wicked—and I had so much free time, Corey. Like, if I had known that I could have spent some time when I was just like hanging out in my hotel room watching TV all day, like, learning how to code. I would have been—I would have done this years ago. If I had known it was even, I don't even know actually if it was an option back then in, like, the early-2010s. I feel like boot camps kind of started around then, but they were mostly in person.But if I was—today, if I was right now starting my career as an artist, I would absolutely learn how to code as a side hustle. Because why wait tables? [laugh]. Why make, like, minimum wage in a terrible job that you hate when you can I have a skillset that you can do from home now because everything is remote for the most part? Why not?It doesn't make sense to me that anybody would go back to those kind of awful side gigs, side hustle jobs. Because at the end of the day, side hustle jobs end up actually being the things that you spend more time doing, just because theater jobs and art jobs and music jobs are so, you know, far apart when you have them. That might as well pick something that's lucrative and makes you feel less stressed out, you know, in the interim, between gigs. I see it as kind of a way to give artists a little more freedom in what they can choose to do with their art. Which I think is… it's kind of magical, right?Like, it takes away that narrative of if you can't see yourself—if you can see yourself doing anything else, you should do it, right? That's what we tell kids when they go into the arts. If you can see yourself doing any other thing, you know, you have to struggle to be an artist; that is part of the gig. That's what you sign up for. And I just call bullshit on it, Corey. I don't know if I can swear on this, but I call bullshit on [crosstalk 00:21:06]—[laugh].Corey: Oh, you absolutely can.Carla: I just think it's so unfair to young people, to how they get to view themselves and their creativity, right? Like, you literally stunt them when you tell them that. You say, “You can only do this one thing.” That's like the opposite of creative, right? That's like telling somebody that they can only do one thing without imagining that they can do all these other things. The most interesting artists that I know do, like, 400 things, they are creative people and they can't stop, right? They're like multi-hyphenates [crosstalk 00:21:39].Corey: It feels like it's setting people up for failure, on some level, in a big way where when you're building your entire life toward this make-or-break thing and then you don't get it, it's, well, what happens then?Carla: Yeah.Corey: I've always liked the idea of failure as a step forward. And well, that thing didn't work out; let's see if we can roll into it and see what comes out next. It's similar to the idea of a lot of folks who are career-changing, where they were working somewhere else in a white-collar environment, well time to go back to square one for an entry-level world. Hell with that. Pivot; take a half step toward what you want to be doing in your next role, and then a year or so later, take the other half step, and now you're doing it full time without having to start back at square one.I think that there are very few things in this world that are that binary as far as you either succeed or you're done and your whole life was a waste. It is easy get stuck in this idea that if your childhood dream doesn't come true, well give up and prepare for a life of misery. I just don't accept that.Carla: Yeah, I—Corey: But maybe it's because I have no choice because getting fired is my stock-in-trade. So, it wasn't until I built a company where I can't get fired from it that I really started to feel a little bit secure in that. But it does definitely leave its marks and its damages. I spent 12 years waiting for the surprise meeting with my boss and someone I didn't recognize from HR where they don't offer you coffee—that's always the tell when they don't offer coffee—and to realize it while I'm back on the job market again; time to find something new. It left me feeling more mercenary that I probably should have, which wasn't great for the career.What about you? Do you think that—did it take, on some level, a sense of letting go of old dreams? Was it—and did it feel like a creeping awareness that this was, like—that you felt almost cornered into it? Or how did you approach it?Carla: Yeah, I think I was the same way. I think I especially when you were younger because of that narrative, right, we tell people that if they decide to go into the arts, they have to be one hundred percent committed to it, and if they aren't one hundred percent and then they don't succeed, it is their fault, right? Like, if you give it everything that you have, and then it doesn't work out, you have clearly done something wrong, therefore you are a failure. You failed at your dream because you gave it everything that you have, so you kind of set yourself up for failure because you don't allow yourself to, you know, be more of who you are in other ways.For me, I just spent so many I had so many moments in my life where I thought that the world was over, right? Like, when I was—right out of college, I went to school to study opera. And I was studying at Cincinnati Conservatory of Music, it was, like, the great, great conservatory, and halfway through my freshman year, I got diagnosed with a cyst on my vocal cords. So, basically what this meant was that I had to have surgery to have it removed, and the doctor told me that I probably would never sing opera. And I was devastated.Like, I was—this was the thing I wanted to do with my life; I had committed myself one hundred percent, and now all of a sudden this thing happened, and I panicked. I thought it was my fault—because there was nobody to help me understand that it wasn't—and I was like, “I have failed this thing. I have failed my dream. What am I going to do with my life?” And I said, “Okay I'll be an actor because acting is a noble thing.” And that's sort of like act—that's sort of like performing; it's performing in a different way, it's just not singing.And I was terrified to sing again because I had this narrative in my head that I was a failed singer if I co,uldn't be an opera singer. And so, it took me, like, years, three years before I finally started singing again I got a voice teacher, and he—I would cry through all of my lessons. He was like, “Carly, you really have a—should be singing. Like, this is something that you're good at.” And I was like, no because if I can't sing, like, the way I want to sing, why would I sing?And he really kind of pushed me and helped me, like, figure out what my voice could do in a new way. And it was really magical for me. It made me realize that this narrative that I've been telling myself of what I thought that I was supposed to be didn't have to be true. It didn't have to be the only one that existed; there could be other possibilities for what I could do and they could look different. But I closed myself off to that idea because I had basically been told no, you can't do this thing that you want to do.So, I didn't even consider the possibilities of the other things that I could do. And when I relearned how to sing, it just blew my mind because I was like, “Oh, my God, I didn't know this was possible. I didn't know in my body it was possible of this. I didn't know if I could do this.” And, like, overcoming that and making me realize that I could do other things, that there were other versions of what I wanted, kind of blew my mind a little bit.And so, when I would hit road bumps and I'd hit these walls, I was like, “Okay, well, maybe I just need to pivot. Maybe the direction I'm going in isn't quite the right one, but maybe if I just, like, open my eyes a little bit, there's another—there's something else over here that is interesting and will be creative and will take me in a different way, an unexpected way that I wasn't expecting.” And so, I've kind of from that point on sort of living my life like that, in this way that, well, this might be a roadblock, and many people might view this thing as a failure, but for me, it allowed me to open up all these other new things that I didn't even know I could do, right? Like, what I'm doing now is something I never would have imagined I'd be doing five years ago. And now I'm also in a place where not only am I doing something completely different as a software engineer, but I have this incredible opportunity to also start incorporating art back into my life in a way that I can own and I can do for myself instead of having to do for other people.Which is also something I never thought because I thought it was all or nothing. I thought if I was an artist, I was an artist; I'm a software engineer, I'm a software engineer. And so, now I have the ability to kind of live in this weird gray area of getting to make those decisions for myself, and recognize that those little failures were, you know—like, I like to call them, like, the lowercase failures instead of the uppercase failure, right? Like, I am not a failure because I experienced failure. Those little failures are kind of what led me to grow my strength and my resilience and my ability to recognize it more free—like, more quickly when I see it so that I can bounce back faster, right?Like, when I hit a wall, instead of living in that feeling of, like, “Ugh, God, this is the worst thing that ever happened,” I allow myself to move faster through it and recognize that there will be light on the other side. I will get there. And I know that it's going to be okay, and I can trust that because it's always been okay. I always figure it out. And so, that's something—taken me a long time to, like, realize, you know? To, like, really learn, you have to fail a lot to learn that you're going to be okay every time it happens. [laugh].Corey: Yeah, what's the phrase? “Sucking at something is the first step to being kind of good at it?”Carla: Yeah. You got to let yourself suck at it. When I used to teach voice, I would make my students make just, like, the ugliest sounds because I was like, if we can just get past the fact that no matter what, when you sing you're going to sound awful at some point. We're going to try something, you're going to crack, it's not going to come out right, and if we can't own that it's going to suck a little bit on the journey to being good, like, you're going to have a really hard time getting there because you're just going to beat yourself up every time it sucks. Like, it's going to suck a lot [laugh] before you get good. And that's just part of it. That's, like, it is just a part of the process, and you have to kind of own it.Corey: I think that as people we are rarely as one-dimensional as we imagine we are when. And for example, I like working with cloud services, let's not kid ourselves on this. But I have a deep and abiding love affair with the sound of my own voice, so I'm always going to find ways to work that into it. I have a hard time seeing a future career for you that does not in some way, shape or form, tie back to your performing background because even now, talking about singing, you lit up when talking about that in a way that no one does—or at least should—light up when they're talking about React. So, do you think that there's a place between the performing side of the world and the technical side of the world, or those phases of your life, that's going to provide interesting paths for you down the road?Carla: That is a good question, Corey. And I don't know if I have the answer. You know, I think one thing—if there's anything I learned from all the crazy things that happened to me, is that I just kind of have to be open. You know, I like to say yes to things. And also learning to say no, which has been really a big deal for me.Corey: Oh, yes.“, no,” is a complete sentence and people know that sometimes at their own peril.Carla: Yes, I have said no to some things lately, and it's felt very good. But I like to be open, you know? I like to feel like if I'm putting out good things into the world, good things will come back to me, and so I'm just trying to keep that open. You know, I'm trying to be the best engineer that I can be. And I'm trying to also, you know—if I can use my voice and my platform to help inspire other people to see that there are other ways of being an artist, there are, you know, there are other paths in this world to take.I hope that, you know, I can, other things will come up to me, there'll be opportunities. And I don't know what those look like, but I'm open. So, if anybody out there hears this and you want to collaborate, hit me up. [laugh].Corey: Careful what you offer. People don't know—people have a disturbing tendency of saying, “Well, all right, I have an idea.” That's where a lot of my ridiculous parody music videos came from. It's like, “So, what's the business case for doing?” It's like, “Mmm, I think it'll be funny.”It's like, “Well, how are you going to justify the expense?” “Oh, there's a line item and the company budget labeled ‘Spite.' That's how.” And it's this weird combination of things that lead to a path that on some level makes perfect sense, but at the time you're building this stuff out, it feels like you're directionless and doing all these weird things. Like, one of the, I guess, strange parts of looking back at a path you take in the course of your career is, in retrospect, it feels like every step for the next was obvious and made intuitive sense, but going through it it's, “I have no idea what I'm doing. I'm like the dog that caught the car, and they need to desperately figure out how to drive the thing before it hits the wall.”It's just a—I don't pretend to understand how the tapestry of careers tie together, but I do know that I'm very glad to see people in this space, who do not all have the same ridiculous story for how they got in here. That's the thing that I find continually obnoxious, this belief that there's only one way to do it, or you're somehow less than because you didn't grow up programming in the '90s. Great. There's a lot of people like that. And yes, it is okay to just view computers as a job that pays the bills; there is nothing inherently wrong with that.Carla: Yeah. And I mean, and I—Corey: I just wish people were told that early on.Carla: Yeah, why not? Right? Why didn't anybody tell us that? Like, you don't—the thing that I did not—it took me a long time to realize is that you do not have to be passionate about your job. And that's like, that's okay, right? All you have to do is enjoy it enough to do it, but it does not have to be, like—Corey: You have to like it, on some level [crosstalk 00:33:10]—Carla: Yeah, you just do have to like it. [laugh].Corey: —dreading the 40 hours a week, that's a miserable life on some level.Carla: Like, I sit in front of a computer now all day, and I enjoy it. Like, I enjoy what I'm doing. But again, like, I don't need to be the greatest software engineer that ever lived; I have other things that I like to do, and it allows me to also do those things. And that is what I love about it. It allows me that ability to just enjoy my weekends and have a stable career and have a stable life and have health insurance. And then when I want—Corey: Oh, the luxuries of modern life.Carla: [laugh]. Yeah, the luxuries of modern life. Health insurance, who knew? Yeah, you know, so it's great. And then when creative projects come up, I can choose to say yes or no to them, and that's really exciting for me.Corey: I have a sneaking suspicion—I'll just place my bet now—that the world of performing is not quite done with you yet.Carla: Probably not. I would be lying if I said it was. I—so before all this stuff, I don't know if your listeners know this, but in January, the thing that kind of happened to me that went a little viral where I went back to Broadway after not being on Broadway for a little while, and the news media and everybody picked up on it, and there were like these headlines of, “Software engineer plays Elphaba on Broadway after seven years.” It surprised me, but it also didn't surprise me, you know? Like, when I left, I left thinking I was done.And I think it was easy to leave when I left because of the pandemic, right? There was nothing going on when I—like, I started my journey before the pandemic, but I fully shifted into software engineering during the pandemic. So, I never had feelings of, like, “I'm missing out on performing,” because performing didn't exist. There was no Broadway for a while. And so, once it kind of started to come back last year in the fall, I was like, “Oh, maybe I miss it a little bit.”And maybe I accidentally manifested it, but, you know, when Wicked called and I flew back to New York for those shows, and I was like, “Oh, this is really wonderful.” Also, I'm really glad I don't have to do this eight times a week. I'm so excited to go home. And I was like, having a little taste of it made me realize, “Oh, I can do this if I want to do this. I also don't have to do this if I don't want to do this.” And that was pretty—it was very empowering. I was like, “That feels nice.”Corey: I really appreciate your taking so much time to talk about how you've gone through what at the time has got to have felt like a very strange set of career steps, but it's starting to form into something that appears to have an arc to it. If people want to learn more and follow along as you continue to figure out what you're going to do next, where's the best place to find you?Carla: Oh, good question, Corey. I do a website, carlastickler.com. Because I've had a lot of people—artists, in particular—reaching out and asking how I did this, I'm starting to build some resources, and so you can sign up for my mailing list.I also am pretty big on Instagram if we're going to choose social media. So, my Instagram is stiglercarla. And there's links to all that stuff on my website. But—Corey: And they will soon be in the [show notes 00:36:26] as well.Carla: Ah yes, add them to the show notes. [laugh]. Yeah, and I want to make sure that I… I want—a lot of people who've seen my story and felt very inspired by it. A lot of artists who have felt that they, too, were failures because they chose not to go into art and get a regular nine to five. And so, I'm trying to, like, kind of put a little bit more of that out there so that people see that they're not alone.And so, on my social media, I do post a lot of stories that people send to me, just telling me their story about how they made the transition and how they keep art in their life in different ways. And so, that's something that also really inspires me. So, I tried to put their voices up, too. So, if anybody is interested in feeling not alone, feeling like there are other people out there, all of us, quote-unquote, “Failed artists,” and there's a lot of us. And so, I'm just trying to create a little space for all of us.Corey: I look forward to seeing it continue to evolve.Carla: Thank you.Corey: Thank you so much for your time. I appreciate it.Carla: Thanks, Corey.Corey: Carla Stickler, software engineer at G2 and also very much more. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, and if it's on the YouTubes, smash the like and subscribe buttons, as the kids of today are saying, whereas if you've hated this podcast, same thing: Five-star review, smash the buttons, but also leave an angry comment telling me exactly what you didn't like about this, and I will reply with the time and date for your audition.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About SimonFounder and CEO of SnapShooter a backup company Links Referenced: SnapShooter.com: https://SnapShooter.com MrSimonBennett: https://twitter.com/MrSimonBennett TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Finding skilled DevOps engineers is a pain in the neck! And if you need to deploy a secure and compliant application to AWS, forgettaboutit! But that's where DuploCloud can help. Their comprehensive no-code/low-code software platform guarantees a secure and compliant infrastructure in as little as two weeks, while automating the full DevSecOps lifestyle. Get started with DevOps-as-a-Service from DuploCloud so that your cloud configurations are done right the first time. Tell them I sent you and your first two months are free. To learn more visit: snark.cloud/duplo. Thats's snark.cloud/D-U-P-L-O-C-L-O-U-D.Corey: What if there were a single place to get an inventory of what you're running in the cloud that wasn't "the monthly bill?" Further, what if there were a way to compare that inventory to what you were already managing via Terraform, Pulumi, or CloudFormation, but then automatically add the missing unmanaged or drifted parts to it? And what if there were a policy engine to immediately flag and remediate a wide variety of misconfigurations? Well, stop dreaming and start doing; visit snark.cloud/firefly to learn more.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. One of the things that I learned early on in my career as a grumpy Unix systems administrator is that there are two kinds of people out there: those who care about backups an awful lot, and people who haven't lost data yet. I lost a bunch of data once upon a time and then I too fell on the side of backups are super important. Here to talk with me about them a bit today is Simon Bennett, founder and CEO of SnapShooter.com. Simon, thanks for joining me.Simon: Thanks for having me. Thank you very much.Corey: It's fun to be able to talk to people who are doing business in the cloud space—in this sense too—that is not venture-backed, that is not, “Well, we have 600 people here that are building this thing out.” And similar to the way that I handle things at The Duckbill Group, you are effectively one of those legacy things known as a profitable business that self-funds. What made you decide to pursue that model as opposed to, well, whatever the polite version of bilking venture capitalists out of enormous piles of money for [unintelligible 00:01:32]?Simon: I think I always liked the idea of being self-sufficient and running a business, so I always wanted to start a physical business when I was younger, but when I got into software, I realized that that's a really easy way, no capital needed, to get started. And I tried for years and years to build products, all of which failed until finally SnapShooter actually gained a customer. [laugh].Corey: “Oh, wait, someone finally is paying money for this, I guess I'm onto something.”Simon: Yeah.Corey: And it's sort of progressed from there. How long have you been in business?Simon: We started in 2017, as… it was an internal project for a company I was working at who had problems with DigitalOcean backups, or they had problems with their servers getting compromised. So, I looked at DigitalOcean API and realized I could build something. And it took less than a week to build a product [with billing 00:02:20]. And I put that online and people started using it. So, that was how it worked.Every other product I tried before, I'd spent months and months developing it and never getting a customer. And the one time I spent less than [laugh] less than a week's worth of evenings, someone started paying. I mean, admittedly, the first person was only paying a couple of dollars a month, but it was something.Corey: There's a huge turning point where you just validate the ability and willingness for someone to transfer one dollar from their bank account to yours. It speaks to validation in a way that social media nonsense generally doesn't. It's the oh, someone is actually willing to pay because I'm adding value to what they do. That's no small thing.Simon: Yeah. There's definitely a big difference between people saying they're going to and they'd love it, and actually doing it. So.Corey: I first heard about you when Patrick McKenzie—or @patio11, as he goes by on Twitter—wound up doing a mini-thread on you about, “I've now used SnapShooter.com for real, and it was such a joy, including making a server migration easier than it would otherwise have been. Now, I have automatically monitored backups to my own S3 account for a bunch of things, which already had a fairly remote risk of failure.” And he keeps talking about the awesome aspects of it. And okay, when Patrick says, “This is neat,” that usually means it's time for me to at least click the link and see what's going on.And the thing that jumped out at me was a few things about what it is that you offer. You talk about making sure that people can sleep well at night, that it's about why backups are important, about—you obviously check the boxes and talk about how you do things and why you do them the way that you do, but it resonates around the idea of helping people sleep well at night. Because no one wants to think about backups. Because no one cares about backups; they just care an awful lot about restores, usually right after they should have cared about the backups.Simon: Yeah. This is actually a big problem with getting customers because I don't think it's on a lot of people's minds, getting backups set up until, as you said in the intro, something's gone wrong. [laugh]. And then they're happy to be a customer for life.Corey: I started clicking around and looking at your testimonials, for example, on your website. And the first one I saw was from the CEO of Transistor.fm. For those who aren't familiar with what they do, they are the company that hosts this podcast. I pay them as a vendor for all the back issues and whatnot.Whenever you download the show. It's routing through their stuff. So yeah, I kind of want them to have backups of these things because I really don't want to have all these conversations [laugh] again with everyone. That's an important thing. But Transistor's business is not making sure that the data is safe and secure; it's making podcasts available, making it easy to publish to them.And in your case, you're handling the backup portion of it so they can pay their money and they set it up effectively once—set it and forget it—and then they can go back to doing the thing that they do, and not having to fuss with it constantly. I think a lot of companies get it wrong, where they seem to think that people are going to make sustained, engaged efforts in whatever platform or tool or service they build. People have bigger fish to fry; they just want the thing to work and not take up brain sweat.Simon: Yeah. Customers hardly ever log in. I think it's probably a good sign when they don't have to log in. So, they get their report emails, and that's that. And they obviously come back when they got new stuff to set up, but from a support point of view is pretty, pretty easy, really, people don't—[laugh] constantly on there.Corey: From where I sit, the large cloud providers—and some of the small ones, too—they all have backup functionality built into the offering that they've got. And some are great, some are terrible. I assume—perhaps naively—that all of them do what it says on the tin and actually back up the data. If that were sufficient, you wouldn't have any customers. You clearly have customers. What is it that makes those things not work super well?Simon: Some of them are inflexible. So, some of the providers have built-in server backups that only happen weekly, and six days of no backups can be a big problem when you've made a mistake. So, we offer a lot of flexibility around how often you backup your data. And then another key part is that we let you store your data where you want. A lot of the providers have either vendor lock-in, or they only store it in themselves. So… we let you take your data from one side of the globe to the other if you want.Corey: As anyone who has listened to the show is aware, I'm not a huge advocate for multi-cloud for a variety of excellent reasons. And I mean that on a per-workload basis, not, “Oh, we're going to go with one company called Amazon,” and you use everything that they do, including their WorkMail product. Yeah, even Amazon doesn't use WorkMail; they use Exchange like a real company would. And great, pick the thing that works best for you, but backups have always been one of those areas.I know that AWS has great region separation—most of the time. I know that it is unheard of for there to be a catastrophic data loss story that transcends multiple regions, so the story from their side is very often, oh, just back it up to a different region. Problem solved. Ignoring the data transfer aspect of that from a pricing perspective, okay. But there's also a risk element here where everyone talks about the single point of failure with the AWS account that it's there, people don't talk about as much: it's your payment instrument; if they suspend your account, you're not getting into any region.There's also the story of if someone gets access to your account, how do you back that up? If you're going to be doing backups, from my perspective, that is the perfect use case, to put it on a different provider. Because if I'm backing up from, I don't know, Amazon to Google Cloud or vice versa, I have a hard time envisioning a scenario in which both of those companies simultaneously have lost my data and I still care about computers. It is very hard for me to imagine that kind of failure mode, it's way out of scope for any disaster recovery or business continuity plan that I'm coming up with.Simon: Yeah, that's right. Yeah, I haven't—[laugh] I don't have that in my disaster recovery plan, to be honest about going to a different cloud, as in, we'll solve that problem when it happens. But the data is, as you say, in two different places, or more. But yeah, the security one is a key one because, you know, there's quite a lot of surface area on your AWS account for compromising, but if you're using either—even a separate AWS account or a different provider purely for storage, that can be very tightly controlled.Corey: I also appreciate the idea that when you're backing stuff up between different providers, the idea of owning both sides of it—I know you offer a solution where you wind up hosting the data as well, and that has its value, don't get me wrong, but there are also times, particularly for regulated industries, where yeah, I kind of don't want my backup data just hanging out with someone else's account with whatever they choose to do with it. There's also the verification question, which again, I'm not accusing you of in any way, shape, or form of being nefarious, but it's also one of those when I have to report to a board of directors of like, “Are you sure that they're doing what they say they're doing?” It's a, “Well, he seemed trustworthy,” is not the greatest answer. And the boards ask questions like that all the time. Netflix has talked about this where they backup a rehydrate-the-business level of data to Google Cloud from AWS, not because they think Amazon is going to disappear off the face of the earth, but because it's easier to do that and explain it than having to say, “Well, it's extremely unlikely and here's why,” and not get torn to pieces by auditors, shareholders, et cetera. It's the path of least resistance, and there is some validity to it.Simon: Yeah, when you see those big companies who've been with ransomware attacks and they've had to either pay the ransom or they've literally got to build the business from scratch, like, the cost associated with that is almost business-ending. So, just one backup for their data, off-site [laugh] they could have saved themselves millions and millions of pounds. So.Corey: It's one of those things where an ounce of prevention is worth a pound of cure. And we're still seeing that stuff continue to evolve and continue to exist out in the ecosystem. There's a whole host of things that I think about like, “Ooh, if I lost, that would be annoying but not disastrous.” When I was going through some contractual stuff when we were first setting up The Duckbill Group and talking to clients about this, they would periodically ask questions about, “Well, what's your DR policy for these things?” It's, “Well, we have a number of employees; no more than two are located in the same city anywhere, and we all work from laptops because it is the 21st century, so if someone's internet goes out, they'll go to a coffee shop. If everyone's internet goes out, do you really care about the AWS bill that month?”It's a very different use case and [unintelligible 00:11:02] with these things. Now, let's be clear, we are a consultancy that fixes AWS bills; we're not a hospital. There's a big difference in the use case and what is acceptable in different ways. But what I like is that you have really build something out that lets people choose their own adventure in how managed they want it to be, what the source is, what the target should be. And it gives people enough control but without having to worry about the finicky parts of aligning a bunch of scripts that wind up firing off in cron jobs.Simon: Yeah. I'd say a fair few people run into issues running scripts or, you know, they silently fail and then you realize you haven't actually been running backups for the last six months until you're trying to pull them, even if you were trying to—Corey: Bold of you to think that I would notice it that quickly.Simon: [laugh]. Yeah, right. True. Yeah, that's presuming you have a disaster recovery plan that you actually test. Lots of small businesses have never even heard of that as a thing. So, having as us, kind of, manage backups sort of enables us to very easily tell people that backups of, like—we couldn't take the backup. Like, you need to address this.Also, to your previous point about the control, you can decide completely where data flows between. So, when people ask us about what's GDPR policies around data and stuff, we can say, “Well, we don't actually handle your data in that sense. It goes directly from your source through almost a proxy that you control to your storage.” So.Corey: The best answer: GDPR is out of scope. Please come again. And [laugh] yeah, just pass that off to someone else.Simon: In a way, you've already approved those two: you've approved the person that you're managing servers with and you've already approved the people that are doing storage with. You kind of… you do need to approve us, but we're not handling the data. So, we're handling your data, like your actual customer; we're not handling your customer's customer's data.Corey: Oh, yeah. Now, it's a valuable thing. One of my famous personal backup issues was okay, “I'm going to back this up onto the shared drive,” and I sort of might have screwed up the backup script—in the better way, given the two possible directions this can go—but it was backing up all of its data and all the existing backup data, so you know, exponential growth of your backups. Now, my storage vendor was about to buy a boat and name it after me when I caught that. “Oh, yeah, let's go ahead and fix that.”But this stuff is finicky, it's annoying, and in most cases, it fails in silent ways that only show up as a giant bill in one form or another. And not having to think about that is valuable. I'm willing to spend a few hours setting up a backup strategy and the rest; I'm not willing to tend it on an ongoing basis, just because I have other things I care about and things I need to get done.Simon: Yeah. It's such a kind of simple and trivial thing that can quickly become a nightmare [laugh] when you've made a mistake. So, not doing it yourself is a good [laugh] solution.Corey: So, it wouldn't have been a @patio11 recommendation to look at what you do without having some insight into the rest of the nuts and bolts of the business and the rest. Your plans are interesting. You have a free tier of course, which is a single daily backup job and half a gig of storage—or bring your own to that it's unlimited storage—Simon: Yep. Yeah.Corey: Unlimited: the only limits are your budget. Yeah. Zombo.com got it slightly wrong. It's not your mind, it's your budget. And then it goes from Light to Startup to Business to Agency at the high end.A question I have for you is at the high end, what I've found has been sort of the SaaS approach. The top end is always been a ‘Contact Us' form where it's the enterprise scope of folks where they tend to have procurement departments looking at this, and they're going to have a whole bunch of custom contract stuff, but they're also not used to signing checks with fewer than two commas in them. So, it's the signaling and the messaging of, “Reach out and talk to us.” Have you experimented with that at all, yet? Is it something you haven't gotten to yet or do you not have interest in serving that particular market segment?Simon: I'd say we've been gearing the business from starting off very small with one solution to, you know, last—and two years ago, we added the ability to store data from one provider to a different provider. So, we're sort of stair-stepping our way up to enterprise. For example, at the end of last year, we went and got certificates for ISO 27001 and… one other one, I can't remember the name of them, and we're probably going to get SOC 2 at some point this year. And then yes, we will be pushing more towards enterprises. We add, like, APIs as well so people can set up backups on the fly, or so they can put it as part of their provisioning.That's hopefully where I'm seeing the business go, as in we'll become under-the-hood backup provider for, like, a managed hosting solution or something where their customers won't even realize it's us, but we're taking the backups away from—responsibility away from businesses.Corey: For those listeners who are fortunate enough to not have to have spent as long as I have in the woods of corporate governance, the correct answer to, “Well, how do we know that vendor is doing what they say that they're doing,” because the, “Well, he seemed like a nice guy,” is not going to carry water, well, here are the certifications that they have attested to. Here's copies under NDA, if their audit reports that call out what controls they claim to have and it validates that they are in fact doing what they say that they're doing. That is corporate-speak that attests that you're doing the right things. Now, you're going to, in most cases, find yourself spending all your time doing work for no real money if you start making those things available to every customer spending 50 cents a year with you. So generally, the, “Oh, we're going to go through the compliance, get you the reports,” is one of the higher, more expensive tiers where you must spend at least this much for us to start engaging down this rabbit hole of various nonsense.And I don't blame you in the least for not going down that path. One of these years, I'm going to wind up going through at least one of those certification approaches myself, but historically, we don't handle anything except your billing data, and here's how we do it has so far been sufficient for our contractual needs. But the world's evolving; sophistication of enterprise buyers is at varying places and at some point, it'll just be easier to go down that path.Simon: Yeah, to be honest, we haven't had many, many of those customers. Sometimes we have people who come in well over the plan limits, and that's where we do a custom plan for them, but we've not had too many requests for certification. But obviously, we have the certification now, so if anyone ever [laugh] did want to see it under NDA, we could add some commas to any price. [laugh].Corey: This episode is sponsored in parts by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on premises, private cloud, and they just announced a fully managed service on AWS and Azure called BigAnimal, all one word.Don't leave managing your database to your cloud vendor because they're too busy launching another half dozen manage databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications, including Oracle, to the cloud.To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: What I like as well is that you offer backups for a bunch of different things. You can do snapshots from, effectively, every provider. I'm sorry, I'm just going to call out because I love this: AWS and Amazon LightSail are called out as two distinct things. And Amazonians will say, “Oh, well, under the hood, they're really the same thing, et cetera.” Yeah, the user experience is wildly different, so yeah, calling those things out as separate things make sense.But it goes beyond that because it's not just, “Well, I took a disk image. There we go. Come again.” You also offer backup recipes for specific things where you could, for example, back things up to a local file and external storage where someone is. Great, you also backup WordPress and MongoDB and MySQL and a whole bunch of other things.A unified cloud controller, which is something I have in my house, and I keep thinking I should find a way to back that up. Yeah, this is great. It's not just about the big server thing; it's about having data living in managed services. It's about making sure that the application data is backed up in a reasonable, responsible way. I really liked that approach. Was that an evolution or is that something you wound up focusing on almost from the beginning?Simon: It was an evolution. So, we started with the snapshots, which got the business quite far to be honest and it was very simple. It was just DigitalOcean to start with, actually, for the first two years. Pretty easy to market in a way because it's just focused on one thing. Then the other solutions came in, like the other providers and, you know, once you add one, it was easy to add many.And then came database backups and file backups. And I just had those two solutions because that was what people were asking for. Like, they wanted to make sure their whole server snapshot, if you have a whole server snapshot, the point in time data for MySQL could be corrupt. Like, there could be stuff in RAM that a MySQL dump would have pulled out, for example. Like… there's a possibility that the database could be corrupt from a snapshot, so people were asking for a bit of, more, peace of mind with doing proper backups of MySQL.So, that's what we added. And it soon became apparent when more customers were asking for more solutions that we really needed to, like, step back and think about what we're actually offering. So, we rebuilt this whole, kind of like, database engine, then that allowed us to consume data from anywhere. So, we can easily add more backup types. So, the reason you can see all the ones you've listed there is because that's kind of what people have been asking for. And every time someone comes up with a new, [laugh], like, a new open-source project or database or whatever, we'll add support, even ones I've never heard of before. When people ask for some weird file—Corey: All it takes is just waiting for someone to reach out and say, hey, can you back this thing up, please?Simon: Yeah, exactly, some weird file-based database system that I've never ever heard of. Yeah, sure. Just give us [laugh] a test server to mess around with and we'll build, essentially, like, we use bash in the background for doing the backups; if you can stream the data from a command, we can then deal with the whole management process. So, that's the reason why. And then, I was seeing in, like, the Laravel space, for example, people were doing MySQL backups and they'd have a script, and then for whatever reason, someone rotated the passwords on the database and the backup script… was forgotten about.So, there it is, not working for months. So, we thought we could build a backup where you could just point it at where the Laravel project is. We can get all the config we need at the runtime because it's all there with the project anyway, and then thus, you never need to tell us the password for your database and that problem goes away. And it's the same with WordPress.Corey: I'm looking at this now just as you go through this, and I'm a big believer in disclaiming my biases, conflicts of interest, et cetera. And until this point, neither of us have traded a penny in either direction between us that I'm ever aware of—maybe you bought a t-shirt or something once upon a time—but great, I'm about to become a customer of this because I already have backup solutions for a lot of the things that you currently support, but again, when you're a grumpy admin who's lost data in the past, it's, “Huh, you know what I would really like? That's right, another backup.” And if that costs me a few hundred bucks a year for the peace of mind is money well spent because the failure mode is I get to rewrite a whole lot of blog posts and re-record all podcasts and pay for a whole bunch of custom development again. And it's just not something that I particularly want to have to deal with. There's something to be said for a holistic backup solution. I wish that more people thought about these things.Simon: Can you imagine having to pull all the blog posts off [unintelligible 00:22:19]? [laugh]—Corey: Oh, my got—Simon: —to try and rebuild it.Corey: That is called the crappiest summer internship someone has ever had.Simon: Yeah.Corey: And that is just painful. I can't quite fathom having to do that as a strategy. Every once in a while some big site will have a data loss incident or go out of business or something, and there's a frantic archiving endeavor that happens where people are trying to copy the content out of the Google Search Engine's cache before it expires at whatever timeline that is. And that looks like the worst possible situation for any sort of giant backup.Simon: At least that's one you can fix. I mean, if you were to lose all the payment information, then you've got to restitch all that together, or anything else. Like, that's a fixable solution, but a lot of these other ones, if you lose the data, yeah, there's no two ways around it, you're screwed. So.Corey: Yeah, it's a challenging thing. And it's also—the question also becomes one of, “Well, hang on. I know about backups on this because I have this data, but it's used to working in an AWS environment. What possible good would it do me sitting somewhere else?” It's, yeah, the point is, it's sitting somewhere else, at least in my experience. You can copy it back to that sort of environment.I'm not suggesting this is a way that you can run your AWS serverless environment on DigitalOcean, but it's a matter of if everything turns against you, you can rebuild from those backups. That's the approach that I've usually taken. Do you find that your customers understand that going in or is there an education process?Simon: I'd say people come for all sorts of reasons for why they want backup. So, having your data in two places for that is one of the reasons but, you know, I think there's a lot of reasons why people want peace of mind: for either developer mistakes or migration mistakes or hacking, all these things. So, I guess the big one we come up with a lot is people talking about databases and they don't need backups because they've got replication. And trying to explain that replication between two databases isn't the same as a backup. Like, you make a mistake you drop—[laugh] you run your delete query wrong on the first database, it's gone, replicated or not.Corey: Right, the odds of me fat-fingering an S3 bucket command are incredibly likelier than the odds of AWS losing an entire region's S3 data irretrievably. I make mistakes a lot more than they tend to architecturally, but let's also be clear, they're one of the best. My impression has always been the big three mostly do a decent job of this. The jury's still out, in my opinion, on other third-party clouds that are not, I guess, tier one. What's your take?Simon: I have to be careful. I've got quite good relationships with some of these. [laugh].Corey: Oh, of course. Of course. Of course.Simon: But yes, I would say most customers do end up using S3 as their storage option, and I think that is because it is, I think, the best. Like, is in terms of reliability and performance, some storage can be a little slow at times for pulling data in, which could or could not be a problem depending on what your use case is. But there are some trade-offs. Obviously, S3, if you're trying to get your data back out, is expensive. If you were to look at Backblaze, for example, as well, that's considerably cheaper than S3, especially, like, when you're talking in the petabyte-scale, there can be huge savings there. So… they all sort of bring their own thing to the table. Personally, I store the backups in S3 and in Backblaze, and in one other provider. [laugh].Corey: Oh, yeah. Like—Simon: I like to have them spread.Corey: Like, every once in a while in the industry, there's something that happens that's sort of a watershed moment where it reminds everyone, “Oh, right. That's why we do backups.” I think the most recent one—and again, love to them; this stuff is never fun—was when that OVH data center burned down. And OVH is a somewhat more traditional hosting provider, in some respects. Like, their pricing is great, but they wind up giving you what amounts to here as a server in a rack. You get to build all this stuff yourself.And that backup story is one of those. Oh, okay. Well, I just got two of them and I'll copy backups to each other. Yeah, but they're in the same building and that building just burned down. Now, what? And a lot of people learned a very painful lesson. And oh, right, that's why we have to do that.Simon: Yeah. The other big lesson from that was that even if the people with data in a different region—like, they'd had cross-regional backups—because of the demand at the time for accessing backups, if you wanted to get your data quickly, you're in a queue because so many other people were in the same boat as you're trying to restore stored backups. So, being off-site with a different provider would have made that a little easier. [laugh].Corey: It's a herd of elephants problem. You test your DR strategy on a scheduled basis; great, you're the only person doing it—give or take—at that time, as opposed to a large provider has lost a region and everyone is hitting their backup service simultaneously. It generally isn't built for that type of scale and provisioning. One other question I have for you is when I make mistakes, for better or worse, they're usually relatively small-scale. I want to restore a certain file or I will want to, “Ooh, that one item I just dropped out of that database really should not have been dropped.” Do you currently offer things that go beyond the entire restore everything or nothing? Or right now are you still approaching this from the perspective of this is for the catastrophic case where you're in some pain already?Simon: Mostly the catastrophic stage. So, we have MySQL [bin logs 00:27:57] as an option. So, if you wanted to do, like, a point-in-time of store, which… may be more applicable to what you're saying, but generally, its whole, whole website recovery. For example, like, we have a WordPress backup that'll go through all the WordPress websites on the server and we'll back them up individually so you can restore just one. There are ways that we have helped customers in the past just pull one table, for example, from a backup.But yeah, we geared towards, kind of, the set and the forget. And people don't often restore backups, to be honest. They don't. But when they do, it's obviously [laugh] very crucial that they work, so I prefer to back up the whole thing and then help people, like, if you need to extract ten megabytes out of an entire gig backup, that's a bit wasteful, but at least, you know, you've got the data there. So.Corey: Yeah. I'm a big believer in having backups in a variety of different levels. Because I don't really want to do a whole server restore when I remove a file. And let's be clear, I still have that grumpy old Unix admin of before I start making changes to a file, yeah, my editor can undo things and remembers that persistently and all. But I have a disturbing number of files and directories whose names end in ‘.bac' with then, like, a date or something on it, just because it's—you know, like, “Oh, I have to fix something in Git. How do I do this?”Step one, I'm going to copy the entire directory so when I make a pig's breakfast out of this and I lose things that I care about, rather than having to play Git surgeon for two more days, I can just copy it back over and try again. Disk space is cheap for those things. But that's also not a holistic backup strategy because I have to remember to do it every time and the whole point of what you're building and the value you're adding, from my perspective, is people don't have to think about it.Simon: Yes. Yeah yeah yeah. Once it's there, it's there. It's running. It's as you say, it's not the most efficient thing if you wanted to restore one file—not to say you couldn't—but at least you didn't have to think about doing the backup first.Corey: I really want to thank you for taking the time out of your day to talk to me about all this. If people want to learn more for themselves, where can they find you?Simon: So, SnapShooter.com is a great place, or on Twitter, if you want to follow me. I am @MrSimonBennett.Corey: And we will, of course, put links to that in the [show notes 00:30:11]. Thank you once again. I really appreciate it.Simon: Thank you. Thank you very much for having me.Corey: Simon Bennett, founder and CEO of SnapShooter.com. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this episode, please leave a five-star review on your podcast platform of choice, whereas if you've hated this episode, please leave a five-star review on your podcast platform of choice, along with an angry insulting comment that, just like your backup strategy, you haven't put enough thought into.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About WesleyWesley Faulkner is a first-generation American, public speaker, and podcaster. He is a founding member of the government transparency group Open Austin and a staunch supporter of racial justice, workplace equity, and neurodiversity. His professional experience spans technology from AMD, Atlassian, Dell, IBM, and MongoDB. Wesley currently works as a Developer Advocate, and in addition, co-hosts the developer relations focused podcast Community Pulse and serves on the board for SXSW.Links Referenced: Twitter: https://twitter.com/wesley83 Polywork: https://polywork.com/wesley83 Personal Website: https://www.wesleyfaulkner.com/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Finding skilled DevOps engineers is a pain in the neck! And if you need to deploy a secure and compliant application to AWS, forgettaboutit! But that's where DuploCloud can help. Their comprehensive no-code/low-code software platform guarantees a secure and compliant infrastructure in as little as two weeks, while automating the full DevSecOps lifestyle. Get started with DevOps-as-a-Service from DuploCloud so that your cloud configurations are done right the first time. Tell them I sent you and your first two months are free. To learn more visit: snark.cloud/duplo. Thats's snark.cloud/D-U-P-L-O-C-L-O-U-D.Corey: What if there were a single place to get an inventory of what you're running in the cloud that wasn't "the monthly bill?" Further, what if there were a way to compare that inventory to what you were already managing via Terraform, Pulumi, or CloudFormation, but then automatically add the missing unmanaged or drifted parts to it? And what if there were a policy engine to immediately flag and remediate a wide variety of misconfigurations? Well, stop dreaming and start doing; visit snark.cloud/firefly to learn more.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I am joined again for a second time this year by Wesley Faulkner. Last time we spoke, he was a developer advocate. And since then, as so many have, he's changed companies. Wesley, thank you for joining me again. You're the Head of Community at SingleStore, now. Congrats on the promotion.Wesley: Thank you. It's been a very welcome change. I love developer advocates and developer advocacy. But I love people, too, so it's almost, I think, very analogous to the ebbs and flow that we all have gone through, through the pandemic, and leaning into my strong suits.Corey: It's a big deal having a ‘head of' in a role title, as opposed to Developer Advocate, Senior Developer Advocate. And it is a different role. It's easy to default into the world of thinking that it's a promotion. Management is in many ways orthogonal to what it takes to succeed in an actual role. And further, you're not the head of DevRel, or DevRelopers or whatever you want to call the term. You are instead the Head of Community. How tied is that to developer relations, developer advocacy, or other things that we are used to using as terms of art in this space?Wesley: If we're talking about other companies, I would say the Head of Community is something that's under the umbrella of developer relations, where it's just a peer to some of the other different elements or columns of developer relations. But in SingleStore specifically, I have to say that developer relations in terms of what you think about whole umbrella is very new to the company. And so, I consider myself the first person in the role of developer relations by being the Head of Community. So, a lot of the other parts are being bolted in, but under the focus of developer as a community. So, I'm liaisoning right now as helping with spearheading some of the design of the activities that the advocates do, as well as architecting the platform and the experiences of people coming in and experiencing SingleStore through the community's perspective.So, all that to say is, what I'm doing is extremely structured, and a lot of stuff that we're doing with the efficacy, I'm using some of my expertise to help guide that, but it's still something that's kind of like an offshoot and not well integrated at the moment.Corey: How has it changed the way that you view the function of someone who's advocating to developers, which is from my cynical perspective, “Oh, it's marketing, but we don't tell people it's marketing because they won't like it.” And yes, I know, I'll get emails about that. But how does it differ from doing that yourself versus being the head of the function of a company? Because leadership is a heck of a switch? I thought earlier in my career that oh, yeah, it's a natural evolution of being a mediocre engineer. Time to be a mediocre manager. And oh, no, no, I aspired to be a mediocre manager. It's a completely different skill set and I got things hilariously wrong. What's it like for you going through that shift?Wesley: First of all, it is kind of like advertising, and people may not think of it that way. Just to give an example, movie trailers is advertising. The free samples at the grocery store is advertising. But people love those because it gives an experience that they like in a package that they are accustomed to. And so, it's the same with developer relations; it's finding the thing that makes the experience worthwhile.On the community side, this is not new to me. I've done several different roles, maybe not in this combination. But when I was at MongoDB, I was a technical community manager, which is like a cog in the whole giant machine. But before that, in my other life, I managed social and community interactions for Walmart, and I had, at the slow period, around 65, but during the holidays, it would ramp up to 95 direct reports that I managed.It's almost—if you're a fan of The Princess Bride, it's different than fighting one person. Sometimes it's easier to fight, like, a squad or a gang of people. So, being Head of Community with such a young company is definitely a lot different than. In some ways, harder to deal with this type of community where we're just growing and emerging, rather than something more well-established.Corey: It probably gives you an interesting opportunity. Because back when I was doing engineering work as an SRE or whatever we call them in that era, it was, “Yeah, wow, my boss is terrible and has no idea what the hell they're doing.” So, then I found myself in the role, and it's, “Cool. Now, do all the things that you said you would do. Put up or shut up.”And it turns out that there's a lot you don't see that our strategic considerations. I completely avoided things like managing up or managing laterally or balancing trade-offs in different ways. Yeah, you're right. If you view the role of management as strictly being something that is between you and your direct reports, you can be an amazing manager from their perspective, but completely ineffective organizationally at accomplishing the goals that have been laid out for you.Wesley: Yeah. The good thing about being head of and the first head of is that you help establish those goals. And so, when you take a role with another company saying, “Hey, we have headcount for this,” and it's an established role, then you're kind of like streamlining into a process that's already underway. What's good about this role specifically, a ‘head of,' is that I help with not only designing what are the goals and the OKRs but deciding what the teams and what the team structure should look like. And so, I'm hiring for a specific position based on how it interacts with everything else.So, when I'm coming in, I don't say, “Well, what do you do?” Or, “How do you do it?” I said, “This is what needs to be done.” And that makes it so much easier just to say that if everything is working the way it should and to give marching orders based on the grand vision, instead of hitting the numbers this quarter or next quarter. Because what is core to my belief, and what's core, too, of how I approach things is at the heart of what I'm trying to do, which is really great, in terms of making something that didn't exist before.Corey: The challenge, too, is that everyone loves to say—and I love to see this at different ways—is the evolution and understanding of the DevRel folks who I work with and I have great relationships with realizing that you have to demonstrate business value. Because I struggle with this my entire career where I know intrinsically, that if I get on stage and tell a story about a thing that is germane to what my company does, that good things are going to happen. But it's very hard to do any form of attribution to it. In a different light, this podcast is a great example of this.We have sponsors. And people are listening. Ideally, they aren't fast-forwarding through sponsor messages; I do have interesting thoughts about the sponsors that I put into these ads. And that's great, but I also appreciate that people are driving while they're listening to this, and they are doing the dishes, they are mowing the lawn, and hopefully not turning up the volume too loudly so it damages their hearing. And the idea that they're going to suddenly stop any of those things and go punch in the link that I give is a little out to lunch there.Instead, it's partially brand awareness and it is occasionally the, “Wait. That resonates exactly with the problem that I have.” So, they get to work or they get back in front of a computer and the odds are terrific they're not going to punch in that URL of whatever I wound up giving; they're going to type in whatever phrases they remember and the company name into Google. Now—and doing attribution on something like that is very hard.It gets even more hard when we're talking about something that is higher up the stack that requires a bit more buy-in than individual developers. There's often a meeting or two about it. And then someone finally approaches the company to have a conversation. Now, does it work? Yes. There are companies that are sponsoring this stuff that spend a lot of time, effort, and money on that.I don't know how you do that sort of attribution; I don't pretend to know, but I know that it works. Because these people whose entire job is making sure that it does tell me it does. So, I smile, I nod, and that's great. But it's very hard to wind up building out a direct, “If you spend X dollars sponsoring this, you will see Y dollars in response.” But in the DevOps world, when your internal doing these things, well, okay because to the company, I look an awful lot like an expensive developer except I don't ever write production code.And then—at least in the before times—“So, what does your job do? Because looking at the achievements and accomplishments last quarter, it looks an awful lot like you traveled to exotic places on the company dime, give talks that are of only vague relevance to what we do, and then hang out at parties with your friends? Nice job, how can I get that?” But it's also first on the chopping block when okay, how do we trim expenses go? And I think it's a mistake to do that. I just don't think that story of the value of developer relations is articulated super-well. And I say that, but I don't know how to do a much better job of it myself.Wesley: Well, that's why corporate or executive buy-in is important because if they know from the get-go while you're there, it makes it a little bit easier to sell. But you do have to show that you are executing. So, there are always two parts to presenting a story, and that's one, the actual quantitative, like, I've done this many talks—so that output part—I've written this many blog posts, or I've stood up this many events that people can attend to. And then there's the results saying, people did read this post, people did show up to my event, people did listen to my talk that I gave. But you also need to give the subjective ones where people respond back and say, “I loved your talk,” or, “I heard you on Corey's podcast,” or, “I read your blog posts,” because even though you might not understand that it goes all the way down in a conversion funnel to a purchase, you can least use that stand-in to say there's probably, like, 20, 30 people behind this person to have that same sentiment, so you can see that your impact is reaching people and that it's having some sort of lasting effect.That said, you have to keep it up. You have to try to increase your output and increase your sphere of influence. Because when people go to solve their problem, they're going to look into their history and their own Rolodex of saying what was the last thing that I heard? What was the last thing that's relevant?There is a reason that Pepsi and Coke still do advertising. It's not because people don't know those brands, but being easily recalled, or a center of relevance based on how many touchpoints or how many times that you've seen them, either from being on American Idol and the logo facing the camera, or seeing a whole display when you go into the grocery store. Same with display advertising. All of this stuff works hand in hand so that you can be front-of-mind with the people and the decision-makers who will make that decision. And we went through this through the pandemic where… that same sentiment, it was like, “You just travel and now you can't travel, so we're just going to get rid of the whole department.”And then those same companies are hunting for those people to come back or to rebuild these departments that are now gone because maybe you don't see what we do, but when it's gone, you definitely notice a dip. And that trust is from the top-up. You have to do not just external advocacy, but you have to do internal advocacy about what impacts you're having so that at least the people who are making that decision can hopefully understand that you are working hard and the work is paying off.Corey: Since the last time that we spoke, you've given your first keynote, which—Wesley: Yes.Corey: Is always an interesting experience to go through. It was at a conference called THAT Conference. And I feel the need to specify that because otherwise, we're going to wind up with a ‘who's on first' situation. But THAT Conference is the name.Wesley: Specify THAT. Yes.Corey: Exactly. Better specify THAT. Yes. So, what was your keynote about? And for a bit of a behind-the-scenes look, what was that like for you?Wesley: Let me do the behind-the-scenes because it's going to lead up to actual the execution.Corey: Excellent.Wesley: So, I've been on several different podcasts. And one of the ones that I loved for years is one called This Week in Tech with Leo Laporte. Was a big fan of Leo Laporte back in the Screen Saver days back in TechTV days. Loved his opinion, follow his work. And I went to a South by Southwest… three, four years ago where I actually met him.And then from that conversation, he asked me to be on his show. And I've been on the show a handful of times, just talking about tech because I love tech. Tech is my passion, not just doing it, but just experiencing and just being on either side of creating or consuming. When I moved—I moved recently also since, I think, from the last time I was on your show—when I moved here to Wisconsin, the organizer of THAT Conference said that he's been following me for a while, since my first appearance on This Week in Tech, and loved my outlook and my take on things. And he approached me to do a keynote.Since I am now Wisconsin—THAT Conference is been in Wisconsin since inception and it's been going on for ten years—and he wanted me to just basically share my knowledge. Clean slate, have enough time to just say whatever I wanted. I said, “Yes, I can do that.” So, my experience on my end was like sheer excitement and then quickly sheer terror of not having a framework of what I was going to speak on or how I was going to deliver it. And knowing as a keynote, that it would be setting the tone for the whole conference.So, I decided to talk on the thing that I knew the most about, which was myself. Talked about my journey growing up and learning what my strengths, what my weaknesses are, how to navigate life, as well as the corporate jungle, and deciding where I wanted to go. Do I want to be the person that I feel like I need to be in order to be successful, which when we look at structures and examples and the things that we hold on a pedestal, we feel that we have to be perfect, or we have to be knowledgeable, and we have to do everything, well rounded in order to be accepted. Especially being a minority, there's a lot more caveats in terms of being socially acceptable to other people. And then the other path that I could have taken, that I chose to take, was to accept my things that are seen as false, but my own quirkiness, my own uniqueness and putting that front and center about, this is me, this is my person that over the years has formed into this version of myself.I'm going to make sure that is really transparent and so if I go anywhere, they know what they're getting, and they know what they're signing up for by bringing me on board. I have an opinion, I will share my opinion, I will bring my whole self, I won't just be the person that is technical or whimsical, or whatever you're looking for. You have to take the good with the bad, you have to take the I really understand technology, but I have ADHD and I might miss some deadlines. [laugh].Corey: This episode is sponsored in parts by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on premises, private cloud, and they just announced a fully managed service on AWS and Azure called BigAnimal, all one word.Don't leave managing your database to your cloud vendor because they're too busy launching another half dozen manage databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications, including Oracle, to the cloud.To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: I have a very similar philosophy, and how I approach these things where it's there is no single speaking engagement that I can fathom even being presented to me, let alone me accepting that is going to be worth me losing the reputation I have developed for authenticity. It's you will not get me to turn into a shill for whatever it is that I am speaking in front of this week. Conversely, whether it's a paid speaking engagement or not, I have a standing policy of not using a platform that is being given to me by a company or organization to make them look foolish. In other words, I will not make someone regret inviting me to speak at their events. Full stop.And I have spoken at events for AWS; I have spoken at events for Oracle, et cetera, et cetera, and there's no company out there that I'm not going to be able to get on stage and tell an entertaining and engaging story, but it requires me to dunk on them. And that's fine. Frankly, if there is a company like that where I could not say nice things about them—such as Facebook—I would simply decline to pursue the speaking opportunity. And that is the way that I view it. And very few companies are on that list, to be very honest with you.Now, there are exceptions to this, if you're having a big public keynote, I will do my traditional live-tweet the keynote and make fun of people because that is, A, expected and, B, it's live-streamed anywhere on the planet I want to be sitting at that point in time, and yeah, if you're saying things in public, you can basically expect that to be the way that I approach these things. But it's a nuanced take, and that is something that is not fully understood by an awful lot of folks who run events. I'll be the first to admit that aspects of who and what I am mean that some speaking engagements are not open to me. And I'm okay with that, on some level, I truly am. It's a different philosophy.But I do know that I am done apologizing for who I am and what I'm about. And at some point that required a tremendous amount of privilege and a not insignificant willingness to take a risk that it was going to work out all right. I can't imagine going back anymore. Now, that road is certainly not what I would recommend to everyone, particularly folks earlier in their career, particularly for folks who don't look just like I do and have a failure mode of a board seat and a book deal somewhere, but figuring out where you will and will not compromise is always an important thing to get straight for yourself before you're presented with a situation where you have to make those decisions, but now there's a whole bunch of incentive to decide in one way or another.Wesley: And that's a journey. You can't just skip sections, right? You didn't get to where you are unless you went through the previous experience that you went through. And it's true for everyone. If you see those success books or how-to books written by people who are extremely rich, and, like, how to become successful and, like, okay, well, that journey is your own. It doesn't make it totally, like, inaccessible to everyone else, but you got to realize that not everyone can walk that path. And—Corey: You were in the right place at the right time, an early employee at a company that did phenomenally well and that catapulted you into reach beyond the wildest dreams of avarice territory. Good for you, but fundamentally, when you give talks like that as a result, what it often presents as is, “I won the lottery, and here's how you can too.” It doesn't work that way. The road you walked was unique to you and that opportunity is closed, not open anyone else, so people have to find their own paths.Wesley: Yeah, and lightning doesn't strike in the same place twice. But there are some things where you can understand some fundamentals. And depending on where you go, I think you do need to know yourself, you do need to know—like, be able to access yourself, but being able to share that, of course, you have to be at a point where you feel comfortable. And so, even if you're in a space where you don't feel that you can be your authentic self or be able to share all parts of you, you yourself should at least know yourself and then make that decision. I agree that it's a point of privilege to be able to say, “Take me how I am.”I'm lucky that I've gotten here, not everyone does, and just because you don't doesn't mean that you're a failure. It just means that the world hasn't caught up yet. People who are part of marginalized society, like, if you are, let's say trans, or if you are even gay, you take the same person, the same stance, the same yearning to be accepted, and then transport it to 50 years ago, you're not safe. You will not necessarily be accepted, or you may not even be successful. And if you have a lane where you can do that, all the power to you, but not everyone could be themselves, and you just need to make sure that at least you can know yourself, even if you don't share that with the world.Corey: It takes time to get there, and I think you're right that it's impossible to get there without walking through the various steps. It's one of the reasons I'm somewhat reluctant to talk overly publicly about my side project gig of paid speaking engagements, for instance, is that the way to get those is you start off by building a reputation as a speaker, and that takes an awful lot of time. And speaking at events where there's no budget even to pay you a speaking fee out of anyway. And part of what gets the keynote invitations to, “Hey, we want you to come and give a talk,” is the fact that people have seen you speak elsewhere and know what you're about and what to expect. Here's a keynote presented by someone who's never presented on stage before is a recipe for a terrifying experience, if not for the speaker or the audience, definitely [laugh] for the event organizers because what if they choke.?Easy example of this, even now hundreds of speaking engagements in, the adrenaline hit right before I go on stage means that sometimes my knees shake a bit before I walk out on stage. I make it a point to warn the people who are standing with me backstage, “Oh, this is a normal thing. Don't worry, it is absolutely expected. It happens every time. Don't sweat it.”And, like, “Thank you for letting us know. That is the sort of thing that's useful.” And then they see me shake, and they get a little skeptical. Like, I thought this guy was a professional. What's the story and I walk on stage and do my thing and I come back. Like, “That was incredible. I was worried at the beginning.” “I told you, we all have our rituals before going on stage. Mine is to shake like a leaf.”But the value there is that people know what to generally expect when I get on stage. It's going to have humor, there's going to be a point interwoven throughout what I tend to say, and in the case of paid speaking engagements, I always make sure I know where the boundaries are of things I can make fun of a big company for. Like, I can get on stage and make fun of service naming or I can make fun of their deprecation policy or something like that, but yeah, making fun of the way that they wind up handling worker relations is probably not going to be great and it could get the person who championed me fired or centered internally. So, that is off the table.Like, even on this podcast, for example, I sometimes get feedback from listeners of, “Well, you have someone from company X on and you didn't beat the crap out of them on this particular point.” It's yeah, you do understand that by having people on the show I'm making a tacit agreement not to attack them. I'm not a journalist. I don't pretend to be. But if I beat someone up with questions about their corporate policy, yeah, very rarely do I have someone who is in a position in those companies to change that policy, and they're certainly not authorized to speak on the record about those things.So, I can beat them up on it, they can say, “I can't answer that,” and we're not going to go anywhere. What is the value of that? It looks like it's not just gotcha journalism, but ineffective gotcha journalism. It doesn't work that way. And that's never been what this show is about.But there's that consistent effort behind the scenes of making sure that people will be entertained, will enjoy what they're seeing, but also are not going to deeply regret giving me a microphone, has always been the balancing act, at least for me. And I want to be clear, my style is humor. It is not for everyone. And my style of humor has a failure mode of being a jerk and making people feel bad, so don't think that my path is the only or even a recommended way for folks who want to get more into speaking to proceed.Wesley: You also mention, though, about, like, punching up versus punching down. And if you really tear down a company after you've been invited to speak, what you're doing is you're punching down at the person who booked you. They're not the CEO; they're not the owner of the company; they're the person who's in charge of running an event or booking speakers. And so, putting that person and throwing them under the bus is punching down because now you're threatening their livelihood, and it doesn't make any market difference in terms of changing the corporate's values or how they execute. So yeah, I totally agree with you in that one.And, like you were saying before, if there's a company you really thought was abhorrent, why speak there? Why give them or lend your reputation to this company if you absolutely feel that it's something you don't want to be associated with? You can just choose not to do that. For me, when I look at speaking, it is important for me to really think about why I'm speaking as well. So, not just the company who's hiring me, but the audience that I'll be serving.So, if I'm going to help with inspiring the next generation of developers, or helping along the thought of how to make the world a better place, or how people themselves can be better people so that we can just change the landscape and make it a lot friendlier, that is also its own… form of compensation and not just speaking for a speaker's fee. So, I do agree that you need to not just be super Negative Nancy, and try to fight all fights. You need to embrace some of the good things and try to make more of those experiences good for everyone, not just the people who are inviting you there, but the people who are attending. And when I started speaking, I was not a good speaker as well. I made a lot of mistakes, and still do, but I think speaking is easier than some people think and if someone truly wants to do it, they should go ahead and get started.What is the saying? If there's something is truly important, you'll be bad at it [laugh] and you'll be okay with it. I started speaking because of my role as a developer advocate. And if you just do a Google search for ‘CFPs,' you can start speaking, too. So, those who are not public speakers and want to get into it, just Google ‘CFP' and then start applying.And then you'll get better at your submissions, you'll get better at your slides, and then once you get accepted, then you'll get better at preparing, then you'll get better at actually speaking. There's a lot of steps between starting and stopping and it's okay to get started doing that route. The other thing I wanted to point out is I feel public speaking is the equivalent of lifting your own bodyweight. If you can do it, you're one of the small few of the population that is willing to do so or that can do it. If you start public speaking, that in itself is an accomplishment and an experience that is something that is somewhat enriching. And being bad at it doesn't take the passion away from you. If you just really want to do it, just keep doing it, even if you're a bad speaker.Corey: Yeah. The way to give a great talk because you have a bunch of terrible talks first.Wesley: Yeah. And it's okay to do that.Corey: And it's not the in entirety of community. It's not even a requirement to be involved with the community. If you're one of those people that absolutely dreads the prospect of speaking publicly, fine. I'm not suggesting that, oh, you need to get over that and get on stage. That doesn't help anyone. Don't do the things you dread doing because you know that it's not going to go well for you.That's the reason I don't touch actual databases. I mean, come on, let's be realistic. I will accidentally the data, and then we won't have a company anymore. So, I know what things I'm good at and things I'm not. I also don't do hostage negotiations, for obvious reasons.Wesley: And also, here's a little, like, secret tip. If you really want to do public speaking and you start doing public speaking and you're not so good at it from other peoples' perspective, but you still love doing it and you think you're getting better, doing public speaking is one of those things where you can say that you do it and no one will really question how good you are at it. [laugh]. If you're just in casual conversation, it's like, “Hey, I wrote a book.” People like, “Oh, wow. This person wrote the book on blah, blah, blah.”Corey: It's a self-published book that says the best way to run Kubernetes. It's a single page; it says, “Don't.” In 150-point type. “The end.” But I wrote a book.Wesley: Yeah.Corey: Yeah.Wesley: People won't probe too much and it'll help you with your development. So, go ahead and get started. Don't worry about doing that thing where, like, I have to be the best before I can present it. Call yourself a public speaker. Check, done.Corey: Always. We are the stories we tell, and nowhere is it more true than in the world of public speaking. I really want to thank you for taking the time out of your day to speak with me about this for a second time in a single year. Oh, my goodness. If people want to learn more about what you're up to, where can they find you?Wesley: I'm on Twitter, @wesley83 on Twitter. And you can find me also on PolyWork. So, polywork.com/wesley83. Or just go to wesleyfaulkner.com which redirects you there. I list pretty much everything that I am working on and any upcoming speaking opportunities, hopefully when they release that feature, will also be on that Polywork page.Corey: Excellent. And of course, I started Polywork recently, and I'm at thoughtleader.cloud because of course I am, which is neither here nor there. Thank you so much for taking the time to speak about this side of the industry that we never really get to talk about much, at least not publicly and not very often.Wesley: Well, thank you for having me on the show. And I wanted to take some time to say thank you for the work that you're doing. Not just elevating voices like myself, but talking truth to power, like we mentioned before, but being yourself and being a great representation of how people should be treating others: being honest without being mean, being snarky without being rude. And other companies and other people who've given me a chance, and given me a platform, I wanted to say thank you to you too, and I wouldn't be here unless it was people like you acknowledging the work that I've been doing.Corey: All it takes is just recognizing what you're doing and acknowledging it. People often want to thank me for this stuff, but it's just, what, for keeping my eyes open? I don't know, I feel like it's just the job; it's not something that is above and beyond any expected normal behavior. The only challenge is I look around the industry and I realize just how wrong that impression is, apparently. But here we are. It's about finding people doing interesting work and letting them tell their story. That's all this podcast has ever tried to be.Wesley: Yeah. And you do it. And doing the work is part of the reward, and I really appreciate you just going through the effort. Even having your ears open is something that I'm glad that you're able to at least know who the people are and who are making noises—or making noise to raise their profile up and then in turn, sharing that with the world. And so, that's a great service that you're providing, not just for me, but for everyone.Corey: Well, thank you. And as always, thank you for your time. Wesley Faulkner, Head of Community at SingleStore. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with a rambling comment telling me exactly why DevRel does not need success metrics of any kind.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About JamesJames has been part of AWS for over 15 years. During that time he's led software engineering for Amazon EC2 and more recently leads the AWS Commerce Platform group that runs some of the largest systems in the world, handling volumes of data and request rates that would make your eyes water. And AWS customers trust us to be right all the time so there's no room for error.Links Referenced:Email: jamesg@amazon.comTranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Vultr. Optimized cloud compute plans have landed at Vultr to deliver lightning-fast processing power, courtesy of third-gen AMD EPYC processors without the IO or hardware limitations of a traditional multi-tenant cloud server. Starting at just 28 bucks a month, users can deploy general-purpose, CPU, memory, or storage optimized cloud instances in more than 20 locations across five continents. Without looking, I know that once again, Antarctica has gotten the short end of the stick. Launch your Vultr optimized compute instance in 60 seconds or less on your choice of included operating systems, or bring your own. It's time to ditch convoluted and unpredictable giant tech company billing practices and say goodbye to noisy neighbors and egregious egress forever. Vultr delivers the power of the cloud with none of the bloat. “Screaming in the Cloud” listeners can try Vultr for free today with a $150 in credit when they visit getvultr.com/screaming. That's G-E-T-V-U-L-T-R dot com slash screaming. My thanks to them for sponsoring this ridiculous podcast.Corey: Finding skilled DevOps engineers is a pain in the neck! And if you need to deploy a secure and compliant application to AWS, forgettaboutit! But that's where DuploCloud can help. Their comprehensive no-code/low-code software platform guarantees a secure and compliant infrastructure in as little as two weeks, while automating the full DevSecOps lifestyle. Get started with DevOps-as-a-Service from DuploCloud so that your cloud configurations are done right the first time. Tell them I sent you and your first two months are free. To learn more visit: snark.cloud/duplo. Thats's snark.cloud/D-U-P-L-O-C-L-O-U-D. Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. And I've been angling to get someone from a particular department at AWS on this show for nearly its entire run. If you were to find yourself in an Amazon building and wander through the various dungeons and boiler rooms and subterranean basements—I presume; I haven't seen nearly as many of you inside of those buildings as people might think—you pass interesting departments labeled things like ‘Spline Reticulation,' or whatnot. And then you come to a very particular group called Commerce Platform.Now, I'm not generally one to tell other people's stories for them. My guest today is James Greenfield, the VP of Commerce Platform at AWS. James, thank you for joining me and suffering the slings and arrows I will no doubt be hurling at you.James: Thanks for having me. I'm looking forward to it.Corey: So, let's start at the very beginning—because I guarantee you, you're going to do a better job of giving the chapter and verse answer than I would from a background mired deeply in snark—what is Commerce Platform? It sounds almost like it's the retail website that sells socks, books, and underpants.James: So, Commerce Platform actually spans a bunch of different things. And so, I'm going to try not to bore you with a laundry list of all of the things that we do—it's a much longer list than most people assume even internal to AWS—at its core, Commerce Platform owns all of the infrastructure and processes and software that takes the fact that you've been running an EC2 instance, or you're storing an object in S3 for some period of time, and turns it into a number at the end of the month. That is what you asked for that service and then proceeds to try to give you as many ways to pay us as easily as possible. There are a few other bits in there that are maybe less obvious. One is we're also responsible for protecting the platform and our customers from fraudulent activity. And then we're also responsible for helping collect all of the data that we need for internal reporting to support some of the back-ends services that a business needs to do things like revenue recognition and general financial reporting.Corey: One of the interesting aspects about the billing system is just how deeply it permeates everything that happens within AWS. I frequently say that when it comes to cloud, cost and architecture are foundationally and fundamentally the same exact thing. If your entire service goes down, a few interesting things happen. One, I don't believe a single customer is going to complain other than maybe a few accountants here and there because the books aren't reconciling, but also you've removed a whole bunch of constraints around why things are the way that they are. Like, what is the most efficient way to run this workload?Well, if all the computers suddenly become free, I don't really care about efficiency, so much is, “Oh, hey. There's a fly, what do I have as a flyswatter? That's right, I'm going to drop a building on it.” And those constraints breed almost everything. I've said, for example, that S3 has infinite storage because it does.They can add drives faster than we're able to fill them—at least historically; they added some more replication services—but they're going to be able to buy hard drives faster than the rest of us are going to be able to stretch our budgets. If that constraint of the budget falls away, all bets are really off, and more or less, we're talking about the destruction of the cloud as a viable business entity. No pressure or anything.James: [laugh].Corey: You're also a recent transplant into AWS billing as a whole, Commerce Platform in general. You spent 15 years at the company, the vast majority of that over an EC2. So, either it was you've been exiled to a basically digital Siberia or it was one of those, “Okay, keeping all the EC2 servers up, this is easy. I don't see what people stress about.” And they say, “Oh, ho ho, try this instead.” How did you find yourself migrating over to the Commerce Platform?James: That's actually one I've had a lot from folks that I've worked with. You're right, I spent the first 15 or so years of my career at AWS in EC2, responsible for various things over there. And when the leadership role in Commerce Platform opened up, the timing was fortuitous, and part of it, I was in the process of relocating my family. We moved to Vancouver in the middle of last year. And we had an opening in the role and started talking about, potentially, me stepping into that role.The reason that I took it—there's a few reasons, but the primary reason is that if I look back over my career, I've kind of naturally gravitated towards owning things where people only really remember that they exist when they're not working. And for some reason, you know, I enjoy the opportunity to try to keep those kinds of services ticking over to the point where people don't notice them. And so, Commerce Platform lands squarely in that space. I've always been attracted to opportunities to have an impact, and it's hard to imagine having much more of an impact than in the Commerce Platform space. It underpins everything, as you said earlier.Every single one of our customers depends on the service, whether they think about it or realize it. Every single service that we offer to customers depends on us. And so, that really is the sort of nexus within AWS. And I'm a platform guy, I've always been a platform guy. I like the force multiplier nature of platforms, and so Commerce Platform, you know, as I kind of thought through all of those elements, really was a great opportunity to step in.And I think there's something to be said for, I've been a customer of Commerce Platform internally for a long time. And so, a chance to cross over and be on the other side of that was something that I didn't want to pass up. And so, you know, I'm digging in, and learning quickly, ramping up. By no means an expert, very dependent on a very smart, talented, committed group of people within the team. That's kind of the long and short of how and why.Corey: Let's say that I am taking on the role of an AWS product team, for the sake of argument. I know, keep the cringe down for a second, as far as oh, God, the wince is just inevitable when the idea of me working there ever comes up to anyone. But I have an idea for a service—obviously, it runs containers, and maybe it does some other things as well—going from idea to six-pager to MVP to barely better than MVP day-one launch, and at some point, various things happen to that service. It gets staff with a team, objectives and a roadmap get built, a P&L and budget, and a pricing model and the rest. One the last thing that happens, apparently, is someone picks the worst name off of a list of candidates, slaps it on the product, and ships it off there.At what point does the billing system and figuring out the pricing dimensions for a given service tend to factor in? Is that a last-minute story? Is that almost from the beginning? Where along that journey does, “Oh, by the way, we're building this thing. Maybe we should figure out, I don't know, how to make money from it.” Factor into the conversation?James: There are two parts to that answer. Pretty early on as we're trying to define what that service is going to look like, we're already typically thinking about what are the dimensions that we might charge along. The actual pricing discussions typically happen fairly late, but identifying those dimensions and, sort of, the right way to present it to customers happens pretty early on. The thing that doesn't happen early enough is actually pulling the Commerce Platform team in. but it is something that we're going to work this year to try to get a little bit more in front of.Corey: Have you found historically that you have a pretty good idea of how a service is going to be priced, everything is mostly thought through, a service goes to either private preview or you're discussing about a launch, and then more or less, I don't know, someone like me crops up with a, “Hey, yeah, let's disregard 90% of what the service does because I see a way to misuse the remaining 10% of it as a database.” And you run some mental math and realize, “Huh. We're suddenly giving, like, eight petabytes of storage per customer away for free. Maybe we should guard against that because otherwise, it's rife with misuse.” It used to be that I could find interesting ways to sneak through the cracks of various services—usually in pursuit of a laugh—those are getting relatively hard to come by and invariably a lot more trouble than they're worth. Is that just better comprehensive diligence internally, is that learning from customers, or am I just bad at this?James: No, I mean, what you're describing is almost a variant of the Defender's Dilemma. They are way more ways to abuse something than you can imagine, and so defending against that is pretty challenging. And it's important because, you know, if you turn the economics of something upside down, then it just becomes harder for us to offer it to customers who want to use it legitimately. I would say 90% of that improvement is us learning. We make plenty of mistakes, but I think, you know, one of the things that I've always been impressed by over my time here is how intentional we are trying to learn from those mistakes.And so, I think that's what you're seeing there. And then we try very hard to listen to customers, talk to folks like you, because one of the best ways to tackle anything it smells of the Defender's Dilemma is to harness that collective creativity of a large number of smart people because you really are trying to cover as much ground as possible.Corey: There was a fun joke going around a while back of what is the most expensive environment you can get running on a free tier account before someone from AWS steps in, and I think I got it to something like half a billion dollars in the first month. Now, I haven't actually tested this for reasons that mostly have to do with being relatively poor compared to, you know, being able to buy Guam. And understanding as well the fraud protections built into something like AWS are largely built around defending against getting service usage for free that in some way, shape or form, benefits the attacker. The easy example of that would be mining cryptocurrency, which is just super-economic as long as you use someone else's AWS account to do it. Whereas a lot of my vectors are, “Yeah, ignore all of that. How do I just make the bill artificially high? What can I do to misuse data transfer? And passing a single gigabyte through, how much can I make that per gigabyte cost be?” And, “Oh, circular replication and the Lambda invokes itself pattern,” and basically every bad architectural decision you can possibly make only this time, it's intentional.And that shines some really interesting light on it. And I have to give credit where due, a lot of that didn't come from just me sitting here being sick and twisted nearly so much as it did having seen examples of that type of misconfiguration—by mistake—in a variety of customer accounts, most confidently my own because it turns out that the way I learn things is by screwing them up first.James: Yeah, you've touched on a couple of different things in there. So, you know, maybe the first one is, I typically try to draw a line between fraud and abuse. And fraud is essentially trying to spend somebody else's money to get something for free. And we spent a lot of time trying to shut that down, and we're getting really good at catching it. And then abuse is either intentional or unintentional. There's intentional abuse: You find a chink in our armor and you try to take advantage of it.But much more commonly is unintentional abuse. It's not really abuse, you know. Abuse has very negative connotations, but it's unintentionally setting something up so that you run up a much larger bill than you intended. And we have a number of different internal efforts, and we're working on a bunch more this year, to try to catch those early on because one of my personal goals is to minimize the frequency with which we surprise customers. And the least favorite kind of surprise for customers is a [laugh] large bill. And so, what you're talking about there is, in a sufficiently complex system, there's always going to be weaknesses and ways to get yourself tied up in knots.We're trying both at the service team level, but also within my teams to try to find ways to make it as hard as possible to accidentally do that to yourself and then catch when you do so that we can stop it. And even more on the intentional abuse side of things, if somebody's found a way to do something that's problematic for our services, then you know, that's pretty much on us. But we will often reach out and engage with whoever's doing and try to understand what they're trying to do and why. Because often, somebody's trying to do something legitimate, they've got a problem to solve, they found a creative way to solve it, and it may put strain on the service because it's just not something we designed for, and so we'll try to work with them to use that to feed into either new services, or find a better place for that workload, or just bolster what they're using. And maybe that's something that eventually becomes a fully-fledged feature that we offer the customers. We're always open to learning from our customers. They have found far more creative ways to get really cool things done with our services than we've ever imagined. And that's true today.Corey: I mean, most of my service criticisms come down to the fact that you have more-or-less built a very late model, high performing iPad, and I'm out there complaining about, “What a shitty hammer this thing is, it barely works at all, and then it breaks in my hand. What gives?” I would also challenge something you said a minute ago that the worst day for some customers is to get a giant surprise bill, but [unintelligible 00:13:53] to that is, yeah, but, on some level, that kind of only money; you do have levers on your side to fix those issues. A worse scenario is you have a customer that exhibits fraud-like behavior, they're suddenly using far more resources than they ever did before, so let's go ahead and turn them off or throttle them significantly, and you call them up to tell them you saved them some money, and, “Our Superbowl ad ran. What exactly do you think you're doing?” Because they don't get a second bite at that kind of Apple.So, there's a parallel on both sides of this. And those are just two examples. The world is full of nuances, and at the scale that you folks operate at. The one-in-a-million events happen multiple times a second, the corner cases become common cases, and I'm surprised—to be direct—how little I see you folks dropping the ball.James: Credit to all of the teams. I think our secret sauce, if anything, really does come down to our people. Like, a huge amount of what you see as hopefully relatively consistent, good execution comes down to people behind the scenes making sure. You know, like, some of it is software that we built and made sure it's robust and tested to scale, but there's always an element of people behind the scenes, when you hit those edge cases or something doesn't quite go the way that you planned, making sure that things run smoothly. And that, if anything, is something that I'm immensely proud of and is kind of amazing to watch from the inside.Corey: And, on some level, it's the small errors that are the bigger concern than the big ones. Back a couple years ago, when they announced GP3 volumes at re:Invent, well, great, well spin up a test volume and kick the tires on it for an hour. And I think it was 80 or 100 gigs or whatnot, and the next day in the bill, it showed up as about $5,000. And it was, “Okay, that's not great. Not great at all.” And it turned out that it was a mispricing error by I think a factor of a million.And okay, at least it stood out. But there are scenarios where we were prepared to pay it because, oops, you got one over on us. Good job. That's never been the mindset I've gotten about AWS's philosophy for pricing. The better example that I love because no one took it seriously, was a few years before that when there was a LightSail bug in the billing system, and it made the papers because people suddenly found that for their LightSail instance, they were getting predicted bills of $4 billion.And the way I see it, you really only had to make that work once and then you've made your numbers for the year, so why not? Someone's going to pay for it, probably. But that was such out-of-the-world numbers that no one saw that and ever thought it was anything other than a bug. It's the small pernicious things that creep in. Because the billing system is vast; I had no idea when I started working with AWS bills just how complicated it really was.James: Yeah, I remember both of those, and there's something in there that you touched on that I think is really important. That's something that I realized pretty early on at Amazon, and it's why customer obsession is our flagship leadership principle. It's not because it's love and butterflies and unicorns; customer obsession is key to us because that's how you build a long-term sustainable business is your customers depend on you. And it drives how we think about everything that we do. And in the billing space, small errors, even if there are small errors in the customer's favor, slowly erode that trust.So, we take any kind of error really seriously and we try to figure out how we can make sure that it doesn't happen again. We don't always get that right. As you said, we've built an enormous, super-complex business to growing really quickly, and really quick growth like that always acts as kind of a multiplier on top of complexity. And on the pricing points, we're managing millions of pricing points at the moment.And our tools that we use internally, there's always room for improvement. It's a huge area of focus for us. We're in the beginning of looking at applying things like formal methods to make sure that we can make very hard guarantees about the correctness of some of those. But at the end of the day, people are plugging numbers in and you need as many belts and braces as possible to make sure that you don't make mistakes there.Corey: One of the things that struck me by surprise when I first started getting deep into this space was the fact that the finalized bill was—what does it mean to have this be ‘finalized?' It can hit the Cost and Usage Report in an S3 bucket and it can change retroactively after the month closed periodically. And that's when I started to have an inkling of a few things: Not just the sheer scale and complexity inherent to something like the billing system that touches everything, but the sheer data retention stories where you clearly have to be able to go back and reconstruct a bill from the raw data years ago. And I know what the output of all of those things are in the form of Cost and Usage Reports and the billing data from our client accounts—which is the single largest expense in all of our AWS accounts; we spent thousands and thousands and thousands of dollars a year just on storing all of that data, let alone the processing piece of it—the sheer scale is staggering. I used to wonder why does it take you a day to record me using something to it's showing up in the bill? And the more I learned the more it became a how can you do that in only a day?James: Yes, the scale is actually mind-boggling. I'm pretty sure that the core of our billing system is—I'm reasonably confident it's the largest or one of the largest data processing systems on the planet. I remember pretty early on when I joined Commerce Platform and was still starting to wrap my head around some of these things, Googling the definition of quadrillion because we measured the number of metering events, which is how we record usage in services, on a daily basis in the quadrillions, which is a billion billions. So, it's just an absolutely staggering number. And so, the scale here is just out of this world.That's saying something because it's not like other services across AWS are small in their own right. But I'm still reasonably sure that being one of a handful of services that is kind of at the nexus of AWS and kind of deals with the aggregate of AWS's scale, this is probably one of the biggest systems on the planet. And that shows up in all sorts of places. You start with that input, just the sheer volume of metering events, but that has to produce as an output pretty fine-grained line item detailed information, which ultimately rolls up into the total that a customer will see in their bill. But we have a number of different systems further down the pipeline that try to do things like analyze your usage, make sensible recommendations, look for opportunities to improve your efficiency, give you the ability to slice and dice your data and allocate it out to different parts of your business in whatever way it makes sense for your business. And so, those systems have to deal with anywhere from millions to billions to recently, we were talking about trillions of data points themselves. And so, I was tangentially aware of some of the scale of this, but being in the thick of it having joined the team really just does underscore just how vast the systems are.Corey: I think it's, on some level, more than a little unfortunate that that story isn't being more widely told, more frequently. Because when Commerce Platform has job postings that are available on the website, you read it and it's very vague. It doesn't tend to give hard numbers about a lot of these things, and people who don't play in these waters can easily be forgiven for thinking the way that you folks do your job is you fire up one of those 24 terabyte of RAM instances that—you know, those monstrous things that you folks offer—and what do you do next? Well, Microsoft Excel. We have a special high memory version that we've done some horse-trading with our friends over at Microsoft for.It's, yeah, you're several steps beyond that, at this point. It's a challenging problem that every one of your customers has to deal with, on some level, as well. But we're only dealing with the output of a lot of the processing that you folks are doing first.James: You're exactly right. And a big focus for some of my teams is figuring out how to help customers deal with that output. Because even if you're talking about couple of orders of magnitude reduction, you're still talking about very large numbers there. So, to help customers make sense of that, we have a range of tools that exist, we're investing in.There's another dimension of complexity in the space that I think is one that's also very easy to miss. And I think of it as arbitrary complexity. And it's arbitrary because some of the rules that we have to box within here are driven by legislative changes. As you operate more and more countries around the world, you want to make sure that we're tax compliant, that we help our customers be tax compliant. Those rules evolve pretty rapidly, and Country A may sit next to Country B, but that doesn't mean that they're talking to one another. They've all got their own ideas. They're trying to accomplish r—00:22:47Corey: A company is picking up and relocating from India to Germany. How do we—James: Exactly.Corey: —change that on the AWS side and the rest? And it's, “Hoo boy, have you considered burning it all down and filing an insurance claim to start over?” And, like, there's a lot of complexity buried underneath that that just doesn't rise to the notice of 99% of your customers.James: And the fact that it doesn't rise to the notice is something that we strive for. Like, these shouldn't be things that customers have to worry about. Because it really is about clearing away the things that, as far as possible, you don't want to have to spend time thinking about so that you can focus on the thing that your business does that differentiates you. It's getting rid of that undifferentiated heavy lifting. And there's a ton of that in this space, and if you're blissfully unaware of it, then hopefully that means that we're doing our job.Corey: What I'm, I think, the most surprised about, and I have been for a long time. And please don't take this as an insult to various other folks—engineers, the rest, not just in other parts of AWS but throughout the other industry—but talking to the people who work within Commerce Platform has always been just a fantastic experience. The caliber of people that you have managed to attract and largely retain—we don't own people, they do matriculate out eventually—but the caliber of people that you've retained on your teams has just been out of this world. And at first, I wondered, why are these awesome people working on something as boring and prosaic as billing? And then I started learning a little bit more as I went, and, “Oh, wow. How did they learn all the stuff that they have to hold in their head in tension at once to be able to build things like this?” It's incredibly inspiring just watching the caliber of the people that you've been able to bring in.James: I've been really, really excited joining this team, as I've gotten other folks on the team because there's some super-smart people here. But what's really jumped out to me is how committed the team is. This is, for the most part, a team that has been in the space for many years. Many of them have—we talk about boomerangs, folks who live AWS, go spend some time somewhere else and come back and there's a surprisingly high proportion of folks in Commerce Platform who have spent time somewhere else and then come back because they enjoy the space, they find that challenging, folks are attracted to the ability to have an impact because it is so foundational. But yeah, there's a super-committed core to this team. And I really enjoy working with teams where you've got that because then you really can take the long view and build something great. And I think we have tons of opportunities to do that here.Corey: It sounds ridiculous, but I've reached out to team members before to explain two-cent variances in my bill, and never once have I been confronted with a, “It's two cents. What do you care?” They understand the requirement that these things be accurate, not just, “Eh, take our word for it.” And also, frankly, they understand that two cents on a $20 bill looks a little different on a $20 million bill. So yeah, let us figure out if this is systemic or something I have managed to break.It turns out the Cost and Usage Report processing systems don't love it when there's a cost allocation tag whose name contains an emoji. Who knew? It's the little things in life that just have this fun way of breaking when you least expect it.James: They're also a surprisingly interesting problem. So like, it turns out something as simple as rounding numbers consistently across a distributed system at this scale, is a non-trivial problem. And if you don't, then you do get small seventh or eighth decimal place differences that add up to something that then shows up as a two-cent difference somewhere. And so, there's some really, really interesting problems in the space. And I think the team often takes these kinds of things as a personal challenge. It should be correct, and it's not, so we should go make sure it is correct. The interesting problems abound here, but at the end of the day, it's the kind of thing that any engineering team wants to go and make sure it's correct because they know that it can be.Corey: This episode is sponsored in parts by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on premises, private cloud, and they just announced a fully managed service on AWS and Azure called BigAnimal, all one word. Don't leave managing your database to your cloud vendor because they're too busy launching another half dozen manage databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications, including Oracle, to the cloud.To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: On the one hand, I love people who just round and estimate—we all do that, let's be clear; I sit there and I back-of-the-envelope everything first. But then I look at some of your pricing pages and I count the digits after the zeros. Like, you're talking about trillionths of a dollar on some of your pricing points. And you add it up in the course of a given hour and it's like, oh, it's $250 a month, most months. And it's you work backwards to way more decimal places of precision than is required, sometimes.I'm also a personal fan of the bill that counts, for example, number of Route 53 zones. Great. And it counts them to four decimal places of precision. Like, I don't even know what half of it Route 53 zone is at this point, let alone something to, like, ah the 1,000th of the zone is going to cause this. It's all an artifact of what the underlying systems are.Can you by any chance shed a little light on what the evolution of those systems has been over a period of time? I have to imagine that anything you built in the early days, 16 years ago or so from the time of this recording when S3 launched to general availability, you probably didn't have to worry about this scope and scale of what you do, now. In fact, I suspect if you tried to funnel this volume through S3 back then, the whole thing would have collapsed under its own weight. What's evolved over the time that you had the billing system there? Because changes come slowly to your environment. And frankly, I appreciate that as a customer. I don't like surprising people in finance.James: Yeah, you're totally right. So, I joined the EC2 team as an engineer myself, some 16 years ago, and the very first thing that I did was our billing integration. And so, my relationship with the Commerce Platform organization—what was the billing team way back when—it goes back over my entire career at AWS. And at the time, the billing team was similar, you know, [unintelligible 00:28:34] eight people. And that was everything. There was none of the scale and complexity; it was all one system.And much like many of our biggest, oldest services—EC2 is very similar, S3 is as well—there's been significant growth over the last decade-and-a-half. A lot of that growth has been rapid, and rapid growth presents its own challenges. And you live with decisions that you make early on that you didn't realize were significant decisions that have pretty deep implications 15 years later. We're still working through some of those; they present their own challenges. Evolving an existing system to keep up with the growth of business and a customer base that's as varied and complex as ours is always challenging.And also harder but I also think more fun than a clean sheet redo at this point. Like, that's a great thought exercise for, well, if we got to do this again today, what would we do now that we've learned so much over the last 15 years? But there's this—I find it personally fascinating challenge with evolving a live system where it's like, “No, no, like, things exist, so how do we go from there to where we want to be next?”Corey: Turn the billing system off for 18 months, rebuild—James: Yeah. [laugh].Corey: The whole thing from first principles. Light it up. I'm sure you'd have a much better billing system, and also not a company left anymore.James: [laugh]. Exactly, exactly. I've always enjoyed that challenge. You know, even prior to AWS, my previous careers have involved similar kinds of constraints where you've got a live system, or you've got an existing—in the one case, it was an existing SDK that was deployed to tens of thousands of customers around the world, and so backwards compatibility was something that I spent the first five years of my career thinking about it way more detail than I think most people do. And it's a very similar mindset. And I enjoy that challenge. I enjoy that: How do I evolve from here to there without breaking customers along the way?And that's something that we take pretty seriously across AWS. I think SimpleDB is the poster child for we never turn things off. But that applies equally to the services that are maybe less visible to customers, and billing is definitely one of them. Like, we don't get to switch stuff off. We don't get to throw things away and start again. It's this constant state of evolution.Corey: So, let's say that I were to find a way to route data through a series of two Managed NAT Gateways and then egress to internet, and the sheer density of the expense of that traffic tears a hole in the fabric of space-time, it goes back 15 years ago, and you can make a single change to how the billing system was built. What would it be? What pisses you off the most about the current constraints that you have to work within or around?James: I think one of the biggest challenges we've got, actually, is the concept of an account. Because an account means half-a-dozen different things. And way back, when it seemed like a great idea, you just needed an account; an account was your customer, and it was the same thing as the boundary that you put all your resources inside. And of course, it's the same thing that you're going to roll all of your usage up and issue a bill against. And that has been one of the areas that's seen the most evolution and probably still has a pretty long way to go.And what's interesting about that is, that's probably something we could have seen coming because we watched the retail business go through, kind of, the same evolution because they started with, well, a customer is a customer is a customer and had to evolve to support the concept of sellers and partners. And then users are different than customers, and you want to log in and that's a different thing. So, we saw that kind of bifurcation of a single entity into a wide range of different related but separate entities, and I think if we'd looked at that, you know, thought out 15 years, then yeah, we could probably have learned something from that. But at the same time, when AWS first kicked off, we had wild ambitions for it, but there was no guarantee that it was going to be the monster that it is today. So, I'm always a little bit reluctant to—like, it's a great thought exercise, but it's easy to end up second-guessing a pretty successful 15 years, so I'm always a little bit careful to walk that line. But I think account is one of the things that we would probably go back and think about a little bit more.Corey: I want to be very clear with this next question that it is intentionally setting up a question I suspect you get a lot. It does not mirror my own thinking on the matter even slightly, but I get a version of it myself all the time. “AWS bills, that sounds boring as hell. Why would you choose to work on such a thing?” Now, I have a laundry list of answers to that aren't nearly as interesting as I suspect yours are going to be. What makes working on this problem space interesting to you?James: There's a bunch of different things. So, first and foremost, the scale that we're talking about here is absolutely mind-blowing. And for any engineer who wants to get stuck into problems that deal with mind-blowingly large volumes of data, incredibly rich dimensions, problems where, honestly, applying techniques like statistical reasoning or machine learning is really the only way to chip away at it, that exists in spades in the space. It's not always immediately obvious, and I think from the outside, it's easy to assume this is actually pretty simple. So, the scale is a huge part of that.Corey: “Oh, petabytes. How quaint.”James: [laugh]. Exactly. Exactly I mean, it's mind-blowing every time I see some of the numbers in various parts of the Commerce Platform space. I talked about quadrillions earlier. Trillions is a pretty common unit of measure.The complexity that I talked about earlier, that's a result of external environments is another one. So, imposed by external entities, whether it's a government or a tax authority somewhere, or a business requirement from customers, or ourselves. I enjoy those as well. Those are different kinds of challenge. They really keep you on your toes.I enjoy thinking of them as an engineering problem, like, how do I get in front of them? And that's something we spend a lot of time doing in Commerce Platform. And when we get it right, customers are just unaware of it. And then the third one is, I personally am always attracted to the opportunity to have an impact. And this is a space where we get to hopefully positively impact every single customer every day. And that, to me is pretty fulfilling.Those are kind of the three standout reasons why I think this is actually a super-exciting space. And I think it's often an underestimated space. I think once folks join the team and sort of start to dig in, I've never heard anybody after they've joined, telling me that what they're doing is boring. Challenging, yes. Is frustrating, sometimes. Hard, absolutely, but boring never comes up.Corey: There's almost no service, other than IAM, that I can think of that impacts every customer simultaneously. And it's easy for me to sit in the cheap seats and say, “Oh, you should change this,” or, “You should change that.” But every change you have is so massive in scale that it's going to break a whole bunch of companies' automations around the bill processing in different ways. You have an entire category of user persona who is used to clicking a certain button in this certain place in the console to generate the report every month, and if that button moves or changes color, or has a different font, suddenly that renders their documentation invalid, and they're scrambling because it's not their core competency—nor should it be—and every change you make is so constricted, just based upon all the different concerns that you've got to be juggling with. How do you get anything done at all? I find that to be one of the most impressive aspects about your organization, bar none.James: Yeah, I'm not going to lie and say that it isn't a challenge, but a lot of it comes down to the talent that we have on the team. We have a super-motivated, super-smart, super-engaged team, and we spend a lot of time figuring out how to make sure that we can keep moving, keep up with the business, keep up with a world that's getting more complicated [laugh] with every passing day. So, you've kind of hit on one of the core challenges there, which is, how do we keep up with all of those different dimensions that are demanding an increasing amount of engineering and new support and new investment from us, while we keep those customers happy?And I think you touched on something else a little bit indirectly there, which is, a lot of our customers are actually pretty technical across AWS. The customers that Commerce Platform supports, are often the least technical of our customers, and so often need the most help understanding why things are the way they are, where the constraints are.Corey: “A big bill from Amazon. How many books did you people buy last month?”—James: [laugh]. Exactly.Corey: —is still very much level of understanding in some cases. And it's not because they're dumb; far from it. It's just, imagine that some people view there as being more to life than understanding the nuances and intricacies of cloud computing. How dare they?James: Exactly. Who would have thought?Corey: So, as you look now over all of your domain, such as it is, what sucks the most? What are you looking to fix as far as impactful changes that the rest of the world might experience? Because I'm not going to accept one of those questions like, “Oh, yeah, on the back-end, we have this storage subsystem for a tertiary thing that just annoys me because it wakes us up once in a whi”—no, no, I want something customer-facing. What's the painful thing you're looking at fixing next?James: I don't like surprising customers. And free tier is, sort of, one of those buckets of surprises, but there are others. Another one that's pretty squarely in my sights is, whether we like it or not, customer accounts get compromised. Usually, it's a password got reused somewhere or was accidentally committed into a GitHub repository somewhere.And we have pretty established, pretty effective mechanisms for finding all of those, we'll scan for passwords and credentials, and alert customers to those, and help them correct that pretty quickly. We're also actually pretty good at detecting when an account does start to do something that suggests that it's been compromised. Usually, the first thing that a compromised account starts to do is cryptocurrency mining. We're pretty quick to catch those; we catch those within a matter of hours, much faster most days.What we haven't really cracked and where I'm focused at the moment is getting back to the customer in a way that's effective. And by that I mean specifically, we detect an account compromised super-quickly, we reach out automatically. And so, you know, a customer has got some kind of contact from us usually within a couple of hours. It's not having the effect that we need it to. Customers are still being surprised a month later by a large bill. And so, we're digging into how much of that is because they never saw the contact, they didn't know what to do with the contact.Corey: It got buried with all the other, “Hey, we saw you spun up an S3 bucket. Have you heard of what S3 is?” Again, that's all valuable, but you have 300-some-odd services. If you start doing that for every service, you're going to hit mail sending limits for Gmail.James: Exactly. It's not just enough that we detect those and notify customers; we have to reduce the size of the surprise. It's one thing to spend 100 bucks a month on average, and then suddenly find that your spend has jumped $250 because you reused the password somewhere and somebody got ahold of it and it's cryptocurrency-mining your account. It's a whole different ballgame to spend 100 bucks a month and then at the end of the month discover that your bill is suddenly $2,000 or $20,000. And so, that's something that I really wanted to make some progress on this year. Corey: I've really enjoyed our conversation. If people want to learn more about how you view these things, how you're approaching some of these problems, or potentially are just the right kind of warped to consider joining up, where's the best place for them to go?James: They should drop me an email at jamesg@amazon.com. That is the most direct way to get hold of me, and I promise I will get back to you. I try to stay on top of my email as much as possible. But that will come straight to me, and I'm always happy to talk to folks about the space, talk to folks about opportunities in this team, opportunities across AWS, or just hear what's not working, make sure that it's something that we're aware of and looking at.Corey: Throughout Amazon, but particularly within Commerce Platform, I've always appreciated the response of, whenever I report something, no matter how ridiculous it is—and I assure you there's an awful lot of ridiculousness in my bug reports—the response has always been the same: “Tell me more. Help me understand what it is you're trying to achieve—even if it is ridiculous—so we can look at this and see what is actually going on.” Every Amazonian team has been great about that or you're not at Amazon very long, but you folks have taken that to an otherworldly level. I just want to thank you for doing that.James: I appreciate you for calling that out. We try, you know, we really do. We take listening to our customers very seriously because, at the end of the day, that's what makes us better, and that's how we make sure we're in it for the long haul.Corey: Thanks once again for being so generous with your time. I really appreciate it.James: Yeah, thanks for having me on. I've enjoyed it.Corey: James Greenfield, VP of Commerce Platform at AWS. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment—possibly on YouTube as well—about how you aren't actually giving this five-stars at all; you have taken three trillions of a star off of the rating.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
About PriyankaPriyanka Vergadia is currently a Staff Developer Advocate at Google Cloud where she works with enterprises to build and architect their cloud platforms. She enjoys building engaging technical content and continuously experiments with new ways to tell stories and solve business problems using Google Cloud tools. You can check out some of the stories that she has created for the developer community on the Google Cloud Platform Youtube channel. These include "Deconstructing Chatbots", "Get Cooking in Cloud", "Pub/Sub Made Easy" and more. ..Links Referenced: LinkedIn: https://www.linkedin.com/in/pvergadia/ Twitter: https://twitter.com/pvergadia Priyanka's book: https://www.amazon.com/Visualizing-Google-Cloud-Illustrated-References/dp/1119816327 TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Finding skilled DevOps engineers is a pain in the neck! And if you need to deploy a secure and compliant application to AWS, forgettaboutit! But that's where DuploCloud can help. Their comprehensive no-code/low-code software platform guarantees a secure and compliant infrastructure in as little as two weeks, while automating the full DevSecOps lifestyle. Get started with DevOps-as-a-Service from DuploCloud so that your cloud configurations are done right the first time. Tell them I sent you and your first two months are free. To learn more visit: snark.cloud/duplo. Thats's snark.cloud/D-U-P-L-O-C-L-O-U-D. Corey: What if there were a single place to get an inventory of what you're running in the cloud that wasn't "the monthly bill?" Further, what if there were a way to compare that inventory to what you were already managing via Terraform, Pulumi, or CloudFormation, but then automatically add the missing unmanaged or drifted parts to it? And what if there were a policy engine to immediately flag and remediate a wide variety of misconfigurations? Well, stop dreaming and start doing; visit snark.cloud/firefly to learn more.Corey: Welcome to Screaming in the Cloud, I'm Corey Quinn. Periodically, I get the privilege of speaking to people who work in varying aspects of some would call it developer evangelism, some would call it developer advocacy, developer relations is a commonly accepted term, and I of course call it devrelopers because I enjoy annoying absolutely everyone by giving things terrible names. My guest today is Priyanka Vergadia, who is a staff developer advocate at Google Cloud. Priyanka, thank you for joining me.Priyanka: Thank you so much for having me. Corey. I'm so excited to be your developer—what did you call it again?Corey: Devreloper. Yes indeed.Priyanka: Devreloper. That is the term I'm going to be using from now on. I am a devreloper. Anyway.Corey: Excellent.Priyanka: Yeah.Corey: I'm starting to spread this out so that eventually we're going to form a giant, insufferable army of people who pronounce it that way, and it's going to be great.Priyanka: It's going to be awesome. [laugh].Corey: One of the challenges, even as I alluded to different titles within this space, everyone has a slightly different definition of where the role starts and stops, just in terms of its function, let alone the myriad ways that can be expressed. In the before times, I knew a number of folks in the developer advocacy space who were more or less worldwide experts in accumulating airline miles and racking up status and going from conference to conference to conference to more or less talk about things that had a tenuous at best connection to where they worked. Great. Other folks have done things in very different ways. Some people write extensively, blog posts and the rest, others build things a sample code, et cetera, et cetera.It seems like every time I talk to someone in the space, they have found some new and exciting way of carrying the message of what their company does to arguably a very cynical customer group. Where do you start and stop with your devrelopment?Priyanka: Yeah. So, that is such—like, all the devrelopers have their own style that they have either adopted or learned over time that works for them. When I started, I think about three years ago, I did go to conferences, did those events, give talks, all of that, but I was also—my actual introduction to DevRel [laugh] was with videos. I started creating my first series was deconstructing chatbots, and I was very interested in learning more about chatbots. So, I was like, you know what, I'm just going to teach everybody, and learn.So like, learn and teach at the same time was my motto, and that's kind of how I got started into, like, okay, I'm going to create a few videos to learn this and teach it. And during the process I was like, “I want to do this more.” And that's kind of transitioned, my move from being in front of customers, which I still end up doing, but I was doing more of just, you know, working with customers extensively to get their deployments done. This was a segue for me to, you know, think back, sit back and think about what's working and what I personally enjoy doing more, and that's what got me into creating videos. And it's like, okay, I'm going to become a devreloper now.And that's kind of how the whole, like, journey started. And for me, like you were pointing out earlier—should I just stop because I've been talking too long? [laugh].Corey: No, keep going. Please, [unintelligible 00:04:10] it's fine.Priyanka: [laugh]. For me, I started—I found my, I would say, in the last two years—it was all before the pandemic, we were all either writing blogs or doing videos or going to conferences, so it was, you know, the pandemic kind of brought us to a point where it's like, “Okay, let's think about—we can't meet each other; let's think about other ways to communicate and how can we make it creative and exciting?”Corey: And the old way started breaking down, too, where it's, “Yay, I'm going to watch an online conference.” “What is it?” “Oh, it's like a crappy Zoom only you don't have to pretend to pay attention in the same way.” And as a presenter, then you've got to modify what you're doing to understand that people's attention spans are shorter, distraction is always a browser tab away, and unlike a physical event, people don't feel the same sense of shame of getting up from the front row and weaving in front of 300 people, and not watching the rest of your talk. I mean, don't get me wrong, I'll still do it, but I'll feel bad about it.Now it's, “Oh, nope, I'm sitting here in my own little… hovel, I'm just going to do and watch whatever I want to do.” So, you've got to—it forces you to up your game, and it—Priyanka: Yep.Corey: Still doesn't quite have the same impact.Priyanka: Yeah. Or just switch off the camera, if you're like me, and just—uh, shut off the camera, go away or do something else. And, yeah, it's very easy to do that. So, it's not the same, which is why it prompted, I think all of us DevRel people to think about new ways to connect, which is for me that way to connect is art and visual aspects, to kind of bring that—because that—we are all whether we accept it or not or like it or not, we're all visual learners, so that's kind of how I think when it comes to creating content is visually appealing, and that's when people can dive in. [laugh].Corey: I am in the, I guess opposite side of the universe from you, where I acknowledge and agree with everything you're saying that people are visual creatures inherently, but I have effectively zero ability in that direction. My medium has always been playing games with words and language. And over time, I had the effectively significantly belated realization that wait a minute, just because I'm not good at a thing doesn't mean that other people might not be good at that thing, and I don't have to do every last part of it myself. Suddenly, I didn't have to do my own crappy graphic design because you can pay people who are worlds better than I'll ever be, and so on and so forth. I don't edit my own podcast audio because I'm bad at that, too.But talking about things is a different story, writing about things, building things is where I tend to see a lot of what I do tend to resonate. But I admit I bias for the things that I enjoy doing and the way that I enjoy consuming things. You do as well because relatively recently, as of time of this recording, you have done what I don't believe anyone actually wants to do. You wrote a book. Now, everyone wants to have written a book, but no one actually wants to write a book.Priyanka: So, true. [laugh].Corey: But it's not like most technical books. Tell me about it.Priyanka: Yeah, I actually never thought I would write a book. If you asked me two years ago—three years ago, I would say, I would have never thought that I would write a book because I am not a text person. So, I don't like to read a lot of texts because it zones out. So, for me, when I started creating some of these sketches, and sharing it on social media and in blogs and things like that, and gotten the attention that it has gotten from people, that's when I was like, okay, ding, ding, ding. I think I can do a visual book with these images.And this was like, halfway through, I'd already created, like, 30 sketches at this point. And I was like, “Okay, maybe I can turn this into a book,” which would be interesting for me because I like doing art-type things along with teaching, and it's not text because I wanted to do this in a very unique way. So yeah, that's kind of how it ended up happening.Corey: I have a keen appreciation for people who approach things with a different point of view. One of your colleagues, Forrest Brazeal, took a somewhat similar approach in the in his book, The Read Aloud Cloud, where it was illustrated, and everything he did was in rhyme, which is a constant source of envy for me, where it's, “Mmm, I've got to find a way to one-up him again.” And it's… he is inexorable, as far as just continuing to self-improve. So, all right, we're going to find a way to wind up defeating that. With you, it's way easier.I read a book, like, wow, this is gorgeous and well-written that it's attractive to look at, and I will never be able to do any of those things. That's all you. It doesn't feel like we're trying to stand at the same spot in the universe in quite the same way. Nothing but love for Forrest. Let's be clear. I am teasing. I consider him a friend.Priyanka: He is amazing. Well honestly, like, I actually got to know Forrest when I decided to do this book. Wiley, who's the publisher, sent me Forrest's book, and he said, “You should look at this book because the idea that you are presenting to me, we could lay it out in this format.” Like, in the, you know, physical format. So, he sent me that book. And that's how I know Forrest, honestly.So, I told him that—this is a little story that I told him after. But anyway, yeah. I—the—[sigh]—I was going to make a point about the vid—the aspect of creating images, like, honestly, like, I designed the aspects of, like, how you layout information in the sketches, I studied a bunch of stuff to come up with, how do I make it precise and things like that. But there's no way this book was possible without some design help. Like, I can't possibly do the entire thing unless I have, like, five years. [laugh]. So—Corey: Right on top of all of this, you do presumptively have a day job as well—and while—Priyanka: Exactly.Corey: This is definitely related. “I'm just going to go write a book.” “Oh, is it a dissertation?” “No, it's going to look more like a children's book than that,” is what they're going to hear. And it's yeah, I'm predicting some problems with the performance evaluation process at large companies when you start down those paths.Priyanka: Exactly. So, I ended up, like, showing all these numbers, like, of the blog views and reads and social media, the presence of some of these images that were going wider. And in the GCPSketchnote GitHub repo got a huge number of stars. And it was like, everybody could see that writing a book would be amazing. From that point on, I was just like, I don't think I can scale that.So, when I was drawing—this is an example—when I drew my first sketch, it took me an entire weekend to just draw one sketch, which is what—I was only doing that the entire weekend—like, assume, like, 16 hours of work, just drawing the one sketch. So, if I went with that pace, this book was not possible. So, you know, after I had the idea laid out, had the process in place, I got some design help, which made it—which expedited the process much, much faster. [laugh].Corey: There's a lot to be said, for doing something that you enjoy. Do you do live sketchnoting during conference talks as well, or do you tend to not do it while someone is talking at a reasonably fast clip, and well, in 45 minutes, this had better be done, so let's go. I've seen people who can do that, and I just marvel in awe at what they do.Priyanka: I don't do live. I don't do live sketching. For me, paper and pen is a better medium so that's just the medium that I like to work with. So, when the talk is happening, I'm actually taking notes on a pen and a paper. And then after, I can sketch it out, faster in a fast way.Like, I did one sketchnote for Next 2020, I think, and that was done, like, a day after Next was over so I could take all the bits and pieces that were important and put it into that sketch. But I can't do it live. That's just one of the things I haven't figured out yet. [laugh].Corey: For me, I was always writing my email newsletter, so it was relatively rapid turnaround, and Twitter was interesting for me. I finally cracked the nut on how to express myself in a way that worked. The challenge that I ran into then was okay, there are thoughts I occasionally have that don't lend themselves to then 140—now 280—characters, so I should probably start writing long-form. And then I want to start writing 1000 to 1500-word blog posts every week that goes out. And that forced me to become a better writer across the board. And then it became about one-upping myself, sort of, live-tweeting conference talks.And the personal secret of why I do that is I'm ADHD in a bottle. Someone gets on stage—you say you zone out when you read a giant quantity of data; you prefer something more visual, more interactive. For me, I'm the opposite, where when someone gets on stage and starts talking, it's, “Okay, get to—yes, you're doing the intro of what a cloud might be. I get that point. This is supposed to be a more advanced talk. Can we speed it up a bit?”And doing the live-tweeting about it, but not just relating what is said, but by making a joke about it, it's how I keep myself engaged and from zoning out. Because let's face it, this industry is extraordinarily boring, if you don't bring a little bit of light to it.Priyanka: Yeah, that is—Corey: And that how to continue and how to do that was hard, and it took me time to get there.Priyanka: Yeah. Yeah, no, I totally agree. Like, that's exactly why I got into, like, training videos and sketches. Like, and videos and also. Like, I come up with, like, fake examples of companies that may or may not exist.Like, I made up a dog shoe making company that ships out shoes when you need them and then return them and there's a size and stuff, like, you have to come up with interesting things to make the content interesting because otherwise, this can get boring pretty quickly, which is going back to your example of, “Speed it up; get to the point.” [laugh].Corey: This episode is sponsored in parts by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on premises, private cloud, and they just announced a fully managed service on AWS and Azure called BigAnimal, all one word. Don't leave managing your database to your cloud vendor because they're too busy launching another half dozen manage databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications, including Oracle, to the cloud.To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: It's always just fun to start experimenting with it, too, because all right, once I was done learn learning how to live-tweet other people's talk and mostly get it correct because someone says something, I have three to five seconds to come up with what I want to talk about and maybe grab a picture and then move on to the next thing. And it's easy to get that wrong and say things you don't necessarily intend to and get taken the wrong way. I've mostly gotten past that. And—I'm not saying I'm always right, but I better than I used to be. And then it was okay, “How do I top this?”And I started live-tweeting conference talks that I was giving live, which is always fun, but being able to pre-write some tweets at certain times, have certain webhooks in your slide deck and whatnot that fire these things off. And again, I'm not saying that he this is recommended or even a good idea, but it definitely wasn't boring. And—Priyanka: Yeah.Corey: And continue to find ways to make the same type of material new and interesting is one of the challenges because the stuff is complex.Priyanka: Also bite-size, right? Like, it's—I think Twitter is, like, the [unintelligible 00:15:54] words are obviously limiting, but it also forces you to think about it in bite-size, right? Like, okay, if I have a blog post then I'm summarizing it, how would I do it in two sentences? It forces me to think about it that way, which makes it very applicable to the time span that we have now, right, which is maybe, like, 30 seconds, you can have somebody on [unintelligible 00:16:18]Corey: Attention is a rare and precious commodity.Priyanka: Yeah. Yeah.Corey: People who [unintelligible 00:16:21] engagement, I think that's the wrong metric to go after because that inspires a whole bunch of terrible incentives, whereas finding something that is interesting, and a way to bring light to it and have a perspective on it that makes people think about it differently. For me, it's been humor, but that's my own approach to things. Your direction, it seems to be telling a story through visual arts. And that is something we don't see nearly as much of.Priyanka: Yeah. I think it's also because it's something that you—you know, like, I grew up drawing and painting. I was drawing since I was three years old, so that's my way of thinking. Like, I don't—I was talking to another devreloper the other day, and we were talking about—Corey: It's catching on. I love it.Priyanka: —[laugh]. Two different ways of how we think. So, for me, when I design a piece of content, I have my visuals first, and then he was talking about when he designs his content, he has his bullet points and a blog post first. So, it's like, two very different ways of approaching this similar thing. And then from that, from the images or the deck that I'm building up, I would come up with the narrative and stuff like that.My thinking starts with images and narrative of tying, like, the images together. But it's, that is the whole, like, fun of being in DevRel, right? Like, you are your own personality, and bringing whatever your personality, like you mentioned, humor and your case, art in my case, in somebody else's case, it could be totally different thing, right? So, yeah.Corey: Now, please correct me if I'm wrong on this, but an area of emphasis for you has been data analytics as well as Kubernetes, more or less things that are traditionally considered to be much more back-end if you're looking at a spectrum of all things technology. Is that directionally accurate, or am I dramatically is understanding a lot of what you're saying?Priyanka: No, that's very much accurate. I like to—I tend to be on the infrastructure back-and creating pipeline, creating easier processes, sort of person, not much into front-end. I dabble into it, but don't enjoy it. [laugh].Corey: This makes you something of a unicorn, in the sense of there are a tremendous number of devreloper types in the front-end slash JavaScript world because their entire career is focused on making things look visually appealing. That is what front-end is. I know this because I am rubbish at it. My idea of a well-designed interface that everyone looks at and smiles at [unintelligible 00:19:12] of command-line arguments when you're writing a script for something. And it's on a green screen, and sometimes I'll have someone helped me coordinate to come up with a better color palette for the way that I'm looking at my terminal on my Mac. Real exciting times over here, I assure you.So, the folks who are working in that space and they have beautifully designed slides, yeah, you tend to expect that. I gave a talk years ago at the front-end conference in Zurich, and I was speaking in the afternoon. And I went there and every presentation, slides were beautiful. And this was before I was working here and had a graphic designer on retainer to make my slides look not horrible. It was black Helvetica text on a white background, and I'm looking at this and I'm feeling ashamed that it's—okay, I have two hours to fix this. What do I do?I did the only thing I could think of; I changed Helvetica text to Comic Sans because if it's going to look terrible and it's going to be a designer thing that puts them off, you may as well go all-in. And that was a recurring meme at the time. I've since learned that there is an argument—I don't know if it's true or not—that Comic Sans is easier to read for folks with dyslexia, for example. And that's fine. I don't know if that's accurate or not, but I stopped making jokes about it just because if people—even if it's not true, and people believe that it's, “Are you being unintentionally crappy to people?” It's, “Well, I sure hope not. I'm rarely intentionally crappy. But when I do, I don't want to be mistaken for not being.” It's, save it up and use it when it counts.Priyanka: Yeah, yeah. I've—yeah, I think, when it comes to these big events—and like front-end for me is—I would think, like, I actually thought that I would be great at front-end because I have interest in art and stuff. I do make things that [crosstalk 00:20:57]—Corey: That's my naive assumption, too. I'm learning as you speak here. Please continue[.Priyanka: Yeah. And I was just—I thought that I would be and I have tried it, and I only like it to an extent, to present my idea. But I don't like to go in deeper and, like, make my CSS pretty or make this—make it look pretty. I am very much intrigued by all the back-end stuff, and most of my experience, over the past ten years in Cloud has been in the back-end stuff, mainly just because I love APIs, I love—like, you know, as long as I can connect, or the idea of creating a demo or something that involves a bunch of APIs and a back-end, to present an idea in a front-end, I would work on that front-end. But otherwise, I'm not going to choose to do it. [laugh]. Which I found interesting for myself as well. It's a realization. [laugh].Corey: Every time I try and do something with front-end, it doesn't matter the framework, I find myself more confused at the end than I was when I started. There's something I don't get. And anytime I see someone on Twitter, for example, talking about how a front-end is easier or somehow less than, I read that and I can't help myself. It's, “You ridiculous clown. You have no idea what you're talking about.”I don't believe that I'm bad at all of the things under engineering—just most of them—and I think I pick things up reasonably quickly. It is a mystery that does not align with this, and if it's easy for you, you don't recognize—arguably—a skill that you have, but not everyone does, by a landslide. And that's a human nature thing, too. It's if it was easy for me, it's obviously easy for everyone. If something's hard for me, no one would understand how this works and the people that do are wizards from the future.Priyanka: Yep. So true.Corey: It never works that way.Priyanka: Yeah. It never works that way. At least we have this in common, that you don't like to work on front-ends. [laugh].Corey: There's that too. And I think that no matter where you fall on the spectrum of technology, I would argue that something that we all share in common is, it doesn't matter how far we are down in the course of our entire career, from the very beginning to the very end, it is always a consistent, constant process of being humbled and made to feel like a fool by things you are supposedly professionally good at. And oh my stars, I've just learned to finally give up and embrace it. It's like, “So, what's going to make me feel dumb today?”Priyanka: Exactly.Corey: It's the learn in public approach, which is important.Priyanka: It's so important. Especially, like, if you're thinking about it, like that's the part of DevRel that makes it so exciting, too, right? Like, just learning a new thing today and sharing it with you. Like, I'm not claiming that I'm an expert, but hey, let's talk about it. And sure, I might end up looking dumb one day, I might end up looking smart the other day, but that's not the point. The point is, I end up learning every day, right? And that's the most important part, which is why I love this particular job, which is—what did we call it—devreloper.Corey: Devreloping. And as a part of that, you're talking to people constantly, be it people in the community and ecosystem, people who—you say you've talk to customers, but you also talk to these other folks. I would challenge you on that, where when you're at a company like Google Cloud, increasingly everyone in the community in the ecosystem is in one way or another, indistinguishable from being your customer; it all starts to converge at some point. All major cloud providers have that luxury, to be perfectly honest. What do you see in the ecosystem that people are struggling with as you talk to them?And again, any one person is going to have a problem or bone to pick with some particular service or implementation, and okay, great. What I'm always interested in is what is the broad sweep of things? Because when I hear someone complaining that a given service from a given cloud provider is terrible. Okay, great. Everyone has an opinion. When I started to hear that four or five, six times, it's okay, there's something afoot here, and now I'm curious as to what it is. What patterns are you seeing emerge these days?Priyanka: Yeah. I think more and more patterns along the lines of how can you make it automated? How can you make anything automated, right? Like, from machine learning's perspective, how do I not need ML skills to build an ML model? Like, how can we get there faster, right?Same for, like, in the infrastructure side, the serverless… aspect? How can you make it easy for me so I can just build an application and just deploy it so it becomes your problem to run it and not mine?Corey: Oh, the—you are preaching to the choir on that. I feel like all of these services that talk about, “This is how you build and train a machine learning model,” yadda, yadda, it's for an awful lot of the use cases out there, it's exposing implementation details about which I could not possibly care less. It's the, I want an API that I throw something at—like, be it a picture—and then I want to get a response of, “Yes, it's a hot dog,” or, “That's disgusting,” or whatever it is that it decides that it wants to say, great because that's the business outcome I'm after, and I do not care what wizardry happens on the back-end, I don't care if it's people who are underpaid and working extremely quickly by hand to do it, as long as it's from a business perspective, it hits a certain level of performance, reliability, et cetera. And then price, of course, yeah.And that is not to say I'm in favor of exploiting people, let's be clear here because I'm pretty sure most of these are not actually humans on the back-end, but okay. I just want that as the outcome that I think people are after, and so much of the conversation around how to build and train models and all misses the point because there are companies out there that need that, absolutely, there are, but there are a lot more that need the outcome, not the focus on this. And let's face it, an awful lot of businesses that would benefit from this don't have the budget to hire the team of incredibly expensive people it takes to effectively leverage these things because I have an awful lot of observations about people in machine learning space, one of them is absolutely not that, “Wow, I bet those people are inexpensive for me to hire.” It doesn't work that way.Priyanka: It doesn't. Yeah. And so, yeah. I think the future of, like, the whole cloud space, like, when it started, we started with how can I run my server not in my basement, but somewhere else, right? Now, we are at a different stage where we have a different sets of problems and requirements for businesses, right?And that's where I see it growing. It's like, how can I make this automated fast, not my problem? How can I make it not my problem is, like, the biggest [laugh] biggest, I think, theme that we are seeing, whether it's infrastructure, data science, data analytics, in all of these spaces.Corey: I get a lot of interesting feedback for my comparative takes on the various cloud providers, and one thing that I've said for a while about Google Cloud has been that its developer experience is unparalleled compared to basically anything else on the market. It makes things just work, and that's important because a bad developer experience has the unfortunate expression—at least for me—of, “Oh, this isn't working the way I want it to. I must be dumb.” No, it's a bad user experience for you. What I am seeing emerge as well from Google Cloud is an incredible emphasis—and I do think they're aligned here—on storytelling, and doing so effectively.You're there communicating visually; Forrest is there, basically trying to be the me of Google Cloud—which is what I assume he's doing; he would argue everything about that and he'd be right to do it, but that's what I'm calling it because this is my show; he can come on and argue with me himself if he takes issue with it. But I love the emphasis on storytelling and unifying solutions and the rest, as opposed to throwing everything at the wall to see what sticks to it. I think there's more intention being put into an awful lot of not just what you're building, but how you're talking about it, now it's integrated with the other things that you're building. That's no small thing.Priyanka: Yeah. That is so hard, especially when you know the cloud space; like, hundreds of products, they all have their unique requirement to solve a problem, but nobody cares, right? Like, as a consumer, I shouldn't have to care that there are 127 products or whatever. It doesn't matter to me as a consumer or customer, all that matters is whether I can solve my business problem with a set of your tools, right? So, that's exactly why, like, we have this team that I work in that I'm a part of, which has an entire focus on storytelling.We do YouTube videos with storytelling, we do art like this, I've also dabbled into comics a little bit. And we continue to go back to the drawing board with how else we can tell these stories. I know—I mentioned this to Forrest—I'm working on a song as well, which I have never done before, and [laugh] I think I'm going to butcher it. I kind of have it ready for, like, six months but never released it, right, because I'm just too scared to do that. [laugh] but anyway.Corey: Ship and then turn the internet off for a week and it'll be gone regardless, by the time you come back. Problem solved until the reporters start calling, and then you have problems.Priyanka: I might have to just do that, and be, like, you know what world? Keep saying whatever you want to say, I'm not here. [laugh]. But anyway, going back to that point of storytelling, and it's so—I think we have weaved it into the process. And it's going really well, and now we are investing more in, like, R&D and doing more of how we can tell stories in different ways.Corey: I have to say, I'm a big fan of the way that you're approaching this. If people want to learn more about what you're up to—and arguably, as I argue they should get a copy of your book because it is glorious—where's the best place to find you?Priyanka: Thank you. Okay, so LinkedIn and Twitter are my platforms that I check every single day, so you can message me, connect with me, I am available as—my handle is pvergadia. I don't know if they have [crosstalk 00:31:11]—Corey: Oh, this is all going in the [show notes 00:31:13] you need not worry.Priyanka: Okay, perfect. So yeah, I don't have to spell it because my last name is hard. [laugh]. So, you'll find it in the show notes. But yeah, you can connect with me there. And you will find at the top of both of my profiles, the link to order the book, so you can do it there.Corey: Excellent. And I've already done so, and I'm just waiting for it to arrive. So, this is—it's going to be an exciting read if nothing else. One of these days, I'd have to actually live-tweet a reading thereof. We'll see how that plays out.Priyanka: That would be amazing.Corey: Be careful what you wish for. Some of the snark could be a little too cutting; we have to be cautious of that.Priyanka: [laugh]. I'm always scared of your tweets. Like, do I want to read this or not? [laugh].Corey: If nothing else, it at least tries to be funny. So, there is that.Priyanka: Yes. Yes, for sure.Corey: I really—Priyanka: No, I'm excited. I'm excited for when you get a chance to read it and just tweet whatever you feel like, from, you know, all the bits and pieces that I've brought together. So, I would love to get your take. [laugh].Corey: Oh, you will, one way or another. That's one of those non-optional things. It's one of the fun parts of dealing with me. It's, “Aw crap. That shitposter is back again.” Like the kid outside of your yard just from across the street, staring at your house and pointing and it's, “Oh, dear. Here we go.” Throwing stones.Priyanka: [laugh]. I'm excited either way. [laugh].Corey: He's got a platypus with him this time. What's going on? It happens. We deal with what we have to. Thank you so much for being so generous with your time. I appreciate it.Priyanka: Thank you so much for having me. It was amazing. You are a celebrity, and I wanted to be, you know, a part of your show for a long time, so I'm glad we're able to make it work.Corey: You are welcome back anytime.Priyanka: I will. [laugh].Corey: An absolute pleasure to talk with you. Thanks again.Priyanka: Thank you.Corey: Priyanka Vergadia staff developer—but you call it developer advocate—at Google Cloud. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on whatever platform you're using to listen to this thing, whereas if you've hated it, please do the exact same thing, making sure to hit the like and subscribe buttons on the YouTubes because that's where it is. But if you did hate it, also leave an insulting, angry comment but not using words. I want you to draw a picture telling me exactly what you didn't like about this episode.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
https://go.dok.community/slack https://dok.community/ ABSTRACT OF THE TALK 6 months have passed since our last DoK webinar about benchmarking PostgreSQL workloads in a Kubernetes environment. In the meantime, many things have happened at EDB, and we're happy to share what we've learned in this timeframe. We'll use cnp-bench and cnp-sandbox to help us describe some of the challenges we might face when running PostgreSQL workloads, how to spot them, and what actions to take to make your databases healthier and more longeve. cnp-bench is a collection of Helm charts that help run storage and database benchmarks, using popular open source tools like fio, pgbench, and HammerDB. cnp-sandbox is a Helm chart that sets up a Prometheus/Grafana stack, including basic metrics and dashboards for Cloud Native PostgreSQL, the Kubernetes operator developed by EDB. Both cnp-sandbox and cnp-bench are open source and recommended for development, testing, and pre-production environments only. BIO A long time open-source programmer and entrepreneur, Gabriele has a degree in Statistics from the University of Florence. After having consistently contributed to the growth of 2ndQuadrant and its members through nurturing a lean and devops culture, he is now leading the Cloud Native initiative at EDB. Gabriele lives in Prato, a small but vibrant city located in the northern part of Tuscany, Italy - famous for having hosted the first European PostgreSQL conferences. His second home is Melbourne, Australia, where he studied at Monash University and worked in the ICT sector. He loves playing the Blues with his Fender Stratocaster, but his major passions are called Elisabeth and Charlotte! KEY TAKE-AWAYS FROM THE TALK - A methodology for benchmarking a PostgreSQL database in Kubernetes - Open source set of tools for benchmarking a PostgreSQL database in Kubernetes - Reasons why benchmarking both the storage and the database is important https://github.com/EnterpriseDB/cnp-sandbox https://github.com/EnterpriseDB/cnp-bench
About EdEd Boyajian, President and CEO of EDB, drives the development and execution of EDB's strategic vision and growth strategy in the database industry, steering the company through 47 consecutive quarters of recurring revenue growth. He also led EDB's acquisition of 2ndQuadrant, a deal that brought together the world's top PostgreSQL experts and positioned EDB as the largest dedicated provider of PostgreSQL products and solutions worldwide. A 15+ year veteran of the open source software movement, Ed is a seasoned enterprise software executive who emphasizes that EDB must be a technology-first business in order to lead the open source data management ecosystem. Ed joined EDB in 2008 after serving at Red Hat, where he rose to Vice President and General Manager of North America. While there, he played a central leadership role in the development of the modern business model for bringing open source to enterprises.Links:EDB: https://enterprisedb.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by Honeycomb. When production is running slow, it's hard to know where problems originate: is it your application code, users, or the underlying systems? I've got five bucks on DNS, personally. Why scroll through endless dashboards, while dealing with alert floods, going from tool to tool to tool that you employ, guessing at which puzzle pieces matter? Context switching and tool sprawl are slowly killing both your team and your business. You should care more about one of those than the other, which one is up to you. Drop the separate pillars and enter a world of getting one unified understanding of the one thing driving your business: production. With Honeycomb, you guess less and know more. Try it for free at Honeycomb.io/screaminginthecloud. Observability, it's more than just hipster monitoring. Corey: This episode is sponsored in part by our friends at Jellyfish. So, you're sitting in front of your office chair, bleary eyed, parked in front of a powerpoint and—oh my sweet feathery Jesus its the night before the board meeting, because of course it is! As you slot that crappy screenshot of traffic light colored excel tables into your deck, or sift through endless spreadsheets looking for just the right data set, have you ever wondered, why is it that sales and marketing get all this shiny, awesome analytics and inside tools? Whereas, engineering basically gets left with the dregs. Well, the founders of Jellyfish certainly did. That's why they created the Jellyfish Engineering Management Platform, but don't you dare call it JEMP! Designed to make it simple to analyze your engineering organization, Jellyfish ingests signals from your tech stack. Including JIRA, Git, and collaborative tools. Yes, depressing to think of those things as your tech stack but this is 2021. They use that to create a model that accurately reflects just how the breakdown of engineering work aligns with your wider business objectives. In other words, it translates from code into spreadsheet. When you have to explain what you're doing from an engineering perspective to people whose primary IDE is Microsoft Powerpoint, consider Jellyfish. Thats Jellyfish.co and tell them Corey sent you! Watch for the wince, thats my favorite part. Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Today's promoted episode is a treasure and a delight. Longtime listeners of this show know that it's not really a database—unless of course, it's Route 53—and of course, I don't solve pronunciation problems with answers that make absolutely everyone hate me. Longtime listeners of the show know that if there's one thing I adore when it comes to databases—you know, other than Route 53—it is solving pronunciation holy wars in such a way that absolutely everyone is furious with me as a result, and today is no exception. My guest is Ed Boyajian, the CEO of EDB, a company that effectively is the driving force behind the Postgres-squeal database. Ed, thank you for joining me.Ed: Hey, Corey.Corey: So, I know that other people pronounce it ‘post-gree,' ‘Postgresql,' ‘Postgres-Q-L,' all kinds of other things. We know it's decidedly not ‘Postgres-squeal,' which is how I go for it. How do you pronounce it?Ed: We say ‘Postgres,' and this is one of the great branding challenges this fantastic open-source project has endured over many years.Corey: So, I want to start at the very beginning because when I say that you folks are the driving force behind Postgres—or Postgres-squeal—I mean it. I've encountered folks from EDB—formerly EnterpriseDB—in the wild in consulting engagements before, and it's great because whenever we found an intractable database problem, back at my hands-on keyboard engineering implementation days, very quickly after you folks got involved, it stopped being a problem, which is kind of the entire point. A lot of companies will get up there and say, “Oh, it's an open-source project,” with an asterisk next to it and 15 other things that follow from it, or, “Now, we're changing our license so the big companies can't compete with us.” Your company's not named after Postgres-squeal and you're also—when you say you have people working on it, we're not talking just one or two folks; your fingerprints are all over the codebase. How do you engage with an open-source project in that sense?Ed: First and foremost, Postgres itself is, as you know, an independent open-source project, a lot like Linux. And that means it's not controlled by a company. I think that's inherently one of Postgres's greatest strengths and assets. With that in mind, it means that a company like EDB—and this started when I came to the company; I came from Red Hat, so I've been in open-source for 20 years—when I came to the company back in 2008, it starts with a commitment and investment in bringing technology leaders in and around Postgres into a business like EDB, to help enterprises and customers. And that dynamic intersection between building the core database in the community and addressing customer needs in a business, at that intersection is where the magic happens. And we've been doing that since I joined EDB in 2008; it was really an explicit focus for the company.Corey: I'd like to explore a little bit, well first and foremost, this story of is there a future for running databases in cloud environments yourself? And I have my own angry, loud opinion on this that I'm sure we'll get to momentarily, but I want to start with yours. Who is writing their own databases in the Year of our Lord 2021, rather than just using whatever managed thing is their cloud provider of choice today is offering for them?Ed: Well, let me give you context, Corey, because I think it matters. We've been bringing enterprise Postgres solutions to companies now, since the inception of the company, which dates back to 2004, and over that trajectory, we've been helping companies as they've done really two things: migrate away, in particular from Oracle, and land on Postgres, and then write new apps. Probably the first ten of the last 13 years since I've been in the company, the focus was in traditional on-prem database transformations that companies were going through. In the last three years, we've really seen an acceleration of that intersection of their traditional deployments and their cloud deployments. Our customers now, who are represented mostly in the Fortune 500 and Global 2000, 40% of our customers report they're deploying EDB's Postgres in the cloud, not in a managed context, but in a traditional EC2 or GCP self-managed cloud deployment.Corey: And that aligns with what I've seen, a fair bit. Years ago, I wound up getting the AWS Cloud Practitioner Certification—did a whole blog post on it—not because it was opening any doors for me, but because it let me get into the certified lounge at re:Invent, and ideally charge a battery and have some mostly crappy coffee. The one question I got wrong was I was honest when I answered, “How long does it take to restore an RDS database from snapshot backup?” Rather than giving the by-the-book answer, which is way shorter than I found in practice a fair bit of the time. And that's the problem I always ran into is that when you're starting out and building something that needs a database, and it needs a relational database that runs in that model so all the no SQL options are not viable for whatever reason, great, RDS is great for getting you started, but there's only so much that you can tune and tweak before you start to run into issues were, for particular workloads as they scale-out, it's no longer a fit for a variety of reasons.And most of the large companies that I work with that are heavily relational-database-driven have either started off or migrated to the idea of, “Oh, we're going to run our own databases on top of EC2 instances,” for a variety of reasons that, again, the cloud providers will say, “Oh, that's not accurate, and they're doing the wrong thing.” But, you know, it takes a certain courage to tell a large-scale customer, “You're doing it wrong.” “Well, why is that?” “Because I have things to sell you,” is kind of a terrible answer. How do you see it? Let's not pick on RDS, necessarily, because all of the cloud providers offered managed database offerings. Where do those make sense and where do they fall down?Ed: Yeah, I think many of our customers who made their first step into cloud picked a single vendor to do it, and we often hear AWS is been that early, early—Corey: Yeah, a five-year head start makes a pretty compelling story.Ed: That's right. And let's remember what these vendors are mostly. They are mostly infrastructure companies, they build massive data centers and set those up, and they do that beautifully well. And they lean on software, but they're not software companies themselves. And I think the early implementation of many of our customers in cloud relied on what I'll call relatively lightweight software offerings from their cloud vendor, including database.They traded convenience, ease of use, an easy on-ramp, and they traded some capability in some depth for that. And it was a good trade, in fact. And for a large number of workloads it may still be a good trade. But our more sophisticated customers, enterprise customers who are running Postgres or databases at scale in their traditional environments have long depended on a very intimate relationship with their database technology vendor. And that relationship is the intersection of their evolving and emerging needs and the actual development of the database capabilities in support of that.And that's the heart of who we are at EDB and what we do with Postgres and the many people we have committed to doing that. And we don't see our customers changing that appetite. So, I think for those customers, they've emerged more aware of the need to have a primary relationship with a database vendor and still be in cloud. And so I think that's how this evolves to see two different kinds of services side-by-side, what they really want is a Database as a Service from the database vendor, which is what we just announced here at Microsoft Ignite event.Corey: So, talk to me a little bit more about that, where it's interesting in 2021 to see a company launching a managed service offering, especially in the database space, when there's been so much pushback in different ways against the large cloud providers—[cough] Amazon—who tend to effectively lose sleep at night over the haunting fear that someone who isn't them is making money, somehow. And they will take whatever is available to them and turn it into a managed service offering. That's always been the fear, so people play games with licenses and the rest. Well, they've been running Postgres offerings for a long time. It is an independent open-source project.I don't think you can wind up forcing a license change through that says everyone except big companies can run this themselves and don't do a managed service with it because that cat is very much out of the bag. How is it that you're taking something to market now and expecting that to fare competitively?Ed: So, I think there's a few things that our customers are clearly telling us they want, and I think this is the most important thing: they want control of their data. And if you step back, Corey, look at it historically, they made a huge trade to big proprietary database companies, companies like Oracle, and they made that trade actually for convenience. They traded data to that database vendor. And we all know the successes Oracle's had, and the sheer extraordinary expense of those technologies. So, it felt like a walled garden.And that's where EDB and Postgres entered to really change that equation. What's interesting is the re-platforming that happened and the transformation to cloud actually had the same, kind of, binding effect; we now moved all that data over to the public cloud vendors, arguably in an even stickier context, and now I think customers are realizing that's created a dimension of inflexibility. It's also created some—as you rightly pointed out—some deficiencies in technical depth, in database, and in software. So, our customers have sorted that out and are kind of coming back to middle. And what they're saying is, “Well, we want all the advantages of an open-source database like a Postgres, but we want control of the data.”And so what control looks like is more the ability to take one version of that software—in our case, we're worrying about Postgres—and deploy the same thing everywhere they go. And that opens the door up for EDB to be their partner as a traditional on-prem partner, in the cloud where they run our Postgres and they manage it themselves, and as their managed service, Postgres Database as a Service Provider, which is what we're doing.Corey: I've been something of a bear on the idea of, “I'm going to build a workload to run everywhere in every cloud provider,” which I get. I think that's generally foolish, and people chasing that, with remarkably few exceptions, are often going after the wrong thing. That said, I'm also a fan of having a path to strategic Exodus, where Google's Cloud Spanner is fascinating, DynamoDB is revelatory, Cosmos DB is a security nightmare, which is neither here nor there, but the idea that I can take a provider's offering that even if it solves a bunch of problems for me, well, if I ever need to move this somewhere else for any reason, I'm re-architecting, my data model and re-architecting the built-in assumptions around how the database acts and behaves, and that is a very heavy lift. We have proof of that from Amazon, who got up on stage and told a story about how much they hate Oracle, and they're migrating everything off of Oracle to Aurora, which they had to build in order to get off of Oracle, and it took them three years to migrate things. And Oracle loves telling that story, too.And it's, you realize you both sound terrible when you tell that story? It's, “This is a massive undertaking that even we struggle with, so you should probably not attempt it.” Well, what I hear from that is good God, don't wind up getting locked into a particular database that is only available from one source. So, if you're all-in on a cloud provider, which I'm a fan of, personally—I don't care which one but pick a cloud provider—having a database that is not only going to work in that environment is just a reasonable step as far as how I view things. Trading up that optionality has got to pay serious dividends, and in many database use cases, I've just don't see it.Ed: Yeah, I think you're bringing up a really important point. So, let's unpack it for a minute.Corey: Please.Ed: Because I think you brought up some really prominent specialty database technologies, and I'm not sure there's ever a way out of that intersection and commitment to a single vendor if you pick their specialty database. But underneath this is exactly one of the things that we've worried about here at EDB, which is to make Postgres a more capable, robust database in its entirety. A Postgres superpower is its ability to run a vast array of workloads. Guess what, it's not sexy. It's not sexy not to be that specialty database, but it's incredibly powerful in the hands of an enterprise who can do more.And that really creates an opportunity, so we're trying to make Postgres apply to a much broader set of workloads, from traditional systems of record, like your ERP systems; systems of analysis, where people are doing lightweight analytic workloads or reporting, you can think in the world of data warehouse; and then systems of engagement, where customers are interacting with a website and have a database on the backend. All areas Postgres has done incredibly well in and we have customer experience with. So, when you separate out that core capability and then you look at it on a broader scale like Postgres, you realize that customers who want to make Postgres strategic, by definition need to be able to deploy it wherever they want to deploy it, and not be gated or bound by one cloud vendor. And all the cloud vendors picked up Postgres offerings, and that's been great for Postgres and great for enterprises. But that corresponding lock-in is what people want to get away from, at this point.Corey: There's something to be said for acknowledging that there is a form of lock-in as far as technology selection goes. If you have a team of folks who are terrific at one database engine and suddenly you're switching over to an entirely different database, well, folks who spent their entire career working on one particular database that's still in widespread use are probably not super thrilled to stick around for that. Having something that can migrate from environment to environment is valuable and important. When you say you're launching this as a database as a service offering, how does that actually work? Is that going to be running in your own cloud environment somewhere and people just make queries across the wire through standard connections to the database like they would something locally? Are you running inside of their account or environment? Is it something else?Ed: So, this is a fully-managed database as a service, just like you'd get from any cloud vendor or DBAAS vendor that you've worked with in the past, just being managed and run by EDB. And with that, you get lot of the goodies that we bring, including our compatibility, and all our deep Postgres expertise, but I think one of the other important attributes is we're going to run that service in our clients' account, which gives them a level of isolation and a level of independence that we think is really important. And as different as that is, it's not heroic; it's exactly what our customers told us they wanted.Corey: There's something to be said for building the thing that your customers have said that they want and make sense for you to build as opposed to, “We're going to build this ridiculous thing and we're sure folks are going to love it.” It's nice to see that shaping up in the proper order. And I've fallen victim to that myself; I think most technologists have to some extent. How big is EDB these days?Ed: So, we have over 650 employees. Now, around the world, we have 6000 customers. And of the 650 employees, about 300 of those are focused on Postgres. A subset of that are 30-odd core team members in the Postgres community, committers in the Postgres community, major contributors, and contributors in the Postgres community. So, we have a density of technical depth that is really unparalleled in Postgres.Corey: You're not, for lack of a better term, pulling an Amazon, insofar as you're, “Well, we have three people working on open-source projects, so we're going to go ahead and claim we're an open-source company,” in other words. Conversely, you're also not going down the path of this is a project that you folks have launched, and it claims to be open-source because we love it when people volunteer for for-profit entities, but we exercise total control over the project. You have a lot of contributors, but you're also still a minority, I think the largest minority, but still a minority of people contributing to Postgres.Ed: That's right. And, look, we're all-in on Postgres, and it's been that way since I got here. As I mentioned earlier, I came from Red Hat where I was—I was at Red Hat for a little over six years, so I've been an open-source now for 20 years. So, my orientation is towards really powerful, independent open-source projects. And I think we'll see Postgres really be the most transformative open-source technology since Linux.I think we'll see that as we look forward. And you're right, though, I think what's powerful about Postgres is it's an independent project, which means it's supported by thousands of contributors who aren't tied to single companies, around the world. And it just makes the software—we develop innovation faster, and I think it makes the software better. Now, EDB plays a big part in there. Roughly, a little less than a third of the last res—actually, the 13 release—were contributions that came from contributors who came from EDB.So, that's not a majority, and that's healthy. But it's a big part of what helps move Postgres along and there aren't—you know, the next set of companies are much, much—next set of combined contributors add up to quite small numbers. But the cloud vendors are virtually non-existent in that contribution.Corey: This episode is sponsored in part by something new. Cloud Academy is a training platform built on two primary goals. Having the highest quality content in tech and cloud skills, and building a good community the is rich and full of IT and engineering professionals. You wouldn't think those things go together, but sometimes they do. Its both useful for individuals and large enterprises, but here's what makes it new. I don't use that term lightly. Cloud Academy invites you to showcase just how good your AWS skills are. For the next four weeks you'll have a chance to prove yourself. Compete in four unique lab challenges, where they'll be awarding more than $2000 in cash and prizes. I'm not kidding, first place is a thousand bucks. Pre-register for the first challenge now, one that I picked out myself on Amazon SNS image resizing, by visiting cloudacademy.com/corey. C-O-R-E-Y. That's cloudacademy.com/corey. We're gonna have some fun with this one!Corey: Something else that does strike me as, I guess, strange, just because I've seen so many companies try to navigate this in different ways with varying levels of success. I always encountered EDB—even back when it was EnterpriseDB, which was, given their love of acronyms, I'm still somewhat partial to. I get it; branding, it's a thing—but the folks that I engaged with were always there in a consulting service's capacity, and they were great at this. Is EDB a services company or a product company?Ed: Yeah, we are unashamedly a product technology company. Our business is over 90% of our revenue is annually recurring subscription revenue that comes from technical products, database server, mostly, but then various adjacent capabilities in replication and other areas that we add around the database server itself. So no, we're a database technology company selling a subscription. Now, we help our customers, so we do have a really talented team of consultants who help our customers with their business strategy for Postgres, but also with migrations and all the things they need to do to get Postgres up and running.Corey: And the screaming, “Help, help, help, fix it, fix it, fix it now,” emergencies as well.Ed: I think we have the best Postgres support operation in the world. It is a global 24/7 organization, and I think a lot of what you likely experienced, Corey, came out of our support organization. So, our support guys, these guys aren't just handling lightweight issues. I mean, they wade into the gnarly questions and challenges that customers face. But that's a support business for us. So, that's part and parcel. You get that, it's included with the subscription.Corey: I would not be remembering this for 11 years later, if it hadn't been an absolutely stellar experience—or a horrible experience, for that matter; one or the other. You remember the superlatives, not the middle of the road ones—and if it hadn't been important. And it was. It also noteworthy; with many vendors that are product-focused, their services may have an asterisk next to it because it's either a, “Buy our product and then we'll support it,” or it's, “Ohh, we're going to sell you a whole thing just to get us on the phone.” And as I recall, there wasn't a single aspect of upsell involved in this.It was, “Let's get you back up and running and solve the problem.” Sure, later in time, there were other conversations, as all good businesses will have, but there was no point during those crisis moments where it felt like, “Oh, if you had gone ahead and bought this thing that we sell, this wouldn't happen,” or, “You need to buy this or we won't help you.” I guess that's why I've contextualized you folks as a services company, first and foremost.Ed: Well, I'm glad you have that [laugh] experience because that's our goal. And I think—look, this is an interesting point where customers want us to bring that capability to their managed DBAAS world. Step back again, go back to what I said about the big cloud vendors; they are, at their core, infrastructure companies. I mean, they're really good at that. They're not particularly well-positioned to take your Postgres call, and I don't think they want that call.We're the other guys; we want to help you run your Postgres, at scale, on-prem, in the cloud, fully managed in the cloud, by EDB, and solve those problems at the same time. And I think that's missing in the market today. And we can step back and look at this overall cloud evolution, and I think some might think, “Gee, we're into the mature phase of cloud adoption.” I would tell you, since the Red Sox have done well this year, I think in a nine-inning baseball game—for those of your listeners who follow American baseball—we're in, like, the top of the second inning, maybe. Maybe the bottom of the second inning. So, we've been able to listen and learn from the experiences our customers have had. I think that's an incredible advantage as we now firmly plant ourselves in the cloud DBAAS market alongside our robust Postgres capabilities that you experienced.Corey: The world isn't generating less data, and it's important that we're able to access that in a bunch of different ways. And the last time I really was playing with relational databases, you can view my understanding of it as Excel with a weirder interface, and you're mostly there. One thing that really struck me since the last time I went deep into database-land over in the Postgres-squeal world has been just the sheer variety of native data types that it winds up supporting. The idea of, “Here's some JSON. Take this and store it that way,” or it's GIS data that it can represent, or the idea of having data types that are beyond just string or var or whatever other somewhat limited boolean values or whatnot. Without having just that traditional list, which is of course all there as well. It also seems to have extensively improved its coverage that just can only hint to my small mind about these things and what sort of use cases people are really putting these things into.Ed: Yeah, I think this is one of Postgres' superpowers. And it started with Mike Stonebraker's original development of Postgres as an object-relational database. Mike is an adviser to EDB, which has been incredibly helpful as we've continued to evolve our thinking about what's possible in Postgres. But I think because of that core technology, or that core—because of that core technical capability within Postgres, we have been able to build a whole host of data types. And so now you see Postgres being used not just as the context of a traditional relational database, but we see it used as a time-series database. You pointed out a geospatial database, more and more is a document-oriented database with JSON and JSONB.These are all the things that make Postgres have much more universal appeal, universal appeal to developers—which is worth talking about in the recent StackOverflow developer survey, but we can come back to that—and I think universal applicability for new applications. This is what's bringing Postgres forward faster, unlike many of the specialty database companies that you mentioned earlier.Corey: Now, this is something that you can use for your traditional CRUD app, the my first hello world app that returns something from a database, yeah, that stuff works. But it also, for example, has [cyter 00:25:09] data types, where you can say, give me the results where the IP range contains this address, and it'll do that. Before that, you're trying to solve a whole bunch of very messy things in application logic that's generally awful. The database now does that for you automatically, and there's something—well, it would if I were smart and used it instead of storing it as strings because I make terrible life choices, but for sensible people, it solves a lot of those problems super well. And it's taken the idea of where logic should live in application versus database, and sort of turn a lot of those assumptions I was starting my career with on their head.Ed: Yeah, I think if you look now at the appeal of Postgres to developers, which we've paid a lot of attention to—one of our stated strategies at EDB is to make Postgres easier. That's been true for many years, so a drive for engineering and development here has been that call to action. And if you measure that, over time, we've been contributing—not alone, but contributing to making Postgres more approachable, easier to use, easier to engage with. Some of those things we do just through edb.com, and the way we handle EDB docs is a great example of that, and our developer advocacy and outreach into adjacent communities that care about Postgres. But here's where that's landed us. If you looked at the last Stack Overflow developer survey—the 2021 Stack Overflow developer survey, which I love because I think it's very independent-oriented—and they surveyed, I think this past year was 80,000 developers.Corey: Oh yeah, if Stack Overflow is captured by any particular constituency, it's got to be ‘Big Copy and Paste' that is really behind them. But yeah, other than the cabal of keyboard manufacturers for those copy-and-paste stories, yeah, they're fairly objective when it comes to stuff like this.Ed: And if you look at that survey, Corey, if you just took and summed it because it's helpful to sum it, most used, most loved, and most wanted database: Postgres wins. And I find it fascinating that if you—having been here, in this company for 13 years and watch the evolution from—you know, 13 years ago, Postgres needed help, both in terms of its awareness in the market and some technical capabilities it just lacked, we've come so far. For that to be the new standard for developers, I think, is a remarkable achievement. And I think it's a representation of why Postgres is doing so well in the market that we've long served, in the cloud market that we are now serving, and I think it speaks to what's ahead as a transformational database for the future.Corey: There really is something to be said for a technology as—please don't take this term the wrong way—old. As a relational database, Postgres has been around for a very long time, but it's also not your grandparents' Postgres. It is continuing to evolve. It continues to be there in a bunch of really interesting ways for developers in a variety of different capacities, and it's not the sort of thing that you're only using in, “Legacy environments,” quote-unquote. Instead, it's something that you'll see all over the place. It is rare that I see an environment that doesn't have Postgres in it somewhere these days.Ed: Yeah, I think quite the contrary to the old-school database, which I love that; I love that shade because when you step away from it, you realize, the Postgres community represents the very best of what's possible with open-source. And that's why Postgres continues to accelerate and move forward at the rate that it does. And obviously, we're proud to be a contributor to that, so we don't just watch that outcome happen; we're actually part of creating it. But I also think that when you see all that Postgres has become and where it's going, you really start to understand why the market is adopting open-source.Corey: It's one of those areas where even if some company comes out with something that is amazing and transformatively better, and you should jump into it with both feet and never look back, yeah, it turns out that it takes a long time to move databases, even when they're terrible. And you can lobby an awful lot of accusations at Postgres—or Postgres-squeal—but you can't call it terrible. It's used in enough interesting applications by enough large-scale companies out there—and small as well—that it's very hard to find a reason not to explore it. It's my default relational database when Route 53 loses steam. It just makes sense in a bunch of ways that other things really didn't for me before.Ed: Yeah, and I think we'll continue to see that. And we're just going to keep making Postgres better. And it gets better because of that intersection, as I mentioned, that intimate intersection between enterprise users, and the project, and the community, and the bridge that a company like EDB provides for that. That's why it'll get better faster; the breadth of use of Postgres will keep it accelerating. And I think it's different than many of the specialty databases.Look, I've been in open-source now for 20 years and it's intriguing to me how many new specialty open-source databases have come to market. We tend to forget the amount of roadkill we've had over the course of the past ten years of some of those open-source projects and companies. We certainly are tuned into some of the more prolific ones, even today. And I think again, here again, this is where Postgres shines, and where I think Postgres is a better call for a long-term. Just like Linux was.Corey: I want to thank you for taking so much time out of your day to talk to me about databases, which given my proclivities, is probably like pulling teeth for you. If people want to learn more, where can they find you?Ed: So, come to enterprisedb.com. You still get EnterpriseDB, Corey. Just come to enterprise—Corey: There we go. It's hidden in the URL, right in plain sight.Ed: Come to enterprisedb.com. You can learn all the things you need about the technology, and certainly more that we can do to help you.Corey: And we will, of course, put links to that in the [show notes 00:31:10]. Thank you once again for your time. I really do appreciate it.Ed: Thanks, Corey. My pleasure.Corey: Ed Boyajian, CEO of EDB. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with a long angry comment because you are one of the two Amazonian developers working on open-source databases.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.
In this episode of Scaling Postgres, we discuss when to use tablespaces, setting up streaming replication, features coming in Postgres 14 and implementing security. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/when-to-use-tablespaces-in-postgresql/ https://www.migops.com/blog/2021/03/31/setting-up-streaming-replication-in-postgresql-13-and-streaming-replication-internals/ https://www.depesz.com/2021/04/01/waiting-for-postgresql-14-add-pg_database_owner-default-role/ https://www.depesz.com/2021/03/31/waiting-for-postgresql-14-add-date_bin-function/ https://www.depesz.com/2021/04/01/waiting-for-postgresql-14-add-unistr-function/ https://blog.crunchydata.com/blog/is-postgres-secure https://www.youtube.com/user/EnterpriseDB/videos https://fluca1978.github.io/2021/04/02/OOMKillerFreeBSD.html https://b-peng.blogspot.com/2021/03/logging-pgpool-on-k8s.html http://blog.cleverelephant.ca/2021/04/psql-binary.html https://postgresql.life/post/jan_karremans/
Jaclyn Jussif, Director of Talent Acquisition at EnterpriseDB, joined us on The Modern People Leader. We talked about hiring, onboarding, and how to create custom ramp plans for new hires. Timestamps: biggest differences hiring internationally (12:29), using the WHO method for behavioral interviewing (16:22), calling out the elephant in the room (COVID) in interviews (22:04), front-loading information in the onboarding process (25:52), creating customized 30, 60, 90-day ramp plans (29:33), tying references to ramp plans (33:08), nudging people to connect (36:00), why the EMPLOYEE needs to drive the onboarding process (38:45), and the magnified imposter syndrome that people are feeling right now (42:56). If you have any feedback for the MPL podcast or would like to connect, don't hesitate to reach out on LinkedIn: Daniel Huerta and Stephen Huerta
In this episode of Scaling Postgres, we discuss a vacuum speed up and faster foreign tables in Postgres 14, running faster queries with union and learning about the query optimizer. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.citusdata.com/blog/2021/03/25/speeding-up-recovery-and-vacuum-in-postgres-14/ http://postgres-road.blogspot.com/2021/03/faster-bulk-insertion-to-foreign-tables.html https://www.foxhound.systems/blog/sql-performance-with-union/ https://www.cybertec-postgresql.com/en/how-the-postgresql-query-optimizer-works/ https://www.highgo.ca/2021/03/20/how-to-check-and-resolve-bloat-in-postgresql/ https://www.depesz.com/2021/03/22/waiting-for-postgresql-14-allow-configurable-lz4-toast-compression/ https://www.citusdata.com/blog/2021/03/20/sharding-postgres-on-a-single-citus-node/ https://www.youtube.com/user/EnterpriseDB/videos https://blog.crunchydata.com/blog/get-started-with-explain-analyze https://www.enterprisedb.com/blog/ansible-collection-postgresql-and-edb-components https://mydbanotebook.org/post/no_space_left_on_device/ https://blog.crunchydata.com/blog/performance-improvements-in-geos https://www.enterprisedb.com/blog/how-get-status-cloud-native-postgresql-cluster https://postgresql.life/post/julien_riou/
In this episode of Scaling Postgres, we discuss Citus 10 open source, time series performance in native Postgres, using subscripting for updates and new target_session_attrs. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.citusdata.com/blog/2021/03/05/citus-10-release-open-source-rebalancer-and-columnar-for-postgres/ https://aws.amazon.com/blogs/database/designing-high-performance-time-series-data-tables-on-amazon-rds-for-postgresql/ https://erthalion.info/2021/03/03/subscripting/ https://www.cybertec-postgresql.com/en/new-target_session_attrs-settings-for-high-availability-and-scaling-in-postgresql-v14/ https://www.depesz.com/2021/03/01/starting-with-pg-where-how-can-i-set-configuration-parameters/ https://www.depesz.com/2021/03/05/starting-with-pg-where-are-logs/ https://www.youtube.com/user/EnterpriseDB/videos https://www.enterprisedb.com/blog/security-and-containers-cloud-native-postgresql https://www.cybertec-postgresql.com/en/postgresql-getting-started-on-ubuntu/ https://www.highgo.ca/2021/03/03/how-postgresql-handles-sub-transaction-visibility-in-streaming-replication/ https://www.highgo.ca/2021/03/04/how-to-create-a-system-information-function-in-postgresql/ https://b-peng.blogspot.com/2021/02/clustering-modes-in-pgpool.html https://www.enterprisedb.com/blog/regression-analysis-postgresql-tensorflow-part-1-getting-started https://postgresql.life/post/david_e_wheeler/
In this episode of Scaling Postgres, we discuss cleaning up your database, function performance, 11 million IOPS and change data capture. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://hakibenita.com/postgresql-unused-index-size https://info.crunchydata.com/blog/cleaning-up-your-postgres-database https://www.ongres.com/blog/boost-your-user-defined-functions-in-postgresql/ https://tanelpoder.com/posts/11m-iops-with-10-ssds-on-amd-threadripper-pro-workstation/ https://info.crunchydata.com/blog/postgres-change-data-capture-with-debezium https://www.cybertec-postgresql.com/en/postgresql-what-is-a-checkpoint/ https://www.youtube.com/user/EnterpriseDB/videos https://blog.rustprooflabs.com/2021/02/postgresql-sqlite-fdw-pihole https://www.percona.com/blog/2021/02/01/postgresql-database-security-authentication/ https://info.crunchydata.com/blog/postgres-and-crypto-supply-chain-of-integrity https://www.depesz.com/2021/02/04/waiting-for-postgresql-14-search-and-cycle-clauses/ https://www.highgo.ca/2021/02/03/how-postgresql-inserts-a-new-record-with-the-help-of-table-access-method-api-and-buffer-manager/ https://info.crunchydata.com/blog/gitops-postgres-kubernetes https://b-peng.blogspot.com/2021/01/statistics-in-pgpool.html http://lin-ear-th-inking.blogspot.com/2021/02/overlayng-and-invalid-geometry.html https://www.cybertec-postgresql.com/en/pg_timetable-asynchronous-chain-execution/ https://postgresql.life/post/alexander_sosna/
Welcome to another episode of Develomentor. Today's guest is Atri Sharma!As a database engineer and open source committer, Atri Sharma has built a career in tech deeply focused some of today’s most popular databases. Some of these include PostgreSQL, Lucene, Elasticsearch, Presto and HAWQ. After getting his bachelor’s from Jaypee Institute of Information Technology, Atri Sharma has held roles at Amazon, EnterpriseDB, Teradata, Barclays, Pivotal, Microsoft and Intuit. Along the way, he’s become a prolific contributor to several open source projects like Lucene, PostgreSQL and a handful of other Apache Software Foundation Projects. He’s also the author of the upcoming book “Practical Lucene 8”. Stay tuned as we take a deep dive with Atri Sharma into the world of databases and open source!-Grant IngersollIf you are enjoying our content please leave us a rating and review or consider supporting usQuotes“Google Summer of Code has been around for a while now. It is designed in a way to introduce college students and academics to the world of open source development.”“One thing that open source really gives you is the people aspect. You build a network of smart and very accomplished people who can actually mentor and help you grow on the right path.”“When I actually decided to commit myself to open source I always thought it would be in my spare time. I never actually anticipated having a job where I would be expected to contribute features back to open source projects.”—Atri SharmaKey MilestonesWhat is Google Summer of CodeHow has Atri continually found jobs in open sourceAtri has spent his career working on some of the hardest challenges in the database world like adding features, making them faster. What’s it like working deep inside some of the world’s most heavily used data stores?What are key skills necessary to work as a database engineer?Atri has worked for many of the world’s largest tech companies. Why did he choose to work in big tech?Atri’s book ‘Practical Lucene 8’Additional ResourcesOrder Atri’s book ‘Practical Apache Lucene 8’ – https://www.amazon.com/Practical-Apache-Lucene-Capabilities-Application/dp/1484263448Mike McCandless is one of Atri’s mentors; read his blog here – http://blog.mikemccandless.com/Connect with Atri SharmaLinkedInGitHubFollow DevelomentorTwitter: @develomentorConnect with Grant IngersollLinkedInTwitter
The database that's making waves in enterprise settings is PostgreSQL (often called Postgres), which would be romping up the database popularity index, if such a thing existed. Why is that the case?An open-source system that runs on Alibaba Cloud, AWS, Azure and ARM alike, you can download it, run it on your virtual or real tech, from 60-core x86s to Raspberry Pi, and it'll happily mince your data, just how you want it!But what happens when your business relies on Postgres, or you need a helping hand? Or an extra feature you can't/won't develop yourself? That's where EnterpriseDB comes in. We speak to Marc Linster, Senior Vice President of Product Development at the company, about paying for "open-souce-PLUS", upstreaming, development communities and the unique capabilities of PostgreSQL.The EnterpriseDB homepage:https://www.enterprisedb.com/The elephant-centric PostgreSQL pages are here:https://www.postgresql.org/Connect with Joe on LinkedIn:https://www.linkedin.com/in/josephedwardgreen/
TheNextCMO's latest podcast is with Rachael Champagne and Frank Days. Rachael is the Marketing Campaign Manager and Frank is the Senior VP of Marketing at EnterpriseDB. Rachael has been with EnterpriseDB for 2 years working in ABM, demand gen, and events. Frank joined the company leading product marketing and now heads the entire marketing department. Rachael and Frank joined us to talk about transitioning their annual physical user conference Postgresvision to a virtual event due to the changing market conditions. In this episode we cover the massive undertaking of shifting a physical event to virtual, the pros/cons of a digital event, advice to marketers who are going through these changes, and some potential roadblocks you may come across and how to navigate through them. Frank and Rachael leveraged Intrado's virtual event solution and have been very happy with the tools and support leading up to the event. EnterpriseDB: https://www.enterprisedb.com/ Postgresvision 2020 - https://www.postgresvision.com/ Intrado Virtual Events: https://www.inxpo.com/virtual-events/ Rachael Champagne - https://www.linkedin.com/in/rachaelchampagne/ Frank Days - https://www.linkedin.com/in/tangyslice/ For more info about Plannuh, check out our website
This episode has 2 subjects: PostgreSQL migration and performance. The two guests of our show will present during the PostgreSQL User group meetup at bol.com which started shortly after this recording. The topic of the meetup is introduced as PG ConfEU - The Dutch talks..GuestsMartijn Dashorst – a software engineer for over 20 years and has worked with various databases over the years, including DB2, Oracle, Sqlserver, SABDB and PostgreSQL. As a Java (enterprise) developer, he has worked in the education sector for over 12 years and developed student information systems for each of the Dutch levels of education. During those years he has worked on the open-source Java web framework Apache Wicket and given numerous presentations on that and other subjects on international conferences and meetups. Senior Software Engineer at Topicus.Sebastiaan Mannem – Technically driven database, container and cloud specialist. Earned his stripes and was battle-hardened by the Dutch Government and Bol.com. Now working for EnterpriseDB as a senior consultant at Enterprise DB.NotesPostgreSQL Conference EuropeHVR - tool used during migration from Oracle to Postgres as safeguardRust - programming language used by Sebastiaan to loadtest the Postgres database
Ed Boyajian, CEO joined EnterpriseDB and helped it pivot from a small organization, to one of the leading Postgres database companies. The company has figured out how to run a profitable business, while embracing and respecting the community and open development process that has formed around Postres for more then two decades.
Today we're talking with Nitzan Shapira, co-founder and CEO of Epsagon, about how and why to use managed services from a cloud provider such as AWS. Epsagon is a sponsor of The New Stack that provides automated observability and cost management for serverless applications using distributed tracing and AI technologies. He wrote a post for us this week on the evolution of managed services such as AWS Fargate and Amazon EKS, the tradeoffs and benefits of using them and his tips for how to choose a managed service. Then later in the show, we discuss the latest news from EnterpriseDB's Postgres Vision, JFrog's swampUp, and QCON New York conferences, as well as our top podcast pick from the week, an interview with Gitlab's Ashish Kuthiala about the changing role of CI/CD.
The buyer is changing, and the sales channels we go to market with must do the same. Buyers are more comfortable than ever purchasing virtually. Inside Sales organizations are handling larger deals than ever before. Today Mike Huseman is here to share his experience leveraging Inside Sales to virtually drive profitable growth.
PostgreSQL is the #1 enterprise-class open source database with a feature set comparable to the major proprietary RDBMS vendors and a customer list that spans every industry. Hosts: Randal Schwartz and Dan Lynch Guest: Ed Boyajian Download or subscribe to this show at https://twit.tv/shows/floss-weekly. We invite you to read, add to, and amend our show notes. Here's what's coming up for FLOSS in the future. Think your open source project should be on FLOSS Weekly? Email Randal at merlyn@stonehenge.com. Thanks to CacheFly for providing the bandwidth for this podcast, and Lullabot's Jeff Robbins, web designer and musician, for our theme music.
PostgreSQL is the #1 enterprise-class open source database with a feature set comparable to the major proprietary RDBMS vendors and a customer list that spans every industry. Hosts: Randal Schwartz and Dan Lynch Guest: Ed Boyajian Download or subscribe to this show at https://twit.tv/shows/floss-weekly. We invite you to read, add to, and amend our show notes. Here's what's coming up for FLOSS in the future. Think your open source project should be on FLOSS Weekly? Email Randal at merlyn@stonehenge.com. Thanks to CacheFly for providing the bandwidth for this podcast, and Lullabot's Jeff Robbins, web designer and musician, for our theme music.
As PostgreSQL celebrates 15 years, EnterpriseDB's Ed Boyajian and Robin Schumacher talk about this popular open source database, its engaged community, how it differs MySQL, and where it fits in the world of big data.
Software Engineering Radio - The Podcast for Professional Software Developers
In this episode we're talking to Dirk Riehle about open source business models. We started looking at the way OS projects work and defined different kinds of open source projects. In the main part of the discussion we looked at various ways of how to make money with open source: consulting, support contracts, commercial variant of an open source project, etc. We then looked at the chances and risks of each of these approaches. The next part focused on different open source licenses and how they are suitable for open source business. We concluded the episode by discussing a couple of specific questions and loose ends. After the show, Dirk informed me about the following three corrections: Black Duck Software's main product is called protexIP not IP Central, there are presently 70 licenses approved by the Open Source Initiative, and EnterpriseDB has so far acquired $37M in venture capital
Software Engineering Radio - The Podcast for Professional Software Developers
In this episode we're talking to Dirk Riehle about open source business models. We started looking at the way OS projects work and defined different kinds of open source projects. In the main part of the discussion we looked at various ways of how to make money with open source: consulting, support contracts, commercial variant of an open source project, etc. We then looked at the chances and risks of each of these approaches. The next part focused on different open source licenses and how they are suitable for open source business. We concluded the episode by discussing a couple of specific questions and loose ends. After the show, Dirk informed me about the following three corrections: Black Duck Software's main product is called protexIP not IP Central, there are presently 70 licenses approved by the Open Source Initiative, and EnterpriseDB has so far acquired $37M in venture capital
Software Engineering Radio - The Podcast for Professional Software Developers
In this episode we're talking to Dirk Riehle about open source business models. We started looking at the way OS projects work and defined different kinds of open source projects. In the main part of the discussion we looked at various ways of how to make money with open source: consulting, support contracts, commercial variant of an open source project, etc. We then looked at the chances and risks of each of these approaches. The next part focused on different open source licenses and how they are suitable for open source business. We concluded the episode by discussing a couple of specific questions and loose ends. After the show, Dirk informed me about the following three corrections: Black Duck Software's main product is called protexIP not IP Central, there are presently 70 licenses approved by the Open Source Initiative, and EnterpriseDB has so far acquired $37M in venture capital