Podcasts about Amazon Aurora

AWS Morning Brief for the week of April 7th, with Corey Quinn. Links:Amazon EC2 now supports more bandwidth and jumbo frames to select destinationsAPI Gateway launches support for dual-stack (IPv4 and IPv6) endpointsAWS Lambda adds support for Ruby 3.4Amazon CloudWatch Logs increases maximum log event size to 1 MBAmazon Neptune announces 99.99% availability Service Level AgreementAnnouncing the general availability of Amazon VPC Route ServerUnder the hood: Amazon EKS Auto ModeOptimizing cost savings: The advantage of Amazon Aurora over self-managed open source databasesHow AWS Sales uses generative AI to streamline account planningIssue with AWS SAM CLI (CVE-2025-3047, CVE-2025-3048)

Quali sono le novità relative ad Amazon Aurora presentate al Re:Invent 2024? Cos'è Aurora Limitless? E Aurora DSQL? Oggi, insieme a Domenico Di Salvia (Senior Specialist Solutions Architect) parliamo di tutte le versioni di Aurora disponibili, le loro differenze e gli use case a cui si adattano meglio.Link utili:- Amazon Aurora- Amazon Aurora Serverless- Amazon Aurora Limitless- Amazon Aurora DSQL

Learn when Amazon Aurora DSQL is the right choice for your applications, what changes you'll need to make during migration, and how to handle its unique approach to optimistic locking.

In this episode, we provide an overview of Amazon Aurora, a relational database solution on AWS. We discuss its unique capabilities like distinct storage architecture for better performance and faster recovery. We cover concepts like Aurora clusters, reader and writer instances, endpoints, and global databases. We also compare the serverless versions V1 and V2, noting that V2 is more enterprise-ready while V1 scales to zero. We touch on billing and additional features like the data API, RDS query editor, and RDS proxy. Overall, Aurora is powerful and scalable but not trivial to use at global scale. It's best for serious enterprise use cases or variable traffic workloads.

Covering when to use Aurora vs. RDS, common challenges when optimizing Aurora after migrating from RDS, I/O optimization and when I/O-optimized makes sense, and how to architect your Aurora databases for cost optimization.

The SM7B is everywhere. Adam and Dax can't wait for their AI coworkers to arrive, but should you be concerned about your job with AI? And what does E/ACC mean? Adam asks Dax about his tweets and rules for Twitter. And the US healthcare system is broken with no fix in sight.Want to carry on the conversation? Join us in Discord.SM7B - Vocal Microphone - Shure USAPerplexityDax on X: “you've all had co2 monitors for over a month now and i haven't seen an improvement”Query Insights — PlanetScale DocumentationIntroducing the Data API for Amazon Aurora Serverless v2 and Amazon Aurora provisioned clusters | AWS Database BlogAI description of this episode.(00:00) - Do it at the beginning (00:29) - Mics and mic technique (05:32) - AI developer coworkers (18:09) - What does the best programming language look like in the future? (24:39) - Are we too optimistic about AI's future? (29:28) - AI is great for cleaning up data (33:17) - You don't have to be human with AI (38:16) - What does E/ACC mean? (40:39) - The White House says we should write Rust (42:07) - Adam asks Dax about his tweets (44:00) - Dax's Twitter avatar rule (46:24) - What's PlanetScale Insights? (53:18) - The state of health care in America (01:02:13) - Interest rates going down - or up? (01:03:39) - Meta ask: leave us a rating please! (01:08:03) - Would you get Neuralink?

Este episodio explica en detalle como hacer migraciones de SQL Server a Amazon Aurora utilizando diversas herramientas nativas de AWS. Material Adicional: https://aws.amazon.com/blogs/database/automate-the-migration-of-microsoft-ssis-packages-to-aws-glue-with-aws-sct

For time-strapped data teams looking to simplify analytics on large transactional data, Amazon Aurora's seamless integration with Amazon Redshift is a game-changer. Tune in to listen to chat with, Jyoti Aggarwal (Product Management lead for Amazon Redshift zero-ETL), Adam Levin (Principal Product Manager on the Amazon Aurora) for a new service called Amazon Aurora with Amazon Redshift Integration. Amazon Aurora with Amazon Redshift Integration is a capability that enables organizations to run analytics directly against their Amazon Aurora transactional data, without needing to manage complex ETL pipelines.

Thomas LaRock, Principal Developer Evangelist at Selector AI, joins Corey on Screaming in the Cloud to discuss why he loves having a career in data and his most recent undertaking at Selector AI. Thomas explains how his new role aligned perfectly with his career goals in his recent job search, and why Selector AI is not in competition with other data analysis tools. Corey and Thomas discuss the benefits and drawbacks to going back to school for additional degrees, and why it's important to maintain a healthy balance of education and practical experience. Thomas also highlights the impact that data can have on peoples' lives, and why he finds his career in data so meaningful. About ThomasThomas' career and life experiences are best described as follows: he takes things that are hard and makes them simple for others to understand. Thomas is a highly experienced data professional with over 25 years of expertise in diverse roles, from individual contributor to team lead. He is passionate about simplifying complex challenges for others and leading with empathy, challenging assumptions, and embracing a systems-thinking approach. Thomas has strong analytical reasoning skills and expertise to identify trends and opportunities for significant impact, and is a builder of cohesive teams by breaking down silos resulting in increased efficiencies and collective success. He has a track record of driving revenue growth, spearheading industry-leading events, and fostering valuable relationships with major tech players like Microsoft and VMware. Links Referenced: Selector: https://www.selector.ai/ LinkedIn: https://www.linkedin.com/in/sqlrockstar/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Do you wish there were cheat codes for database optimization? Well, there are – no seriously. If you're using Postgres or MySQL on Amazon Aurora or RDS, OtterTune uses AI to automatically optimize your knobs and indexes and queries and other bits and bobs in databases. OtterTune applies optimal settings and recommendations in the background or surfaces them to you and allows you to do it. The best part is that there's no cost to try it. Get a free, thirty-day trial to take it for a test drive. Go to ottertune dot com to learn more. That's O-T-T-E-R-T-U-N-E dot com.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. There are some guests I have been nagging-slash-angling to have on this show for years on end, and that you almost give up, until they wind up having a job change. At which point, there's no better opportunity to pounce like some sort of scavenger or hyena or whatnot in order to get them on before their new employer understands what I am, and out of an overabundance of caution, decides not to talk with me. Thomas LaRock is a recently minted Principal Developer Evangelist at Selector. Thomas, thank you for finally deigning to appear on the show. It is deeply appreciated.Thomas: Oh, thanks for having me. Thanks for extending invitation. I'm sorry. It's my fault I haven't come here before now; it's just been one of those scheduling things. And I always think I'm going to see you. Like, I'll go to re:Invent, and I'm like, “I'll see Corey there.” And then, nah, Corey is a little busy.Corey: Yeah, I have no recollection of basically anything that ever happens at re:Invent, just because it is eight days of ridiculous Cloud Chanukah and thing to thing to thing to thing to thing. It's just overload and I wind up effectively blocking all of it out. You are one of those very interesting people where, depending upon the context in which someone encounters you, it's difficult to actually put a finger on where you start and where you stop. You are, for example, a Microsoft MVP, which means you presumably have a fair depth of experience with at least some subset of Microsoft products. You have been working at SolarWinds for a while now, and you also have the username of SQLRockstar on a number of social media environments, which leads me to think, oh, you're a database person. What are you exactly? Where do you start? Where do you stop?Thomas: Yeah, in my heart-of-hearts, a data professional. And that can mean a lot of things to a lot of different people. My latest thing I've taken from a friend where I just call myself a data janitor because that's pretty much what I do all day, right? I'll clean data up, I'll move it around, it's a pile here and a pile there. But that's my heart of hearts. I've been a database administrator, I've been the data advocate. I've done a lot of roles, but it's always been heavily focused on data.Corey: So, these days, your new role—let's start at the present and see if we work our way backwards or not—you've been, at the time of this recording, in your role for a week where you are a principal developer evangelist at Selector, which to my understanding, is an AIOps or MLOps or whatever buzzword that we're sprinkling on top of things today is, which of course presupposes having some amount of data to wind up operating on. What do you folks do over there?Thomas: That's a great question. I'm hoping to figure that out eventually. No. So, here's the thing, Corey. So, when I started my unforced sabbatical this past June, I was, of course, doing what everybody does: panicking. And I was looking for job opportunities just about anywhere.But I, again, data professional. I really wanted a role that would allow me to use my math skills—I have a master's in mathematics—I wanted to use those math and analytical skills and go beyond the data into the application of the data. So, in the past five, six years, I've been earning a lot of data science certifications, I've been just getting back into my roots, right, statistical analysis, even my Six Sigma training is suddenly relevant again. So, what happened was I was on LinkedIn and friend had posted a note and mentioned Selector. I clicked on the link, and [all of sudden 00:04:17] I read, I go, “So, here's a company that is literally building new tools and it's data-science-centric. Is data-science-first.”It is, “We are going to find a way to go through your data and truly build out a better set of correlations to get you a signal through the noise.” Traditional monitoring tools, you know, collect a lot of things and then they kind of tell you what's wrong. Or you're collecting a lot of different things, so they slap, like, I don't know, timestamps in there and they guess at correlations. And these people are like, “No, no, no. We're going to go through everything and we will tell you what the data really says about your environment.”And I thought it was crazy how at the moment I was looking for a role that involve data and advocacy, the moment I'm looking for that role, that company was looking for someone like me. And so, I reached out immediately. They wanted not just a resume, but they're like, where's your portfolio? Have you spoken before? I'm like, “Yeah, I've spoken in a couple places,” right?So, I gave them everything, I reached right out to the recruiter. I said, “In case it doesn't arrive, let me know. I'll send it again. But this sounds very interesting.” And it didn't take more than—Corey: Exactly. [unintelligible 00:05:24] delivery remains hard.Thomas: Yeah. And it didn't take more than a couple of weeks. And I had gone through four or five interviews, they said that they were going to probably fly me out to Santa Clara to do, like, a last round or whatever. That got changed at some point and we went from, “Hey, we'll have you fly out,” to, “Hey, here's the offer. Why don't you just sign?” And I'm like, “Yeah, I'll start Monday. Let's go.”Corey: Fantastic. I imagine at some point, you'll be out in this neck of the woods just for an off-site or an all-hands or basically to stare someone down when you have a sufficiently large disagreement.Thomas: Yes, I do expect to be out there at some point. Matter of fact, I think one of my trips coming up might be to San Diego if you happen to head down south.Corey: Oh, I find myself all over the place these days, which is frankly, a welcome change after a few years of seclusion during the glorious pandemic years. What I like about Selector's approach, from what I can tell at least, is that it doesn't ask all of its customers to, “Hey, you know, all that stuff that you've instrumented over the last 20 years with a variety of different tools in the observability pipeline? Yeah, rip them all out and replace them with our new shiny thing.” Which never freaking happens. It feels like it's a better step toward meeting folks where they are.Thomas: Yeah. So, we're finding—I talk like I've been there forever: “What we're finding,”—in the past 40 hours of my work experience there, what we're finding, if you just look at the companies that are listed on the website, you'll get an idea for the scale that we're talking about. So no, we're not there to rip and replace. We're not going to show up and tell you, “Yeah, get rid of everything. We're going to do that for you.”Matter of fact, we think it's great you have all of those different things because it just reflects the complexity of your environment right now, is that you've grown, you've got so many disparate systems, you've got some of the technologies trying to monitor it all, and you're really hoping to have everything rolled into one big dashboard, right? Instead of right now, you've got to go through three, four, or five dashboards, to even think you have an idea of the problem. And you never really—you guess. We all guess. We think we know where it is, and you start looking and then you figure it out.But yeah, we take kind of a different approach right from the start, and we say, “Great, you've got all that data? Ingest it. Bring it right to us, okay? We don't care where it comes from, we can bring it in, and we can start going through it and start giving you true actionable insights.” We can filter out the noise, right, instead of one node going down, triggering a thousand alerts, we can just filter all of that out for you and just let you focus on the things that you need to be looking at right now.Corey: One of the things that I think gets overlooked in this space a lot is, “Well, we have this tool that does way better than that legacy tool that you're using right now and it's super easy to do a just drop-in replacement with our new awesomeness.” Great. What that completely misses is that there are other business units who perhaps care about data interchange and the idea that yeah, thing's a legacy piece of junk and replacing it would take an afternoon. And then it would take 14 years to wind up redoing all the other reports that other things are generating downstream of that because they integrate with that thing. So yeah, it's easy to replace the thing itself, but not in a way that anything else can take advantage of it.Thomas: Right.Corey: And when it turns out also when you sit there making fun of people's historical technological decisions, they don't really like becoming customers as it turns out. This was something of a shock for an awful lot of very self-assured startup founders in the early days.Thomas: Yeah. And again, you're talking about how, you know some of the companies we're looking at, it's y—we don't want to rip and replace things. Like you just said, you've got an ecosystem. It's a delicate ecosystem that has [laugh] developed over time. We aren't interested in replacing all that. We want to enhance it, we want to be on top of it and amplify what's in there for you.So yeah, we're not interested in coming in and say, “Yeah, rip every tool out.” And in some ways, when somebody will ask, you know, “Who do you compete with?” I'll go, “Nobody.” Because I'm not looking to replace anybody. I'm looking to go on top.And again, the companies we're dealing with have lots of data. We're talking very large companies. Some of these are the backbone of the internet. They just have way too much data for any of these legacy tools to help with, you know? They can help with, like, little things, but in terms of making sense of it all, in terms of doing the real big data analytics, yeah, that's where our tool comes in and it really shines.Corey: Yeah, it turns out that is not a really compelling sales pitch to walk it and say, “Hey, listen up, idiots, you all are doing it wrong. Now, pay me and we'll do it right.” Yeah, even if you're completely right, you've already lost the room at that point.Thomas: Exactly.Corey: People make decisions based upon human aspects, not about arithmetic, in most cases. I will say, taking a glance at the website, a couple of things are very promising. One, your picture and profile are already up there, which is good. No one is still on the fence about that, and further as a bonus, they've taken your job role down off the website, which is always disconcerting when you're there and, “Why is that job still open?” “Oh, we're preserving optionality. Don't you worry your head about that. We've got it.” No one finds that a reassuring story when it's about the role that they're in. So, good selection.Thomas: I went to—after I signed, it was within the day, I went to send somebody the link to the job req. Like, they're like, “What are”—I go, “Here, let me show you.” It was already down. The ink was even dry on the DocuSign and it was already down. So, I thought that—Corey: Good on them.Thomas: —was a good sign, too.Corey: Oh, yeah. Now, looking at the rest of your website, I do see a couple of things that lead to natural questions. One of the first things I look at on a web page is, okay, how is this thing priced? Because you always want to see the free tier option when I'm trying to solve a problem the middle of the night that I can just sign up for and see if it works for a small use case, but you also, in a big company definitely want to have the ‘Contact Us' option because we're procurement and we don't know how to sign a deal that doesn't have two commas in it with a bunch of special terms that ride along with it. Selector does not at the time of this recording, have a pricing page at all, which usually indicates if you have to ask, it might not be for you.Then I look at your customer case studies and they talk about very large enterprises, such as a major cable operator, for example, or TracFone. And oh okay, yeah, that is probably not the scale that I tend to be operating at. So, if I were to envision this as a carnival ride and there's a sign next to it, “You must be at least this tall to ride,” how tall should someone be?Thomas: That is a great way of putting it and I would—I can't really go into specifics because I'm still kind of new. But my understanding—Corey: Oh yeah. Make sweeping policy statements about your new employer 40 hours in. What could possibly go wrong?Thomas: My understanding is the companies that we—that are our target market today are fairly large enterprises with real data challenges, real monitoring data challenges. And so no, we're not doing—it's not transactional. You can't just come to our website and say, “Here, click this, you'll be up and running.” Because the volumes of data we're talking about, this requires a little bit of specialty in helping make sure that things are getting set up and correct.Think of it this way. Like if somebody said, “Here, do the statistical analysis on whatever, and here's Excel and go at it and get me that report by the end of the day and tell me how we're doing,” most people would be like, “I don't have enough information on that. Can you help me?” So, we're still at that, hey, we're going to need to help you through this and make sure it's correctly configured. And it's doing what you expect. So, how tall are you? I think that goes both ways. I think you're at a height where you still need some supervision [laugh]. Does that make sense?Corey: I think that's probably a good way of framing it. It's a—again, I'm not saying that you should never ever, ever, ever have a ‘you must contact us to get started.' There are a bunch of products like that out there. It turns out that even at The Duckbill Group here, we always want to have a series of conversations first. We don't have a shopping cart that's, “One consulting, please,” just because we'll get into trouble with that.Though I think our first pass offering of a two-day engagement might have one of those somewhere still lurking around. Don't quote me on that. Hell is other people's websites. It's great. But your own yeah, whoever reads that thing“. Wait, we're saying what?” Don't quote me on any of that, my God.Thomas: But I think that's a good way of putting it. Like, you want to have some conversations first. Yeah, so you—and again, we're still, we're fairly young. We've only—we're Series A, so we've been around 16 months, like… you know, the other website you're looking at is probably going to change within the next six or eight weeks just because information gets outdated—Corey: It already has. It put your picture on it.Thomas: Right. But I mean, things are going to things move pretty fast with startups, especially this one. So, I just expect that over time, I envision some type of a free tier, but we're not there yet.Corey: That's one of those challenges as far as in some cases moving down market. I found that anything that acts like a security tool, for example, has to, on some level, charge enough to be worth the squeeze. One of the challenges there is, I'm either limited for anything that does CloudTrail analysis over in AWS-land, for example. I can either find a bunch of janky things off GitHub or I can spend what starts at $1,000 a month and increases rapidly from there, which is about twice the actual AWS bill that it would wind up alerting on. Not that the business value isn't there, but because a complex sale is, in many cases, always going to be attendant with some of these products, so why not go after the larger companies where the juice is worth the squeeze rather than the folks who are not going to see the value and it'd be just as challenging to wind up launching a sale into?The corollary, of course, is that some of those small companies do in fact, grow meteorically. But it's a bit of a lottery.Thomas: Yep.Corey: Ugh. So, I have to ask as well, while we're talking about strange decisions that people might have made, in the world of tech, in many cases, when someone gets promoted—like, “So, does that mean extra money?” “No, not really. We just get extra adjectives added to our job title.” Good for us. You have decided to add letters in a different way, by going back for a second master's degree. What on earth would possess you to do such a thing?Thomas: I—man, that is—you know, so I got my first master's degree because I thought I was going to, I thought I was be a math teacher and basketball coach. And I had a master's degree in math and I thought that was going to be a thing. I'll get a job, you know, coaching and teaching at some small school somewhere. But then I realized that I enjoyed things like eating and keeping the wind off me, and so I realized I had to go get a jobby-job. And so, I took my masters in math, I ended—I got a job as a software analyst, and just rolled that from one thing to another until where I am today.But about four years ago, when I started falling back in love with my roots in math, and statistical analysis became a real easy thing for people to really start doing for themselves—well actually, that was about eight years ago—but the past four or five years, I've been earning more certifications in data science technologies. And then I found this program at Georgia Tech. So, Georgia Tech has an online masters of science and data analytics. And it's extremely affordable. So, I looked at a lot of programs, Corey, over the past few years, especially during the pandemic.I had some free time, so I browsed the love these places, and they were charging 50, $60,000 and you had to do it within two, three years. And in one case, the last class you had to take, your practicum, had to be all done on campus. So, you had to go, like, live somewhere. And I'm looking at all—none of that was practical. And all of a sudden, somebody shows up and goes, “So, you can go online, fully online, Georgia Tech, $275 a credit. Costs ten grand for the entire program.”And you can—it's geared towards a working professional and you can take anywhere from two to six years. So, you take, like, one class a semester if you want, or two or even three if they allow you, but they usually restrict you. So, it just blew my mind. Like, this exists today that I can start earning another Master's degree in data analytics and I'll say, be… classically trained in how—it's funny because when I learn things in class, I'm like, I feel like I'm Thornton Melon in Back to School, and I'm just like, “Oh, you left out a bunch of stuff. That isn't how you do it all,” right?That's kind of my reaction. I'm like, “Calm down. I'm sure the professor has point. I'll hear [laugh] him out.” But to me, you asked why, and I just the challenge. Am I really good at what I do? Like, I feel I am. I already have a master's degree. I'm not worried about the level of work and the commitment involved in earning another one.I just wanted to show to myself that could—I want to learn and make sure I can do things like code in Python. If anybody has a chance to take a programming class, a graduate-level programming classes at Georgia Tech, you should do it. You should see where your skills rate at that level, right? So, it was for the challenge. I want to know if I can do it. I'm three classes in. I just started my fourth, actually, today was the start of the fall semester.And so, I'm about halfway through, and I'm loving it. It's not too taxing. It's just the right speed for me. I get to do it in my leisure hours as they were. Yeah, so I did it for the challenge. I'm really glad I'm doing it. I encourage anybody interested in obtaining a degree in data analytics to look at the Georgia Tech program. It's well worth it. Georgia Tech's not a bad school. Like, if you had to go to school in the South, it's all right.Corey: I always find it odd, just, you had your first master's degree in, you know, mathematics, and now you're going for data analytics, which sounds like mathematics with extra steps.Thomas: It is.Corey: Were there opportunities that you were hoping to pursue that were not available to you with just the one master's degree?Thomas: So, it's interesting you say that because I'm so old that when I went to school, all we had was math, that was it. It was pure mathematics. I could have been a statistics major, I think, and computer science was a thing. And one day I met a guy who transferred into math from computer science. I'm like, “Why would you do that? What are you going to do with the degree in math?”And his response is, “What am I going to do with a degree in computer science?” And I look back and I realized how we were both right. So, I think at the time if there had been a course in applied mathematics, that would have piqued my interest. Like, what am I going to do with this math degree other than become an actuary because that was about all I knew at the time. You were a teacher or an actuary, and that was about it.So, the idea now that they have these programs in data analytics or data science that are little more narrow of focus, like, “This is what we're going to do: we're going to apply a little bit of math, some calculus, some stats; we're going to show you how to build your own simulations; we're going to show you how to ask the right questions of the data.” To give you a little bit of training. Because they can't teach you everything. You really have to have real-world experience in whatever domain you're going to focus on, be it finance or marketing or whatever. All these bright financial operations, that's just analytics for finance, marketing operations, that's analytics for marketing. It's just, to me, I think just the opportunity to have that focus would have been great back then and it didn't exist. And I want to take advantage of it now.Corey: I've always been a fan of advising people who ask me, “Should I go back to school,” because usually, there's something else driving that. Like, I am honestly not much of a career mentor. My value basically comes in as being a horrible warning to others. On paper, I have an eighth-grade education. I am not someone to follow for academic approaches.But when someone early or mid-career asks, “Should I get another degree?” Unpacking that is always a bit of a fun direction for me to go in. Because at some level, we've sold entire generations a bill of goods, where oh, if you don't know what to do, just get more credentials and then your path will be open to you in a bunch of new and exciting ways. Okay, great. I'm not saying that's inherently wrong, but talk to people doing the thing you'd want to do after you have that degree, maybe, you know, five or six years down the professional line from where you are and get their take on it.Because in some cases, yeah, there are definite credentials you're going to need—I don't want you to be a self-taught surgeon, for example—but there are other things where it doesn't necessarily open doors. People are just reflexively deciding that I'm going to go after that instead. And then you can start doing the math of, okay, assume that you have whatever the cost of the degree is in terms of actual cost and opportunity cost. Is this the best path forward for you to wind up getting where you want to go? It sounds like in your particular case, this is almost a labor of love or a hobby style of approach, as opposed to, “Well, I really want Job X, but I just can't get it without the right letters after my name.” Is that a fair assessment?Thomas: It's not unfair. It is definitely fair, but I would also say, you know, if somebody came and said, “Hey Tom, we need somebody to run our data science team or our data engineering team,” I've got the experience for—the only thing I would be lacking is, you know, production experience, like, with machine-learning pipelines or something. I don't have that today.Corey: Which is basically everyone else, too, but that's a little—bit of a quiet secret in the industry.Thomas: Yeah, that's—okay. Bad example. But you know what I'm saying is that the only thing I'd be lacking would be that practical experience, so this is one way that—to at least start that little bit of experience, especially with the end result being the practicum that we'll be doing. It's, like, six credits at the very end. So yes, it's a fair thing.I wouldn't—hobby isn't really the right—this is really something that makes me get out of bed in the morning. I get to work with data today and I'm going to get—I'm going to tell a great story using data today. I really do enjoy those things. But then at the tail end of this, if it happens to lead to a position that somebody says, “Hey, we need somebody, vice president of data engineering. This a really good”—honestly, the things I look for are the roles and the roles I want are to have a role that allows me to really have an impact on other people's lives.And that's one of the things about Selector. The things that we're able to do for these admins that are just drowning in data, the data is just in their way, and that we can help them make sense of it all, to me, that's impactful. So, those are the types of roles that I will be looking for as well in the future, especially at the high level of something data science-y.Corey: I think that that is a terrific example of what I'm talking about. Because I've met a number of folks, especially very early-20s range where, okay, they've gotten the degree, but now they don't know what to do because every time they're applying for jobs, it doesn't seem to work for them. You've been around this industry for 25 years. Everyone needs a piece of paper that says they know certain things, and in your case, it long ago transitioned into being—I would assume—your resumé, the history of things you have done that look equivalent. Part of me, on some level, wonders if there isn't an academic snobbery going on at some level, where a number of teams are, “Oh, we'd love to have you in, but you don't have a PhD.”And then people get the PhD. “From the right school, in the right area of concentration.” It's like, you just keep moving these very expensive goalposts super quickly. Remember, I have an eighth-grade education. I'm not coming at this from a place of snobbery and I'm also not one of those folks who's well it didn't work for me, therefore, it won't work for anyone else either because that's equally terrible in a different direction.It's just making sure that people are going into these things with their eyes open. With you, it's never been a concern. You've been around this industry so long that it is extremely unlikely to me [laugh] that you, “Oh, wait. You mean a degree won't magically solve all of my problems and regrow some of my hair and make me two inches taller, et cetera, et cetera?” But yeah, do I remember in the early days just how insipid and how omnipresent that pressure was.Thomas: Yeah. I've been at companies where we've brought in people because of the education and—or I'm sorry. Let's be more specific. I've been at companies where we've sent current employees—as we used to call it—off the charm school, which is basically [MBA 00:25:44].Corey: [laugh].Thomas: And I swear, so many of them came back and they just forgot how to think, how to have common sense. Like, they were very much focused on one particular thing and this is just it, and they forgot there were maybe humans involved, and maybe look for a human answer instead of the statistically correct one. So, I think that was a good thing for me as well to be around that because, yeah, somebody put it me best years ago: “Education by itself isn't enough. If you combine education with motivation, now you've really got something.” And your case, I don't know where you went for eighth grade, it could have been the best eighth-grade program ever, but you definitely have the motivation through the years to overcome anything that might have been lacking in the form of education. So, it's really the combination—Corey: Oh, you'd be surprised. A lot of those things are still readily apparent to people who work with me, so I've done a good job of camouflaging them. Hazzah.Thomas: Just it's, you got to have both. You can't just rely on one or the other.Corey: So, last question, given that you are the data guy and SQLRockstar is your username in a bunch of places. What's the best database? I mean, I would always say it's Route 53, but I understand that can be controversial for some folks, given that their SQL implementation is not yet complete. What's your take?Thomas: So clearly, I'm partial to anything inside the Microsoft data platform, with the exception being Access. I think if Access disappeared from the universe… society might be better off. But that's for a different day, I think the best database is the one that does the job you need it to do. Honestly, the database shouldn't really matter. It's just an abstraction. The database engine is just something in between you and the data you need, right?So, whatever you're using, if it's doing the job that you need it to do, then that's the best database you could have. I learned a long time ago to not pick sides, choose fiefdoms. Like, it just didn't matter. It's all kind of the same. And in a lot of cases, if you go to, like, the DB-Engines Rankings, you'll see how many of these systems these days, there's a lot of overlap. They offer all the same features and the differences between them are getting smaller and smaller in a lot of cases. So yeah, it's… you got to database, it does what you need to do? That's great. That's the best database.Corey: Especially since any database, I suspect, can be made to perform a given task, even if sub-optimally. Which states back to my core ethos of, quite frankly, anything is a database if you hold it wrong.Thomas: Yeah, it really is. I mean, we've had those discussions. I kid about Access because it's just a painful thing for a lot of different reasons. But is Excel a database? And I would say no but, you know—because it can't do certain things that I would expect a relational engine to do. And then you find out, well, I can make it do those things. So, now is it a database? And, yeah…Corey: [laugh]. Yeah. Well, what if I apply some brute force? Will it count then? Like, you have information, Thomas. Can I query you?Thomas: Yes. Yes, yes, [laugh] you can. I also have latency.Corey: Exactly. That means you are a suboptimal database.Thomas: [laugh].Corey: Good job. I really want to thank you for taking the time to talk about what you're up to these days and finally coming on the show. If people want to learn more, where's the best place for them to find you?Thomas: Well, I'm becoming more active on LinkedIn. So, it's linkedin/in/sqlrockstar. Just search for SQLRockstar, you'll find me everywhere. I mean, I do have a blog. I rarely blog these days. Most of the posts I do is over at LinkedIn.And you might find me at some networking events coming up since Selector really does focus on network observability. So, you could see me there. And you know what? I'm also going to have an appearance on the Screaming in the Cloud podcast, so you can listen to me there.Corey: Excellent. And I imagine that's the one we don't have to put into these [show notes. 00:29:44]. Thank you so much for taking the time to speak with me. I really do appreciate it.Thomas: Thanks for having me, Corey. I look forward to coming back.Corey: As I look forward to seeing you again over here. Thomas LaRock, Principal Developer Evangelist at Selector. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an insulting comment because then we're going to use all those together as a distributed database.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

В свежем выпуске мы обсудили множество актуальных новостей об AWS за последнее время (Q1-Q2 2023): LightSail for Research - новый сервис для начала работы с Data Science в AWS с низким порогом входа. Обновления в Amazon Aurora - новая ценовая модель без отдельной оплаты операций ввода-вывода, возможные экономии до 40%. AWS S3 Object Lambda - теперь возможно запускать лямбда-функции при GET-запросах к объектам S3. AWS User Notifications - новый сервис для получения объединенных нотификаций от более чем 100 AWS-сервисов. Обзор Amazon CodeWhisperer  - генеративный AI-ассистент для ускорения написания кода. Рассмотрены возможности Amazon Service Catalog для автоматизации развертывания стандартных инфраструктурных компонентов. Улучшения видимости ресурсов VPC на новой ресурс-карте. Как всегда, в подкасте много интересных подробностей и примеров. Заходите на наш канал, слушайте, оставляйте отзывы!   Упомянутые  telegram каналы: https://t.me/aws_notes и https://t.me/awsweekly    Если у вас есть вопросы, предложения темы, пишите мне в LinkedIn https://www.linkedin.com/in/vedmich/ или телеграмм https://t.me/ViktorVedmich  

Kelsey Hightower joins Corey on Screaming in the Cloud to discuss his reflections on how the tech industry is progressing. Kelsey describes what he's been getting out of retirement so far, and reflects on what he learned throughout his high-profile career - including why feature sprawl is such a driving force behind the complexity of the cloud environment and the tactics he used to create demos that are engaging for the audience. Corey and Kelsey also discuss the importance of remaining authentic throughout your career, and what it means to truly have an authentic voice in tech. About KelseyKelsey Hightower is a former Distinguished Engineer at Google Cloud, the co-chair of KubeCon, the world's premier Kubernetes conference, and an open source enthusiast. He's also the co-author of Kubernetes Up & Running: Dive into the Future of Infrastructure. Recently, Kelsey announced his retirement after a 25-year career in tech.Links Referenced:Twitter: https://twitter.com/kelseyhightower TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Do you wish there were cheat codes for database optimization? Well, there are – no seriously. If you're using Postgres or MySQL on Amazon Aurora or RDS, OtterTune uses AI to automatically optimize your knobs and indexes and queries and other bits and bobs in databases. OtterTune applies optimal settings and recommendations in the background or surfaces them to you and allows you to do it. The best part is that there's no cost to try it. Get a free, thirty-day trial to take it for a test drive. Go to ottertune dot com to learn more. That's O-T-T-E-R-T-U-N-E dot com.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. You know, there's a great story from the Bible or Torah—Old Testament, regardless—that I was always a big fan of where you wind up with the Israelites walking the desert for 40 years in order to figure out what comes next. And Moses led them but could never enter into what came next. Honestly, I feel like my entire life is sort of going to be that direction. Not the biblical aspects, but rather always wondering what's on the other side of a door that I can never cross, and that door is retirement. Today I'm having returning guest Kelsey Hightower, who is no longer at Google. In fact, is no longer working and has joined the ranks of the gloriously retired. Welcome back, and what's it like?Kelsey: I'm happy to be here. I think retirement is just like work in some ways: you have to learn how to do it. A lot of people have no practice in their adult life what to do with all of their time. We have small dabs in it, like, you get the weekend off, depending on what your work, but you never have enough time to kind of unwind and get into something else. So, I'm being honest with myself. It's going to be a learning curve, what to do with that much time.You're probably still going to do work, but it's going to be a different type of work than you're used to. And so, that's where I am. 30 days into this, I'm in that learning mode, I'm on-the-job training.Corey: What's harder than you expected?Kelsey: It's not the hard part because I think mentally I've been preparing for, like, the last ten years, being a minimalist, learning how to kind of live within my means, learn to appreciate things that are just not work-related or status symbols. And so, to me, it felt like a smooth transition because I started to value my time more than anything else, right? Just waking up the next day became valuable to me. Spending time in the moment, right, you go to these conferences, there's, like, 10,000 people, but you learn to value those one-on-one encounters, those one-off, kind of, let's just go grab lunch situations. So, to me, retirement just makes more room for that, right? I no longer have this calendar that is super full, so I think for me, it was a nice transition in terms of getting more of that valuable time back.Corey: It seems to me that you're in a similar position to the one that I find myself in where the job that you were doing and I still am is tied, more or less, to a sense of identity as opposed to a particular task or particular role that you fill. You were Kelsey Hightower. That was a complete sentence. People didn't necessarily need to hear the rest of what you were working on or what you were going to be talking about at a given conference or whatnot. So, it seemed, at least from the outside, that an awful lot of what you did was quite simply who you were. Do you feel that your sense of identity has changed?Kelsey: So, I think when you have that much influence, when you have that much reputation, the words you say travel further, they tend to come with a little bit more respect, and so when you're working with a team on new product, and you say, “Hey, I think we should change some things.” And when they hear those words coming from someone that they trust or has a name that is attached to reputation, you tend to be able to make a lot of impact with very few words. But what you also find is that no matter what you get involved in—configuration management, distributed systems, serverless, working with customers—it all is helped and aided by the reputation that you bring into that line of work. And so yes, who you are matters, but one thing that I think helped me, kind of greatly, people are paying attention maybe to the last eight years of my career: containers, Kubernetes, but my career stretches back to the converting COBOL into Python days; the dawn of DevOps, Puppet, Chef, and Ansible; the Golang appearance and every tool being rewritten from Ruby to Golang; the Docker era.And so, my identity has stayed with me throughout those transitions. And so, it was very easy for me to walk away from that thing because I've done it three or four times before in the past, so I know who I am. I've never had, like, a Twitter bio that said, “Company X. X person from company X.” I've learned long ago to just decouple who I am from my current employer because that is always subject to change.Corey: I was fortunate enough to not find myself in the public eye until I owned my own company. But I definitely remember times in my previous incarnations where I was, “Oh, today I'm working at this company,” and I believed—usually inaccurately—that this was it. This was where I really found my niche. And then surprise I'm not there anymore six months later for, either their decision, my decision, or mutual agreement. And I was always hesitant about hanging a shingle out that was tied too tightly to any one employer.Even now, I was little worried about doing it when I went independent, just because well, what if it doesn't work? Well, what if, on some level? I think that there's an authenticity that you can bring with you—and you certainly have—where, for a long time now, whenever you say something, I take it seriously, and a lot of people do. It's not that you're unassailably correct, but I've never known you to say something you did not authentically believe in. And that is an opinion that is very broadly shared in this industry. So, if nothing else, you definitely were a terrific object lesson in speaking the truth, as you saw it.Kelsey: I think what you describe is one way that, whether you're an engineer doing QA, working in the sales department, when you can be honest with the team you're working with, when you can be honest with the customers you're selling into when you can be honest with the community you're part of, that's where the authenticity gets built, right? Companies, sometimes on the surface, you believe that they just want you to walk the party line, you know, they give you the lines and you just read them verbatim and you're doing your part. To be honest, you can do that with the website. You can do that with a well-placed ad in the search queries.What people are actually looking for are real people with real experiences, sharing not just fact, but I think when you mix kind of fact and opinion, you get this level of authenticity that you can't get just by pure strategic marketing. And so, having that leverage, I remember back in the day, people used to say, “I'm going to do the right thing and if it gets me fired, then that's just the way it's going to be. I don't want to go around doing the wrong thing because I'm scared I'm going to lose my job.” You want to find yourself in that situation where doing the right thing, is also the best thing for the company, and that's very rare, so when I've either had that opportunity or I've tried to create that opportunity and move from there.Corey: It resonates and it shows. I have never had a lot of respect for people who effectively are saying one thing today and another thing the next week based upon which way they think that the winds are blowing. But there's also something to be said for being able and willing to publicly recant things you have said previously as technology evolves, as your perspective evolves and, in light of new information, I'm now going to change my perspective on something. I've done that already with multi-cloud, for example. I thought it was ridiculous when I heard about it. But there are also expressions of it that basically every company is using, including my own. And it's a nuanced area. Where I find it challenging is when you see a lot of these perspectives that people are espousing that just so happen to deeply align with where their paycheck comes from any given week. That doesn't ring quite as true to me.Kelsey: Yeah, most companies actually don't know how to deal with it either. And now there has been times at any number of companies where my authentic opinion that I put out there is against party line. And you get those emails from directors and VPs. Like, “Hey, I thought we all agree to think this way or to at least say this.” And that's where you have to kind of have that moment of clarity and say, “Listen, that is undeniably wrong. It's so wrong in fact that if you say this in public, whether a small setting or large setting, you are going to instantly lose credibility going forward for yourself. Forget the company for a moment. There's going to be a situation where you will no longer be effective in your job because all of your authenticity is now gone. And so, what I'm trying to do and tell you is don't do that. You're better off saying nothing.”But if you go out there, and you're telling what is obviously misinformation or isn't accurate, people are not dumb. They're going to see through it and you will be classified as a person not to listen to. And so, I think a lot of people struggle with that because they believe that enterprise's consensus should also be theirs.Corey: An argument that I made—we'll call it a prediction—four-and-a-half years ago, was that in five years, nobody would really care about Kubernetes. And people misunderstood that initially, and I've clarified since repeatedly that I'm not suggesting it's going away: “Oh, turns out that was just a ridiculous fever dream and we're all going back to running bare metal with our hands again,” but rather that it would slip below the surface-level of awareness. And I don't know that I got the timing quite right on that, I think it's going to depend on the company and the culture that you find yourself in. But increasingly, when there's an application to run, it's easy to ask someone just, “Oh, great. Where's the Kubernetes cluster live so we can throw this on there and just add it to the rest of the pile?”That is sort of what I was seeing. My intention with that was not purely just to be controversial, as much fun as that might be, but also to act as a bit of a warning, where I've known too many people who let their identities become inextricably tangled with the technology. But technologies rise and fall, and at some point—like, you talk about configuration management days; I learned to speak publicly as a traveling trainer for Puppet. I wrote part of SaltStack once upon a time. But it was clear that that was not the direction the industry was going, so it was time to find something else to focus on. And I fear for people who don't keep an awareness or their feet underneath them and pay attention to broader market trends.Kelsey: Yeah, I think whenever I was personally caught up in linking my identity to technology, like, “I'm a Rubyist,” right?“, I'm a Puppeteer,” and you wear those names proudly. But I remember just thinking to myself, like, “You have to take a step back. What's more important, you or the technology?” And at some point, I realized, like, it's me, that is more important, right? Like, my independent thinking on this, my independent experience with this is far more important than the success of this thing.But also, I think there's a component there. Like when you talked about Kubernetes, you know, maybe being less relevant in five years, there's two things there. One is the success of all infrastructure things equals irrelevancy. When flights don't crash, when bridges just work, you do not think about them. You just use them because they're so stable and they become very boring. That is the success criteria.Corey: Utilities. No one's wondering if the faucet's going to work when they turn it on in the morning.Kelsey: Yeah. So, you know, there's a couple of ways to look at your statement. One is, you believe Kubernetes is on the trajectory that it's going to stabilize itself and hit that success criteria, and then it will be irrelevant. Or there's another part of the irrelevancy where something else comes along and replaces that thing, right? I think Cloud Foundry and Mesos are two good examples of Kubernetes coming along and stealing all of the attention from that because those particular products never gained that mass adoption. Maybe they got to the stable part, but they never got to the mass adoption part. So, I think when it comes to infrastructure, it's going to be irrelevant. It's just what side of that [laugh] coin do you land on?Corey: It's similar to folks who used to have to work at a variety of different companies on very specific Linux kernel subsystems because everyone had to care because there were significant performance impacts. Time went on and now there's still a few of those people that very much need to care, but for the rest of us, it is below the level of things that we have to care about. For me, the signs of the unsustainability were, oh, you can run Kubernetes effectively in production? That's a minimum of a quarter-million dollars a year in comp or up in some cases. Not every company is going to be able to field a team of those people and still remain a going concern in business. Nor frankly, should they have to.Kelsey: I'm going to pull on that thread a little bit because it's about—we're hitting that ten-year mark of Kubernetes. So, when Kubernetes comes out, why were people drawn to it, right? Why did it even get the time of day to begin with? And I think Docker kind of opened Pandora's box there. This idea of Chef, Puppet, Ansible, ten thousand package managers, and honestly, that trajectory was going to continue forever and it was helping no one. It was literally people doing duplicate work depending on the operating system you're dealing with and we were wasting time copying bits to servers—literally—in a very glorified way.So, Docker comes along and gives us this nicer, better abstraction, but it has gaps. It has no orchestration. It's literally this thing where now we've unified the packaging situation, we've learned a lot from Red Hat, YUM, Debian, and the various package repo combinations out there and so we made this universal thing. Great. We also learned a little bit about orchestration through brute force, bash scripts, config management, you name it, and so we serialized that all into this thing we call Kubernetes.It's pretty simple on the surface, but it was probably never worthy of such fanfare, right? But I think a lot of people were relieved that now we finally commoditized this expertise that the Googles, the Facebooks of the world had, right, building these systems that can copy bits to other systems very fast. There you go. We've gotten that piece. But I think what the market actually wants is in the mobile space, if you want to ship software to 300 million people that you don't even know, you can do it with the app store.There's this appetite that the boring stuff should be easy. Let's Encrypt has made SSL certificates beyond easy. It's just so easy to do the right thing. And I think for this problem we call deployments—you know, shipping apps around—at some point we have to get to a point where that is just crazy easy. And it still isn't.So, I think some of the frustration people express ten years later, they're realizing that they're trying to recreate a Rube Goldberg machine with Kubernetes is the base element and we still haven't understood that this whole thing needs to simplify, not ten thousand new pieces so you can build your own adventure.Corey: It's the idea almost of what I'm seeing AWS go through, and to some extent, its large competitors. But building anything on top of AWS from scratch these days is still reminiscent of going to Home Depot—or any hardware store—and walking up and down the aisles and getting all the different components to piece together what you want. Sometimes just want to buy something from Target that's already assembled and you have to do all of that work. I'm not saying there isn't value to having a Home Depot down the street, but it's also not the panacea that solves for all use cases. An awful lot of customers just want to get the job done and I feel that if we cling too tightly to how things used to be, we lose it.Kelsey: I'm going to tell you, being in the cloud business for almost eight years, it's the customers that create this. Now, I'm not blaming the customer, but when you start dealing with thousands of customers with tons of money, you end up in a very different situation. You can have one customer willing to pay you a billion dollars a year and they will dictate things that apply to no one else. “We want this particular set of features that only we will use.” And for a billion bucks a year times ten years, it's probably worth from a business standpoint to add that feature.Now, do this times 500 customers, each major provider. What you end up with is a cloud console that is unbearable, right? Because they also want these things to be first-class citizens. There's always smaller companies trying to mimic larger peers in their segment that you just end up in that chaos machine of unbound features forever. I don't know how to stop it. Unless you really come out maybe more Apple style and you tell people, “This is the one and only true way to do things and if you don't like it, you have to go find an alternative.” The cloud business, I think, still deals with the, “If you have a large payment, we will build it.”Corey: I think that that is a perspective that is not appreciated until you've been in the position of watching how large enterprises really interact with each other. Because it's, “Well, what customer the world is asking for yet another way to run containers?” “Uh, this specific one and their constraints are valid.” Every time I think I've seen everything there is to see in the world of cloud, I just have to go talk to one more customer and I'm learning something new. It's inevitable.I just wish that there was a better way to explain some of this to newcomers, when they're looking at, “Oh, I'm going to learn how this cloud thing works. Oh, my stars, look at how many services there are.” And then they wind up getting lost with analysis paralysis, and every time they get started and ask someone for help, they're pushed in a completely different direction and you keep spinning your wheels getting told to start over time and time again when any of these things can be made to work. But getting there is often harder than it really should be.Kelsey: Yeah. I mean, I think a lot of people don't realize how far you can get with, like, three VMs, a load balancer, and Postgres. My guess is you can probably build pretty much any clone of any service we use today with at least 1 million customers. Most people never reached that level—I don't even want to say the word scale—but that blueprint is there and most people will probably be better served by that level of simplicity than trying to mimic the behaviors of large customers—or large companies—with these elaborate use cases. I don't think they understand the context there. A lot of that stuff is baggage. It's not [laugh] even, like, best-of-breed or great design. It's like happenstance from 20 years of trying to buy everything that's been sold to you.Corey: I agree with that idea wholeheartedly. I was surprising someone the other day when I said that if you were to give me a task of getting some random application up and running by tomorrow, I do a traditional three-tier architecture, some virtual machines, a load balancer, and a database service. And is that the way that all the cool kids are doing it today? Well, they're not talking about it, but mostly. But the point is, is that it's what I know, it's where my background is, and the thing you already know when you're trying to solve a new problem is incredibly helpful, rather than trying to learn everything along that new path that you're forging down. Is that architecture the best approach? No, but it's perfectly sufficient for an awful lot of stuff.Kelsey: Yeah. And so, I mean, look, I've benefited my whole career from people fantasizing about [laugh] infrastructure—Corey: [laugh].Kelsey: And the truth is that in 2023, this stuff is so powerful that you can do almost anything you want to do with the simplest architecture that's available to us. The three-tier architecture has actually gotten better over the years. I think people are forgotten: CPUs are faster, RAM is much bigger quantities, the networks are faster, right, these databases can store more data than ever. It's so good to learn the fundamentals, start there, and worst case, you have a sound architecture people can reason about, and then you can go jump into the deep end, once you learn how to swim.Corey: I think that people would be depressed to understand just how much the common case for the value that Kubernetes brings is, “Oh yeah, now we can lose a drive or a server and the application stays up.” It feels like it's a bit overkill for that one somewhat paltry use case, but that problem has been hounding companies for decades.Kelsey: Yeah, I think at some point, the whole ‘SSH is my only interface into these kinds of systems,' that's a little low level, that's a little bare bones, and there will probably be a feature now where we start to have this not Infrastructure as Code, not cloud where we put infrastructure behind APIs and you pay per use, but I think what Kubernetes hints at is a future where you have APIs that do something. Right now the APIs give you pieces so you can assemble things. In the future, the APIs will just do something, “Run this app. I need it to be available and here's my money budget, my security budget, and reliability budget.” And then that thing will say, “Okay, we know how to do that, and here's roughly what is going to cost.”And I think that's what people actually want because that's how requests actually come down from humans, right? We say, “We want this app or this game to be played by millions of people from Australia to New York.” And then for a person with experience, that means something. You kind of know what architecture you need for that, you know what pieces that need to go there. So, we're just moving into a realm where we're going to have APIs that do things all of a sudden.And so, Kubernetes is the warm-up to that era. And that's why I think that transition is a little rough because it leaks the pieces part, so where you can kind of build all the pieces that you want. But we know what's coming. Serverless also hints at this. But that's what people should be looking for: APIs that actually do something.Corey: This episode is sponsored in part by Panoptica.  Panoptica simplifies container deployment, monitoring, and security, protecting the entire application stack from build to runtime. Scalable across clusters and multi-cloud environments, Panoptica secures containers, serverless APIs, and Kubernetes with a unified view, reducing operational complexity and promoting collaboration by integrating with commonly used developer, SRE, and SecOps tools. Panoptica ensures compliance with regulatory mandates and CIS benchmarks for best practice conformity. Privacy teams can monitor API traffic and identify sensitive data, while identifying open-source components vulnerable to attacks that require patching. Proactively addressing security issues with Panoptica allows businesses to focus on mitigating critical risks and protecting their interests. Learn more about Panoptica today at panoptica.app.Corey: You started the show by talking about how your career began with translating COBOL into Python. I firmly believe someone starting their career today listening to this could absolutely find that by the time their career starts drawing to their own close, that Kubernetes is right in there as far as sounding like the deprecated thing that no one really talks about or thinks about anymore. And I hope so. I want the future to be brighter than the past. I want getting a business or getting software together in a way that helps people to not require the amount of, “First, spend six weeks at a boot camp,” or, “Learn how to write just enough code that you can wind up getting funding and then have it torn apart.”What's the drag-and-drop story? What's the describe the application to a robot and it builds it for you? I'm optimistic about the future of infrastructure, just because based upon its power to potentially make reliability and scale available to folks who have no idea of what's involved with that. That's kind of the point. That's the end game of having won this space.Kelsey: Well, you know what? Kubernetes is providing the metadata to make that possible, right? Like in the early days, people were writing one-off scripts or, you know, writing little for loops to get things in the right place. And then we get config management that kind of formalizes that, but it still had no metadata, right? You'd have things like Puppet report information.But in the world of, like, Kubernetes, or any cloud provider, now you get semantic meaning. “This app needs this volume with this much space with this much memory, I need three of these behind this load balancer with these protocols enabled.” There is now so much metadata about applications, their life cycles, and how they work that if you were to design a new system, you can actually use that data to craft a much better API that made a lot of this boilerplate the defaults. Oh, that's a web application. You do not need to specify all of this boilerplate. Now, we can give you much better nouns and verbs to describe what needs to happen.So, I think this is that transition as all the new people coming up, they're going to be dealing with semantic meaning to infrastructure, where we were dealing with, like, tribal knowledge and intuition, right? “Run this script, pipe it to this thing, and then this should happen. And if it doesn't, run the script again with this flag.” Versus, “Oh, here's the semantic meaning to a working system.” That's a game-changer.Corey: One other topic I wanted to ask you about—I've it's been on my list of things to bring up the next time I ran into you and then you went ahead and retired, making it harder to run into you. But a little while back, I was at a tech conference and someone gave a demo, and it didn't go as well as they had hoped. And a few of us were talking about it afterwards. We've all been speakers, we've all lived that life. Zero shade.But someone brought you up in particular—unprompted; your legend does precede you—and the phrase that they used was that Kelsey's demos were always picture-perfect. He was so lucky with how the demos worked out. And I just have to ask—because you don't strike me as someone who is not careful, particularly when all eyes are upon you—and real experts make things look easy, did you have demos periodically go wrong that the audience just didn't see going wrong along the way? Or did you just actually YOLO all of your demos and got super lucky every single time for the last eight years?Kelsey: There was a musician who said, “Hey, your demos are like jazz. You improvise the whole thing.” There's no script, there's no video. The way I look at the demo is, like, you got this instrument, the command prompt, and the web browser. You can do whatever you want with them.Now, I have working code. I wrote the code, I wrote the deployment scenarios, I delete it all and I put it all back. And so, I know how it's supposed to work from the ground up. And so, what that means is if anything goes wrong, I can improvise. I could go into fixing the code. I can go into doing a redeploy.And I'll give you one good example. The first time Kubernetes came out, there was this small meetup in San Francisco with just the core contributors, right? So, there is no community yet, there's no conference yet, just people hacking on Kubernetes. And so, we decided, we're going to have the first Kubernetes meetup. And everyone got, like, six, seven minutes, max. That's it. You got to move.And so, I was like, “Hey, I noticed that in the lineup, there is no ‘What is Kubernetes?' talk. We're just getting into these nuts and bolts and I don't think that's fair to the people that will be watching this for the first time.” And I said, “All right, Kelsey, you should give maybe an intro to what it is.” I was like, “You know what I'll do? I'm going to build a Kubernetes cluster from the ground up, starting with VMs on my laptop.”And I'm in it and I'm feeling confident. So, confidence is the part that makes it look good, right? Where you're confident in the commands you type. One thing I learned to do is just use your history, just hit the up arrow instead of trying to copy all these things out. So, you hit the up arrow, you find the right command and you talk through it and no one looks at what's happening. You're cycling through the history.Or you have multiple tabs where you know the next up arrow is the right history. So, you give yourself shortcuts. And so, I'm halfway through this demo. We got three minutes left, and it doesn't work. Like, VMware is doing something weird on my laptop and there's a guy calling me off stage, like, “Hey, that's it. Cut it now. You're done.”I'm like, “Oh, nope. Thou shalt not go out like this.” It's time to improvise. And so, I said, “Hey, who wants to see me finish this?” And now everyone is locked in. It's dead silent. And I blow the whole thing away. I bring up the VMs, I [pixie 00:28:20] boot, I installed the kubelet, I install Docker. And everyone's clapping. And it's up, it's going, and I say, “Now, if all of this works, we run this command and it should start running the app.” And I do kubectl apply-f and it comes up and the place goes crazy.And I had more to the demo. But you stop. You've gotten the point across, right? This is what Kubernetes is, here's how it works, and look how you do it from scratch. And I remember saying, “And that's the end of my presentation.” You need to know when to stop, you need to know when to pivot, and you need to have confidence that it's supposed to work, and if you've seen it work a couple of times, your confidence is unshaken.And when I walked off that stage, I remember someone from Red Hat was like—Clayton Coleman; that's his name—Clayton Coleman walked up to me and said, “You planned that. You planned it to fail just like that, so you can show people how to go from scratch all the way up. That was brilliant.” And I was like, “Sure. That's exactly what I did.”Corey: “Yeah, I meant to do that.” I like that approach. I found there's always things I have to plan for in demos. For example, I can never count on having solid WiFi from a conference hall. The show has to go on. It's, okay, the WiFi doesn't work. I've at one point had to give a talk where the projector just wasn't working to a bunch of students. So okay, close the laptop. We're turning this into a bunch of question-and-answer sessions, and it was one of the better talks I've ever given.But the alternative is getting stuck in how you think a talk absolutely needs to go. Now, keynotes are a little harder where everything has been scripted and choreographed and at that point, I've had multiple fallbacks for demos that I've had to switch between. And people never noticed I was doing it for that exact reason. But it takes work to look polished.Kelsey: I will tell you that the last Next keynote I gave was completely irresponsible. No dry runs, no rehearsals, no table reads, no speaker notes. And I think there were 30,000 people at that particular Next. And Diane Greene was still CEO, and I remember when marketing was like, “Yo, at least a backup recording.” I was like, “Nah, I don't have anything.”And that demo was extensive. I mean, I was building an app from scratch, starting with Postgres, adding the schema, building an app, deploying the app. And something went wrong halfway. And there's this joke that I came up with just to pass over the time, they gave me a new Chromebook to do the demo. And so, it's not mine, so none of the default settings were there, I was getting pop-ups all over the place.And I came up with this joke on the way to the conference. I was like, “You know what'd be cool? When I show off the serverless stuff, I would just copy the code from Stack Overflow. That'd be like a really cool joke to say this is what senior engineers do.” And I go to Stack Overflow and it's getting all of these pop-ups and my mouse couldn't highlight the text.So, I'm sitting there like a deer in headlights in front of all of these people and I'm looking down, and marketing is, like, “This is what… this is what we're talking about.” And so, I'm like, “Man do I have to end this thing here?” And I remember I kept trying, I kept trying, and came to me. Once the mouse finally got in there and I cleared up all the popups, I just came up with this joke. I said, “Good developers copy.” And I switched over to my terminal and I took the text from Stack Overflow and I said, “Great developers paste,” and the whole room start laughing.And I had them back. And we kept going and continued. And at the end, there was like this Google Assistant, and when it was finished, I said, “Thank you,” to the Google Assistant and it was talking back through the live system. And it said, “I got to admit, that was kind of dope.” So, I go to the back and Diane Greene walks back there—the CEO of Google Cloud—and she pats me on the shoulder. “Kelsey, that was dope.”But it was the thrill because I had as much thrill as the people watching it. So, in real-time, I was going through all these emotions. But I think people forget, the demo is supposed to convey something. The demo is supposed to tell some story. And I've seen people overdo their demos with way too much code, way too many commands, almost if they're trying to show off their expertise versus telling a story. And so, when I think about the demo, it has to complement the entire narrative. And so, sometimes you don't need as many commands, you don't need as much code. You can keep things simple and that gives you a lot more ins and outs in case something does go crazy.Corey: And I think the key takeaway here that so many people lose sight of is you have to know the material well enough that whatever happens, well, things don't always go the way I planned during the day, either, and talking through that is something that I think serves as a good example. It feels like a bit more of a challenge when you're trying to demo something that a company is trying to sell someone, “Oh, yeah, it didn't work. But that's okay.” But I'm still reminded by probably one of the best conference demo fails I've ever seen on video. One day, someone was attempting to do a talk that hit Amazon S3 and it didn't work.And the audience started shouting at him that yeah, S3 is down right now. Because that was the big day that S3 took a nap for four hours. It was one of those foundational things you'd should never stop to consider. Like, well, what if the internet doesn't work tomorrow when I'm doing my demo? That's a tough one to work around. But rough timing.Kelsey: [breathy sound]Corey: He nailed the rest of the talk, though. You keep going. That's the thing that people miss. They get stuck in the demo that isn't working, they expect the audience knows as much as they do about what's supposed to happen next. You're the one up there telling a story. People forget it's storytelling.Kelsey: Now, I will be remiss to say, I know that the demo gods have been on my side for, like, ten, maybe fifteen years solid. So, I retired from doing live demos. This is why I just don't do them anymore. I know I'm overdue as an understatement. But the thing I've learned though, is that what I found more impressive than the live demo is to be able to convey the same narratives through story alone. No slides. No demo. Nothing. But you can still make people feel where you would try to go with that live demo.And it's insanely hard, especially for technologies people have never seen before. But that's that new challenge that I kind of set up for myself. So, if you see me at a keynote and you've noticed why I've been choosing these fireside chats, it's mainly because I'm also trying to increase my ability to share narrative, technical concepts, but now in a new form. So, this new storytelling format through the fireside chat has been my substitute for the live demo, normally because I think sometimes, unless there's something really to show that people haven't seen before, the live demo isn't as powerful to me. Once the thing is kind of known… the live demo is kind of more of the same. So, I think they really work well when people literally have never seen the thing before, but outside of that, I think you can kind of move on to, like, real-life scenarios and narratives that help people understand the fundamentals and the philosophy behind the tech.Corey: An awful lot of tools and tech that we use on a day-to-day basis as well are thankfully optimized for the people using them and the ergonomics of going about your day. That is orthogonal, in my experience, to looking very impressive on stage. It's the rare company that can have a product that not only works well but also presents well. And that is something I don't tend to index on when I'm selecting a tool to do something with. So, it's always a question of how can I make this more visually entertaining? For while I got out of doing demos entirely, just because talking about things that have more staying power than a screenshot that is going to wind up being irrelevant the next week when they decide to redo the console for some service yet again.Kelsey: But you know what? That was my secret to doing software products and projects. When I was at CoreOS, we used to have these meetups we would used to do every two weeks or so. So, when we were building things like etcd, Fleet was a container management platform that came before Kubernetes, we would always run through them as a user, start install them, use them, and ask how does it feel? These command line flags, they don't feel right. This isn't a narrative you can present with the software alone.But once we could, then the meetups were that much more engaging. Like hey, have you ever tried to distribute configuration to, like, a thousand servers? It's insanely hard. Here's how you do with Puppet. But now I'm going to show you how you do with etcd. And then the narrative will kind of take care of itself because the tool was positioned behind what people would actually do with it versus what the tool could do by itself.Corey: I think that's the missing piece that most marketing doesn't seem to quite grasp is, they talk about the tool and how awesome it is, but that's why I love customer demos so much. They're showing us how they use a tool to solve a real-world problem. And honestly, from my snarky side of the world and the attendant perspective there, I can make an awful lot of fun about basically anything a company decides to show me, but put a customer on stage talking about how whatever they've built is solving a real-world problem for them, that's the point where I generally shut up and listen because I'm going to learn something about a real-world story. Because you don't generally get to tell customers to go on stage and just make up a story that makes us sound good, and have it come off with any sense of reality whatsoever. I haven't seen that one happen yet, but I'm sure it's out there somewhere.Kelsey: I don't know how many founders or people building companies listen in to your podcast, but this is right now, I think the number one problem that especially venture-backed startups have. They tend to have great technology—maybe it's based off some open-source project—with tons of users who just know how that tool works, it's just an ingredient into what they're already trying to do. But that isn't going to ever be your entire customer base. Soon, you'll deal with customers who don't understand the thing you have and they need more than technology, right? They need a product.And most of these companies struggle painting that picture. Here's what you can do with it. Or here's what you can't do now, but you will be able to do if you were to use this. And since they are missing that, a lot of these companies, they produce a lot of code, they ship a lot of open-source stuff, they raise a lot of capital, and then it just goes away, it fades out over time because they can bring on no newcomers. The people who need help the most, they don't have a narrative for them, and so therefore, they're just hoping that the people who have all the skills in the world, the early adopters, but unfortunately, those people are tend to be the ones that don't actually pay. They just kind of do it themselves. It's the people who need the most help.Corey: How do we monetize the bleeding edge of adoption? In many cases you don't. They become your community if you don't hug them to death first.Kelsey: Exactly.Corey: Ugh. None of this is easy. I really want to thank you for taking the time to catch up and talk about how you seen the remains of a career well spent, and now you're going off into that glorious sunset. But I have a sneaking suspicion you'll still be around. Where should people go if they want to follow up on what you're up to these days?Kelsey: Right now I still use… I'm going to keep calling it Twitter.Corey: I agree.Kelsey: I kind of use that for my real-time interactions. And I'm still attending conferences, doing fireside chats, and just meeting people on those conference floors. But that's what where I'll be for now. So yeah, I'll still be around, but maybe not as deep. And I'll be spending more time just doing normal life stuff, maybe less building software.Corey: And we will, of course, put a link to that in the show notes. Thank you so much for taking the time to catch up and share your reflections on how the industry is progressing.Kelsey: Awesome. Thanks for having me, Corey.Corey: Kelsey Hightower, now gloriously retired. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment that you're going to type on stage as part of a conference talk, and then accidentally typo all over yourself while you're doing it.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

Nikolay and Michael discuss a listener question — about products that take Postgres and transform it to something that decouples compute from storage (RDS Aurora, GC AlloyDB, Neon etc.) and whether they see something like this landing upstream in the medium term.Here are some links to some things they mentioned:Amazon Aurora https://aws.amazon.com/rds/aurora/Google Cloud AlloyDB for PostgreSQL https://cloud.google.com/alloydbNeon https://neon.tech/ Google Cloud Spanner https://cloud.google.com/spannerIs Aurora PostgreSQL really faster and cheaper than RDS PostgreSQL? (blog post by Avinash Vallarapu from MigOps) https://www.migops.com/blog/is-aurora-postgresql-really-faster-and-cheaper-than-rds-postgresql-benchmarking/ Deep dive on Amazon Aurora with PostgreSQL compatibility (presentation by Grant McAllister) https://www.youtube.com/watch?v=HQg8wqlxefo Intro to Aurora PostgreSQL Query Plan Management https://aws.amazon.com/blogs/database/introduction-to-aurora-postgresql-query-plan-management/ Michael Stonebraker Turing Award Lecture Interview with Stas Kelvich from Neon on Postgres TV https://www.youtube.com/watch?v=4PUKNznq_eM Interview with Ben Vandiver from Google Cloud Spanner on Postgres TV https://www.youtube.com/watch?v=BW-Uexhv-bk Timescale Cloud bottomless storage feature (data tiering to Amazon S3) https://www.timescale.com/blog/expanding-the-boundaries-of-postgresql-announcing-a-bottomless-consumption-based-object-storage-layer-built-on-amazon-s3/ Testing Database Changes the Right Way (Heap Analytics article) https://www.heap.io/blog/testing-database-changes-right-way  ~~~What did you like or not like? What should we discuss next time? Let us know via a YouTube comment, on social media, or by commenting on our Google doc!~~~Postgres FM is brought to you by:Nikolay Samokhvalov, founder of Postgres.aiMichael Christofides, founder of pgMustardWith special thanks to:Jessie Draws for the amazing artwork 

Welcome to another episode of the DevOps Toolchain podcast! In this episode, we have a special guest, Lukas Fittl, an experienced engineer and entrepreneur known for his work in optimizing Postgres performance. Lukas will share his insights and tips on tuning Postgres databases for optimal performance. We'll explore the different tuning aspects, including using managed services like Amazon RDS or Amazon Aurora, general best practices, and understanding where bottlenecks occur. Lukas will also discuss his journey in developing a comprehensive tool called pganalyze, which provides valuable insights and recommendations for optimizing Postgres performance. But that's not all! We'll also dive into the evolving role of the traditional DBA, the impact of AI on decision-making processes, and the latest trends in database indexing. Whether you're a developer, a database administrator, or just interested in learning more about Postgres performance, this episode is packed with valuable information. So grab your headphones and get ready to dive deep into Postgres performance at any scale with our guest Lukas Fittl. Let's get started!  

Welcome to the newest episode of The Cloud Pod podcast - where the forecast is always cloudy! Ryan, Jonathan, and Matt are your hosts this week as we discuss all things cloud, including updates to Terraform, pricing updates in GCP SCC, AWS Blueprint, DMS Serverless, and Snowball - as well as all the discussion on Microsoft quantum safe computing and ethical AI you could possibly want!  A big thanks to this week's sponsor: Foghorn Consulting, provides top-notch cloud and DevOps engineers to the world's most innovative companies. Initiatives stalled because you have trouble hiring?  Foghorn can be burning down your DevOps and Cloud backlogs as soon as next week.

Welcome to the newest episode of The Cloud Pod podcast! Justin, Ryan and Matthew are your hosts this week as we discuss all the latest news and announcements in the world of the cloud and AI. Do people really love Matt's Azure know-how? Can Google make Bard fit into literally everything they make? What's the latest with Azure AI and their space collaborations? Let's find out! Titles we almost went with this week: Clouds in Space, Fictional Realms of Oracles, Oh My.  The cloudpod streams lambda to the cloud A big thanks to this week's sponsor:  Foghorn Consulting, provides top-notch cloud and DevOps engineers to the world's most innovative companies. Initiatives stalled because you have trouble hiring?  Foghorn can be burning down your DevOps and Cloud backlogs as soon as next week.

On this episode of The Cloud Pod, the team discusses the new Amazon Linux 2023, Google Bard,  new features of Google Chronicle Security Operations, GPT-4 from Azure Open AI, and Oracle's Kubernetes platform comparison. They also talk about cloud-native architecture as a way to adapt applications for a pivot to the cloud. A big thanks to this week's sponsor, Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights

In this episode, Emily and Dave chat with Keyur Diwan Sr. Product Manager, Amazon RDS. Amazon Relational Database Service (Amazon RDS) is a collection of managed services that makes it simple to set up, operate, and scale databases in the cloud. Amazon RDS recently launched fully managed Blue/Green Deployments to help you with safer, simpler, and faster updates to your Amazon Aurora and Amazon RDS databases. Blue/Green Deployments create a fully managed staging environment that allows you to deploy and test production changes, keeping your current production database safe. With a single click, you can promote the staging environment to be the new production system in as fast as a minute, with no changes to your application and no data loss. Keyur walks us through this new feature, how to get started, how customers are using it today, and what's next for RDS. [BLOG] Announcing Amazon RDS Blue/Green Deployments for safer, simpler, and faster updates - https://aws.amazon.com/about-aws/whats-new/2022/11/amazon-rds-blue-green-deployments-safer-simpler-faster-updates/ [BLOG] New – Fully Managed Blue/Green Deployments in Amazon Aurora and Amazon RDS - https://aws.amazon.com/blogs/aws/new-fully-managed-blue-green-deployments-in-amazon-aurora-and-amazon-rds/ [DOCS] Amazon RDS and Aurora Documentation - https://docs.aws.amazon.com/rds/ [PORTAL] Amazon Relational Database Service (RDS) https://aws.amazon.com/rds/ [YOUTUBE] Introduction to Amazon RDS Blue/Green Deployments - https://youtu.be/mGAjzAzBOsk Origins of the term "blue-green deployment" - https://gitlab.com/-/snippets/1846041 Martin Fowler - Blue Green Deployment - https://martinfowler.com/bliki/BlueGreenDeployment.html Subscribe: Amazon Music: https://music.amazon.com/podcasts/f8bf7630-2521-4b40-be90-c46a9222c159/aws-developers-podcast Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-developers-podcast/id1574162669 Google Podcasts: https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkcy5zb3VuZGNsb3VkLmNvbS91c2Vycy9zb3VuZGNsb3VkOnVzZXJzOjk5NDM2MzU0OS9zb3VuZHMucnNz Spotify: https://open.spotify.com/show/7rQjgnBvuyr18K03tnEHBI TuneIn: https://tunein.com/podcasts/Technology-Podcasts/AWS-Developers-Podcast-p1461814/ RSS Feed: https://feeds.soundcloud

В этот раз re:Invent 2022 посетили 50 000+ человек, прошло 2000 технических сессий на которых презентовали более 100 технологических новинок - и это все только за одну неделю! По традиции в нашем подкасте мы делаем обзор того, что было анонсировано: и это не только новые сервисы, но и детальное обсуждение новых фич в уже существующих.  В одном выпуске очень трудно уместить все, поэтому в первой части мы рассмотрели: -  Amazon Security Lake -  Amazon CloudWatch Internet Monitor  -  AWS Verified Access  -  Amazon Verified permissions -  Amazon EventBridge Pipes -  Trusted Language Extensions for PostgreSQL on Amazon Aurora and Amazon RDS -   Amazon DocumentDB Elastic Clusters -   Fully Managed Blue/Green Deployments in Amazon Aurora and Amazon RDS  -   AWS Control Tower – Comprehensive Controls Management  

The Cloud Pod recaps all of the positives and negatives of Amazon ReInvent 2022, the annual conference in Las Vegas, bringing together 50,000 cloud computing professionals.  This year's keynote speakers include Adam Selpisky, CEO of Amazon Web Services, Swami Sivasubramanian, Vice President of Data and Machine Learning at AWS and Werner Vogels, Amazon's CTO.  Attendees and web viewers were treated to new features and products, such as AWS Lambda Snapstart for Java Functions, New Quicksight capabilities and quality-of-life improvements to hundreds of services.  Justin, Jonathan, Ryan, Peter and Special guest Joe Daly from the Finops foundation talk about the show and the announcements. Thank you to our sponsor, Foghorn Consulting, which provides top notch cloud and DevOps engineers to the world's most innovative companies. Initiatives stalled because you're having trouble hiring? Foghorn can be burning down your DevOps and Cloud backlogs as soon as next week. Episode Highlights ⏰ AWS Pricing Calculator now supports modernization cost estimates for Microsoft workloads. ⏰ AWS Re:Invent 2022 announcements and keynote updates. Top Quote

In this episode, Emily and Dave chat with John Dalton, Sr. Product Manager, Amazon RDS Open Source, and Jonathan Katz, Principal PMT, Amazon RDS Open Source, about the recent launch of Trusted Language Extensions PostgreSQL. Trusted Language Extensions for PostgreSQL is an open source development kit for building PostgreSQL extensions that allows developers to build high performance PostgreSQL extensions and safely run them on your RDS for PostgreSQL DB instance. By using Trusted Language Extensions (TLE) for PostgreSQL, developers can create PostgreSQL extensions that follow the documented approach for extending PostgreSQL functionality. Both Johns discuss building this new service update, and their approach to open source. [BLOG] New – Trusted Language Extensions for PostgreSQL on Amazon Aurora and Amazon RDS: https://aws.amazon.com/blogs/aws/new-trusted-language-extensions-for-postgresql-on-amazon-aurora-and-amazon-rds/ [BLOG] Announcing Trusted Language Extensions for PostgreSQL on Amazon Aurora and Amazon RDS: https://aws.amazon.com/about-aws/whats-new/2022/11/trusted-language-extensions-postgresql-amazon-aurora-rds/ [DOCS] Working with Trusted Language Extensions for PostgreSQL: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/PostgreSQL_trusted_language_extension.html [GITHUB] Open Source Framework for Building Trusted Language Extensions for PostgreSQL: https://github.com/aws/pg_tle [PORTAL] Amazon Aurora: https://aws.amazon.com/rds/aurora/ [PORTAL] Amazon RDS for PostgreSQL: https://aws.amazon.com/rds/postgresql/ Subscribe: Amazon Music: https://music.amazon.com/podcasts/f8bf7630-2521-4b40-be90-c46a9222c159/aws-developers-podcast Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-developers-podcast/id1574162669 Google Podcasts: https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkcy5zb3VuZGNsb3VkLmNvbS91c2Vycy9zb3VuZGNsb3VkOnVzZXJzOjk5NDM2MzU0OS9zb3VuZHMucnNz Spotify: https://open.spotify.com/show/7rQjgnBvuyr18K03tnEHBI TuneIn: https://tunein.com/podcasts/Technology-Podcasts/AWS-Developers-Podcast-p1461814/ RSS Feed: https://feeds.soundcloud

This week we recap the news from AWS re:Invent and discuss application vendors mandating use of specific Kubernetes distros. Plus, some thoughts on dog boarding… Watch the YouTube Live Recording of Episode 389 (https://www.youtube.com/watch?v=h8L0QEIMvOs) Runner-up Titles Everyone gets a Graviton Instance What a Boring re:Invent Part of our brand 17 Days in the Hole Under the Stars, Under the Sea Tighten it up Don't make me pay for security Secure by default That's a great message and I don't believe it Works with Lambda Security, it keeps getting better? Rundown AWS re:Invent What's New at AWS – Cloud Innovation & News - 2022 Archive (https://aws.amazon.com/about-aws/whats-new/2022/?whats-new-content-all.sort-by=item.additionalFields.postDateTime&whats-new-content-all.sort-order=desc&awsf.whats-new-analytics=*all&awsf.whats-new-app-integration=*all&awsf.whats-new-arvr=*all&awsf.whats-new-blockchain=*all&awsf.whats-new-business-applications=*all&awsf.whats-new-cloud-financial-management=*all&awsf.whats-new-compute=*all&awsf.whats-new-containers=*all&awsf.whats-new-customer-enablement=*all&awsf.whats-new-customer%20engagement=*all&awsf.whats-new-database=*all&awsf.whats-new-developer-tools=*all&awsf.whats-new-end-user-computing=*all&awsf.whats-new-mobile=*all&awsf.whats-new-gametech=*all&awsf.whats-new-iot=*all&awsf.whats-new-machine-learning=*all&awsf.whats-new-management-governance=*all&awsf.whats-new-media-services=*all&awsf.whats-new-migration-transfer=*all&awsf.whats-new-networking-content-delivery=*all&awsf.whats-new-quantum-tech=*all&awsf.whats-new-robotics=*all&awsf.whats-new-satellite=*all&awsf.whats-new-security-id-compliance=*all&awsf.whats-new-serverless=*all&awsf.whats-new-storage=*all) Compute Amazon EC2 C7g instances – Compute –Amazon Web Services (https://aws.amazon.com/ec2/instance-types/c7g/?sc_icampaign=aware_ec2-c7gn-instances_reinvent22&sc_ichannel=ha&sc_icontent=awssm-11814_aware_reinvent22&sc_iplace=ribbon&trk=1b39069e-86fc-466c-99c7-4ab2427ddb3a~ha_awssm-11814_aware_reinvent22) Announcing Amazon EC2 M6in, M6idn, R6in, and R6idn network optimized instances (https://aws.amazon.com/about-aws/whats-new/2022/11/amazon-ec2-m6in-m6idn-r6in-r6idn-network-optimized-instances/) Announcing Amazon EC2 Hpc6id instances (https://aws.amazon.com/about-aws/whats-new/2022/11/announcing-amazon-ec2-hpc6id-instances/) AWS Nitro Enclaves now supports Amazon EKS and Kubernetes (https://aws.amazon.com/about-aws/whats-new/2022/11/aws-nitro-enclaves-supports-amazoneks-kubernetes/) Introducing Finch: An Open Source Client for Container Development (https://aws.amazon.com/blogs/opensource/introducing-finch-an-open-source-client-for-container-development/) New – Accelerate Your Lambda Functions with Lambda SnapStart (https://aws.amazon.com/blogs/aws/new-accelerate-your-lambda-functions-with-lambda-snapstart/) Data Announcing Amazon Redshift integration for Apache Spark with Amazon EMR (https://aws.amazon.com/about-aws/whats-new/2022/11/amazon-redshift-integration-apache-spark-amazon-emr/) AWS announces Amazon Redshift integration for Apache Spark (https://aws.amazon.com/about-aws/whats-new/2022/11/aws-announces-amazon-redshift-integration-apache-spark/) AWS announces Amazon Aurora zero-ETL integration with Amazon Redshift (https://aws.amazon.com/about-aws/whats-new/2022/11/amazon-aurora-zero-etl-integration-redshift/) Serverless Open-Source Search Engine – Amazon OpenSearch Serverless (https://aws.amazon.com/opensearch-service/features/serverless/) Introducing AWS Glue 4.0 (https://aws.amazon.com/about-aws/whats-new/2022/11/introducing-aws-glue-4-0/) Security Introducing Amazon Security Lake (Preview) (https://aws.amazon.com/about-aws/whats-new/2022/11/amazon-security-lake-preview/) AWS co-announces release of the Open Cybersecurity Schema Framework (OCSF) (https://aws.amazon.com/blogs/security/aws-co-announces-release-of-the-open-cybersecurity-schema-framework-ocsf-project/) Amazon GuardDuty now protects Amazon Elastic Kubernetes Service clusters (https://aws.amazon.com/about-aws/whats-new/2022/01/amazon-guardduty-elastic-kubernetes-service-clusters/) Solutions AWS CEO: The cloud isn't just about technology (https://www.protocol.com/enterprise/aws-adam-selipsky-cloud) AWS Supply Chain (https://aws.amazon.com/aws-supply-chain/) AWS Clean Room (https://aws.amazon.com/clean-rooms/) Announcing AWS SimSpace Weaver (https://aws.amazon.com/about-aws/whats-new/2022/11/aws-simspace-weaver-available/) Amazon Connect announces Contact Lens agent performance evaluation forms (https://aws.amazon.com/about-aws/whats-new/2022/11/amazon-connect-contact-lens-agent-performance-evaluation-forms/) Introducing Amazon Omics (https://aws.amazon.com/about-aws/whats-new/2022/11/amazon-omics-generally-available/) Corey Quinn on re:Invent (https://twitter.com/QuinnyPig/status/1597664998234345472) Ask SDT — “using a "supported platform" list to drive cross sales.” (https://softwaredefinedtalk.slack.com/archives/C6CDLDCVB/p1669255641385689) (SDT Slack) Relevant to your Interests SigmaOS raises $4 million to build a browser for productivity nerds (https://techcrunch.com/2022/11/16/sigmaos-raises-4-million-to-build-a-browser-for-productivity-nerds/) The Distributed Computing Manifesto (https://www.allthingsdistributed.com/2022/11/amazon-1998-distributed-computing-manifesto.html) Unpacking Musk's "hardcore" marching orders (https://www.axios.com/newsletters/axios-login-3bf3c6e4-d8cd-492c-942d-c7f80719e66b.html?chunk=0&utm_term=emshare#story0) Akeyless secures a cash infusion to help companies manage their passwords, certificates and keys (https://techcrunch.com/2022/11/16/akeyless-secures-a-cash-infusion-to-help-companies-manage-their-passwords-certificates-and-keys/) Vista passes halfway mark to $20bn target for latest flagship (https://www.privateequityinternational.com/vista-passes-halfway-mark-to-20bn-target-for-latest-flagship/) 1Password Will Support Passkeys Starting in Early 2023 (https://www.macrumors.com/2022/11/17/1password-passkeys-support-2023/) Passkeys: the future of authentication in 1Password (https://www.future.1password.com/passkeys/?utm_medium=sign-in-side-panel&utm_source=1password&utm_campaign=passkeys) 10,000 Google Employees Could Be Rated as Low Performers (https://www.theinformation.com/articles/10-000-google-employees-could-be-rated-as-low-performers) Resignations Roil Twitter as Elon Musk Tries Persuading Some Workers to Stay (https://www.nytimes.com/2022/11/17/technology/twitter-elon-musk-ftc.html) Hundreds of employees say no to being part of Elon Musk's ‘extremely hardcore' Twitter (https://www.theverge.com/2022/11/17/23465274/hundreds-of-twitter-employees-resign-from-elon-musk-hardcore-deadline) Security of Passkeys in the Google Password Manager (https://security.googleblog.com/2022/10/SecurityofPasskeysintheGooglePasswordManager.html) With $8.6M in seed funding, Nx wants to take monorepos mainstream (https://techcrunch.com/2022/11/17/with-8-6m-in-seed-funding-nx-wants-to-take-monorepos-mainstream/) Facebook parent Meta winding down some non-core hardware projects (https://www.reuters.com/technology/facebook-parent-meta-winding-down-some-non-core-hardware-projects-2022-11-11/) OpenStack passes 40 million cores in production use (https://www.theregister.com/2022/11/18/openstack_thriving_survey/) A note from CEO Andy Jassy about role eliminations (https://www.aboutamazon.com/news/company-news/a-note-from-ceo-andy-jassy-about-role-eliminations) Twitter is Going Great (https://twitterisgoinggreat.com/) Building Kubernetes Applications with Acorn (https://acorn.io/building-kubernetes-applications-with-acorn/) Platforms at Kubecon 2022 (https://blog.joshgav.com/posts/kubecon-platforms-review) Zoom's looming squeeze (https://www.axios.com/newsletters/axios-login-149ea16b-be11-451a-b4de-5a1e2f8f0ce7.html?chunk=0&utm_term=emshare#story0) Sony's VR headset-console integration could limit sales, but allow depth (https://www.emergingtechbrew.com/stories/2022/11/18/sony-s-vr-headset-console-integration-could-limit-sales-but-allow-depth?utm_campaign=etb&utm_medium=newsletter&utm_source=morning_brew&mid=f642abf4dca6751d0ec109d4cbc6782e) The State of Kubernetes {Open-Source} Security | ARMO (https://www.armosec.io/blog/the-state-of-kubernetes-open-source-security/) Considerations when implementing developer portals in regulated enterprise environments (https://www.redhat.com/en/blog/considerations-when-implementing-developer-portals-regulated-enterprise-environments) Broadcom's proposed $61B VMware acquisition scrutinized by UK regulators (https://techcrunch.com/2022/11/21/broadcoms-proposed-61b-vmware-acquisition-scrutinized-by-uk-regulators/) 2023 may be the year of multicloud Kubernetes (https://www.infoworld.com/article/3679752/2023-may-be-the-year-of-multicloud-kubernetes.html?utm_source=substack&utm_medium=email) Server-side WebAssembly prepares for takeoff in 2023 (https://www.techtarget.com/searchitoperations/news/252527414/Server-side-WebAssembly-prepares-for-takeoff-in-2023?utm_source=substack&utm_medium=email) Zoom shares drop on light forecast as company faces 'heightened deal scrutiny' (https://www.cnbc.com/2022/11/21/zoom-zm-earnings-q3-2023.html?utm_source=newsletter&utm_medium=email&utm_campaign=newsletter_axioslogin&stream=top) What's coming for cloud computing in 2023 (https://www.infoworld.com/article/3680553/whats-coming-for-cloud-computing-in-2023.html) The Rise of Platform Engineering - Software Engineering Daily (https://softwareengineeringdaily.com/2020/02/13/setting-the-stage-for-platform-engineering/) IBM sues Micro Focus, claims it copied mainframe software (https://www.theregister.com/2022/11/22/ibm_sues_micro_focus_for/) How to beat the Kubernetes skills shortage (https://www.infoworld.com/article/3679749/how-to-beat-the-kubernetes-skills-shortage.html) TikTok Couldn't Ensure Accurate Responses To Government Inquiries, A ByteDance Risk Assessment Said (https://www.forbes.com/sites/emilybaker-white/2022/11/28/tiktok-inaccurate-government-inquiries-internal-bytedance-risk-assessment/?sh=7f57dc9723fe) Exclusive: Sam Bankman-Fried says he's down to $100,000 (https://www.axios.com/2022/11/29/sam-bankman-fried-100000-ftx-cftc-regulation?utm_source=newsletter&utm_medium=email&utm_campaign=newsletter_axiosprorata&stream=top) Why Big Tech is not rushing to clone Twitter (https://www.axios.com/newsletters/axios-login-1cea6d1a-1428-448d-b0d3-5da3ae9425ef.html?chunk=0&utm_term=emshare#story0) Amazon Alexa is a “colossal failure,” on pace to lose $10 billion this year (https://arstechnica.com/gadgets/2022/11/amazon-alexa-is-a-colossal-failure-on-pace-to-lose-10-billion-this-year/) I analyzed 290 booths at KubeCon - here are the DevOps trends for 2023 (https://www.uptime.build/post/i-analyzed-290-booths-at-kubecon-here-are-the-devops-trends-for-2023?utm_source=substack&utm_medium=email) Nonsense Billionaires like Elon Musk want to save civilization by having tons of genetically superior kids. Inside the movement to take 'control of human evolution.' (https://www.businessinsider.com/pronatalism-elon-musk-simone-malcolm-collins-underpopulation-breeding-tech-2022-11) Australia: How 'bin chickens' learnt to wash poisonous cane toads (https://www.bbc.com/news/world-australia-63699884) A 12,000 lb. metal sculpture of Elon Musk's head on a goat body riding a rocket parked outside Tesla HQ failed to elicit a response from the billionaire (https://www.businessinsider.com/elon-musk-head-on-goat-body-riding-a-rocket-sculpture-2022-11) The leap second's time will be up in 2035—and tech companies are thrilled (https://www.popsci.com/technology/bipm-abandon-leap-second/) Conferences THAT Conference Texas Speakers and Schedule (https://that.us/events/tx/2023/schedule/). Jan 15th-18th use code SDT for 5% off CloudNativeSecurityCon North America (https://events.linuxfoundation.org/cloudnativesecuritycon-north-america/), Seattle, Feb 1 – 2, 2023 DevOpsDays Birmingham, AL 2023 (https://devopsdays.org/events/2023-birmingham-al/welcome/), April 20 - 21, 2023 Listener Feedback Sudesh shared a list of Tech Companies Hiring (https://airtable.com/shrAPDHg8apj4mnRR/tbl6Kz4KeeCp3HrSM) Send “End of Year” listener questions to questions@softwaredefinedtalk.com (mailto:questions@softwaredefinedtalk.com). SDT news & hype Join us in Slack (http://www.softwaredefinedtalk.com/slack). Get a SDT Sticker! Send your postal address to stickers@softwaredefinedtalk.com (mailto:stickers@softwaredefinedtalk.com) and we will send you free laptop stickers! Follow us on Twitch (https://www.twitch.tv/sdtpodcast), Twitter (https://twitter.com/softwaredeftalk), Instagram (https://www.instagram.com/softwaredefinedtalk/), LinkedIn (https://www.linkedin.com/company/software-defined-talk/) and YouTube (https://www.youtube.com/channel/UCi3OJPV6h9tp-hbsGBLGsDQ/featured). Use the code SDT to get $20 off Coté's book, Digital WTF (https://leanpub.com/digitalwtf/c/sdt), so $5 total. Become a sponsor of Software Defined Talk (https://www.softwaredefinedtalk.com/ads)! Recommendations Brandon: The Complete History & Strategy of Qualcomm (https://www.acquired.fm/episodes/qualcomm) Matt: Kishi Bashi This Must Be The Place (https://www.youtube.com/watch?v=IslMHJFkIME) Carma (https://carma.com.au) car purchase: referral code: REF22-872E Photo Credits Header (https://unsplash.com/photos/K8i-gRJHT_0) CoverArt (https://twitter.com/DevchicaJasmin/status/1597874321510526978)

Cloud Posse holds public "Office Hours" every Wednesday at 11:30am PST to answer questions on all things related to DevOps, Terraform, Kubernetes, CICD. Basically, it's like an interactive "Lunch & Learn" session where we get together for about an hour and talk shop. These are totally free and just an opportunity to ask us (or our community of experts) any questions you may have. You can register here: https://cloudposse.com/office-hoursJoin the conversation: https://slack.cloudposse.com/Find out how we can help your company:https://cloudposse.com/quizhttps://cloudposse.com/accelerate/Learn more about Cloud Posse:https://cloudposse.comhttps://github.com/cloudpossehttps://sweetops.com/https://newsletter.cloudposse.comhttps://podcast.cloudposse.com/[00:00:00] Intro[00:01:37] Terraform Provider Lint Toolhttps://github.com/bflad/tfproviderlint[00:02:49] Validates AWS IAM Policies in a Terraform HCL AWS IAM best practiceshttps://github.com/awslabs/terraform-iam-policy-validator[00:03:49] AWS re:Invent Highlights?https://aws.amazon.com/blogs/aws/top-announcements-of-aws-reinvent-2022/AWS Config rules now support proactive compliancehttps://aws.amazon.com/about-aws/whats-new/2022/11/aws-config-rules-support-proactive-compliance/Fully Managed Blue/Green Deployments in Amazon Aurora and Amazon RDShttps://aws.amazon.com/blogs/aws/new-fully-managed-blue-green-deployments-in-amazon-aurora-and-amazon-rds/Amazon CloudFront launches continuous deployment supporthttps://aws.amazon.com/about-aws/whats-new/2022/11/amazon-cloudfront-continuous-deployment-support/Accelerate Your Lambda Functions with Lambda SnapStarthttps://aws.amazon.com/blogs/aws/new-accelerate-your-lambda-functions-with-lambda-snapstart/Introducing Amazon Security Lake (Preview)https://aws.amazon.com/about-aws/whats-new/2022/11/amazon-security-lake-preview/Introducing VPC Lattice – Simplify Networking for Service-to-Service Communication (Preview)https://aws.amazon.com/blogs/aws/introducing-vpc-lattice-simplify-networking-for-service-to-service-communication-preview/Announcing Amazon OpenSearch Serverless (Preview)https://aws.amazon.com/about-aws/whats-new/2022/11/announcing-amazon-opensearch-serverless-preview/AWS announces lower latencies for Amazon Elastic File Systemhttps://aws.amazon.com/about-aws/whats-new/2022/11/aws-announces-lower-latencies-amazon-elastic-file-system/Verified Permissions https://aws.amazon.com/verified-permissions/[00:57:54]  What do you think of AWS KMS External Key Store announcement, and what are some of the use-cases you can think of?[01:01:31]  Outro#officehours,#cloudposse,#sweetops,#devops,#sre,#terraform,#kubernetes,#awsSupport the show

On The Cloud Pod this week, Amazon announces Neptune Serverless, Google introduces Google Blockchain Node Engine, and we get some cost management updates from Microsoft. Thank you to our sponsor, Foghorn Consulting, which provides top notch cloud and DevOps engineers to the world's most innovative companies. Initiatives stalled because you're having trouble hiring? Foghorn can be burning down your DevOps and Cloud backlogs as soon as next week. General News [1:24]

Amazon Aurora PostgreSQL-Compatible Edition now supports PostgreSQL major version 14 (14.3). Let us discuss this news. https://aws.amazon.com/about-aws/whats-new/2022/06/amazon-aurora-supports-postgresql-14/ 0:00 Intro 1:00 Database on the Cloud options 3:45 Amazon Aurora supports Postgres 14 6:00 Postgres 14 vs Postgres 13 --- Support this podcast: https://anchor.fm/hnasr/support

Stephanie Wong and Debi Cabrera are learning all about BigLake from guests Gaurav Saxena and Justin Levandoski of the BigQuery team. BigLake offers unified data management from both data warehouses and data lakes. What exactly is the difference between a data warehouse and a data lake? Justin explains what a data lake is, how they came to be, and the benefits. Each data option has its cons too, like the limitations of data lakes for enterprise use. Enter BigLake built on BigQuery, which helps enterprise clients manage and analyze their data from both data warehouses and data lakes. The best features of BigQuery are now available for Google Cloud Storage and across multi-cloud solutions. Guarav describes BigLake behind the scenes and how the principles of BigQuery's data management can now be used for open file formats in BigLake. It's BigQuery for more data formats, Justin explains. BigLake solves many data problems quickly with a special emphasis on improving security. Our guests talk specifically about clients who gain the most from using BigLake, especially those looking to analyze distributed data and those who need easy and fast security and compliance solutions. With tightened security, BigLake offers access delegation and secure APIs that work over object storage. We hear about the user experience and how easy it is to get started, especially for customers already familiar with and using other GCP products. Google's advocacy of open source projects means many clients are coming in with workloads built with open source software. BigLake supports multi-cloud projects so that tables can be built on top of any data system. No matter the format of your data, you can run analytics with BigLake. We talk more about the security features of BigLake and how easy it is to unify data warehouses and data lakes with optimal data security. The customers have helped shape BigLake, and Gaurav describes how these clients are using this data software. We hear about integration with BigQuery Omni and Dataplex and how BigLake is different. In the future, Google will continue to make simple, effective solutions for data management and analytics, building further off of BigQuery. Gaurav Saxena Gaurav Saxena is a product management lead at Google BigQuery. He has 12+ years of experience building products at the intersection of cloud, data and AI. Before Google, Gaurav led product management at Microsoft Azure and Amazon Web Services for some of the most widely used cloud offerings in storage and data. Justin Levandoski Justin is a tech lead/manager in BigQuery leading BigLake and other projects pushing the frontier of BigQuery. Prior to Google, just worked on Amazon Aurora and was part of the Database research group at Microsoft Research. Cool things of the week Your ultimate guide to Speech on Google Cloud blog Announcing the Climate Innovation Challenge—grants to support cutting-edge earth research blog Interview BigLake site BigQuery site Cloud Storage site Spark site Apache Ranger site BigQuery Omni docs Apache Iceberg site Delta Lake site Presto site TensorFlow site Dataplex site What's something cool you're working on? Debi is working on a series about automatic DLP. Cloud Data Loss Prevention is now automatic and allows you to scan data across your whole org with the click of one button! Hosts Stephanie Wong and Debi Cabrera

On The Cloud Pod this week, the team discusses Peter's concept of fun. Plus digital adventures with AWS Cloud Quest game, much-wanted Google price increases, and a labyrinthine run-through of the details of Azure Health Data Services. A big thanks to this week's sponsor, Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights

About EmilyEmily is an Android engineer by day, but makes tech jokes and satires videos by night. She lives in San Francisco with two ridiculously fluffy dogs.Links: Uber: https://eng.uber.com/ Blog: https://www.emilykager.com/ Twitter: https://twitter.com/EmilyKager TikTok: https://www.tiktok.com/@shmemmmy TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Couchbase Capella Database-as-a-Service is flexible, full-featured and fully managed with built in access via key-value, SQL, and full-text search. Flexible JSON documents aligned to your applications and workloads. Build faster with blazing fast in-memory performance and automated replication and scaling while reducing cost. Capella has the best price performance of any fully managed document database. Visit couchbase.com/screaminginthecloud to try Capella today for free and be up and running in three minutes with no credit card required. Couchbase Capella: make your data sing.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance query accelerator for the Oracle MySQL Database Service, although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLAP and OLTP—don't ask me to pronounce those acronyms again—workloads directly from your MySQL database and eliminate the time-consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Today's episode is a little bit off of the beaten path because, you know, normally we talk to folks doing things in the world of cloud. What is cloud, you ask? Great question. Whatever someone's trying to sell you that day happens to be cloud.But it usually looks like SaaS products, Platform as a Service products, Infrastructure as a Service products, with ridiculous names because no one ever really thought what that might look like to pronounce out loud. But today, we're going in a completely different direction. My guest is Emily Kager, a senior Android engineer at a small scrappy startup called Uber. Emily, thank you for joining me.Emily: Thanks for having me.Corey: So, I'm going to outright come out and say it I know remarkably little about, I don't even want to say the mobile ecosystem in general, but even Android specifically because I fell down the iPhone hole a long time ago, and platform lock-in is a very real thing. Whenever you start talking about technical things, that generally tends to sail completely past me. You're talking about things like Promises and whatnot. And it's like, oh, that sounds suspiciously close to JavaScript, a language that I cannot make sense of to save my life. And it's clear you know an awful lot about what you're doing. It's also clear, I don't know, a whole heck of a lot about that side of the universe.Emily: Well, that's good because I don't know much about the cloud.Corey: Exactly. Which sounds like well, we don't have a whole lot of points of commonality to have a show on, except for this small little thing, where recently, I decided in an attempt to recapture my lost youth and instead wound up feeling older than I ever have before, I joined the TikToks and started making small videos that I would consider humorous, but almost no one else will. And okay, great. I give it a hearty, sensible chuckle and move on, and then I start scrolling to see what else is out there. And I started encountering you, kind of a lot.And oh, my God, this is content that it's relatable, it is educational, dare I say, and most of all, it's engaging without being overbearing. And this is a new type of content creation that I hadn't really spent a lot of time with before. So, I want to talk to you about that.Emily: Awesome. I want to apologize for having to see my face as you're just scrolling throughout your day, but happy to chat about it. [laugh].Corey: No, no, it's—compared to some of the things I wind up on the TikTok algorithm, it is ridiculous. I think it's about 80% confident that I'm a lesbian for some Godforsaken reason. Which hey, power to the people. I don't think I qualify, but you know, that's just how it works. And what I found really interesting about it, what does tie it back to the world of cloud, is that a recurring theme of this show has been, since the beginning, where does the next generation of cloud-engineering-type come from?Because I've been in this space, almost 20 years, and it turns out that my path of working to help desk until you realize that you like the computers, but not so much being screamed at by the general public, then go find a unicorn job somewhere you can bluff your way into because the technical interviewer is out sick that day, and so on and so forth, isn't really a path that is A) repeatable by a whole lot of people, and B) something that exists anymore. So, how do people who are just entering the workforce now or transitioning into tech from other fields learn about this stuff? And we've had a bunch of people talking about approaches to educating people on these sorts of things, but I don't think I've ever spoken to someone who's been as effective at it in minute or less long videos as you are.Emily: That's super kind. Yeah, I think there's actually a whole discussion and joke set on TikTok of people's parents suggesting why don't you just go slide your resume under the CEOs door? Like, why don't you just go get a job [laugh] that way? I think the realities of—what year are we in? 2022? [laugh]—Corey: All year long, I'm told.Emily: Yeah, [laugh] yeah. Yeah. I think that's not going to be the reality anymore, right? You can't just go shake hands with the CEO and work your way up from the mailroom and yeah, that's not the way anymore. So yeah, I think I, you know, started just putting some feelers out, making educational content mostly about my own experiences as a change career person in the tech world.I have some, I would say interesting perspectives on how to enter the industry, you know, either through undergrad or after undergrad, so. And it's done really well. I think people are really interested in tech is a career at this point. Like, it's kind of well known that they're good jobs, well paid, and, you know, pretty, like, good work-life balance, most of the time. So yeah, the youth are interested.Corey: It's something that offers a path forward that lends itself to folks with less traditional backgrounds. For example, you have a master's degree; I have an eighth-grade education on paper. And, yes, I'm proof-positive that it is possible to get into this space and, by some definitions, excel in it without having a degree, but let's also be clear, here, I have the winds of privilege at my back, and I was stupendously lucky. It is harder to do without the credential than it is with the credential.Emily: Yep.Corey: But the credential is not required in the same way that it is if I want to be a surgeon. Yeah, you're going to spend a lot of time in either school or prison with that approach. So, you have really two paths there; one is preferable over the other. Tech, it feels like there's always more than one way to get in. And there's always, it seems, as many stories as there are people out there about how they wound up approaching their own path to it. What was yours?Emily: Yeah. First of all, it's funny, you mentioned surgeons because I actually just today saw on my ‘For You' page some surgeons sharing, you know, their own suturing techniques. And I think it's a really interesting platform even, you know, within different fields and different subsets to kind of share information and keep up to date and connect with people in your own industry. So, beyond learning how to get into [laugh] an industry, it can also be helpful for other things. But sorry, I completely forgot the original question. How—what was my path? Is that what the question was?Corey: Yeah. How did you get here is always a good question. It's the origin stories that we sometimes tell, sometimes we wind up occluding aspects of it. But I find it's helpful to tell these stories just because, if nothing else, it reaffirms to folks who are watching or listening or reading depending on how they want to consume this, that when they feel like well, I tried to get a credential and didn't succeed, or I applied for a job and didn't get it, there are other paths. There is not only one way to get there.Emily: Yeah. And I think it's also super important to talk about failures that we've had, right? So, when I was in undergrad, I was studying neuroscience and I was pre-med. And I thought I wanted to go to med school, kind of decided halfway through, I was only lukewarm about it, and I don't think med school is the type of thing that you want to feel lukewarm about as you're [laugh] approaching, you know, hundreds of thousands of dollars of debt and a ten-plus year commitment to schooling and whatever else, right? So yeah, I felt very lukewarm about the whole thing.Both my parents were doctors, so I just didn't really have exposure to many other careers or job options. I'm from a pretty, like, rural area, so tech had never really [laugh] occurred to me either. So yeah, then I decided to just take a year off after undergrad, felt super lost. I think when you're 22, everything feels so important, [laugh] and you look at everyone else who already has their first job at 22, and I was like, “Wow, I'm a huge failure. I'm never going to have a job.” Which is, you know, hilarious looking back because 22-year-olds are so young. And yeah, just decided to take a year off. I worked at a nonprofit. I hated it, hated the work. Decided, like I, you know, can never do this forever.Corey: I can't do nonprofit stuff. I'm going to do for-profit stuff. And it turns out that most—when you say nonprofit, it doesn't mean what I thought. It ap—usually means, you know, something that's dedicated to a charitable cause, not, you know, a VC-backed company that doesn't know how to make any money.Emily: Yeah. I mean, it could still be very corporate at nonprofit. After that, actually—Corey: Oh, yes. Money is the root of all good as well as evil.Emily: Yeah. And I actually had a task at the nonprofit where I was sorting a ton of things in spreadsheets. And I was like, wow, it'd be easy if there was just, like, some program I could write to, like, do this. So, I actually reached out to my brother, who was a computer science nerd—affectionately—and he helped me write some, like, Excel macros, and I was like, “This is so cool.” And I ended up taking a free course, CS50, which is great, by the way, great course, super high quality from Harvard, totally free to take online.And really liked it, so I did something a little crazy and decided to just dive right in. [laugh]. And I applied to a post-bacc program to kind of take all the courses that a CS undergrad would have taken just after. And that post-bacc turned into a master's program.Corey: And here you are now on the other side of having done it. If—sort of the dangerous questions: If you had known then what you know now, would you have gone down the same path, or would you have done something different to get into the space?Emily: Yeah, I mean, I think it's hard once you've kind of made it, to be like, “I would change all this.” I think I would probably try more things in undergrad. That would be the real answer to that. It obviously would have been a lot easier and more time-efficient if I didn't have to go back to school and do something. But that being said, I don't think that getting a post-bacc or a Master's is the only way into tech; it was just my path.And I try not to… I try not to promote other paths that I don't really know much about independently, right? So—on me. So—but plenty of people are successful going through boot camps or self-teaching, even, I think they're just much more difficult paths because the reality is, like, having a degree is still definitely an easier path when you show up to an interview and you can just kind of show your piece of paper, which, for better or worse, that's the reality sometimes.Corey: My wife's a corporate attorney, so I've been law adjacent for over a decade now, and one of the things that always struck me about that field is the big law approach is you go to a top-tier law school, you wind up putting your nose to the grindstone for all three years, and you hope to get an offer at one of the big law firms. And they all keep their salaries in lockstep. I think right now they're all—they just upgraded again to $235,000 a year starting. And if you don't get one of those rare, prestigious jobs at a number of select firms, it's almost a bimodal distribution where you're making somewhere between 60 and $80,000 a year to start somewhere else. It is the one path to make big money in law as you're fresh out of school, and there are no real do-overs in most cases.So, it's easy to apply that type of thinking to tech, and it's just not true. Talking to folks who have this dream of working at Google and they finally go through the interview process. And it turns out that oh no, they froze when asked to solve Fizz Buzz, or invert a binary tree on a whiteboard, or whatever ridiculous brainteaser question they're being asked, and, “Oh, no, my life is over.” And it's, you know, you can go to, I don't know, Stripe, two blocks down the street and try again. And if that doesn't work, Microsoft, or Amazon, or go down the entire list of tech companies you've heard of and haven't heard of, and they all compensate directionally the same way. It's not a one-shot, ‘this is it' moment in the same way. And I—Emily: Yeah.Corey: —I think that's a unique thing to tech right now.Emily: Yeah, definitely. And I think a lot of kids—I say kids, but really, like, you know, 18 to 20-year-olds—Corey: Oh, believe me, after being on TikTok for a couple of weeks, let me say that every one of you are children, to my perspective. I am now Grandpa Quinn over here.Emily: [laugh]. I'll take it. Yeah, but a lot of them have reached out like, “I didn't get hired at FAANG right out of school. Is my life over? Is my career over?” And I've never worked at a FAANG. [laugh]. I'm pretty happy. I definitely think I have a successful career, and I almost think I'm better for not having gone right into it, you know?I think it can be great for some people. There's great, you know… definitely great salaries, great mentorship options, but it's not the only option. And I think maybe tech is unique in that way, but there's just so many good companies to work at, and so many great opportunities, you really don't need to go to the name brand in the same way that maybe you would have to in law. It's funny you say that because my partner is also a lawyer [laugh] and [crosstalk 00:13:00]—Corey: Oh, dear. We should start a support group of our own, on some level.Emily: I know, yeah. He just went through the whole big law recruiting thing. So, I know much about that. [laugh].Corey: It's always an experience. The way that I have found across the board as well is there's also a shared, I guess, esprit de corps almost across the industry. I mean, you are on the Android side of the world, and I historically was on the DevOps side of the universe, although now mocking cloud services—but not the way test engineers say when they use the term ‘mocking'—is what I do. But there are shared experiences that tie us together, and that's part of what I found so interesting about a lot of your content.Because yes, there is some of the deep dive stuff into Android and, cool, sails right over my head—I hear the whistling sound vaguely as it goes over—but then there's other stories about things that are unique—that are, I guess, a shared experience. For me, one of the things that tied all of tech together, regardless of where in the ecosystem you fit in, is a shared sense of being utterly intimidated to hell by the miracle of Git, where it's like, Git's entire superpower is making you feel dumb. Doesn't matter who you are, from someone who doesn't know what Git is all the way to Linus himself. Someone is go—at some point, you're going to look at it and wonder, “What the hell is going on?” It's just a question of how far you get along the path before it changes your understanding of the universe.And I wound up starting to give talks, in the before times, at front-end conferences about this, which you want to talk about dispiriting things. I would build slides like, you know, a DevOps person would: Black Helvetica text on a white slide. Everyone else has these beautifully pristine, great slides. I have 20 minutes to go.How can I fix it? Change the font to Comic Sans because if you're going to have something that looks crappy, make it look like it was intentionally so.Emily: And did it work?Corey: Oh, it worked swimmingly. It was fantastic. I like the idea of being able to reach people in different areas, no matter where they are in their journey, and one of the things that appeals to me about TikTok in general in your content in particular, is it seems like we have something of a shared perspective on, getting people's attention is required in order to teach them something, and I think we both use the same vehicle for that, which is humor.Emily: Yeah, I would agree. I think the other interesting thing I just wanted to touch on; you were talking about is, we don't really know too much about each other's fields in tech. And I think when you're talking to a younger audience, maybe who you want to get interested in tech, it's really hard to communicate all the different avenues into tech that they can take. And this is something that I'm still struggling with because I know my experience as an Android developer, a mobile developer, I probably medium I understand, you know, back end development, but I don't think I could explain to a college student why or what even is, [laugh] you know, cloud development and how they could get involved in that, or all these other fields that I just really don't know much about. And I think that's kind of what ties a lot of people in tech together as well, right? Because we know our little corners of the world, and you have to start to get comfortable with the things that you don't know. And I think that's really hard to explain to [laugh] the younger generation as you're trying to get them excited about things.Corey: Oh, yeah. And the reality, too, of what we tell people and how the world works is radically different. Like, I want to learn a technology that will absolutely last for an entire career and then some, and I want to be able to be employed anytime, anywhere, at any company. The easy slam dunk answer that I think will not change in either of our lifetimes is Microsoft Excel. It powers the world.People think I'm kidding, but it is the IDE of back-office processes and communications. If Excel were to go away or even worse, Microsoft were to change Excel's interface, people would be storming Redmond by noon.Emily: Yeah, I believe it. Yeah, you know, it's interesting, right? Like, it's hard to tell people—because people will tell to me, “Well, do you have to keep learning things?” And I'm like, “Yeah. You got to keep learning things, like, all the time.”But I don't think that should be, you know, a deterrent from the career; it's just a reality. But to try to manage, like, the fears a lot of people have coming into tech and also encouraging them to still, you know, try it, go after it, I think that's something I struggle with when I'm creating my content for—towards, like, younger people. [laugh].Corey: Today's episode is brought to you in part by our friends at MinIO the high-performance Kubernetes native object store that's built for the multi-cloud, creating a consistent data storage layer for your public cloud instances, your private cloud instances, and even your edge instances, depending upon what the heck you're defining those as, which depends probably on where you work. It's getting that unified is one of the greatest challenges facing developers and architects today. It requires S3 compatibility, enterprise-grade security and resiliency, the speed to run any workload, and the footprint to run anywhere, and that's exactly what MinIO offers. With superb read speeds in excess of 360 gigs and 100 megabyte binary that doesn't eat all the data you've gotten on the system, it's exactly what you've been looking for. Check it out today at min.io/download, and see for yourself. That's min.io/download, and be sure to tell them that I sent you.Corey: Something I found on Twitter is that among other things that Twitter has going on for it, it doesn't do nuance, it does, effectively, things that are black and white, yes or no, it's always a binary in many respects. And one of those is that, like, should—like, is passion or requirement for working in tech. And there's the, “Yes, you absolutely have to be passionate for this and power through it.” And the answer, “No, you don't need to be passionate about it's okay to do it for the money and not kill yourself working 20 hours a day.” And from my perspective, I take a more moderate stance, which is how you get both sides of that argument to hate you, but it's, I don't think you need to have this all-consuming drive for tech, but I do think you need to like it.Emily: Oh yeah.Corey: I think you need to enjoy what you're doing or it's going to feel like unmitigated toil and misery, and you will not be happy in the space. And if you're not happy, really is the rest of it all worth it?Emily: I think that applies to most careers, though, right? Like that—definitely, when I was looking to switch careers, that was the main thing I was looking for. Number one was like, you know, pretty solid salary. And number two was, do I just not hate it? [laugh]. And I think if you're doing anything and you hate it, you're going to be miserable, right?Like, even if you're doing it to make a paycheck if you actually hate every single day when you wake up in the morning and you dread, you know, going to bed because the next morning, you have to wake up and do it again, like, you're going to be miserable. But I do think, yeah, like, to your point, there's a middle ground in all this, right? You don't have to dream about tech, but I think you do have to realize that, yeah, if you're going to be in this industry for decades, you're going to have to be able to learn and be interested enough in things that, you know, learning isn't a huge slog either. So.Corey: I've never understood the folks who don't want to learn as they go through their career because it just seems like a recipe to do the same thing every year for 40 years, and then you retire with what 40 years of experience—one year experience repeated 40 times. It's a… any technology or any disruption change happens, and suddenly you're in a very uncomfortable situation when we're talking about knowledge workers.Emily: Yeah, I think people—you know, I think we talk a lot about, like, imposter syndrome in our industry right? So, I think people already feel like maybe, “I don't know anything so why would I put myself out there and learn new things?” I mean, I definitely sometimes struggle with this where I'm like, “I'm very comfortable [laugh] in, like, what I do day-to-day. I know what I'm doing.” So yeah, when you have to learn, like, a totally new language or new architecture, whatever, it can feel very overwhelming to be like, wow, I actually am, you know, super stupid. [laugh]. But it's just new things, right? You're learning new things, and—Corey: Like, “Find the imposter. Oh, no, it's me.” Yes, it's a consistent problem.Emily: But it's a really powerful thing to acknowledge that you can feel stupid and you can ask questions and you can be new to something, and that's, like, totally valid. And I started taking a new language course a year or two ago, and showing up every day and speaking a new language and feeling like an idiot, it was actually super empowering because everyone in the class is doing it, you know? We didn't know the language and we were just, you know, talking gibberish to each other, and that's fine. We were learning.Corey: The emotional highs and lows are also—they hit quickly. I have never felt smarter or dumber in a two-minute span of each other than when working on technology. It's one of those, “I will never understand how this works—oh my God, it works. I'm a genius. Just kidding. It doesn't work. Nevermind. Forget everything I just said.” It's a real emotional roller coaster.Emily: [laugh]. There's only two ends of the spectrum, right? Like, there's no middle ground in this situation. It's, “I'm a genius,” or, “I should quit and never work on technology ever again.”Corey: So, I've been experimenting on TikTok a bit and you've been on it significantly longer. You have, as of this recording, something in the direction of 65,000 followers on the TikToks. I have a bit more than that on the Twitters, which only took me a brief 14 years to do. So, great. I've noticed that as I wind up—as you hit certain inflection points on Twitter, your experience definitely changes, when—as far as just, like, the unfortunate comments coming out of the woodwork.Like, I was making fun of LinkedIn at some point, and then there was some troll comment in the comments, and I looked at who the commenter was and it was the official LinkedIn brand account. And okay, well, that's novel, but all right. I'd like to add them to my professional network on TikTok. So, there we go. But have you noticed inflection points as well, in your—experience changes on the platform as you continue to grow?Emily: Yeah. I think—I saw something once that Twitter is only fun if you have less than, like, [laugh] 5000 followers or something. So, I think we both surpassed that a while ago. And yeah, I think it can be a very interesting experience as you start to gain followers. And to be honest, like, I'm on both platforms, just to kind of make content.It's a very, like, creative outlet for me. I don't necessarily care that much about how many followers I have. But it is an interesting progression to see, like, you know, you get a little bit of engagement, and it's usually, like, a back and forth; you're kind of like actually connecting to people, and then as you kind of surpass maybe five or ten-thousand followers, there's all these people who come in who you don't know who they are, they don't know who you are, they make assumptions about you, they are saying really mean things that I think just because you have, like, a high follower account that they're like, “I can say whatever I want to this person.” And it's definitely an interesting change. I think over the years—because I've been fairly public for a number of years now—you kind of get more immune to it. I'm sure you feel the same way, but you're like, whatever, just kind of brush off a lot of these things. But—Corey: Oh, yeah. You become more of a persona to people than an actual person.Emily: Yeah.Corey: And that is—Emily: Yeah.Corey: —people forget that—you know, everyone yells at you about, “That was an unkind thing, express more empathy all the ti”—I mean, you get that all the time when you get—when set a slight foot wrong. And they're right—don't think I'm saying otherwise—but they're not expressing a lot of empathy for you at the same time, either. So, it's one of those you have to disengage and disconnect on certain levels and just start to ignore it. But it's been a wild ride.Emily: I used to wonder, I used to see, like, accounts that have you know, 50, 60,000 followers on Twitter back when I was a smaller account, and they didn't—they never tweeted, and I was like, “How'd they get so many followers? They never tweet.” And now I understand. It's that they gained that many followers and then they left. [laugh]. They're done.Corey: [unintelligible 00:23:18] like, “This platform sucks now.” And it's—a lot of folks, like, “Oh, Twitter's not as good as it used to be.” It's like, well hang on. Has the platform itself changed or has your exposure to it changed? And it's a question that doesn't really have a great answer or way to find out, but it's… it's been a—it's an ongoing struggle for folks. And I do have empathy for that. I try to avoid getting involved in pile-ons wherever possible.Emily: Yeah. That's been a new change for me, too. I think a lot of my early brand on Twitter—as dumb as that word is—was, you know, kind of finding, like, misogynists in tech and really, like, creating a pile-on on them. And, you know, I think there is a space for calling out bad behavior in the industry, but you want to be careful because really, there are other people on the other side of the screen. And unless someone's really implying—like, unless they're really intending ill intent, you know, I think I've kind of now moved less towards that type of [laugh] pile-on. It is fun though. That's the thing. It's fun.Corey: Plus the algorithm rewards engagement. Say horrifying things and get a bunch of attention and more followers. But you don't necessarily want to participate in that.Emily: Yeah, exactly. And that's the other thing I realized that if someone is really saying something stupid, me bringing attention to it is only going to amplify it more. So. Especially as you gain followers and you have more of an audience to whatever you quote, tweet, or retweet, or comment on, right? So.Corey: As I look at, like, the sheer amount of content that you've put out—it's weird because if someone asked me this question, I don't know that I would have a good answer, but I am curious. You are consistently exploring new boundaries in terms of the humor, the content, the topics, the rest. How do you come up with it?Emily: This is going to be a really unsatisfying answer. [laugh]. I don't know. [laugh]. I'm a runner, and a lot of times when I'm running I don't use headphones. A lot of people say I'm sociopathic because I just am by myself in the world, and—this is such, like, a weird answer—but yeah, I just kind of—I'm thinking about things, usually I'm like digesting my day, things that happened, things that were annoying.And to be honest, I think it's pretty easy to identify things that are relatable, right? So, a lot of the gripes that all engineers have, right? So, you're like, “Wow, it was really annoying that I had to make a ticket in Jira today.” And you can kind of think about how is it annoying, and how can I make this funny and relatable to someone else? So—and to be hon—like, when I had, you know, a group of coworkers that I worked really closely in my last job, I would just send them the jokes, and then if they thought it was funny, I would just, like, post it on Twitter.And that's kind of… you know, it's just, like, the basic chit-chat that you do. But now we're all remote, so I found an outlet through Twitter and TikTok, where I would just express all my, you know, stupid engineering jokes to the world. [laugh]. Whether they want it or not.Corey: Something I found is that—and it always has frustrated me, and I figured, one day, I too, would figure out how to solve for this. And no. There are things I will tweet out that I think are screamingly funny and hilarious, and no one cares. Conversely, I'll jot off something right before I dive into a meeting, and I'll come back and find out it's gone around the internet three times. And there seems to be no rhyme or reason to it, other than that my sense of humor is not quite dialed into exactly where most folks in this industries are. It's close enough that could be overlooked, but I still feel like the best jokes go unappreciated.Emily: Oh, I agree. I mean, I send jokes by friends all the time that I'm like, “I'm posting this,” and it gets, like, you know, 20 likes. And I don't even care. I think, you know—I think that's the—you know can—you start to learn as a content creator that you're like, “I'm going to put out the content that I want to put out and hope other people find it funny, but at the end of the day, I don't really care.” So, I'm laughing at my own jokes. I'll admit that. So. I think they're funny. My—Corey: [crosstalk 00:26:58]—Emily: —[crosstalk 00:26:58] funny, too.Corey: —for me because if—I'm keeping myself engaged, otherwise it gets boring, and I lose interest in the sound of my own voice, which is just a terrible sin for me. So, it's—I have to keep it engaging or I'll lose interest.Emily: Yeah, exactly.Corey: Do you find when you're trying to put together content, that—for TikTok, for example—that you've come up with something that, “Huh, this doesn't really fit the video format. Maybe it's more of a blog post or something else.” Do you find that one content venue feeds another? Do you reuse content across multiple platforms? And if so—Emily: Yeah.Corey: —what have you learned from all that?Emily: That's an interesting question. I think—I do maintain a blog, but I don't post so often on it, and I find that the—for the more serious content I'm making that's not jokes, right? I think TikTok just really hits a different audience. Like, people don't find my blog, it's not discoverable, maybe they're not checking it, and I think definitely the younger audience prefers to consume things in video content. And a lot of my content is also aimed towards people who maybe are exploring tech who don't work in tech yet, and so to really hit them, they probably aren't following me and they probably don't know who I am, they probably don't even know what to look for in my blog.So, for example, I have a blog post all about how I transitioned into tech, blah, blah, blah, and people still ask me all the time on TikTok, “How did you transition into tech? How did you”—I'm like, “It's in my blog.” On my—like, you know, linked my bio. But you still have to just kind of—I think, like, I tend to just recreate the content into the different platforms. And it can be a bit tedious, but I try to keep my blog up to date with, like, different stories of things that have happened to me. But these days, I mostly just post on TikTok, to be honest. [laugh].Corey: I had the same problem, but content reuse saved me. I started writing a long-form blog post of roughly 1000 to 1500 words every week, then reading it into a microphone. It became the AWS Morning Brief podcast and emailing out to the newsletter as well. So, it's one piece of content used three different times, which was awesome, but then there's the other side of it, which is, I need to come up with an interesting idea or concept or something to talk about for 1000 words every week, like clockwork. And one of the things that made this way easier is a tip I got from Scott Hanselman that I have been passing on whenever it seems appropriate—like in this conversation—which is if you find yourself explaining something a third time, turn it into a blog post because then you'll just be able to link people to the thing that you wrote where you go into significantly more depth around what you're talking about than you can in a two-tweet exchange, and that in turn, gives you a place to dump that stuff out.And I found that has worked super well for me because once I've written it and gotten it out, I also often find I stopped making the same reference all the time because now I've said it, I've said my piece. Now, I can move on and come up with a second analogy, or a new joke or something.Emily: Yeah. I've also found that um—that's a great idea from Scott; he's also great on the TikToks [laugh]—Corey: Oh, yes he is.Emily: —[crosstalk 00:29:45] [laugh]. Building his account. Yeah, I think another interesting thing is, specifically on TikTok and Twitter because it's more of a conversation between you and your community, I tend to get a lot of ideas just from people asking me questions, right? So, in the comments of something, it could be related to the video I just made and it really helps me expand upon, you know, what I was just saying and maybe answer a follow-up question in a different video. Or maybe it's just a totally unrelated question.So, someone finds, you know, one of my comedy videos and is like, “Hey, you work in tech. Like, what is that like in San Francisco?” Right? So, I think I've found a ton of inspiration just from community people and really what they're asking for, right? Because at the end of the day, you want to make content that people actually care about and want to know the answers to.Corey: Yeah, seems like that does help. If it's, “How do I wind up building a following or getting a lot of traffic or the rest?” And it's Lord knows, once you have a website that has a certain amount of Google juice, you just get besieged by random requests from basically every channel. “Hey, I saw this great article linked to a back issue of the newsletter talking about this thing. Would you mind including my link to it, this would help your readers.” And it's just it's a pure SEO scam.And it's yeah, I don't—my approach to SEO has been this, again, ancient, old-timey idea of I'm going to write compelling original content that ideally other people find valuable and then assume that the rest is going to take care of itself. Because, on some level, that is what all these algorithms are trying to do is surface the useful stuff. I feel like as long as you hold to that, you're not going to go too far wrong.Emily: No, that's true. Also, something funny about reusing content is sometimes I'll post a joke on Twitter, and if it does well, I'll make it into a video format. And you know, sometimes I change the format of the joke around, whatever. But I—a couple times this happened—I'll post something on Twitter, and then, like, a day or two later, I'll make a TikTok about it, and a lot of people will come in and be like, “I already saw this joke on Twitter.” And they won't know it's from me, so they're basically accusing me of joke stealing when really I'm just content-raising is what I should tell them. But it is funny. [laugh].Corey: That's happened me a couple times on Twitter. People are like, “Hey, that's a stolen joke.” And then they'll google it and they'll dig it out. Like, “Here's the original—oh, wait, you said it two years ago.” “Yeah. No one liked it then, so here we are.” “If you liked it then, why didn't you blow it up like you did now?” So.Emily: They remembered it from two years ago, but they didn't remember it was yours. [laugh].Corey: At some level, I feel like I could almost loop my Twitter account and just let it continue to play out again for the next seven years, and other than the live-streaming stuff and the live-tweeting various events, I feel like it would do fairly well, but who knows.Emily: Yeah. Yeah. But at the end of the day, I think there's also a finite amount of funny tech jokes, and we're all just kind of recycling each other's jokes at some point. So, I don't get too offended by that. I'm like, “Sure. We all made the same joke about NFTs. Great.” Like, I don't care. [laugh].Corey: I really want to thank you for taking the time to speak with me today.Emily: [crosstalk 00:32:36] been fun.Corey: If people want to learn more and appreciate some of that awesome content, where's the best place to find you?Emily: Yeah, I'm on the Twitters and the TikToks, just like you.Corey: Excellent. And we will, of course, put links to that in the [show notes 00:32:45].Emily: Had a great time. Thank you so much for having me again.Corey: No, thank you for coming. Emily Kager, senior Android engineer at Uber. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment that links to a TikTok video of you ranting for a solid minute, but because computers and phones alike are very hard, you're using the wrong camera, and we just get that video of your floor.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

About KellyKellyAnn Fitzpatrick is a Senior Industry Analyst at RedMonk, the developer-focused industry analyst firm. Having previously worked as a QA analyst, test & release manager, and tech writer, she has experience with containers, CI/CD, testing frameworks, documentation, and training. She has also taught technical communication to computer science majors at the Georgia Institute of Technology as a Brittain Postdoctoral Fellow.Holding a Ph.D. in English from the University at Albany and a B.A. in English and Medieval Studies from the University of Notre Dame, KellyAnn's side projects include teaching, speaking, and writing about medievalism (the ways that post-medieval societies reimagine or appropriate the Middle Ages), and running to/from donut shops.Links: RedMonk: https://redmonk.com/ Twitter: https://twitter.com/drkellyannfitz TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Today's episode is brought to you in part by our friends at MinIO the high-performance Kubernetes native object store that's built for the multi-cloud, creating a consistent data storage layer for your public cloud instances, your private cloud instances, and even your edge instances, depending upon what the heck you're defining those as, which depends probably on where you work. It's getting that unified is one of the greatest challenges facing developers and architects today. It requires S3 compatibility, enterprise-grade security and resiliency, the speed to run any workload, and the footprint to run anywhere, and that's exactly what MinIO offers. With superb read speeds in excess of 360 gigs and 100 megabyte binary that doesn't eat all the data you've gotten on the system, it's exactly what you've been looking for. Check it out today at min.io/download, and see for yourself. That's min.io/download, and be sure to tell them that I sent you.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance query accelerator for the Oracle MySQL Database Service, although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLAP and OLTP—don't ask me to pronounce those acronyms again—workloads directly from your MySQL database and eliminate the time-consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: Welcome to Screaming in the Cloud, I'm Corey Quinn. It's always a good day when I get to sit down and have a chat with someone who works over at our friends at RedMonk. Today is no exception because after trying for, well, an embarrassingly long time, my whining and pleading has finally borne fruit, and I'm joined by Kelly Fitzpatrick, who's a senior industry analyst at RedMonk. Kelly, thank you for, I guess, finally giving in to my always polite, but remarkably persistent requests to show up on the show.Kelly: Great, thanks for having me. It's great to finally be on the show.Corey: So, let's start at the very beginning because I am always shockingly offended whenever it happens, but some people don't actually know what RedMonk is. What is it you'd say it is that you folks do?Kelly: Oh, I love this question. Because it's like, “What do you do,” versus, “What are you?” And that's a very big difference. And I'm going to start with maybe what we are. So, we are a developer-focused industry analyst firm. You put all those things, kind of, together.And in terms of what we do, it means that we follow tech trends. And that's something that many industry analysts do, but our perspective is really interested in developers specifically and then practitioners more broadly. So, it's not just, “Okay, these are things that are happening in tech that you care about if you're a CIO,” but what tech things affect developers in terms of how they're building software and why they want to build software and where they're building software?Corey: So, backing it up slightly because it turns out that I don't know the answer to this either. What exactly is an industry analyst firm? And the reason I bring this up is I've been invited to industry analyst events, and that is entirely your colleague, James Governor's, fault because he took me out for lunch at I think it was Google Next a few years ago and said, “Oh, you're definitely an analyst.” “Okay, cool. Well, I don't think I am. Why should I be an analyst?”“Oh, because companies have analyst budgets.” “Oh, you said, analyst”—protip: Never get in the way of people trying to pay you to do things. But I still feel like I don't know what an analyst is, in this sense. Which means I'm about to get a whole bunch of refund requests when this thing airs.Kelly: I should hope not. But industry analysts, one of the jokes that we have around RedMonk is how do we explain to our families what an industry analyst is? And I think even Steve and James, who are RedMonk's founders, they've been doing this for quite a long time, like, much longer than they ever want to admit that they do, and they still are like, “Okay, how do I explain this to my parents?” Or you know, anyone else who's asking, and partly, it's almost like a very—a term that you'll see in the tech industry, but outside of it doesn't really have that much, kind of, currency in the same way that you can tell someone that you're like, maybe a business analyst or something like that, or any of those, almost like spy-like versions of analyst. I think was it The Hunt for Red October, the actual hero of that is an analyst, but not the type of analyst that I am in any way, shape or form.But you know, industry analyst firms, specifically, it's like we keep up on what tech is out there. People engage with us because they want to know what to buy for the things that they're doing and the things that they're building, or how to better create and sell the stuff that they are building to people who build software. So, in our case, it's like, all right, what type of tools are developers using? And where does this particular tool that our company is building fit into that? And how do you talk about that with developers in a way that makes sense to them?Corey: On some level, what I imagine your approach to this stuff is aligns somewhat with my own. Before you became an industry analyst, which I'm still not entirely sure I know what that is—I'm sorry, not your fault; just so many expressions of it out there—before you wound up down that path, you were a QA manager; you wound up effectively finding interesting bugs in software, documentation, et cetera. And, on some level, that's, I think, what has made me even somewhat useful in the space is I'll go ahead and try and build something out of something that a vendor has released, and huh, the documentation says it should work this way, but I try it and it breaks and it fails. And the response is always invariably the same, which is, “That's interesting,” which is engineering-speak for, “What the hell is that?” I have this knack for stumbling over weird issues, and I feel like that aligns with what makes for a successful QA person. Is that directionally correct, or am I dramatically misunderstanding things and I'm just accident-prone?Kelly: [laugh]. No, I think that makes a lot of sense. And especially coming from QA where it's like, not just making sure that something works, but making sure that something doesn't break if you try to break it in different ways, the things that are not necessarily the expected, you know, behaviors, that type of mindset, I think, for me translated very easily to, kind of, being an analyst. Because it's about asking questions; it's about not just taking the word of your developers that this software works, but going and seeing if it actually does and kind of getting your hands dirty, and in some cases, trying to figure out where certain problems or who broke the build, or why did the build break is always kind of super fun mystery that I love doing—not really, but, like, everyone kind of has to do it—and I think that translates to the analyst world where it's like, what pieces of these systems, or tech stacks, or just the way information is being conveyed about them is working or is not, and in what ways can people kind of maybe see things a different way that the people who are building or writing about these things did not anticipate?Corey: From my position, and this is one of the reasons I sort of started down this whole path is if I'm trying to build something with a product or a platform—or basically anything, it doesn't really matter what—and the user experience is bad, or there are bugs that get in my way, my default response—even now—is not, “Oh, this thing's a piece of crap that's nowhere near ready for primetime use,” but instead, it's, “Oh, I'm not smart enough to figure out how to use it.” It becomes a reflection on the user, and they feel bad as a result. And I don't like that for anyone, for any product because it doesn't serve the product well, it certainly doesn't serve the human being trying to use it and failing well, and from a pure business perspective, it certainly doesn't serve the ability to solve a business problem in any meaningful respect. So, that has been one of the reasons that I've been tilting at that particular windmill for as long as I have.Kelly: I think that makes sense because you can have the theoretically best, most innovative, going to change everyone's lives for the better, product in the world, but if nobody can use it, it's not going to change the world.Corey: As you take a look at your time at RedMonk, which has been, I believe, four years, give or take?Kelly: We're going to say three to four.Corey: Three to four? Because you've been promoted twice in your time there, let's be very clear, and this is clearly a—Kelly: That's a very, very astute observation on your part.Corey: It is a meteoric rise. And what makes that also fascinating from my perspective, is that despite being a company that is, I believe, 19 years old, you aren't exactly a giant company that throws bodies at problems. I believe you have seven full-time employees, two of whom have been hired in the last quarter.Kelly: That's true. So, seven full-time employees and five analysts. So, we have—of that it's five analysts, and we only added a fifth analyst the beginning of this year, with Dr. Kate Holterhoff. [unintelligible 00:08:09], kind of, bring her on the team.So, we had been operating with, like, kind of, six full-time employees. We were like, “We need some more resources in this area.” And we heard another analyst, which if you talk about, okay, we hired one more, but when you're talking about hiring one more and adding that to a team of, like, four analysts, it's such a big difference, just in terms of, kind of, resources. And I think your observation about you ca—we don't just throw bodies at problems is kind of correct. That is absolutely not the way we go about things at all.Corey: At a company that is taking the same model that The Duckbill Group does—by which I mean not raising a bunch of outside money is, as best I can tell—that means that you have to fall back on this ancient business model known as making more money than it costs to run the place every month, you don't get to do this massive scaled out hiring thing. So, bringing on multiple employees at a relatively low turnover company means that suddenly you're onboarding not just one new person, but two. What has that been like? Because to be very clear, if you're hiring 20 engineers or whatnot, okay, great, and you're having significant turnover, yeah, onboarding two folks is not that big of a deal, but this is a significant percentage of your team.Kelly: It is. And so for us—and Kate started at the beginning of this year, so she's only been here for a bit—but in terms of onboarding another analyst, this is something where I haven't done before, but, like, my colleagues have, whereas the other new member of our team, Morgan Harris, who is our Account Engagement Manager, and she is amazing, and has also, like, very interesting background and client success in, like, fashion, which is, you know, awesome when I'm trying to figure out what [unintelligible 00:09:48] fit I need to do, we have someone in-house who can actually give me advice on that. But that's not something that we have onboarded for that role very much in the past, so bringing on someone where they're the only person in their role and, like, having to begin to learn the role. And then also to bring in another analyst where we have a little bit more experience onboarding analysts, it takes a lot of patience for everybody involved. And the thing I love about RedMonk and the people that I get to work with is that they actually have that patience and we function very well as, like, a team.And because of that, I think things that could really have thrown us off course, like losing an account engagement or onboarding one and then onboarding a new analyst, like, over the holidays, during a pandemic, and everything else that is happening, it's going much more smoothly than it could have otherwise.Corey: These are abnormal times, to be sure. It's one of those things where it's, we're a couple years into a pandemic now, and I still feel like we haven't really solved most of the problems that this has laid bare, which kind of makes me despair of ever really figuring out what that's going to look like down the road.Kelly: Yeah, absolutely. And there is very much the sense that, “Okay, we should be kind of back to normal, going to in-person conferences.” And then you get to an in-person conference, and then they all move back to virtual or, as in your case, you go to an in-person conference and then you have to sequester yourself away from your family for a couple of weeks to make sure that you're not bringing something home.Corey: So, I have to ask. You have been quoted as saying that 2022—for those listening, that is this year—is the year of documentation. You're onboarding two new people into a company that does not see significant turnover, which means that invariably, “Oh, it's been a while since we've updated the documentation. Whoops-a-doozy,” is a pretty common experience there. How much of your assertion that this is the year of documentation comes down to the, “Huh. Our onboarding stuff is really out of date,” versus a larger thing that you're seeing in the industry?Kelly: That is a great question because you never know what your documentation is like until you have someone new, kind of, come in with fresh eyes, has a perspective not only on, “Okay, I have no idea what this means,” or, “This is not where I thought it would be,” or, “This, you know, system is not working in any… in any way similar to anything I have ever seen in any other part of my, like, kind of, working career.” So, that's where you really see what kind of gaps you have, but then you also kind of get to see which parts are working out really well. And not to spend, kind of, too much on that, but one of the best things that my coworkers did for me when I started was, Rachel Stephens had kept a log of, like, all the questions that she had as a new analyst. And she just, like, gave that to me with some advice on different things, like, in a spreadsheet, which I think is—I love spreadsheets so much and so does Rachel. And I think I might love spreadsheets more than Rachel at this point, even though she actually has a hat that says, “Spreadsheets.”But when Kate started, it was fascinating to go through that and see what parts of that were either no longer relevant because the entire world had changed, or because the industry had advanced, or because there's all these new things you need to know now that we're not on the list of things that you needed to know three years ago. And then what other, even, topics belong down on that kind of list of things to know. So, I think documentation is always a good, like, check-in for things like that.But going back to, like, your larger question. So, documentation is important, not just because we happened to be onboarding, but a lot of people, I think once they no longer could be in the office with people and rely on that kind of face-to-face conversations to smooth over things began, I think, to realize how essential documentation was to just their everyday to day, kind of, working lives. So, I think that's something that we've definitely seen from the pandemic. But then there are certainly other signals in the software industry-specific, which we can go into or not depending on your level of interest.Corey: Well, something that I see that I have never been a huge fan of in corporate life—and it feels like it is very much a broad spectrum—has been that on one side of the coin, you have this idea that everything we do is bespoke and we just hire smart people and get out of their way. Yeah, that's more uncontrolled anarchy than it is a repeatable company process around anything. And the other extreme is this tendency that companies have, particularly the large, somewhat slow-moving companies, to attempt to codify absolutely everything. It almost feels like it derives from the what I believe to be mistaken belief that with enough process, eventually you can arrive at the promised land where you don't have to have intelligent, dynamic people working behind things, you can basically distill it down to follow the script and push the buttons in the proper order, and any conceivable outcome is going to be achieved. I don't know if that's accurate, but that's always how it felt when you start getting too deeply mired in documentation-slash-process as almost religion.Kelly: And I think—you know, I agree. There has to be something between, “All right, we don't document anything and it's not necessary and we don't need it.” And then—Corey: “We might get raided by the FBI. We want nothing written down.” At which point it's like, what do you do here? Yeah.Kelly: Yeah. Leave no evidence, leave no paper trail of anything like that. And going too far into thinking that processes is absolutely everything, and that absolutely anyone can be plugged into any given role and things will be equally successful, or that we'll just be automated away or become just these, kind of, automatons. And I think that balance, it's important to think about that because while documentation is important, and you know, I will say 2022, I think we're going to hear more and more about it, we see it more as an increasingly valuable thing in tech, you can't solve everything with documentation. You can use it as the, kind of, duct tape and baling wire for some of the things that your company is doing, but throwing documentation at it is not going to fix things in the same way that throwing engineers at a problem is not going to fix it either. Or most problems. I mean, there are some that you can just throw engineers at.Corey: Well, there's a company wiki, also known as where documentation goes to die.Kelly: It is. And those, like, internal wikis, as horrible as they can be in terms of that's where knowledge goes to die as well, places that have nothing like that, it can be even more chaotic than places that are relying on the, kind of, company internal wiki.Corey: So, delving into a bit of a different topic here, before you were in the QA universe, you were what distills down to an academic. And I know that sometimes that can be interpreted as a personal attack in some quarters; I assure you, despite my own eighth grade level of education, that is not how this is intended at all. Your undergraduate degree was in medieval history—or medieval studies and your PhD was in English. So, a couple of questions around that. One, when we talk about medieval studies, are we talking about writing analyst reports about Netscape Navigator, or are we talking things a bit later in the sweep of history than that?Kelly: I appreciate the Netscape Navigator reference. I get that reference.Corey: Well, yeah. Medieval studies; you have to.Kelly: Medieval studies, when you—where we study the internet in the 1990s, basically. I completely lost the line of questioning that you're asking because I was just so taken by the Netscape Navigator reference.Corey: Well, thank you. Started off with the medieval studies history. So, medieval studies of things dating back to, I guess, before we had reasonably recorded records in a consistent way. And also Twitter. But I'm wondering how much of that lends itself to what you do as an analyst.Kelly: Quite a bit. And as much as I want to say, it's all Monty Python references all the time, it isn't. But the disciplinary rigor that you have to pick up as a medievalist or as anyone who's getting any kind of PhD ever, you know, for the most part, that very much easily translated to being an analyst. And even more so tech culture is, in so many ways, like, enamored—there's these pop culture medieval-isms that a lot of people who move in technical circles appreciate. And that kind of overlap for me was kind of fascinating.So, when I started, like, working in tech, the fact that I was like writing a dissertation on Lord of the Rings was this little interesting thing that my coworkers could, like, kind of latch on to and talk about with me, that had nothing to do with tech and that had nothing to do with the seemingly scary parts of being an academic.Corey: This episode is sponsored in part by our friends at Vultr. Spelled V-U-L-T-R because they're all about helping save money, including on things like, you know, vowels. So, what they do is they are a cloud provider that provides surprisingly high performance cloud compute at a price that—while sure they claim its better than AWS pricing—and when they say that they mean it is less money. Sure, I don't dispute that but what I find interesting is that it's predictable. They tell you in advance on a monthly basis what it's going to going to cost. They have a bunch of advanced networking features. They have nineteen global locations and scale things elastically. Not to be confused with openly, because apparently elastic and open can mean the same thing sometimes. They have had over a million users. Deployments take less that sixty seconds across twelve pre-selected operating systems. Or, if you're one of those nutters like me, you can bring your own ISO and install basically any operating system you want. Starting with pricing as low as $2.50 a month for Vultr cloud compute they have plans for developers and businesses of all sizes, except maybe Amazon, who stubbornly insists on having something to scale all on their own. Try Vultr today for free by visiting: vultr.com/screaming, and you'll receive a $100 in credit. Thats V-U-L-T-R.com slash screaming.Corey: I want to talk a little bit about the idea of academic rigor because to my understanding, in the academic world, the publication process is… I don't want to say it's arduous. But if people subjected my blog post anything approaching this, I would never write another one as long as I lived. How does that differ? Because a lot of what I write is off-the-cuff stuff—and I'm not just including tweets, but also tweets—whereas academic literature winds up in peer-reviewed journals and effectively expands the boundaries of our collective societal knowledge as we know it. And it does deserve a different level of scrutiny, let's be clear. But how do you find that shifts given that you are writing full-on industry analyst reports, which is something that we almost never do on our side, just honestly, due to my own peccadilloes?Kelly: You should write some industry reports. They're so fun. They're very fun.Corey: I am so bad at writing the long-form stuff. And we've done one or two previously, and each time my business partner had to basically hold my nose to the grindstone by force to get me to ship, on some level.Kelly: And also, I feel like you might be underselling the amount of writing talent it takes to tweet.Corey: It depends. You can get a lot more trouble tweeting than you can in academia most of the time. Every Twitter person is Reviewer 2. It becomes this whole great thing of, “Well, did you consider this edge corner case nuance?” It's, “I've got to say, in 208 any characters, not really. Kind of ran out of space.”Kelly: Yeah, there's no space at all. And it's not what that was intended. But going back to your original question about, like, you know, academic publishing and that type of process, I don't miss it. And I have actually published some academic pieces since I became an analyst. So, my book finally came out after I had started as—it came out the end of 2019 and I had already been at RedMonk for a year.It's an academic book; it has nothing to do with being an industry analyst. And I had an essay come out in another collection around the same time. So, I've had that come out, but the thing is, the cycle for that started about a year earlier. So, the timeframe for getting things out in, especially the humanities, can be very arduous and frustrating because you're kind of like, “I wrote this thing. I want it to actually appear somewhere that people can read it or use it or rip it apart if that's what they're going to do.”And then the jokes that you hear on Twitter about Reviewer 2 are often real. A lot of academic publishing is done in, like, usually, like, a double-blind process where you don't know who's reviewing you and the reviewers don't know who you are. I've been a reviewer, too, so I've been on that side of it. And—Corey: Which why you run into the common trope of people—Kelly: Yes.Corey: —suggesting, “Oh, you don't know what you're talking about. You should read this work by someone else,” who is in fact, the author they are reviewing.Kelly: Absolutely. That I think happens even when people do know who [laugh] who's stuff they're reviewing. Because it happens on Twitter all the time.Corey: Like, “Well, have you gotten to the next step beyond where you have a reviewer saying you should wind up looking at the work cited by”—and then they name-check themselves? Have we reached that level of petty yet, or has that still yet to be explored?Kelly: That is definitely something that happens in academic publishing. In academic circles, there can be these, like, frenemy relations among people that you know, especially if you are in a subfield that is very tiny. You tend to know everybody who is in that subfield, and there's, like, a lot of infighting. And it does not feel that far from tech, sometimes. [unintelligible 00:21:52] you could look at the whole tech industry, and you look at the little areas that people specialize in, and there are these communities around these specializations that—you can see some of them on Twitter.Clearly, not all of them exist in the Twitterverse, but in some ways, I think that translated over nicely of, like, the year-long publication and, like, double peer-review process is not something that I have to deal with as much now, and it's certainly something that I don't miss.Corey: You spent extensive amounts of time studying the past, and presumably dragons as well because, you know, it's impossible to separate medieval studies from dragons in my mind because basically, I am a giant child who lives through fantasy novels when it comes to exploring that kind of past. And do you wind up seeing any lessons we can take from the things you have studied to our current industry? That is sort of a strange question, but they say that history doesn't repeat, but it rhymes, and I'm curious to how far back that goes. Because most people are citing, you know, 1980s business studies. This goes centuries before that.Kelly: I think the thing that maybe stands out for me the most the way that you framed that is, when we look at the past and we think of something like the Middle Ages, we will often use that term and be like, “Okay, here's this thing that actually existed, right?” Here's, like, this 500 years of history, and this is where the Middle Ages began, and here's where it ended, and this is what it was like, and this is what the people were like. And we look at that as the some type of self-evident thing that exists when in reality, it's a concept that we created, that people who lived in later ages created this concept, but then it becomes something that has real currency and, really, weight in terms of, like, how we talk about the world.So, someone will say, you know, I like that film. It was very medieval. And it'll be a complete fantasy that has nothing to do with Middle Ages but has a whole bunch of these tropes and signals that we translate as the Middle Ages. I feel like the tech industry has a great capacity to do that as well, to kind of fold in along with things that we tend to think of as being very scientific and very logical but to take a concept and then just kind of begin to act as if it is an actual thing when it's something that people are trying to make a thing.Corey: Tech has a lot of challenges around the refusing to learn from history aspect in some areas, too. One of the most common examples I've heard of—or at least one that resonated the most with me—is hiring, where tech loves to say, “No one really knows how to hire effectively and well.” And that is provably not true. Ford and GM and Coca-Cola have run multi-decade studies on how to do this. They've gotten it down to a science.But very often, we look at that in tech and we're trying to invent everything from first principles. And I think, on some level, part of that comes out as, “Well, I wouldn't do so well in that type of interview scenario, therefore, it sucks.” And I feel like we're too willing in some cases to fail to heed the lessons that others have painstakingly learned, so we go ahead and experiment on our own and try and reinvent things that maybe we should not be innovating around if we're small, scrappy, and trying to one area of the industry. Maybe going back to how we hire human beings should not be one of those areas of innovation that you spend all your time on as a company.Kelly: I think for some companies, I think it depends on how you're hiring now. It's like, if your hiring practices are horrible, like, you probably do need to change them. But to your point, like, spending all of your energy on how are we hiring, can be counterproductive. Am I allowed to ask you a question?Corey: Oh, by all means. Mostly, the questions people ask me is, “What the hell is wrong with you?” But that's fine, I'm used to that one, too. Bonus points if you have a different one.Kelly: Like, your hiring processes at Duckbill Group. Because you've hired, you know, folks recently. How do you describe that? Like, what points of that you think… are working really well?Corey: The things that have worked out well for us have been being very transparent at the beginning around things like comp, what the job looks like, where it starts, where it stops, what we expect from people, what we do not expect from people, so there are no surprises down that path. We explain how many rounds of interviews there are, who they'll be meeting with at each stage. If we wind up declining to continue with a candidate in a particular cycle, anything past the initial blind resume submission, we will tell them; we don't ghost people. Full stop. Originally, we wanted to wind up responding to every applicant with a, “Sorry, we're not going to proceed,” if the resume was a colossal mismatch. For example, we're hiring for a cloud economist, and we have people with PhDs in economics, and… that's it. They have not read the job description.And then when you started doing that people would argue with us on a constant basis, and it just became a soul-sucking time sink. So, it's unfortunate, but that's the reality of it. But once we've had a conversation with you, doing that is the right answer. We try and move relatively quickly. We're honest with folks because we believe that an interview is very much a two-way street.And even if we declined to proceed—or you declined to proceed with us; either way—that you should still think well enough of us that you would recommend us to people for whom it might be a fit. And if we treat you like crap, you're never going to do that. Not to mention, I just don't like making people feel like crap as a general rule. So, that stuff that has all come out of hiring studies.So, has the idea of a standardized interview. We don't have an arbitrary question list that we wind up smacking people with from a variety of different angles. And if you drew the lucky questions, you'll do fine. We also don't set this up as pass-fail, we tend to presume that by the time you've been around the industry for as long as generally is expected for years of experience for the role, we're not going to suddenly unmask you as not knowing how computers work through our ridiculous series of trivia questions. We don't ask those.We also make the interview look a lot like what the job is, which is apparently a weird thing. It's in a lot of tech companies it's, “Go and solve whiteboard algorithms for us.” And then, “Great. Now, what's the job?” “It's going to be moving around some CSS nonsense.”It's like, first that is very different, and secondly, it's way harder to move CSS than to implement quicksort, for most folks. At least for me. So, it's… yeah, it just doesn't measure the right things. That's our approach. I'm not saying we cracked it by any means to be very clear here. This is just what we have found that sucks the least.Kelly: Yeah, I think the, ‘we're not going to do obscure whiteboarding exercises' is probably one of the key things. I think some people are still very attached those personal reasons. And I think the other thing I liked about what you said, is to make the interview as similar to the job as you can, which based on my own getting hired process at RedMonk and then to some levels of being involved in hiring our, kind of, new hires, I really like that. And I think that for me, the process will like, okay, you submit your application. There'd be—I think I'd to do a writing sample.But then it was like, you get on a call and you talk to Steve. And then you get on a call and you talk to James. And talking to people is my job. Like for the most part. I write things, but it's mostly talking to people, which you may not believe by the level of articulate, articulate-ness, I am stumbling my way through in this sentence.And then the transparency angle, I think it's something that most companies are not—may not be able to approach hiring in such a transparent way for whatever reason, but at least the motion towards being transparent about things like salaries, as opposed to that horrible salary negotiation part where that can be a nightmare for people, especially if there's this code of silence around what your coworkers or potential coworkers are making.Corey: We learned we were underpaying our clouds economists, so we wound up adjusting the rate advertised; at the same time we wound up improving the comp for existing team because, “Yeah, we're just going to make you apply again to be paid a fair wage for what you do,” no. Not how we play these games.Kelly: Yeah, which is, you know, one of the things that we're seeing in the industry now. Of course, the term ‘The Great Resignation' is out there. But with that comes, you know, people going to new places partly because that's how they can get, like, the salary increase or whatever it is they want for among other reasons.Corey: Some of the employees who have left have been our staunchest advocates, both for new applicants as well as new clients. There's something to be said for treating people as you mean to go on. My business partner, I've been clear that we aspire for this to be a 20, 25-year company, and you don't do that by burning bridges.Kelly: Yeah. Or just assuming that your folks are going to stay for three years and move on, which tends to be the kind of the lifespan of where people stay.Corey: Well, if they do, that's fine because it is expected. I don't want people to wind up feeling that they owe us anything. If it no longer makes sense for them to be here because they're not fulfilled or whatnot—this has happened to us before we've tried to change their mind, talked to them about what they wanted, and okay, we can't offer what you're after. How can we help you move on? That's the way it works.And like, the one thing we don't do in interviews—and this is something I very much picked up from the RedMonk culture as well—is we do a lot of writing here, so there's a writing sample of here's a list of theoretical findings for an AWS bill—if we're talking about a cloud economist role—great. Now, the next round is people are going to talk to you about that, and we're going to roleplay as if we were a client. But let's be clear, I won't tolerate abusive behavior from clients to our team, I will fire a client if it happens. So, we're not going to wind up bullying the applicant and smacking ‘em around on stuff—or smacking them around to be clear. That was an ‘em not a him, let's be clear.It's a problem of not wanting to even set the baseline expectation that you just have to sit there and take it when clients decide to go down unfortunate paths. And I believe it's happened all of maybe once in our five-and-a-half-year history. So, why would you ever sit around and basically have a bunch of people chip away at an applicant's self-confidence? By virtue of being in the room and having the conversation, they are clearly baseline competent at a number of things. Now, it's just a question of fit and whether their expression of skills is what we're doing right now as a company.At least that's how I see it. And I think that there is a lot of alignment here, not just between our two companies, but between the kinds of companies I look at and can actively recommend that people go and talk to.Kelly: Yeah. I think that emphasis on, it's not just about what a company is doing—like, what is their business, you know, how they're making money—but how they're treating people, like, on their way in and on the way out. I don't think you can oversell how important that is.Corey: Culture is what you wind up with instead of what you intend. And I think that's something that winds up getting lost a fair bit.Kelly: Yeah, culture is definitely not something you can just go buy, right? [laugh], where you can, like—this is what our culture will be.Corey: No, no. But if there is, “Culture-in-a-box. Like, you may not be able to buy it, but I would love to sell it to you,” seems to be the watchwords of a number of different companies out there. Kelly, I really want to thank you for taking the time to speak with me today. If people want to learn more, where can they find you?Kelly: They can find me on Twitter at @drkellyannfitz, that's D-R-K-E-L-L-Y-A-N-N-F-I-T-Z—I apologize for having such a long Twitter handle—or my RedMonk work and of my colleagues, you can find that at redmonk.com.Corey: And we will, of course, include links to that in the [show notes 00:33:14]. Thank you so much for your time. I appreciate it.Kelly: Thanks for having me.Corey: Kelly Fitzpatrick, senior industry analyst at RedMonk. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment telling me how terrible this was and that we should go listen to Reviewer 2's podcast instead.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

About AnnaAnna has nearly ten years of experience researching and advising organizations on cloud adoption with a focus on security best practices. As a Gartner Analyst, Anna spent six years helping more than 500 enterprises with vulnerability management, security monitoring, and DevSecOps initiatives. Anna's research and talks have been used to transform organizations' IT strategies and her research agenda helped to shape markets. Anna is the Director of Thought Leadership at Sysdig, using her deep understanding of the security industry to help IT professionals succeed in their cloud-native journey.Anna holds a PhD in Materials Engineering from the University of Michigan, where she developed computational methods to study solar cells and rechargeable batteries.How do I adapt my security practices for the cloud-native world?How do I select and deploy appropriate tools and processes to address business needs?How do I make sense of new technology trends like threat deception, machine learning, and containers?Links: Sysdig: https://sysdig.com/ “2022 Cloud-Native Security and Usage Report”: https://sysdig.com/2022-cloud-native-security-and-usage-report/ Twitter: https://twitter.com/aabelak LinkedIn: https://www.linkedin.com/in/aabelak/ Email: anna.belak@sysdig.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Today's episode is brought to you in part by our friends at MinIO the high-performance Kubernetes native object store that's built for the multi-cloud, creating a consistent data storage layer for your public cloud instances, your private cloud instances, and even your edge instances, depending upon what the heck you're defining those as, which depends probably on where you work. It's getting that unified is one of the greatest challenges facing developers and architects today. It requires S3 compatibility, enterprise-grade security and resiliency, the speed to run any workload, and the footprint to run anywhere, and that's exactly what MinIO offers. With superb read speeds in excess of 360 gigs and 100 megabyte binary that doesn't eat all the data you've gotten on the system, it's exactly what you've been looking for. Check it out today at min.io/download, and see for yourself. That's min.io/download, and be sure to tell them that I sent you.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance query accelerator for the Oracle MySQL Database Service, although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLAP and OLTP—don't ask me to pronounce those acronyms again—workloads directly from your MySQL database and eliminate the time-consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Once upon a time, I went to a conference talk at, basically, a user meetup. This was in the before times, when that wasn't quite as much of a deadly risk because of a pandemic, and mostly a deadly risk due to me shooting my mouth off when it wasn't particularly appreciated.At that talk, I wound up seeing a new open-source project that was presented to me, and it was called Sysdig. I wasn't quite sure on what it did at the time and I didn't know what it would be turning into, but here we are now, what is it, five years later. Well, it's turned into something rather interesting. This is a promoted episode brought to us by our friends at Sysdig and my guest today is their Director of Thought Leadership, Anna Belak. Anna, thank you for joining me.Anna: Hi, Corey. I'm very happy to be here. I'm a big fan.Corey: Oh, dear. So, let's start at the beginning. Well, we'll start with the title: Director of Thought Leadership. That is a lofty title, it sounds like you sit on the council of the Lords of Thought somewhere. Where does your job start and stop?Anna: I command the Council of the Lords of thought, actually. [laugh].Corey: Supply chain issues mean the robe wasn't available. I get it, I get it.Anna: There is a robe. I'm just not wearing it right now. So, the shortest way to describe the role is probably something that reports into engineering, interestingly, and it deals with product and marketing in a way that is half evangelism and half product strategy. I just didn't feel like being called any of those other things, so they were like, “Director of Thought Leadership you are.” And I was like, “That sounds awesome.”Corey: You know, it's one of those titles that people generally don't see a whole lot of, so if nothing else, I always liked those job titles that cause people to sit up and take notice as opposed to something that just people fall asleep by the time you get halfway through it because, in lieu of a promotion, people give you additional adjectives in your title. And we're going to go with it. So, before you wound up at Sysdig, you were at Gartner for a number of years.Anna: That's right, I spent about six years at Gartner, and there half the time I covered containers, Kubernetes, and DevOps from an infrastructure perspective, and half the time I spent covering security operations, actually, not specifically with respect to containers, or cloud, but broadly. And so my favorite thing is security operations, as it relates to containers and cloud-native workloads, which is kind of how I ended up here.Corey: I wouldn't call that my favorite thing. It's certainly something that is near and dear to the top of mind, but that's not because I like it, let's put it [laugh] that way. It's one of those areas where getting it wrong is catastrophic. Back in 2017, when I went to that meetup in San Francisco, Sysdig seemed really interesting to me because it looked like it tied together a whole bunch of different diagnostic tools, LSOF, strace, and the rest. Honestly—and I mean no slight to the folks who built out this particular tool—it felt like DTrace, only it understood the value of being accessible to its users without basically getting a doctorate in something.I like the idea, and it felt like it was very much aimed at an in-depth performance analysis story or an observability play. But today, it seems that you folks have instead gone in much more of a direction of DevSecOps, if the people listening to this, and you, will pardon the term. How did that happen? What was that product evolution like?Anna: Yeah, I think that's a fair assessment, actually. And again, no disrespect to DTrace of which I'm also a fan. So, we certainly started out in the container observability space, essentially because this whole Docker Kubernetes thing was exploding in popularity—I mean, before it was exploding, it was just kind of like, peaking out—and very quickly, our founder Loris, who is the co-founder of Wireshark, was like, “Hey, there's a visibility issue here. We can't see inside these things with the tools that we have that are built for host instrumentation, so I'm going to make a thing.” And he made a thing, and it was an awesome thing that was open-sourced.And then ultimately, what happened is, the ecosystem of containers and communities evolved, and more and more people started to adopt it. And so more people needed kind of a more, let's say, hefty, serious tool for observability, and then what followed was another tool for security because what we actually discovered was the data that we're able to collect from the system with Sysdig is incredibly useful for noticing security problems. So, that caused us to kind of expand into that space. And today we are very much a tool that still has an observability component that is quite popular, has a security component which is it's fairly broad: We cover CSPM use cases, we cover [CIEM 00:05:04] use cases, and we are very, kind of let's say, very strong and very serious about our detection response and runtime security use cases, which come from that pedigree of the original Sysdig as well.Corey: You can get a fairly accurate picture of what the future of technology looks like by taking a look at what my opinion of something is, and then doing the exact opposite of that. I was a big believer that virtualization, “Complete flash in the pan; who's going to use that?” Public cloud, “Are you out of your tree? No one's going to trust other companies with their holy of holies.” And I also spent a lot of time crapping on containers and not actually getting into them.Instead, I leapfrogged over into the serverless land, which I was a big fan of, which of course means that it's going to be doomed sooner or later. My security position has also somewhat followed similar tracks where, back when you're running virtual machines that tend to be persistent, you really have to care about security because you are running full-on systems that are persistent, and they run all kinds of different services simultaneously. Looking at Lambda Functions, for example, in the modern serverless world, I always find a lot of the tooling and services and offerings around security for that are a little overblown. They have a defined narrow input, they have a defined output, there usually aren't omnibus functions shoved in here where they have all kinds of different code paths. And it just doesn't have the same attack surface, so it often feels like it's trying to sell me something I don't need. Security in the container world is one of those areas I never had to deal with in anger, as a direct result. So, I have to ask, how bad is it?Anna: Well, I have some data to share with you, but I'll start by saying that I maybe was the opposite of you, so we'll see which one of us wins this one. I was an instant container fangirl from the minute I discovered them. But I crapped out—Corey: The industry shows you were right on that one. I think the jury [laugh] is pretty much in on this one.Anna: Oh, I will take it. But I did crap on Lambda Functions pretty hard. I was like, “Serverless? This is dumb. Like, how are we ever going to make that work?” So, it seems to be catching on a little bit, at least it. It does seem like serverless is playing the function of, like, the glue between bits, so that does actually make a lot of sense. In retrospect, I don't know that we're going to have—Corey: Well, it feels like it started off with a whole bunch of constraints around it, and over time, they've continued to relax those constraints. It used to be, “How do I package this?” It's, “Oh, simple. You just spent four days learning about all the ins and outs of this,” and now it's, “Oh, yeah. You just give it a Docker file?” “Oh. Well, that seems easier. I could have just been stubborn and waited.” Hindsight.Anna: Yeah, exactly. So, containers as they are today, I think are definitely much more usable than they were five-plus years ago. There are—again there's a lot of commercial support around these things, right? So, if you're, you know, like, a big enterprise client, then you don't really have time to fool around in open-source, you can go in, buy yourself a thing, and they'll come with support, and somebody will hold your hand as you figure it out, and it's actually quite, quite pleasant. Whether or not that has really gone mainstream or whether or not we've built out the entire operational ecosystem around it in a, let's say, safe and functional way remains to be seen. So, I'll share some data from our report, which is actually kind of the key thing I want to talk about.Corey: Yeah, I wanted to get into that. You wound up publishing this somewhat recently, and I regret that as of the time of this recording, I have not yet had time to go into it in-depth, and of course eviscerate it in my typical style on Twitter—although that may have been rectified by the time that this show airs, to be very clear—but it's the Sysdig “2022 Cloud-Native Security and Usage Report”.Anna: Please at me when you Twitter-shred it. [laugh].Corey: Oh, when I read through and screenshot it, and I'd make what observations that I imagine are witty. But I'm looking forward to it; I've done that periodically with the Flexera, “State of the Cloud” report for last few years, and every once in a while, whatever there's a, “We've done a piece of thought leadership, and written a report,” it's, “Oh, great. Let's make fun of it.” That's basically my default position on things. I am not a popular man, as you might imagine. But not having had the chance to go through it in-depth, what did this attempt to figure out when the study was built, and what did you learn that you found surprising?Anna: Yeah, so the first thing I want to point out because it's actually quite important is that this report is not a survey. This is actual data from our actual back end. So, we're a SaaS provider, we collect data for our customers, we completely anonymize it, and then we show in aggregate what in fact we see them doing or not doing. Because we think this is a pretty good indicator of what's actually happening versus asking people for their opinion, which is, you know, their opinion.Corey: Oh, I love that. My favorite lies that people tell are the lies they don't realize that they're telling. It's, I'll do an AWS bill analysis and, “Great. So, tell me about all these instances you have running over in Frankfurt.” “Oh, we don't have anything there.”I believe you're being sincere when you say this, however, the data does show otherwise, and yay, now we're in a security incident.Anna: Exactly.Corey: I'm a big believer of going to the actual source for things like this where it's possible.Anna: Exactly. So, I'll tell you my biggest takeaway from the whole thing probably was that I was surprised by the lack of… surprise. And I work in cloud-native security, so I'm kind of hoping every single day that people will start adopting these modern patterns of, like, discarding images, and deploying new ones when they found a vulnerability, and making ephemeral systems that don't run for a long time like a virtual machine in disguise, and so on. And it appears that that's just not really happening.Corey: Yeah, it's always been fun, more than a little entertaining, when I wind up taking a look at the aspirational plans that companies have. “Great, so when are you going to do”—“Oh, we're going to get to that after the next sprint.” “Cool.” And then I just set a reminder and I go back a year later, and, “How's that coming?” “Oh, yeah. We're going to get to that next sprint.”It's the big lie that we always tell ourselves that right after we finished this current project, then we're going to suddenly start doing smart things, making the right decisions, and the rest. Security, cost, and a few other things all tend to fall on the side of, you can spend infinite money and infinite time on these things, but it doesn't advance what your business is doing, but if you do none of those things, you don't really have a business anymore. So, it's always a challenge to get it prioritized by the strategic folks.Anna: Exactly. You're exactly right because what people ultimately do is they prioritize business needs, right? They are prioritizing whatever makes them money or creates the trinkets their selling faster or whatever it is, right? The interesting thing, though, is if you think about who our customers would be, like, who the people in this dataset are, they are all companies who are probably more or less born in the cloud or at least have some arm that is born in the cloud, and they are building software, right? So, they're not really just your average enterprises you might see in a Gartner client base which is more broad; they are software companies.And for software companies, delivering software faster is the most important thing, right, and then delivering secure software faster, should be the most important thing, but it's kind of like the other thing that we talk about and don't do. And that's actually what we found. We found that people do deliver software faster because of containers and cloud, but they don't necessarily deliver secure software faster because as is one of our data points, 75% of containers that run in production have critical or high vulnerabilities that have a patch available. So, they could have been fixed but they weren't fixed. And people ask why, right? And why, well because it's hard; because it takes time; because something else took priority; because I've accepted the risk. You know, lots of reasons why.Corey: One of the big challenges, I think, is that I can walk up and down the expo hall at the RSA Conference, which until somewhat recently, you were not allowed to present that or exhibit at unless you had the word ‘firewall' in your talk title, or wound up having certain amounts of FUD splattered across your banners at the show floor. It feels like there are 12 products—give or take—for sale there, but there are hundreds of booths because those products have different names, different messaging, and the rest, but it all feels like it distills down to basically the same general categories. And I can buy all of those things. And it costs an enormous pile of money, and at the end of it, it doesn't actually move the needle on what my business is doing. At least not in a positive direction, you know? We just set a giant pile of money on fire to make sure that we're secure.Well, great. Security is never an absolute, and on top of that, there's always the question of what are we trying to achieve as a business. As a goal—from a strategic perspective—security often looks a lot like, “Please let's not have a data breach that we have to report to people.” And ideally, if we have a lapse, we find out about it through a vector that is other than the front page of The New York Times. That feels like it's a challenging thing to get prioritized in a lot of these companies. And you have found in your report that there are significant challenges, of course, but also that some companies in some workloads are in fact getting it right.Anna: Right, exactly. So, I'm very much in line with your thinking about this RSA shopping spree, and the reality of that situation is that even if we were to assume that all of the products you bought at the RSA shopping center were the best of breed, the most amazing, fantastic, perfect in every way, you would still have to somehow build a program on top of them. You have to have a process, you have to have people who are bought into that process, who are skilled enough to execute on that process, and who are more or less in agreement with the people next door to them who are stuck using one of the 12 trinkets you bought, but not the one that you're using. So, I think that struggle persists into the cloud and may actually be worse in the cloud because now, not only are we having to create a processor on all these tools so that we can actually do something useful with them, but the platform in which we're operating is fundamentally different than what a lot of us learned on, right?So, the priorities in cloud are different; the way that infrastructure is built is a little different, like, you have to program a YAML file to make yourself an instance, and that's kind of not how we are used to doing it necessarily, right? So, there are lots of challenges in terms of skills gap, and then there's just this eternal challenge of, like, how do we put the right steps into place so that everybody who's involved doesn't have to suffer, right, and that the thing that comes out at the end is not garbage. So, our approach to it is to try to give people all the pieces they need within a certain scope, so again, we're talking about people developing software in a cloud-native world, we're focused kind of on containers and cloud workloads even though it's not necessarily containers. So that's, like, our sandbox, right? But whoever you are, right, the idea is that you need to look to the left—because we say ‘shift left'—but then you kind of have to follow that thread all the way to the right.And I actually think that the thing that people most often neglect is the thing on the right, right? They maybe check for compliance, you know, they check configurations, they check for vulnerabilities, they check, blah, blah, blah, all this checking and testing. They release their beautiful baby into the world, and they're like, okay, I wash my hands of it. It's fine. [laugh]. Right but—Corey: It has successfully been hurled over the fence. It is the best kind of problem, now: Someone else's.Anna: It's gone. Yeah. But it's someone else's—the attacker community, right, who are now, like, “Oh, delicious. A new target.” And like, that's the point at which the fun starts for a lot of those folks who are on the offensive side. So, if you don't have any way to manage that thing's security as it's running, you're kind of like missing the most important piece, right? [laugh].Corey: One of the challenges that I tend to see with a lot of programmatic analysis of this is that it doesn't necessarily take into account any of the context because it can't. If I have, for example, a containerized workload that's entire job is to take an image from S3, run some analysis or transformation on it then output the results of that to some data store, and that's all it's allowed to talk to you, it can't ever talk to the internet, having a system that starts shrieking about, “Ah, there's a vulnerability in one of the libraries that was used to build that container; fix it, fix it, fix it,” doesn't feel like it's necessarily something that adds significant value to what I do. I mean, I see this all the time with very purpose-built Lambda Functions that I have doing one thing and one thing only. “Ah, but one of the dependencies in the JSON processing library could turn into something horrifying.” “Yeah, except the only JSON it's dealing with is what DynamoDB returns. The only thing in there is what I've put in there.”That is not a realistic vector of things for me to defend against. The challenge then becomes when everything is screaming that it's an emergency when you know, due to context, that it's not, people just start ignoring everything, including the, “Oh, and by the way, the building is on fire,” as one of—like, on page five, that's just a small addendum there. How do you view that?Anna: The noise insecurity problem, I think, is ancient and forever. So, it was always bad, right, but in cloud—at least some containers—you would think it should be less bad, right, because if we actually followed these sort of cloud-native philosophy, of creating very purp—actually it's called the Unix philosophy from, like, I don't know, before I was born—creating things that are fairly purposeful, like, they do one thing—like you're saying—and then they disappear, then it's much easier to know what they're able to do, right, because they're only able to do what we've told them, they're able to do. So, if this thing is enabled to make one kind of network connection, like, I'm not really concerned about all the other network connections it could be making because it can't, right? So, that should make it easier for us to understand what the attack surface actually is. Unfortunately, it's fairly difficult to codify and productize the discovery of that, and the enrichment of the vulnerability information or the configuration information with that.That is something we are definitely focusing on as a vendor. There are other folks in the industry that are also working on this kind of thing. But you're exactly right, the prioritization of not just a vulnerability, but a vulnerability is a good example. Like, it's a vulnerability, right? Maybe it's a critical or maybe it's not.First of all, is it exposed to the outside world somehow? Like, can we actually talk to this system? Is it mitigated, right? Maybe there's some other controls in place that is mitigating that vulnerability. So, if you look at all this context, at the end of the day, the question isn't really, like, how many of these things can I ignore? The question is at the very least, which are the most important things that I actually can't ignore? So, like you're saying, like, the buildings on fire, I need to know, and if it's just, like, a smoldering situation, maybe that's not so bad. But I really need to know about the fire.Corey: This episode is sponsored in part by LaunchDarkly. Take a look at what it takes to get your code into production. I'm going to just guess that it's awful because it's always awful. No one loves their deployment process. What if launching new features didn't require you to do a full-on code and possibly infrastructure deploy? What if you could test on a small subset of users and then roll it back immediately if results aren't what you expect? LaunchDarkly does exactly this. To learn more, visit launchdarkly.com and tell them Corey sent you, and watch for the wince.Corey: It always becomes a challenge of prioritization, and that has been one of those things that I think, on some level, might almost cut against a tool that works at the level that Sysdig does. I mean, something that you found in your report, but I feel like, on some level, is one of those broadly known, or at least unconsciously understood things is, you can look into a lot of these tools that give incredibly insightful depth and explore all kinds of neat, far-future, bleeding edge, absolute front of the world, deep-dive security posture defenses, but then you have a bunch of open S3 buckets that have all of your company's database backups living in them. It feels like there's a lot of walk before you can run. And then that, on some level, leads to the wow, we can't even secure our S3 buckets; what's the point of doing anything beyond that? It's easy to, on some level, almost despair, want to give up, for some folks that I've spoken to. Do you find that is a common thing or am I just talking to people who are just sad all the time?Anna: I think a lot of security people are sad all the time. So, the despair is real, but I do think that we all end up in the same solution, right? The solution is defense in depth, the solution is layer control, so the reality is if you don't bother with the basic security hygiene of keeping your buckets closed, and like not giving admin access to every random person and thing, right? If you don't bother with those things, then, like, you're right, you could have all the tools in the world and you could have the most advanced tools in the world, and you're just kind of wasting your time and money.But the flipside of that is, people will always make mistakes, right? So, even if you are, quote-unquote, “Doing everything right,” we're all human, and things happen, and somebody will leave a bucket open on accident, or somebody will misconfigure some server somewhere, allowing it to make a connection it shouldn't, right? And so if you actually have built out a full pipeline that covers you from end-to-end, both pre-deployment, and at runtime, and for vulnerabilities, and misconfigurations, and for all of these things, then you kind of have checks along the way so that this problem doesn't make it too far. And if it does make it too far and somebody actually does try to exploit you, you will at least see that attack before they've ruined everything completely.Corey: One thing I think Sysdig gets very right that I wish this was not worthy of commenting on, but of course, we live in the worst timeline, so of course it is, is that when I pull up the website, it does not market itself through the whole fear, uncertainty, and doubt nonsense. It doesn't have the scary pictures of, “Do you know what's happening in your environment right now?” Or the terrifying statistics that show that we're all about to die and whatnot. Instead, it talks about the value that it offers its customers. For example, I believe its opening story is, “Run with confidence.” Like, great, you actually have some reassurance that it is not as bad as it could be. That is, on the one hand, a very uplifting message and two, super rare. Why is it that so much of the security industry resorts to just some of the absolute worst storytelling tactics in order to drive sales?Anna: That is a huge compliment, Corey, and thank you. We try very hard to be kind of cool in our marketing.Corey: It shows. I'm tired of the 1990s era story of, “Do you know where the hackers are?” And of course, someone's wearing, like, a ski mask and typing with gloves on—which is always how I break into things; I don't know about you—but all right, we have the scary clip art of the hacker person, and it just doesn't go anywhere positive.Anna: Yeah. I mean, I think there certainly was a trend for a while have this FUD approach. And it's still prevalent in the industry, in some circles more than others. But at the end of the day, Cloud is hard and security is hard, and we don't really want to add to the suffering; we would like to add to the solution, right? So, I don't think people don't know that security is hard and that hackers are out there.And you know, there's, like, ransomware on the news every single day. It's not exactly difficult to tell that there's a challenge there, so for us to have to go and, like, exacerbate this fear is almost condescending, I feel, which is kind of why we don't. Like, we know people have problems, and they know that they need to solve them. I think the challenge really is just making sure that A) can folks know where to start and how to build a sane roadmap for themselves? Because there are many, many, many things to work on, right?We were talking about context before, right? Like, so we actually try to gather this context and help people. You made a comment about how having a lot of telemetry might actually be a little bit counterproductive because, like, there's too much data, what do I do well—Corey: Here's the 8000 findings we found that you fail—great. Yeah. Congratulations, you're effectively the Nessus report as a company. Great. Here you go.Anna: Everything is over.Corey: Yeah.Anna: Well, no shit, Nessus, you know. Nessus did its thing. All right. [laugh].Corey: Oh, Nessus was fantastic. Nessus was—for those who are unaware, Nessus was an open-source scanner made by the folks at Tenable, and what was great about it was that you could run it against an environment, it would spit out all the things that it found. Now, one of the challenges, of course, is that you could white-label this and slap whatever logo you wanted on the top, and there were a lot of ‘security consultancies' that use the term incredibly… lightly, that would just run a Nessus report, drop off the thick print out. “Here's the 800 things you need to fix. Pay me.” And wander on off into the sunset.And when you have 800 things you need to fix, you fix none of them. And they would just sit there and atrophy on the shelf. Not to say that all those things weren't valid findings, but you know, the whole, you're using an esoteric, slightly deprecated TLS algorithm on one of your back-end services, versus your Elasticsearch database does not have a password set. Like, there are different levels of concern here. And that is the problem.Anna: Yeah. That is in fact one of the problems we're aggressively trying to solve, right? So, because we see so much of the data, we're actually able to piece together a lot of context to gives you a sense of risk, right? So, instead of showing all the data to the customer—the customer can see it if they want; like, it's all in there, you can look at it—one of the things we're really trying to do is collect enough information about the finding or the event or the vulnerability or whatever, so we can kind of tell you what to do.For example, one you can do this is super basic, but if you're looking at a specific vulnerability, like, let's say it's like Log4j or whatever, you type it in, and you can see all your systems affected by this thing, right? Then you can, in the same tool, like, click to the other tab, and you can see events associated with this vulnerability. So, if you can see the systems that the vulnerability is on and you can see there's weird activity on those systems, right? So, if you're trying to triage some weird thing in your environment, during the Log4j disaster, it's very easy for you to be like, “Huh. Okay, these are the relevant systems. This is the vulnerability. Like, here's all that I know about this stuff.”So, we kind of try to simplify as much as possible—my design team uses the word ‘easify,' which I love; it's a great word—to easify, the experience of the end-user so that they can get to whatever it is they're trying to do today. Like, what can I do today to make my company more secure as quickly as possible? So, that is sort of our goal. And all this huge wealth of information we gather, we try to package for the users in a way that is, in fact, digestible. And not just like, “Here's a deluge of suffering,” like, “Look.” [laugh]. You know?Corey: This is definitely complicated in the environment I tend to operate in which is almost purely AWS. How much more complex is get when people start looking into the multi-cloud story, or hybrid environments where they have data center is talking to things within AWS? Because then it's not just the expanded footprint, but the entire security model works slightly differently in all of those different environments as well, and it feels like that is not a terrific strategy.Anna: Yeah, this is tough. My feelings on multi-cloud are mostly negative, actually.Corey: Oh, thank goodness. It's not just me.Anna: I was going to say that, like, multi-cloud is not a strategy; it's just something that happens to you.Corey: Same with hybrid. No one plans to do hybrid. They start doing a cloud migration, realize halfway through some things are really hard to move, give up, plant the flag, declare victory, and now it's called hybrid.Anna: Basically. But my position—and again, as an analyst, you kind of, I think, end up in this position, you just have a lot of sympathy for the poor people who are just trying to get these stupid systems to run. And so I kind of understand that, like, nothing's ideal, and we're just going to have to work with it. So multi-cloud, I think is one of those things where it's not really ideal, we just have to work with it. There's certainly advantages to it, like, there's presumably some level of mythical redundancy or whatever. I don't know.But the reality is that if you're trying to secure a pile of junk in Azure and a pile of junk in AWS, like, it'd be nice if you had, like, one tool that told you what to do with both piles of junk, and sometimes we do do that. And in fact, it's very difficult to do that if you're not a third-party tool because if you're AWS, you don't have much incentive to, like, tell people how to secure Azure, right? So, any tool in the category of, like, third-party CSPM—Gartner calls them CWPP—kind of, cloud security is attempting to span those clouds because they always have to be relevant, otherwise, like, what's the point, right?Corey: Well, I would argue cynically there's also the VC model, where, “Oh, great. If we cover multiple cloud providers, that doubles or triples our potential addressable market.” And, okay, great, I don't have those constraints, which is why I tend to focus on one cloud provider where I tend to see the problems I know how to solve as opposed to trying to conquer the world. I guess I have my bias on that one.Anna: Fair. But there's—I think the barrier to entry is lower as a security vendor, right? Especially if you're doing things like CSPMs. Take an example. So, if you're looking at compliance requirements, right, if your team understands, like, what it means to be compliant with PCI, you know, like, [line three 00:28:14] or whatever, you can apply that to Azure and Amazon fairly trivially, and be like, “Okay, well, here's how I check in Azure, and here's how I check in Amazon,” right?So, it's not very difficult to, I think, engineer that once you understand the basic premise of what you're trying to accomplish. It does become complicated as you're trying to deal with more and more different cloud services. Again, if you're kind of trying to be a cloud security company, you almost have no choice. Like, you have to either say, “I'm only doing this for AWS,” which is kind of a weird thing to do because they're kind of doing their own half-baked thing already, or I have to do this for everybody. And so most default to doing it for everybody.Whether they do it equally well, for everybody, I don't know. From our perspective, like, there's clearly a roadmap, so we have done one of them first and then one of them second and one of the third, and so I guarantee you that we're better in some than others. So, I think you're going to have pluses and minuses no matter what you do, but ultimately what you're looking for is coverage of the tool's capabilities, and whether or not you have a program that is going to leverage that tool, right? And then you can check the boxes of like, “Okay. Does it do the AWS thing? Does it do this other AWS thing? Does it do this Azure thing?”Corey: I really appreciate your taking the time out of your day to speak with me. We're going to throw a link to the report itself in the [show notes 00:29:23], but other than that, if people want to learn more about how you view these things, where's the best place to find you?Anna: I am—rarely—but on Twitter at @aabelak. I am also on LinkedIn like everybody else, and in the worst case, you could find me by email, at anna.belak@sysdig.com.Corey: And we will of course put links to that in the [show notes 00:29:44]. Thank you so much for taking the time to speak with me today. I appreciate it.Anna: Thanks for having me, Corey. It's been fun.Corey: Anna Belak, Director of Thought Leadership at Sysdig. I'm Cloud Economist Corey Quinn and this is streaming on the cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment telling me not only why this entire approach to security is awful and doomed to fail, but also what booth number I can find you at this year's RSA Conference.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

About LynnCloud Architect who codes, Angel InvestorLinks: Lynn Langit Consulting: https://lynnlangit.com/ Groove Capital: https://www.groovecap.com/groove-capital-minnesotas-first-check-fund Twitter: https://twitter.com/lynnlangit GitHub: https://github.com/lynnlangit TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Today's episode is brought to you in part by our friends at MinIO the high-performance Kubernetes native object store that's built for the multi-cloud, creating a consistent data storage layer for your public cloud instances, your private cloud instances, and even your edge instances, depending upon what the heck you're defining those as, which depends probably on where you work. It's getting that unified is one of the greatest challenges facing developers and architects today. It requires S3 compatibility, enterprise-grade security and resiliency, the speed to run any workload, and the footprint to run anywhere, and that's exactly what MinIO offers. With superb read speeds in excess of 360 gigs and 100 megabyte binary that doesn't eat all the data you've gotten on the system, it's exactly what you've been looking for. Check it out today at min.io/download, and see for yourself. That's min.io/download, and be sure to tell them that I sent you.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance query accelerator for the Oracle MySQL Database Service, although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLAP and OLTP—don't ask me to pronounce those acronyms again—workloads directly from your MySQL database and eliminate the time-consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. So, I've been doing this podcast for a little while now—by my understanding, this is episode 300 and something—but back when the very first episode aired, I had pre-recorded the first twelve episodes. Episode number ten was with Lynn Langit who is, among many other things, the CEO of Lynn Langit Consulting, she is also the first person to achieve the AWS Community Hero and equivalent designations at all three of the primary tier-one hyperscale cloud providers, which I can't even wrap my head around what it takes to get that at one of those companies. Lynn, thank you so much for agreeing to come back now that I'm no longer scared of the microphone.Lynn: Well, thank you for having me. It's great to be back, Corey.Corey: So, it's been a few years now since we really sat down and caught up. And what an interesting few years it's been. There's been a whole minor global pandemic thing that wound up hitting us from unexpected and unpleasant places. There's been a significant, I would say, not revolution but evolution in how adoption of cloud services has been proceeding. The types of problems that customers are encountering, the conversational discourse has moved significantly away from, “Should we be using cloud?” Into, “Okay, we obviously should be using Cloud. How should we be using it?” And the industry keeps on churning. Sure there's still rough parts, there are still ridiculous aspects of it, but what have you been up to?Lynn: Well, as you might remember, I have an independent consultancy where I do really what my customers need. I work across different clouds, which keeps it interesting and fun, but I've had a focus over the past few years in supporting bioinformatics research. Before the pandemic, it was mostly cancer research. Since the pandemic, it's been all Covid, all the time.Corey: All Covid, all the time sort of has been the unofficial theme of this. And it's weird. I know, we're in 2022, now, but it still feels like on some level, it's like, “Man, this is March 2020; it's still dragging on, on some level.” There have been a number of stories in the world that is, let's say medicine-adjacent, more so than—we're all sort of medicine adjacent these days, but there's been a lot of refocusing away from things like cancer research into Covid and similar pandemic respiratory diseases. Do you think that there's a longer-term story where we're going to start seeing progress stall on things that were previously areas of focus—in your case cancer—in favor of reducing infectious disease, or is it really one of those ‘rising tide lifts all boats' type of scenarios?Lynn: Yeah, it's the latter. It's been really interesting. Without getting too much into the details, you know, you think of genomic research for drug discovery, you know, we started with this idea of different DNA sequencing cohorts. So, like people from the—you know, that started from the United States, people that started from Africa, you know, different cohort as a normative to evaluate the effectiveness of diseases, what was an area of research already was to go down to the level of what's called single-cell RNA. So, look at the expression of the genomics by cell area, so by the different parts of your body.Well, this is similar to what has been done to understand the impact and the efficacy of potential Covid drugs. So, this whole single-cell RNA mapping cohorts of what is normal for different types of populations has resulted in this data explosion that I've never seen before. And I see it as positive for the impact of human health. However, it really drives the need for adoption to the cloud. These research facilities are running out of space if they're still working on-prem.Corey: I spend an awful lot of time thinking about data and its storage from a primarily cost-focused perspective, for obvious reasons, and that is nuanced and intricate and requires, sort of, an end-to-end lifecycle policy. There's this idea of, ideally, you would delete old data you don't need anymore, but failing that you, maybe aspirationally, don't need 500 copies of the same thing lying around. Maybe there are ways to fix that. And that's all within one cloud ecosystem. You work across all of the clouds. How do you keep it all straight in your head trying to figure out things around lifecycles, things around just understanding the capabilities of the various platforms? Because I got to say, from my perspective, it's challenging enough only bounding it to one.Lynn: Yeah, it's the constant problem. The big clients I had over this past year were not on Amazon, they were on other platforms. So, it seems like it sort of goes in cycles. And what I'll sometimes need to do is hire subcontractors that have been working on those platforms because you can't, I mean, you can't even know one platform, much less all of them to the level of complexity in order to implement. One thing that is kind of interesting though, in bioinformatics is—and different than the other domains—is when you talk about data, it's a function of time first and cost second.So, they will run on less computational resources, so that they can, for example, not overspend their research grant, and wait longer for the results. And this has been really an interesting shift in my work because I used to work with FinTech and ad tech, where it's all about, get it out there fast. And we don't really care how much it costs, we just want it super fast. So, this continuum of time or money shifts by vertical. And that's been something that—I don't know, it's kind of obvious, in hindsight, but I didn't really expect until I got into the different domains.Corey: It's always been fascinating to me watching how different organizations and different organization types wind up have interacting with cost. I mean, I've been saying for a while now that cost and architecture are the same thing when it comes to cloud. What are your trade-offs? What are your constraints? In many venture-backed companies, it's when you have a giant pile of other people's money raring to go, and it's a spend it and hit your milestone if you want to get another round of funding, or this has been an incredible journey Medium post in the making, then, yeah, okay, go ahead and make the result happen faster. Save money is not the first, second or third order of business as far as what you're trying to achieve.In academia, where everything's grant powered. And it's a question of, we need to be able to deliver, and we need to be able to show results and be able to go and play the game and understand the cultural context we're operating in, and ideally get another grant next year, it completely shifts the balance of what needs to be prioritized and when. And I don't think there's been a lot of discussion around that because most cloud cost discussions inherently center around industry.Lynn: They do and they focus on the industries where they're willing to spend most. So, most of the reference examples are, they always prioritize for time and money is sort of unlimited. I'll give you an example—this was from a few years back—some work I did with a research group in Australia, and again, it was a genomics example. They were running on-prem, and to do a single query, it took them 500 hours. And I was just like, “Are you kidding me?”And they're like, “Hey, cloud lady, what can you do?” Right? So, we gave two solutions, and the first solution was kind of a more of a lift-and-shift kind of a solution because they didn't know anything about cloud. And it took a few hours. The second solution was what was in our opinion, super elegant, it was one of the earliest data lakes, it took minutes.Well, it was a big hit to the ego that they adopted… the easier solution. But again, it's a learning because another dimension about cloud architecture is usability. The FinTechs are like, “We're going to get it really done fast; we'll hire who we need to hire.” The biotechs, they can't afford to hire who they need to hire because there all being hired by the FinTechs. So, you have these different dimensions you need to optimize for that aren't really obvious if you just work in the industries that optimize for time.Corey: And the thing that always gets overlooked is that in most environments, the people working on things are more expensive than the infrastructure themselves. And back when Lambda and all the serverless joy came out, my first iteration of lastweekinaws.com website was powered entirely by Lambda functions, S3, and other assorted bits of nonsense. Today, it's on WordPress.And it's not because I think that is somehow the superior architecture from a purely technologist point of view, but because I have to find other people who aren't me or one of the other six people in the world at the time who could stuff all that into their head and work on it effectively, should be able to make changes to the website. That is not something I need to be focusing on. There's something to be said for going to where there's a significant talent pool, rather than pushing the frontiers of innovation in areas that don't directly benefit whatever it is your organization is targeting.Lynn: Yeah, it's really interesting, when Covid hit back in 2020—kind of an interesting little story here—one of my clients is the Broad Institute at MIT and Harvard—they're a well-known research organization for, you know, cancer genomic datasets—they were tasked with pivoting their labs so that they could provide Covid testing capability. And I was a long-term contractor with them, so they brought me in for an architectural cloud consultant. I said, “This clearly is a serverless. I know you guys haven't done this before, but this is going to be burstable, you don't know how big this is going to need to go.” And then just to make life interesting, in the middle of the build of that, I was one of the first people in Minnesota to get Covid, so I actually wasn't able to go and complete it, nor was I able to get a test because there weren't tests.I mean, you know, I can't make this stuff up. I was in the ER saying, “Okay, is this the end of me, or can I go back and get you some tests?” [laugh]. So, it's really kind of two things—kind of a weird story. And also, life situations will cause change, and so the Broad did launch that pipeline, and it was serving up to 10% of the Covid tests in the United States.But they had never done anything serverlessly or had considered it before because they didn't need to have that amount of change. It was really, again, a big thing when I came into human health. Prior to that, I was doing all serverless all the time. You know, I came into human health, and they were saying, “Okay, we're going to have massive VMs.” And I was like, “No…” but you know, you have to meet the client where they are.Corey: I think it's the easiest thing in the world, particularly as a junior consultant—because you do not see senior consultants doing this ever, you know, after the first time—to walk into an environment, look around and have zero context into what's going on—because you're a consultant; you haven't been there and say, “This is ridiculous. What fool built this?” Invariably, to said fool. Now, most people don't show up in the morning hoping to do a terrible job at work today, so there are constraints that you are certainly not seeing. And maybe it was an offering wasn't available that maybe they weren't aware of it. Maybe there was a constraint that you're not seeing.But the best case is you're right and you just made them feel terrible, which is not generally a great way to land more consulting projects. It's always frustrating to me because even looking at a bill and having a pretty good idea of what's going on, I always frame it as, “Can you help me understand why this is the case? Had you considered this, or is that not an option?” As opposed to categorically saying, well, this is not the way to do it. Because once you're wrong when you're delivering expertise, it takes a lot to build that back, if it's even possible.Lynn: Well, again, from human health because, you know, they were consuming the vendor information, they thought they wanted to learn how to use Kubernetes, but what they really needed to learn was how to do archiving to reduce their storage costs.Corey: Yes. Kubernetes is a terrific solution for a bunch of problems and create several orders of magnitude more somewhere along the way. My somewhat accurate, somewhat snarky observation is that Kubernetes is great if your primary problem is you want to pretend you work at Google but didn't pass their technical screen. I don't really want to cosplay as a cloud provider myself, most days. That said, there are use cases for which it makes sense, but context is everything, and generally speaking, I don't tend to follow a hype trend to figure out whether or not it's going to solve my particular problem.Lynn: Well, here's the soundbite: “Kubernetes is today's Hadoop.”Corey: Oh, there are people who are not going to like that. I made a tweet, I think—Lynn: Tough.Corey: —three years ago now—Lynn: It's true. [laugh].Corey: Oh, yeah. Tweet three years ago or so that said, “Hot take: In five years, nobody's going to care about Kubernetes.” And I think I have a year or two left on that prediction. And what I said at the time was that not that it's going to go away and not be anywhere—because enterprises do not move that quickly—but it's no longer going to be the sort of thing that everyone is concerned about at a very high level. The Linux kernel has a bunch of aspects to it that we used to have to care about a fair bit. Now, a few people really, really need to care about those things; because of those folks' hard work, the rest of us don't have to think about it at all. And that is the nature of technology, in the fullness of time.Lynn: Well, another way to think about it is Kubernetes is a C++. Certain people are going to be experts in it and need to, and that's valid, right, but what percentage of developers code in C++. Like, ten? Five? You know, it's kind of analogous, right?So, it's one of the signatures of my consultancy. You know, I'm this pragmatic midwesterner, and I love to say, “Look,”—like you said—“If you think you need this, you really need to understand the actual cost of it because it's non-trivial on all clouds.” And I get to say that because I'm independent. You know, they're doing solid work to abstract it into a higher-level implementation, but when I hear a customer say, “I need Kubernetes,” the burden of proof is on them [laugh] before I'm going to build that.Corey: Speaking of hype-driven emerging technologies, you are arguably one of the few people on the planet I can have this conversation with, and I do not mean that as an insult other people operating in this space. For context, a couple of years ago, AWS launched Brakets—which they spelled Braket without a C because it's Amazon and spelling is hard, presumably; I know, I know, there's a reason behind it—and it is their service that enables you to get access to quantum computers the same way we get access to any other AWS service: Through a somewhat janky console and some APIs. And, okay, quantum computing. We've heard a lot about it forever; it always seemed a bit like science fiction and it was never really clearly articulated what kind of value it can solve for us.So, “Aha, now it's here. I don't need to go and build or buy a quantum computer somewhere else.” And I tried using the Quickstart, and it turns out that the Hello World tutorial for quantum computing—at least to my mind—is basically an application for a PhD program at Berkeley. And I am not that type of academic for better or worse, so I kept smacking my head off of that and realizing, okay, whatever this is, is clearly not for me. You have been doing some deep dives in the quantum computing space, but as we've just mentioned, your day job is not, to my understanding, a college professor. You are a consultant, you run your own consultancy, solving data problems, particularly towards bioinformatics. What is the deal—to the layperson—of quantum computing these days?Lynn: Well, yeah, like you, I was introduced years ago and tried to read the books, and I didn't have the math and just, you know, saw it as a curiosity. Last year, I picked up a book from O'Reilly called Practical Quantum Computing, which of course, because the name was attractive to me. I read it, felt like I was getting a little bit more knowledge, implemented a learning JavaScript library with a browser-based editor—so zero-install—and it was a simulator, you couldn't run it on actual QPUs. So, I decided to see if there's any other interest in my tech community, and I got about five other developers and we ran a 15-week long book club because we all just wanted to move forward with our knowledge. Because there is this fundamental difference in the information you can get from a qubit versus a bit because a qubit can basically be, like, a globe, and so it has a superposition, and so you can have all the different mathematical points on the globe, versus a bit is on or off.I mean, that's intuitive, like, “Hey, I could get more information out of that.” So, the potential usages—it's always been tech that leads the way—is on figuring out of what are called NP-hard or computationally complex problems, and, again, this is at the edge of my knowledge, but this is where bioinformatics is. I think of it in an oversimplified way, as [N by N by N by N, all by all by all 00:16:49]. We want to see all possible combinations of all possible inputs. So, for example, we can figure out which Covid drug we should try—which set of drugs we should try—and we want that as fast as possible.So, I wanted to see, okay, you know, where's this at? Plus, like you said, Amazon introduced Braket; when Amazon introduces something, then there's some customers somewhere that are using it. I mean, that's—you know, kind of pay attention to it now. So, as I was doing this book club, I investigated all the different cloud vendors and captured all that learning in a GitHub, and just recently recorded a LinkedIn Learning course. Which again, in the learning ladder is, if this is, you know, Hello World and this is actual implementation, it's like right here.But right here doesn't exist. Like, there's nothing there, so I tried to make something to say, okay, the Amazon Braket example, how does that actually work? What is a Hadamard Gate? Why do you care? What is amplification? How do you measure it? Like, what would you do with that? And so, you know, I tried to interpret some academic papers and do that learning layer in the middle to help move people towards productivity. Am I fully there? No. Did I move further? I hope so. Do you want to come along with me? Great.Corey: You've done something, though, that I don't think anyone else yet has when I had conversations with them about quantum computing, which is we all are shaped by our own needs and our own experiences when we interact with a cloud provider. To me, I, perhaps foolishly, took Amazon seriously when they called it Amazon Web Services. “Oh, okay. Clearly, this is going to be things to help me build websites and website accessories, more or less.” So, it's always odd to me when I'll see something like oh, and here's our IoT solution that winds up powering a fleet of 10,000 robots, and I'm looking around my website going, “I don't really have a problem that could be solved by the 10,000 robots. I have a bunch that could be made a lot worse.”But it feels like it's this orthogonal thing that is removed. But some areas, it's okay. I can see the points of commonality and how you get there from here, and if I think really hard, I can do that with IoT stuff. For example, iRobot is a cloud-connected robot that talks to something that looks like a website and vacuums my house. Whereas with quantum computing, it always felt very isolated, very much an island as far as being connected to anything else that I can recognize. Bioinformatics research, as you describe it, well, yeah, I can see you get the bioinformatics research from web services. And now I can see how you can get to quantum computing through the bioinformatics side of things.Lynn: Well, the other thing that really was useful for me, I am doing TensorFlow, finally. Took me a few years, but for neural networks. And so I am using, with some of my bioinformatics clients, acceleration with GPUs and TPUs, if I happen to be on Google because it's a known thing that when you're training a neural network, again, similar you have complexity, so you have a specialized chip, where you can offload some of the linear algebra onto that chip. So, you split the classic and the tensor portion, if you will, and you do computation on both sides. And so it's not a huge leap to say, “Well, I'm not going to use a GPU, I'm going to use a QPU,” because you split. And that's the way it actually works.There's actually a really interesting paper I put in my GitHub. It is a QCNN, and it is—that's a Quantum Convolutional Neural Network that is used to analyze images of breast cancer. Because again, on the image, you can think of the pixels as what's called a tensor, which is just vectors in multiple dimensions, you need the [all by all by all 00:20:17] again; that's really how it goes in my head. You know, you have the globe of the qubit and you want to get the all possible combinations faster, so that you can analyze all combinations in the, in this case, the image. And they found, not only was it faster, it was more accurate. And that's why I am interested in this.Corey: Couchbase Capella Database-as-a-Service is flexible, full-featured and fully managed with built in access via key-value, SQL, and full-text search. Flexible JSON documents aligned to your applications and workloads. Build faster with blazing fast in-memory performance and automated replication and scaling while reducing cost. Capella has the best price performance of any fully managed document database. Visit couchbase.com/screaminginthecloud to try Capella today for free and be up and running in three minutes with no credit card required. Couchbase Capella: make your data sing.Corey: The neat part is that this might be one of the first clear-cut stories where, “What could I use a quantum computer for?” And the answer isn't something that's forward-looking or theoretical. I mean, the obvious gag when you said reading about Practical Quantum Computing is that book is probably in pre-release, I would assume.Lynn: [laugh].Corey: But it's a hard thing to solve for, and I do have the awareness that I am not an academic, academia has never been my friend, so I bias heavily for, “Well, can we use this to solve real-world problems slash make money?”—because industry—and academia focuses, ideally and aspirationally on the expansion of the limits of human knowledge. And sometimes it's okay to do those things without an immediate, “Well, how can I turn a profit on it next quarter?” What a dismal, bleak society we have if that's all that we wind up focusing on any given point in time.Lynn: Yeah, that's for sure.Corey: Which, of course, sets us up for one other thing that's a relatively recent change for you. You now have mentioned in your bio, which I believe is new since the last time we spoke, that you are an angel investor. And that is something that I recently found being applied to me as well after I made an investment in a startup that I was very excited about. I talked about in the show previously; it's called Byte Check. But honestly, I didn't realize that what I was doing was called angel investing until I read the press release because ‘strategic angel' are two words that no one ever applies to me, particularly in that order. What happened? What are you doing these days?Lynn: Well, I live in Minneapolis. So—and I moved there in 2019, so you know, my 2020 story is first I had Covid, got over that, and then I was there during the tragedy of George Floyd. So, I wanted to understand more about what were the root causes, and what I could do to make an impact in the recovery of my city. And I was really surprised to find that Minnesota is one of the most charitable states in the United States, it ranks one or two, but yet we have in the Twin Cities of Minneapolis and St. Paul, we have really unacceptable income inequality and poverty. So, something's not working.I'm a pretty charitable person; I always allocate a certain percentage of my money to charity, but I said, “I want to accelerate this.” So, at the same time, there was a new angel investment fund launched, it's called Groove Capital, that was going to focus on women-owned and BIPOC businesses. And I thought, “Hmm, this seems good.”Now, I was super intimidated because I lived in California for so many years, and check sizes in California, you just add a zero. And I thought, you know, “I don't have generational wealth. This is my own money.” You know, I'm well-compensated, but I'm not loaded.Corey: Yeah there's a common trope right now that oh, angel investor is a polite way of saying I am rich—Lynn: Right.Corey: —but I rent my home at this point, living in San Francisco. It is, I am not exactly sitting here diving into a money bin out back, Scrooge McDuck-style either.Lynn: Right. Well, I mean, you know, I'll just be transparent about it. Like everybody else, or many people, I moved out of California because of the cost of doing business there and reduced my cost of living by 40% move into the Midwest, which is awesome. So anyway, I joined this fund, and it's been just fantastic because I've listened to deals on my own and felt just like a complete, like, I don't know what I'm doing. But I'm taking advantage—Corey: How do you evaluate an idea that someone has that's early-stage, barely better in some cases than back-of-an-envelope scrawlings?Lynn: For sure, right. But what I found through the fund is I can contribute both money and time because, you know, I did this cloud expertise, and in addition to writing checks for a couple companies that I really believe in, for example, I got all these companies on the X cloud company for startups program. Because that wasn't just a known thing in my ecosystem. I was like, “Why are you paying a cloud bill? You could be on the startup program for the first year.”So, I'm impacting these new businesses with both my experience and my dollars, and I just really love it. I just really, really love it. And you know, the reasons I want to talk about it is because more people who have expertise in tech should do this because you can really, really be impactful. One of the companies that I invested in is called TurnSignl. They are coming to Los Angeles.It was three attorneys and one of their brothers is a police officer. They wanted to de-escalate situations that happen with traffic stops. So, it's a mobile app, where you push a button and you're connected to an attorney. And they do training for the community and police officers, and the idea to record the conversation and to get an attorney involved to de-escalate and get everybody home safely. And that was my first investment and I'm—it's going national, and I'm like, really, really—the kind of things I want to do you know.Corey: It is simultaneously such a terrific idea and such a stunning indictment of the society that makes something like that necessary.Lynn: Well, you know, we have to find practical solutions. We have to find ways forward.Corey: Oh, please. Don't interpret anything I'm saying a shade on that. It's like, “Well, I wish the world were differently.” Yeah, I think most people do. But you have to deal for better or worse with the hand that you're dealt, and this is, for better or worse, at the time of recording this, the society that we have, and finding the best path forward is often not easy.But it beats just sitting here complaining about everything every day, and not doing anything to be part of that change. The surprising thing I learned as I went through it was that in many cases, the value of individual angel investors is not the check that they're writing, that's basically just almost a formality, on some level. It is the expertise, it is the insight into particular markets, and the rest. The part of what you're saying that surprises me that I hadn't really considered, but of course, it must exist, is the idea of angel funds. Is this generally run by an existing VC firm? Is it a group of like-minded friends who decide, ah, we're going to just basically do the investing equivalent of a giving circle where everyone puts some money in the pot and then that decides where to go? How is it structured?Lynn: Yeah, the way ours worked is you do pay a fee—it's a small fee—to be part of it, and then they have people who vet deals for you. And then what I really like about it is the community aspect because just like in tech, when you're learning something new in tech, you have community, same thing here. We have a Slack, we have a website for each deal, we have in-person meetups when Covid situation allows, and we have chosen to start by investing in Minnesota, although we're going to, in fund two we're going to invest in Upper Midwest. And for example, here's something I would have never known. There's an angel tax credit Minnesota, that for certain businesses, you can get a 25% tax credit. Which hey, do good, be good, get good. I would have never known about that, I would have never known how to do it. All my investments so far have qualified. Fantastic. My money goes further.Corey: Yeah, it's about well, what are you talking about worrying about taxes? That there's about to be doing something good? Yeah, great. If you believe in a cause, take advantage of the tax code as written—I am not advocating tax fraud; pay every cent that you owe, let's be serious here. They have no sense of humor about that—Lynn: [laugh].Corey: —and take advantage of that. That means you have additional money to do good with. I wish that more people had an awareness around that particular school of thought.Lynn: Well, make your money go further, make your money effective.Corey: Oh yes.Lynn: Because like it or not, we run on money. We run on money. And so be smart, from everything where you shop to how you spend. That's how we're going to make change.Corey: One last area I want to explore with you is that for a long time you've been working on, effectively, data pipelines and similar things in that space, tied to your consulting work. You are clearly skilled across all of the various cloud providers and even tieing into the expertise side of what you're doing as an angel investor, you've always been a staunch advocate for, I guess we'll call it doing security the right way. And I've always been tangentially related to security throughout the course of my career. And somewhat recently, I launched another day of my newsletter focused on security within AWS, for folks who are not themselves in the security space of what do you need to know. But so much of it comes down to the do the easy thing now, the right way to do it before you wind up having to do a whole bunch of damage control. And you've been advocating for that since before it was trendy to do so. I imagine you're still somewhat passionate about that perspective.Lynn: Well, I always like to say, you know, Werner Vogels doesn't talk anything about tech; he just talks about, “Please use our security.” And I don't blame him. I mean, you know, I joke that I am an AWS Community Hero because I made a bunch of YouTube videos about securing buckets. And that was, like, seven years ago and I just had a financial client, literally in November, and their buckets, you know, was made public because it was easy for the developer. I'm like, “Ugh, can we just do our foundations?”I don't know why it is not seen as a valuable skill. I mean, I've made craploads of money because people come after they have an incident, but you know, I wish we would be better. And I'm worried because as we start to get more and more of our health information in these big repositories—granted, we have some laws; yay, good—but it's just not valued like coding up a new feature with node or something. And why not? I don't understand.So, I make all these educational resources: I make courses, I have GitHub repos, I have videos. You know, just do it. Plus the people who learned security. I mean, we are always in demand. I'm not a security professional, but I always do security kind of like as a courtesy. And people are like, “Oh, you know, you're great. Oh, my friend needs you.” Dah-dah-dah… I mean, you'll be working forever.Corey: It feels like it's aligned with cost in that it is almost a reactive function. You can spend all your time on it, but it's not going to advance the state of your org further toward its stated goals. You've got to do it, but there's also never really any ‘done' there. It's just easier for me on the cost side because I can very easily quantify the return on investment, whereas with security, it's much more nebulous. And, of course, you wind up with the vendor—I'm going to call it what it is, in some cases—nonsense that is in this space, where, “Oh, you're completely doomed, unless you buy their particular product.” You know, walk up or down the aisle at RSA a few times and your shopping cart is full. And great, are you more secure? You're a lot more complex, but does this get you to a better outcome?And it's, I am so continually frustrated by all of these fancy whiz-bang solutions that are sort of going around the easy stuff—not easy, but it's the baseline level of things: Secure your S3 buckets, or—for users themselves—it's use a password manager that has a strong password on it, use it for everything, use MFA for the important things that you need to use, make sure your email is secure, don't click random nonsense. There's a whole separate pile of things. If I can click the wrong link in an email and it destroys my company, maybe it's not me clicking that link in the email that's the root problem here. Maybe there's an entire security model revisitation that's due. But I'm sorry, I will rant like a loon about the dismal state of security these days, if you let me, and you absolutely should not.Lynn: Well, I would just entreat the audience, basic threat modeling is not complicated. It's like cost modeling. It's just a basic of having successful business on the cloud.Corey: [sigh]. I wish the world work differently than it does, and yet here we are. Lynne, I really want to thank you for taking the time to come on the show a second time. If people want to learn more about what you're up to and talk to you about anything we've discussed, what's the best way to find you?Lynn: So, if you can't find me, you're not looking. I have an internet-easy name. But two places that I'm pretty active: Twitter—just my name, @lynnlangit—and go to my GitHub. In particular, I have a learning cloud kind of meta-repository that has over 100 links to mostly free things on every cloud and just use them. Have at it, learn, be a practitioner, use the cloud more effectively.Corey: And we will, of course, put links to that in the [show notes 00:32:25]. Thanks so much for coming back on. I really appreciate it.Lynn: Thanks for having me. It's been fun.Corey: Lynn Langit, CEO of Lynn Langit Consulting, and oh so much more. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment talking about how security really isn't that important, and right before you submit that comment accidentally type your banking password into the form, too.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

About TylerLifelong learner, passionate coach, obsessed with continuous improvement, avid solver of people puzzles.Links: United Airlines: https://www.united.com/ LinkedIn: https://www.linkedin.com/in/tylerslove/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Couchbase Capella database as a service is flexible, full-featured, and fully managed with built-in access via Key-Value SQL, and full-text search. Flexible JSON documents align to your applications and workloads. Build faster with blazing fast in-memory performance and automated replication and scaling, while reducing costs. Capella has the best price-performance of any fully managed document database. Visit couchbase.com/ScreamingintheCloud to try Capella today for free, and be up and running in 3 minutes. No credit card required. Couchbase Capella make your data sing.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance query accelerator for the Oracle MySQL Database Service, although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLAP and OLTP—don't ask me to pronounce those acronyms again—workloads directly from your MySQL database and eliminate the time-consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Calling this show Screaming in the Cloud has been pretty… easy most of the time because that's mostly what I do: I shake my fist and I yell at clouds. And most companies are okay with that. Today's guest is likely a little bit on the other side of that because when I'm screaming at clouds, it's often out the window, when I'm in a plane.Today, I'm joined by Tyler Slove, who's a Senior Manager in the Enterprise Cloud and DevOps Group at United Airlines, a company I spend way too much time dealing with when we're not in the midst of a global pandemic. Tyler, thank you for joining me.Tyler: Yeah. Thanks for the invite, Corey. Really excited to be here.Corey: So, I want to talk a little bit about, first, how glad I am to finally talk to you because airlines are kind of like computers—and particularly cloud—where when you first see it, it is magic; it is transformative, it's endless possibilities, the power of flight slash instant provisioning of computer resources. Okay, so not everyone is going to find those quite the same way. What's novel today is commonplace tomorrow, and then you get annoyed because your plane is 20 minutes late as it hurls you through the sky to the other side of the planet with the miracle of flight while you're on the internet the whole way. And it's one of those problems where it is sort of definitionally, a thankless job. It is either in the background that just empowers things, or everyone's yelling at you on Twitter. So, given that you work with both sides of that, how do you find that commonality to play out in your world?Tyler: Yeah, it's an interesting thought, and I hadn't necessarily connected the dots before. Because I, like you, are just as frustrated when that flight is, like, 20 minutes delayed. It's like, “Oh, I wanted to be—[laugh]—where I wanted to be at that time.” And, you know, when you think about it, it's actually an ongoing joke I have with one of my mentors. Like, airlines should not work; when you think about the maintenance, the aircraft, the crews, the weather, legal stuff, like, it's amazing how complex they are, and it's something that's kept me interested for, you know, the first three years that I've been here.But it is similar, actually, to being in an operational role, right? You do everything right, everything's resilient, you roll through an Amazon, like, region-specific issue without any blips, and no one reaches out to you. But you know, you have one issue, and then it's you're getting out of bed at three in the morning, and everyone's got a big retrospective about why you didn't do something that could have resulted in that not happening. And I can see the parallel.Corey: We all tend to have blind spots, and I more or less had my idea of big enterprise technology fixed a while back. And it occurred to me a few years ago that this is probably no longer accurate because I'm sitting here thinking of, well, United Airlines—with whom I do extortionately large amount of travel, let's be very clear here; we're talking I think I did 140,000 miles domestically flown in 2019, the last year that was even close to normal. Protip: Don't fly that much. It really winds up doing a number on your internal clock and having any semblance of life. But I'm sitting there thinking that it's old-school technology; there's a mainframe that powers all of this, and all of the staff checking me in are using these ancient Unix green screens has always been my assumption.And that thought occurred to me as I'm staring at my iPhone, checking in automatically in the mobile app—that was very modern and working at the same time—and the penny finally dropped for me of this is probably not accurate, how I'm envisioning the technology on the back end working. And there have been announcements that United is moving an awful lot of its systems to AWS specifically. What is that—I don't want to call it modernization because that sends the wrong undertone or subtext to it, but what has that cloud transformation been like?Tyler: So, it's the marrying together of those two things without the time that you would potentially want to just rewrite the functionality that the mainframes that have gotten us to do the amount of you know flights and revenue that we do, and that are rock solid, like, we don't get the chance to shut that thing down for three months and rebuild it—or what would be, realistically, more like three years. So, it's how do we build a—Corey: Yeah, it's a heck of a delay notice to put on the airport flight thing: “Flight delayed?” “Oh, when is it rescheduled to?” “2025.” Yeah, turns out that doesn't usually happen.Tyler: Yeah, and so we've got to do it at the same time. And there's, you know, analogies of, like, changing the tire while you're driving or changing the engine on the jet while it's flying. And we've actually—it's felt like that, but it's been in an exciting way. So, we really are able to decouple the front end from the back end or some of the core systems and then, piece-by-piece, modernize them, and do them in a way that is safe and responsible, given you know, the amount of folks that are relying on us to get to where they want to go every day.So yeah, it's been challenging for sure, but it's also the right thing to do. It's the direction we need to go where we can focus more of our engineering talent, which is scarce or limited, you know, we would rather have folks invested in improving the user experience instead of—what we have is a world-class data center, but you know, the number of people that are focused on making that what it is, I would much rather see that happen—or that investment be put into a higher up the value chain.Corey: It's also, on some level, on a baseline trying to understand how it all fits together. You look at the challenges that an airline has, you have challenges with labor, with press, with you know, the big problem of the logistics of not just the scheduling and the rest of making sure that everything flows throughout an enormous what is effectively logistics network, but also the, you know, the minor detail of keeping the planes in the sky when they're supposed to be in the sky. And it feels like on some other you flip through the list of concerns a company has, and technology in the computer sense feels like it's going to be, like, chapter 47 of that giant book. Obviously, that's not true because technology is an empowering story. It is not just the booking system; it controls, more or less, everything.At some level, I'd like to make fun of big companies saying, “Oh, we're not a”—insert whatever the company really does here—“We're a tech company.” But without technology, I don't think you, at this point, have much of an airline. How do you see yourselves in the broader sense? Are you increasingly a tech company?Tyler: We are increasingly a tech company. I think we're… we're seen as partners with the VPs of the different functional areas, right? It's not a separation of the business and IT the way that maybe we would have thought about it five or ten years ago. It's, both of us can't be successful without each other, and the functions have come to trust that we will spend the time we need to understand the problems that they're solving, and we'll bring different perspectives, we're going to bring technical solutions, but we're also going to bring, you know, potentially system or flow changes and business process improvements. And that takes some getting—that right a few times and building up that trust and spending the time you need to, like, go past, “Oh, here's a set of user stories. Just do them.” Of, like, “What are we trying to solve here? Could we just remove this process? Do we even need to do this thing anymore?” And once you prove yourself, I've never felt like we've been put in a backroom or seen as a lower priority. We're working on the same stuff together, and we win or lose together.Corey: I know a lot about the airline industry because I go to tech conferences, and when I'm at tech conferences, invariably the speaker—who's usually J. Paul Reed, but not always—decides to talk about computers, and incident response, and the rest through the lens of the airline industry, which for some reason has always been one of those neck and neck things that are just completely inseparable for those types of talks. And they talk about airline incidents, and very often it's not even, like, the horrifying headline-making stuff, but things like two aircraft passed closer to one another than they should have, and the NTSB does a full investigation. And they talk about how, “Oh, this is exactly the sort of thing you should do whenever there's a computer-related issue.” And I am curious, given that you do in fact have those investigations with the plane-facing stuff, how much of that culture carries over into the, “Hmm. We took a systems outage on the computer side.” And how much of that is similar versus how much of this is just conference-ware.Tyler: It's actually quite similar; that part of our culture permeates through. And we're actually looking at what's the right level of time to spend to get to the root cause when sometimes it's hard to explain in computers. Or there's so many variables that it's going to take us, you know, weeks or dozens of hours to really get there. But yeah, after any significant incident, we're religious about having a follow-up problem review where we get all the information that we need, and we, kind of, are expected to figure out exactly—like, replay what happened, step-by-step, and what were the controls that were in place to avoid such a thing, and were those complied with or not, et cetera. And earlier at my time in United, definitely was frustrated with how—I'm like, “I just need to get back to delivery. We've got this—this sprint is ending, and I can't spend four hours doing this.”Like, that was a… what was seen as, like, a one-time event. And I don't think that all the things that culminated in that are going to happen again, and I've done a few things that I feel are going to mitigate the risk moving forward, but actually, I've changed my perspective on this now. So, we are forcing—or not even forcing; we're simulating major incidents and then doing that type of a problem review so that we can learn ahead of time and we can make it a heck of a lot more fun [laugh] and open and transparent conversation. So hey, me or someone from my team gets behind the curtain and, like, creates some simulation of a major issue in one of our pre-production environments, and then the team that's responsible for the operations and whatnot of that response.And we look at what alerts went off? What alerts do we expect to go off that didn't? What was maybe a leading indicator that we aren't yet looking at? And kind of so we're calling that a game day, and we took that, you know, from—AWS has influenced our thinking on that, or they contributed to it. And it's a really good way to build those relationships, when there's not a lot on the line, you're not coming around what could be a customer-impacting negative experience, which is, you know, really what drives us to do good work is to make sure that never happens.And it does happen, but you know, we're getting more and more resilient. And this is a way to turn that on its head and be able to take the positive of that, and get the spirit, and get people to collaborate better because they—like, “Hey, I did that fun thing together. Now, when we're in the heat of it, we're going to collaborate better, we're going to be, kind of, more open with the information we're sharing because we understand each other's people and their intentions, and you know, where someone's coming from.” So, yeah, we were pretty excited about that.Corey: I have to admit I'm a little on the envious side about how your timing has worked out. Because back in 2008, when the cloud was still a new thing and some of the early adopters were diving in, the experience really sucked. I mean, this was before CloudFormation and other ways of managing systems. And by migrating over the last few years, so many of those sharp edges have been smoothed, and established patterns and processes, and understanding of how cloud interplays with enterprise IT has evolved dramatically. What has been your experience migrating to AWS? What's worked well and what hasn't?Tyler: Yeah, so the migration itself has been very deliberate. So, we were focused on AWS from the beginning, and it was—we believe that they're a leader, that they're going to give us what we need, but also we didn't want to fragment our engineers across multiple platforms and have them have to pick a team. Like, “Am I going to choose to learn how to build stuff in AWS, or GCP?” So, from just a transformation, and to get everybody on the same page, and upskill the organization, we're focused on AWS. And there's definitely, like, some learning curve, or moving into an environment where there used to be a centralized team that handled a lot of stuff for you and made it magic—like, as an engineer; I just have to make sure that my app builds, and then I can send it to someone, and they're going to deploy it, and it's going to work and then you know, we… shifting the responsibility to, okay, we actually believe that if—we could do that; we could just have the same function that did that in the on-prem world, do that for you in the cloud world, but our belief is that we come up with better software when the engineer understands and can control the entire workload and that it's like, “Hey, I can configure my app to take advantage of this particular portion of the underlying infrastructure.”And that became very clear with, like, Lambda or things like that, where it's… you know, there's only so many configurations, and it doesn't make sense to try to get someone else to do that for you. So, there's mindset changes that had to happen. There's also just, like, proving it out. Like, is this going to be more reliable than our data center, which is extremely reliable? And there have been issues in the cloud, like, where we have something running parallel, and we have a cloud issue and it didn't impact on-prem.So, how do we learn from that? And then how do we kind of continue on and figure out, how do we build resilient workloads in the cloud? How do we make sure that we cover our bases on not just getting it running, but like, getting it running the right way, and then doing the testing that we need to do—like I mentioned earlier on the game days—to really be confident in it so that we can ultimately move away from needing to have any sort of backup in the data center.Corey: I was poking around in an AWS account recently, and it looked like there were seven different ways of managing the systems that have been brought to bear in that account, and different design philosophies, competing approaches. And the sad part is that this was my personal AWS account. No one else has ever built anything in that account except for me. And if I have that problem as one person—admittedly a strange person—I can't imagine what the governance story around something like AWS looks like for an organization that has thousands of people working in your IT org. How do you wind up managing the way to build things appropriately?I can't fathom—even though I am a fan of ClickOps—just letting everyone loose with admin rights in the AWS console. There has to be some form of gating approach. Is that done through patterns? Is that done through some sort of internal platform that abstracts away for folks? How are you managing this?Tyler: Yeah, so this is one of the things that led to a learning curve at the beginning, but I think it's worthwhile. And I can't take credit for this because it was a decision that happened before I came, but we're all-in on infrastructure as code. So, we're not extremely prescriptive about what that means across the entire enterprise, but you cannot deploy anything into an environment, like, higher than a development area without it being defined as CloudFormation and promoted through. And that allows us consistency, auditability, [laugh] and a lot of other things.So, that was kind of phase one, and that's been—I believe—in place since we started in the cloud. Like, maybe there were some pocket accounts and some things that existed before, but once we were all-in, and it was, kind of, official that's been in place. And I'm glad we held to that because there's been a lot of, like, “Oh, just remove that. Let people build stuff through the console because they need to move fast.” And we're like, “Yes, that would move them fast right now, but the level of inconsistency would be extremely risky to be able to handle that, and handle production incidents if you don't have a pre-prod environment to test the patch that you're trying to put in on the fly, that manages hundreds of orders a second.”So, we started with CloudFormation. We were kind of all-in on CloudFormation, and then over the last year or so—maybe a little bit longer—it's become apparent that CloudFormation has some limitations. And it can be also intimidating to have to, in excruciating detail, like, define every single parameter of every resource you're trying to create. And—Corey: It's wordy. It's YAML or JSON, whichever one you hate the most, invariably, is the one you're dealing with today. And yeah, it has its limitations.Tyler: Yeah. And then they're sharing that happens, right? So, it's like, I've got someone that I go to lunch with, that's like, “Oh, I just built this solution. It's all in CloudFormation.” They send it over, and then I'm looking at, it's like, “Can I reuse this? Which parameters here are things that I should change for my app, and which ones are there because security mandated it, or it's part of, like, a corporate compliance thing, or other reasons why?”So, what we are really excited about in the last few months, we've really invested in CDK constructs and being able to define. You know, as my small team, we have visibility and strong, like, partnerships with our cloud engineering group, with our security groups, and whatnot, and we can say, “Hey, if you want to build an ECS cluster, like, this is a good, known way to start.” And you can just provide, like, X number of parameters that are meaningful to you, and you can inherit all the rest. And you're going to get our logging standards, you're going to get our security standards, all that, like, more or less built-in. And we also can version that.So, we can know, hey, this person built off the CDK App 1.1, and then we have some sort of security change, right? So say, now we want to install some other agent on all these things. And it's like, “Okay, all the ones that were deployed on 1.1, we need to move it from 1.1 to 1.2.”And we can test what that upgrade path looks like in a lab environment, and then we can, you know, release it and have, you know, 30 different app teams all consume that update in a relatively self-service manner that means we don't have to do it one by one. And then, yeah, it just gives us the ability to respond to stuff as quickly as we need to in the current environment.Corey: Today's episode is brought to you in part by our friends at MinIO the high-performance Kubernetes native object store that's built for the multi-cloud, creating a consistent data storage layer for your public cloud instances, your private cloud instances, and even your edge instances, depending upon what the heck you're defining those as, which depends probably on where you work. It's getting that unified is one of the greatest challenges facing developers and architects today. It requires S3 compatibility, enterprise-grade security and resiliency, the speed to run any workload, and the footprint to run anywhere, and that's exactly what MinIO offers. With superb read speeds in excess of 360 gigs and 100 megabyte binary that doesn't eat all the data you've gotten on the system, it's exactly what you've been looking for. Check it out today at min.io/download, and see for yourself. That's min.io/download, and be sure to tell them that I sent you.Corey: It's a constant challenge and it's really neat seeing the adoption of things like the CDK, which I've always sort of mentally put on the same stack as, “Oh, yeah, this is something that scrappy tiny startups use.” But you're the exact opposite of that. The fact that you're using it and finding success with it says a lot. I think you're also right there with the most nimble, advanced, tiniest of startups in the world, and you're still trying to figure out how to contextualize this into the broader lifecycle and understand the long-term architectural implications of how this stuff works. If it helps anything, I can assure you, you are very far from alone.If anyone else is feeling that way, exactly the same position. And if you're out there saying, “Oh, yeah. We've solved this. This is how we do it.” Find a second person to agree with you. But then come talk to me. Because everyone solves it locally; no one solves that globally. It's a hard problem.Tyler: Yeah. We've had this vision of, like, a vending machine for stuff. And then we've tried that in different ways and templates, and we think that this is the right pattern.Corey: Yeah, every time AWS builds a vending machine for accounts and whatnot, it's like the worst kind of vending machine; the kind that eats all your money.Tyler: Service catalog. Yeah.Corey: Yeah. It becomes a disaster. So, I want to talk about a couple of other things as well. When we started talking a year or so ago, you were a team lead. Today, you are a senior manager, and it turns out that, unlike when you start your own company and can invent your own made-up title, like, Cloud Economist, those words mean things. So first, congratulations on the promotion, how'd it come about?Tyler: Thank you. Yeah, it came about—I guess, I really have always been passionate about people leadership, but I know that in order to properly lead and, like, have the context, and you need to know what it's like to do these hard things that my team is solving, and be responsible for those, kind of, as an individual. So, you know, I've been spending the last, like, five or so years as an individual contributor, kind of learning how all this stuff works, and then learning from a lot of different managers. You know, I've been really lucky to have some people that, kind of, took me under their wing, coached me, and is just, like, the person that puts the wind in your sails, but like, not in a… not in a fake way, but like actually sees you and puts you into situations that are going to force you to grow and have your back if something goes wrong. And I kind of saw that and I wanted to be that for someone else.So, you know, it's… yeah, it was something that I kind of put my hat in the ring, and a position came and I was tapped to step up and do it. But it was initially for a very small team, right, so a three-person team. But it's since expanded to be six or seven over the next month or so.Corey: One of the things that I found always interesting slash admirable about you is we travel in somewhat similar circles. We both have pitched in from time to time as mentors in Forrest Brazeal's cloud resume challenge, and it's nice to see people who are working at established companies who are very busy with their day jobs, also taking the time out of the day to help, effectively, what is the next generation of cloud engineer find their way within this industry. How did you get onto that track?Tyler: Yeah, so I guess it's, you got to send the elevator back down. I have the experience of, kind of, being on the edge of, like—I was on the waitlist for my university, I had to—also was on the waitlist for my first job as a rotational program, and there was always kind of this, like, I had to claw for it, I had to prove myself, and also had to—I was the first in my family to pursue opportunities like this. And I got the itch for it, then I also see there's so much potential in folks. And like, even looking at my parents as examples, right? My father's an auto mechanic, and he's probably one of the smartest people I know, but didn't really… have the opportunity to get into technology. [unintelligible 00:22:44] kind of in a blue-collar job.But I just feel like there's so much untapped potential, and I am passionate about helping people at least, like, understand what opportunities are available to them. And not just assume that if you don't have an example of someone who's a software engineer in your life, or a sibling, or a parent, like, that's outside of your reach.Corey: I love the phrase, ‘send the elevator back down' because it's true. I feel like the only reason that anyone that you have ever heard of in tech, who you have any modicum of respect for—and I include both of us on that list as well, but basically everyone else in the industry, too—the only reason all of us are here in the roles that we're in is that at some point, someone did a favor for us that they didn't have to, but they did. And it's almost impossible to pay that back, so instead, I've stopped trying. I instead try to do those favors in a forward-looking way for other people whenever I can. And there's a lot to be said for expressing that through a way of helping people find their way and see what happens.Because let's face it, the industry that you and I came up in doesn't really exist in the same way. There is no fleet of help desk positions out there the way there was when I first started getting exposed to technology, that would get me into this direction, so people have to come through alternate paths. And some people try and express that through advice that no longer applies for a world long gone. I try and at least keep up with what's going on in this space.Tyler: Yeah, absolutely. It's a dynamic environment for sure, and when I look at just how challenging it is to try to, like, find a senior cloud engineer, and then looking at, okay, is what we're doing here, like, really rocket science? Does it require ten years of experience? And I think the answer is no, like, we've got a small enough group here, we know what we're doing, and everyone's passionate about bringing other people up and, like, finding their strengths, giving them a problem, not giving them the answer to the problem, and kind of strategically building to bigger, bigger things until the next day, you know—or before you know it, they're able to solve problems that you would have previously thought, like, “Oh, that's something that I have to get my hands on.” And it's just so powerful to see that and to be part of that. So, that's kind of the approach we're taking.Corey: It refreshing to see. So, many companies are requiring that they hire senior talent, and they can't take junior talent because, “Oh, that person would take six months to come up to speed in this environment. We want to hit the ground running.” And the job req has been open for nine months. At some point, building talent becomes the best slash only way forward.I'm still at a scale now where I'm not in a position be able to do that, just because we are dropping principal consultants into dynamic strange situations, and that is a terrible environment for a junior, but as you scale past a certain point—I don't really know what that point is, but yes, United Airlines has scaled past that point—bringing folks up, taking interns, making interns job offers, and continuing to expand what is happening, I think, on some level, one of the big hiring challenges for United and other similarly situated companies has been that, oh, the technology must be ancient caribou-era of trekking across the tundra level of development. But we just talked about using the CDK, and pattern design for things. The public perception and the reality are incredibly divergent.Tyler: Yeah. Maybe I'm strange in this regard. But since college, I've worked only in very, very large organizations. And seeing the satisfaction that you have, or you can get from working with those systems, and being able to churn out a modern customer experience, or modernizing the system for operational efficiency, just it's very satisfying to me to be in that environment. I know that it probably scares other people away.But it's just the scale; it's hard to get that scale somewhere more—I don't know, I guess, like, younger, newer because you don't have years of legacy. But I don't necessarily see that as a bad thing. Like, years of success and technology that's supported that success that you need to figure out how to handle.Corey: One last question that I have for you harkens back to something that I said earlier, where I congratulated you on your promotion to management. It's not really a promotion, at least not the way that I think it should be thought about. Because it's very much an orthogonal skill. You were a great engineer and architect building things yourself. And now you manage a team where if you're diving into fix things by hand, you are misunderstanding the role in many respects, suddenly, your toolkit is no longer doing the thing yourself, but rather delegating the thing to be done and making sure that it gets done and your primary slash only toolkit to do all of that is hiring and developing talent. How have you negotiated that transition? Do you still find yourself itching to dive in and fix the work yourself? Are you better at letting go than I was for a long time? Where do you find yourself on that?Tyler: Yeah, so that the inclination is still there, but I've learned to, like, recognize it and let it go. But I also have told my team members, like, 90% of the time, I'm going to give you all the latitude in the world, and I'm going to spend all my time helping you understand the problem that we're facing as I understand it, and the potential roadblocks, and then there may be some times where I'm going to be like, “I really want it done this way.” And I ask them to give me that… give me that ability. I have yet to really break that one out. But that's the only way that you can scale, and you get so much satisfaction about over… empowering someone to solve a hard challenge, and then seeing that they did it in a way different than you did it, and they did it better. [laugh].And that's a little bit of an ego hit, but you're like, that's what it's about. And then they can build that confidence and then take on larger challenges. And that's what gets me out of bed in the morning; that's what gets me excited is working with people who just really want to do good work. And I can help put the right challenges in front of them, help shield them from stuff that's not adding value, but like, asking for their time, connecting them with others that is going to kind of get that wind in their sails, and just get out of their way.And then once the success is there, do everything I can to get that out and make sure that people know the good work that we're doing. Because as much as you can say your work speaks for itself, in a huge organization, it's not so much the case. Like, good work often goes unacknowledged if there's not someone if you're—like, promoting that. And most individuals aren't comfortable—myself included—promoting my own work. Like, I wouldn't do that, but I'm more than happy to promote the work of someone on my team.Corey: On some level, as managers, you get recognized and evaluated based upon the performance of your team, not the things that you personally achieve. And that has always been a difficult transition. I got to level with you; I never handled it super well. It sounds like you are way better suited for the role than I ever was.Tyler: Well, it's early on, but yeah, I'm very excited.Corey: If I really want to evaluate a manager, all I have to do is really talk to their team, more often than not, and you start to see things when you probe properly. I really want to thank you for taking so much time out of your day to speak with me. If people want to learn more about what you're up to and how you see things, where can they find you?Tyler: I'm probably most active on LinkedIn. So, just tylerslove at LinkedIn.Corey: We'll be sure to add that to both the [show notes 00:29:58], as well as I will add you to my professional network on LinkedIn, which I believe is the catchphrase that they're using. Thanks so much for your time. I appreciate it.Tyler: All right. Thanks, Corey.Corey: Tyler Slove, Senior Manager for Enterprise Cloud and DevOps at United Airlines. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud, of the usual kind. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment disavowing all of this newfangled technology we've been talking about and that's why you only travel via steamship.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

About AnaAna Visneski is the founder of Merewif, a crisis communications and management consulting firm. She is a veteran of the U.S. Coast Guard where she was a first responder to major disasters from Hurricane Katrina to the BP Oil Spill, and various other incidents. After the USCG, Ana moved on to a whole new disaster that needed an experienced crisis operator - running Launch Operations for AWS. Following that she was the global lead for AWS Disaster Response, overseeing deploying AWS technology response to natural disasters and overseeing the response to COVID. She has a Master of Communication Digital Media and a Master of Communication in Networks from the University of Washington, where she currently teaching Crisis Communications. Links: Mirewif: https://www.themerewif.com/ Oracle HeatWave: https://www.oracle.com/mysql/heatwave/ Twitter: https://twitter.com/acvisneski The—T-H-E—merewif—M-E-R-E-W-I-F dot com: https://www.themerewif.com/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Vultr. Spelled V-U-L-T-R because they're all about helping save money, including on things like, you know, vowels. So, what they do is they are a cloud provider that provides surprisingly high performance cloud compute at a price that—while sure they claim its better than AWS pricing—and when they say that they mean it is less money. Sure, I don't dispute that but what I find interesting is that it's predictable. They tell you in advance on a monthly basis what it's going to going to cost. They have a bunch of advanced networking features. They have nineteen global locations and scale things elastically. Not to be confused with openly, because apparently elastic and open can mean the same thing sometimes. They have had over a million users. Deployments take less that sixty seconds across twelve pre-selected operating systems. Or, if you're one of those nutters like me, you can bring your own ISO and install basically any operating system you want. Starting with pricing as low as $2.50 a month for Vultr cloud compute they have plans for developers and businesses of all sizes, except maybe Amazon, who stubbornly insists on having something to scale all on their own. Try Vultr today for free by visiting: vultr.com/screaming, and you'll receive a $100 in credit. Thats v-u-l-t-r.com slash screaming.Corey: This episode is sponsored in part by our friends at Sysdig. Sysdig is the solution for securing DevOps. They have a blog post that went up recently about how an insecure AWS Lambda function could be used as a pivot point to get access into your environment. They've also gone deep in-depth with a bunch of other approaches to how DevOps and security are inextricably linked. To learn more, visit sysdig.com and tell them I sent you. That's S-Y-S-D-I-G dot com. My thanks to them for their continued support of this ridiculous nonsense.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. My guest today has been on this show before, generally at a previous point in her career where she was making a transition. That time, she was leaving AWS, as happens to awesome people a fair bit of the time—more than it potentially should—and going to work at H2O.ai, a company that does some sort of machine learning thing that I can't be bothered to remember offhand. I talked to her again, as she has just left that company to start her own thing. Ana Visneski is the Chief Chaos Coordinator at Mirewif. Ana, thank you for joining me yet again.Ana: Oh, I mean, how could I not when you're the one who got me to get off my butt and actually start my own company?Corey: What's fun is that your company is a crisis communications firm, and first that's definitely useful for me because I do put the ‘crisis' in ‘crisis comms,' let's not kid ourselves.Ana: You're not wrong. [laugh].Corey: But I'm also your first customer.Ana: Mm-hm.Corey: And you're in one of the harder niches to get people to stand up and say, “Yeah. Oh, yeah. Can I get a testimonial on this?” “Absolutely not. We hired you because we did something horrible.”And that's not really how I tend to view crisis comms. I mean, it's sort of a similar problem to what I had when I started The Duckbill Group of, “Hey, can I use you as a testimonial about your horrifying AWS bill?” “No.” And I understand how it looks, which is not the reality of it. And in time, I found ways to get people to slap their logo on their website. But I want to be the first logo, the fact that I have a platypus associated is just a nice bonus.Ana: Absolutely. You will be the first logo when I finally get around adding logos. The interesting thing is, that it's not just crisis comms that I'm doing with the company. I also do threat assessment, violence assessment, so risk analysis, basically, on if you have an employee that might be a risk or, for some of my video game or gaming companies, if you have someone in your fan organization that is a potential risk.I also do crisis management planning. So, I will put together an operational plan—similar to what I built when I was at AWS—a top to bottom, this is how you run a crisis to make sure your people don't burn out, make sure your leadership is aware of what's going on and gets the proper daily briefings, that sort of thing. And then lastly, I've actually been doing some consulting with governments on their disaster response technology needs. So, there's a lot of different aspects to it.Corey: Yeah, to be very clear, none of those things are things that I have roped you in for. I don't have employees that I'm looking there with, “Oh, if they blow their stack this is going to be a disaster.” Like that is not the nature of the work we're doing together. What we're doing is more along the lines of, “Okay, great. I have a bad tweet that blows up. How do I handle this without, ‘All right, pass me that shovel. We're digging this puppy deeper. Now, okay. Holes dug nice and deep. Let's work on the edging details a little bit.'”Ana: [laugh]. Yep.Corey: It's the, “How do I avoid making things worse in moments of crisis?” And we're building plans for things that I hope to never need around things like data breaches, like, the stuff that every business should have a plan for. Because when disaster strikes, as it tends to in various ways, I don't want to be sitting here flipping through the Yellow Pages for, “I've messed up.” Like, I don't know what section that would be in. Having a plan ready to go is important.Ana: I would say it's actually critical.Corey: Yeah.Ana: So, that's the thing is, unfortunately—and as Covid taught a lot of people—having that plan in place before things go wrong before the shit hits the fan, is what's going to save you or not. It'll save you millions of dollars, it'll save your employees, and it could potentially save lives. And so what I think a lot of companies have finally figured out is, “Oh, wait. We weren't ready for Covid. We actually need to be ready for the next thing.”But I also teach crisis communications for the communication leadership program for the University of Washington; it's a graduate program. You've been a guest speaker there. You were one of the favorite guest speakers. And there I tell them all the time is that you have to plan. The two critical things before anything even starts is planning and trust.If you don't have plans in place on how you're going to do things, you're going to have people running around like chickens with their heads cut off going, “Oh, what do we do?” And someone's going to do something that makes it worse—inevitably—with the best of intentions. And then the other thing is, if your audience, if your customers don't trust you to be doing the right thing in the first place, then no amount of planning is going to help from that deficit.Corey: It also, in my experience working with you, comes down to avoiding putting your foot in your mouth with the best of intentions.Ana: Yes.Corey: Heaven forbid if you have an employee pass and tweeting out something like, “We are heartbroken to announce the loss of our dear friend and colleague, [Shtephen 00:06:45]. Also, we're hiring.” Like, make sure you don't wind up coming across as the worst example of humanity. It's the basic stuff.Ana: Even more than just that basic of don't put your I'm hiring—because you saw that tweet that was going around with, “So-and-so has passed. Please mourn off the clock.” Whether that was a joke or not—and it's up for debate if it was real or not, like—Corey: We've all known people who would have said such a thing and it would not have been a joke.Ana: Exactly. But the other thing is, it's not even just that. It's knowing the timelines for notifications. So, for example, there should be at least a 24-hour next-of-kin notification window, where if someone is passed, the friends and family grieving can be notified. The last thing you want is a friend of Shtephen to find out that he died because you tweeted about it. That is traumatizing.So, you actually have to have a plan in place of, you've received notification from Shtephen's wife that he has passed. Obviously, you're going to be offering her your support. Say, “Hey, here's the things we can offer you to help.” You have, you know, your package of, like, here's the ways we can help you. But then you also say, “Can you let me know when it's appropriate for me to tell the other employees?”Because the moment you start telling employees—this recently happened; a friend of mine in the Coast Guard passed, and unfortunately, some others found out about his passing because someone posted about it on Facebook. That is not the way you need to find out. So, it's not even the blatantly obvious things, like, “Oh, hey, don't post about hiring,” it's also just the order in which you notify so that things don't leak.Corey: I didn't even know Shtephen was married. I mean, what kind of—Ana: [laugh].Corey: —crappy employer am I here? Yeah, it's the human side of it.Ana: Mm-hm.Corey: And that's one of the things I've always admired about you. It's—and again, when I started doing all these nonsense things, I had a circle of friends that I could run things past of, “Hey, is this tweet a bridge too far?” And in time, I needed to rely on those people a little bit less because it turns out that I have a pretty good eye for what's going to make people feel bad. And that's really the only thing I care about is if it makes someone feel bad, then I'm not thrilled with the tweet most of the time.And I figured out where that line lies. And then I got loud and big enough on Twitter where I started having to think about it again, where, all right, I know it's not mean, but I'm going to hear about it. Is the juice worth the squeeze? And the reason I like working with you on things like that is I've grown well past the point where I'm comfortable asking people to volunteer for basically what amounts to something of my own brand-building exercise. Paying people for advice has always been something that I'm a big fan of, and now I'm able to do that and have a professional way.And I don't think you've ever once been wrong. There are times you've given guidance that I have not followed, but that's what you see anytime you're talking about someone a downside, risk side of the business. That's the entire function of an attorney for a business is to identify risk. If you start letting attorneys, for example, my wife, great attorney, great wife, wound up—Ana: And very tolerant human being. [laugh].Corey: Oh, extraordinarily—living saint. But she wound up editing a proposal that I was going to send out—back when I was independent—once. And I looked at it and she's like, “Oh, well that could go wrong, and that could go wrong and no, we're going to change that and the rest.” It's like, this is—I understand where you're coming from, but this is a sales document. And it was for a proposal, it was something like $7,000 back then.It's like, worst-case scenario, I'm a nice person, I will fall over myself apologizing and give them a full refund. The end. That sort of caps my downside risk here, if they want to be obnoxious and go to court, well, I've been doing this for three months, I guess I'm shutting down the LLC because that's been sued into oblivion. I'm getting a real job. Like that was the risk mitigation there.She's used to doing risk analysis for a company with 250,000 employees, and yeah, they have more to lose than I do in those things, so I get it. But you don't generally have lawyers on your sales team that are proactively over-promising things, for obvious reasons. At least—because there's no way to get a salesperson disbarred. I've checked.Ana: Of course you did. When I'm teaching class, one of the other things I do is I actually have some lawyers come in and talk. And the reason is, I learned this one when I was in the Coast Guard, and I was running District Eight. So, it's basically the entire Gulf Coast and all the way up the Mississippi to the Canadian border. So, all of the units contained in that area, I was in charge of their media relations, their community relations.And this was, like, right after Katrina. I learned pretty quickly that having a very good relationship with my lawyer—so the head of legal—it made us a one-two punch that was unbeatable because I could look at it from the human empathy, communication, subtext aspect, and he'd look at it from the legal aspect, and the two of us would be like, “Okay, you can do this legally, but here's the impact of it if you say it this way, or if you do this.” Or, “Ehh, don't do this one, legally.” Like, it's just a great thing. But risk analysis, from my perspective versus a lawyer's, are slightly different.I do, of course, talk to lawyers, obviously, a lot, and look at the legal side of stuff. But a lot of what I'm looking at is perception, subtext, potential pitfalls. You and I've had many conversations, and you know me well enough to know that most of the time I'm giving you guidance, but if I see one more, I'm like, “Absolutely not. Do not do that.” I will lean into it so heavily, and be like, “Corey, here's the eight ways this is going to go badly for you. You're going to end up in The Times for bad stuff.”Corey: And you say that so infrequently that I definitely pay attention when you do. I don't always listen, I mean, [crosstalk 00:12:14] I wound up posting that Andy Jassy birthday video. But you know—Ana: I helped with that video, though. [laugh].Corey: —you were instrumental behind that video. Thank you for that.Ana: You're welcome. But that's—so what's fun about working with you, and different than my other clients is there are these moments where I get to also express my weird sense of humor, you know, where it's just like, calling Jeff Bezos, a space cowboy. Those moments of getting to find—help you with that line. Because I have that same sense of humor line and I don't get to express it a lot with my other clients because most of them are very, very serious bidness. And not to say your business isn't serious, but you yourself are almost—Corey: But we do have fun with it.Ana: —never serious. Exactly, exactly. And that one, like, I really enjoy that aspect of it. But with a lot of the other stuff, it is incredibly serious. And like the risk analysis that your wife does, versus the risk analysis type I do, I'm actually looking at emotional stuff.So, when we're talking about acts of violence, for example, acts of violence are, almost to a one, about power. So, what I do is I actually sit and look at okay, this person is lashing out. What power dynamic has them wanting to lash out? So like, if you look at a lot of the school shootings, it's about kids who feel bullied, they want to regain power by showing they have power or the guys who write their manifesto about hating women, et cetera, et cetera. So, it's always about a power dynamic.So, it's not about, is it legal to go in and shoot the office? It's clearly not. But has the system taught them that they can push the line far enough that this sort of behavior, they might get famous for it? Or might get away with it? And then how do you mitigate that particular power dynamic? And so that gets real tricky. And luckily, with you, I have not had to deal with that one.Corey: For better or worse, I come out from a good place to place a good intention. I'm trying to imagine if I just said, “To hell with it,” and decided to just take off the gloves and be a complete bully every time I felt like it. I could do some damage at this point. But… no.Ana: You could, but the thing is remember what I said at the very beginning: It's about trust. What has made you so very successful, what has made you so good at what you do is you're very intentional and very careful. Not to say you're not a pain in the ass. I will agree with some—Corey: And I do get wrong. Let's be clear. I'm no saint.Ana: Oh, no, no, no. No. You've gotten stuff wrong, but you immediately apologize for it. So, when I'm talking about this from a space of trust, it's not that you're not obnoxious; you totally can be.Corey: Extraordinarily so.Ana: You can totally be a snarky pain in the ass. Like I said, your wife is a saint. And sometimes—like, we were talking about recently, backing off on mocking people for working for Facebook because you and I both saw what it did to Chloe. And it's just not cool to do that to someone who's making a career choice, whether we agree with it or not. I personally have companies I would never work for. You and I have discussed contracts—not with you, but contracts I wouldn't take. Me personally, it's in my contract, I will not defend someone who is a sexual harasser or sexual assaulter. Like, I won't defend them. If they do #MeToo stuff—Corey: Mm-hm. The way that we've codified that—Ana: —I won't do it.Corey: —here is generally speaking—and this is a truism, I would encourage everyone in business to consider is, if you don't respect a client's business, you probably should not take their money. And—Ana: [laugh].Corey: —that leads to a lot of things.Ana: Yeah. I wish that was more common. [laugh].Corey: Yeah. It's—and again, I've never once shamed a company for this. I have declined to work with a number of companies in different capacities. And I've never been very open about this because I don't want companies to be listening to this and think, “Ohh, we sell ads. He might not want to work with us, so we're not going to reach out.” First, I will never mention, name, or drag anyone publicly.Ana: Oh, yeah. Same.Corey: Secondly, there's no such thing as any saint in these industries.Ana: Oh, no.Corey: I'm not talking about, “Oh, you display ads to people? [tsking noise].” No, I'm talking about, “You make landmines.” Let's be clear here. This is a whole other side of the universe. And I still never drag the companies that I declined to work with, in public, for having the temerity to reach out. Just seems like it's the wrong incentive structure if I start down that path.Ana: I was just talking to a client that I firmly believe we're at a pivot point in the way businesses are run. I was calling 2022 the Year of Transparency. And the reason I'm saying that is because in the last couple years with people working from home, with Covid, with Black Lives Matter, with all the stuff that's been going on in the world, and then, like, Activision Blizzard, and the lawsuits, and pay disparity, and Paizo unionizing—Paizo is a tabletop company that makes Pathfinder RPG—Corey: Mmm.Ana: —you know, all these companies. So, we're starting to see the game industry see unionization, we're seeing Starbucks employees want to unionize. People are not going to accept, “No comment,” anymore. They're not going to accept, “We're just not going to answer this.” And I can already see your brain ticking on who you're about to—I know where you're thinking.But my point is, when I've been talking to some of them, “I'm like, you have to be prepared that the old-school mentality of people not sharing their pay, like, not sharing how much they make compared to the person sitting next to them, that's gone.” People share that information now. There are companies where they are having spreadsheets. Now, one thing I did like about AWS was I always knew, like, my peers and I were encouraged if we want—my manager was awesome—my first manager was like, “If you guys want to talk about what you're making, go ahead.” And I was able to find out that because I had the masters, and more experience, and all this other stuff, I was actually—in my level group—the highest-paid one, even though I was the only woman at first. That's pretty cool to know.Corey: That's the kind of story that never makes the rounds.Ana: Well, and the thing is, we're not going to see people accepting obfuscation anymore. I think that's done. It's too easy to share information now for companies to think that their dirty laundry isn't going to come out, to think that they can lie and get away with stuff. As you know, we've talked about this a bit, I'm actually working on a book with a comic book artist—I didn't get his permission to say his name, so I'm not going to say it yet—and it's literally a picture book on how to not screw things up in today's digital media age when it comes to how you communicate with people. It's called Oh, Noes: A Picture Book for Execs. [laugh]. Um, but you know, you got to focus on the fact that people aren't going to accept obfuscation and lies anymore. They're not going to accept, “Oh, we're the company. We've got your best interests at heart.” It's not how it works anymore.Corey: That's what I see in this entire industry, where there's this idea that we're not going to say anything, we're just going to do our thing and not comment on any of these things. Which, okay, it's a strategy. But customers and the community and loud obnoxious—Ana: They talk to each other.Corey: —people on Twitter are going to comment in your absence. And that becomes a problem.Ana: Have you seen the movie—what is it?—John Tucker Must Die?Corey: I have not.Ana: It's a movie about three girls at the same high school who find out the guy is dating all three of them, and how they plot to destroy him. And every time I see one of these things happen where a big tech company—or any company—doesn't say anything, but then their customers start talking to each other going, “Wait a second,” I always think of that movie. And it's like, you can't think that people aren't going to talk to each other anymore.Especially once you get huge. When you're looking at these big, big companies, people want to take you down. Like, they're over this idea of monopolization and this idea that you can do things and there's no accountability. So yeah, I've been calling this the Year of Transparency because I think we're going to see huge shifts in what is and isn't okay to hide from your customers. Trust is your most valuable asset. And it can be lost in seconds.Corey: It's the easiest thing in the world to get, and it's incredibly easy to lose it, and almost impossible to regain it once you've lost it.Ana: Yes. And I think my students get sick of me saying this because I say it every week: “Trust is easy to get if you do it right, but you got to do it right.” You actually have to be honest, you have to, you know—and I'm not saying share secrets. But you can be—like, a good example with AWS is, they do great COEs after they have a big splat. You know, 2017, when they had a service disruption, and the latest ones, like, they do a good COE. Being able to rely on that sort of thing is critical.Corey: For me, it's one of the things that we do here just because of the sensitive information with which we are entrusted, and the way that we operate in the industry, we hold ourselves to a bar that is pretty similar to what you'll see in regulated industries and the rest. I periodically disclose all of my investments, which is nowhere near as interesting as most people would think.Ana: [laugh].Corey: I make it clear exactly where my interests are. This is the reason we have no partners with any company in this space, just because it is the perception of conflict of interest is huge. I mean, half our consulting business is doing contract negotiation on behalf of customers, with AWS directly. As soon as it comes out that we have a back channel deal with someone, everyone's going to question what's going on. It's easier never to enter into those engagements rather than having to try and back-walk it later. No. Does that leave opportunities on the table? Sometimes. But I think this is the better long-term play if I can think beyond next quarter's numbers.Ana: Yeah, absolutely. And that's, like, similar for me is that I have to be mindful of not taking contracts with companies that are in conflict with each other. And I don't mean conflict like they're at war, but like, where my working with each of them puts me in a position where there could be questions on who my loyalties are to.Corey: On the sponsorship side of our business, we refuse to do anything that even looks like an exclusivity contract, of, “All right. None of our direct competitors will be allowed to sponsor for a fixed period of ti”—sure, if you buy out the ads you don't want them to take, I guess, sure. But you don't get editorial control, either. It's the same approach: You can buy my attention, but never my opinion. Paying me does not make me say nicer things about you, directly.It does force me to look more closely into what your company does, and no one's purely good or purely evil. I will talk more about what I see, good and bad. That is the nature of what you get with me, and that is something that I don't think a number of folks realize, out of that ecosystem.Ana: Well, there's a level of professional maturity that goes with taking criticism. And when you have worked on something for a very, very, very long time, and it is your baby and you're getting criticized, it can be natural to have an emotional response. And that's something that, as a crisis communicator, I look at. Are the attacks coming in—and attacks, or commentary, or negative press—is it coming in, in an emotional way, like, what's happening is there's been a nerve hit because there's an emotional investment in whatever's going on? Or is it an impact of concern over finances, concern over jobs? So, there's different reasons why people will react and things. And that's one of the things I have to always keep in mind when I'm looking at stuff. As you well know. We've had many conversations about this. [laugh].Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance query accelerator for the Oracle MySQL Database Service, although I insist on calling it, “My squirrel.” While MySQL has long been the worlds most popular open-source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLAP and OLTP—don't ask me to pronounce those acronyms again—workloads directly from your MySQL database and eliminate the time-consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: One of the things that I always admired about you—and I have never once incidentally tried to change this in any way—but you have never leaked confidential information to me about anyone or anything. And to be clear, I have never asked. Back when you're running launch operations at AWS, I don't want to know things that are coming out if I can avoid it because then it gets very challenging for me to remember what I can talk about versus what I can't. My insight into AWS product roadmaps is not much better than anyone else in the industry. I just pay attention and I have a knack for being able to see what's coming.But because of the perception that I have the inside track, I don't break the news; I don't create the news; I just talk about what other people have already written about publicly. It's safer that way for me, and I've always appreciated your ability to respect confidentiality because for stuff like this, it matters more than anything else.Ana: Absolutely. My confidentiality is huge thing. I just don't talk about stuff. And in fact, like, my husband doesn't even know who half my clients are. He knows the number of clients I have, but he doesn't know who I'm working with. And that's because, you know, I don't need him to know. And it's a confidentiality thing. And you know, spouse, you're my husband, I have ten clients. And that's what I'll say. You know? He knows about you, obviously.Corey: Well, I should hope so. He's lovely. I was at your wedding, lovely though it was.Ana: That is true.Corey: So, one other thing that you're in the process of launching as we speak is apparently your own podcast. Loathe though I am to drive people to the competition, tell me about it.Ana: [laugh]. It's not actually competition, and we do have to give you credit for the name. One of your superpowers is giving really funny, punny names to just about anything. Next time we get a pet, I'm going to be like, “I want a pun that goes around this. What can I name the dog?”Corey: And how long did it take me to name your podcast?Ana: God, like, two minutes. It's so annoying because I'd been—Corey: It took that long?Ana: You let me finish typing.Corey: Yeah, that was nice of me, I thought.Ana: Yeah, you let me finish typing, and then you're like—okay, so not even two minutes. Like, a minute.Corey: It's not that I'm that good at naming things. It's just that I've never worked at AWS, and people who are so bad at it, that when some—they just encounter someone who's average with these things, we look like wizards from the future.Ana: So yeah, we're launching a podcast called [Disasterpiece Theater 00:26:15]. And it's actually a podcast where we're going to have subject matter experts from NASA, from medical fields, cybersecurity folks, we're going to actually have a shark expert so we can talk about The Meg and how that works. But the whole point of the podcast is taking pop culture movies—so like Jurassic Park, Alien: Covenant, all of these—and talking about how they'd actually work in the real world. How would Alien: Covenant have gone down if these people were trained the way people on ships are trained now? Or The Meg, what would you actually do if you had a shark in that scene where there's hundreds of thousands of people in the water? How would that actually go? I mean, the shark wouldn't be three miles long, but same concept.So, it's going to be a lot of fun, just kind of going through. One of my favorite guests is my dad. My dad's going to be talking to us about the movie 2012. My dad's a naval architect marine engineer. And he and I had the most fascinating conversation after watching that movie on how ships like that would actually be built, and what would happen, and what would have to happen, and the different rules and regulations that would have to change, and how you would actually—like, and his pet peeves with lazy things the writers did. So, it's going to be a lot of fun. We're doing it as a short run to see how it sticks. It'll be eight episodes to start, and then if there's a desire for more, we'll do a second season.Corey: I'm really looking forward to seeing how it comes out. I'd ask, “What's going on? You're starting a company and this side project for funsies? What's the point?” But I started this podcast show not too long after I started what became The Duckbill Group. So yeah.Ana: What's funny is, this has all kind of cascaded in weird ways because next month, the company's—Merewif's been around a year next month.Corey: Wow. Hard to believe.Ana: Which is totally crazy to think about. But I only—I was doing it as a side gig while I was at H2O until October—end of September. So, it's only been full-time since October. The podcast idea—Corey: Why now? Why now, though? What drove you—Ana: [laugh].Corey: —you went from giant company to start-up to launching your own thing. And you're launching your own thing in the same way that I launched my own company, which I'm going to shorthand to ‘the dumb way,' which is right now there is so much constipated capital sloshing around the VC ecosystem, and we both started companies that are absolutely never going to be a VC-scale opportunity because, you know, what can you do with $4 billion in investment? Oh, something monstrous, for damn sure. But there's no—there's no good answer to that. But we're never going to be the VC-scale opportunity.Ana: [laugh]. I dread to think what you would figure out what to do with, like, $400 million. It's terrifying.Corey: Oh, the video would be ridiculous. We're talking, like, Pixar quality…ridiculousness, making fun of various things in this industry, on a lark.Ana: Oh, I can imagine. I can imagine. I can imagine a weekly game show with you, too, where you brought in engineers from the different services and ask them random questions, kind of like Jeopardy, but with, like, the floor dropping out underneath them. Then they just get replaced with the next engineer or whatever. Like, they get an answer wrong; they drop through the floor; the next one slides in.Corey: I like that, yeah. That has legs.Ana: And this is why you and I are not allowed to come up with ideas together.Corey: Yeah this is—Ana: Anyway.Corey: —what we do to break on us from time to time. Yeah.Ana: [laugh]. So, the timing was a couple things. One, I've wanted to do this company since I got out of the Coast Guard. Like, it's something I wanted to do, but I needed to get more private experience. Because up until 2016, all of my experience was public sector. It was military, it was Coast Guard.So, while I'd worked with—in disasters, I had been side-by-side with BP dealing with that disaster and all sorts of stuff, I didn't actually have the experience myself. And I kept going, “Oh, well. I'll do it eventually. I'll do it eventually. I'm not ready yet. I'm not ready yet.”And then literally, you and one other person were like, “No, I literally need you to set this up right now because I need your help with something and I need an official way to pay you.”Corey: It seemed like the right thing to do. Yeah. Yeah.Ana: So, it was, like, “Okay.” And the name Merewif actually means ‘sea witch' or ‘siren' in Old English. Little known fact: I majored in English specializing in medieval and ancient literature when I was in college.Corey: That explains your depth of insight into the AWS documentation.Ana: [laugh]. Yeah. I can read like nobody's business. And so, in traditional stories, a lot of times, the hero will go to a witch or a sea witch for advice, or for knowledge, or for medicines, or whatever. So, it kind of tied together the fact that I was in the Coast Guard—so I've always been around oceans—my Old English, Middle English background.And yeah, it just—the name made sense to me. So, it was like, “Well, I have a name now. Let's just do it.” And so I did it. And then as the year went on, I started getting a lot of interest, different friends in the industry found out what I was doing, or they found out through a friend, an alumni classmate of mine pinged me going, “Hey, this company really needs your help. Can I do an introduction?” I said, “Okay.”And so it started taking off. And so by September, I was like, “Well, if I can get a couple things lined up, I'm going to have too much to do with the job I love, which is Merewif, to stay at a day job that I'm like, ‘Ehh. It's a job.'” And it's been incredible. Like, it's busy. It sometimes means waking up at two in the morning to see what you're up to.Corey: It happens, sometimes. To be clear, that is out of your own choice. The beautiful thing about my business is that it's strictly a business hours problem.Ana: Yes, except I knew that the video was launching today, and I wanted to take one more scrub on it to make sure that [laugh] there wasn't anything over the line.Corey: Yeah. We go right up to it, but try not to cross it.Ana: Yes. And so—and that's the killer thing is, like, I'm loving every day. Like, it's crazy. It's different things. I do hate being my own finance department. But you know.Corey: Fractional CFOs are one of our first strategic hires that we made here, and it was a bit of a stretch, and it's a, “We think we can afford it because, Dan”—who's been a guest on this show—“As our CFO says we can, and that's sort of his job, so all right. Let's see what happens.” And sure, it's great way to fail if he's not good at his job, but he was right. And it has been an absolute Godsend just for the things I don't have to worry about that have been taken off of my head, are—it's like not having to plan a wedding anymore. That level of relief.Ana: [laugh].Oh, yeah. Covid messed up my wedding, too. So, that ended up being in our backyard. But you know, at the end of the day, every day I'm doing work that I've spent my whole career becoming really good at, and becoming an expert at, and being able to talk with [countries 00:32:30] that can't necessarily afford to hire someone like me full time, but to be able to walk them through, “All right, here's the cloud technologies that are available for you, but you're also going to want to have, for example, a snowball edge in your area because you're going to lose connectivity.” And, “Oh, hey, talk to the guys over at Project OWL.”It's a cool one if you haven't looked at it. They're basically these floating little—they look like little ducks; well, the original versions of them did—and they basically allow—they're WiFi repeaters in some ways, where they float. So, if you disperse them in an area where disasters happen, even if it flooded, it's going to keep that wireless network up and available in that entire area, for everyone who's impacted. Which is a huge problem in the last mile. So, getting to do this stuff that I love anyway, it was just time.And I'm loving teaching at the UW. I'm back at the program I actually graduated from. And this will be of no shock to you, at some point in the near future, I'm going to be applying to do my PhD. It's been a goal of mine since I was little to be the first PhD in my family. Were weirdly competitive about very strange things.Corey: I will be extremely disappointed if your dissertation does not feature the word ‘shitposting,' and of course, a link to something that cites my work.Ana: Actually shitposting could end up in there because what I really want to study is the impact of emerging technologies, including social media and things like that, and how they're impacting the ability of responders to have a common operating picture. So, it's clouding the ability. So, a common operating picture is how the Coast Guard and the Fish and Wildlife and the local fire department all know what's going on when a disaster happens, right? That's great, but they now all have separate systems. And if you think the local fire department or the local fisheries guys have the same level of security as, say, the Coast Guard does on their systems, they don't.So, how do you get them into the same common operating picture? And then what happens if it's a hurricane, and you have people tweeting pictures of the hurricane, and they're not even in the area from the hurricane? So, you have all this additional noise, you have all these additional security needs that weren't there, say, during Katrina, when we were doing everything by, like—no joke—a lot of faxing and text messaging and driving things back and forth. How do you deal with that? So yeah, that's actually what I'm looking at doing.So yeah, shitposting might end up in there as a what do you do when you're in a disaster and you have shitposting cluttering up your mess? So yeah, that's what I'm hoping to do at some point. But I've got so much work right now with Merewif that, right now, I don't have time to get the PhD. [laugh]. So.Corey: Industry and academia tend to be a little on the different side. And for what it's worth, like, there are a lot of companies doing PR, crisis comms work, et cetera, et cetera. The reason that there was really—this was one of those no-bid contracts because you understand this industry in a way that few people do. You've worked within it, you understand the dynamics within it, as well as adjacent industries like gaming, for example. Having someone who understands the moving parts of an industry, who the major players are and how that all fits together, it's something that you can't take some random comms firm off the street and expect them to understand it in the evolving way that social media, among others, has really shifted the entire narrative. So, I don't know of anyone else who's doing it the way that you do. They're certainly not talking about it the same.Ana: Way. There are a few firms that do something similar, but they're bigger and they have a lot of people and they're not as specialized as I am. So, they have an idea of it, but they're not necessarily from that industry. Or, you know, I've been playing video games since I was—what—ten. And I've been very involved. I do panels about women in the military, and how we're represented in video games and comic books, I do those quite often.Actually, real quick, that reminds me back to the PhD thing.Corey: Of course.Ana: The other reason I want to get the PhD is because, as a woman, having that extra boost of not only have I been doing this for—oh God, almost 20 years; that makes me feel really old—almost 20 years, but I also have a PhD in this specific technique. In order to get this PhD, I have to convince a university to let me combine an IT PhD, like, either an information technology or an IS tech—like, a science PhD and a communication PhD into one. There is no school that quite offers what I want, so I'm going to actually have to combine them. But I will say that one of the other reasons I really want to do it, other than the fact that I get to look at my little brother—who you know—and go, “Pttht, I got it first,” is because as a woman, it does give me one more way to keep the door open that my male counterparts don't necessarily need. And as you know, in this industry, that's a lot. I mean, it's not easy being a younger-looking blue-haired woman who's like, “Hi, I know my shit.”Corey: Meanwhile, I am presumed competent in a way that people who aren't over-represented are not. And when I say something, it is presumed true, as opposed to being nibbled to death by ducks with, “Well, can you back up that assertion?” Because sometimes, no. I'm speculating, but I am presumed to be right as a default.Ana: Yep.Corey: And people love to say that, “Oh, yeah, privilege isn't really a thing.” Let's be very clear here. I did have to build a lot of the stuff that's here. None of this was handed to me. But I didn't have a headwind at fighting against me every step of the way the I would have if I didn't look like this.Ana: One of the things I've joked about a lot is my being a veteran, has actually helped me with some of those headwinds because there are assumptions made about my personality—[laugh] the fact that I'm blunt, the fact that I—Corey: No.Ana: —tend to be very straightforward. And I believe my very first meeting with Ariel Kelman when he was a VP at Amazon—at AWS—was, in one of the meetings, the very first one was the words, “Are you shitting me?” Came out of my mouth over something. [laugh]. Could help it; just came out of my mouth.I am very good at filtering when I need to, but in that moment, whoof, I couldn't have. So, being a veteran does help a bit because there's some personality assumptions that other women deal with the, “Oh, she's a bitch.” With me. It's, “Oh, she's scary because she was a veteran.” I'm like, “All right. [laugh]. Cool. We'll lean into that. We will tell you this has been my personality since I was five. We'll let you think it was the Coast Guard that made me this way.”Corey: You joined early. Got it.Ana: Oh, totally. Joined at five. Well, my dad was Coast Guard, so let's just count that. I grew up in the Coast Guard.Corey: I just never grew up. It was easier.Ana: You know, when they're going to let you drive a 378-foot ship, you kind of have to grow up a little bit.Corey: One would hope anyway.Ana: [laugh]. Well, and I mean, you know, there's the other factor is that, you know—actually, in my AWS interview, I think I scared my Bar Raiser by telling one of these stories—there were times where when I made a decision, someone could get killed if I was wrong.Corey: So, that does happen at Amazon scale, but less frequently than it does in the armed services.Ana: Well, yeah. I mean, there it's you're literally being dumb and leaving people in place in front of a tornado, which I'm not going to get into. I'm very—Corey: Or a power bus is—a safety isn't put on and someone gets electrocuted. But it's always small-scale stuff, not—it's not as common.Ana: Yeah. And when you're doing—like, I was a search and rescue controller, and I had to know the area I was operating in the winds, the potential risks, what type of vessels were in that area, and then we had a computer software called SAROPS that helped me search. But, like, growing up in an industry where if I screwed up someone could die gives you a completely different perspective on a lot of things.Corey: Compared to that, there is no stress in the computer industry. There really isn't.Ana: I used to joke at launch when people were freaking out—and I told Ariel this once and I thought he was going to snort his coffee—but we were sitting there and people were like, “Oh, my gosh,” for re:Invent I was like, “Is the building flooding?” “No.” “Is it on fire?” “No.” “Is anyone shooting at us?” “No.” “Okay, cool. Chill out. [laugh]. It'll be okay.”Corey: Yeah, “You can weather some mean tweets. I promise. It'll be okay.”Ana: “Deep breaths.” But you know, at the same time, on the empathy scale is understanding that not everyone has that experience. So, that's the other thing that's critical to understand as a crisis communicator or as a leader of any kind, is that the stresses and crazy things I've been through have made me who I am. The stresses and crazy things you've been through have made you who you are, right? Well, what you find—what will trigger your brain to go, “This is fight or flight. Oh, my gosh, this is terrifying. Oh, gosh, I could”—you know, for some of these people at re:Invent, “Oh, my gosh, I could lose my job. If I lose my job, I can't feed my family.”So, even though I don't panic because I'm like, “Meh, no one's shooting at me. Cool.” Understanding that for the person next to them, they could physically be having that response of fight or flight is a critical part of leadership and crisis comms. You know, I think too often people are like, “Oh, my hardship beats your hardship.” Well, yeah.Not everyone has been in 60-foot seas where they literally bounce off bulkheads and pass a mushroom through their nose because, by the way, you can get that seasick. But it's true. And if you look at some of the younger people you're hiring, what they consider as, “Oh, my gosh, this could be a problem.” You're like, “Well, okay. We're going to be okay. Take a breath.”Corey: Perspective is one of those things that comes with experience, for better or worse.Ana: [laugh]. Yeah, right?Corey: So, I want to thank you for taking so much time to speak with me today.Ana: Oh, absolutely.Corey: If people want to learn more, where can they find you?Ana: So, I am @acvisneski, on Twitter. And also, my webpage is the—T-H-E—merewif—M-E-R-E-W-I-F dot com. Those are the two best places.Corey: And we'll put them in the [show notes 00:42:00], of course.Ana: Awesome.Corey: Thank you so much for joining me today. I really appreciate it.Ana: Oh, happy to. It's always fun.Corey: It really is. Ana Visneski, Chief Chaos Coordinator at the Merewif. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment telling me that this was the worst possible way to find out that Shtephen was no longer with us.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

About LauraLaura leads the research program at Jeli.io.  She has a Master's degree in Human Factors & Systems Safety and a PhD in Cognitive Systems Engineering. Her doctoral work focused on distributed incident response practices in DevOps teams responsible for critical digital services. She was a researcher with the SNAFU Catchers Consortium from 2017-2020 and her research interests lie in resilience engineering, coordination design and enabling adaptive capacity across distributed work teams. As a backcountry skier and alpine climber, she also studies cognition & resilient performance in high risk, high consequence mountain environments.  Links: Howie: The Post-Incident Guide: https://www.jeli.io/howie-the-post-incident-guide/ Jeli: https://www.jeli.io Twitter: https://twitter.com/lauramdmaguire TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Today's episode is brought to you in part by our friends at MinIO the high-performance Kubernetes native object store that's built for the multi-cloud, creating a consistent data storage layer for your public cloud instances, your private cloud instances, and even your edge instances, depending upon what the heck you're defining those as, which depends probably on where you work. It's getting that unified is one of the greatest challenges facing developers and architects today. It requires S3 compatibility, enterprise-grade security and resiliency, the speed to run any workload, and the footprint to run anywhere, and that's exactly what MinIO offers. With superb read speeds in excess of 360 gigs and 100 megabyte binary that doesn't eat all the data you've gotten on the system, it's exactly what you've been looking for. Check it out today at min.io/download, and see for yourself. That's min.io/download, and be sure to tell them that I sent you.Corey: This episode is sponsored in part by our friends at Sysdig. Sysdig is the solution for securing DevOps. They have a blog post that went up recently about how an insecure AWS Lambda function could be used as a pivot point to get access into your environment. They've also gone deep in-depth with a bunch of other approaches to how DevOps and security are inextricably linked. To learn more, visit sysdig.com and tell them I sent you. That's S-Y-S-D-I-G dot com. My thanks to them for their continued support of this ridiculous nonsense.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. One of the things that's always been a treasure and a joy in working in production environments is things breaking. What do you do after the fact? How do you respond to that incident?Now, very often in my experience, you dive directly into the next incident because no one has time to actually fix the problems but just spend their entire careers firefighting. It turns out that there are apparently alternate ways. My guest today is Laura Maguire who leads the research program at Jeli, and her doctoral work focused on distributed incident response in DevOps teams responsible for critical digital services. Laura, thank you for joining me.Laura: Happy to be here, Corey, thanks for having me.Corey: I'm still just trying to wrap my head around the idea of there being a critical digital service, as someone whose primary output is, let's be honest, shitposting. But that's right, people do use the internet for things that are a bit more serious than making jokes that are at least funny only to me. So, what got you down this path? How did you get to be the person that you are in the industry and standing in the position you hold?Laura: Yeah, I have had a long circuitous route to get to where I am today, but one of the common threads is about safety and risk and how do people manage safety and risk? I started off in natural resource industries, in mountain safety, trying to understand how do we stop things from crashing, from breaking, from exploding, from catching fire, and how do we help support the people in those environments? And when I went back to do my PhD, I was tossed into the world of software engineers. And at first I thought, now, what do firefighters, pilots, you know, emergency room physicians have to do with software engineers and risk in software engineering? And it turns out, there's actually a lot, there's a lot in common between the types of people who handle real-time failures that have widespread consequences and the folks who run continuous deployment environments.And so one of the things that the pandemic did for us is it made it immediately apparent that digital service delivery is a critical function in society. Initially, we'd been thinking about these kinds of things as being financial markets, as being availability of electronic health records, communication systems for disaster recovery, and now we're seeing things like communication and collaboration systems for schools, for businesses, this helps keep society functioning.Corey: What makes part of this field so interesting is that the evolution in the space where, back when I first started my career about a decade-and-a-half ago, there was a very real concern in my first Linux admin gig when I accidentally deleted some of the data from the data warehouse that, “Oh, I don't have a job anymore.” And I remember being surprised and grateful that I still did because, “Oh, you just learned something. You going to do it again?” “No. Well, not like that exactly, but probably some other way, yeah.”And we have evolved so far beyond that now, to the point where when that doesn't happen after an incident, it becomes almost noteworthy in its own right and it blows up on social media. So, the Overton window of what is acceptable disaster response and incident management, and how we learn from those things has dramatically shifted even in the relatively brief window of 15 years. And we're starting to see now almost a next-generation approach to this. One thing that you were, I believe the principal author behind is Howie: The Post-Incident Guide, which is a thing that you have up on jeli.io—that's J-E-L-I dot I-O—talking about how to run post-incident investigations. What made you decide to write something like this?Laura: Yeah, so what you described at the beginning there about this kind of shift from blameless—blameful-type approaches to incident response to thinking more broadly about the system of work, thinking about what does it mean to operate in continuous deployment environments is really fundamental. Because working in these kinds of worlds, we don't have an established knowledge base about how these systems work, about how they break because they're continuously changing, the knowledge, the expertise required to manage them is continuously changing. And so that shift towards a blameless or blame-aware post-incident review is really important because it creates this environment where we can actually share knowledge, share expertise, and distribute more of our understandings of how these systems work and how they break. So that, kind of, led us to create the Howie Guide—the how we got here post-incident guide. And it was largely because companies were kind of coming from this position of, we find the person who did the thing that broke the system and then we can all rest easy and move forward. And so it was really a way to provide some foundation, introduce some ideas from the resilience engineering literature, which has been around for, you know, the last 30 or 40 years—Corey: It's kind of amazing, on some level, how tech as an industry has always tried to reinvent things from first principles. I mean, we figured out long before we started caring about computers in the way we do that when there was an incident, the right response to get the learnings from it for things like airline crashes—always a perennial favorite topic in this space for conference talks—is to make sure that everyone can report what happened in a safe way that's non-accusatory, but even in the early-2010s, I was still working in environments where the last person to break production or break the bill had the shame trophy hanging out on their desk, and it would stay there until the next person broke it. And it was just a weird, perverse incentive where it's, “Oh if I broke something, I should hide it.”That is absolutely the most dangerous approach because when things are broken, yes, it's generally a bad thing, so you may as well find the silver lining in it from my point of view and figure out, okay, what have we learned about our systems as a result of the way that these things break? And sometimes the things that we learn are, in fact, not that deep, or there's not a whole lot of learnings about it, such as when the entire county loses power, computers don't work so well. Oh, okay. Great, we have learned that. More often, though, there seem to be deeper learnings.And I guess what I'm trying to understand is, I have a relatively naive approach on what the idea of incident response should look like, but it's basically based on the last time I touched things that were production-looking, which was six or seven years ago. What is the current state of the art that the advanced leaders in the space as they start to really look at how to dive into this? Because I'm reasonably certain it's not still the, “Oh, you know, you can learn things when your computers break.” What is pushing the envelope these days?Laura: Yeah, so it's kind of interesting. You brought up incident response because incident response and incident analysis are the, sort of like, what do we learn from those things are very tightly coupled. What we can see when we look at someone responding in real-time to a failure is, it's difficult to detect all of the signals; they don't pop up and wave a little flag and say, like, “I am what's broken.” There's multiple compounding and interacting factors. So, there's difficulty in the detection phase; diagnosis is always challenging because of how the systems are interrelated, and then the repair is never straightforward.But when we stop and look at these kinds of things after the fact, of really common theme emerges, and that it's not necessarily about a specific technical skill set or understanding about the system, it's about the shared, distributed understanding of that. And so to put that in plain speak, it's what do you know that's important to the problem? What do I know that's important to the problem? And then how do we collectively work together to extract that specific knowledge and expertise, and put that into practice when we're under time pressure, when there's a lot of uncertainty, when we've got the VP DMing us and being like, “When's the system going to be back up?” and Twitter's exploding with unhappy customers?So, when we think about the cutting edge of what's really interesting and relevant, I think organizations are starting to understand that it's how do we coordinate and we collaborate effectively? And so using incident analysis as a way to recognize not only the technical aspects of what went wrong but the social aspects of that as well. And the teamwork aspects of that is really driving some innovation in this space.Corey: It seems to me, on some level, that the increasing sophistication of what environments look like is also potentially driving some of these things. I mean, again, when you have three web servers and one of them's broken, okay, it's a problem; we should definitely jump on that and fix it. But now you have thousands of containers running hundreds of microservices for some Godforsaken reason because what we decided this thing that solves the problem of 500 engineers working on the same repository is a political problem, so now we're going to use microservices for everything because, you know, people. Great. But then it becomes this really difficult to identify problem of what is actually broken?And past a certain point of scale, it's no longer a question of, “Is it broken?” so much as, “How broken is it at any given point in time?” And getting real-time observability into what's going on does pose more than a little bit of a challenge.Laura: Yeah, absolutely. So, the more complexity that you have in the system, the more diversity of knowledge and skill sets that you have. One person is never going to know everything about the system, obviously, and so you need kind of variability in what people know, how current that knowledge is, you need some people who have legacy knowledge, you have some people who have bleeding edge, my fingers were on the keyboard just moments ago, I did the last deploy, that kind of variability in whose knowledge and skill sets you have to be able to bring to bear to the problem in front of you. One of the really interesting aspects, when you step back and you start to look really carefully about how people work in these kinds of incidents, is you have folks that are jumping, get things done, probe a lot of things, they look at a lot of different areas trying to gather information about what's happening, and then you have people who sit back and they kind of take a bit of a broader view, and they're trying to understand where are people trying to find information? Where might our systems not be showing us what's going on?And so it takes this combination of people working in the problem directly and people working on the problem more broadly to be able to get a better sense of how it's broken, how widespread is that problem, what are the implications, what might repair actually look like in this specific context?Corey: Do you suspect that this might be what gives rise, sometimes, to it seems middle management's perennial quest to build the single pane of glass dashboard of, “Wow, it looks like you're poking around through 15 disparate systems trying to figure out what's going on. Why don't we put that all on one page?” It's a, “Great, let's go tilt at that windmill some more.” It feels like it's very aligned with what you're saying. And I just, I don't know where the pattern comes from; I just know I see it all the time, and it drives me up a wall.Laura: Yeah, I would call that pattern pretty common across many different domains that work in very complex, adaptive environments. And that is—like, it's an oversimplification. We want the world to be less messy, less unstructured, less ad hoc than it often is when you're working at the cutting edge of whatever kind of technology or whatever kind of operating environment you're in. There are things that we can know about the problems that we are going to face, and we can defend against those kinds of failure modes effectively, but to your point, these are very largely unstructured problem spaces when you start to have multiple interacting failures happening concurrently. And so Ashby, who back in 1956 started talking about, sort of, control systems really hammered this point home when he was talking about, if you have a world where there's a lot of variability—in this case, how things are going to break—you need a lot of variability in how you're going to cope with those potential types of failures.And so part of it is, yes, trying to find the right dashboard or the right set of metrics that are going to tell us about the system performance, but part of it is also giving the responders the ability to, in real-time, figure out what kinds of things they're going to need to address the problem. So, there's this tension between wanting to structure unstructured problems—put those all in a single pane of glass—and what most folks who work at the frontlines of these kinds of worlds know is, it's actually my ability to be flexible and to be able to adapt and to be able to search very quickly to gather the information and the people that I need, that are what's really going to help me to address those hard problems.Corey: Something I've noticed for my entire career, and I don't know if it's just unfounded arrogance, and I'm very much on the wrong side of the Dunning-Kruger curve here, but it always struck me that the corporate response to any form of outage has is generally trending toward oh, we need a process around this, where it seems like the entire idea is that every time a thing happens, there should be a documented process and a runbook on how to perform every given task, with the ultimate milestone on the hill that everyone's striving for is, ah, with enough process and enough runbooks, we can then eventually get rid of all the people who know all this stuff works, and basically staff at up with people who'd know how to follow a script and run push the button when told to buy the instruction manual. And that's always rankled, as someone who got into this space because I enjoy creative thinking, I enjoy looking at the relationships between things. Cost and architecture are the same thing; that's how I got into this. It's not due to an undying love of spreadsheets on my part. That's my business partner's problem.But it's this idea of being able to play with the puzzle, and the more you document things with process, the more you become reliant on those things. On some level, it feels like it ossifies things to the point where change is no longer easily attainable. Is that actually what happens, or am I just wildly overstating the case? Either as possible. Or a third option, too. You're the expert; I'm just here asking ridiculous questions.Laura: Yeah, well, I think it's a balance between needing some structure, needing some guidelines around expected actions to take place. This is for a number of reasons. One, we talked about earlier about how we need multiple diverse perspectives. So, you're going to have people from different teams, from different roles in the organization, from different levels of knowledge, participating in an incident response. And so because of that, you need some form of script, some kind of process that creates some predictability, creates some common ground around how is this thing going to go, what kinds of tools do we have at our disposal to be able to either find out what's going on, fix what's going on, get the right kinds of authority to be able to take certain kinds of actions.So, you need some degree of process around that, but I agree with you that too much process and the idea that we can actually apply operational procedures to these kinds of environments is completely counterproductive. And what it ends up doing is it ends up, kind of, saying, “Well, you didn't follow those rules and that's why the incident went the way it did,” as opposed to saying, “Oh, these rules actually didn't apply in ways that really matter, given the problem that was faced, and there was no latitude to be able to adapt in real-time or to be able to improvise, to be creative in how you're thinking about the problem.” And so you've really kind of put the responders into a bit of a box, and not given them productive avenues to, kind of, move forward from. So, having worked in a lot of very highly regulated environments, I recognize there's value in having prescription, but it's also about enabling performance and enabling adaptive performance in real-time when you're working at the speeds and the scales that we are in this kind of world.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance query accelerator for the Oracle MySQL Database Service, although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLAP and OLTP—don't ask me to pronounce those acronyms again—workloads directly from your MySQL database and eliminate the time-consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: Yeah, and let's be fair, here; I am setting up something of a false dichotomy. I'm not suggesting that the answer is oh, you either are mired in process, or it is the complete Wild West. If you start a new role and, “Great. How do I get started? What's the onboarding process?” Like, “Step one, write those docs for us.”Or how many times have we seen the pattern where day-one onboarding is, “Well, here's the GitHub repo, and there's some docs there. And update it as you go because this stuff is constantly in motion.” That's a terrible first-time experience for a lot of folks, so there has to be something that starts people off in the right direction, a sort of a quick guide to this is what's going on in the environment, and here are some directions for exploration. But also, you aren't going to be able to get that to a level of granularity where it's going to be anything other than woefully out of date in most environments without resorting to draconian measures. I feel like—Laura: Yeah.Corey: —the answer is somewhere in the middle, and where that lives depends upon whether you're running Twitter for Pets or a nuclear reactor control system.Laura: Yeah. And it brings us to a really important point of organizational life, which is that we are always operating under constraints. We are always managing trade-offs in this space. It's very acute when you're in an incident and you're like, “Do I bring the system back up but I still don't know what's wrong or do I leave it down a little bit longer and I can collect more information about the nature of the problem that I'm facing?”But more chronic is the fact that organizations are always facing this need to build the next thing, not focus on what just happened. You talked about the next incident starting and jumping in before we can actually really digest what just happened with the last incident; these kinds of pressures and constraints are a very normal part of organizational life, and we are balancing those trade-offs between time spent on one thing versus another as being innovating, learning, creating change within our environment. The reason why it's important to surface that is that it helps change the conversation when we're doing any kind of post-incident learning session.It's like, oh, it allows us to surface things that we typically can't say in a meeting. “Well, I wasn't able to do that because I know that team has a code freeze going on right now.” Or, “We don't have the right type of, like, service agreement to get our vendor on the phone, so we had to sit and wait for the ticket to get dealt with.” Those kinds of things are very real limiters to how people can act during incidents, and yet, don't typically get brought up because they're just kind of chronic, everyday things that people deal with.Corey: As you look across the industry, what do you think that organizations are getting, I guess, it's the most wrong when it comes to these things today? Because most people are no longer in the era of, “All right. Who's the last person to touch it? Well, they're fired.” But I also don't think that they're necessarily living the envisioned reality that you described in the Howie Guide, as well as the areas of research you're exploring. What's the most common failure mode?Laura: Hmm. I got to tweak that a little bit to make it less about the failure mode and more about the challenges that I see organizations facing because there are many failure modes, but some common issues that we see companies facing is they're like, “Okay, we buy into this idea that we should start looking at the system, that we should start looking beyond the technical thing that broke and more broadly at how did different aspects of our system interact.” And I mean, both people as a part of the system, I mean processes part of the system, as well as the software itself. And so that's a big part of why we wrote the Howie Guide, is because companies are struggling with that gap between, “Okay, we're not entirely sure what this means to our organization, but we're willing to take steps to get there.” But there's a big gap between recognizing that and jumping into the academic literature that's been around for many, many years from other kinds of high-risk, high-consequence type domains.So, I think some of the challenges they face is actually operationalizing some of these ideas, particularly when they already have processes and practices in place. There's ideas that are very common throughout an organization that take a long time to shift people's thinking around, the implicit biases or orientations towards a problem that we as individuals have, all of those kinds of things take time. You mentioned the Overton window, and that's a great example of it is intolerable in some organizations to have a discussion about what do people know and not know about different aspects of the system because there's an assumption that if you're the engineer responsible for that, you should know everything. So, those challenges, I think, are quite limiting to helping organizations move forward. Unfortunately, we see not a lot of time being put into really understanding how an incident was handled, and so typically, reviews get done on the side of the desk, they get done with a minimal amount of effort, and then the learnings that come out of them are quite shallow.Corey: Is there a maturity model, where it makes sense to begin investing in this, whereas if you've do it too quickly, you're not really going to be able to ship your MVP and see what happens; if you go too late, you have a globe-spanning service that winds up being down all the time so no one trusts it. What is the sweet spot for really started to care about incident response? In other words, how do people know that it's time to start taking this stuff more seriously?Laura: Ah. Well… you have kids?Corey: Oh, yes. One and four. Oh yeah.Laura: Right—Corey: Demons. Little demons whom I love very much.Laura: [laugh]. They look angelic, Corey. I don't know what you're talking about. Would you not teach them how to learn or not teach them about the world until they started school?Corey: No, but it would also be considered child abuse at this age to teach them about the AWS bill. So, there is a spectrum as far as what is appropriate learnings at what stage.Laura: Yeah, absolutely. So, that's a really good point is that depending on where you are at in your operation, you might not have the resources to be able to launch full-scale investigations. You may not have the complexity within your system, within your teams, and you don't have the legacy to, sort of, draw through, to pull through, that requires large-scale investigations with multiple investigators. That's really why we were trying to make the Howie Guide very applicable to a broad range of organizations is, here are the tools, here are the techniques that we know can help you understand more about the environment that you're operating in, the people that you're working with, so that you can level up over time, you can draw more and more techniques and resources to be able to go deeper on those kinds of things over time. It might be appropriate at an early stage to say, hey, let's do these really informally, let's pull the team together, talk about how things got set up, why choices were made to use the kinds of components that we use, and talk a little bit more about why someone made a decision they did.That might be low-risk when you're small because y'all know each other, largely you know the decisions, those conversations can be more frank. As you get larger, as more people you don't know are on those types of calls, you might need to handle them differently so that people have psychological safety, to be able to share what they knew and what they didn't know at the time. It can be a graduated process over time, but we've also seen very small, early-stage companies really treat this seriously right from the get-go. At Jeli, I mean, one of our core fundamentals is learning, right, and so we do, we spend time on sharing with each other, “Oh, my mental model about this was X. Is that the same as what you have?” “No.” And then we can kind of parse what's going on between those kinds of things. So, I think it really is an orientation towards learning that is appropriate any size or scale.Corey: I really want to thank you for taking the time to speak with me today. If people want to learn more about what you're up to, how you view these things and possibly improve their own position on these areas, where can they find you?Laura: So, we have a lot of content on jeli.io. I am also on Twitter at—Corey: Oh, that's always a mistake.Laura: [laugh]. @lauramdmaguire. And I love to talk about this stuff. I love to hear how people are interpreting, kind of, some of the ideas that are in the resilience engineering space. Should I say, “Tweet at me,” or is that dangerous, Corey?Corey: It depends. I find that the listeners to this show are all far more attractive than the average, and good people, through and through. At least that's what I tell the sponsors. So yeah, it should be just fine. And we will of course include links to those in the [show notes 00:27:11].Laura: Sounds good.Corey: Thank you so much for your time. I really appreciate it.Laura: Thank you. It's been a pleasure.Corey: Laura Maguire, researcher at Jeli. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please give a five-star review on your podcast platform of choice along with an angry, insulting comment that I will read just as soon as I get them all to display on my single-pane-of-glass dashboard.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

About Serena Serena is a Network Engineer who specializes in Data Center Compute and Virtualization. She has degrees in Computer Information Systems with a concentration on networking and information security and is currently pursuing a master's in Data Center Systems Engineering. She is most known for her content on TikTok and Twitter as Shenetworks. Serena's content focuses on networking and security for beginners which has included popular videos on bug bounties, switch spoofing, VLAN hoping, and passing the Security+ certification in 24 hours.Links: Cisco cert Discord study group:https://discord.com/invite/uXQ8yWnN8a Beacons:https://beacons.page/shenetworks TikTok:https://www.tiktok.com/@shenetworks sysengineer's TikTok:https://www.tiktok.com/@sysengineer Twitter:https://twitter.com/notshenetworks TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Sysdig. Sysdig is the solution for securing DevOps. They have a blog post that went up recently about how an insecure AWS Lambda function could be used as a pivot point to get access into your environment. They've also gone deep in-depth with a bunch of other approaches to how DevOps and security are inextricably linked. To learn more, visit sysdig.com and tell them I sent you. That's S-Y-S-D-I-G dot com. My thanks to them for their continued support of this ridiculous nonsense.Corey: Today's episode is brought to you in part by our friends at MinIO the high-performance Kubernetes native object store that's built for the multi-cloud, creating a consistent data storage layer for your public cloud instances, your private cloud instances, and even your edge instances, depending upon what the heck you're defining those as, which depends probably on where you work. It's getting that unified is one of the greatest challenges facing developers and architects today. It requires S3 compatibility, enterprise-grade security and resiliency, the speed to run any workload, and the footprint to run anywhere, and that's exactly what MinIO offers. With superb read speeds in excess of 360 gigs and 100 megabyte binary that doesn't eat all the data you've gotten on the system, it's exactly what you've been looking for. Check it out today at min.io/download, and see for yourself. That's min.io/download, and be sure to tell them that I sent you.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Today's guest was on relatively recently, but it turns out that when I have people on the show to talk about things, invariably I tend to continue talking to them about things and that leads down really interesting rabbit holes. Today is a stranger rabbit hole than most. Joining me once again is @SheNetworks or Serena [DiPenti 00:00:51]. Thanks for coming back and subjecting yourself to, basically, my nonsense all over again in the same month.Serena: Thanks for having me back. Excited.Corey: So, you have a, I think study group is the term that you're using. I don't know how to describe it in a way that doesn't make me sound ridiculous and describing and speaking with my hands and the rest. It's a Discord, as the kids of today tend to use. There are some private channels on an existing Discord group, and we'll get to the mechanics of that in a second. But it's a study group for various Cisco certifications, which it's been a while since I had one; my CCNA is something I took back in 2009. I've checked, it's expired to the point where they can't even look it up anymore to figure out who I might have been, once upon a time. What is this group and where did it come from?Serena: Yeah, so the Discord itself is kind of a collective of a bunch of people that are creators on TikTok. And it's just, like, a cool place to connect, especially people from TikTok join, people from Twitter join, they want to interact, you know, a great place to get resources if you're early in your career. I—you know, new year, new me resolution was [laugh] I wanted to start studying for the CCNP a little bit, and I've been doing it pretty loosely for a while, but I kind of was like, all right, time to actually sit down and dedicate some real time to this. And I put on Twitter, you know, if anybody else was interested—I know there's other various study groups out there and things like that, but I was just like, hey, you know, it was anyone interested and a study group and I got really good response. Of course, a lot of people are at the CCNA level, so I made a channel for CCNA and CCNP, so whatever level you're at, you can come in and ask questions. It's really great.Corey: One thing that irked me when I first joined, as well, there's no CCENT which was sort of the entry-level Cisco cert, the first half of the CCNA, and I did a bit of googling before shooting my mouth off. And it turns out that Cisco sunset that cert a while back, so CCNA is now the entry-level cert, as I understand it.Serena: Yeah. So, when I did my CCNA, I did the C-C-E-N-T—the CCENT, and then the ICND2, and that's how I got my CCNA. And then I went and got the Data Center CCNA, which was two exams… two? Or maybe it was just one. I can't remember fully. But they basically got rid of all of their CCNAs and created one new one that's just the CCNA Enterprise.Corey: What I found worked out for me when I was going through the process of getting the CCNA—the CCENT, I forget how at the time, came along for the ride. And it was the CCENT, the baseline stuff that really added value to my entire career. That piece of advice that I would give anyone in the technical space is when your hand-waving over a thing you don't really understand. Maybe stop doing that one afternoon when you don't have anything else going on, dig into it.For me, it was always, “What the hell is a subnet mask?” “I don't know. It's the thing that I put the right numbers in, the box stops turning gray and will turn black and let me click the button; life goes on.” Figuring out what that meant and how it was calculated was interesting and it made me understand what's going on at a deeper level. Which means that invariably when things break as—they're computers; they break—I could have a better understanding of the holistic system and ideally have a better chance of getting to an outcome of fixing it.So, I'm not sitting here suggesting that anyone who wants to, “Oh, you want to work in the cloud and go and build things out on top of AWS or GCP. Great, go and get a Cisco certification is the first stop along your journey.” But understanding how the network works is absolutely going to serve you well for the rest of your technical career because not a lot has changed in the networking sense over the past 13 years since I sat the certification exam. It turns out that the TCP handshake still works the same way: Badly.Serena: [laugh]. Yeah, and to your point, the troubleshooting part is really where you need that depth of knowledge, right? And that's typically when it's crunch time and things are gone awry. And you really need to have an understanding of okay, is it the subnet mask? And the quicker that you can identify that outage, that problem, the quicker you get a resolution. And you do need depth of knowledge for that, and understanding that kind of underlying infrastructure is so helpful.Corey: And that was always the useful part of the certification—and the exam that went along with it—to me was, “Okay, with a subnet mask of whatever you're talking about here, great. How many usable IP addresses are there in the network?” And yeah, that's the kind of thing that we really care about.The stuff that drove me nuts was the other half of it, where it's the, “Ah, what is the proper syntactical command on the Cisco command line to display this thing?” And it's, “First, I can probably look that up or tab-complete it or whatnot. Secondly, I get it's a Cisco exam, but this is a world where interoperability is very much a thing and it is incredibly likely that the thing I need to find that out on is not going to ultimately be a Cisco device, once I'm working in enterprise.”Serena: Yeah, I do have similar feedback when it comes to that because right now, I've been trying to do kind of a chapter a day out of the Cisco Press book, and that's my main source of studying right now. I like to read a lot, so reading is usually my main method of studying, I guess. But I'm in a chapter right now that's, like, 100 pages of just hardware specifics. And we're talking about, like, PCIe cards and VICs and the different models and which ports are unified and you can configure for Fibre Channel, and which are uplink on the different generations. And I'm like, “Ohh.”I hate that. It's my least favorite part of studying because for that, I mean, I always just pull up the documentation. And it's like, “Okay, here's the ports that can be, you know, configured as Fibre Channel over Ethernet or Fibre Channel,” or whatever. Remembering it off the top of my head, which model, which year, which ports, I'm not great with that. And I don't think it's, honestly, that valuable when it comes to certification exams because you really should be using the documentation when you are doing those types of configurations between hardware and generations and compatibility.Corey: We sort of see the same thing in the development space, where, okay, the job we're hiring you to do is to work on some front end work and change how things are rendered, but when we're doing the job interview for that role, oh, now we have an empty whiteboard, we want you to write syntactically valid code that will implement some sorting algorithm or whatnot, while some condescending jerk sits there. And, “Nope, that's not it,” in the background in a high-pressure environment because for that jackwagon, it's any given Thursday, but for you, it determines the next phase of your career. And I hated that stuff. Whereas in the real world, I'm not going to be implementing an algorithm like that in any realistic sense; I'll be using the one built into whatever language I'm using. It's important from a computer science perspective to know it, but from a day-to-day job environment, not so much.And I can't recall the last time that I had to fix a technical issue where I did not have the internet as a resource while I was fixing that issue, even when it's the internet is down because it turns out without the network, I just have a whole bunch of expensive space heaters here, great, my phone still worked. I could check, “Oh, what is the command to get back into that firewall?” That it turns out, I just locked myself out of by—yeah, it turns out when you close a port and you're using that port, mistakes show.Serena: Yeah, I agree with that. And I mean, that goes into the much broader conversation of technical interviews because even as a network engineer, one time I had a whiteboard technical interview where they were asking, like, routing questions, but I didn't have access to any equipment, and so it was just basically asking them questions. And I'm a very visual person, so for me to not be able to, like, kind of put my hands on something and, like, run some commands and look over it myself. I did so horribly in that interview, and I left feeling just, like—I left feeling really bad about myself, honestly, because I had done so bad. And for me, I was assuming they were using some routing protocol. And they're like, “No, it's actually all statically configured.” And I was like, I would be able to know that if I could run commands and, like, actually look. But it was so bad.Corey: Right. And it's stressful working in front of people. I know that whatever I'm typing in front of an audience, I don't do it, but it feels like what I did first is, all right, let me put my mittens on, and then I—because I can't type to save my life, and I look incompetent across five different levels at that point. And yeah, it's these contrived problems. One of the things I like about the study group is when there's a question that is, I guess, not the answer, I would expect, it's okay, we can talk about that. Give me more context behind why.I thought it was this. Clearly, I'm missing something—or the bot is broken—so what is going on here? Help me understand why this is the way that it is? And back when I was learning how this stuff all worked, I went through originally a class at a community college and then finished it up with apparently with sort of a brain dump style boot camp, which I didn't really realize was a thing until after the fact. It was just memorization of these things.Which okay, great. I could memorize my way through some things I would never use again like EIGRP, one of Cisco's proprietary routing protocols that I've never heard of anyone using in the real world before, but I'm sure it's a thing and they're trying to push it. Great. I can skate past that well enough to hang a cert, but it didn't feel like the way to learn it because there was no context. It was just the rote memorization.Serena: Mm-hm. Yeah, and that is very difficult. I'm a big fan of theory, so you know, when we're talking about VIC cards, I was going through each generation, and which you would use for a blade or a rack server, whatever. I think that your time is better spent understanding what a VIC card is, why it's important, maybe, like, the history, and all that instead of being, like, “This version isn't compatible with this UCS blade server,” or whatever. Because I am studying for the Data Center flavor of the CCNP right now, so it's a little bit of a different path. I think most people take the enterprise, that's the more traditional route, switch, IOS. Mine's more UCS, Nexus, HyperFlex type questions.Corey: One thing that I always appreciate is, for example, take subnet mask [crosstalk 00:10:57] calculations. Yeah, I can figure that out on a whiteboard now. But here in the real world, everyone uses a subnet calculator. It's the way that things work. And there's a lot of discussion back and forth about things like that, without talking about the real-world implications, such as, if you're building out two subnets inside of a larger range, don't put them right next to each other because if you need to expand the network later, you're in a world of pain compared to if you had given them some significant breathing room.And okay, great. You probably don't need to use all the [10.0.0.0/8 00:11:30] network in your small-scale environment, and even some larger-scale ones you're hard-pressed to use all those things.It's just the real-world experience, and you understand that you don't want to do that. The second time. The first time you do it because why not? It's easy to remember for humans. And then you run into weird issues with oh, well, why would I ever have more than 254 servers sitting in a subnet—or 253, whatever the number is these days, don't yell at me—great.What about containers running on top of those things? Oh, right, the worst answer to so many architectural patterns, we'll throw some containers at it. And you're back into those problems.Serena: Yeah.Corey: It's the real-world scars you get.Serena: Yeah. And I think that there is such a difference between when you're studying and learning versus—and taking certifications or tests—than in the real world. And that was very discouraging for me when I was first learning because I would take these exams—and we had a Cisco academy where I went to college—and I would take these exams, and my professor was just known for her very difficult test, so I think her advanced routing course, maybe only 30% of the people who took it passed it their first try. And so I would take these exams, I'd walk away being like, “I don't know anything. I'm never going to be a good network engineer, I'm never going to be able to get a job or anything,” because I couldn't regurgitate which show command was showing me errors on a switch, right?And then now in the real world, I'm like, okay, relieved because I was like, I can look this up, like, I can take my time. And then you know, with getting your hands on—I mean, you learned so much within your first year; that is probably more than I learned in all four years of school. But saying that, it was really great for me to have that base of all of that underlying networking and already kind of understanding the terminology alone is such a big… barrier, I would say, like, just being able to sit in a room and listen to these conversations and understand what's going on. That's half the battle in the beginning. [laugh].Corey: I have never heard anyone be prouder of being bad at their job than a professor saying, “I have a 30% pass rate.” Isn't your whole ethos of that role to be someone who teaches people how to do a thing? So, if two-thirds of your class is not learning that thing, it doesn't mean you're a hard grader, it means you're bad at conveying the concept and/or testing for understanding of the thing that you've just taught them. If you're a teacher listening to this, please don't email me until you fix your problem first.Serena: [laugh]. See, and… she would come in and say on the first day class—I took multiple classes with her and she was like, “If you read everything in the book, and pay attention to all the slides, you're still going to fail.” She wanted you to really go above and beyond, and commit and run all these labs and do all these things, and in college, I hated it. I was so resentful and angry because it really did make me feel bad. But at the same time, there was one point someone had asked her a question, and she was like, “Why don't you ask Serena? She has the highest grade in the class.”And I was shocked because I had, like, a C in the [laugh] class. And I was like, “Me? I'm the one that has the highest grade in the class?” And I would definitely do things a little bit differently if I were teaching that course because it, I think, turned off a lot of people into the field. But me passing those grades, I mean, I really could have probably taken the CCNP right when I was done with those courses and passed with flying colors. But I didn't have the money to take the CCNP exams until much later when I had a job. And now it's like so much has changed. The exams have changed. I'm in Data Center now. So, a little bit different. But yeah. [laugh].Corey: I never understood the idea of charging for certs. If people are spending the time and energy to learn about your company's specific technology well enough to take the exam, they're probably going to want to use it in their career as they move forward, so charging a few 100 bucks to sit the test has never struck me as a good idea. And the cloud companies do the exact same things as well. And every company that attains some level of success launches a certification exam, but then they charge a few 100 bucks for it, which… does that money really matter because either you're an engineer, and your company is going to be paying for it, or you're making engineering money these days, and it's just an irritant, but it feels to me like the people that really get disadvantaged by that are the early learners, the students, the folks who are planning to have a career in this, but a few 100 bucks becomes a barrier.Serena: Oh, it's a huge barrier. I mean, it was a big barrier for me. I didn't have money to go to college, so I took out student loans. I worked my way through college and constantly had a job, which then was difficult because my grades suffered because I didn't have the same amount of time.Corey: You did have the highest grade in class, I recall.Serena: [laugh]. For that one course. For the one course. [laugh]. But I didn't have the same amount of time in a day to study as some of my classmates who didn't have to have a job in college.But then also, I couldn't afford $300 to take one exam out of the three that you needed at the time for the CCNP. And that's when I was early in my career. The CCNA, too, like, I didn't have the money to take that exam either. And I think a lot of people are in that position because they are trying to better their knowledge. They're trying to achieve a new job.That's what those certifications are geared towards, right? And so putting that $300—I mean, that person might be working a minimum wage job, and they're trying to get out of that minimum wage job into a higher—paying tech job. And $300 is a lot of money. It is a lot of money. My rent in college was $300. That's a whole month's rent for me, right, to put it in perspective. So yeah.Corey: Yeah. We'll be throwing a bunch of credit codes your way for folks who are learning and [unintelligible 00:17:10] the financial burden because it's important that people be able to not have money being the obstacle to learning a technical field. I am curious, though, as to the genesis of this whole Discord because I heard you talking about it, I joined, but there are a lot of other people talking about different things. Most notably and importantly, there's an Ohio slander channel—Serena: [laugh].Corey: —in there, which is just spot-on perfect from where I sit. But it's not just you, and it's not just networking stuff. It's a systems engineering Slack. Where did it come from?Serena: Yeah so sysengineer, my friend [Chris Lynd 00:17:43]—she's also a TikTok creator—and she set up her own Discord server, which I have kind of like inserted myself into. It's very hard to run your own server, right, so it's kind of more of a collective at this point. But she's sysengineer on TikTok, and so her server is just sysengineer. And there's a lot of memes, right? Because we have a lot of, like, Gen Z—I mean, who doesn't love a good meme? And Chris Lynd, sysengineer, is from Ohio, I'm from Ohio. So, the Ohio slander thing is kind of funny because we're just like always talking crap about Ohio. [laugh].Corey: Which it deserves, let's be very clear here. I have family in Ohio, myself. Every time I visited them, my favorite part was leaving Ohio. I mean, data transfer between AWS regions, the least expensive one is the one cent instead of two cents between Ohio and Virginia because even data wants to get out of Ohio.Serena: It was like, 11 of the astronauts are from Ohio. And it was like, “What about Ohio makes me want to leave the Earth?” [laugh].Corey: Yeah, “How far can I get from Ohio, the absolute furthest place away?” “Well, here's the furthest place on earth.” “Not far enough.” I know, if you're from Ohio, I know you're going to be very upset. You're going to be listening to this and angrily riding your horse to Pennsylvania to send an angry email my way, but that's okay. You'll get there eventually.Serena: But yeah, there's a lot of memes and stuff from TikTok. It's funny because we love to joke; we love to keep it light-hearted; we want to attract people who are younger, a lot of the memes come from TikTok. And so it's a fun, good time. And there's developers on there, there's tons of people that work other jobs that aren't systems engineering, or network engineering. So, we have a bunch of different opportunities and channels for other people to kind of ask questions and connect with other people in the field. Especially with everyone being remote for the most part now, and Covid, you don't have a ton of social interaction, so it's a good place to go get some social interaction.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance query accelerator for the Oracle MySQL Database Service, although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLAP and OLTP—don't ask me to pronounce those acronyms again—workloads directly from your MySQL database and eliminate the time-consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: It's also great because when I was early in my career, I was a traveling consultant, and periodically I would find myself, well, working 40 hours a week and then in a hotel room for the rest of it. That's sort of depressing; I would go to local meetups. I'll never forget going to one Linux user group meeting. In this town, apparently, Linux wasn't really a thing, so the big conversational topic is how to sneak Linux into your Windows job. And I'm sitting around here going, “I don't know if that's necessarily the best way to go about it.”But I checked; there were no reasonable Linux jobs in that community. So, all of their focus in these user groups was about doing it as a side project, as this aspirational thing. And I'm sitting here visiting from out of town, I'm thinking, “Well, I have a job in the Linux environment. And how did I find it? I just went online and looked for jobs that had the word Linux in the title, and there you go.”That option is not open to everyone in every geography, so being able to get exposed to folks who aren't all in your neighborhood is one of the big benefits I found online forums like this.Serena: Yeah. One of the things that I think was positive that came out of Covid is, if you are in a smaller region—one of the reasons I left Ohio was because of a lack of jobs, right. And because there was more opportunity in other areas. And now I wouldn't have had to move. Not that say—I mean, I would have probably moved out of Ohio anyway.But if you don't want to, if your whole family's there now, you're luckily not really stuck with just the jobs that are in your local area. There's tons of remote jobs now. I think that's fantastic, and like I said, one of the positive things that did come out of Covid.Corey: The thing that I don't fully understand is folks who are working for remote companies—we're a distributed companies outside The Duckbill Group, and we pay the same for a role, regardless of where on—or off—the planet you happen to be sitting, just because the value you're adding makes zero difference to me based upon where you happen to be. And there are a number of companies out there who are being very particular about well, where are you geographically because then we need to adjust your comp so you're appropriate for that market. And it's, really? Is the work you're doing this month materially different than the work you're doing next month, as far as value goes, based upon where you're sitting? I don't buy it. But it's also challenging at giant companies to wind up paying the same across the board for all of your staff in one fell swoop.Serena: I think it's particularly bad. I had seen some companies that were basically saying if they're already employed and already getting some salary, and then, like, if you move, we're going to lower your salary. And I was like, it just to me seems so greedy, especially coming from these massive companies that charge huge profits, that you're going to be concerned over a ten, twenty, thirty-thousand dollar difference, right? And it's like, it just seems greedy to me because it's like, well, you had no problem paying that while I was living there, but now it's a problem that I move closer to family or something like that? I luckily was not in that position, but it would have put a distaste in my mouth towards that company, I think, as an employee in that position.Corey: We want to know where people are for tax purposes, we have this whole thing about not committing tax fraud, but aside from that, we don't care where you happen to be. We've had people take a month in Costa Rica, for example. Great. Have fun. Let us know what you think. As long as you have internet there and you make the scheduled meetings you've committed to make, great.But that's part of the benefit of having a company has been distributed since before the pandemic. What I really have sympathy for is folks who had built companies that depended on an in-office culture, and suddenly you're forced into remote during a very stressful time.Serena: Mm-hm. Yeah. Luckily, I mean, most of my jobs are very easily remote, but I can see that. I don't know. The whole—I don't ever want to work in an office again, personally. It's just not for me. I have done really well transitioning to work from home and still keeping up with all my coworkers, and reaching out to them, having meetings.I think, at this point, after two years in, companies are going to have a really hard time justifying to their employees, like, oh, we have to be back in office. And it's like, well, why? Is productivity down? Are we not as profitable? Like, what happened within these last two years that is making you think, like, we need to go back into the office? And they don't really have anything besides, “Culture?” And it's like, yeah, you're going to need to do more than that. [laugh].Corey: It's important for us to see our co-workers from time to time, and once it's safe to do so we're going to be doing quarterly meetups in various places, but that's also… it's not every day.Serena: Right.Corey: The technology problems, I have less sympathy for it now than I did at the start of the pandemic, where network engineers were basically calling the data center and, “Yeah, can you go reboot the VPN concentrator?” “Uh, okay. Which server is that? Probably the one that's glowing white-hot right now.” Because they aren't designed for the entire company to be using it simultaneously all the time. Two years later, we have mostly fixed those problems.Serena: Yeah, yeah. Two years later, it's like, okay, you're going to really have to convince me to go back into the office. [laugh]. And I like the flexibility. Like, I really do. If I want to move, I can move. If I want to, like you said, go to Costa Rica for a month, I could do that. But there's a lot of options, flexibility. I've been having a great time work from home.Corey: And I've been having a lot of fun exploring the bounds of this new Discord group, and I'll throw a link to it in the [show notes 00:24:49] because anyone who wants to show up and can validate that their human being is welcome to join until they turn into a jerk which is basically the [audio break 00:24:57] the community these days, let's be clear, but I found there are a couple of Discord bots—and yeah, it's all the same thing now—that ask test questions, and you can give an answer and it tells you in a DM whether you got it right or not, which is always fun when the bot is broken, and you're sitting there going well, that doesn't make much sense. But what other stuff has been built into this? For those of us who spend all of our time in Slack these days, what is the advantage of the Discord way of doing things?Serena: I guess for me, I'm not, like, a huge Discord person. This is really the only one that I participate in. I'm in a couple of my friends Discord as well, but there's a lot of stickers that are customizable, that relate back to memes a lot of the times. But yeah, the bot that you had mentioned is a great feature that Discord has where @terranovatech, who's also another TikTok content creator—his name's Anthony—he created from Python a practice question bot for CCNA and CCNP. And so, uploaded some questions to those.The bot is in beta guys, so you know, just like, [laugh] be aware of that. We are trying to constantly improve it and add new features. I have been adding a ton of questions for [D core 00:26:05] as I go through my book studying; I'll, you know, create practice questions. And that's typically a part of my normal studying routine, is creating practice questions that I can then go back to after I've read something to solidify it in my mind. And you know, you can use those questions, too, you can suggest questions. If you're like, “Hey, I was doing studying and I think this would be a cool question to add to the Discord bot.” We can do that as well. And so that's great. I love that feature.Corey: One last question before we wind up calling it an episode. Recently, you have caused a bit of TikTok controversy, for lack of a better term. And sure enough, we've had people swing in from all over the planet that chime in and yell at you in the comments. What's going on there?Serena: Okay. Yeah, so that's not unusual for me to cause some TikTok drama in the tech space. Okay, so there's a TikTok trend right now where it's a song and the song lyrics are, “You look so dumb right now.” Okay? And the other videos, like, if you click the sound, you can see, like, some of the videos will say, like, “They told me I needed to rotate my tires, but they rotate every time I drive.”And someone was like, “My girlfriend said she needs new foundation, but our house is just fine.” And so in the background, you hear the song that says, like, “You look so dumb right now.” So, it's just, like, a funny… funny joke. I did it, and I was like, I knew some people were going to miss the joke. And I said, you know, “When they say you need a backup, but you use RAID.” [laugh]. And so the sound is, “You look so dumb right now.”And I was definitely expecting people to miss the joke. And so I even tweeted at the same time, I was like, “I posted a new video, like, about that joke.” And so I was like, “Be prepared for the comments.” Because I knew even someone would be, like, she's just backtracking now. Like, she just is embarrassed. But I was like, “It's the joke guys.”I even put in the caption #thisisajoke. And, like, 90% of people that commented on it just completely missed that joke and were very upset that I made that—that I said that.Corey: Anyone who believes RAID is a backup only has to make one mistake deleting the wrong thing or overwriting something important before they realize that is very much not the case. And if you've been in tech for longer than about 20 minutes, you probably made a mistake like that at one point. It's not one of those things that could reasonably be expected that someone would take seriously. But yet, here we are with entire legions of people with no sense of humor.Serena: Yeah, it ended up in, like, Facebook groups and stuff, too, where these people thought I was being serious. And in the comments, I started making more jokes because someone's like, well, what if your data center catches on fire? And I was like, “Well, don't have a fire at your data center. Like, I don't understand. Obviously.” And so I just tried to, like, you know, make more jokes back to, kind of, keep it up and people were very upset. [laugh].Corey: That's why you're not allowed to smoke in them. Problem solved. Where would the fire come from? Yeah.Serena: There was, like, someone was like, “Well, what if you get ransomware?” And I was like, “We have Norton.” Like, what—[laugh] like, just, like, making the most red—and I was trying to really go outlandish with some of them because they're like, “RAID is not a replacement for cold storage.” And I was like, “Well, we have a lot of fans, so our RAID is very cold.” [laugh]. And, like, just kept it going. Some people were not happy.Corey: I love that. They just keep doubling down on the dumb. The problem is some people are lifelong experts at it, and they're always going to beat you with experience when you try it. It's…Serena: [laugh]. Yeah.Corey: Honestly, the hardest thing to learn, one it was valuable, least from my perspective, is learning when to just ignore the comments and keep going.Serena: Yeah. I definitely get some that I ignore. I mean, if they're, like, overly mean, I'll block somebody or something like that. You know, for someone just missing a joke, it's like, “Okay, whatever.” But yeah, some people—even after they're like, “Hey, man. This is just a joke.” They're like, “Well, this isn't a funny joke.” And I was like, “I will never make a joke about RAID as a backup again. I promise.” [laugh].Corey: No, you already told that joke. There are better ones you can explore.Serena: Yeah. For sure.Corey: So, if people want to come and hang out in this Discord, what's the best way for them to find it? We'll put it in the [show notes 00:30:05], but sometimes people listen rather than read.Serena: Yeah, I think if you even just Google ‘sysengineer Discord' it should come up like that; it's on the Google returned searches. It's a link in my Beacons on my TikTok. It's in a link in sysengineer's TikTok. So, there's a couple different places that you can find and join.Corey: And of course, in the [show notes 00:30:27] for this podcast, as well.Serena: And the [show notes 00:30:30] of this podcast, of course. [laugh].Corey: Thank you so much for taking the time to talk to me about all this. If people want to follow you beyond just the Discord, where's the best place for them to find you?Serena: So, I'm @SheNetworks on TikTok and then I'm @notshenetworks on Twitter. So, you can find me in both of those locations.Corey: Fantastic. Thanks so much for taking the time to speak with me today. I appreciate it.Serena: Thanks for having me on.Corey: Serena DiPenti, network engineer and of course@SheNetworks on the internet. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment telling me which RAID level makes the best backup.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

About NatalieI'm interested in solving human problems through technology (she/her). Share your screen (or I'll share mine) and we'll figure this out!Links: Netlify: https://www.netlify.com/ Twitter: https://twitter.com/codeFreedomRitr TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Sysdig. Sysdig is the solution for securing DevOps. They have a blog post that went up recently about how an insecure AWS Lambda function could be used as a pivot point to get access into your environment. They've also gone deep in-depth with a bunch of other approaches to how DevOps and security are inextricably linked. To learn more, visit sysdig.com and tell them I sent you. That's S-Y-S-D-I-G dot com. My thanks to them for their continued support of this ridiculous nonsense.Corey: It seems like there is a new security breach every day. Are you confident that an old SSH key or a shared admin account isn't going to come back and bite you? If not, check out Teleport. Teleport is the easiest, most secure way to access all of your infrastructure. The open source Teleport Access Plane consolidates everything you need for secure access to your Linux and Windows servers—and I assure you there is no third option there. Kubernetes clusters, databases, and internal applications like AWS Management Console, Yankins, GitLab, Grafana, Jupyter Notebooks, and more. Teleport's unique approach is not only more secure, it also improves developer productivity. To learn more visit: goteleport.com. And no, that is not me telling you to go away, it is: goteleport.com.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. A recurring theme of this show has been where does the next generation of cloud engineer come from because of the road that a lot of us walked is closed, and a lot of the jobs that some of us took no longer exist in any meaningful form. There are a bunch of answers around oh, we're going to get people right out of school from computer science programs into this space, but that doesn't always solve some of the answers. Here to talk to me today is someone who took a different path. Natalie Davis is a software engineer at Netlify, and she entered tech by changing careers from another industry. Natalie, how are you? Thank you for joining me.Natalie: I'm really good, Corey. Thanks for having me. I'm very excited to be here and kind of share my experiences.Corey: So, you have entered tech within the last few years. You went to a boot camp, you spent a year as an engineer at a different company, and now you're at Netlify, one of those companies that, at least for some of us was one of those things you vaguely hear about in the background, sort of a buzz, and the buzz gets louder and louder and louder, and no seems that every time I turn around, I'm tripping over Netlify. In good ways, to be clear.Natalie: I mean, that's definitely good news for me. [laugh]. Yeah, Netlify is a company I first grew familiar with while I was in boot camp. It was the first place I ever hosted a website, a nice little to-do app. And now a couple of years later, here I am, in the guts of it.Corey: So, what were you doing before you decided, “You know what? I'm going to enter tech.” Because if you stand back and you look at it, like that seems like a great culture with no problems whatsoever inherent to it in any way, shape, or form. That's where I want to be. Honestly, I find myself in tech these days, in spite of a lot of things rather than because of it. But again, I am cynical, jaded, again, old and grumpy because you don't get to be a Unix sysadmin without being old and grumpy by somewhere around week three.Natalie: So, that's something I actually find very interesting. Because I came to tech after having existed in another industry—and I'll talk about that in a moment—for about 15 years, I don't find tech as toxic as people who have always been in tech find it. There are problems in tech, but we're talking about those problems; we're trying to come up with solutions. Whereas in retail, where I spent the first 15 years of my career, no one's talking about those problems. And they exist, and they exist on an amplified level because not only are people being treated horribly, not only are people consistently being profiled and discriminated against, but they're doing it for $10 an hour, so there's not even the incentive of at least I get to live well. So, I always push back just a little bit on that, tech is so toxic.Corey: That is a fantastic approach. I hadn't considered it from that perspective. I mean, I sit here in something of an ivory tower. My clients tend to be big companies doing things in a B2B level, whether I'm talking about media sponsorships or consulting projects. The one time a year that I deal with the quote-unquote, “General public,” or a B2C type of thing is my annual charity t-shirt fundraiser.And I have remarked before on this show that those $35 t-shirts cause more customer service headaches for me than the entire rest of the year put together because you sell someone $100,000 consulting project, and you're responsible adults, and you can have conversations and figure out how to move forward, but when someone spends $35 on a shirt—for charity, I will point out—and it doesn't show up, or it's the wrong size or something, they have opinions, and they will in some cases put you on blast. But even in that sense, it's not the quote-unquote, “General public,” it's people in this industry, by and large, who are themselves working professionals, not people walking into a retail store and deciding the best way to get what they want is to basically abuse the staff.Natalie: Yeah, yeah. I noticed that even within retail. I spent most of my retail career in better or luxury retail, but there was one year that I worked in an outlet—and I won't name them—but that was the worst experience of my life. People calling corporate on me over 40 cent discounts. It was just unbelievable. [laugh].Corey: It's a different era, so coming from that, you look at tech and your perspective then is that you see that it has challenges in it, but it's, “Oh, compared to what I used to deal with, this is nothing.”Natalie: Correct. Although I did know that there were challenges in tech, but I viewed it more from a standpoint of how tech was impacting communities like mine. And that was part of what drew me to tech because obviously, there weren't enough people like me in the room, and that meant that there was room for someone like me to enter the room and shake some tables. So, that was part of why I wanted to come to tech.Corey: This is evocative of other conversations I've had, generally with people in the midst of an outage, where everyone's running around with their hair on fire because the computers aren't working, and there's one person sitting there who's just, you would think it is any random Tuesday, and at people ask them, “How on earth are you so calm?” And their answer is, “Oh, I'm a veteran. No one's shooting at me. The computers don't work. I know everyone here is going to go home to their families tonight. This isn't stress. You haven't seen stress.”I have seen shades of that from folks who have transitioned into this industry from, honestly, industries that treat people far worse. So, that's an area I haven't considered. I'd like the direction, I like the angle you have on this. This is sort of a strange follow-up to that, but what inspired you to enter tech from retail? I mean, the easy answer is you look around, you're like, “Okay, I've had enough of this, I'm going to go learn how tech works.” It's never that easy.Natalie: Yeah, it definitely wasn't that easy. So, I married a wonderful man who is a firefighter. My brother-in-law works with non-traditional students at the high school age, his wife is a nurse. So, I'm surrounded by these people who actually have careers, who actually are doing things that they're passionate about. And that wasn't a part of my life before marrying into this family.So, it kind of woke something up in me like, hey, I don't just have to work for a living; I can work for a passion. And no, no one dreams of labor, sure. Like, one day, I'll win the lotto and I won't have to do anything except be a professional student, which would be my ideal path, but it did awaken the possibility that even people in my life can go have these passions. So, then I started thinking, “Well, what can I do aside from retail, without incurring another $100,000 worth of college debt?” And then I started—I jumped on Twitter. Following tech accounts now, and—Corey: Oh, geez, you are a glutton for punishment. It's one of those, “All right. So, I don't think the industry is that bad. I'm going to prove it by going on Twitter.” Okay, let's scrap it on that one.Natalie: But around this time was the time where there was an article about automatic hand dryers and how they weren't recognizing black hands as hands. And I think maybe there was something about an automated self-driving car—that's what I'm looking for—that wasn't recognizing black people as people in the same way that it was recognizing others. And I've always been a fighter. I've always been a rebel. You might not be able to tell it now I seem to have grown up quite a bit, and you know, I'm more conservative with the way I respond to the issues that I see in the world.If I'm going to pursue my passion, it needs to be me fighting for something that's important to me. Tech, okay, cool. Then there's this thing about tech where, sure you can go the CS degree route, and I think that's a great route. I don't think it's the right route for everybody. There's almost like this Wild West aspect where if you can build, that's it. If you can do the job, you can do the job.And I didn't think that it was going to be easy, but I know I've got grit, I know, I've got determination. I know if I set my mind to a thing, I can do a thing. And I liked that you could come in and just be able to do the work, and that would be enough. So, I jumped in a boot camp.Corey: Would you recommend boot camps as a way for people to break into tech? The reason I asked i—I'm not talking about any particular boot camp here—Natalie: Sure.Corey: —but I'm interested in what is the common guidance for folks who find themselves in similar situations and decide that, “You know what? I think that I want to go deal with tech because tech does have its problems, but people aren't literally spitting on you, most days, or throwing drinks at you and, let's be very direct because there's a taboo against talking about this sometimes the pay is a lot better in tech than it is in most other industries.” And we all like to—Natalie: Oh yeah.Corey: —dance around the fact that, “Oh, compensation. No, no, no. You should do it because you love it.” It's, yeah, being able to do what you love is one of those privileges that comes along with having money and making money doing the thing that you love. If the thing that you love is getting screamed at on Black Friday by hordes of people, great. You're still going to not necessarily be able to afford the same trappings of a life that you can by having something that compensates better.Natalie: Thank you for bringing that up because I certainly should have mentioned that the pay was attractive to me in the industry as well. Like, I thought only doctors and lawyers made six figures or better. I didn't realize I could get there.Corey: I've always had the baseline assumption that everyone is in tech to some degree for the money. Whenever I meet someone who's like, “No, I'm in tech and I'm not doing it for the money.” I like to follow up with that because sometimes they're right. “Really? So, what do you do?” Like, “Oh, yeah, I work for this nonprofit doing tech stuff.” “Okay. I believe you when you say that.” When I work for one of the FAANG big tech companies, and people are, “Oh, yeah, I'm here because I love the work.” [pause] “Really? Like, you're out there making the world a better place by improving ad conversion rates? Okay.”Like, we all tell ourselves lies to get through the day, and I'm also not suggesting by any means that money is a bad motivator for anything. The thing that always irked me is when people don't acknowledge, yeah, part of the reason I'm in this industry is because it pays riches beyond the wildest dreams of avarice that I had growing up. I never expected to find myself in a situation where I'm making, as you say, lawyer and doctor money. Honestly, I look around and I'm still astounded that the things that I do on computers—badly, may I point out—is valued by anyone. Yet, here we are.Natalie: I wholeheartedly agree. Every time that direct deposit hits my account, my mind is just blown. Like, “You all know I was just putzing around on my computer all week, right? And like, this is what I get? Cool. Cool.” But to get back to your question is, boot camp—I'm sorry, I don't remember exactly how you phrased it.Corey: No, no, the question I really have is, is boot camp the common case recommendation now for folks who want to break in? Are there better slash alternate paths—if you had to do it all again—that you might have pursued?Natalie: I have to say, people reach out to me for advice: How did you do what you did, they never liked what I have to say because I'm going to start with, you have to understand who you are. You have to understand what works for you. I know that I'm incredibly capable, and I learn quite well, but I need structure in order to do so because if you leave me to my own devices, I will get lost in the weeds of something that does not matter much, but it's quite interesting. And now I've spent a month learning about event handlers, but I don't know how to do anything else. So, for me, boot camp provided both the structure and the baked-in community that I need it because no one in my life is in tech; no one can talk to me about these things. I needed a group of people who I could share the struggle that learning to code is. Because my God, that was a struggle. I've done a lot of hard things in my life, and I don't think many of them had me doubting my abilities the way learning to code did.Corey: There's always that constant ebb and flow of it, where you—it's a rush, like, “I am a genius,” and then something doesn't work it, “Oh, I'm a fool. Why didn't anyone bother to tell me this at any point in my life?” And it's the constant, almost swing between highs and lows on a constant basis. There's a support group for that in tech, it's called everyone, and we made it the bar.Natalie: [laugh]. Yeah, I haven't stopped experiencing that since I've gotten—although I've gotten much better with dealing with the emotions that come along with that.Corey: Yes, sometimes I find going for a walk and calming down helps because if I keep staring at this thing, I'm going to say something unfortunate, possibly on Twitter, and no one wants that.Natalie: Well, I kind of want it. It's fun to watch. [laugh].Corey: Yeah, but it's tied to my name, and that's the challenge.Natalie: Ah, yes, yes. So yeah, I mean, there are people out there who have gone the self-taught route, and oh, my goodness, those people are so inspiring and amazing to me because I don't think I could have pulled it off that way. I think something else you have to think about is the support system you have. I don't know that I would have been able to dedicate myself the way I did in boot camp if I didn't have my husband, who was able to kind of shoulder the financial burden on our family, while I was just living in this office for 14 hours a day. And that's unfortunate, and I think that's something that I hope gets addressed by someone. I don't know who; I don't have the solution.But yeah, it took a certain level of privilege for me to pour myself in the way that I did. So, that's something that you have to think about, what kind of time do you have to dedicate? Now, when you're thinking about that, also understand that it's a marathon, not a race, right? It doesn't matter if Billy did it in a year, if it takes you five years to get there, that's how long it took you to get there. But once you're there, you're there.Corey: There are certain one-way doors that people pass through. Another common one that we see a lot of in the industry is the idea of going from engineer to management. Once you have crossed through that door and become a manager, you can go back to being an engineer and then back to being a manager, but crossing into the management realm the first time is one of those things that is not clearly defined in many places. And every time you talk to somebody like, “How do you break that barrier?” And the answer is, “Oh. I was in the right place at the right time, and I got lucky,” is generally the common answer to it.I keep looking for ways to systematically get there, and that was interesting to me because I wanted to be a manager very much back in the first part of the 2010s. And I put myself in weird roles chasing that, and I think I wanted to do it for the right reasons, namely, to inspire and to be the manager I wished I'd always had. And it turns out I was really bad at it on a variety of different levels. And okay, this is not for me. I decided to go in a bit of a different direction, even now, the entire company rolls up the reporting chain that does not include me. I have a business partner who handles that. No one has to report to me on a weekly basis, which is really something we should put on our careers page as a benefit to help attract people.Natalie: [laugh]. Absolutely. I mean, I'm thinking about that, and like, what does my next five years look like? Do I want to go into management role? I've got a ton of leadership experience in retail.It's not a direct translation, but of course, there are some transferable skills there. But also, it is beautiful to be an individual contributor, to not have to follow up with a team of 12 to see where they're at and what they're working on. So, I still haven't decided where I want to go.Corey: When I have the privilege of talking to high-level executives about the hardest part on their journey, very often the story they say is that—especially if they started off in the engineering world, where, “Yeah, I love what I do, my job is great, but…” and then they pause a minute, and, “Back in the before times, it was easier.” [unintelligible 00:16:13] you're like, “Oh, here. Let me buy you eight drinks.” And then they get really honest. And they say the hard part really is that you don't get to do anything yourself.Your only tool to solve all of these problems is delegation. So, you've got to build and manage and maintain and develop the team, and then you have to give them context and basically let them go and hope that they can deliver the thing that you need when you need it delivered. And for a lot of us who are used to working on the computer of, I push the button and the computer does what I say—you know, aspirationally, after you wind up fixing it eight times in a row, only to figure out that comma should have been a semicolon. Great—and then you're, “Oh, yeah. Okay, that makes sense.”It is hard for folks in an engineering sense to often let go and that leads to things like micromanagement, and the failure mode of a boss who shows up and basically winds up writing code and reverting your commits in the middle of the night and they're treating main as their feature branch. And yeah, we've all seen those weird patterns there. It's a hard, hard thing to do. You've been management in a retail role. Do you aspire to manage people in the tech industry as your career in this zany place evolves?Natalie: I just haven't decided, I think in some ways, it makes a lot of sense. I did enjoy mentoring and coaching and helping people level up. That was kind of my specialty. I got a lot of people promoted, and that felt good to see them kind of take off and fly. But I am kind of in love with the, how do I make this thing do what I want it to do.That digging in and the mystery and the following the trail and console logging 6000 different variables, and then finally, finally, finally, it works, and I don't know if I want to give that up. Honestly, the thing that pushed me into management and retail, initially, was I can make a lot more money in management than I can as a sales associate. And with that incentive kind of removed—and sure I can make more money as a manager, but money ceases to be the same kind of motivator once your needs are met. Like, I'm in a good place, I don't have to worry. So, now I have to think about, do I really want to go back to not being able to do the work—because I found it difficult even in retail not to just jump in and make the sale because I know how to make a sale and I can see where you're going wrong. And I've got to let you fail, but then I've lost the sale.So, I don't know that I want to give up the individual contributor role. But I'm very open. I feel like in this stage of my career, anything is possible. I'm just kind of exploring what's out there and seeing where it leads.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance query accelerator for the Oracle MySQL Database Service, although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLAP and OLTP—don't ask me to pronounce those acronyms again—workloads directly from your MySQL database and eliminate the time-consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: Very often there's this mistaken belief that, “All right, I've been an engineer, so now I need to be a manager to get promoted.” And they're orthogonal skills. Whenever I looked at management roles, and the requirements are well, there's going to be a coding on the whiteboard component to the interview, it's, “What exactly do you think a manager does here?” Or the, “Oh, yeah. You're going to be half managing the team and half participating in the team's work.” It's great. Those are two jobs. Which one would you rather I fail at?Because let's be very realistic here. There's also a bias, it's linked to ageism, for sure in this industry, but you look at someone who's in their 40s or 50s, or 60s or whatever it happens to be, who's an individual contributor, and you look at them, and there's a lot of people that see that either overtly or subtly think that oh, yeah, they got lost somewhere along the way. They have gone in a different direction, they missed some opportunities. And I don't think that's necessarily fair. I think that it fails to acknowledge exactly what you're talking about, that there's a love and a passion behind some of the things you get to deal with and some things you don't have to deal with when you're working as an engineer versus working as management.From my perspective, I'd argue everyone should at least do a stint in management at some point or another just because I have a lot more empathy for those quote-unquote, “Crappy managers” that I had back in the early part of my career, now that I've been on the other side of that table. It's like, I used to be like, “Why would that person fire me?” And now looking at it from that perspective, it's, “Why did that person wait three whole months to fire me?” It's one of those areas where I see it now with the broader context.And it's strange, I've always said I'm a terrible employee, but I would be a much better one now as a result. So, I learned the lesson just in time for it to be completely useless to me, personally, but if I can pass that on to people, that's why I have a microphone.Natalie: Absolutely, yeah. There's a lot of tension, especially when you're kind of middle-level management because you're trying to make your people happy, but then you've got these demands coming from the top, and they don't want what your people want at all. And that's difficult.Corey: That was my failure when I would—I failed to manage up completely. I was obstinate as an employee and got myself fired a lot and figured as a manager, I'm going to do exactly the same thing because it'll work great now.Natalie: [laugh].Corey: Yeah, turns out it doesn't work that way at all for anyone.Natalie: But I think there's something else interesting in that perspective in that I came to tech at what is considered a late age. I joined boot camp, I think maybe… I was 38 when I joined boot camp.Corey: Understand, some people say, “I came to tech late—I was 14 years old—compared to some folks.” And it's like this whole, “Oh, if you weren't in the cradle with a keyboard in your hand, you're too late for this.” And that is some bullshit.Natalie: I laughed so much. I want to see more people like me join late because I can tell you, I haven't had the typical boot camp experience. I've been extremely fortunate in that I have had a community that's really supportive of me, but within a week of telling Twitter I was officially looking for work, I had three interviews with three different companies lined up. And that happened because I had previous experience, both in life and in the industry, so I understood how important it was to build my network and what that looked like, and kind of did that consistently throughout the whole time that I was in boot camp. If I had come at the age of 20, or 14, I wouldn't have had those skills that—kind of—made it relatively—not relatively. That's easy. That was an easy journey. I'm still blown away, and I pinch myself almost every day to think about the fairy tale entry I've had into tech.But again, it happened because I came at an older age because I had those life skills. So please, if you're out there and thinking you're too old, you have to stop listening to people who haven't lived enough life to understand how life works. You have to understand who you are, understand what your skills are, and then understand that tech is thirsty for those skills.Corey: I wish that this were a more common approach. At some level, I feel like there are headwinds against people moving into tech later into their career, gatekeeping, and whatnot. And I used to think that it was this, “Oh because, you know, people just want to hire more folks that look like them.” And I'm increasingly realizing that is actually the more benevolent answer; I suspect, there's at least some element as well, where when someone is new to their career, they're in their early-20s, fresh out of school, they are not nearly as cynical, they are not as good at drawing boundaries. So, they'll work for magic equity at a startup that might one day possibly turn into something, earning significantly below market rate salaries, and they'll be putting in 80 hours a week because they're building something.You only do that once or twice in most people's careers before they realize, wait a minute, that's kind of a scam. Or they'll have an exit and the founder buys a yacht and they get enough to buy a used Toyota. And it's, “Hmm. Seems like that was an awful lot of late nights, weekends, a time away from my family that I could have been spending doing more productive things.” And they work out what it is by the hour that I put in, and it's like fractions of a penny by the time they're all done. And it's, “Yeah, that was ill-advised.”Natalie: Yeah.Corey: There's a cynicism that comes to it, where folks who are further along in their career or come into this industry, from other careers as well, have a lot better understanding of the dynamics of interpersonal relationships in the workplace, as well as understanding that when something smells off, it very well might be off. And early in your career, you just think, “Oh, this is just how it is. This is what workplaces must be. Why didn't anyone ever tell me that?” To me at least, that's why mentorship, especially mentorship from people in other companies at times and career growth is just such a critical thing.Because I used to do the exact same thing till someone took me aside and said, “You know, you just did that thing today at 4:45 and your coworker came up with an emergency it has to be pushed out? Yeah. Watch what happens someone does it to me next.” And he did—great. Because I wasn't able to get to it—“Okay, when did you first find out about this? When does it need to get done? Why didn't you mention this earlier because I'm packing up to go home now? Well, I guess it's not going to get done. I will do it tomorrow instead.”And that's not being a jerk; that's drawing boundaries. And that was transformative to me because I used to think that my job was to just do whatever my boss said, regardless of the rest. Like, call my then fiance, “Oh, sorry. I'm not able to be there for dinner tonight because I've got to do this emergency at work.” That's not an emergency. It's really not.Natalie: Yeah.Corey: Basic stuff like that, but it's the thing you only learned by working in the workforce and having a career for a period of time because it's so different than what the public education system is, coming up through it, where it's basically, comply, obey, et cetera. You aren't really going to have much luck drawing boundaries when you don't do your homework at night.Natalie: Absolutely. I mean, two of the things that you just said that I love is, when you come to it after having lived a bit of life, you absolutely are able to suss out certain things, and kind of sense, “Ooh, that's not good, and I don't want to pursue this any longer.” I've been really fortunate not to experience a ton of things that a lot of people experience, regardless of race, gender, age, there are just some parts of tech that—I don't want to say allegedly; that can be toxic because I don't want to invalidate anyone's experience. But because I've lived so much life, and so much of my career was understanding people, that the moment I started to see those signs, I just kind of separated myself from affiliation with that person, or that group, or that entity, and kind of pursued what I knew would work for me.And then mentorship, and especially mentorship outside of your company. I've got great mentors at my company, but I've got at least three mentors who all work at different places who had just—I wouldn't be here without them. They're my place to go when, hey, is this normal? Because I didn't have any experience in the tech industry. And I'd run everything by them.I don't always do what they tell me to do. Sometimes I get their advice, I listen to it, I think about how it might apply in my life, and then I just tuck it in my back pocket and do what I intended to do in the first place.Corey: One of the things people get wrong about mentorship is that it has to be mentee-led, not mentor-led. And again, it's never expected whenever you're asking someone for advice that you're going to do exactly what they say, but if you're going to go to all the trouble of taking someone's time, you should at least consider what they say. And it may not apply; it may be completely wrong. Every once in a while, we rotate through paid advisors at our company where we have people come in for time to advise us, and sometimes some of those valuable advisors we have, we never did a single thing that they tell us to do, but listening to them and how they articulate and how they clear it out. It's, “Okay, we strongly agree with aspects of this, but here's why it is a complete non-starter for us.”And that is valuable, even though from their perspective, “You never take my advice.” And it's not that, like, “Well, we think your advice is garbage.” No, it's well reasoned, and it's nuanced, but it's not quite right because of the following reasons. That's something that I think gets lost on.Natalie: Yeah, yeah, I would agree with that. And I think you made a really good point. You have to consider the advice if this is someone whom you've come to ask how you might handle a certain situation, and they take the time to give their insight, you have to consider that. If you don't consider it, why are you wasting everyone's time?Corey: One last question I want to get into before we call this an episode. It is abundantly clear that you are a net add to virtually any team that you find yourself on based upon a variety of things that you've evinced during this episode. Why did you choose to work at Netlify? And let's be clear, that is not casting shade at Netlify.Natalie: [laugh].Corey: Like, “You can work anywhere. Why are you at that crap hole?” No, I have a bunch of friends that Netlify and every story I have heard about that company has been positive. So, great. Why are you there?Natalie: For me, it's always going to start with people. I was happy at Foxtrot, my first employer. I was growing there, I was doing well. I liked everyone I worked with. But when Cassidy slides in your DMs and you have a chance to work directly with her and learn from her, you have to explore that opportunity.So, that's what at least led me to having the conversation. And then the way I was treated by everyone through the interview process. No one was trying to trip me up, no one was asking me ridiculous questions. And they were actively fighting to make sure that I came in at a pay rate that made sense, and that I was trusted and given responsibility. And I have to say, once I got there, I found out that I had taken the wrong role.I asked questions about what I was doing. I joined as part of the DX team and my role was to be a template engineer. So, I asked some questions: How much of my role would be coding? Because I knew I couldn't stray too far from the keyboard at this stage of my career. And I got answers, but I didn't know the right questions to ask.When I heard I was—be coding, I thought that meant like how I do now. I work on a product team with a PM and a designer, and they cut issues for me. But what happened in DX is it was much more self-directed, and the work was very different over there. It's incredibly important work. It's valuable work, but it didn't line up with my skill set.So, having that conversation with Cassidy, and then going on to have that conversation with my VP of engineer, a woman named Dana, and having the safety to have those conversations to say, “Hey, I know I just got here. This isn't right for me. I owe more to the DX team and I owe more to myself.” And to be well-received, and to immediately begin to have conversations with engineering managers to find out the right place for me, made me incredibly happy that I chose Netlify, and it kind of reinforced the things they were telling me in the interview process were real.Corey: The fact that you were able to make that transition within the first six months of working at a company and not transition to a different company, either by your choice or not, speaks volumes about how Netlify approaches engineering talent, and its business, and human beings.Natalie: I agree one hundred percent because they could have very easily told me, “Hey, you were hired to do this role. You didn't interview for a product team role, you're welcome to continue to do the work that you were hired to do or move on.” But they didn't do that. No one—in fact, they encouraged me to find the right place for myself.Corey: We talked a minute ago about the one of the values of mentors being able to normalize, is this normal or is this not? Let me just say from what I've seen for almost 20 years in this industry, that is not normal. That is an outlier in one of the most exceptional ways possible, and it is a great story to hear.Natalie: I tell you, I've had an absolutely termed entrance into tech. But also it goes back to, like, when I was in the interview process, I wasn't really focusing on, like, what I would be doing as much as who would I be doing it with and getting a feel for both Cassidy and Jason. And I was one hundred percent confident that at the end of the day, what they wanted was to bring me into the company and for me to do work that fulfills me.Corey: And it sounds like you've got there.Natalie: Absolutely. I'm very happy with the things I'm learning. This codebase is huge. I'm digging in. It's amazing. I couldn't ask for more in life right now.Corey: I want to thank you for being so generous with your time to talk with me today. If people want to learn more, where can they find you?Natalie: I am on Twitter. My username is @codeFreedomRitr, but that's spelled C-O-D-E-F-R-E-E-D-O-M-R-I-T-R.Corey: Excellent. That is some startup to your word spelling there. That is fantastic. You could raise a $20 million seed round on that alone.Natalie: [laugh]. I mean, can I count that as, like, an endorsement? Can I—Corey: Oh, absolutely. Yeah. I have strong opinions on the naming of various things. No, well done. Thank you so much for speaking with me today. I really appreciate it.Natalie: Thank you for having me, Corey. This has been a lovely experience.Corey: Natalie Davis, software engineer at Netlify. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment that you are then going to send to corporate and demand your 40 cents back.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

About NancyNancy Wang is a global product and technical leader at Amazon Web Services, where she leads P&L, product, engineering, and design for its data protection and governance businesses. Prior to Amazon, she led SaaS product development at Rubrik, the fastest-growing enterprise software unicorn and built healthdata.gov for the U.S. Department of Health and Human Services. Passionate about advancing more women into technical roles, Nancy is the founder & CEO of Advancing Women in Tech, a global 501(c)(3) nonprofit with 16,000+ members worldwide.Nancy is an angel investor in data security and compliance companies, and an LP with several seed- and growth-stage funds such as Operator Collective and IVP. She earned a degree in computer science from the University of Pennsylvania.Links: https://coursera.org/awit Advancing Women in Technology: https://www.advancingwomenintech.org LinkedIn: https://www.linkedin.com/in/wangnancy/ Advancing Women in Technology LinkedIn: https://www.linkedin.com/company/advancingwomenintech/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Sysdig. Sysdig is the solution for securing DevOps. They have a blog post that went up recently about how an insecure AWS Lambda function could be used as a pivot point to get access into your environment. They've also gone deep in-depth with a bunch of other approaches to how DevOps and security are inextricably linked. To learn more, visit sysdig.com and tell them I sent you. That's S-Y-S-D-I-G dot com. My thanks to them for their continued support of this ridiculous nonsense.Corey: This episode is sponsored in part by our friends at Rising Cloud, which I hadn't heard of before, but they're doing something vaguely interesting here. They are using AI, which is usually where my eyes glaze over and I lose attention, but they're using it to help developers be more efficient by reducing repetitive tasks. So, the idea being that you can run stateless things without having to worry about scaling, placement, et cetera, and the rest. They claim significant cost savings, and they're able to wind up taking what you're running as it is, in AWS, with no changes, and run it inside of their data centers that span multiple regions. I'm somewhat skeptical, but their customers seem to really like them, so that's one of those areas where I really have a hard time being too snarky about it because when you solve a customer's problem, and they get out there in public and say, “We're solving a problem,” it's very hard to snark about that. Multus Medical, Construx.ai, and Stax have seen significant results by using them, and it's worth exploring. So, if you're looking for a smarter, faster, cheaper alternative to EC2, Lambda, or batch, consider checking them out. Visit risingcloud.com/benefits. That's risingcloud.com/benefits, and be sure to tell them that I said you because watching people wince when you mention my name is one of the guilty pleasures of listening to this podcast.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I've said repeatedly on this show—and I stand by it—that absolutely nobody cares about backups. Because they don't. They do care tremendously about restores, usually right after they really should have been caring about backups.My guest today has more informed opinions on these things than I do, just because I'm bad at computers. But Nancy Wang is someone else entirely. She is AWS's general manager of the AWS Backup service, and heads the Data Protection Team. Nancy, thank you for tolerating me, I appreciate it.Nancy: Hey, no worries because you know, when I heard you say I don't care about backups, I knew I had to come on the show and correct you. [laugh].Corey: It's the sort of thing where there's no one is fanatical as a convert. And every grumpy old sysadmin that is in my cohort either cares a lot about backups or just doesn't even think about it at all. And the question is—the only thing that separates those two groups is have you lost data yet? And once you've lost data and you feel like a heel, you realize, “Wow, this was eminently preventable. What can I do differently to fix this?”And that's when people start preaching the virtues of backups, and you know, this novel ridiculous idea of testing the backups you've made to make sure that it isn't just—yeah, it says it's completing correctly, but if you haven't restored it, you don't really know.Nancy: Yeah. I mean, that's so true, right? And that's why when we're thinking about our holistic data protection strategy, it's less so about, “Hey, make sure that you take backups”—which is albeit a very important part of the data protection hygiene—but is making sure that you can regularly test the things that you're backing up to make sure that, frankly, when you happen to be in a disaster scenario, or someone fat fingers a restore process, that you have good known bits to restore from.Corey: So, people will be forgiven for not, potentially, understanding what AWS Backup is, where it starts and where it stops. I mean, let's be clear, this is sort of the price you as a company get to pay for having 300-some-odd services; not everyone is conversant with every single one of them. I know, I'm as offended as anyone at that fact, but apparently other people have lives. So, what is AWS Backup?Nancy: So, on that note, Corey, I do have to say that I'm probably at a more of an advantage in terms of my name being very descriptive and what it does versus, maybe, Athena or Redshift where it's very clear, hey, we do backups. But actually, if you parse apart the product—and this is why the team itself is called data protection—there are various axes to think about what we do, right? So, to help illustrate, perhaps if you think about axes one as in, what are the different types of application data that we protect, right? There's obviously database data, there's going to be file system data, there's various storage platform data, right? And those are comprised by AWS services that I'm sure you all are very familiar with, love dearly, like RDS, EBS, with EC2, VMs, et cetera, but also, more recently, we added S3, which we'll get to that in just a bit, but because I'd love to talk about, you know, how folks think about S3 and why you might want to back it up, right? So, that's axis number one.Now, if we turn to axis number two, it's about the different platforms where these application data might reside. So there's, of course, in-cloud, and that's the place where most people are familiar with and why they might choose to seek out a first party native data protection provider like AWS Backup. And by the way, we just extended our support to on-premises as well, starting with VMware, which is a thing that a lot of backup admins were super excited to hear about, and all those vExperts out there.And of course, the final axis is we think about how we make sure that we not just protect your data, but we are also able to give you tools like compliance reporting, which we announced in August at re:Inforce, via our CISO, Stephen Schmidt, about, “Hey, once you take your backups, are you monitoring continuously the resource configurations of the application data that you're protecting?” Are your backup plans architected to meet RPO requirements that your organization needs to meet? Are they being, for example, retained for the right amount of times? Is it seven years or is it a month? Many different organizations have widely varying RPO requirements, so making sure that all of that is captured, monitored, and also reportable so when, hey, those, that auditor decides to knock on your door, you have a report ready to say, “Hey, I'm in compliance. And by the way, I'm proactively thinking about how my organization can meet evolving regulations.”Corey: Please tell me you're familiar with AWS Audit Manager, which is, to my understanding, aimed at solving exactly this problem. If the answer is no, this would admittedly not be the first time there I found, “Oh, wow. We have a complete service duplicate hanging out somewhere at AWS.” “Oh, good. How do we make it run in containers?” Being the next obvious question there.Nancy: Sure. Which is actually a great lead-in to, again, another descriptive name of an AWS service, which is AWS Backup Audit Manager. So, if you recall from the re:Inforce keynote, it was one of the slides that was highlighted. The reason being, I'm a firm believer of a managed solution. Because look, we all know that AWS is great at building, I would say, tools or building blocks, or primitives to design end-to-end solutions.Corey: It's the Lego approach to cloud services. “What can I build with this?” “You're only constrained by your imagination.” “Okay, but what can I build?” “Here to talk about that is someone from Netflix.”Great. I want to build Twitter for Pets, which I guess now has to stream video? Yeah, it becomes a very different story. The higher-level service offerings are generally not a common area that AWS has excelled in, but this seems to be a notable exception.Nancy: That's actually where my background is, right? So, previous to AWS, I worked at a not-so-small startup anymore, called Rubrik, down in Silicon Valley, where we spent a lot of time thinking about what is the end-to-end solution for customers. How can customers simply deploy with one click, make sure that they can create policies that are repeatable, that are automated, and go off when you want them to, and make sure that you have reporting, at the end of the day. So, that's really what we focused on, right?But I digress, Corey. To your question about AWS Audit Manager, the name of the service within AWS Backup that handles compliance reporting, and auditing is called AWS Audit Manager, and we certainly didn't pick that name by fluke. The reason being, we wanted AWS Backup, from that managed solution point of view, to be the single central platform where customers come to create data protection policies, where they come to execute those data protection policies, in backup plans, store their backups in encrypted backup vaults, and have the ability to restore them when they want, and finally, report on them. So, it is that single platform.Now, with that said, if, for example, you wanted that reporting to come from AWS Audit Manager, which is a service that does a lot of reporting across many AWS services, you also have that ability. So, depending on what user persona you might be, whether you're from the central compliance office or you're a member of the data protection team within an organization, you might choose to use that functionality separately. And that's the flexibility that my team strived to provide.Corey: One of the most interesting things about AWS Backup is that I did not affirmatively go out of my way to use your service. I did not—to my recollection—wind up saying, “Oh, time to learn about this new thing, and set it up, and be very diligent about it.” But sure enough, I find it showing up on the AWS inventory—which is of course, the bill. And I look at this in a random account I use for various, you know, shitposting extravaganzas, and sure enough, it's last—so far, this month, it is—I'm recording this near the end of the month—it charged me $3.40 to backup 70 gigs of data.Which is first, like on the one hand, there is an argument of, “Now, wait a minute. I didn't opt into this. What gives?” The other side of it though, is how dare you make sure that my data isn't going to be lost, not through your negligence, but through my own, when I get sloppy with an rm -rf. And because I've been using ZFS a fair bit, and it is integrated extraordinarily tightly with that service. It goes super well.It works out when setting this up, unless you go out of your way to disable it, it will set up a backup plan. And first, that is not generally aligned with how AWS thinks about things, which you across the board, generally the philosophy I've gotten is, “Oh, you want to do this thing? That's a different service team. Do it yourself.” But also, it's one of those areas that is the least controversial. If you have to make a decision one way or another, yeah, it's opt people into backups. Was that as hard to get approved as I would suspect it would be, or was that sort of a no-brainer?Nancy: Hopefully you can let me know what your account number is, Corey, so I can make sure it doesn't get marked for fraud—A—but B, going into, you know, our philosophy on protecting data: So, EFS actually was one of our first AWS services that was supported by the AWS Backup service, which is actually quite a fascinating story in itself because the service [AWS Backup] only launched in 2019. Now, AWS has been around for much, much longer than that—Corey: And it feels even three times longer than that. But yes.Nancy: [laugh]. Exactly, right. So, as a central data protection platform for the AWS overall cloud platform, it's quite interesting that from a managed solution perspective, the service is not yet, you know, four years old. We're barely embarking on our third year together. So, with that said, why we started with EFS and a few other services is we wanted to cover the most commonly used stateful data stores for AWS Cloud, EFS being one of them, as the first cloud-native—as Wayne Duso would say—Elastic File System in the cloud.And so what we did is a deeper level integration, what we call our “data plane integration.” So, what does that mean? Customers protecting EFS file systems have the ability to not just restore their entire file system as a file system volume, but also have the ability to specify individual files, folders, that they want to restore from. And so, file level recovery, super, super important. And it's something that we also want to bring for other file systems down the road as well.And so, to your question, Corey, a common design principle that we think about is, how do we make sure that customers are protected? Obviously, in a world where we cannot yet use AI to transcribe every part of a customer's intent when they're looking to protect their data, the closest that we can get is, “Hey, you create a file system. We assume that you want it protected, unless you tell us you don't want to.” And so for certain resources, like EFS, where we have a deeper level integration to our own data plane, we can then say, “Once you create a file system will opt you automatically into AWS Backup protection until you tell us to stop.” And from there, you have all the goodness that comes with AWS Backup, such as file-level restore, such as for example now, WORM [write-once-read-many] lock, which disables the ability to mutate backups from anyone, even someone with admin access.Corey: So, a big announcement in your area at re:Invent, was AWS Backup support for S3. Allow me to set up an intentionally insulting straw man argument here. S3 has vaunted 11 nines of durability, which I think exceeds the likelihood the gravity is going to continue to function. So, are they lying by having AWS Backups supporting it now, or are you just basically selling us something we don't need? Which is it?Nancy: Well, you know, Corey, judging by the hundreds of customers who have been filling up my inbox—and that's why I actually ended up creating a special email alias for the S3 preview—so what we launched at re:Invent was a public preview of the ability to start baking in S3 backup protection—or bucket protection—into their existing data protection workflows, right? And so judging by the hundreds of customers, many of them in highly regulated industries, and FinServ, in healthcare, as well as in the US government, I would say that I think they find it pretty important, and we're not just peddling things they don't need. So, I'm getting ahead of myself. We're actually—we should probably start the conversation—is a deeper dive into how we think about data protection on AWS.And so there's two really core schools of thought, right? One is, you know, focused on data durability, which in itself is a function of technology. So, to your point of 11 nines, right? That is very much true, and that's why S3 increasingly becomes the platform of choice, now, for all of customer's, you know, analytics information, and other stateful stores that they want to keep an S3 buckets for applications, right? But second of all—and this is a part where AWS Backup wants to focus on—is that concept of data resiliency, which itself is a function of external factors. Because, for example, human errors, such as fat-fingering, or miscellaneous entries, could impact for example, how you can access information that's stored in your S3 bucket, or unfortunately, sometimes what we've heard is accidentally deleting an S3 bucket or certain objects in your S3 bucket.Corey: This speaks to the idea of that RAID is not a backup. Sure, you want to make sure a drive failure doesn't lose your data, but you also want to make sure that you overwriting a file that was super important doesn't happen either and RAID, nor data durability and S3, are going to save you from that.Nancy: Yeah. Because for example, we have built in—and this is actually very core to not just AWS Backup, but really how we think about data protection on AWS—is again, that separation of control. So, I encourage you to try to delete, let's say, an EBS volume that is protected by AWS Backup, from the EBS console. You'll likely find a very glaring error in your face that says, “You do not have sufficient privileges to do so.” And the reason we actually make such a separation of control, or our role-based access control—RBAC—so core to our product design is so that, for example, whoever creates that primary volume should not be the same person that deletes it, unless they do happen to be the same person with two different roles.And that prevents, for example, unintended mutations. That also enables the data protection administrator to have the ability to, let's say, do cross-region copies: Having your S3 bucket or objects stored in another region, in another account, that can be completely locked down to anyone, even those with administrator access, right? So, like I said, before, all the platform goodness, AWS Backup, such as version control, WORM locks, having multiple copies of those backups, as well as different protection domains, that's what customers look for when they come to this service.And to your point, especially even with highly durable platforms like S3, there's still external factors that you simply can't control for all the time, right? And having that peace of mind, having that protection that you know is on 24/7, hey, that keeps businesses up, right? And that keeps consumers like you and me able to enjoy all the goodness that those businesses offer.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance query accelerator for the Oracle MySQL Database Service, although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLAP and OLTP—don't ask me to pronounce those acronyms again—workloads directly from your MySQL database and eliminate the time-consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: I agree wholeheartedly with everything that you're saying. I had a consulting client where it's coming in optimize the AWS bill, and, “Wow, that sure is a lot of petabytes over in that S3 infrequent access bucket. How about you change the Infrequent Access-One Zone?” “Oh, no, no, no. We lose this data, it basically ends a division of the company.” “Cool. Do you have multi-factor delete turned on?” “No.” “Do you have versioning turned on?” “No.” “Okay. This is why I call it cost optimization, not cost cutting. You should be backing that up somewhere because there is far likelier—by several orders of magnitude—that you or someone on your team intentionally—unlikely—or by accident—very likely, as someone who's extremely accident prone with computers, from my own perspective because I am—is going to accidentally cause data loss there. So yeah, spend more money and back that up.”And they started doing that. So, it's always nice when your recommendations get accepted. But yeah, if data is that important, you absolutely need to have a strategy around that. What I love so far about what I've seen from AWS Backup is—and please don't take this in any way as criticism on it—is that it's so brainless. It just works. Because people don't think about backups until it's too late to have thought about backups.Nancy: Yeah, don't worry, I don't take that as offense, Corey, otherwise I wouldn't be on the show. Absolutely, right? My motto is set it and forget it, right? Just as I want to make it super simple for our mission, for customers to understand our mission, as well as, frankly, the engineers who build the service to understand our mission, it is, “We protect our customers' data on AWS. How? With set-it-and-forget-it data protection policies.”And we try to configure these policies to be fairly comprehensive. You can set everything from, like I mentioned, warm lock, where you want your backup copies created to: Which regions? Which accounts, for example? Which user role do you want to use with these data protection policies? Which services do you want to protect?And even recently, we created the selection ability—or as we call it, AWS Backup Select—so you can include, exclude different resources, even when you have the common union of tags specified on your backup plan. So, the reason we went this comprehensive is so that once you configure a data protection policy, you can really rest assured that, hey, I've done everything in my power to make sure that these resources, this application data that is so critical to my business, is being protected. And oh, by the way, I can see these backups—or as we call in our lexicon, Recovery Points—directly in my console, in my account.Corey: And there's tremendous value to doing that. That is the sort of thing that customers like to see. This is—if you have to move up the stack somewhere, this feels like the place to begin doing it, just because it's so critical to the rest of it. We all have side projects as well. Like, for example, I wind up making insulting parody music videos for people's birthdays when they're not expecting it. You have 80 hours of training content on Coursera. What is that about? Because I don't think it's all about backups.Nancy: No. Although at some point, we should probably get AWS Backup as one of the modules in AWS certification. But I digress. The reason why training is so important to me is one of the ways, actually, that folks find me online is through my presence in the nonprofit world. So, I'm the founder and CEO of a 501(c)(3) organization that's called Advancing Women in Technology, or AWIT, or A-W-I-T for short.The mission of AWIT is really to get more women leaders into visible, into senior tech leadership roles, so frankly and from a selfish perspective, I'm not the only woman in a room many of the times when decisions are being made, right? And that's not just, you know, I'm talking about my current role, but in various roles that I've had throughout the tech industry. So, where does that start? And there's a lot of different amazing organizations that focus on the early career, beginning in the pipeline, which is super important because it is important to get women, underrepresented groups in the door so that they can advance and they can accelerate their careers to becoming leaders, but the areas where AWIT focus is actually in that mid-career.Because once folks, and especially women and underrepresented groups are in the door 10 to 15 years, they're maybe in their first managerial role, or they're in their first leadership role, that's the core time when you want to retain that population, where you want to advance that population, so that in the next, I would say, generation—or hopefully it doesn't even take that long; next 5, 10 years—we see a much more representative leadership room, or board table, right? So, that's really where that goal starts. And so, why do we have 80 hours of training content because part of advancing your career and accelerating your career is having the right skills. Of course having a right network is also very important, and that's something else that we preach, but upskilling yourself, constantly learning about new technologies—I mean, the tech world changes by the minute, right, and so being familiar with new technologies, new frameworks, new ways of thinking about product problems, is really what we focus on. So, we were the first to create the Real-World Product Management Specialization, which you can check out on Coursera. You'll see my mug shot in a lot of those videos.But actually, also of those of some of the best and brightest underrepresented leaders in the industry, such as Sandy Carter, Mai-Lan Tomsen Bukovec, Sabrina Farmer, I mean, the list goes on and on. Including, you know, personal friend who created Coffee Meets Bagel. So hey, for all those connections made out there on that platform, you know, she's also a woman CEO, and used to be a product manager at Amazon.Corey: A dear friend met his partner on Coffee Meets Bagel. I hear good things.Nancy: Oh, awesome.Corey: Fortunately, I was married before it launched, so I've never used the service myself. If I were a reference customer now, that would raise questions.Nancy: [laugh]. Well, let's just say I'm not on the platform, either, so I can't verify or deny that you have a profile. Yeah. So, just having those underrepresented groups and individuals, really stellar rock stars, role models that we would all consider to be super inspirational, as speakers, as instructors on the courses have given so many folks the inspiration, the encouragement that they need to upskill themselves. And so yes, now educated over 20,000 learners worldwide using those courses.And I still receive just amazing notes from them on a daily basis, all over LinkedIn about how they've managed to get promotions from taking these courses, or how they've managed to get jobs in FAANG tech companies as a result of taking these courses. And really, that's the impact that I want to make is one to n, being able to impact a global audience, upskilling a global audience. And so again, in the future, and not so distant future, the leadership room gets so much more representative.Corey: And to complete the trifecta of interesting things you do, you are also an early angel investor and a limited partner in a number of startups. Tell me a little bit about that. It's odd to—at least in my experience—to see folks who are heavily involved in the nonprofit space, the corporate space at a giant tech company, and doing investment all at the same time. It seems like that is not a particularly common combination, at least in the circles in which I travel.Nancy: You could also probably blame it on my extreme ADHD. That's probably very true. Don't worry, I try to control it, most of the time.Corey: I've been struggling to control my own my entire life, which probably explains a lot about why I do the things that I do. I hear you.Nancy: It makes sense, right? From one to another. It honestly makes me better at my job. And I'll explain why. So, if you look at some of the new or joint marketing campaigns that AWS Backup or data protection team has done this past year with various startups—namely Open Raven; there'll be others we're working with in the new year—being able to just get some of that inspiration from founders, so thinking about how can we have a better together story?You specialize in, let's say with the case of Open Raven, in data visibility and let's say scanning S3 buckets for vulnerabilities, for different content. And hey, we specialize in data recovery process, or then that data protection policy creation process. How do we come together to form a really awesome solution for our highly regulated customers, or compliance-minded customers? That's the story that I love to tell, and frankly, I just get so inspired from talking to startup founders. The reason why I have also advised a few venture capitalists—namely Felicis Ventures—on, for example, their investment thesis is I just see so much potential in this environment, right?And there's really that adage, where it's big enough sandbox for a lot of players. Just like, for example, how Snowflake and Redshift have managed to coexist together on the AWS platform, there's a lot of just goodness, too, that exists between the data security world, how they customers think about securing their data, to the data protection world because, hey, you can't protect what you can't see, so you need to be make sure that you have that data visibility angle, along with that protection angle, along with that recovery angle. And hey, all of this needs to be within your data perimeter, within a secure zone, right? How do you securitize your data? So, all of that really comes together in this melding world.And of course, there's also adjacent themes such as, well, once you protect your data, how can you also make sure that the quality of your data is high? And that's where pretty interesting startups in the data observability space, such as Monte Carlo, have come up. Which is, “Hey, I need to rely on my business data to make important decisions that affect my customers, so how can I make sure that what's ever coming out of my data lake or data warehouse is correct, it truly reflects the state of the business?” So, all of that is converging, and that's why, you know, it's just super exciting to be a part of this space, to not only create net new, I would say greenfield opportunities on the AWS platform, but also use this as an opportunity to partner with startup CEOs and various startups in the data space, data infrastructure space, to create more use cases, more solutions for customers who otherwise we'd have to rely on either custom scripts, or simply not having any solutions in this space at all.Corey: There's something to be said for doing the—how do I frame this?—the boring work that's always behind the scenes, that is never top of mind. People don't get excited about things like data protection, about compliance, about cost optimization, about making sure that the fire insurance is paid up on the building before you wind up insulting execs at big companies, et cetera, et cetera. And that—but it is incredibly important—in my case, especially that last one—just because if you don't get that done, there's massive risk, and managing that risk is important. It's nice to see that it's not just the shiny features that are getting the attention. It's the stuff of, “Okay, how do we do this safely and securely?” That is the area that I think is not being particularly well served these days, so it's honestly refreshing to see someone focusing on that as an area of active investment.Nancy: I mean, absolutely. Perhaps one data point I should also share, because I do get questions asked of, “What gets you so excited about compliance, about audit?” Well, I used to work for the US government. So, if that tells you anything—and I used to hold an active secret clearance—that hopefully explains some things about why I'm passionate about the areas I am. But, that's really where, you know, back to your comment that you made on the core tenet or the ethos of the AWS Backup service, which is, “Set it, forget it, make it super simple,” is I want to design systems or solutions that enable customers to focus on developing applications, working on building business logic, whereas we will create the comprehensive data protection policies that protect your data.And especially in the world of ever evolving cyber attacks where the attackers are getting more and more sophisticated, they have more backdoor methods that go undetected for many months, as was the case in attacks over the past recent years, or in the case of pesky ransomware attacks, where certain insurance companies have even stopped paying ransoms, right, and you're wondering, “Well, how do I get my data back?” This is the world that we live in. And so, you know, yes, there might be ever-evolving more, I would say, sophisticated ways to detect vulnerabilities, or attacks, or do pattern matching between known attack patterns, but really what remains core and should be core to a lot of companies' recovery strategies, as per the NIST cybersecurity framework, is actually having a good way to restore. And that goes back to something that you mentioned at the beginning of this recording, Corey, which is making sure that you're regularly testing your backups because as you said, no one cares that you're taking backups, but people do care about the ability to restore. So, having known good bits that exist in a secure vault, that exists maybe in some air gap account or region, where you know that it's going to be there for you, that it's restorable is going to be super key.And we're already seeing that trend in a lot of customers that I speak with. And by the way, these aren't just customers in highly regulated industries. They're really customers that now are increasingly relying on data to make business decisions. Just like, for example, there's that adage that says, you know, “Software is eating the world,” well, now most businesses are data-driven businesses, and so data is core to their business mission. And so protecting that, it should also be core to their business mission.Corey: I really wish that were the case a bit more than it is.Nancy: True that. So, I would have to say, “Hear, hear.” And this is actually what makes my job so, just, fun frankly, is that I get to have these conversations with thought leaders at various different companies, who are my clients or customers of AWS. And these are different, I would say, leaders, ranging from IT leaders, to compliance leaders, to CISOs who I have these conversations with. And oftentimes it does start with this very, I would say, innocuous question, which is, “Well, why should I think about protecting my data?” And then we're able to go into, “Well, this is how you think about tiering your data, this is how you think about different SLAs that you might have for your data, and then finally, this is how you would think about architecting a data protection solution into your environment.”Corey: Nancy, I want to thank you for taking some time out of your day to speak with me. If people want to learn more about what you're up to and how you're viewing these things, where can they find you?Nancy: Feel free to connect with me on LinkedIn, whether you have a service that you desperately want AWS Backup to protect—yes, I get a lot of those tweets or LinkedIn posts—absolutely happy to consider them and to prioritize them on the future roadmap. Or if you want to give me a feedback about your experience, more than happy to take those as well. Also, if you're a startup founder and you have a brilliant new idea, and data infrastructure, always happy to grab coffee or drinks and hear about those ideas.And lastly, if you're looking to upskill yourself either product management or cloud tech skills, find us on Coursera at https://www.coursera.org/awit, or on LinkedIn as Advancing Women in Technology. Either way, whether you fit into one or more or all of these buckets, I'd love to hear from you.Corey: And we will, of course, put links to that in the [show notes 00:32:36]. Thank you so much for speaking with me today. I really appreciate it.Nancy: Well, thank you, Corey. It's always a pleasure, and I'll see you very soon in person in SF.Corey: I look forward to it. Nancy Wang, General Manager of AWS Backup and AWS Data Protection. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an insulting comment that I will then delete because it wasn't backed up.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

About AndrewI create free cloud certification courses and somehow still make money.Links: ExamPro Training, Inc.: https://www.exampro.co/ PolyWork: https://www.polywork.com/andrewbrown LinkedIn: https://www.linkedin.com/in/andrew-wc-brown Twitter: https://twitter.com/andrewbrown TranscriptAndrew: Hello, and welcome to Screaming in the Cloud with your host, Chief cloud economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Redis, the company behind the incredibly popular open source database that is not the bind DNS server. If you're tired of managing open source Redis on your own, or you're using one of the vanilla cloud caching services, these folks have you covered with the go to manage Redis service for global caching and primary database capabilities; Redis Enterprise. To learn more and deploy not only a cache but a single operational data platform for one Redis experience, visit redis.com/hero. Thats r-e-d-i-s.com/hero. And my thanks to my friends at Redis for sponsoring my ridiculous non-sense.  Corey: This episode is sponsored in part by our friends at Rising Cloud, which I hadn't heard of before, but they're doing something vaguely interesting here. They are using AI, which is usually where my eyes glaze over and I lose attention, but they're using it to help developers be more efficient by reducing repetitive tasks. So, the idea being that you can run stateless things without having to worry about scaling, placement, et cetera, and the rest. They claim significant cost savings, and they're able to wind up taking what you're running as it is in AWS with no changes, and run it inside of their data centers that span multiple regions. I'm somewhat skeptical, but their customers seem to really like them, so that's one of those areas where I really have a hard time being too snarky about it because when you solve a customer's problem and they get out there in public and say, “We're solving a problem,” it's very hard to snark about that. Multus Medical, Construx.ai and Stax have seen significant results by using them. And it's worth exploring. So, if you're looking for a smarter, faster, cheaper alternative to EC2, Lambda, or batch, consider checking them out. Visit risingcloud.com/benefits. That's risingcloud.com/benefits, and be sure to tell them that I said you because watching people wince when you mention my name is one of the guilty pleasures of listening to this podcast.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. My guest today is… well, he's challenging to describe. He's the co-founder and cloud instructor at ExamPro Training, Inc. but everyone knows him better as Andrew Brown because he does so many different things in the AWS ecosystem that it's sometimes challenging—at least for me—to wind up keeping track of them all. Andrew, thanks for joining.Andrew: Hey, thanks for having me on the show, Corey.Corey: How do I even begin describing you? You're an AWS Community Hero and have been for almost two years, I believe; you've done a whole bunch of work as far as training videos; you're, I think, responsible for #100daysofcloud; you recently started showing up on my TikTok feed because I'm pretending that I am 20 years younger than I am and hanging out on TikTok with the kids, and now I feel extremely old. And obviously, you're popping up an awful lot of places.Andrew: Oh, yeah. A few other places like PolyWork, which is an alternative to LinkedIn, so that's a space that I'm starting to build up on there as well. Active in Discord, Slack channels. I'm just kind of everywhere. There's some kind of internet obsession here. My wife gets really mad and says, “Hey, maybe tone down the social media.” But I really enjoy it. So.Corey: You're one of those folks where I have this challenge of I wind up having a bunch of different AWS community Slacks and cloud community, Slacks and Discords and the past, and we DM on Twitter sometimes. And I'm constantly trying to figure out where was that conversational thread that I had with you? And tracking it down is an increasingly large search problem. I really wish that—forget the unified messaging platform. I want a unified search platform for all the different messaging channels that I'm using to talk to people.Andrew: Yeah, it's very hard to keep up with all the channels for myself there. But somehow I do seem to manage it, but just with a bit less sleep than most others.Corey: Oh, yeah. It's like trying to figure out, like, “All right, he said something really useful. What was that? Was that a Twitter DM? Was it on that Slack channel? Was it that Discord? No, it was on that brick that he threw through my window with a note tied to it. There we go.”That's always the baseline stuff of figuring out where things are. So, as I mentioned in the beginning, you are the co-founder and cloud instructor at ExamPro, which is interesting because unlike most of the community stuff that you do and are known for, you don't generally talk about that an awful lot. What's the deal there?Andrew: Yeah, I think a lot of people give me a hard time because they say, Andrew, you should really be promoting yourself more and trying to make more sales, but that's not why I'm out here doing what I'm doing. Of course, I do have a for-profit business called ExamPro, where we create cloud certification study courses for things like AWS, Azure, GCP, Terraform, Kubernetes, but you know, that money just goes to fuel what I really want to do, is just to do community activities to help people change their lives. And I just decided to do that via cloud because that's my domain expertise. At least that's what I say because I've learned up on in the last four or five years. I'm hoping that there's some kind of impact I can make doing that.Corey: I take a somewhat similar approach. I mean, at The Duckbill Group, we fixed the horrifying AWS bill, but I've always found that's not generally a problem that people tend to advertise having. On Twitter, like, “Oh, man, my AWS bill is killing me this month. I've got to do something about it,” and you check where they work, and it's like a Fortune 50. It's, yeah, that moves markets and no one talks about that.So, my approach was always, be out there, be present in the community, talk about this stuff, and the people who genuinely have billing problems will eventually find their way to me. That was always my approach because turning everything I do into a sales pitch doesn't work. It just erodes confidence, it reminds people of the used mattress salesman, and I just don't want to be that person in that community. My approach has always been if I can help someone with a 15-minute call or whatnot, yeah, let's jump on a phone call. I'm not interested in nickel-and-diming folks.Andrew: Yeah. I think that if you're out there doing a lot of hard work, and a lot of it, it becomes undeniable the value you're putting out there, and then people just will want to give you money, right? And for me, I just feel really bad about taking anybody's money, and so even when there's some kind of benefit—like my courses, I could charge for access for them, but I always feel I have to give something in terms of taking somebody's money, but I would never ask anyone to give me their money. So, it's bizarre. [laugh] so.Corey: I had a whole bunch of people a year or so after I started asking, like, “I really find your content helpful. Can I buy you a cup of coffee or something?” And it's, I don't know how to charge people a dollar figure that doesn't have a comma in it because it's easy for me to ask a company for money; that is the currency of effort, work, et cetera, that companies are accustomed to. People view money very differently, and if I ask you personally for money versus your company for money, it's a very different flow. So, my solution to it was to build the annual charity t-shirt drive, where it's, great, spend 35 bucks or whatever on a snarky t-shirt once a year for ten days and all proceeds go to benefit a nonprofit that is, sort of, assuaged that.But one of my business philosophies has always been, “Work for free before you work for cheap.” And dealing with individuals and whatnot, I do not charge them for things. It's, “Oh, can you—I need some advice in my career. Can I pay you to give me some advice?” “No, but you can jump on a Zoom call with me.” Please, the reason I exist at all is because people who didn't have any reason to did me favors, once upon a time, and I feel obligated to pay that forward.Andrew: And I appreciate, you know, there are people out there that you know, do need to charge for their time. Like—Corey: Oh. Oh, yes.Andrew: —I won't judge anybody that wants to. But you know, for me, it's just I can't do it because of the way I was raised. Like, my grandfather was very involved in the community. Like, he was recognized by the city for all of his volunteer work, and doing volunteer work was, like, mandatory for me as a kid. Like, every weekend, and so for me, it's just like, I can't imagine trying to take people's money.Which is not a great thing, but it turns out that the community is very supportive, and they will come beat you down with a stick, to give you money to make sure you keep doing what you're doing. But you know, I could be making lots of money, but it's just not my priority, so I've avoided any kind of funding so like, you know, I don't become a money-driven company, and I will see how long that lasts, but hopefully, a lot longer.Corey: I wish you well. And again, you're right; no shade to anyone who winds up charging for their time to individuals. I get it. I just always had challenges with it, so I decided not to do it. The only time I find myself begrudging people who do that are someone who picked something up six months ago and decided, oh, I'm going to build some video course on how to do this thing. The end. And charge a bunch of money for it and put myself out as an expert in that space.And you look at what the content they're putting out is, and one, it's inaccurate, which just drives me up a wall, and two, there's a lack of awareness that teaching is its own skill. In some areas, I know how to teach certain things, and in other areas, I'm a complete disaster at it. Public speaking is a great example. A lot of what I do on the public speaking stage is something that comes to me somewhat naturally. So, can you teach me to be a good public speaker? Not really, it's like, well, you gave that talk and it was bad. Could you try giving it only make it good? Like, that is not a helpful coaching statement, so I stay out of that mess.Andrew: Yeah, I mean, it's really challenging to know, if you feel like you're authority enough to put something out there. And there's been a few courses where I didn't feel like I was the most knowledgeable, but I produced those courses, and they had done extremely well. But as I was going through the course, I was just like, “Yeah, I don't know how any this stuff works, but this is my best guess translating from here.” And so you know, at least for my content, people have seen me as, like, the lens of AWS on top of other platforms, right? So, I might not know—I'm not an expert in Azure, but I've made a lot of Azure content, and I just translate that over and I talk about the frustrations around, like, using scale sets compared to AWS auto-scaling groups, and that seems to really help people get through the motions of it.I know if I pass, at least they'll pass, but by no means do I ever feel like an expert. Like, right now I'm doing, like, Kubernetes. Like, I have no idea how I'm doing it, but I have, like, help with three other people. And so I'll just be honest about it and say, “Hey, yeah, I'm learning this as well, but at least I know I passed, so you know, you can pass, too.” Whatever that's worth.Corey: Oh, yeah. Back when I was starting out, I felt like a bit of a fraud because I didn't know everything about the AWS billing system and how it worked and all the different things people can do with it, and things they can ask. And now, five years later, when the industry basically acknowledges I'm an expert, I feel like a fraud because I couldn't possibly understand everything about the AWS billing system and how it works. It's one of those things where the more you learn, the more you realize that there is yet to learn. I'm better equipped these days to find the answers to the things I need to know, but I'm still learning things every day. If I ever get to a point of complete and total understanding of a given topic, I'm wrong. You can always go deeper.Andrew: Yeah, I mean, by no means am I even an expert in AWS, though people seem to think that I am just because I have a lot of confidence in there and I produce a lot of content. But that's a lot different from making a course than implementing stuff. And I do implement stuff, but you know, it's just at the scale that I'm doing that. So, just food for thought for people there.Corey: Oh, yeah. Whatever, I implement something. It's great. In my previous engineering life, I would work on large-scale systems, so I know how a thing that works in your test environment is going to blow up in a production scale environment. And I bring those lessons, written on my bones the painful way, through outages, to the way that I build things now.But the stuff that I'm building is mostly to keep my head in the game, as opposed to solving an explicit business need. Could I theoretically build a podcast transcription system on top of Transcribe or something like that for these episodes? Yeah. But I've been paying a person to do this for many years to do it themselves; they know the terms of art, they know how this stuff works, and they're building a glossary as they go, and understanding the nuances of what I say and how I say it. And that is the better business outcome; that's the answer. And if it's production facing, I probably shouldn't be tinkering with it too much, just based upon where the—I don't want to be the bottleneck for the business functioning.Andrew: I've been spending so much time doing the same thing over and over again, but for different cloud providers, and the more I do, the less I want to go deep on these things because I just feel like I'm dumping all this information I'm going to forget, and that I have those broad strokes, and when I need to go deep dive, I have that confidence. So, I'd really prefer people were to build up confidence in saying, “Yes, I think I can do this.” As opposed to being like, “Oh, I have proof that I know every single feature in AWS Systems Manager.” Just because, like, our platform, ExamPro, like, I built it with my co-founder, and it's a quite a system. And so I'm going well, that's all I need to know.And I talk to other CTOs, and there's only so much you need to know. And so I don't know if there's, like, a shift between—or difference between, like, application development where, let's say you're doing React and using Vercel and stuff like that, where you have to have super deep knowledge for that technical stack, whereas cloud is so broad or diverse that maybe just having confidence and hypothesizing the work that you can do and seeing what the outcome is a bit different, right? Not having to prove one hundred percent that you know it inside and out on day one, but have the confidence.Corey: And there's a lot of validity to that and a lot of value to it. It's the magic word I always found in interviewing, on both sides of the interview table, has always been someone who's unsure about something start with, “I'm not sure, but if I had to guess,” and then say whatever it is you were going to say. Because if you get it right, wow, you're really good at figuring this out, and your understanding is pretty decent. If you're wrong, well, you've shown them how you think but you've also called them out because you're allowed to be wrong; you're not allowed to be authoritatively wrong. Because once that happens, I can't trust anything you say.Andrew: Yeah. In terms of, like, how do cloud certifications help you for your career path? I mean, I find that they're really well structured, and they give you a goal to work towards. So, like, passing that exam is your motivation to make sure that you complete it. Do employers care? It depends. I would say mostly no. I mean, for me, like, when I'm hiring, I actually do care about certifications because we make certification courses but—Corey: In your case, you're a very specific expression of this that is not typical.Andrew: Yeah. And there are some, like, cases where, like, if you work for a larger cloud consultancy, you're expected to have a professional certification so that customers feel secure in your ability to execute. But it's not like they were trying to hire you with that requirement, right? And so I hope that people realize that and that they look at showing that practical skills, by building up cloud projects. And so that's usually a strong pairing I'll have, which is like, “Great. Get the certifications to help you just have a structured journey, and then do a Cloud project to prove that you can do what you say you can do.”Corey: One area where I've seen certifications act as an interesting proxy for knowledge is when you have a company that has 5000 folks who work in IT in varying ways, and, “All right. We're doing a big old cloud migration.” The certification program, in many respects, seems to act as a bit of a proxy for gauging where people are on upskilling, how much they have to learn, where they are in that journey. And at that scale, it begins to make some sense to me. Where do you stand on that?Andrew: Yeah. I mean, it's hard because it really depends on how those paths are built. So, when you look at the AWS certification roadmap, they have the Certified Cloud Practitioner, they have three associates, two professionals, and a bunch of specialties. And I think that you might think, “Well, oh, solutions architect must be very popular.” But I think that's because AWS decided to make the most popular, the most generic one called that, and so you might think that's what's most popular.But what they probably should have done is renamed that Solution Architect to be a Cloud Engineer because very few people become Solutions Architect. Like that's more… if there's Junior Solutions Architect, I don't know where they are, but Solutions Architect is more of, like, a senior role where you have strong communications, pre-sales, obviously, the role is going to vary based on what companies decide a Solution Architect is—Corey: Oh, absolutely take a solutions architect, give him a crash course in finance, and we call them a cloud economist.Andrew: Sure. You just add modifiers there, and they're something else. And so I really think that they should have named that one as the cloud engineer, and they should have extracted it out as its own tier. So, you'd have the Fundamental, the Certified Cloud Practitioner, then the Cloud Engineer, and then you could say, “Look, now you could do developer or the sysops.” And so you're creating this path where you have a better trajectory to see where people really want to go.But the problem is, a lot of people come in and they just do the solutions architect, and then they don't even touch the other two because they say, well, I got an associate, so I'll move on the next one. So, I think there's some structuring there that comes into play. You look at Azure, they've really, really caught up to AWS, and may I might even say surpass them in terms of the quality and the way they market them and how they construct their certifications. There's things I don't like about them, but they have, like, all these fundamental certifications. Like, you have Azure Fundamentals, Data Fundamentals, AI Fundamentals, there's a Security Fundamentals.And to me, that's a lot more valuable than going over to an associate. And so I did all those, and you know, I still think, like, should I go translate those over for AWS because you have to wait for a specialty before you pick up security. And they say, like, it's intertwined with all the certifications, but, really isn't. Like—and I feel like that would be a lot better for AWS. But that's just my personal opinion. So.Corey: My experience with AWS certifications has been somewhat minimal. I got the Cloud Practitioner a few years ago, under the working theory of I wanted to get into the certified lounge at some of the events because sometimes I needed to charge things and grab a cup of coffee. I viewed it as a lounge pass with a really strange entrance questionnaire. And in my case, yeah, I passed it relatively easily; if not, I would have some questions about how much I actually know about these things. As I recall, I got one question wrong because I was honest, instead of going by the book answer for, “How long does it take to restore an RDS database from a snapshot?”I've had some edge cases there that give the wrong answer, except that's what happened. And then I wound up having that expire and lapse. And okay, now I'll do it—it was in beta at the time, but I got the sysops associate cert to go with it. And that had a whole bunch of trivia thrown into it, like, “Which of these is the proper syntax for this thing?” And that's the kind of question that's always bothered me because when I'm trying to figure things like that out, I have entire internet at my fingertips. Understanding the exact syntax, or command-line option, or flag that needs to do a thing is a five-second Google search away in most cases. But measuring for people's ability to memorize and retain that has always struck me as a relatively poor proxy for knowledge.Andrew: It's hard across the board. Like Azure, AWS, GCP, they all have different approaches—like, Terraform, all of them, they're all different. And you know, when you go to interview process, you have to kind of extract where the value is. And I would think that the majority of the industry, you know, don't have best practices when hiring, there's, like, a superficial—AWS is like, “Oh, if you do well, in STAR program format, you must speak a communicator.” Like, well, I'm dyslexic, so that stuff is not easy for me, and I will never do well in that.So like, a lot of companies hinge on those kinds of components. And I mean, I'm sure it doesn't matter; if you have a certain scale, you're going to have attrition. There's no perfect system. But when you look at these certifications, and you say, “Well, how much do they match up with the job?” Well, they don't, right? It's just Jeopardy.But you know, I still think there's value for yourself in terms of being able to internalize it. I still think that does prove that you have done something. But taking the AWS certification is not the same as taking Andrew Brown's course. So, like, my certified cloud practitioner was built after I did GCP, Oracle Cloud, Azure Fundamentals, a bunch of other Azure fundamental certifications, cloud-native stuff, and then I brought it over because was missing, right? So like, if you went through my course, and that I had a qualifier, then I could attest to say, like, you are of this skill level, right?But it really depends on what that testament is and whether somebody even cares about what my opinion of, like, your skillset is. But I can't imagine like, when you have a security incident, there's going to be a pop-up that shows you multiple-choice answer to remediate the security incident. Now, we might get there at some point, right, with all the cloud automation, but we're not there yet.Corey: It's been sort of thing we've been chasing and never quite get there. I wish. I hope I live to see it truly I do. My belief is also that the value of a certification changes depending upon what career stage someone is at. Regardless of what level you are at, a hiring manager or a company is looking for more or less a piece of paper that attests that they're to solve the problem that they are hiring to solve.And entry-level, that is often a degree or a certification or something like that in the space that shows you have at least the baseline fundamentals slash know how to learn things. After a few years, I feel like that starts to shift into okay, you've worked in various places solving similar problems on your resume that the type that we have—because the most valuable thing you can hear when you ask someone, “How would we solve this problem?” Is, “Well, the last time I solved it, here's what we learned.” Great. That's experience. There's no compression algorithm for experience? Yes, there is: Hiring people with experience.Then, at some level, you wind up at the very far side of people who are late-career in many cases where the piece of paper that shows that they know what they're doing is have you tried googling their name and looking at the Wikipedia article that spits out, how they built fundamental parts of a system like that. I think that certifications are one of those things that bias for early-career folks. And of course, partners when there are other business reasons to get it. But as people grow in seniority, I feel like the need for those begins to fall off. Do you agree? Disagree? You're much closer to this industry in that aspect of it than I am.Andrew: The more senior you are, and if you have big names under your resume there, no one's going to care if you have certification, right? When I was looking to switch careers—I used to have a consultancy, and I was just tired of building another failed startup for somebody that was willing to pay me. And I'm like—I was not very nice about it. I was like, “Your startup's not going to work out. You really shouldn't be building this.” And they still give me the money and it would fail, and I'd move on to the next one. It was very frustrating.So, closed up shop on that. And I said, “Okay, I got to reenter the market.” I don't have a computer science degree, I don't have big names on my resume, and Toronto is a very competitive market. And so I was feeling friction because people were not valuing my projects. I had, like, full-stack projects, I would show them.And they said, “No, no. Just do these, like, CompSci algorithms and stuff like that.” And so I went, “Okay, well, I really don't want to be doing that. I don't want to spend all my time learning algorithms just so I can get a job to prove that I already have the knowledge I have.” And so I saw a big opportunity in cloud, and I thought certifications would be the proof to say, “I can do these things.”And when I actually ended up going for the interviews, I didn't even have certifications and I was getting those opportunities because the certifications helped me prove it, but nobody cared about the certifications, even then, and that was, like, 2017. But not to say, like, they didn't help me, but it wasn't the fact that people went, “Oh, you have a certification. We'll get you this job.”Corey: Yeah. When I'm talking to consulting clients, I've never once been asked, “Well, do you have the certifications?” Or, “Are you an AWS partner?” In my case, no, neither of those things. The reason that we know what we're doing is because we've done this before. It's the expertise approach.I question whether that would still be true if we were saying, “Oh, yeah, and we're going to drop a dozen engineers on who are going to build things out of your environment.” “Well, are they certified?” is a logical question to ask when you're bringing in an external service provider? Or is this just a bunch of people you found somewhere on Upwork or whatnot, and you're throwing them at it with no quality control? Like, what is the baseline level experience? That's a fair question. People are putting big levels of trust when they bring people in.Andrew: I mean, I could see that as a factor of some clients caring, just because like, when I used to work in startups, I knew customers where it's like their second startup, and they're flush with a lot of money, and they're deciding who they want to partner with, and they're literally looking at what level of SSL certificate they purchased, right? Like now, obviously, they're all free and they're very easy to get to get; there was one point where you had different tiers—as if you would know—and they would look and they would say—Corey: Extended validation certs attend your browser bar green. Remember those?Andrew: Right. Yeah, yeah, yeah. It was just like that, and they're like, “We should partner with them because they were able to afford that and we know, like…” whatever, whatever, right? So, you know, there is that kind of thought process for people at an executive level. I'm not saying it's widespread, but I've seen it.When you talk to people that are in cloud consultancy, like solutions architects, they always tell me they're driven to go get those professional certifications [unintelligible 00:22:19] their customers matter. I don't know if the customers care or not, but they seem to think so. So, I don't know if it's just more driven by those people because it's an expectation because everyone else has it, or it's like a package of things, like, you know, like the green bar in the certifications, SOC 2 compliance, things like that, that kind of wrap it up and say, “Okay, as a package, this looks really good.” So, more of an expectation, but not necessarily matters, it's just superficial; I'm not sure.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance accelerator for the Oracle MySQL Database Service. Although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLTP and OLAP, don't ask me to ever say those acronyms again, workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: You've been building out certifications for multiple cloud providers, so I'm curious to get your take on something that Forrest Brazeal, who's now head of content over at Google Cloud, has been talking about lately, the idea that as an engineer is advised to learn more than one cloud provider; even if you have one as a primary, learning how another one works makes you a better engineer. Now, setting aside entirely the idea that well, yeah, if I worked at Google, I probably be saying something fairly similar.Andrew: Yeah.Corey: Do you think there's validity to the idea that most people should be broad across multiple providers, or do you think specialization on one is the right path?Andrew: Sure. Just to contextualize for our listeners, Google Cloud is highly, highly promoting multi-cloud workloads, and one of their flagship products is—well, they say it's a flagship product—is Anthos. And they put a lot of money—I don't know that was subsidized, but they put a lot of money in it because they really want to push multi-cloud, right? And so when we say Forrest works in Google Cloud, it should be no surprise that he's promoting it.But I don't work for Google, and I can tell you, like, learning multi-cloud is, like, way more valuable than just staying in one vertical. It just opened my eyes. When I went from AWS to Azure, it was just like, “Oh, I'm missing out on so much in the industry.” And it really just made me such a more well-rounded person. And I went over to Google Cloud, and it was just like… because you're learning the same thing in different variations, and then you're also poly-filling for things that you will never touch.Or like, I shouldn't say you never touch, but you would never touch if you just stayed in that vertical when you're learning. So, in the industry, Azure Active Directory is, like, widespread, but if you just stayed in your little AWS box, you're not going to notice it on that learning path, right? And so a lot of times, I tell people, “Go get your CLF-C01 and then go get your AZ-900 or AZ-104.” Again, I don't care if people go and sit the exams. I want them to go learn the content because it is a large eye-opener.A lot of people are against multi-cloud from a learning perspective because say, it's too much to learn all at the same time. But a lot of people I don't think have actually gone across the cloud, right? So, they're sitting from their chair, only staying in one vertical saying, “Well, you can't learn them all at the same time.” And I'm going, “I see a way that you could teach them all at the same time.” And I might be the first person that will do it.Corey: And the principles do convey as well. It's, “Oh, well I know how SNS works on AWS, so I would never be able to understand how Google Pub/Sub works.” Those are functionally identical; I don't know that is actually true. It's just different to interface points and different guarantees, but fine. You at least understand the part that it plays.I've built things out on Google Cloud somewhat recently, and for me, every time I do, it's a refreshing eye-opener to oh, this is what developer experience in the cloud could be. And for a lot of customers, it is. But staying too far within the bounds of one ecosystem does lend itself to a loss of perspective, if you're not careful. I agree with that.Andrew: Yeah. Well, I mean, just the paint more of a picture of differences, like, Google Cloud has a lot about digital transformation. They just updated their—I'm not happy that they changed it, but I'm fine that they did that, but they updated their Google Digital Cloud Leader Exam Guide this month, and it like is one hundred percent all about digital transformation. So, they love talking about digital transformation, and those kind of concepts there. They are really good at defining migration strategies, like, at a high level.Over to Azure, they have their own cloud adoption framework, and it's so detailed, in terms of, like, execution, where you go over to AWS and they have, like, the worst cloud adoption framework. It's just the laziest thing I've ever seen produced in my life compared to out of all the providers in that space. I didn't know about zero-trust model until I start using Azure because Azure has Active Directory, and you can do risk-based policy procedures over there. So, you know, like, if you don't go over to these places, you're not going to get covered other places, so you're just going to be missing information till you get the job and, you know, that job has that information requiring you to know it.Corey: I would say that for someone early career—and I don't know where this falls on the list of career advice ranging from, “That is genius,” to, “Okay, Boomer,” but I would argue that figuring out what companies in your geographic area, or the companies that you have connections with what they're using for a cloud provider, I would bias for learning one enough to get hired there and from there, letting what you learn next be dictated by the environment you find yourself in. Because especially larger companies, there's always something that lives in a different provider. My default worst practice is multi-cloud. And I don't say that because multi-cloud doesn't exist, and I'm not saying it because it's a bad idea, but this idea of one workload—to me—that runs across multiple providers is generally a challenge. What I see a lot more, done intelligently, is, “Okay, we're going to use this provider for some things, this other provider for other things, and this third provider for yet more things.” And every company does that.If not, there's something very strange going on. Even Amazon uses—if not Office 365, at least exchange to run their email systems instead of Amazon WorkMail because—Andrew: Yeah.Corey: Let's be serious. That tells me a lot. But I don't generally find myself in a scenario where I want to build this application that is anything more than Hello World, where I want it to run seamlessly and flawlessly across two different cloud providers. That's an awful lot of work that I struggle to identify significant value for most workloads.Andrew: I don't want to think about securing, like, multiple workloads, and that's I think a lot of friction for a lot of companies are ingress-egress costs, which I'm sure you might have some knowledge on there about the ingress-egress costs across providers.Corey: Oh, a little bit, yeah.Andrew: A little bit, probably.Corey: Oh, throwing data between clouds is always expensive.Andrew: Sure. So, I mean, like, I call multi-cloud using multiple providers, but not in tandem. Cross-cloud is when you want to use something like Anthos or Azure Arc or something like that where you extend your data plane or control pla—whatever the plane is, whatever plane across all the providers. But you know, in practice, I don't think many people are doing cross-cloud; they're doing multi-cloud, like, “I use AWS to run my primary workloads, and then I use Microsoft Office Suite, and so we happen to use Azure Active Directory, or, you know, run particular VM machines, like Windows machines for our accounting.” You know?So, it's a mixed bag, but I do think that using more than one thing is becoming more popular just because you want to use the best in breed no matter where you are. So like, I love BigQuery. BigQuery is amazing. So, like, I ingest a lot of our data from, you know, third-party services right into that. I could be doing that in Redshift, which is expensive; I could be doing that in Azure Synapse, which is also expensive. I mean, there's a serverless thing. I don't really get serverless. So, I think that, you know, people are doing multi-cloud.Corey: Yeah. I would agree. I tend to do things like that myself, and whenever I see it generally makes sense. This is my general guidance. When I talk to individuals who say, “Well, we're running multi-cloud like this.” And my response is, “Great. You're probably right.”Because I'm talking in the general sense, someone building something out on day one where they don't know, like, “Everyone's saying multi-cloud. Should I do that?” No, I don't believe you should. Now, if your company has done that intentionally, rather than by accident, there's almost certainly a reason and context that I do not have. “Well, we have to run our SaaS application in multiple cloud providers because that's where our customers are.” “Yeah, you should probably do that.” But your marketing, your billing systems, your back-end reconciliation stuff generally does not live across all of those providers. It lives in one. That's the sort of thing I'm talking about. I think we're in violent agreement here.Andrew: Oh, sure, yeah. I mean, Kubernetes obviously is becoming very popular because people believe that they'll have a lot more mobility, Whereas when you use all the different managed—and I'm still learning Kubernetes myself from the next certification I have coming out, like, study course—but, you know, like, those managed services have all different kind of kinks that are completely different. And so, you know, it's not going to be a smooth process. And you're still leveraging, like, for key things like your database, you're not going to be running that in Kubernetes Cluster. You're going to be using a managed service.And so, those have their own kind of expectations in terms of configuration. So, I don't know, it's tricky to say what to do, but I think that, you know, if you have a need for it, and you don't have a security concern—like, usually it's security or cost, right, for multi-cloud.Corey: For me, at least, the lock-in has always been twofold that people don't talk about. More—less lock-in than buy-in. One is the security model where IAM is super fraught and challenging and tricky, and trying to map a security model to multiple providers is super hard. Then on top of that, you also have the buy-in story of a bunch of engineers who are very good at one cloud provider, and that skill set is not in less demand now than it was a year ago. So okay, you're going to start over and learn a new cloud provider is often something that a lot of engineers won't want to countenance.If your team is dead set against it, there's going to be some friction there and there's going to be a challenge. I mean, for me at least, to say that someone knows a cloud provider is not the naive approach of, “Oh yeah, they know how it works across the board.” They know how it breaks. For me, one of the most valuable reasons to run something on AWS is I know what a failure mode looks like, I know how it degrades, I know how to find out what's going on when I see that degradation. That to me is a very hard barrier to overcome. Alternately, it's entirely possible that I'm just old.Andrew: Oh, I think we're starting to see some wins all over the place in terms of being able to learn one thing and bring it other places, like OpenTelemetry, which I believe is a cloud-native Kubernetes… CNCF. I can't remember what it stands for. It's like Linux Foundation, but for cloud-native. And so OpenTelemetry is just a standardized way of handling your logs, metrics, and traces, right? And so maybe CloudWatch will be the 1.0 of observability in AWS, and then maybe OpenTelemetry will become more of the standard, right, and so maybe we might see more managed services like Prometheus and Grafa—well, obviously, AWS has a managed Prometheus, but other things like that. So, maybe some of those things will melt away. But yeah, it's hard to say what approach to take.Corey: Yeah, I'm wondering, on some level, whether what the things we're talking about today, how well that's going to map forward. Because the industry is constantly changing. The guidance I would give about should you be in cloud five years ago would have been a nuanced, “Mmm, depends. Maybe for yes, maybe for no. Here's the story.” It's a lot less hedge-y and a lot less edge case-y these days when I answer that question. So, I wonder in five years from now when we look back at this podcast episode, how well this discussion about what the future looks like, and certifications, and multi-cloud, how well that's going to reflect?Andrew: Well, when we look at, like, Kubernetes or Web3, we're just seeing kind of like the standardized boilerplate way of doing a bunch of things, right, all over the place. This distributed way of, like, having this generic API across the board. And how well that will take, I have no idea, but we do see a large split between, like, serverless and cloud-natives. So, it's like, what direction? Or we'll just have both? Probably just have both, right?Corey: [Like that 00:33:08]. I hope so. It's been a wild industry ride, and I'm really curious to see what changes as we wind up continuing to grow. But we'll see. That's the nice thing about this is, worst case, if oh, turns out that we were wrong on this whole cloud thing, and everyone starts exodusing back to data centers, well, okay. That's the nice thing about being a small company. It doesn't take either of us that long to address the reality we see in the industry.Andrew: Well, that or these cloud service providers are just going to get better at offering those services within carrier hotels, or data centers, or on your on-premise under your desk, right? So… I don't know, we'll see. It's hard to say what the future will be, but I do believe that cloud is sticking around in one form or another. And it basically is, like, an essential skill or table stakes for anybody that's in the industry. I mean, of course, not everywhere, but like, mostly, I would say. So.Corey: Andrew, I want to thank you for taking the time to speak with me today. If people want to learn more about your opinions, how you view these things, et cetera. Where can they find you?Andrew: You know, I think the best place to find me right now is Twitter. So, if you go to twitter.com/andrewbrown—all lowercase, no spaces, no underscores, no hyphens—you'll find me there. I'm so surprised I was able to get that handle. It's like the only place where I have my handle.Corey: And we will of course put links to that in the [show notes 00:34:25]. Thanks so much for taking the time to speak with me today. I really appreciate it.Andrew: Well, thanks for having me on the show.Corey: Andrew Brown, co-founder and cloud instructor at ExamPro Training and so much more. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment telling me that I do not understand certifications at all because you're an accountant, and certifications matter more in that industry.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

About ShirShir Tamari is the Head of Research of Wiz, the cloud security company. He is an experienced security and technology researcher specializing in vulnerability research and practical hacking. In the past, he served as a consultant to a variety of security companies in the fields of research, development and product.About SagiSagi Tzadik is a security researcher in the Wiz Research Team. Sagi specializes in research and exploitation of web applications vulnerabilities, as well as network security and protocols. He is also a Game-Hacking and Reverse-Engineering enthusiast.About NirNir Ohfeld is a security researcher from Israel. Nir currently does cloud-related security research at Wiz. Nir specializes in the exploitation of web applications, application security and in finding vulnerabilities in complex high-level systems.Links: Wiz: https://www.wiz.io Cloud CVE Slack channel: https://cloud-cve-db.slack.com/join/shared_invite/zt-y38smqmo-V~d4hEr_stQErVCNx1OkMA Wiz Blog: https://wiz.io/blog Twitter: https://twitter.com/wiz_io TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Redis, the company behind the incredibly popular open source database that is not the bind DNS server. If you're tired of managing open source Redis on your own, or you're using one of the vanilla cloud caching services, these folks have you covered with the go to manage Redis service for global caching and primary database capabilities; Redis Enterprise. To learn more and deploy not only a cache but a single operational data platform for one Redis experience, visit redis.com/hero. Thats r-e-d-i-s.com/hero. And my thanks to my friends at Redis for sponsoring my ridiculous non-sense.  Corey: This episode is sponsored in part by our friends at Rising Cloud, which I hadn't heard of before, but they're doing something vaguely interesting here. They are using AI, which is usually where my eyes glaze over and I lose attention, but they're using it to help developers be more efficient by reducing repetitive tasks. So, the idea being that you can run stateless things without having to worry about scaling, placement, et cetera, and the rest. They claim significant cost savings, and they're able to wind up taking what you're running as it is in AWS with no changes, and run it inside of their data centers that span multiple regions. I'm somewhat skeptical, but their customers seem to really like them, so that's one of those areas where I really have a hard time being too snarky about it because when you solve a customer's problem and they get out there in public and say, “We're solving a problem,” it's very hard to snark about that. Multus Medical, Construx.ai and Stax have seen significant results by using them. And it's worth exploring. So, if you're looking for a smarter, faster, cheaper alternative to EC2, Lambda, or batch, consider checking them out. Visit risingcloud.com/benefits. That's risingcloud.com/benefits, and be sure to tell them that I said you because watching people wince when you mention my name is one of the guilty pleasures of listening to this podcast.Corey: Welcome to Screaming in the Cloud, I'm Corey Quinn. One of the joyful parts of working with cloud computing is that you get to put a whole lot of things you don't want to deal with onto the shoulders of the cloud provider you're doing business with—or cloud providers as the case may be, if you fallen down the multi-cloud well. One of those things is often significant aspects of security. And that's great, right, until it isn't. Today, I'm joined by not one guest, but rather three coming to us from Wiz, which I originally started off believing was, oh, it's a small cybersecurity research group. But they're far more than that. Thank you for joining me, and could you please introduce yourself?Shir: Yes, thank you, Corey. My name is Shir, Shir Tamari. I lead the security research team at Wiz. I working in the company for the past year. I'm working with these two nice teammates.Nir: Hi, my name is Nir Ohfield,. I'm a security researcher at the Wiz research team. I've also been working for the Wiz research team for the last year. And yeah.Sagi: I'm Sagi, Sagi Tzadik. I also work for the Wiz research team for the last six months.Corey: I want to thank you for joining me. You folks really burst onto the scene earlier this year, when I suddenly started seeing your name come up an awful lot. And it brought me back to my childhood where there was an electronics store called Nobody Beats the Wiz. It was more or less a version of Fry's on a different coast, and they went out of business and oh, good. We're going back in time. And suddenly it felt like I was going back in time in a different light because you had a number of high profile vulnerabilities that you had discovered, specifically in the realm of Microsoft Azure. The two that leap to mind the most readily for me are ChaosDB and the OMIGOD exploits. There was a third as well, but why don't you tell me, in your own words, what it is that you discovered and how that played out?Shir: We, sort of, found the vulnerabilities in Microsoft Azure. We did report multiple vulnerabilities also in GCP, and AWS. We had multiple vulnerabilities in AWS [unintelligible 00:02:42] cross-account. It was a cross-account access to other tenants; it just was much less severe than the ChaosDB vulnerability that we will speak on more later. And a both we've present in Blackhat in Vegas in [unintelligible 00:02:56]. So, we do a lot of research. You mentioned that we have a third one. Which one did you refer to?Corey: That's a good question because you had the I want to say it was called as Azurescape, and you're doing a fantastic job with branding a number of your different vulnerabilities, but there's also, once you started reporting this, a lot of other research started coming out as well from other folks. And I confess, a lot of it sort of flowed together and been very hard to disambiguate, is this a systemic problem; is this, effectively, a whole bunch of people piling on now that their attention is being drawn somewhere; or something else? Because you've come out with an awful lot of research in a short period of time.Shir: Yeah, we had a lot of good research in the past year. It's a [unintelligible 00:03:36] mention Azurecape was actually found by a very good researcher in Palo Also. And… do you remember his name?Sagi: No, I can't recall his name is.Corey: Yeah, they came out of unit 42 as I recall, their cybersecurity division. Every tech company out there seems to have some sort of security research division these days. What I think is, sort of, interesting is that to my understanding, you were founded, first and foremost, as a security company. You're not doing this as an ancillary to selling something else like a firewall, or, effectively, you're an ad comp—an ad tech company like Google, we you're launching Project Zero. You are first and foremost aimed at this type of problem.Shir: Yes. Wiz is not just a small research company. It's actually pretty big company with over 200 employees. And the purpose of this product is a cloud security suite that provides [unintelligible 00:04:26] scanning capabilities in order to find risks in cloud environments. And the research team is a very small group. We are [unintelligible 00:04:35] researchers.We have multiple responsibilities. Our first responsibility is to find risks in cloud environments: It could be misconfigurations, it could be vulnerabilities in libraries, in software, and we add those findings and the patterns we discover to the product in order to protect our customers, and to allow them for new risks. Our second responsibility is also to do a community research where we research everyone vulnerabilities in public products and cloud providers, and we share our findings with the cloud providers, then also with the community to make the cloud more secure.Corey: I can't shake the feeling that if there weren't folks doing this sort of research and shining a light on what it is that the cloud providers are doing, if they were to discover these things at all, they would very quietly, effectively, fix it in the background and never breathe a word of it in public. I like the approach that you're taking as far as dragging it, kicking and screaming, into the daylight, but I also have to imagine that probably doesn't win you a whole lot of friends at the company that you're focusing on at any given point in time. Because whenever you talk to a company about a security issue, it seems like the first thing they're concerned about is, “Okay, how do we wind up spinning this or making sure that we minimize the reputational damage?” And then there's a secondary reaction of, “Oh, and how do we protect our customers? But mostly, how do we avoid looking bad as a result?” And I feel like that's an artifact of corporate culture these days. But it feels like the relationship has got to be somewhat interesting to navigate from your perspective.Shir: So, once we found a vulnerability and we discuss it with the vendor, okay, first, I will mention that most cloud providers have a bug bounty program where they encourage researchers to find vulnerabilities and to discover new security threats. And all of them, as a public disclosure, [unintelligible 00:06:29] program will researchers are welcome and get safe harbor, you know, where the disclosure vulnerabilities. And I think it's, like, common interest, both for customers, but for researchers, and the cloud providers to know about those vulnerabilities, to mitigate it down. And we do believe that sometimes cloud providors does resolve and mitigate vulnerabilities behind the scenes, and we know—we don't know for sure, but—I don't know about everything, but just by the vulnerabilities that we find, we assume that there is much more of them that we never heard about. And this is something that we believe needs to be changed in the industry.Cloud providers should be more transparent, they should show more information about the result vulnerabilities. Definitely when a customer data was accessible, or where it was at risk, or at possible risk. And this is actually—it's something that we actually trying to change in the industry. We have a community and, like, innovative community. It's like an initiative that we try to collect, we opened a Slack channel called the Cloud CVE, and we try to invite as much people as we can that concern about cloud's vulnerabilities, in order to make a change in the industry, and to assist cloud providers, or to convince cloud providers to be more transparent, to enumerate cloud vulnerabilities so they have an identifier just, like cloud CVE, like a CVE, and to make the cloud more protected and more transparent customers.Corey: The thing that really took me aback by so much of what you found is that we've become relatively accustomed to a few patterns over the past 15 to 20 years. For example, we're used to, “Oh, this piece of software you run on your desktop has a horrible flaw. Great.” Or this thing you run in your data center, same story; patch, patch, patch, patch patch. That's great.But there was always the sense that these were the sorts of things that were sort of normal, but the cloud providers were on top of things, where they were effectively living up to their side of the shared responsibility bargain. And that whenever you wound up getting breached, for whatever reason—like in the AWS world, where oh, you wound up losing a bunch of customer data because you had an open S3 bucket? Well, yeah, that's not really something you can hang super effectively around the neck of the cloud provider, given that you're the one that misconfigured that. But what was so striking about what you found with both of the vulnerabilities that we're talking about today, the customer could have done everything absolutely correctly from the beginning and still had their data exposed. And that feels like it's something relatively new in the world of cloud service providers.Is this something that's been going on for a while and we're just now shining a light on it? Have I just missed a bunch of interesting news stories where the clouds have—“Oh, yeah, by the way, people, we periodically have to go in and drag people out of our cloud control plane because oops-a-doozy, someone got in there again with the squirrels,” or is this something that is new?Shir: So, we do see an history other cases where probability [unintelligible 00:09:31] has disclosed vulnerabilities in the cloud infrastructure itself. There was only few, and usually, it was—the research was conducted by independent researchers. And I don't think it had such an impact, like ChaosDB, which allowed [cross-system 00:09:51] access to databases of other customers, which was a huge case. And so if it wasn't a big story, so most people will not hear about it. And also, independent researchers usually don't have the back that we have here in Wiz.We have a funding, we have the marketing division that help us to get coverage with reporters, who make sure to make—if it's a big story, we make sure that other people will hear about it. And I believe that in most bug bounty programs where independent researchers find vulnerabilities, usually they more care about the bounty than the aftereffect of stopping the vulnerability, sharing it with the community. Usually also, independent [unintelligible 00:10:32] usually share the findings with the research community. And the research community is relatively small to the IT community. So, it is new, but it's not that new.There was some events back in history, [unintelligible 00:10:46] similar vulnerabilities. So, I think that one of the points here is that everyone makes a mistake. You can find bugs which affected mostly, as you mentioned previously, this software that you installed on your desktop has bugs and you need to patch it, but in the case of cloud providers, when they make mistakes, when they introduce bugs to the service, it affects all of their customers. And this is something that we should think about. So, mistakes that are being made by cloud providers have a lot of impact regarding their customers.Corey: Yeah. It's not a story of you misconfigured, your company's SAN, so you're the one that was responsible for a data breach. It's suddenly, you're misconfiguring everyone's SAN simultaneously. It's the sheer scale and scope of what it is that they've done. And—Shir: Yeah, exactly.Corey: —I'm definitely on board with that. But the stuff I've seen in the past, from cloud providers—AWS, primarily, since that is admittedly where I tend to focus most of my time and energy—has been privilege escalation style stuff, where, okay, if you assign some users at your company—or wherever—access to this managed IAM policy, well, they'll have suddenly have access to things that go beyond the scope of that. And that's not good, let's be very clear on that, but it is a bit different between that and oh, by the way, suddenly, someone in another company that has no relationship established with you at all can suddenly rummage through your data that you're storing in Cosmos DB, their managed database offering. That's the thing to me that I think was the big head-turning aspect of this, not just for me, but for a number of folks I've spoken to, in financial services, in government, in a bunch of environments where data privacy is not optional in the same way that it is when, you know, you're running a social media for pets app.Nir: [laugh]. Yeah, but the thing is, that until the publication of ChaosDB, no one ever heard about the [unintelligible 00:12:40] data tampering in any cloud providers. Meaning maybe in six months, you can see a similar vulnerabilities in other cloud providers that maybe other security research groups find. So yeah, so Azure was maybe the first, but we don't think they will be the last.Shir: Yes. And also, when we do the community research, it is very important to us to take big targets. We enjoy the research. One day, the research will be challenging and we want to do something that it was new and great, so we always put a very big targets. To actually find vulnerability in the infrastructure of the cloud provider, it was very challenging for us.When didn't came ChaosDB by that; we actually found it by mistake. But now we think actively that this is our next goal is to find vulnerabilities in the infrastructure and not just vulnerabilities that affect only the—vulnerabilities within the account itself, like [unintelligible 00:13:32] or bad scoped policies that affects only one account.Corey: That seems to be the transformative angle that you don't see nearly as much in existing studies around vulnerabilities in this space. It's always the, “Oh, no. We could have gotten breached by those people across the hallway from us in our company,” as opposed to folks on the other side of the planet. And that is, I guess, sort of the scary thing. What has also been interesting to me, and you obviously have more experience with this than I do, but I have a hard time envisioning that, for example, AWS, having a vulnerability like this and not immediately swinging into disaster firefighting mode, sending their security execs on a six month speaking tour to explain what happened, how it got there, all of the steps that they're taking to remediate this, but Azure published a blog post explaining this in relatively minor detail: Here are the mitigations you need to take, and as far as I can tell, then they sort of washed their hands of the whole thing and have enthusiastically begun saying absolutely nothing since.And that I have learned is sort of fairly typical for Microsoft, and has been for a while, where they just don't talk about these things when it arises. Does that match your experience? Is this something that you find that is common when a large company winds up being, effectively, embarrassed about their security architecture, or is this something that is unique to Microsoft tends to approach these things?Shir: I would say in general, we really like the Microsoft MSRC team. The group in Microsoft that's responsible for handling vulnerabilities, and I think it's like the security division inside Microsoft, MSRC. So, we have a really good relationship and we had really good time working with them. They're real professionals, they take our findings very seriously. I can tell that in the ChaosDB incident, they didn't plan to publish a blog post, and they did that after the story got a lot of attention.So, I'm looking at a PR team, and I have no idea out there decide stuff and what is their strategy, but as I mentioned earlier, we believe that there is much more cloud vulnerabilities that we never heard of, and it should change; they should publish more.Nir: It's also worth mentioning that Microsoft acted really quick on this vulnerability and took it very seriously. They issued the fix in less than 48 hours. They were very transparent in the entire procedure, and we had multiple teams meeting with them. The entire experience was pretty positive with each of the vulnerability we've ever reported to Microsoft.Sagi: So, it's really nice working with the guys that are responsible for security, but regarding PR, I agree that they should have posted more information regarding this incident.Corey: The thing that I found interesting about this, and I've seen aspects of it before, but never this strongly is, I was watching for, I guess, what I would call just general shittiness, for lack of a better term, from the other providers doing a happy dance of, “Aha, we're better than you are,” and I saw none of that. Because when I started talking to people in some depth at this at other companies, the immediate response—not just AWS, to be clear—has been no, no, you have to understand, this is not good for anyone because this effectively winds up giving fuel to the slow-burning fire of folks who are pulling the, “See, I told you the cloud wasn't secure.” And now the enterprise groundhog sees that shadow and we get six more years of building data centers instead of going to the cloud. So, there's no one in the cloud space who's happy with this kind of revelation and this type of vulnerability. My question for you is given that you are security researchers, which means you are generally cynical and pessimistic about almost everything technological, if you're like most of the folks in that space that I've spent time with, is going with cloud the wrong answer? Should people be building their own data centers out? Should they continue to be going on this full cloud direction? I mean, what can they do if everything's on fire and terrible all the time?Shir: So, I think that there is a trade-off when you embrace the cloud. On one hand, you get the fastest deployment times, and a good scalability regarding your infrastructure, but on the other end, when there is a security vulnerability in the cloud provider, you are immediately affected. But it is worth mentioning that the security teams or the cloud providers are doing extremely good job. Most likely, they are going to patch the vulnerability faster than it would have been patched in on-premise environment. And it's good that you have them working for you.And once the vulnerability is mitigated—depends on the vulnerability but in the case of ChaosDB—when the vulnerability was mitigated on Microsoft's end, and it was mitigated completely. No one else could have exploited after the mitigated it once. Yes, it's also good to mention that the cloud provides organization and companies a lot of security features, [unintelligible 00:18:34] I want to say security features, I would say, it provides a lot of tooling that helps security. The option to have one interface, like one API to control all of my devices, to get visibility to all of my servers, to enforce policies very easily, it's much more secure than on-premise environments, where there is usually a big mess, a lot of vendors.Because the power was in the on-prem, the power was on the user, so the user had a lot of options. Usually used many types of software, many types of hardware, it's really hard to mitigate the software vulnerability in on-prem environments. It's really helped to get the visibility. And the cloud provides a lot of security, like, a good aspects, and in my opinion, moving to the cloud for most organization would be a more secure choice than remain on-premise, unless you have a very, very small on-prem environment.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance accelerator for the Oracle MySQL Database Service. Although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLTP and OLAP, don't ask me to ever say those acronyms again, workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: The challenge I keep running into is that—and this is sort of probably the worst of all possible reasons to go with cloud, but let's face it, when us-east-1 recently took an outage and basically broke a decent swath of the internet, a lot of companies were impacted, but they didn't see their names in the headlines; it was all about Amazon's outage. There's a certain value when a cloud provider takes an outage or a security breach, that the headlines screaming about it are about the provider, not about you and your company as a customer of that provider. Is that something that you're seeing manifest across the industry? Is that an unhealthy way to think about it? Because it feels almost like it's cheating in a way. It's, “Yeah, we had a security problem, but so did the entire internet, so it's okay.”Nir: So, I think that if there would be evidence that these kind of vulnerabilities were exploited while disclosure, then you wouldn't see headlines of companies, shouting in the headlines. But in the case of the us reporting the vulnerabilities prior to anyone exploiting them, results in nowhere a company showing up in the headlines. I think it's a slightly different situation than an outage.Shir: Yeah, but also, when one big provider have an outage or a breach, so usually, the customers will think it's out of my responsibility. I mean, it's bad; my data has been leaked, but what can I do? I think it's very easy for most people to forgive companies [unintelligible 00:21:11]. I mean, you know what, it's just not my area. So, maybe I'm not answer that into that. [laugh].Corey: No, no, it's very fair. The challenge I have, as a customer of all of these providers, to be honest, is that a lot of the ways that the breach investigations are worded of, “We have seen no evidence that this has been exploited.” Okay, that simultaneously covers the two very different use cases of, “We have pored through our exhaustive audit logs and validated that no one has done this particular thing in this particular way,” but it also covers the use case, “Of, hey, we learned we should probably be logging things, but we have no evidence that anything was exploited.” Having worked with these providers at scale, my gut impression is that they do in fact, have fairly detailed logs of who's doing what and where. Would you agree with that assessment, or do you find that you tend to encounter logging and analysis gaps as you find these exploits?Shir: We don't really know. Usually when—I mean, ChaosDB scenario, we got access to a Jupyter Notebook. And from the Jupyter Notebook, we continued to another internal services. And we—nobody stopped us. Nobody—we expected an email, like—Corey: “Whatcha doing over there, buddy?”Shir: Yeah. “Please stop doing that, and we're investigating you.” And we didn't get any. And also, we don't really know if they monitor it or not. I can tell from my technical background that logging so many environments, it's hard.And when you do decide to log all these events, you need to decide what to log. For example, if I have a database, a managed database, do I log all the queries that customers run? It's too much. If I have an HTTP application—a managed HTTP application—do I save all the access logs, like all the requests? And if so, what will be the retention time? For how long?We believe that it's very challenging on the cloud provider side, but it just an assumption. And doing the discussion with Microsoft, the didn't disclose any, like, scenarios they had with logging. They do mention that they're [unintelligible 00:23:26] viewing the logs and searching to see if someone exploited this vulnerability before we disclosed it. Maybe someone discovered before we did. But they told us they didn't find anything.Corey: One last area I'd love to discuss with you before we call it an episode is that it's easy to view Wiz through the lens of, “Oh, we just go out and find vulnerabilities here and there, and we make companies feel embarrassed—rightfully so—for the things that they do.” But a little digging shows that you've been around for a little over a year as a publicly known entity, and during that time, you've raised $600 million in funding, which is basically like what in the world is your pitch deck where you show up to investors and your slides are just, like, copies of their emails, and you read them to them?[laugh]I mean, on some level, it seems like that is a… as-, astounding amount of money to raise in a short period of time. But I've also done a little bit of digging, and to be clear, I do not believe that you have an extortion-based business model, which is a good thing. You're building something very interesting that does in-depth analysis of cloud workloads, and I think it's got an awful lot of promise. How does the vulnerability research that you do tie into that larger platform, other than, let's be honest, some spectacularly effective marketing.Sagi: Specifically in the ChaosDB vulnerability, we were actually not looking for a vulnerability in the cloud service providers. We were originally looking for common misconfigurations that our customers can make when they set up their Cosmos DB accounts, so that our product will be able to alert our customers regarding such misconfigurations. And then we went to the Azure portal and started to enable all of the features that Cosmos DB has to offer, and when we enabled enough features, we noticed some feature that could be vulnerable, and we started digging into it. And we ended up finding ChaosDB.But our original work was to try and find misconfigurations that our customers can make in order to protect them and not to find a vulnerability in the [CSP 00:25:31]. This was just, like, a byproduct of this research.Shir: Yes. There is, as I mentioned earlier, our main responsibility is to add a little security rist content to the product, to help customers to find new security risks in their environment. As you mentioned, like, the escalation possibilities within cloud accounts, and bad scoped policies, and many other security risks that are in the cloud area. And also, we are a very small team inside a big company, so most of the company, they are doing heavy [unintelligible 00:26:06] and talk with customers, they understand the risks, they understand the market, what the needs for tomorrow, and maybe we are well known for our vulnerabilities, but it just a very small part of the company.Corey: On some level, it says wonderful things about your product, and also terrifying things from different perspectives of, “Oh, yeah, we found one of the worst cloud breaches in years by accident,” as opposed to actively going in trying to find the thing that has basically put you on the global map of awareness around these things. Because there a lot of security companies out there doing different things. In fact, go to RSA, and you'll see basically 12 companies that just repeated over and over and over with different names and different brandings, and they're all selling some kind of firewall. This is something actively different because everyone can tell beautiful pictures with slides and whatnot, and the corporate buzzwords. You're one of those companies that actually did something meaningful, and it felt almost like a proof of concept. On some level, the fact that you weren't actively looking for it is kind of an amazing testament for the product itself.Shir: Yeah. We actually used the product in the beginning, in order to overview our own environment, and what is the most common services we use. In order—and we usually we mix this information with our product managers, know to understand what customers use and what products and services we need to research in order to bring value to the product.Sagi: Yeah, so the reason we chose to research Cosmos DB was that, we found that a lot of our Azure customers are using Cosmos DB on their production environments, and we wanted to add mitigations for common misconfigurations to our product in order to protect our customers.Nir: Yeah, the same goes with our other research, like OMIGOD, where we've seen that there is a excessive amount of [unintelligible 00:27:56] installations in an Azure environment, and it raised our [laugh] it raised our attention, and then found this vulnerability. It's mostly, like, popularity-guided research. [laugh].Shir: Yeah. And also [unintelligible 00:28:11] mention that maybe we find vulnerabilities by accident, but the service, we are doing vulnerability itself for the past ten years, and even more. So, we are very professional and this is what we do, and this is what we like to do. And we came skilled to the [crosstalk 00:28:25].Corey: It really is neat to see, just because every other security tool that I've looked at in recent memory tells you the same stuff. It's the same problem you see in the AWS billing space that I live in. Everyone says, “Oh, we can find these inactive instances that could be right-sized.” Great, because everyone's dealing with the same data. It's the security stuff is no different. “Hey, this S3 bucket is open.” Yes, it's a public web server. Please stop waking me up at two in the morning about it. It's there by design.But it goes back and forth with the same stuff just presented differently. This is one of the first truly novel things I've seen in ages. If nothing else, you convince me to kick the tires on it, and see what kind of horrifying things I can learn about my own environments with it.Shir: Yeah, you should. [laugh]. Let's poke [unintelligible 00:29:13].[laugh].Corey: I want to thank you so much for taking the time to speak with me today. If people want to learn more about the research you're up to and the things that you find interesting, where can they find you all?Shir: Most of our publication—I mean, all of our publications are under the Wiz, which is wiz.io/blog, and people can read all of our research. Just today we are announcing a new one, so feel free to go and read there. And they also feel free to approach us on Twitter, the service, we have a Twitter account. We are open for, like, messages. Just send us a message.Corey: And we will certainly put links to all of that in the [show notes 00:29:49]. Shir, Sagi, Nir, thank you so much for joining me today. I really appreciate your time.Shir: Thank you.Sagi: Thank you.Nir: Thank you much.Shir: It was very fun. Yeah.Corey: This has been Screaming in the Cloud. I'm Cloud Economist Corey Quinn and thank you for listening. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry insulting comment from someone else's account.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

About PetePete does many startup things at Allma. Links: Last Tweet in AWS: https://lasttweetinaws.com Twitter: https://twitter.com/petecheslock LinkedIn: https://www.linkedin.com/in/petecheslock/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part byLaunchDarkly. Take a look at what it takes to get your code into production. I'm going to just guess that it's awful because it's always awful. No one loves their deployment process. What if launching new features didn't require you to do a full-on code and possibly infrastructure deploy? What if you could test on a small subset of users and then roll it back immediately if results aren't what you expect? LaunchDarkly does exactly this. To learn more, visitlaunchdarkly.com and tell them Corey sent you, and watch for the wince.Corey: This episode is sponsored in part by our friends at Redis, the company behind the incredibly popular open source database that is not the bind DNS server. If you're tired of managing open source Redis on your own, or you're using one of the vanilla cloud caching services, these folks have you covered with the go to manage Redis service for global caching and primary database capabilities; Redis Enterprise. To learn more and deploy not only a cache but a single operational data platform for one Redis experience, visit redis.com/hero. Thats r-e-d-i-s.com/hero. And my thanks to my friends at Redis for sponsoring my ridiculous non-sense.  Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I am joined—as is tradition, for a post re:Invent wrap up, a month or so later, once everything is time to settle—by my friend and yours, Pete Cheslock. Pete, how are you?Pete: Hi, I'm doing fantastic. New year; new me. That's what I'm going with.Corey: That's the problem. I keep hoping for that, but every time I turn around, it's still me. And you know, honestly, I wouldn't wish that on anyone.Pete: Exactly. [laugh]. I wouldn't wish you on me either. But somehow I keep coming back for this.Corey: So, in two-thousand twenty—or twenty-twenty, as the children say—re:Invent was fully virtual. And that felt weird. Then re:Invent 2021 was a hybrid event which, let's be serious here, is not really those things. They had a crappy online thing and then a differently crappy thing in person. But it didn't feel real to me because you weren't there.That is part of the re:Invent tradition. There's a midnight madness thing, there's a keynote where they announce a bunch of nonsense, and then Pete and I go and have brunch on the last day of re:Invent and decompress, and more or less talk smack about everything that crosses our minds. And you weren't there this year. I had to backfill you with Tim Banks. You know, the person that I backfield you with here at The Duckbill Group as a principal cloud economist.Pete: You know, you got a great upgrade in hot takes, I feel like, with Tim.Corey: And other ways, too, but it's rude of me to say that to you directly. So yeah, his hot takes are spectacular. He was going to be doing this with me, except you cannot mess with tradition. You really can't.Pete: Yeah. I'm trying to think how many—is this third year? It's at least three.Corey: Third or fourth.Pete: Yeah, it's at least three. Yeah, it was, I don't want to say I was sad to not be there because, with everything going on, it's still weird out there. But I am always—I'm just that weird person who actually likes re:Invent, but not for I feel like the reasons people think. Again, I'm such an extroverted-type person, that it's so great to have this, like, serendipity to re:Invent. The people that you run into and the conversations that you have, and prior—like in 2019, I think was a great example because that was the last one I had gone to—you know, having so many conversations so quickly because everyone is there, right? It's like this magnet that attracts technologists, and venture capital, and product builders, and all this other stuff. And it's all compressed into, like, you know, that five-day span, I think is the biggest part that makes so great.Corey: The fear in people's eyes when they see me. And it was fun; I had a pair of masks with me. One of them was a standard mask, and no one recognizes anyone because, masks, and the other was a printout of my ridiculous face, which was horrifyingly uncanny, but also made it very easy for people to identify me. And depending upon how social I was feeling, I would wear one or the other, and it worked flawlessly. That was worth doing. They really managed to thread the needle, as well, before Omicron hit, but after the horrors of last year. So, [unintelligible 00:03:00]—Pete: It really—Corey: —if it were going on right now, it would not be going on right now.Pete: Yeah. I talk about really—yeah—really just hitting it timing-wise. Like, not that they could have planned for any of this, but like, as things were kind of not too crazy and before they got all crazy again, it feels like wow, like, you know, they really couldn't have done the event at any other time. And it's like, purely due to luck. I mean, absolute one hundred percent.Corey: That's the amazing power of frugality. Because the reason is then is it's the week after Thanksgiving every year when everything is dirt cheap. And, you know, if there's one thing that I one-point-seve—sorry, their stock's in the toilet—a $1.6 trillion company is very concerned about, it is saving money at every opportunity.Pete: Well, the one thing that was most curious about—so I was at the first re:Invent in-what—2012 I think it was, and there was—it was quaint, right?—there was 4000 people there, I want to say. It was in the thousands of people. Now granted, still a big conference, but it was in the Sands Convention Center. It was in that giant room, the same number of people, were you know, people's booths were like tables, like, eight-by-ten tables, right? [laugh].It had almost a DevOpsDays feel to it. And I was kind of curious if this one had any of those feelings. Like, did it evoke it being more quaint and personable, or was it just as soulless as it probably has been in recent years?Corey: This was fairly soulless because they reduced the footprint of the event. They dropped from two expo halls down to one, they cut the number of venues, but they still had what felt like 20,000 people or something there. It was still crowded, it was still packed. And I've done some diligent follow-ups afterwards, and there have been very few cases of Covid that came out of it. I quarantined for a week in a hotel, so I don't come back and kill my young kids for the wrong reasons.And that went—that was sort of like the worst part of it on some level, where it's like great. Now I could sit alone at a hotel and do some catch-up and all the rest, but all right I'd kind of like to go home. I'm not used to being on the road that much.Pete: Yeah, I think we're all a little bit out of practice. You know, I haven't been on a plane in years. I mean, the travel I've done more recently has been in my car from point A to point B. Like, direct, you know, thing. Actually, a good friend of mine who's not in technology at all had to travel for business, and, you know, he also has young kids who are under five, so he when he got back, he actually hid in a room in their house and quarantine himself in the room. But they—I thought, this is kind of funny—they never told the kids he was home. Because they knew that like—Corey: So, they just thought the house was haunted?Pete: [laugh].Corey: Like, “Don't go in the west wing,” sort of level of nonsense. That is kind of amazing.Pete: Honestly, like, we were hanging out with the family because they're our neighbors. And it was like, “Oh, yeah, like, he's in the guest room right now.” Kids have no idea. [laugh]. I'm like, “Oh, my God.” I'm like, I can't even imagine. Yeah.Corey: So, let's talk a little bit about the releases of re:Invent. And I'm going to lead up with something that may seem uncharitable, but I don't think it necessarily is. There weren't the usual torrent of new releases for ridiculous nonsense in the same way that there have been previously. There was no, this service talks to satellites in space. I mean, sure, there was some IoT stuff to manage fleets of cars, and giant piles of robots, and cool, I don't have those particular problems; I'm trying to run a website over here.So okay, great. There were enhancements to a number of different services that were in many cases appreciated, in other cases, irrelevant. Werner said in his keynote, that it was about focusing on primitives this year. And, “Why do we have so many services? It's because you asked for it… as customers.”Pete: [laugh]. Yeah, you asked for it.Corey: What have you been asking for, Pete? Because I know what I've been asking for and it wasn't that. [laugh].Pete: It's amazing to see a company continually say yes to everything, and somehow, despite their best efforts, be successful at doing it. No other company could do that. Imagine any other software technology business out there that just builds everything the customers ask for. Like from a product management business standpoint, that is, like, rule 101 is, “Listen to your customers, but don't say yes to everything.” Like, you can't do everything.Corey: Most companies can't navigate the transition between offering the same software in the Cloud and on a customer facility. So, it's like, “Ooh, an on-prem version, I don't know, that almost broke the company the last time we tried it.” Whereas you have Amazon whose product strategy is, “Yes,” being able to put together a whole bunch of things. I also will challenge the assertion that it's the primitives that customers want. They don't want to build a data center out of popsicle sticks themselves. They want to get something that solves a problem.And this has been a long-term realization for me. I used to work at Media Temple as a senior systems engineer running WordPress at extremely large scale. My websites now run on WordPress, and I have the good sense to pay WP Engine to handle it for me, instead of doing it myself because it's not the most productive use of my time. I want things higher up the stack. I assure you I pay more to WP Engine than it would cost me to run these things myself from an infrastructure point of view, but not in terms of my time.What I see sometimes as the worst of all worlds is that AWS is trying to charge for that value-added pricing without adding the value that goes along with it because you still got to build a lot of this stuff yourself. It's still a very janky experience, you're reduced to googling random blog posts to figure out how this thing is supposed to work, and the best documentation comes from externally. Whereas with a company that's built around offering solutions like this, great. In the fullness of time, I really suspect that if this doesn't change, their customers are going to just be those people who build solutions out of these things. And let those companies capture the up-the-stack margin. Which I have no problem with. But they do because Amazon is a company that lies awake at night actively worrying that someone, somewhere, who isn't them might possibly be making money somehow.Pete: I think MongoDB is a perfect example of—like, look at their stock price over the last whatever, years. Like, they, I feel like everyone called for the death of MongoDB every time Amazon came out with their new things, yet, they're still a multi-billion dollar company because I can just—give me an API endpoint and you scale the database. There's is—Corey: Look at all the high-profile hires that Mongo was making out of AWS, and I can't shake the feeling they're sitting there going, “Yeah, who's losing important things out of production now?” It's, everyone is exodus-ing there. I did one of those ridiculous graphics of the naming all the people that went over there, and in—with the hurricane evacuation traffic picture, and there's one car going the other way that I just labeled with, “Re:Invent sponsorship check,” because yeah, they have a top tier sponsorship and it was great. I've got to say I've been pretty down on MongoDB for a while, for a variety of excellent reasons based upon, more or less, how they treated customers who were in pain. And I'd mostly written it off.I don't do that anymore. Not because I inherently believe the technology has changed, though I'm told it has, but by the number of people who I deeply respect who are going over there and telling me, no, no, this is good. Congratulations. I have often said you cannot buy authenticity, and I don't think that they are, but the people who are working there, I do not believe that these people are, “Yeah, well, you bought my opinion. You can buy their attention, not their opinion.” If someone changes their opinion, based upon where they work, I kind of question everything they're telling me is, like, “Oh, you're just here to sell something you don't believe in? Welcome aboard.”Pete: Right. Yeah, there's an interview question I like to ask, which is, “What's something that you used to believe in very strongly that you've more recently changed your mind on?” And out of politeness because usually throws people back a little bit, and they're like, “Oh, wow. Like, let me think about that.” And I'm like, “Okay, while you think about that I want to give you mine.”Which is in the past, my strongly held belief was we had to run everything ourselves. “You own your availability,” was the line. “No, I'm not buying Datadog. I can build my own metric stack just fine, thank you very much.” Like, “No, I'm not going to use these outsourced load balancers or databases because I need to own my availability.”And what I realized is that all of those decisions lead to actually delivering and focusing on things that were not the core product. And so now, like, I've really flipped 180, that, if any—anything that you're building that does not directly relate to the core product, i.e. How your business makes money, should one hundred percent be outsourced to an expert that is better than you. Mongo knows how to run Mongo better than you.Corey: “What does your company do?” “Oh, we handle expense reports.” “Oh, what are you working on this month?” “I'm building a load balancer.” It's like that doesn't add the value. Don't do that.Pete: Right. Exactly. And so it's so interesting, I think, to hear Werner say that, you know, we're just building primitives, and you asked for this. And I think that concept maybe would work years ago, when you had a lot of builders who needed tools, but I don't think we have any, like, we don't have as many builders as before. Like, I think we have people who need more complete solutions. And that's probably why all these businesses are being super successful against Amazon.Corey: I'm wondering if it comes down to a cloud economic story, specifically that my cloud bill is always going to be variable and it's difficult to predict, whereas if I just use EC2 instances, and I build load balancers or whatnot, myself, well, yeah, it's a lot more work, but I can predict accurately what my staff compensation costs are more effectively, that I can predict what a CapEx charge would be or what the AWS bill is going to be. I'm wondering if that might in some way shape it?Pete: Well, I feel like the how people get better in managing their costs, right, you'll eventually move to a world where, like, “Yep, okay, first, we turned off waste,” right? Like, step one is waste. Step two is, like, understanding your spend better to optimize but, like, step three, like, the galaxy brain meme of Amazon cost stuff is all, like, unit economics stuff, where trying to better understand the actual cost deliver an actual feature. And yeah, I think that actually gets really hard when you give—kind of spread your product across, like, a slew of services that have varying levels of costs, varying levels of tagging, so you can attribute it. Like, it's really hard. Honestly, it's pretty easy if I have 1000 EC2 servers with very specific tags, I can very easily figure out what it costs to deliver product. But if I have—Corey: Yeah, if I have Corey build it, I know what Corey is going to cost, and I know how many servers he's going to use. Great, if I have Pete it, Pete's good at things, it'll cut that server bill in half because he actually knows how to wind up being efficient with things. Okay, great. You can start calculating things out that way. I don't think that's an intentional choice that companies are making, but I feel like that might be a natural outgrowth of it.Pete: Yeah. And there's still I think a lot of the, like, old school mentality of, like, the, “Not invented here,” the, “We have to own our availability.” You can still own your availability by using these other vendors. And honestly, it's really heartening to see so many companies realize that and realize that I don't need to get everything from Amazon. And honestly, like, in some things, like I look at a cloud Amazon bill, and I think to myself, it would be easier if you just did everything from Amazon versus having these ten other vendors, but those ten other vendors are going to be a lot better at running the product that they build, right, that as a service, then you probably will be running it yourself. Or even Amazon's, like, you know, interpretation of that product.Corey: A few other things that came out that I thought were interesting, at least the direction they're going in. The changes to S3 intelligent tiering are great, with instant retrieval on Glacier. I feel like that honestly was—they talk a good story, but I feel like that was competitive response to Google offering the same thing. That smacks of a large company with its use case saying, “You got two choices here.” And they're like, “Well, okay. Crap. We're going to build it then.”Or alternately, they're looking at the changes that they're making to intelligent tiering, they're now shifting that to being the default that as far as recommendations go. There are a couple of drawbacks to it, but not many, and it's getting easier now to not have the mental overhead of trying to figure out exactly what your lifecycle policies are. Yeah, there are some corner cases where, okay, if I adjust this just so, then I could save 10% on that monitoring fee or whatnot. Yeah, but look how much work that's going to take you to curate and make sure that you're not doing something silly. That feels like it is such an in the margins issue. It's like, “How much data you're storing?” “Four exabytes.” Okay, yeah. You probably want some people doing exactly that, but that's not most of us.Pete: Right. Well, there's absolutely savings to be had. Like, if I had an exabyte of data on S3—which there are a lot of people who have that level of data—then it would make sense for me to have an engineering team whose sole purpose is purely an optimizing our data lifecycle for that data. Until a point, right? Until you've optimized the 80%, basically. You optimize the first 80, that's probably, air-quote, “Easy.” The last 20 is going to be incredibly hard, maybe you never even do that.But at lower levels of scale, I don't think the economics actually work out to have a team managing your data lifecycle of S3. But the fact that now AWS can largely do it for you in the background—now, there's so many things you have to think about and, like, you know, understand even what your data is there because, like, not all data is the same. And since S3 is basically like a big giant database you can query, you got to really think about some of that stuff. But honestly, what I—I don't know if—I have no idea if this is even be worked on, but what I would love to see—you know, hashtag #AWSwishlist—is, now we have countless tiers of EBS volumes, EBS volumes that can be dynamically modified without touching, you know, the physical host. Meaning with an API call, you can change from the gp2 to gp3, or io whatever, right?Corey: Or back again if it doesn't pan out.Pete: Or back again, right? And so for companies with large amounts of spend, you know, economics makes sense that you should have a team that is analyzing your volumes usage and modifying that daily, right? Like, you could modify that daily, and I don't know if there's anyone out there that's actually doing it at that level. And they probably should. Like, if you got millions of dollars in EBS, like, there's legit savings that you're probably leaving on the table without doing that. But that's what I'm waiting for Amazon to do for me, right? I want intelligent tiering for EBS because if you're telling me I can API call and you'll move my data and make that better, make that [crosstalk 00:17:46] better [crosstalk 00:17:47]—Corey: Yeah it could be like their auto-scaling for DynamoDB, for example. Gives you the capacity you need 20 minutes after you needed it. But fine, whatever because if I can schedule stuff like that, great, I know what time of day, the runs are going to kick off that beat up the disks. I know when end-of-month reporting fires off. I know what my usage pattern is going to be, by and large.Yeah, part of the problem too, is that I look at this stuff, and I get excited about it with the intelligent tiering… at The Duckbill Group we've got a few hundred S3 buckets lurking around. I'm thinking, “All right, I've got to go through and do some changes on this and implement all of that.” Our S3 bill's something like 50 bucks a month or something ridiculous like that. It's a no, that really isn't a thing. Like, I have a screenshot bucket that I have an app installed—I think called Dropshare—that hooks up to anytime I drag—I hit a shortcut, I drag with the mouse to select whatever I want and boom, it's up there and the URL is not copied to my clipboard, I can paste that wherever I want.And I'm thinking like, yeah, there's no cleanup on that. There's no lifecycle policy that's turning into anything. I should really go back and age some of it out and do the rest and start doing some lifecycle management. It—I've been using this thing for years and I think it's now a whopping, what, 20 cents a month for that bucket. It's—I just don't—Pete: [laugh].Corey: —I just don't care, other than voice in the back of my mind, “That's an unbounded growth problem.” Cool. When it hits 20 bucks a month, then I'll consider it. But until then I just don't. It does not matter.Pete: Yeah, I think yeah, scale changes everything. Start adding some zeros and percentages turned into meaningful numbers. And honestly, back on the EBS thing, the one thing that really changed my perspective of EBS, in general, is—especially coming from the early days, right? One terabyte volume, it was a hard drive in a thing. It was a virtual LUN on a SAN somewhere, probably.Nowadays, and even, like, many years after those original EBS volumes, like all the limits you get in EBS, those are actually artificial limits, right? If you're like, “My EBS volume is too slow,” it's not because, like, the hard drive it's on is too slow. That's an artificial limit that is likely put in place due to your volume choice. And so, like, once you realize that in your head, then your concept of how you store data on EBS should change dramatically.Corey: Oh, AWS had a blog post recently talking about, like, with io2 and the limits and everything, and there was architecture thinking, okay. “So, let's say this is insufficient and the quarter-million IOPS a second that you're able to get is not there.” And I'm sitting there thinking, “That is just ludicrous data volume and data interactivity model.” And it's one of those, like, I'm sitting here trying to think about, like, I haven't had to deal with a problem like that decade, just because it's, “Huh. Turns out getting these one thing that's super fast is kind of expensive.” If you paralyze it out, that's usually the right answer, and that's how the internet is mostly evolved. But there are use cases for which that doesn't work, and I'm excited to see it. I don't want to pay for it in my view, but it's nice to see it.Pete: Yeah, it's kind of fun to go into the Amazon calculator and price out one of the, like, io2 volumes and, like, maxed out. It's like, I don't know, like $50,000 a month or a hun—like, it's some just absolutely absurd number. But the beauty of it is that if you needed that value for an hour to run some intensive data processing task, you can have it for an hour and then just kill it when you're done, right? Like, that is what is most impressive.Corey: I copied 130 gigs of data to an EFS volume, which was—[unintelligible 00:21:05] EFS has gone from “This is a piece of junk,” to one of my favorite services. It really is, just because of its utility and different ways of doing things. I didn't have the foresight, just use a second EFS volume for this. So, I was unzipping a whole bunch of small files onto it. Great.It took a long time for me to go through it. All right, now that I'm done with that I want to clean all this up. My answer was to ultimately spin up a compute node and wind up running a whole bunch of—like, 400, simultaneous rm-rf on that long thing. And it was just, like, this feels foolish and dumb, but here we are. And I'm looking at the stats on it because the instance was—all right, at that point, the load average [on the instance 00:21:41] was like 200, or something like that, and the EFS volume was like, “Ohh, wow, you're really churning on this. I'm now at, like, 5% of the limit.” Like, okay, great. It turns out I'm really bad at computers.Pete: Yeah, well, that's really the trick is, like, yeah, sure, you can have a quarter-million IOPS per second, but, like, what's going to break before you even hit that limit? Probably many other things.Corey: Oh, yeah. Like, feels like on some level if something gets to that point, it a misconfiguration somewhere. But honestly, that's the thing I find weirdest about the world in which we live is that at a small-scale—if I have a bill in my $5 a month shitposting account, great. If I screw something up and cost myself a couple hundred bucks in misconfiguration it's going to stand out. At large scale, it doesn't matter if—you're spending $50 million a year or $500 million a year on AWS and someone leaks your creds, and someone spins up a whole bunch of Bitcoin miners somewhere else, you're going to see that on your bill until they're mining basically all the Bitcoin. It just gets lost in the background.Pete: I'm waiting for those—I'm actually waiting for the next level of them to get smarter because maybe you have, like, an aggressive tagging system and you're monitoring for untagged instances, but the move here would be, first get the creds and query for, like, the most used tags and start applying those tags to your Bitcoin mining instances. My God, it'll take—Corey: Just clone a bunch of tags. Congratulations, you now have a second BI Elasticsearch cluster that you're running yourself. Good work.Pete: Yeah. Yeah, that people won't find that until someone comes along after the fact that. Like, “Why do we have two have these things?” And you're like—[laugh].Corey: “Must be a DR thing.”Pete: It's maxed-out CPU. Yeah, exactly.Corey: [laugh].Pete: Oh, the terrible ideas—please, please, hackers don't take are terrible ideas.Corey: I had a, kind of, whole thing I did on Twitter years ago, talking about how I would wind up using the AWS Marketplace for an embezzlement scheme. Namely, I would just wind up spinning up something that had, like, a five-cent an hour charge or whatnot on just, like, basically rebadge the CentOS Community AMI or whatnot. Great. And then write a blog post, not attached to me, that explains how to do a thing that I'm going to be doing in production in a week or two anyway. Like, “How to build an auto-scaling group,” and reference that AMI.Then if it ever comes out, like, “Wow, why are we having all these marketplace charges on this?” “I just followed the blog post like it said here.” And it's like, “Oh, okay. You're a dumbass. The end.”That's the way to do it. A month goes by and suddenly it came out that someone had done something similarly. They wound up rebadging these community things on the marketplace and charging big money for it, and I'm sitting there going like that was a joke. It wasn't a how-to. But yeah, every time I make these jokes, I worry someone's going to do it.Pete: “Welcome to large-scale fraud with Corey Quinn.”Corey: Oh, yeah, it's fraud at scale is really the important thing here.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance accelerator for the Oracle MySQL Database Service. Although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLTP and OLAP, don't ask me to ever say those acronyms again, workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: I still remember a year ago now at re:Invent 2021 was it, or was it 2020? Whatever they came out with, I want to say it wasn't gp3, or maybe it was, regardless, there was a new EBS volume type that came out that you were playing with to see how it worked and you experimented with it—Pete: Oh, yes.Corey: —and the next morning, you looked at the—I checked Slack and you're like well, my experiments yesterday cost us $5,000. And at first, like, the—my response is instructive on this because, first, it was, “Oh, my God. What's going to happen now?” And it's like, first, hang on a second.First off, that seems suspect but assume it's real. I assumed it was real at the outset. It's “Oh, right. This is not my personal $5-a-month toybox account. We are a company; we can absolutely pay that.” Because it's like, I could absolutely reach out, call it a favor. “I made a mistake, and I need a favor on the bill, please,” to AWS.And I would never live it down, let's be clear. For a $7,000 mistake, I would almost certainly eat it. As opposed to having to prostrate myself like that in front of Amazon. I'm like, no, no, no. I want one of those like—if it's like, “Okay, you're going to, like, set back the company roadmap by six months if you have to pay this. Do you want to do it?” Like, [groans] “Fine, I'll eat some crow.”But okay. And then followed immediately by, wow, if Pete of all people can mess this up, customers are going to be doomed here. We should figure out what happened. And I'm doing the math. Like, Pete, “What did you actually do?” And you're sitting there and you're saying, “Well, I had like a 20 gig volume that I did this.” And I'm doing the numbers, and it's like—Pete: Something's wrong.Corey: “How sure are you when you say ‘gigabyte,' that you were—that actually means what you think it did? Like, were you off by a lot? Like, did you mean exabytes?” Like, what's the deal here?Pete: Like, multiple factors.Corey: Yeah. How much—“How many IOPS did you give that thing, buddy?” And it turned out what happened was that when they launched this, they had mispriced it in the system by a factor of a million. So, it was fun. I think by the end of it, all of your experimentation was somewhere between five to seven cents. Which—Pete: Yeah. It was a—Corey: Which is why you don't work here anymore because no one cost me seven cents of money to give to Amazon—Pete: How dare you?Corey: —on my watch. Get out.Pete: How dare you, sir?Corey: Exactly.Pete: Yeah, that [laugh] was amazing to see, as someone who has done—definitely maid screw-ups that have cost real money—you know, S3 list requests are always a fun one at scale—but that one was supremely fun to see the—Corey: That was a scary one because another one they'd done previously was they had messed up Lightsail pricing, where people would log in, and, like, “Okay, so what is my Lightsail instance going to cost?” And I swear to you, this is true, it was saying—this was back in 2017 or so—the answer was, like, “$4.3 billion.” Because when you see that you just start laughing because you know it's a mistake. You know, that they're not going to actually demand that you spend $4.3 billion for a single instance—unless it's running SAP—and great.It's just, it's a laugh. It's clearly a mispriced, and it's clearly a bug that's going to get—it's going to get fixed. I just spun up this new EBS volume that no one fully understands yet and it cost me thousands of dollars. That's the sort of thing that no, no, I could actually see that happening. There are instances now that cost something like 100 bucks an hour or whatnot to run. I can see spinning up the wrong thing by mistake and getting bitten by it. There's a bunch of fun configuration mistakes you can make that will, “Hee, hee, hee. Why can I see that bill spike from orbit?” And that's the scary thing.Pete: Well, it's the original CI and CD problem of the per-hour billing, right? That was super common of, like, yeah, like, an i3, you know, 16XL server is pretty cheap per hour, but if you're charged per hour and you spin up a bunch for five minutes. Like, it—you will be shocked [laugh] by what you see there. So—Corey: Yeah. Mistakes will show. And I get it. It's also people as individuals are very different psychologically than companies are. With companies it's one of those, “Great we're optimizing to bring in more revenue and we don't really care about saving money at all costs.”Whereas people generally have something that looks a lot like a fixed income in the form of a salary or whatnot, so it's it is easier for us to cut spend than it is for us to go out and make more money. Like, I don't want to get a second job, or pitch my boss on stuff, and yeah. So, all and all, routing out the rest of what happened at re:Invent, they—this is the problem is that they have a bunch of minor things like SageMaker Inference Recommender. Yeah, I don't care. Anything—Pete: [laugh].Corey: —[crosstalk 00:28:47] SageMaker I mostly tend to ignore, for safety. I did like the way they described Amplify Studio because they made it sound like a WYSIWYG drag and drop, build a React app. It's not it. It basically—you can do that in Figma and then it can hook it up to some things in some cases. It's not what I want it to be, which is Honeycode, except good. But we'll get there some year. Maybe.Pete: There's a lot of stuff that was—you know, it's the classic, like, preview, which sure, like, from a product standpoint, it's great. You know, they have a level of scale where they can say, “Here's this thing we're building,” which could be just a twinkle in a product managers, call it preview, and get thousands of people who would be happy to test it out and give you feedback, and it's a, it's great that you have that capability. But I often look at so much stuff and, like, that's really cool, but, like, can I, can I have it now? Right? Like—or you can't even get into the preview plan, even though, like, you have that specific problem. And it's largely just because either, like, your scale isn't big enough, or you don't have a good enough relationship with your account manager, or I don't know, countless other reasons.Corey: The thing that really throws me, too, is the pre-announcements that come a year or so in advance, like, the Outpost smaller ones are finally available, but it feels like when they do too many pre-announcements or no big marquee service announcements, as much as they talk about, “We're getting back to fundamentals,” no, you have a bunch of teams that blew the deadline. That's really what it is; let's not call it anything else. Another one that I think is causing trouble for folks—I'm fortunate in that I don't do much work with Oracle databases, or Microsoft SQL databases—but they extended RDS Custom to Microsoft SQL at the [unintelligible 00:30:27] SQL server at re:Invent this year, which means this comes down to things I actually use, we're going to have a problem because historically, the lesson has always been if I want to run my own databases and tweak everything, I do it on top of an EC2 instance. If I want to managed database, relational database service, great, I use RDS. RDS Custom basically gives you root into the RDS instance. Which means among other things, yes, you can now use RDS to run containers.But it lets you do a lot of things that are right in between. So, how do you position this? When should I use RDS Custom? Can you give me an easy answer to that question? And they used a lot of words to say, no, they cannot. It's basically completely blowing apart the messaging and positioning of both of those services in some unfortunate ways. We'll learn as we go.Pete: Yeah. Honestly, it's like why, like, why would I use this? Or how would I use this? And this is I think, fundamentally, what's hard when you just say yes to everything. It's like, they in many cases, I don't think, like, I don't want to say they don't understand why they're doing this, but if it's not like there's a visionary who's like, this fits into this multi-year roadmap.That roadmap is largely—if that roadmap is largely generated by the customers asking for it, then it's not like, oh, we're building towards this Northstar of RDS being whatever. You might say that, but your roadmap's probably getting moved all over the place because, you know, this company that pays you a billion dollars a year is saying, “I would give you $2 billion a year for all of my Oracle databases, but I need this specific thing.” I can't imagine a scenario that they would say, “Oh, well, we're building towards this Northstar, and that's not on the way there.” Right? They'd be like, “New Northstar. Another billion dollars, please.”Corey: Yep. Probably the worst release of re:Invent, from my perspective, is RUM, Real User Monitoring, for CloudWatch. And I, to be clear, I wrote a shitposting Twitter threading client called Last Tweet in AWS. Go to lasttweetinaws.com. You can all use it. It's free; I just built this for my own purposes. And I've instrumented it with RUM. Now, Real User Monitoring is something that a lot of monitoring vendors use, and also CloudWatch now. And what that is, is it embeds a listener into the JavaScript that runs on client load, and it winds up looking at what's going on loading times, et cetera, so you can see when users are unhappy. I have no problem with this. Other than that, you know, liking users? What's up with that?Pete: Crazy.Corey: But then, okay, now, what this does is unlike every other RUM tool out there, which charges per session, meaning I am going to be… doing a web page load, it charges per data item, which includes HTTP errors, or JavaScript errors, et cetera. Which means that if you have a high transaction volume site and suddenly your CDN takes a nap like Fastly did for an hour last year, suddenly your bill is stratospheric for this because errors abound and cascade, and you can have thousands of errors on a single page load for these things, and it is going to be visible from orbit, at least with a per session basis thing, when you start to go viral, you understand that, “Okay, this is probably going to cost me some more on these things, and oops, I guess I should write less compelling content.” Fine. This is one of those one misconfiguration away and you are wailing and gnashing teeth. Now, this is a new service. I believe that they will waive these surprise bills in the event that things like that happen. But it's going to take a while and you're going to be worrying the whole time if you've rolled this out naively. So it's—Pete: Well and—Corey: —I just don't like the pricing.Pete: —how many people will actively avoid that service, right? And honestly, choose a competitor because the competitor could be—the competitor could be five times more expensive, right, on face value, but it's the certainty of it. It's the uncertainty of what Amazon will charge you. Like, no one wants a surprise bill. “Well, a vendor is saying that they'll give us this contract for $10,000. I'm going to pay $10,000, even though RUM might be a fraction of that price.”It's honestly, a lot of these, like, product analytics tools and monitoring tools, you'll often see they price be a, like, you know, MAU, Monthly Active User, you know, or some sort of user-based pricing, like, the number of people coming to your site. You know, and I feel like at least then, if you are trying to optimize for lots of users on your site, and more users means more revenue, then you know, if your spend is going up, but your revenue is also going up, that's a win-win. But if it's like someone—you know, your third-party vendor dies and you're spewing out errors, or someone, you know, upgraded something and it spews out errors. That no one would normally see; that's the thing. Like, unless you're popping open that JavaScript console, you're not seeing any of those errors, yet somehow it's like directly impacting your bottom line? Like that doesn't feel [crosstalk 00:35:06].Corey: Well, there is something vaguely Machiavellian about that. Like, “How do I get my developers to care about errors on consoles?” Like, how about we make it extortionately expensive for them not to. It's, “Oh, all right, then. Here we go.”Pete: And then talk about now you're in a scenario where you're working on things that don't directly impact the product. You're basically just sweeping up the floor and then trying to remove errors that maybe don't actually affect it and they're not actually an error.Corey: Yeah. I really do wonder what the right answer is going to be. We'll find out. Again, we live, we learn. But it's also, how long does it take a service that has bad pricing at launch, or an unfortunate story around it to outrun that reputation?People are still scared of Glacier because of its original restore pricing, which was non-deterministic for any sensible human being, and in some cases lead to I'm used to spending 20 to 30 bucks a month on this. Why was I just charged two grand?Pete: Right.Corey: Scare people like that, they don't come back.Pete: I'm trying to actually remember which service it is that basically gave you an estimate, right? Like, turn it on for a month, and it would give you an estimate of how much this was going to cost you when billing started.Corey: It was either Detective or GuardDuty.Pete: Yeah, it was—yeah, that's exactly right. It was one of those two. And honestly, that was unbelievably refreshing to see. You know, like, listen, you have the data, Amazon. You know what this is going to cost me, so when I, like, don't make me spend all this time to go and figure out the cost. If you have all this data already, just tell me, right?And if I look at it and go, “Yeah, wow. Like, turning this on in my environment is going to cost me X dollars. Like, yeah, that's a trade-off I want to make, I'll spend that.” But you know, with some of the—and that—a little bit of a worry on some of the intelligent tiering on S3 is that the recommendation is likely going to be everything goes to intelligent tiering first, right? It's the gp3 story. Put everything on gp3, then move it to the proper volume, move it to an sc or an st or an io. Like, gp3 is where you start. And I wonder if that's going to be [crosstalk 00:37:08].Corey: Except I went through a wizard yesterday to launch an EC2 instance and its default on the free tier gp2.Pete: Yeah. Interesting.Corey: Which does not thrill me. I also still don't understand for the life of me why in some regions, the free tier is a t2 instance, when t3 is available.Pete: They're uh… my guess is that they've got some free t—they got a bunch of t2s lying around. [laugh].Corey: Well, one of the most notable announcements at re:Invent that most people didn't pay attention to is their ability now to run legacy instance types on top of Nitro, which really speaks to what's going on behind the scenes of we can get rid of all that old hardware and emulate the old m1 on modern equipment. So, because—you can still have that legacy, ancient instance, but now you're going—now we're able to wind up greening our data centers, which is part of their big sustainability push, with their ‘Sustainability Pillar' for the well-architected framework. They're talking more about what the green choices in cloud are. Which is super handy, not just because of the economic impact because we could use this pretty directly to reverse engineer their various margins on a per-service or per-offering basis. Which I'm not sure they're aware of yet, but oh, they're going to be.And that really winds up being a win for the planet, obviously, but also something that is—that I guess puts a little bit of choice on customers. The challenge I've got is, with my serverless stuff that I build out, if I spend—the Google search I make to figure out what the most economic, most sustainable way to do that is, is going to have a bigger carbon impact on the app itself. That seems to be something that is important at scale, but if you're not at scale, it's one of those, don't worry about it. Because let's face it, the cloud providers—all of them—are going to have a better sustainability story than you are running this in your own data centers, or on a Raspberry Pi that's always plugged into the wall.Pete: Yeah, I mean, you got to remember, Amazon builds their own power plants to power their data centers. Like, that's the level they play, right? There, their economies of scale are so entirely—they're so entirely different than anything that you could possibly even imagine. So, it's something that, like, I'm sure people will want to choose for. But, you know, if I would honestly say, like, if we really cared about our computing costs and the carbon footprint of it, I would love to actually know the carbon footprint of all of the JavaScript trackers that when I go to various news sites, and it loads, you know, the whatever thousands of trackers and tracking the all over, like, what is the carbon impact of some of those choices that I actually could control, like, as a either a consumer or business person?Corey: I really hope that it turns into something that makes a meaningful difference, and it's not just greenwashing. But we'll see. In the fullness of time, we're going to figure that out. Oh, they're also launching some mainframe stuff. They—like that's great.Pete: Yeah, those are still a thing.Corey: I don't deal with a lot of customers that are doing things with that in any meaningful sense. There is no AWS/400, so all right.Pete: [laugh]. Yeah, I think honestly, like, I did talk to a friend of mine who's in a big old enterprise and has a mainframe, and they're actually replacing their mainframe with Lambda. Like they're peeling off—which is, like, a great move—taking the monolith, right, and peeling off the individual components of what it can do into these discrete Lambda functions. Which I thought was really fascinating. Again, it's a five-year-long journey to do something like that. And not everyone wants to wait five years, especially if their support's about to run out for that giant box in the, you know, giant warehouse.Corey: The thing that I also noticed—and this is probably the—I guess, one of the—talk about swing and a miss on pricing—they have a—what is it?—there's a VPC IP Address Manager, which tracks the the IP addresses assigned to your VPCs that are allocated versus not, and it's 20 cents a month per IP address. It's like, “Okay. So, you're competing against a Google Sheet or an Excel spreadsheet”—which is what people are using for these things now—“Only you're making it extortionately expensive?”Pete: What kind of value does that provide for 20—I mean, like, again—Corey: I think Infoblox or someone like that offers it where they become more cost-effective as soon as you hit 500 IP addresses. And it's just—like, this is what I'm talking about. I know it does not cost AWS that kind of money to store an IP address. You can store that in a Route 53 TXT record for less money, for God's sake. And that's one of those, like, “Ah, we could extract some value pricing here.”Like, I don't know if it's a good product or not. Given its pricing, I don't give a shit because it's going to be too expensive for anything beyond trivial usage. So, it's a swing and a miss from that perspective. It's just, looking at that, I laugh, and I don't look at it again.Pete: See I feel—Corey: I'm not usually price sensitive. I want to be clear on that. It's just, that is just Looney Tunes, clown shoes pricing.Pete: Yeah. It's honestly, like, in many cases, I think the thing that I have seen, you know, in the past few years is, in many cases, it can honestly feel like Amazon is nickel-and-diming their customers in so many ways. You know, the explosion of making it easy to create multiple Amazon accounts has a direct impact to waste in the cloud because there's a lot of stuff you have to have her account. And the more accounts you have, those costs grow exponentially as you have these different places. Like, you kind of lose out on the economies of scale when you have a smaller number of accounts.And yeah, it's hard to optimize for that. Like, if you're trying to reduce your spend, it's challenging to say, “Well, by making a change here, we'll save, you know, $10,000 in this account.” “That doesn't seem like a lot when we're spending millions.” “Well, hold on a second. You'll save $10,000 per account, and you have 500 accounts,” or, “You have 1000 accounts,” or something like that.Or almost cost avoidance of this cost is growing unbounded in all of your accounts. It's tiny right now. So, like, now would be the time you want to do something with it. But like, again, for a lot of companies that have adopted the practice of endless Amazon accounts, they've almost gone, like, it's the classic, like, you know, I've got 8000 GitHub repositories for my source code. Like, that feels just as bad as having one GitHub repository for your repo. I don't know what the balance is there, but anytime these different types of services come out, it feels like, “Oh, wow. Like, I'm going to get nickeled and dimed for it.”Corey: This ties into the re:Post launch, which is a rebranding of their forums, where, okay, great, it was a little crufty and it need modernize, but it still ties your identity to an IAM account, or the root email address for an Amazon account, which is great. This is completely worthless because as soon as I change jobs, I lose my identity, my history, the rest, on this forum. I'm not using it. It shows that there's a lack of awareness that everyone is going to have multiple accounts with which they interact, and that people are going to deal with the platform longer than any individual account will. It's just a continual swing and a miss on things like that.And it gets back to the billing question of, “Okay. When I spin up an account, do I want them to just continue billing me—because don't turn this off; this is important—or do I want there to be a hard boundary where if you're about to charge me, turn it off. Turn off the thing that's about to cost me money.” And people hem and haw like this is an insurmountable problem, but I think the way to solve it is, let me specify that intent when I provision the account. Where it's, “This is a production account for a bank. I really don't want you turning it off.” Versus, “I'm a student learner who thinks that a Managed NAT Gateway might be a good thing. Yeah, I want you to turn off my demo Hello World app that will teach me what's going on, rather than surprising me with a five-figure bill at the end of the month.”Pete: Yeah. It shouldn't be that hard. I mean, but again, I guess everything's hard at scale.Corey: Oh, yeah. Oh yeah.Pete: But still, I feel like every time I log into Cost Explorer and I look at—and this is years it's still not fixed. Not that it's even possible to fix—but on the first day of the month, you look at Cost Explorer, and look at what Amazon is estimating your monthly bill is going to be. It's like because of your, you know—Corey: Your support fees, and your RI purchases, and savings plans purchases.Pete: [laugh]. All those things happened, right? First of the month, and it's like, yeah, “Your bill's going to be $800,000 this year.” And it's like, “Shouldn't be, like, $1,000?” Like, you know, it's the little things like that, that always—Corey: The one-off charges, like, “Oh, your Route 53 zone,” and all the stuff that gets charged on a monthly cadence, which fine, whatever. I mean, I'm okay with it, but it's also the, like, be careful when that happen—I feel like there's a way to make that user experience less jarring.Pete: Yeah because that problem—I mean, in my scenario, companies that I've worked at, there's been multiple times that a non-technical person will look at that data and go into immediate freakout mode, right? And that's never something that you want to have happen because now that's just adding a lot of stress and anxiety into a company that is—with inaccurate data. Like, the data—like, the answer you're giving someone is just wrong. Perhaps you shouldn't even give it to them if it's that wrong. [laugh].Corey: Yeah, I'm looking forward to seeing what happens this coming year. We're already seeing promising stuff. They—give people a timeline on how long in advance these things record—late last night, AWS released a new console experience. When you log into the AWS console now, there's a new beta thing. And I gave it some grief on Twitter because I'm still me, but like the direction it's going. It lets you customize your view with widgets and whatnot.And until they start selling widgets on marketplace or having sponsored widgets, you can't remove I like it, which is no guarantee at some point. But it shows things like, I can move the cost stuff, I can move the outage stuff up around, I can have the things that are going on in my account—but who I am means I can shift this around. If I'm a finance manager, cool. I can remove all the stuff that's like, “Hey, you want to get started spinning up an EC2 instance?” “Absolutely not. Do I want to get told, like, how to get certified? Probably not. Do I want to know what the current bill is and whether—and my list of favorites that I've pinned, whatever services there? Yeah, absolutely do.” This is starting to get there.Pete: Yeah, I wonder if it really is a way to start almost hedging on organizations having a wider group of people accessing AWS. I mean, in previous companies, I absolutely gave access to the console for tools like QuickSight, for tools like Athena, for the DataBrew stuff, the Glue DataBrew. Giving, you know, non-technical people access to be able to do these, like, you know, UI ETL tasks, you know, a wider group of a company is getting access into Amazon. So, I think anything that Amazon does to improve that experience for, you know, the non-SREs, like the people who would traditionally log in, like, that is an investment definitely worth making.Corey: “Well, what could non-engineering types possibly be doing in the AWS console?” “I don't know, jackhole, maybe paying the bill? Just a thought here.” It's the, there are people who look at these things from a variety of different places, and you have such sprawl in the AWS world that there are different personas by a landslide. If I'm building Twitter for Pets, you probably don't want to be pitching your mainframe migration services to me the same way that you would if I were a 200-year-old insurance company.Pete: Yeah, exactly. And the number of those products are going to grow, the number of personas are going to grow, and, yeah, they'll have to do something that they want to actually, you know, maintain that experience so that every person can have, kind of, the experience that they want, and not be distracted, you know? “Oh, what's this? Let me go test this out.” And it's like, you know, one-time charge for $10,000 because, like, that's how it's charged. You know, that's not an experience that people like.Corey: No. They really don't. Pete, I want to thank you for spending the time to chat with me again, as is our tradition. I'm hoping we can do it in person this year, when we go at the end of 2022, to re:Invent again. Or that no one goes in person. But this hybrid nonsense is for the birds.Pete: Yeah. I very much would love to get back to another one, and yeah, like, I think there could be an interesting kind of merging here of our annual re:Invent recap slash live brunch, you know, stream you know, hot takes after a long week. [laugh].Corey: Oh, yeah. The real way that you know that it's a good joke is when one of us says something, the other one sprays scrambled eggs out of their nose. Yeah, that's the way to do it.Pete: Exactly. Exactly.Corey: Pete, thank you so much. If people want to learn more about what you're up to—hopefully, you know, come back. We miss you, but you're unaffiliated, you're a startup advisor. Where can people find you to learn more, if they for some unforgivable reason don't know who or what a Pete Cheslock is?Pete: Yeah. I think the easiest place to find me is always on Twitter. I'm just at @petecheslock. My DMs are always open and I'm always down to expand my network and chat with folks.And yeah, right, now, I'm just, as I jokingly say, professionally unaffiliated. I do some startup advisory work and have been largely just kind of—honestly checking out the state of the economy. Like, there's a lot of really interesting companies out there, and some interesting problems to solve. And, you know, trying to spend some of my time learning more about what companies are up to nowadays. So yeah, if you got some interesting problems, you know, you can follow my Twitter or go to LinkedIn if you want some great, you know, business hot takes about, you know, shitposting basically.Corey: Same thing. Pete, thanks so much for joining me, I appreciate it.Pete: Thanks for having me.Corey: Pete Cheslock, startup advisor, professionally unaffiliated, and recurring re:Invent analyst pal of mine. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment calling me a jackass because do I know how long it took you personally to price CloudWatch RUM?Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

About MattMatt is an AWS DevTools Hero, Serverless Architect, Author and conference speaker. He is focused on creating the right environment for empowered teams to rapidly deliver business value in a well-architected, sustainable and serverless-first way.You can usually find him sharing reusable, well architected, serverless patterns over at cdkpatterns.com or behind the scenes bringing CDK Day to life.Links: AWS CDK Patterns: https://cdkpatterns.com The CDK Book: https://thecdkbook.com CDK Day: https://www.cdkday.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: It seems like there is a new security breach every day. Are you confident that an old SSH key, or a shared admin account, isn't going to come back and bite you? If not, check out Teleport. Teleport is the easiest, most secure way to access all of your infrastructure. The open source Teleport Access Plane consolidates everything you need for secure access to your Linux and Windows servers—and I assure you there is no third option there. Kubernetes clusters, databases, and internal applications like AWS Management Console, Yankins, GitLab, Grafana, Jupyter Notebooks, and more. Teleport's unique approach is not only more secure, it also improves developer productivity. To learn more visit: goteleport.com. And not, that is not me telling you to go away, it is: goteleport.com.Corey: This episode is sponsored in part by our friends at Rising Cloud, which I hadn't heard of before, but they're doing something vaguely interesting here. They are using AI, which is usually where my eyes glaze over and I lose attention, but they're using it to help developers be more efficient by reducing repetitive tasks. So, the idea being that you can run stateless things without having to worry about scaling, placement, et cetera, and the rest. They claim significant cost savings, and they're able to wind up taking what you're running as it is in AWS with no changes, and run it inside of their data centers that span multiple regions. I'm somewhat skeptical, but their customers seem to really like them, so that's one of those areas where I really have a hard time being too snarky about it because when you solve a customer's problem and they get out there in public and say, “We're solving a problem,” it's very hard to snark about that. Multus Medical, Construx.ai and Stax have seen significant results by using them. And it's worth exploring. So, if you're looking for a smarter, faster, cheaper alternative to EC2, Lambda, or batch, consider checking them out. Visit risingcloud.com/benefits. That's risingcloud.com/benefits, and be sure to tell them that I said you because watching people wince when you mention my name is one of the guilty pleasures of listening to this podcast.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I'm joined today by Matt Coulter, who is a Technical Architect at Liberty Mutual. You may have had the privilege of seeing him on the keynote stage at re:Invent last year—in Las Vegas or remotely—that last year of course being 2021. But if you make better choices than the two of us did, and found yourself not there, take the chance to go and watch that keynote. It's really worth seeing.Matt, first, thank you for joining me. I'm sorry, I don't have 20,000 people here in the audience to clap this time. They're here, but they're all remote as opposed to sitting in the room behind me because you know, social distancing.Matt: And this left earphone, I just have some applause going, just permanently, just to keep me going. [laugh].Corey: That's sort of my own internal laugh track going on. It's basically whatever I say is hilarious, to that. So yeah, doesn't really matter what I say, how I say it, my jokes are all for me. It's fine. So, what was it like being on stage in front of that many people? It's always been a wild experience to watch and for folks who haven't spent time on the speaking circuit, I don't think that there's any real conception of what that's like. Is this like giving a talk at work, where I just walk on stage randomly, whatever I happened to be wearing? And, oh, here's a microphone, I'm going to say words. What is the process there?Matt: It's completely different. For context for everyone, before the pandemic, I would have pretty regularly talked in front of, I don't know, maybe one, two hundred people in Liberty, in Belfast. So, I used to be able to just, sort of, walk in front of them, and lean against the pillar, and use my clicker, and click through, but the process for actually presenting something as big as a keynote and re:Invent is so different. For starters, you think that when you walk onto the stage, you'll actually be able to see the audience, but the way the lights are set up, you can pretty much see about one row of people, and they're not the front row, so anybody I knew, I couldn't actually see.And yeah, you can only see, sort of like, the from the void, and then you have your screens, so you've six sets of screens that tell you your notes as well as what slides you're on, you know, so you can pivot. But other than that, I mean, it feels like you're just talking to yourself outside of whenever people, thankfully, applause. It's such a long process to get there.Corey: I've always said that there are a few different transition stages as the audience size increases, but for me, the final stage is more or less anything above 750 people. Because as you say, you aren't able to see that many beyond that point, and it doesn't really change anything meaningfully. The most common example that you see in the wild is jokes that work super well with a small group of people fall completely flat to large audiences. It's why so much corporate numerous cheesy because yeah, everyone in the rehearsals is sitting there laughing and the joke kills, but now you've got 5000 people sitting in a room and that joke just sounds strained and forced because there's no longer a conversation, and no one has the shared context that—the humor has to change. So, in some cases when you're telling a story about what you're going to say on stage, during a rehearsal, they're going to say, “Well, that joke sounds really corny and lame.” It's, “Yeah, wait until you see it in front of an audience. It will land very differently.” And I'm usually right on that.I would also advise, you know, doing what you do and having something important and useful to say, as opposed to just going up there to tell jokes the whole time. I wanted to talk about that because you talked about how you're using various CDK and other serverless style patterns in your work at Liberty Mutual.Matt: Yeah. So, we've been using CDK pretty extensively since it was, sort of, Q3 2019. At that point, it was new. Like, it had just gone GA at the time, just came out of dev preview. And we've been using CDK from the perspective of we want to be building serverless-first, well-architected apps, and ideally we want to be building them on AWS.Now, the thing is, we have 5000 people in our IT organization, so there's sort of a couple of ways you can take to try and get those people onto the cloud: You can either go the route of being, like, there is one true path to architecture, this is our architecture and everything you want to build can fit into that square box; or you can go the other approach and try and have the golden path where you say this is the paved road that is really easy to do, but if you want to differentiate from that route, that's okay. But what you need to do is feed back into the golden path if that works. Then everybody can improve. And that's where we've started been using CDK. So, what you heard me talk about was the software accelerator, and it's sort of a different approach.It's where anybody can build a pattern and then share it so that everybody else can rapidly, you know, just reuse it. And what that means is effectively you can, instead of having to have hundreds of people on a central team, you can actually just crowdsource, and sort of decentralize the function. And if things are good, then a small team can actually come in and audit them, so to speak, and check that it's well-architected, and doesn't have flaws, and drive things that way.Corey: I have to confess that I view the CDK as sort of a third stage automation approach, and it's one that I haven't done much work with myself. The first stage is clicking around in the console; the second is using CloudFormation or Terraform; the third stage is what we're talking about here is CDK or Pulumi, or something like that. And then you ascend to the final fourth stage, which is what I use, which is clicking around in the AWS console, but then you lie to people about it. ClickOps is poised to take over the world. But that's okay. You haven't gotten that far yet. Instead, you're on the CDK side. What advantages does CDK offer that effectively CloudFormation or something like it doesn't?Matt: So, first off, for ClickOps in Liberty, we actually have the AWS console as read-only in all of our accounts, except for sandbox. So, you can ClickOps in sandbox to learn, but if you want to do something real, unfortunately, it's going to fail you. So.—Corey: I love that pattern. I think I might steal that.Matt: [laugh]. So, originally, we went heavy on CloudFormation, which is why CDK worked well for us. And because we've actually—it's been a long journey. I mean, we've been deploying—2014, I think it was, we first started deploying to AWS, and we've used everything from Terraform, to you name it. We've built our own tools, believe it or not, that are basically CDK.And the thing about CloudFormation is, it's brilliant, but it's also incredibly verbose and long because you need to specify absolutely everything that you want to deploy, and every piece of configuration. And that's fine if you're just deploying a side project, but if you're in an enterprise that has responsibilities to protect user data, and you can't just deploy anything, they end up thousands and thousands and thousands of lines long. And then we have amazing guardrails, so if you tried to deploy a CloudFormation template with a flaw in it, we can either just fix it, or reject the deploy. But CloudFormation is not known to be the fastest to deploy, so you end up in this developer cycle, where you build this template by hand, and then it goes through that CloudFormation deploy, and then you get the failure message that it didn't deploy because of some compliance thing, and developers just got frustrated, and were like, sod this. [laugh].I'm not deploying to AWS. Back the on-prem. And that's where CDK was a bit different because it allowed us to actually build abstractions with all of our guardrails baked in, so that it just looked like a standard class, for developers, like, developers already know Java, Python, TypeScript, the languages off CDK, and so we were able to just make it easy by saying, “You want API Gateway? There's an API Gateway class. You want, I don't know, an EC2 instance? There you go.” And that way, developers could focus on the thing they wanted, instead of all of the compliance stuff that they needed to care about every time they wanted to deploy.Corey: Personally, I keep lobbying AWS to add my preferred language, which is crappy shell scripting, but for some reason they haven't really been quick to add that one in. The thing that I think surprises me, on some level—though, perhaps it shouldn't—is not just the adoption of serverless that you're driving at Liberty Mutual, but the way that you're interacting with that feels very futuristic, for lack of a better term. And please don't think that I'm in any way describing this in a way that's designed to be insulting, but I do a bunch of serverless nonsense on Twitter for Pets. That's not an exaggeration. twitterforpets.com has a bunch of serverless stuff behind it because you know, I have personality defects.But no one cares about that static site that's been a slide dump a couple of times for me, and a running joke. You're at Liberty Mutual; you're an insurance company. When people wind up talking about big enterprise institutions, you're sort of a shorthand example of exactly what they're talking about. It's easy to contextualize or think of that as being very risk averse—for obvious reasons; you are an insurance company—as well as wanting to move relatively slowly with respect to technological advancement because mistakes are going to have drastic consequences to all of your customers, people's lives, et cetera, as opposed to tweets or—barks—not showing up appropriately at the right time. How did you get to the, I guess, advanced architectural philosophy that you clearly have been embracing as a company, while having to be respectful of the risk inherent that comes with change, especially in large, complex environments?Matt: Yeah, it's funny because so for everyone, we were talking before this recording started about, I've been with Liberty since 2011. So, I've seen a lot of change in the length of time I've been here. And I've built everything from IBM applications right the way through to the modern serverless apps. But the interesting thing is, the journey to where we are today definitely started eight or nine years ago, at a minimum because there was something identified in the leadership that they said, “Listen, we're all about our customers. And that means we don't want to be wasting millions of dollars, and thousands of hours, and big trains of people to build software that does stuff. We want to focus on why are we building a piece of software, and how quickly can we get there? If you focus on those two things you're doing all right.”And that's why starting from the early days, we focused on things like, okay, everything needs to go through CI/CD pipelines. You need to have your infrastructure as code. And even if you're deploying on-prem, you're still going to be using the same standards that we use to deploy to AWS today. So, we had years and years and years of just baking good development practices into the company. And then whenever we started to move to AWS, the question became, do we want to just deploy the same thing or do we want to take full advantage of what the cloud has to offer? And I think because we were primed and because the leadership had the right direction, you know, we were just sitting there ready to say, “Okay, serverless seems like a way we can rapidly help our customers.” And that's what we've done.Corey: A lot of the arguments against serverless—and let's be clear, they rhyme with the previous arguments against cloud that lots of people used to make; including me, let's be clear here. I'm usually wrong when I try to predict the future. “Well, you're putting your availability in someone else's hands,” was the argument about cloud. Yeah, it turns out the clouds are better at keeping things up than we are as individual companies.Then with serverless, it's the, “Well, if they're handling all that stuff for you on their side, when they're down, you're down. That's an unacceptable business risk, so we're going to be cloud-agnostic and multi-cloud, and that means everything we build serverlessly needs to work in multiple environments, including in our on-prem environment.” And from the way that we're talking about servers and things that you're building, I don't believe that is technically possible, unless some of the stuff you're building is ridiculous. How did you come to accept that risk organizationally?Matt: These are the conversations that we're all having. Sort of, I'd say once a week, we all have a multi-cloud discussion—and I really liked the article you wrote, it was maybe last year, maybe the year before—but multi-cloud to me is about taking the best capabilities that are out there and bringing them together. So, you know, like, Azure [ID 00:12:47] or whatever, things from the other clouds that they're good at, and using those rather than thinking, “Can I build a workload that I can simultaneously pay all of the price to run across all of the clouds, all of the time, so that if one's down, theoretically, I might have an outage?” So, the way we've looked at it is we embraced really early the well-architected framework from AWS. And it talks about things like you need to have multi-region availability, you need to have your backups in place, you need to have things like circuit breakers in place for if third-party goes down, and we've just tried to build really resilient architectures as best as we can on AWS. And do you know what I think, if [laugh] it AWS is not—I know at re:Invent, there it went down extraordinarily often compared to normal, but in general—Corey: We were all tired of re:Invent; their us-east-1 was feeling the exact same way.Matt: Yeah, so that's—it deserved a break. But, like, if somebody can't buy insurance for an hour, once a year, [laugh] I think we're okay with it versus spending millions to protect that one hour.Corey: And people make assumptions based on this where, okay, we had this problem with us-east-1 that froze things like the global Route 53 control planes; you couldn't change DNS for seven hours. And I highlighted that as, yeah, this is a problem, and it's something to severely consider, but I will bet you anything you'd care to name that there is an incredibly motivated team at AWS, actively fixing that as we speak. And by—I don't know how long it takes to untangle all of those dependencies, but I promise they're going to be untangled in relatively short order versus running data centers myself, when I discover a key underlying dependency I didn't realize was there, well, we need to break that. That's never going to happen because we're trying to do things as a company, and it's just not the most important thing for us as a going concern. With AWS, their durability and reliability is the most important thing, arguably compared to security.Would you rather be down or insecure? I feel like they pick down—I would hope in most cases they would pick down—but they don't want to do either one. That is something they are drastically incentivized to fix. And I'm never going to be able to fix things like that and I don't imagine that you folks would be able to either.Matt: Yeah, so, two things. The first thing is the important stuff, like, for us, that's claims. We want to make sure at any point in time, if you need to make a claim you can because that is why we're here. And we can do that with people whether or not the machines are up or down. So, that's why, like, you always have a process—a manual process—that the business can operate, irrespective of whether the cloud is still working.And that's why we're able to say if you can't buy insurance in that hour, it's okay. But the other thing is, we did used to have a lot of data centers, and I have to say, the people who ran those were amazing—I think half the staff now work for AWS—but there was this story that I heard where there was an app that used to go down at the same time every day, and nobody could work out why. And it was because someone was coming in to clean the room at that time, and they unplugged the server to plug in a vacuum, and then we're cleaning the room, and then plugging it back in again. And that's the kind of thing that just happens when you manage people, and you manage a building, and manage a premises. Whereas if you've heard that happened that AWS, I mean, that would be front page news.Corey: Oh, it absolutely would. There's also—as you say, if it's the sales function, if people aren't able to buy insurance for an hour, when us-east-1 went down, the headlines were all screaming about AWS taking an outage, and some of the more notable customers were listed as examples of this, but the story was that, “AWS has massive outage,” not, “Your particular company is bad at technology.” There's sort of a reputational risk mitigation by going with one of these centralized things. And again, as you're alluding to, what you're doing is not life-critical as far as the sales process and getting people to sign up. If an outage meant that suddenly a bunch of customers were no longer insured, that's a very different problem. But that's not your failure mode.Matt: Exactly. And that's where, like, you got to look at what your business is, and what you're specifically doing, but for 99.99999% of businesses out there, I'm pretty sure you can be down for the tiny window that AWS is down per year, and it will be okay, as long as you plan for it.Corey: So, one thing that really surprised me about the entirety of what you've done at Liberty Mutual is that you're a big enterprise company, and you can take a look at any enterprise company, and say that they have dueling mottos, which is, “I am not going to comment on that,” or, “That's not funny.” Like, the safe mode for any large concern is to say nothing at all. But a lot of folks—not just you—at Liberty have been extremely vocal about the work that you're doing, how you view these things, and I almost want to call it advocacy or evangelism for the CDK. I'm slightly embarrassed to admit that for a little while there, I thought you were an AWS employee in their DevRel program because you were such an advocate in such strong ways for the CDK itself.And that is not something I expected. Usually you see the most vocal folks working in environments that, let's be honest, tend to play a little bit fast and loose with things like formal corporate communications. Liberty doesn't and yet, there you folks are telling these great stories. Was that hard to win over as a culture, or am I just misunderstanding how corporate life is these days?Matt: No, I mean, so it was different, right? There was a point in time where, I think, we all just sort of decided that—I mean, we're really good at what we do from an engineering perspective, and we wanted to make sure that, given the messaging we were given, those 5000 teck employees in Liberty Mutual, if you consider the difference in broadcasting to 5000 versus going external, it may sound like there's millions, billions of people in the world, but in reality, the difference in messaging is not that much. So, to me what I thought, like, whenever I started anyway—it's not, like, we had a meeting and all decided at the same time—but whenever I started, it was a case of, instead of me just posting on all the internal channels—because I've been doing this for years—it's just at that moment, I thought, I could just start saying these things externally and still bring them internally because all you've done is widened the audience; you haven't actually made it shallower. And that meant that whenever I was having the internal conversations, nothing actually changed except for it meant external people, like all their Heroes—like Jeremy Daly—could comment on these things, and then I could bring that in internally. So, it almost helped the reverse takeover of the enterprise to change the culture because I didn't change that much except for change the audience of who I was talking to.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance accelerator for the Oracle MySQL Database Service. Although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLTP and OLAP, don't ask me to ever say those acronyms again, workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: One thing that you've done that I want to say is admirable, and I stumbled across it when I was doing some work myself over the break, and only right before this recording did I discover that it was you is the cdkpatterns.com website. Specifically what I love about it is that it publishes a bunch of different patterns of ways to do things. This deviates from a lot of tutorials on, “Here's how to build this one very specific thing,” and instead talks about, “Here's the architecture design; here's what the baseline pattern for that looks like.” It's more than a template, but less than a, “Oh, this is a messaging app for dogs and I'm trying to build a messaging app for cats.” It's very generalized, but very direct, and I really, really like that model of demo.Matt: Thank you. So, watching some of your Twitter threads where you experiment with new—Corey: Uh oh. People read those. That's a problem.Matt: I know. So, whatever you experiment with a new piece of AWS to you, I've always wondered what it would be like to be your enabling architect. Because technically, my job in Liberty is, I meant to try and stay ahead of everybody and try and ease the on-ramp to these things. So, if I was your enabling architect, I would be looking at it going, “I should really have a pattern for this.” So that whenever you want to pick up that new service the patterns in cdkpatterns.com, there's 24, 25 of them right there, but internally, there's way more than dozens now.The goal is, the pattern is the least amount to code for you to learn a concept. And then that way, you can not only see how something works, but you can maybe pick up one of the pieces of the well-architected framework while you're there: All of it's unit tested, all of it is proper, you know, like, commented code. The idea is to not be crap, but not be gold-plated either. I'm currently in the process of upgrading that all to V2 as well. So, that [unintelligible 00:21:32].Corey: You mentioned a phrase just now: “Enabling architect.” I have to say this one that has not crossed my desk before. Is that an internal term you use? Is that an enterprise concept I've somehow managed to avoid? Is that an AWS job role? What is that?Matt: I've just started saying [laugh] it's my job over the past couple of years. That—I don't know, patent pending? But the idea to me is—Corey: No, it's evocative. I love the term, I'd love to learn more.Matt: Yeah, because you can sort of take two approaches to your architecture: You can take the traditional approach, which is the ‘house of no' almost, where it's like, “This is the architecture. How dare you want to deviate. This is what we have decided. If you want to change it, here's the Architecture Council and go through enterprise architecture as people imagine it.” But as people might work out quite quickly, whenever they meet me, the whole, like, long conversational meetings are not for me. What I want to do is teach engineers how to help themselves, so that's why I see myself as enabling.And what I've been doing is using techniques like Wardley Mapping, which is where you can go out and you can actually take all the components of people's architecture and you can draw them on a map for—it's a map of how close they are to the customer, as well as how cutting edge the tech is, or how aligned to our strategic direction it is. So, you can actually map out all of the teams, and—there's 160, 170 engineers in Belfast and Dublin, and I can actually go in and say, “Oh, that piece of your architecture would be better if it was evolved to this. Well, I have a pattern for that,” or, “I don't have a pattern for that, but you know what? I'll build one and let's talk about it next week.” And that's always trying to be ahead, instead of people coming to me and I have to say no.Corey: AWS Proton was designed to do something vaguely similar, where you could set out architectural patterns of—like, the two examples that they gave—I don't know if it's in general availability yet or still in public preview, but the ones that they gave were to build a REST API with Lambda, and building something-or-other with Fargate. And the idea was that you could basically fork those, or publish them inside of your own environment of, “Oh, you want a REST API; go ahead and do this.” It feels like their vision is a lot more prescriptive than what yours is.Matt: Yeah. I talked to them quite a lot about Proton, actually because, as always, there's different methodologies and different ways of doing things. And as I showed externally, we have our software accelerator, which is kind of our take on Proton, and it's very open. Anybody can contribute; anybody can consume. And then that way, it means that you don't necessarily have one central team, you can have—think of it more like an SRE function for all of the patterns, rather than… the Proton way is you've separate teams that are your DevOps teams that set up your patterns and then separate team that's consumer, and they have different permissions, different rights to do different things. If you use a Proton pattern, anytime an update is made to that pattern, it auto-deploys your infrastructure.Corey: I can see that breaking an awful lot.Matt: [laugh]. Yeah. So, the idea is sort of if you're a consumer, I assume you [unintelligible 00:24:35] be going to change that infrastructure. You can, they've built in an escape hatch, but the whole concept of it is there's a central team that looks to what the best configuration for that is. So, I think Proton has so much potential, I just think they need to loosen some of the boundaries for it to work for us, and that's the feedback I've given them directly as well.Corey: One thing that I want to take a step beyond this is, you care about this? More than most do. I mean, people will work with computers, yes. We get paid for that. Then they'll go and give talks about things. You're doing that as well. They'll launch a website occasionally, like, cdkpatterns.com, which you have. And then you just sort of decide to go for the absolute hardest thing in the world, and you're one of four authors of a book on this. Tell me more.Matt: Yeah. So, this is something that there's a few of us have been talking since one of the first CDK Days, where we're friends, so there's AWS Heroes. There's Thorsten Höger, Matt Bonig, Sathyajith Bhat, and myself, came together—it was sometime in the summer last year—and said, “Okay. We want to write a book, but how do we do this?” Because, you know, we weren't authors before this point; we'd never done it before. We weren't even sure if we should go to a publisher, or if we should self-publish.Corey: I argue that no one wants to write a book. They want to have written a book, and every first-time author I've ever spoken to at the end has said, “Why on earth would anyone want to do this a second time?” But people do it.Matt: Yeah. And that's we talked to Alex DeBrie, actually, about his book, the amazing Dynamodb Book. And it was his advice, told us to self-publish. And he gave us his starter template that he used for his book, which took so much of the pain out because all we had to do was then work out how we were going to work together. And I will say, I write quite a lot of stuff in general for people, but writing a book is completely different because once it's out there, it's out there. And if it's wrong, it's wrong. You got to release a new version and be like, “Listen, I got that wrong.” So, it did take quite a lot of effort from the group to pull it together. But now that we have it, I want to—I don't have a printed copy because it's only PDF at the minute, but I want a copy just put here [laugh] in, like, the frame. Because it's… it's what we all want.Corey: Yeah, I want you to do that through almost a traditional publisher, selfishly, because O'Reilly just released the AWS Cookbook, and I had a great review quote on the back talking about the value added. I would love to argue that they use one of mine for The CDK Book—and then of course they would reject it immediately—of, “I don't know why you do all this. Using the console and lying about it is way easier.” But yeah, obviously not the direction you're trying to take the book in. But again, the industry is not quite ready for the lying version of ClickOps.It's really neat to just see how willing you are to—how to frame this?—to give of yourself and your time and what you've done so freely. I sometimes make a joke—that arguably isn't that funny—that, “Oh, AWS Hero. That means that you basically volunteer for a $1.6 trillion company.”But that's not actually what you're doing. What you're doing is having figured out all the sharp edges and hacked your way through the jungle to get to something that is functional, you're a trailblazer. You're trying to save other people who are working with that same thing from difficult experiences on their own, having to all thrash and find our own way. And not everyone is diligent and as willing to continue to persist on these things. Is that a somewhat fair assessment how you see the Hero role?Matt: Yeah. I mean, no two Heroes are the same, from what I've judged, I haven't met every Hero yet because pandemic, so Vegas was the first time [I met most 00:28:12], but from my perspective, I mean, in the past, whatever number of years I've been coding, I've always been doing the same thing. Somebody always has to go out and be the first person to try the thing and work out what the value is, and where it'll work for us more work for us. The only difference with the external and public piece is that last 5%, which it's a very different thing to do, but I personally, I like even having conversations like this where I get to meet people that I've never met before.Corey: You sort of discovered the entire secret of why I have an interview podcast.Matt: [laugh]. Yeah because this is what I get out of it, just getting to meet other people and have new experiences. But I will say there's Heroes out there doing very different things. You've got, like, Hiro—as in Hiro, H-I-R-O—actually started AWS Newbies and she's taught—ah, it's hundreds of thousands of people how to actually just start with AWS, through a course designed for people who weren't coders before. That kind of thing is next-level compared to anything I've ever done because you know, they have actually built a product and just given it away. I think that's amazing.Corey: At some level, building a product and giving it away sounds like, “You know, I want to never be lonely again.” Well, that'll work because you're always going to get support tickets. There's an interesting narrative around how to wind up effectively managing the community, and users, and demands, based on open-source maintainers, that we're all wrestling with as an industry, particularly in the wake of that whole log4j nonsense that we've been tilting at that windmill, and that's going to be with us for a while. One last thing I want to talk about before we wind up calling this an episode is, you are one of the organizers of CDK Day. What is that?Matt: Yeah, so CDK Day, it's a complete community-organized conference. The past two have been worldwide, fully virtual just because of the situation we're in. And I mean, they've been pretty popular. I think we had about 5000 people attended the last one, and the idea is, it's a full day of the community just telling their stories of how they liked or disliked using the CDK. So, it's not a marketing event; it's not a sales event; we actually run the whole event on a budget of exactly $0. But yeah, it's just a day of fun to bring the community together and learn a few things. And, you know, if you leave it thinking CDK is not for you, I'm okay with that as much as if you just make a few friends while you're there.Corey: This is the first time I'd realized that it wasn't a formal AWS event. I almost feel like that's the tagline that you should have under it. It's—because it sounds like the CDK Day, again, like, it's this evangelism pure, “This is why it's great and why you should use it.” But I love conferences that embrace critical views. I built one of the first talks I ever built out that did anything beyond small user groups was “Heresy in the Church of Docker.”Then they asked me to give that at ContainerCon, which was incredibly flattering. And I don't think they made that mistake a second time, but it was great to just be willing to see some group of folks that are deeply invested in the technology, but also very open to hearing criticism. I think that's the difference between someone who is writing a nuanced critique versus someone who's just [pure-on 00:31:18] zealotry. “But the CDK is the answer to every technical problem you've got.” Well, I start to question the wisdom of how applicable it really is, and how objective you are. I've never gotten that vibe from you.Matt: No, and that's the thing. So, I mean, as we've worked out in this conversation, I don't work for AWS, so it's not my product. I mean, if it succeeds or if it fails, it doesn't impact my livelihood. I mean, there are people on the team who would be sad for, but the point is, my end goal is always the same. I want people to be enabled to rapidly deliver their software to help their customers.If that's CDK, perfect, but CDK is not for everyone. I mean, there are other options available in the market. And if, even, ClickOps is the way to go for you, I am happy for you. But if it's a case of we can have a conversation, and I can help you get closer to where you need to be with some other tool, that's where I want to be. I just want to help people.Corey: And if I can do anything to help along that axis, please don't hesitate to let me know. I really want to thank you for taking the time to speak with me and being so generous, not just with your time for this podcast, but all the time you spend helping the rest of us figure out which end is up, as we continue to find that the way we manage environments evolves.Matt: Yeah. And, listen, just thank you for having me on today because I've been reading your tweets for two years, so I'm just starstruck at this moment to even be talking to you. So, thank you.Corey: No, no. I understand that, but don't worry, I put my pants on two legs at a time, just like everyone else. That's right, the thought leader on Twitter, you have to jump into your pants. That's the rule. Thanks again so much. I look forward to having a further conversation with you about this stuff as I continue to explore, well honestly, what feels like a brand new paradigm for how we manage code.Matt: Yeah. Reach out if you need any help.Corey: I certainly will. You'll regret asking. Matt [Coulter 00:33:06], Technical Architect at Liberty Mutual. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, write an angry comment, then click the submit button, but lie and say you hit the submit button via an API call.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

About LeviLevi's passion lies in helping others learn to cloud better.Links: Jamf: https://www.jamf.com Twitter: https://twitter.com/levi_mccormick TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: It seems like there is a new security breach every day. Are you confident that an old SSH key, or a shared admin account, isn't going to come back and bite you? If not, check out Teleport. Teleport is the easiest, most secure way to access all of your infrastructure. The open-source Teleport Access Plane consolidates everything you need for secure access to your Linux and Windows servers, and I assure you there is no third option there. Kubernetes clusters, databases, and internal applications like AWS Management Console, Yankins, GitLab, Grafana, Jupyter Notebooks, and more. Teleport's unique approach is not only more secure, it also improves developer productivity. To learn more visit: goteleport.com. And not, that is not me telling you to go away, it is: goteleport.com.Corey: This episode is sponsored in part by our friends at Rising Cloud, which I hadn't heard of before, but they're doing something vaguely interesting here. They are using AI, which is usually where my eyes glaze over and I lose attention, but they're using it to help developers be more efficient by reducing repetitive tasks. So, the idea being that you can run stateless things without having to worry about scaling, placement, et cetera, and the rest. They claim significant cost savings, and they're able to wind up taking what you're running as it is in AWS with no changes, and run it inside of their data centers that span multiple regions. I'm somewhat skeptical, but their customers seem to really like them, so that's one of those areas where I really have a hard time being too snarky about it because when you solve a customer's problem and they get out there in public and say, “We're solving a problem,” it's very hard to snark about that. Multus Medical, Construx.ai and Stax have seen significant results by using them. And it's worth exploring. So, if you're looking for a smarter, faster, cheaper alternative to EC2, Lambda, or batch, consider checking them out. Visit risingcloud.com/benefits. That's risingcloud.com/benefits, and be sure to tell them that I said you because watching people wince when you mention my name is one of the guilty pleasures of listening to this podcast.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I am known-slash-renowned-slash-reviled for my creative pronunciations of various technologies, company names, et cetera. Kubernetes, for example, and other things that get people angry on the internet. The nice thing about today's guest is that he works at a company where there is no possible way for me to make it more ridiculous than it sounds because Levi McCormick is a cloud architect at Jamf. I know Jamf sounds like I'm trying to pronounce letters that are designed to be silent, but no, no, it's four letters: J-A-M-F. Jamf. Levi, thanks for joining me.Levi: Thanks for having me. I'm super excited.Corey: Exactly. Also professional advice for anyone listening: Making fun of company names is hilarious; making fun of people's names makes you a jerk. Try and remember that. People sometimes blur that distinction.So, very high level, you're a cloud architect. Now, I remember the days of enterprise architects where their IDEs were basically whiteboards, and it was a whole bunch of people sitting in a room. They call it an ivory tower, but I've been in those rooms; I assure you there is nothing elevated about this. It's usually a dank sub-basement somewhere. What do you do, exactly?Levi: Well, I am part of the enterprise architecture team at Jamf. My roles include looking at our use of cloud; making sure that we're using our resources to the greatest efficacy possible; coordinating between many teams, many products, many architectures; trying to make sure that we're using best practices; bringing them from the teams that develop them and learn them, socializing them to other teams; and just trying to keep a handle on this wild ride that we're on.Corey: So, what I find fun is that Jamf has been around for a long time. I believe it is not your first name. I want to say Casper was originally?Levi: I believe so, yeah.Corey: We're Jamf customers. You're not sponsoring this episode or anything, to the best of my knowledge. So, this is not something I'm trying to shill the company, but we're a customer; we use you to basically ensure that all of our company MacBooks, and laptops, et cetera, et cetera, are basically ensured that there's disk encryption turned on, that people have a password, and that screensaver is turned on, basically to mean that if someone gets their laptop stolen, it's a, “Oh, I have to spend more money with Apple,” and not, “Time to sound the data breach alarm,” for reasons that should be blindingly obvious. And it's great not just at the box check, but also fixing the real problem of I [laugh] don't want to lose data that is sensitive for obvious reasons. I always thought of this is sort of a thing that worked on the laptops. Why do you have a cloud team?Levi: Many reasons. First of all, we started in the business of providing the software that customers would run in their own data centers, in their own locations. Sometime in about 2015, we decided that we are properly equipped to run this better than other people, and we started to provide that as a service. People would move in, migrate their services into the cloud, or we would bring people into the cloud to start with.Device management isn't the only thing that we do. We provide some SSO-type services, we recently acquired a company called Wandera, which does endpoint security and a VPN-like experience for traffic. So, there's a lot of cloud powering all of those things.Corey: Are you able to disclose whether you're focusing mostly on AWS, on Azure, on Google Cloud, or are you pretending a cloud with something like IBM?Levi: All of the above, I believe.Corey: Excellent. That tells you it's a real enterprise, in seriousness. It's the—we talk about the idea of going all in on one providers being a general best practice of good place to start. I believe that. And then there are exceptions, and as companies grow and accumulate technical debt, that also is load-bearing and generates money, you wind up with this weird architectural series of anti-patterns, and when you draw it on a whiteboard of, “Here's our architecture,” the junior consultant comes in and says, “What moron built this?” Usually two said quote-unquote, “Moron,” and then they've just pooched the entire engagement.Yeah, most people don't show up in the morning hoping to do a terrible job today, unless they work at Facebook. So, there are reasons things are the way they are; they're constraints that shape these things. Yeah, if people were going to be able to shut down the company for two years and rebuild everything from scratch from the ground up, it would look wildly different. But you can't do that most of the time.Levi: Yeah. Those things are load bearing, right? You can't just stop traffic one day, and re-architect it with the golden image of what it should have been. We've gone through a series of acquisitions, and those architectures are disparate across the different acquired products. So, you have to be able to leverage lessons from all of them, bring them together and try and just slowly, incrementally march towards a better future state.Corey: As we take a look at the challenges we see The Duckbill Group over on my side of the world, where we talk to customers, it's I think it is surprising to folks to learn that cloud economics as I see it is—well, first, cost and architecture the same thing, which inherently makes sense, but there's a lot more psychology that goes into it than math. People often assume I spend most of my time staring into spreadsheets. I assure you that would not go super well. But it has to do with the psychological elements of what it is that people are wrestling with, of their understanding of the environment has not kept pace with reality, and APIs tend to, you know, tell truths.It's always interesting to me to see the lies that customers tell, not intentionally, but the reality of it of, “Okay, what about those big instances you're running in Australia?” “Oh, we don't have any instances in Australia.” “Look, I understand that you are saying that in good faith, however…” and now we're in a security incident mode and it becomes a whole different story. People's understanding always trails. What do you spend the bulk of your time doing? Is it building things? Is it talking to people? Is it trying to more or less herd cats in certain directions? What's the day-to-day?Levi: I would say it varies week-to-week. Depends on if we have a new product rolling out. I spend a lot of my time looking at architectural diagrams, reference architectures from AWS. The majority of the work I do is in AWS and that's where my expertise lies. I haven't found it financially incentivized to really branch out into any of the other clouds in terms of expertise, but I spend a lot of my time developing solutions, socializing them, getting them in front of teams, and then educating.We have a wide range of skills internally in terms of what people know or what they've been exposed to. I'd say a lot of engineers want to learn the cloud and they want to get opportunities to work on it, and their day-to-day work may not bring them those opportunities as often as they'd like. So, a good portion of my time is spent educating, guiding, joining people's sprints, joining in their stand-ups, and just kind of talking through, like, how they should approach a problem.Corey: Whenever you work at a big company, you invariably wind up with—well, microservices becomes the right answer, not because of the technical reasons; because of the people reason, the way that you get a whole bunch of people moving in roughly the same direction. You are a large scale company; who owns services in your idealized view of the world? Is it, “Well, I wrote something and it's five o'clock. Off to production with it. Talk to you in two days, if everything—if we still have a company left because I didn't double-check what I just wrote.”Do you think that the people who are building services necessarily should be the ones supporting it? Like, in other words, Amazon's approach of having the software engineers being responsible for the ones running it in production from an ops perspective. Is that the direction you trend towards, or do you tend to be from my side of the world—which is grumpy sysadmin—where people—developers hurl applications into your yard for you to worry about?Levi: I would say, I'm an extremist in the view of supporting the Amazon perspective. I really like you build it, you run it, you own it, you architect it, all of it. I think the other teams in the organization should exist to support and enable those paths. So, if you have platform teams are a really common thing you see hired right now, I think those platforms should be built to enable the company's perspective on operating infrastructure or services, and then those service teams on top of that should be enabled to—and empowered to make the decisions on how they want to build a service, how they want to provide it. Ultimately, the buck should stop with them.You can get into other operational teams, you could have a systems operation team, but I think there should be an explicit contract between a service team, what they build, and what they hand off, you know, you could hand off, like, a tier one level response, you know, you can do playbooks, you could do, you know, minimal alert, response, routing, that kind of stuff with a team, but I think that even that team should have a really strong contract with, like, here's what our team provides, here's how you engage with our team, here's how you will transition services to our team.Corey: The challenge with doing that, in some shops, has been that if you decide to roll out a, you build it, you own it, approach that has not been there since the beginning, you wind up with a lot of pushback from engineers who until now really enjoyed their 5:30 p.m. quitting time, or whenever it was they wound up knocking off work. And they started pushing back, like, “Working out of hours? That's inhumane.” And the DevOps team would be sitting there going, “We're right here. How dare you? Like, what do you think our job is?” And it's a, “Yes, but you're not people.” And then it leads to this whole back and forth acrimonious—we'll charitably call it a debate. How do you drive that philosophy?Levi: It's a challenge. I've seen many teams fracture, fall apart, disperse, if you will, under the transition of going through, like, an extreme service ownership. I think you balance it out with the carrot of you also get to determine your own future, right? You get to determine the programming language you use, you get to determine the underlying technologies that you use. Again, there's a contract: You have to meet this list of security concerns, you need to meet these operational concerns, and how you do that is up to you.Corey: When you take a look across various teams—let's bound this to the industry because I don't necessarily want you to wind up answering tough questions at work the day this episode airs—what do you see the biggest blockers to achieving, I guess, a functional cultural service ownership?Levi: It comes down to people's identity. They've established their own identity, “As I am X,” right? I'm a operations engineer. I'm a developer, I'm an engineer. And getting people to kind of branch out of that really fixed mindset is hard, and that, to me, is the major blocker to people assuming ownership.I've seen people make the transition from, “I'm just an engineer. I just want to write code.” I hate those lines. That frustrates me so much: “I just want to write code.” Transitioning into that, like, ownership of, “I had an idea. I built the platform or the service. It's a huge hit.” Or you know, “Lots of people are using it.” Like, seeing people go through that transformation become empowered, become fulfilled, I think is great.Corey: I didn't really expect to get called out quite like this, but you're absolutely right. I was against the idea, back when I was a sysadmin type because I didn't know how to code. And if you have developers supporting all of the stuff that they've built, then what does that mean for me? It feels like my job is evaporating. I don't know how to write code.Well, then I started learning how to write code incredibly badly. And then wow, it turns out, everyone does this. And here we are. But it's—I don't build applications, for obvious reasons. I'm bad at it, but I found another way to proceed in the wide world that we live in of high technology.But yeah, it was hard because this idea of my sense of identity being tied to the thing that I did, it really was an evolve-or-die dinosaur kind of moment because I started seeing this philosophy across the board. You take a look, even now at modern SRE is, or modern DevOps folks, or modern sysadmins, what they're doing looks a lot less like logging into Linux systems and tinkering on the command line a lot more like running and building distributed applications. Sure, this application that you're rolling out is the one that orchestrates everything there, but you're still running this in the same way the software engineers do, which is, interestingly.Levi: And that doesn't mean a team has to be only software engineers. Your service team can be multiple disciplines. It should be multiple disciplines. I've seen a traditional ops team broken apart, and those individuals distributed into the services that they were chiefly skilled in supporting in the past, as the ops team, as we transitioned those roles from one of the worst on-call rotations I've ever seen—you know, 13 to 14 alerts a night—transitioning those out to those service teams, training them up on the operations, building the playbooks. That was their role. Their role wasn't necessarily to write software, day one.Corey: I quit a job after six weeks because of that style of, I guess, mismanagement. Their approach was that, oh, we're going to have our monitoring system live in AWS because one of our VPs really likes AWS—let's be clear, this was 2008, 2009 era—latency was a little challenging there. And [unintelligible 00:17:04] he really liked Big Brother, which was—not to—now before that became a TV show and at rest, it was a monitoring system—but network latency was always a weird thing in AWS in those days, so instead, he insisted we set up three of them. And whenever—if we just got one page, it was fine. But if we got three, then we had to jump in. And two was always undefined.And they turned this off from I think, 10 p.m. to 6 a.m. every night, just so the person I call could sleep. And I'm looking at this, like, this might be the worst thing I've ever seen in my life. This was before they released the Managed NAT Gateway, so possibly it was.Levi: And then the flood, right, when you would get—Corey: Oh, God this was the days, too—Levi: Yeah.Corey: —when you were—if you weren't careful, you'd set this up to page you on the phone with a text message and great, now it takes time for my cell provider to wind up funneling out the sudden onslaught of 4000 text messages. No thanks.Levi: If your monitoring system doesn't have the ability to say, you know, the alert flood, funnel them into one alert, or just pause all alerts, while—because we know there's an incident; you know, us-east-1 is down, right? We know this; we don't need to get 500 text messages to each engineer that's on call.Corey: Well, my philosophy at that point was no, I'm going to instead take a step beyond. If I'm not empowered to fix this thing that is waking me up—and sometimes that's the monitoring system, and sometimes it's the underlying application—I'm not on call.Levi: Yes, exactly. And that's why I like the model of extre—you know, the service ownership: Because those alerts should go to the people—the pain should be felt by the people who are empowered to fix it. It should not land anywhere else. Otherwise, that creates misaligned incentives and nothing gets better.Corey: Yeah. But in large distributed systems, very often the person is on call more or less turns into a traffic router.Levi: Right. That's unfair to them.Corey: That's never fun—yeah, that's unfair, and it's not fun, either, and there's no great answer when you've all these different contributory factors.Levi: And how hard is it to keep the team staffed up?Corey: Oh, yeah. It's a, “Hey, you want a really miserable job one week out of every however many there are in the cycle?” Eh, people don't like that.Levi: Exactly.Corey: This episode is sponsored by our friends at Oracle HeatWave, a new high-performance accelerator for the Oracle MySQL Database Service, although I insist on calling it, “My squirrel.” While MySQL has long been the world's most popular open source database, shifting from transacting to analytics required way too much overhead and, you know, work. With HeatWave you can run your OLAP and OLTP—don't ask me to ever say those acronyms again—workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: So, I've been tracking what you're up to for little while now—you're always a blast to talk with—what is this whole Cloud Builder thing that you were talking about for a bit, and then I haven't seen much about it.Levi: Ah, so at the beginning of the pandemic, our mutual friend, Forrest Brazeal, released the Cloud Resume Challenge. I looked at that, and I thought, this is a fantastic idea. I've seen lots of people going through it. I recommend the people I mentor go through it. Great way to pick up a couple cloud skills here and there, tell an interesting story in an interview, right? It's a great prep.I intended the Cloud Builder Challenge to be a natural kind of progression from that Resume Challenge to the Builder Challenge where you get operational experience. Again, back to that, kind of, extreme service ownership mentality, here's a project where you can build, really modeled on the Amazon GameDays from re:Invent, you build a service, we'll send you traffic, you process those payloads, do some matching, some sorting, some really light processing on these payloads, and then send it back to us, score some points, we'll build a public dashboard, people can high five each other, they can razz each other, kind of competition they want to do. Really low, low pressure, but just a fun way to get more operational experience in an area where there is really no downside. You know, playing like that at work, bad idea, right?Corey: Generally, yes. [crosstalk 00:21:28] production, we used to have one of those environments; oops-a-doozy.Levi: Yeah. I don't see enough opportunities for people to gain that experience in a way that reflects a real workload. You can go out and you can find all kinds of Hello Worlds, you can find all kinds of—like, for front end development, there are tons of activity activities and things you can do to learn the skills, but for the middleware, the back end engineers, there's just not enough playgrounds out there. Now, standing up a Hello World app, you know, you've got your infrastructures code template, you've got your pre-written code, you deploy it, congratulations. But now what, right?And I intended this challenge to be kind of a series of increasingly more difficult waves, if you will, or levels. I really had a whole gamification aspect to it. So, it would get harder, it would get bigger, more traffic, you know, all of those things, to really put people through what it would be like to receive your, “Post got slash-dotted today,” or those kinds of things where people don't get an opportunity to deal with large amounts of traffic, or variable payloads, that kind of stuff.Corey: I love the idea. Where is it?Levi: It is sitting in a bunch of repos, and I am afraid to deploy it. [laugh].Corey: What is it that scares you about it specifically?Levi: The thing that specifically scares me is encouraging early career developers to go out there, deploy this thing, start playing with it, and then incur a huge cloud bill.Corey: Because they failed to secure something or other reasons behind that?Levi: There are many ways that this could happen, yeah. You could accidentally push your access key, secret key up into a public repo. Now, you've got, you know, Bitcoin miners or Monero miners running in your environment. You forget to shut things off, right? That's a really common thing.I went through a SageMaker demo from AWS a couple years ago. Half the room of intelligent, skilled engineers forgot to shut off the SageMaker instances. And everybody ran out of the $25 of credit they had from the demo—Corey: In about ten minutes. Yeah.Levi: In about ten minutes, yeah. And we had to issue all kinds of requests for credits and back and forth. But granted, AWS was accommodating to all of those people, but it was still a lot of stress.Corey: But it was also slow. They're very slow on that, which is fair. Like, if someone's production environment is down, I can see why you care more about that than you do about someone with, “Ah, I did something wrong and lost money.” The counterpoint to that is that for early career folks, that money is everything. We remember earlier this year, that tragic story from the Robinhood customer who committed suicide after getting a notification that he was $730,000 in debt. Turns out it wasn't even accurate; he didn't owe anything when all was said and done.I can see a scenario in which that happens in the AWS world because of their lack of firm price controls on a free tier account. I don't know what the answer on this is. I'm even okay with a, “Cool you will—this is a special kind of account that we will turn you off at above certain levels.” Fine. Even if you hard cap at the 20 or 50 bucks, yeah, it's going to annoy some people, but no one is going to do something truly tragic over that. And I can't believe that Oracle Cloud of all companies is the best shining example of this because you have to affirmatively upgrade your account before they'll charge you a dime. It's the right answer.Levi: It is. And I don't know if you've ever looked at—well, I'm sure you'd have. You've probably looked at the solutions provided by AWS for monitoring costs in your accounts, preventing additional spend. Like, the automation to shut things down, right, it's oftentimes more engineering work to make it so that your systems will shut down automatically when you reach a certain billing threshold than the actual applications that are in place there.Corey: And I don't for the life of me understand why things are the way that they are. But here we go. It's a—[sigh] it just becomes this perpetual strange world. I wish things were better than they are, but they're not.Levi: It makes me terribly sad. I mean, I think AWS is an incredible product, I think the ecosystem is great, and the community is phenomenal; everyone is super supportive, and it makes me really sad to be hesitant to recommend people dive into it on their own dime.Corey: Yeah. And that is a—[sigh] I don't know how you fix that or square that circle. Because I don't want to wind up, I really do not want to wind up, I guess, having to give people all these caveats, and then someone posts about a big bill problem on the internet, and all the comments are, “Oh, you should have set up budgets on that.” Yeah, that's thing still a day behind. So okay, great, instead of having an enormous bill at the end of the month, you just have a really big one two days later.I don't think that's the right answer. I really don't. And I don't know how to fix this, but, you know, I'm not the one here who's a $1.7 trillion company, either, that can probably find a way to fix this. I assure you, the bulk of that money is not coming from a bunch of small accounts that forgot to turn something off or got exploited.Levi: I haven't done my 2021 taxes yet, but I'm pretty sure I'm not there either.Corey: The world in which we live.Levi: [laugh]. I would love this challenge. I would love to put it out there. If I could, on behalf of, you know, early career people who want to learn—if I could issue credits, if I could spin up sandboxes and say, like, “Here's an account, I know you're going to be safe. I have put in a $50 limit.” Right?Corey: Yeah.Levi: “You can't spend more than $50,” like, if I had that control or that power, I would do this in a heartbeat. I'm passionate about getting people these opportunities to play, you know, especially if it's fun, right? If we can make this thing enjoyable, if we can gamify it, we can play around, I think that'd be great. The experience, though, would be a significant amount of engineering on my side, and then a huge amount of outreach, and that to me makes me really sad.Corey: I would love to be able to do something like that myself with a, “Look, if you get a bill, they will waive it, or I will cover it.” But then you wind up with the whole problem of people not operating in good faith as well. Like, “All right, I'm going to mine a bunch of Bitcoin and claim someone else did it.” Or whatnot. And it's just… like, there are problems with doing this, and the whole structure doesn't lend itself to that working super well.Levi: Exactly. I often say, you know, I face a lot of people who want to talk about mining cryptocurrency in the cloud because I'm a cloud architect, right? That's a really common conversation I have with people. And I remind them, like, it's not economical unless you're not paying for it.Corey: Yeah, it's perfectly economical on someone else's account.Levi: Exactly.Corey: I don't know why people do things the way that they do, but here we are. So, re:Invent. What did you find that was interesting, promising there, promising but not there yet, et cetera? What was your takeaway from it? Since you had the good sense not to be there in person?Levi: [laugh]. To me, the biggest letdown was Amplify Studio.Corey: I thought it was just me. Thank you. I just assumed it was something I wasn't getting from the explanation that they gave. Because what I heard was, “You can drag and drop, basically, a front end web app together and then tie it together with APIs on the back end.” Which is exactly what I want, like Retool does; that's what I want only I want it to be native. I don't think it's that.Levi: Right. I want the experience I already have of operating the cloud, knowing the security posture, knowing the way that my users access it, knowing that it's backed by Amazon, and all of their progressively improving services, right? You say it all the time. Your service running on Amazon is better today than it was two years ago. It was better than it was five years ago. I want that experience. But I don't think Amplify Studio delivered.Corey: I wish it had. And maybe it will, in the fullness of time. Again, AWS services do not get worse as they age they get better.Levi: Some gets stale, though.Corey: Yeah. The worst case scenario is they sit there and don't ever improve.Levi: Right. I thought the releases from S3 in terms of, like, the intelligent tiering, were phenomenal. I would love to see everybody turn on intelligent tiering with instant access. Those things to me were showing me that they're thinking about the problem the right way. I think we're missing a story of, like, how do we go from where we're at today—you know, if I've got trillions of objects in storage, how do I transition into that new world where I get the tiering automatically? I'm sure we'll see blog posts about people telling us; that's what the community is great for.Corey: Yeah, they explain these things in a way that the official docs for some reason fail to.Levi: Right. And why don't—Corey: Then again, it's also—I think—I think it's because the people that are building these things are too close to the thing themselves. They don't know what it's like to look at it through fresh eyes.Levi: Exactly. They're often starting from a blank slate, or from a greenfield perspective. There's not enough thought—or maybe there's a lot of thought to it, but there's not enough communication coming out of Amazon, like, here's how you transition. We saw that with Control Tower, we saw that with some of the releases around API Gateway. There's no story for transitioning from existing services to these new offerings. And I would love to see—and maybe Amazon needs a re:Invent Echo, where it's like, okay, here's all the new releases from re:Invent and here's how you apply them to existing infrastructure, existing environments.Corey: So, what's next for you? What are you looking at that's exciting and fun, and something that you want to spend your time chasing?Levi: I spend a lot of my time following AWS releases, looking at the new things coming out. I spend a lot of energy thinking about how do we bring new engineers into the space. I've worked with a lot of operations teams—those people who run playbooks, they hop on machines, they do the old sysadmin work, right—I want to bring those people into the modern world of cloud. I want them to have the skills, the empowerment to know what's available in terms of services and in terms of capabilities, and then start to ask, “Why are we not doing it that way?” Or start looking at making plans for how do we get there.Corey: Levi, I really want to thank you for taking the time to speak with me. If people want to learn more. Where can they find you?Levi: I'm on Twitter. My Twitter handle is @levi_mccormick. Reach out, I'm always willing to help people. I mentor people, I guide people, so if you reach out, I will respond. That's a passion of mine, and I truly love it.Corey: And we'll of course, include a link to that in the [show notes 00:32:28]. Thank you so much for being so generous with your time. I appreciate it.Levi: Thanks, Corey. It's been awesome.Corey: Levi McCormick, cloud architect at Jamf. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with a comment telling me that service ownership is overrated because you are the storage person, and by God, you will die as that storage person, potentially in poverty.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

About Serena Serena is a Network Engineer who specializes in Data Center Compute and Virtualization. She has degrees in Computer Information Systems with a concentration on networking and information security and is currently pursuing a master's in Data Center Systems Engineering. She is most known for her content on TikTok and Twitter as Shenetworks. Serena's content focuses on networking and security for beginners which has included popular videos on bug bounties, switch spoofing, VLAN hoping, and passing the Security+ certification in 24 hours.Links: TikTok: https://www.tiktok.com/@shenetworks Twitter: https://twitter.com/notshenetworks?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: It seems like there is a new security breach every day. Are you confident that an old SSH key, or a shared admin account, isn't going to come back and bite you? If not, check out Teleport. Teleport is the easiest, most secure way to access all of your infrastructure. The open source Teleport Access Plane consolidates everything you need for secure access to your Linux and Windows servers—and I assure you there is no third option there. Kubernetes clusters, databases, and internal applications like AWS Management Console, Yankins, GitLab, Grafana, Jupyter Notebooks, and more. Teleport's unique approach is not only more secure, it also improves developer productivity. To learn more visit: goteleport.com. And not, that is not me telling you to go away, it is: goteleport.com.Corey: This episode is sponsored in part by our friends at Redis, the company behind the incredibly popular open source database that is not the bind DNS server. If you're tired of managing open source Redis on your own, or you're using one of the vanilla cloud caching services, these folks have you covered with the go to manage Redis service for global caching and primary database capabilities; Redis Enterprise. To learn more and deploy not only a cache but a single operational data platform for one Redis experience, visit redis.com/hero. Thats r-e-d-i-s.com/hero. And my thanks to my friends at Redis for sponsoring my ridiculous non-sense.  Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Once upon a time, I was a grumpy Unix systems administrator—because it's not like there's a second kind of Unix systems administrator—then I decided it was time to get better at the networking piece, so I got a CCNA one year. Did this make me a competent network engineer? Absolutely not. But it made me a slightly better systems person.My guest today is coming from the other side of the world, specifically someone who is, in fact, good at the networking things. Serena—or @SheNetworks as you might know her from TikTok or @notshenetworks from the Twitters—thank you for joining me, I appreciate your time.Serena: Yeah, thanks for inviting me on.Corey: So, at a very high level, you are a network engineer, and you specialize in data center compute and virtualization, which is fun because I remember doing a lot of that once upon a time before I went basically all in on Cloud consulting, and then sort of forgot that data centers existed. That's still a thing that's still going well, and there are computers out there that don't belong to what are the three biggest tech companies in the world?Serena: Yeah. Shockingly, there's still a ton of data centers out there, still a lot of private hosting, and a lot of the environments that we see are mixed environment; they will have some cloud, some on-prem. But yes, data centers are still relevant. [laugh].Corey: On some level, it feels like once you get into the world of cloud, you don't have to really think about networking anymore. You know, until there's a big outage, and suddenly everyone had think about the networks. But it also feels like it is abstractions piled upon abstractions in the cloud infrastructure space. How much of what happens in data centers these days maps to what happens in these hyperscaler provider environments?Serena: That's a good question. I think—so I have two CCNAs; I'm very familiar with networking, I'm very familiar with virtualization, and I went and got my AWS certification because as we're talking about a lot of cloud things happening now, it's big, it's good to know about it. And underlying infrastructure under the cloud is all the data centers that I work with, all the networking things that I work with. So, it maps very well to me. I thought I had, like, a really easy time studying for my AWS certification because a lot of the concepts just had, like, a different fancy name for AWS versus just what you know, as, like, NAT, or, you know, DNS, different things like that.Corey: Of course, NAT used to be a thing that was—everyone would yell at you, “It's not security,” even though there are—I would argue there are security elements tied into it. But honestly, that feels like one of the best ways to pick fights with people who are way better at this than I am. Nowadays, of course, I just view NAT through a lens of, “Yeah, I totally want to pay an extra four-and-a-half cents per gigabyte passing through a managed NAT gateway,” which remains, of course, my nemesis. The intersection of security, networking, and billing leads to basically just being very angry all the time.Serena: Yeah. You come into the field, like, so ready to go, and then sometimes you do get beat down. But it's worth it, I think. I really like what I do.Corey: And what you do is something of an anomaly because most people who focus on this world of data center networking and the security aspects thereof, and the virtualization stuff, are all—how do I put it politely?—old, grumpy and unpleasant. I mean, I guess I'm not going to put it politely because I'm just going to be honest with it. Because I'm one of those people, let's be clear here. Instead, you are creating a whole bunch of content on Twitter and on TikTok, where I've got to say that the union set in the Venn diagram between TikTok and deep-dive networking and cybersecurity is basically you. How did you get there?Serena: That's a really good question. To your first point, the, you know, old grumpy, kind of, stereotype, those are honestly some of my favorite people, truly, because I don't know what it is, but I just vibe with them in a work environment so well. And it's funny, you know, when I got my first job out of college, I was definitely the youngest person on my team by far. And we would all go out to lunch, I would mess with all of them, we'd all play pranks on each other. Just integrating into the teams was always super easy for me, which I'm really lucky that—not everybody has that experience, especially in their first job; things are a little rough.But it's always great. Like, I love the diversity in tech. And to your second point, how did I end up here, right, with this kind of intersection from this networking world to TikTok? People are always confused. Like, how did that happen? How are you finding followers on TikTok that are interested in networking?And I'm just as shocked honestly. [laugh]. I started making this content this time last year, and… you know, at first I was like, nobody wants to learn about DNS on TikTok. This is where people dance and play pranks and all this stuff.Corey: And if there's dancing when it comes to DNS, at some point, something has gone other hilarious or terrifyingly. That again, I use it as a database, so who am I to talk?Serena: [laugh]. Yeah, but it's been fun. I am shocked. But there's such a wide variety of people now using TikTok and it's growing so quickly. Early on in my TikTok career, I had messages and emails from people who are vice presidents at major Fortune 100 companies asking me, you know, if I'd be interested in working there or, you know, something like that, and I was just—I was so shocked because there was a company that was a Fortune 100, and one of their VPs joined one of my Lives, and was asking me questions, just about, like, my background career, and then they sent me a follow up email [laugh] to be like, “Hey.”So, I was like, “Did I just get interviewed on my Live on TikTok?” And that they always, like, cracked me up. And at that point, I knew I was like, okay, this is something different; like, this is interesting. Because, you know, at the end of the day, you see the views and the numbers and the followers, but you don't have, really, faces to put to them or names, and you don't really know where a lot of these people are from, so you don't know who's seeing it. And a lot of times, I think I made the assumption that they are younger kids. Which is true, but there are also a lot of very seasoned professionals that have been in this field for a very long time that also follow me, and comment on my videos, and add great input and things like that.Corey: There's a giant misunderstanding, I think across the industry, that the executives at the big serious companies, you know, the ones whose mottos may as well be, “That's not funny,” have no personality themselves as people and that they live their entire lives in this corporate bubble where they talk to their kids primarily via I don't know, Microsoft Teams, or WebEx, or something else equally sad. And in practice, that just doesn't work that way. They're human beings, too. And granted, you have to present in certain ways in certain rooms, but the idea that, oh, you're only going to reach developers with attitude problems by having a personality of being on modern platforms. I mean, it's an easy mistake to make.I know this because I spent years making it myself with the nonsense that I do until suddenly people are reaching out and it's, “Huh. You sure did use a lot of high-level strategic terms for a developer.” And you start digging into it, and it's like, “Oh, you're your chief operating officer to giant company. I bet your code is terrible.” Is it? It's like, “Yeah. Turns out, maybe I'm not looking at that through the right lens.” Meeting people where they are with engaging content is important, and I think that a lot of folks completely miss that bus.Serena: Yeah, I agree. And this is a small field, right, so it gets kind of nerve wracking sometimes because sometimes you say things and it's so easy to be like, this is how I joke with my friends. But I'm still somewhat in a professional capacity because of me associating with my career, right? And then when my videos reach a million, half-a-million views, when we think about how many people are actually in this field that would be interested in viewing that content, you realize, oh, wow. Like, this is a huge mixed bag of people, which does include very high level executives, all the way to people that are in high school that are just interested in learning more. So, it's definitely been interesting to figure that out along the way. [laugh]. But yeah, they will have regular personalities. They all like TikTok too. If they don't, they're lying. [laugh].Corey: I used to be very down on the whole TikTok thing, but I started experimenting with it. And yeah, it turns out I have a face for radio and, you know, the social graces for Twitter. So, it's not really my cup of tea, but I enjoy watching it. I found that I'm not really a video person, but something about the TikTok format means I'm just going to start scrolling. And oh, dear, it's been six hours and my phone battery died. Thank God, or I'd still be there. There's something very captivating about it and I really like the format.The problem I always had with looking at a lot of the deeply technical content out there is so many companies are out there producing this and selling this. And that's fine. Like, money is not the end all, be all [of this 00:09:40]. I'm about to spend weeks of my life on something, the fact that it cost me 30 or 50 bucks or whatnot is really not economic thing I should be concerning myself with. But it all feels like it's classroom stuff. It's if you give people an option, are you going to go to a college lecture or are you going to go to a comedy show? Does the idea of, I want to be entertained. If you can teach me something while entertaining me, that feels like the winning combination, and you've absolutely nailed that.Serena: I think a lot of these companies that are producing content, hold themselves back a lot. And that is why they're not successful, right? Because there's so many stipulations, and there's teams of people, and boardrooms of approvals, and all these things, and me, all I'm doing—I record all my TikToks on my iPhone, and I just use in-app editing. I spend a lot of time kind of researching, right, maybe I will experiment with different formats, but the best format that's worked for me is just being authentic, kind of, not having that corporate vibe, right? And also not really expecting anything in return.So, a lot of times, corporations are putting out content because they obviously want to drive traffic to their websites, and different things like that, but the companies that do the best are the ones that are just putting out content for free, and really not necessarily expecting anything in return. And they also give themselves so much more leeway into the type of content that they create because they're not thinking about the numbers at the end of it, right? You just got to put stuff out there and people will see it. For me, I just put stuff out there, I don't need to wait for someone to approve my TikTok for me to push it out and have this content there. So, that is a big difference.And I've learned that through working with sponsors where they'll send you a giant list of talking points they want you to say and I'm like, “You guys know this is a 60-second video, right?” It needs to be really small. You need to, like, really learn how to get the really important stuff out there because the rest of the smaller stuff doesn't matter as much. Like, sell them on one big thing, and that really makes a difference.Corey: Oh, very much so. I see that sometimes with this show where people will reach out and ask about sponsoring, and they'll want to have a URL that I read into the microphone, and it's with UTM tracking parameters and the rest. And it's, like, “I appreciate where you're coming from and your intention here, however, that is not generally how this format works, so let's talk about this and the outcome.” And again, it's a brave new world out there. Yeah, if you're used to buying display ads in various places, that is exactly what you do.For some reason, there's this corporate mentality toward we're going to spend $25 million on a billboard saturation campaign, and not really give any thought about what we're actually going to say now that we have all of that visual real estate to get people's attention with. It's, there's not enough focus on the message itself, and I think that is a giant lost opportunity. Enterprise marketing doesn't have to be boring, it can be a lot of fun.Serena: I agree. And I think podcasting was the last, probably, big area that people budgeted for marketing, right? So, you have your traditional TV commercials and there was YouTube, and—you know, TV commercials, billboards, newspapers, then there's YouTube, and then podcasts, I would say, probably came a little bit later, as far as these companies look at for marketing potential. And now TikTok is so new and a lot of these marketing companies have no idea how to be successful on it because it's just so different. It's Gen Z, the humor is different.It's kind of like [laugh] the wild west on social media where things are just, like, crazy, and you have to fight the algorithm because on TikTok it's, if you don't like it, you just scroll within three seconds. The attention span is so short. So, you really have to capture people's attention within those first three seconds. Versus a podcast, you have the whole, let's say, first 20 minutes to get people, kind of, interested before you can be like, oh, hey, and here's my sponsor. So, it's very different versus TikTok, they'll just, like, oh, scroll. So, [laugh] you have to get creative and think differently.Corey: Many moons ago, when I was getting my CCNA, I worked at a company where we wound up getting a core switches for the data center, which was at the time, something like 65 grand. Great. And then we rented—because we had configured it in our office—and then a couple of us had to rent a commercial van, which I think ran something like $30,000 itself to transport this thing 20 miles to the data center, and I'm sitting there going, like, “Wow, the switch is worth way more than the van that's sitting within. Also were really shitty movers and that doesn't seem like the best idea for anything.” But I just think they remember that, and it left an impression on me.What I like about cloud with what I do is I can take a credit card and then spend less than $10 on AWS—or theoretically, Azure, or Google Cloud or, you know, $2 million on IBM because oops-a-doozy, but fine—and I wind up coming out the other side of that with having done some interesting disaster stuff. You are teaching people about how this stuff works, but in a data center world, it seems to me that the startup costs of, “Oh, I'm going to buy this random router or switch to wind up doing some demonstration stuff for,” it feels like the startup costs of getting hands on that equipment would be out of reach for an awful lot of people. Am I just completely out of touch with how that world works?Serena: No, you're right, you're one hundred percent, right. It is difficult. So, in college, my undergraduate degree is computer information systems, and they had a Cisco Networking Academy. And so we had old switches, old layer 3 switches, and then we had some routers, and this is all stuff that was EOL, donated equipment, right? And this is going to—Corey: It breaks down you're bidding against very faraway places with no budget on eBay for replacements. Oh, yes.Serena: Yeah, exactly. And it was a lot of IOS stuff, right? And so when I was in college, I had no idea that NX-OS existed, which is the data center Nexus version operating system for their switches and things. And so when I got to my first job and saw NX-OS, I was like, “Oh, crap, [laugh] like, what is this?” Right?Because I honestly didn't even know. I graduated and did not know that existed. And I didn't know a lot of the stuff that I was working on at my first shop existed. And I really had to rely on, kind of, the fundamentals. And they are transferable, right? That's why it's good to kind of get into—like, I know what these routing protocols are. I know, layer 2, I know this cabling, so let me just learn these command differences and things like that.And once you get into a production environment in general, out of a lab, it hits the fan. Like, everything you feel like you've learned is gone almost because there's so many layers and now all of a sudden, you have these firewalls, when before you were just trying to get, like, your routing neighborships to establish [laugh] and you weren't worried about rules on a firewall somewhere. And [crosstalk 00:16:39]—Corey: “Oh, and by the way, in this environment, that link that you're working on goes down, every minute it's down, here is the number of commas in the amount of money that we're losing, and yes, that's a plural.” It's, “Okay, so I guess I'm going to double-check everything I run first.” Yeah, it's that caution that gives people a bit of credence there. [unintelligible 00:16:58] do these things in a, more or less, cowboy style in these environments, at least not for very long. Because you can break individual servers; that's fine, but if you break the network suddenly, you may as well not have the computers.Serena: Yeah. It can be paralyzing, truly. It can be very overwhelming your first networking job. Especially for me, I was just dealing with outages constantly because I worked for a vendor, and I was [laugh] like, I was just scared, you know? Because I would get these cases and it would be a hospital outage.And I'm like, “I just graduated college. Like, what do you want from me?” You know, and back to your original point, it is difficult in a data center space because the equipment's so expensive. So, a lot of people ask, “Do you have a home lab?” And one—there's a couple of reasons I don't really have a significant home lab. One, I move so much.Corey: Oh, and in the spare room basically is always 90 degrees and sounds like a jet engine taking off.Serena: Yeah.Corey: Yeah, it's one of those, I should probably find a different place where I don't live, to have that equipment. Yeah.Serena: Yeah. And I have access, like, remotely to all the lab equipment that I really need. So, I don't personally have one, but a lot of things that I do work with are so expensive, that I'm like, I can't afford to put this data center equipment in my house. That doesn't make any sense.And there is luckily now a lot of virtual labs that you can do. There's some sandboxes by Cisco and other vendors, where you can kind of get a little bit of hands-on experience. A lot of it relates to their certifications. You can rent racks, but that gets pretty pricey, too. So, it is difficult, and sometimes that's why a lot of these jobs, I think I have a lot of people who are looking for entry-level work, and it's hard to get into a specifically a data center space.And aside from racking, stacking, working in a data center—maybe a NOC—if you want to get into the actual,s I'm configuring Nexus switches, I'm configuring, you know, Palo Alto firewalls, it can be difficult because it's hard to get to that point, there's not a clear path.Corey: What is the entry path these days? I entered tech by working on a help desk, and those aren't really the jobs that they once were, in a lot of different ways. So, I've stopped talking to entry-level folks with the position of, “Oh, yeah, this is what you should do because that's what I did.” It turns into, like, “Okay, Boomer. Great job. Tell me a little bit more, though, about what the Great War was like, first.” No, we aren't going to go down that path. It's just I don't know what the entry-level point is for someone who's legitimately interested in these things these days.Serena: Nobody does. It's crazy. And you're right at the, “Okay, Boomer,” thing. See, networking was one of those… things that just got pushed onto people in, just, a general IT department, right? So, that's when everything was like, “Okay, we need to get on the internet, so, you know, hey, you handle some of the computer stuff. It's your job now. Good luck. Figure it out.”And so, people started doing that and they kind of just got pushed into it, and then as the internet grew, as our capabilities grew, then the job became, like, a little bit more specialized. And now we have, you know, dedicated network engineers, we have people running data centers. But that's not necessarily a viable path now for people just because there's so much to it now. There's cloud, there's security risks, there's data center, wireless, pho—I mean, you can be an engineer just for phones, right? So, it's a little bit difficult for, especially, the younger people coming in, and the people that I talk to, and figuring out, well, how do I get to what you're doing?And the way that I did is I went and got a four-year degree and then joined a new college graduate program at a Fortune 100 company. Which is a great path, I highly recommend it to anybody that can do it, but it's also not available for everybody, right, because not everybody has the means to get a four-year education, nor do you necessarily need one to do what I do. So, everybody's kind of has this different path, and it's very confusing for people who are aspiring network engineers, or aspiring cloud engineers, even.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance accelerator for the Oracle MySQL Database Service. Although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLTP and OLAP, don't ask me to ever say those acronyms again, workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: The narrative the cloud companies have been pushing for a while—like, and I'm in that space deeply enough that I haven't really thought to go super deep into questioning this—is that well, the future is all cloud, the data center is basically this legacy thing that the tide is slowly eroding, in the fullness of time, because everything will one day be cloud. Do you think that's accurate?Serena: I don't. I really don't think that's accurate. Don't get me wrong, I think that the cloud is here to stay, and a lot of people are going to be using it. And it's going to be—and it currently is a huge part of our lives. Like, as we've seen recently with a few of the AWS outages, when it goes down and goes down hard because everything's so centralized.And people like to think, like, oh, you know, we have all this redundancy, yadda, yadda. That has not protected us so far, [laugh] like, from these major outages, right? And a lot of places that I see—especially when you're looking at public sector—is a hybrid, where you do have data center on-prem and you have cloud. And I think that, personally, is the best way to go. Unless, you know, maybe you're a fast growing startup and AWS or Azure makes a lot of sense to you.And it does. There's great use cases for that, right? But they're—not only aside from the whole cloud shift, there's another shift of, you know, making our data centers eco-friendly, too, and workload optimization. So, maybe the price point that you're looking for, what's going to save your business the most money, is doing that hybrid. So, I'm going to store a lot of my private documents on site, I'm going to have this as a backup disaster recovery, but we're also going to operate in the cloud. I don't think that the data centers as we know them are going to go extinct. [laugh]. I think they will be around.Corey: Well, AWS finally made their Outpost—the smaller ones; read as servers that run AWS services on in your facility—available a year after announcing them. And I looked at it like, oh, wow, these things are 600 bucks a month. Which is not nothing, but certainly something I could afford to wind up exploring and doing some content. But okay, first, it's a three-year commitment. So, that's 20 grand or so. Okay, not ideal, but fine.That would effectively almost double my AWS bill, but that's not the hardest part because, oh, and to get one of these, you have to have enterprise support. And when I pointed this out to some Amazonian friends, their response was, “Well, what's the problem on this?” Yeah, enterprise support starts at $15,000 a month minimum, and that means that people aren't going to pick these up to do proof of concept work. They're going to do it when they already have a significant infrastructure out there, and I think that's leaving an awful lot of money on the table by making people jump through sales hoops, and getting proof of concept credits, and doing all the other stuff for this. It's just ship me a box for a few weeks and let me kick the tires on in my environment and see if it works or doesn't work.Worst case, I'll ship it back to you. Worst, worst case, I lose the thing, and then you charge me whatever it costs to replace this. But it still feels like they are really doing the whole, “Oh, it's only big legacy companies that have on-premises stuff.” I don't like that narrative.Serena: I don't either. And I honestly think it's a bad idea, right, because if you do put all of your eggs in the AWS basket and they have all the power, that's not going to give us a lot of bargaining, right? That's not going to give people a lot of—because they'll know. They know how hard it is to get off of AWS at that point: They know it's costly, it takes manpower, it takes knowledge, right? And I think that it is in people's best interest to kind of have that mixed environment. Just for long-term, I'm just very wary of centralizing everything in one area. I think it's a bad idea. [laugh]. I think that we need to be prepared for ourselves, and that means also relying a little bit on ourselves. We can't just, in my opinion, put everything in the AWS basket. [laugh].Corey: Not very long anyway. It just doesn't seem to work.Serena: Right. And it's a great product.Corey: Oh, it absolutely is, but—Serena: There's so many positive things about using cloud. Because I'm not the type of person that likes to, kind of, talk crap about any vendor. I think everybody has their pros, cons, flaws, whatever. It's really about what works best for your environment, and that's part of being a network engineer or an architect is evaluating your environment and figuring out what is going to be the best for you, right? There's no one size fits all, unfortunately.Corey: Yeah. And AWS is uniformly excellent, let's be very clear. Okay, not—maybe not uniformly. Some services are significantly better than others, but I have an opinion piece in the information—paywalled, unfortunately, but I'm working on i—the general thesis that AWS has gotten too big to fail, in that when it's not—like, first, they are going to have better uptime than you or I will running our own data centers, across the board.They are very good at keeping things up, but when they do go down, it's not just your company or my company anymore having an outage, it is a significant portion of, you know, the global economy, and that is an awful lot of systemic concentrated risk. I'm not suggesting they did anything wrong, as far as how they sold these things—though, some people will want to argue with that—but it's the, “What does this mean?” Are we ready to reckon with that as a society that whenever us-east-1 has a bad day, so does the stock market? Is that something we're really prepared to accept or wrangle with? Or worse than that, there are life-critical services now. Does that mean that we're going to accept there is some number of people who will die when there's an outage of a data center? And that's new territory for me. I have not worked in environments where it was life or death consequential. At least not directly.Serena: Yeah, I have. So, I have definitely worked in those environments, right, and it's very scary, and especially when it's outside of your control. So, if you are relying, or just waiting on AWS to get back up, you don't have the control to get in there and start fixing things yourself, which is my instinct, right? Like, I immediately want to get hands-on. I put my troubleshooting hat on, like, let's figure this out, let me look through logs, let me do this.And you don't have that option with AWS when it's a significant outage that's impacting multiple people, it's not some configuration internally to you, right?And that's scary. It's a scary place to be. And I think that we need to really consider the cascading effects that will happen, which a lot of these outages that are kind of starting to show us, right? And luckily, there hasn't been anything major catastrophic, but we do need to really consider life when we're talking about, you know, hospitals, 911 systems, all of these critical infrastructures that are going to be cloud managed, and out of our control, and centralized.So, you know, you lose one 911 system, okay, well, you can do a backup, right? You may be able to route all your calls to the city over because their 911 systems are up and running. Well, what if there's are out now, too, because you're both hosted on AWS?Corey: Or you're, “Ah, we're going to diversify and we're going to have this other one on a different cloud provider.” That's great, but there's a critical third-party dependency that's right back to the thing you're trying to avoid. And there you go again.Serena: Yep. And that's dependency hell, right? [laugh].Corey: Oh, yeah. And I don't know how we get away from that.Serena: Yeah.Corey: Like, we don't want everyone writing all their own stuff from scratch, like starting with assembly, move up the stack. But here we are.Serena: Right. And it's funny because these AWS outages specifically effects—or cloud outages, right? I feel like I'm picking on them. I'm not trying to—sorry, AWS, but [laugh] don't come for me.But you know, explaining to my mom, why her Ring doorbell is not working and her Roomba stopped working when that outage happened, right, she's like, “Why is this not—it won't connect.” Like, “I don't understand.” She's like, “What's AWS?” And then to tell my mom that the company that she buys her socks from, like, that she goes online and, like, buys on Amazon is the company that also is hosting her Roomba, you know, services, her Ring services, it's so interesting to have those conversations. And a lot of people who aren't in our field don't understand that. They don't understand cloud, they don't understand on-prem versus, you know, hosted by a third-party. So, it's interesting to watch that kind of unfold now because it's very new. It's very new territory.Corey: And one last question before we wind up calling it an episode. It is remarkably clear in talking to you that you are in no way, shape, or form, junior. You are not a beginner. You know exactly how this stuff works in significant depth. Your content that you put out is aimed at beginners. I do something very similar. So, to be very clear, this is not a criticism in the slightest, but I am curious as to why that's the direction you went in.Serena: I think there's a few reasons. Well, I might have this knowledge, right? I still consider myself very junior in my career, very early in my career. There's so many things that I don't know and I recognize that. When you're first starting out, you might have this kind of inflated sense of knowledge where you're like—like, me, I was like, “Oh, yeah. I know all about OSPF and running on IOS and the command line,” until I figured out there was an NX-OS and I'm like, “Oh crap, what else do I not know about?” Right? [laugh].Corey: Oh, by the way, that never goes away. I feel exactly the same way 20 years into my career, now. I still have absolutely no idea what I'm doing. So smile, nod, and get used to it is the only insight I've got there. But please, go on.Serena: And even on Twitter sometimes, I'm reading people's stuff, and I'm like, “How did you get into these obscure protocols and all these things?” And, you know, I just kind of dive deeper into there. But I think the big reason that I create a lot of my content for beginners is because I remember so well how it was at the beginning, learning about subnetting, and that IOS—[laugh]—[unintelligible 00:30:52] learning about subnetting, and all of the different models that we have, right? And I was overwhelmed, and I was stressed out, and it just seems so… just, like, a giant mountain to climb. It seems so daunting in the beginning, for me it did because there's so much, right?And it felt like everybody was so far ahead of me. And I don't want other people to really feel like that. Like, I don't want people to be turned off from networking because they feel like the bar is too high, that we're not letting enough new people enter because we're discouraging them from the beginning by saying, “Oh, well, you're going to have to know all this. And let me throw this certification book at you.” And they're big. Like, my certification books—and these are massive. And this is for one half of the CCNA.Corey: For those who aren't, like, on the video call—it's not being recorded video-wise—she's holding a book that you could use to kill a mid-sized dog by accident if it falls off a table. It looks like a phonebook with a hardcover on it.Serena: Yeah. [laugh]. It's huge, right? And there are thousands of pages, and we just give this to somebody and say, like, “Here you go. Make sure you remember all this.” And this is all new information.Corey: And does it still cover things like EIGRP? Like Cisco's proprietary routing protocols that I've never once seen in the wild?Serena: Yeah. So, sometimes you will have to learn that, and they've changed it recently, too. They update their certification exam. So, you will learn about some legacy protocols because sometimes you do run into them.Corey: Oh, yes. That's when I have the good sense to pay professionals who know what they're doing.Serena: [laugh]. Yeah. Exactly. So yeah, you do run into those sometimes. But it feels so daunting for new people, and I totally recognize that. And by nature of TikTok I, especially when I first start making content, I assume that most of the people on there are going to be people who are younger, who are interested in this career.And as you know, in tech in general, especially networking, security, cloud, there's a massive shortage of people, and how are we solving that, right? And my contribution to helping solve that is by getting people interested. And now I have people that DM me and say, “I passed my [Network+ 00:33:01],” or, “I just took the CCNA,” or, “This has been helping me with my class so much.” And that is like, okay, this is great.Like, that's exactly what I want. I want to help the pipeline, I want to get more people interested and help a diverse group of people get interested in tech and say, “Hey, like, this is, you know, where I came from. And I did it; you can do it; let's do it together,” type situation.Corey: I really want to thank you for being so generous with your time. If people want to learn more, as they absolutely should, where can they find you?Serena: I am on TikTok as @SheNetworks. I am on Twitter as @notshenetworks because somebody else—Corey: That is very confusing.Serena: [laugh]. I know. Well, my initial thing was like, I didn't really use Twitter that much, and I would just like—I kind of used it as, like, a backchannel to my TikTok, right, where I would just, like, “Hey, I'm going to go live,” or do this. And then my Twitter, kind of, got a little out of control [laugh] and out of my hands. And so—Corey: It does that sometimes.Serena: Yeah. I had no idea there would be so much interest. And it surprises me every day. So, it's exciting though. I really love all the people that I've met, and I feel like I fit in, and I've met so many good friends that it's been great. But yeah, so @notshenetworks on Twitter because somebody had shenetworks and it was a joke. And [laugh] so if you want to find me there, you could also find me there.Corey: And we will, of course, put links to that in the [show notes 00:34:20]. Thank you so much for taking the time to speak with me today. I really do appreciate it.Serena: Thank you for having me. This has been great. [laugh].Corey: Serena, also known as @SheNetworks, networking content creator to the stars. I'm cloud economist, Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice and then a long, angry, rambling comment about how the network isn't that important that you're then not going to be able to submit because the network isn't working.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

About AnilAnil Dash is the CEO of Glitch, the friendly developer community where coders collaborate to create and share millions of web apps. He is a recognized advocate for more ethical tech through his work as an entrepreneur and writer. He serves as a board member for organizations like the Electronic Frontier Foundation, the leading nonprofit defending digital privacy and expression, Data & Society Research Institute, which researches the cutting edge of tech's impact on society, and The Markup, the nonprofit investigative newsroom that pushes for tech accountability. Dash was an advisor to the Obama White House's Office of Digital Strategy, served for a decade on the board of Stack Overflow, the world's largest community for coders, and today advises key startups and non-profits including the Lower East Side Girls Club, Medium, The Human Utility, DonorsChoose and Project Include.As a writer and artist, Dash has been a contributing editor and monthly columnist for Wired, written for publications like The Atlantic and Businessweek, co-created one of the first implementations of the blockchain technology now known as NFTs, had his works exhibited in the New Museum of Contemporary Art, and collaborated with Hamilton creator Lin-Manuel Miranda on one of the most popular Spotify playlists of 2018. Dash has also been a keynote speaker and guest in a broad range of media ranging from the Obama Foundation Summit to SXSW to Desus and Mero's late-night show.Links: Glitch: https://glitch.com Web.dev: https://web.dev Glitch Twitter: https://twitter.com/glitch Anil Dash Twitter: https://twitter.com/anildash TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: It seems like there is a new security breach every day. Are you confident that an old SSH key, or a shared admin account, isn't going to come back and bite you? If not, check out Teleport. Teleport is the easiest, most secure way to access all of your infrastructure. The open source Teleport Access Plane consolidates everything you need for secure access to your Linux and Windows servers—and I assure you there is no third option there. Kubernetes clusters, databases, and internal applications like AWS Management Console, Yankins, GitLab, Grafana, Jupyter Notebooks, and more. Teleport's unique approach is not only more secure, it also improves developer productivity. To learn more visit: goteleport.com. And not, that is not me telling you to go away, it is: goteleport.com.Corey: It seems like there is a new security breach every day. Are you confident that an old SSH key, or a shared admin account, isn't going to come back and bite you? If not, check out Teleport. Teleport is the easiest, most secure way to access all of your infrastructure. The open source Teleport Access Plane consolidates everything you need for secure access to your Linux and Windows servers—and I assure you there is no third option there. Kubernetes clusters, databases, and internal applications like AWS Management Console, Yankins, GitLab, Grafana, Jupyter Notebooks, and more. Teleport's unique approach is not only more secure, it also improves developer productivity. To learn more visit: goteleport.com. And not, that is not me telling you to go away, it is: goteleport.com.Corey: This episode is sponsored in part by our friends at Redis, the company behind the incredibly popular open source database that is not the bind DNS server. If you're tired of managing open source Redis on your own, or you're using one of the vanilla cloud caching services, these folks have you covered with the go to manage Redis service for global caching and primary database capabilities; Redis Enterprise. To learn more and deploy not only a cache but a single operational data platform for one Redis experience, visit redis.com/hero. Thats r-e-d-i-s.com/hero. And my thanks to my friends at Redis for sponsoring my ridiculous non-sense.  Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Today's guest is a little bit off the beaten path from the cloud infrastructure types I generally drag, kicking and screaming, onto the show. If we take a look at the ecosystem and where it's going, it's clear that in the future, not everyone who wants to build a business, or a tool, or even an application is going to necessarily spring fully-formed into the world from the forehead of some God, knowing how to code. And oh, “I'm going to go to a boot camp for four months to learn how to do it first,” is increasingly untenable. I don't know if you would call it low-code or not. But that's how it feels. My guest today is Anil Dash, CEO of Glitch. Anil, thank you for joining me.Anil: Thanks so much for having me.Corey: So, let's get the important stuff out of the way first, since I have a long-standing history of mispronouncing the company Twitch as ‘Twetch,' I should probably do the same thing here. So, what is Gletch? And what does it do?Anil: Glitch is, at its simplest, a tool that lets you build a full-stack app in your web browser in about 30 seconds. And, you know, for your community, your audience, it's also this ability to create and deploy code instantly on a full-stack server with no concern for deploy, or DevOps, or provisioning a container, or any of those sort of concerns. And what it is for the users is, honestly, a community. They're like, “I looked at this app that was on Glitch; I thought it was cool; I could do what we call [remixing 00:02:03].” Which is to kind of fork that app, a running app, make a couple edits, and all of a sudden live at a real URL on the web, my app is running with exactly what I built. And that's something that has been—I think, just captured a lot of people's imagination to now where they've built over 12 or 15 million apps on the platform.Corey: You describe it somewhat differently than I would, and given that I tend to assume that people who create and run successful businesses don't generally tend to do it without thought, I'm not quite, I guess, insufferable enough to figure out, “Oh, well, I thought about this for ten seconds, therefore I've solved a business problem that you have been needling at for years.” But when I look at Glitch, I would describe it as something different than the way that you describe it. I would call it a web-based IDE for low-code applications and whatnot, and you never talk about it that way. Everything I can see there describes it talks about friendly creators, and community tied to it. Why is that?Anil: You're not wrong from the conventional technologist's point of view. I—sufficient vintage; I was coding in Visual Basic back in the '90s and if you squint, you can see that influence on Glitch today. And so I don't reject that description, but part of it is about the audience we're speaking to, which is sort of a next generation of creators. And I think importantly, that's not just age, right, but that could be demographic, that can be just sort of culturally, wherever you're at. And what we look at is who's making the most interesting stuff on the internet and in the industry, and they tend to be grounded in broader culture, whether they're on, you know, Instagram, or TikTok, or, you know, whatever kind of influencer, you want to point at—YouTube.And those folks, they think of themselves as creators first and they think of themselves as participating in the community first and then the tool sort of follow. And I think one of the things that's really striking is, if you look at—we'll take YouTube as an example because everyone's pretty familiar with it—they have a YouTube Creator Studio. And it is a very rich and deep tool. It does more than, you know, you would have had iMovie, or Final Cut Pro doing, you know, 10 or 15 years ago, incredibly advanced stuff. And those [unintelligible 00:04:07] use it every day, but nobody goes to YouTube and says, “This is a cloud-based nonlinear editor for video production, and we target cinematographers.” And if they did, they would actually narrow their audience and they would limit what their impact is on the world.And so similarly, I think we look at that for Glitch where the social object, the central thing that people organize around a Glitch is an app, not code. And that's this really kind of deep and profound idea, which is that everybody can understand an app. Everybody has an idea for an app. You know, even the person who's, “Ah, I'm not technical,” or, “I'm not really into technology,” they're like, “But you know what? If I could make an app, I would make this.”And so we think a lot about that creative impulse. And the funny thing is, that is a common thread between somebody that literally just got on the internet for the first time and somebody who has been doing cloud deploys for as long as there's been a cloud to deploy to, or somebody has been coding for decades. No matter who you are, you have that place that is starting from what's the experience I want to build, the app I want to build? And so I think that's where there's that framing. But it's also been really useful, in that if you're trying to make a better IDE in the cloud and a better text editor, and there are multiple trillion-dollar companies that [laugh] are creating products in that category, I don't think you're going to win. On the other hand, if you say, “This is more fun, and cooler, and has a better design, and feels better,” I think we could absolutely win in a walk away compared to trillion-dollar companies trying to be cool.Corey: I think that this is an area that has a few players in it could definitely stand to benefit by having more there. My big fear is not that AWS is going to launch stuff in your space and drive you out of business; I think that is a somewhat naive approach. I'm more concerned that they're going to try to launch something in your space, give it a dumb name, fail that market and appropriately, not understand who it's for and set the entire idea back five years. That is, in some cases, it seems like their modus operandi for an awful lot of new markets.Anil: Yeah, I mean, that's not an uncommon problem in any category that's sort of community driven. So, you know, back in the day, I worked on building blogging tools at the beginning of this, sort of, social media era, and we worried about that a lot. We had built some of the first early tools, Movable Type, and TypePad, and these were what were used to launch, like, Gawker and Huffington Post and all the, sort of, big early sites. And we had been doing it a couple years—and then at that time, major player—AOL came in, and they launched their own AOL blog service, and we were, you know, quaking in our boots. I remember just being kind of like, pit in your stomach, “Oh, my gosh. This is going to devastate the category.”And as it turns out, people were smart, and they have taste, and they can tell. And the domain that we're in is not one that is about raw computing power or raw resources that you can bring to bear so much as it is about can you get people to connect together, collaborate together, and feel like they're in a place where they want to make something and they want to share it with other people? And I mean, we've never done a single bit of advertising for Glitch. There's never been any paid acquisition. There's never done any of those things. And we go up against, broadly in the space, people that have billboards and they buy out all the ads of the airport and, you know, all the other kind of things we see—Corey: And they do the typical enterprise thing where they spend untold millions in acquiring the real estate to advertise on, and then about 50 cents on the message, from the looks of it. It's, wow, you go to all this trouble and expense to get something in front of me, and after all of that to get my attention, you don't have anything interesting to say?Anil: Right.Corey: [crosstalk 00:07:40] inverse of that.Anil: [crosstalk 00:07:41] it doesn't work.Corey: Yeah. Oh, yeah. It's brand awareness. I love that game. Ugh.Anil: I was a CIO, and not once in my life did I ever make a purchasing decision based on who was sponsoring a golf tournament. It never happened, right? Like, I never made a call on a database platform because of a poster that was up at, you know, San Jose Airport. And so I think that's this thing that developers in particular, have really good BS filters, and you can sort of see through.Corey: What I have heard about the airport advertising space—and I but a humble cloud economist; I don't know if this is necessarily accurate or not—but if you have a company like Accenture, for example, that advertises on airport billboards, they don't even bother to list their website. If you go to their website, it turns out that there's no shopping cart function. I cannot add ‘one consulting' to my cart and make a purchase.Anil: “Ten pounds of consult, please.”Corey: Right? I feel like the primary purpose there might very well be that when someone presents to your board and says, “All right, we've had this conversation with Accenture.” The response is not, “Who?” It's a brand awareness play, on some level. That said, you say you don't do a bunch traditional advertising, but honestly, I feel like you advertise—more successfully—than I do at The Duckbill Group, just by virtue of having a personality running the company, in your case.Now, your platform is for the moment, slightly larger than mine, but that's okay,k I have ambition and a tenuous grasp of reality and I'm absolutely going to get there one of these days. But there is something to be said for someone who has a track record of doing interesting things and saying interesting things, pulling a, “This is what I do and this is how I do it.” It almost becomes a personality-led marketing effort to some degree, doesn't it?Anil: I'm a little mindful of that, right, where I think—so a little bit of context and history: Glitch as a company is actually 20 years old. The product is only a few years old, but we were formerly called Fog Creek Software, co-founded by Joel Spolsky who a lot of folks will know from back in the day as Joel on Software blog, was extremely influential. And that company, under leadership of Joel and his co-founder Michael Pryor spun out Stack Overflow, they spun out Trello. He had created, you know, countless products over the years so, like, their technical and business acumen is off the charts.And you know, I was on the board of Stack Overflow from, really, those first days and until just recently when they sold, and you know, you get this insight into not just how do you build a developer community that is incredibly valuable, but also has a place in the ecosystem that is unique and persists over time. And I think that's something that was very, very instructive. And so when it came in to lead Glitch I, we had already been a company with a, sort of, visible founder. Joel was as well known as a programmer as it got in the world?Corey: Oh, yes.Anil: And my public visibility is different, right? I, you know, I was a working coder for many years, but I don't think that's what people see me on social media has. And so I think, I've been very mindful where, like, I'm thrilled to use the platform I have to amplify what was created on a Glitch. But what I note is it's always, “This person made this thing. This person made this app and it had this impact, and it got these results, or made this difference for them.”And that's such a different thing than—I don't ever talk about, “We added syntax highlighting in the IDE and the editor in the browser.” It's just never it right. And I think there are people that—I love that work. I mean, I love having that conversation with our team, but I think that's sort of the difference is my enthusiasm is, like, people are making stuff and it's cool. And that sort of is my lens on the whole world.You know, somebody makes whatever a great song, a great film, like, these are all things that are exciting. And the Glitch community's creations sort of feel that way. And also, we have other visible people on the team. I think of our sort of Head of Community, Jenn Schiffer, who's a very well known developer and her right. And you know, tons of people have read her writing and seen her talks over the years.And she and I talk about this stuff; I think she sort of feels the same way, which is, she's like, “If I were, you know, being hired by some cloud platform to show the latest primitives that they've deployed behind an API,” she's like, “I'd be miserable. Like, I don't want to do that in the world.” And I sort of feel the same way. But if you say, “This person who never imagined they would make an app that would have this kind of impact.” And they're going to, I think of just, like, the last couple of weeks, some of the apps we've seen where people are—it could be [unintelligible 00:11:53]. It could be like, “We made a Slack bot that finally gets this reporting into the right channel [laugh] inside our company, but it was easy enough that I could do it myself without asking somebody to create it even though I'm not technically an engineer.” Like, that's incredible.The other extreme, we have people that are PhDs working on machine learning that are like, “At the end of the day, I don't want to be responsible for managing and deploying. [laugh]. I go home, and so the fact that I can do this in create is really great.” I think that energy, I mean, I feel the same way. I still build stuff all the time, and I think that's something where, like, you can't fake that and also, it's bigger than any one person or one public persona or social media profile, or whatever. I think there's this bigger idea. And I mean, to that point, there are millions of developers on Glitch and they've created well over ten million apps. I am not a humble person, but very clearly, that's not me, you know? [laugh].Corey: I have the same challenge to it's, effectively, I have now a 12 employee company and about that again contractors for various specialized functions, and the common perception, I think, is that mostly I do all the stuff that we talk about in public, and the other 11 folks sort of sit around and clap as I do it. Yeah, that is only four of those people's jobs as it turns out. There are more people doing work here. It's challenging, on some level, to get away from the myth of the founder who is the person who has the grand vision and does all the work and sees all these things.Anil: This industry loves the myth of the great man, or the solo legend, or the person in their bedroom is a genius, the lone genius, and it's a lie. It's a lie every time. And I think one of the things that we can do, especially in the work at Glitch, but I think just in my work overall with my whole career is to dismantle that myth. I think that would be incredibly valuable. It just would do a service for everybody.But I mean, that's why Glitch is the way it is. It's a collaboration platform. Our reference points are, you know, we look at Visual Studio and what have you, but we also look at Google Docs. Why is it that people love to just send a link to somebody and say, “Let's edit this thing together and knock out a, you know, a memo together or whatever.” I think that idea we're going to collaborate together, you know, we saw that—like, I think of Figma, which is a tool that I love. You know, I knew Dylan when he was a teenager and watching him build that company has been so inspiring, not least because design was always supposed to be collaborative.And then you think about we're all collaborating together in design every day. We're all collaborating together and writing in Google Docs—or whatever we use—every day. And then coding is still this kind of single-player game. Maybe at best, you throw something over the wall with a pull request, but for the most part, it doesn't feel like you're in there with somebody. Certainly doesn't feel like you're creating together in the same way that when you're jamming on these other creative tools does. And so I think that's what's been liberating for a lot of people is to feel like it's nice to have company when you're making something.Corey: Periodically, I'll talk to people in the AWS ecosystem who for some reason appear to believe that Jeff Barr builds a lot of these services himself then writes blog posts about them. And it's, Amazon does not break out how many of its 1.2 million or so employees work at AWS, but I'm guessing it's more than five people. So yeah, Jeff probably only wrote a dozen of those services himself; the rest are—Anil: That's right. Yeah.Corey: —done by service teams and the rest. It's easy to condense this stuff and I'm as guilty of it as anyone. To my mind, a big company is one that has 200 people in it. That is not apparently something the world agrees with.Anil: Yeah, it's impossible to fathom an organization of hundreds of thousands or a million-plus people, right? Like, our brains just aren't wired to do it. And I think so we reduce things to any given Jeff, whether that's Barr or Bezos, whoever you want to point to.Corey: At one point, I think they had something like more men named Jeff on their board than they did women, which—Anil: Yeah. Mm-hm.Corey: —all right, cool. They've fixed that and now they have a Dave problem.Anil: Yeah [unintelligible 00:15:37] say that my entire career has been trying to weave out of that dynamic, whether it was a Dave, a Mike, or a Jeff. But I think that broader sort of challenge is this—that is related to the idea of there being this lone genius. And I think if we can sort of say, well, creation always happens in community. It always happens influenced by other things. It is always—I mean, this is why we talk about it in Glitch.When you make an app, you don't start from a blank slate, you start from a working app that's already on the platform and you're remix it. And there was a little bit of a ego resistance by some devs years ago when they first encountered that because [unintelligible 00:16:14] like, “No, no, no, I need a blank page, you know, because I have this brilliant idea that nobody's ever thought of before.” And I'm like, “You know, the odds are you'll probably start from something pretty close to something that's built before.” And that enabler of, “There's nothing new under the sun, and you're probably remixing somebody else's thoughts,” I think that sort of changed the tenor of the community. And I think that's something where like, I just see that across the industry.When people are open, collaborative, like even today, a great example is web browsers. The folks making web browsers at Google, Apple, Mozilla are pretty collaborative. They actually do share ideas together. I mean, I get a window into that because they actually all use Glitch to do test cases on different bugs and stuff for them, but you see, one Glitch project will add in folks from Mozilla and folks from Apple and folks from the Chrome team and Google, and they're like working together and you're, like—you kind of let down the pretense of there being this secret genius that's only in this one organization, this one group of people, and you're able to make something great, and the web is greater than all of them. And the proof, you know, for us is that Glitch is not a new idea. Heroku wanted to do what we're doing, you know, a dozen years ago.Corey: Yeah, everyone wants to build Heroku except the company that acquired Heroku, and here we are. And now it's—I was waiting for the next step and it just seemed like it never happened.Anil: But you know when I talked to those folks, they were like, “Well, we didn't have Docker, and we didn't have containerization, and on the client side, we didn't have modern browsers that could do this kind of editing experience, all this kind of thing.” So, they let their editor go by the wayside and became mostly deploy platform. And—but people forget, for the first year or two Heroku had an in-browser editor, and an IDE and, you know, was constrained by the tech at the time. And I think that's something where I'm like, we look at that history, we look at, also, like I said, these browser manufacturers working together were able to get us to a point where we can make something better.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance accelerator for the Oracle MySQL Database Service. Although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLTP and OLAP, don't ask me to ever say those acronyms again, workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: I do have a question for you about the nuts and bolts behind the scenes of Glitch and how it works. If I want to remix something on Glitch, I click the button, a couple seconds later it's there and ready for me to start kicking the tires on, which tells me a few things. One, it is certainly not using CloudFormation to provision it because I didn't have time to go and grab a quick snack and take a six hour nap. So, it apparently is running on computers somewhere. I have it on good authority that this is not just run by people who are very fast at assembling packets by hand. What does the infrastructure look like?Anil: It's on AWS. Our first year-plus of prototyping while we were sort of in beta and early stages of Glitch was getting that time to remix to be acceptable. We still wish it were faster; I mean, that's always the way but, you know, when we started, it was like, yeah, you did sit there for a minute and watch your cursor spin. I mean, what's happening behind the scenes, we're provisioning a new container, standing up a full stack, bringing over the code from the Git repo on the previous project, like, we're doing a lot of work, lift behind the scenes, and we went through every possible permutation of what could make that experience be good enough. So, when we start talking about prototyping, we're at five-plus, almost six years ago when we started building the early versions of what became Glitch, and at that time, we were fairly far along in maturity with Docker, but there was not a clear answer about the use case that we're building for.So, we experimented with Docker Swarm. We went pretty far down that road; we spent a good bit of time there, it failed in ways that were both painful and slow to fix. So, that was great. I don't recommend that. In fairness, we have a very unusual use case, right? So, Glitch now, if you talk about ten million containers on Glitch, no two of those apps are the same and nobody builds an orchestration infrastructure assuming that every single machine is a unique snowflake.Corey: Yeah, massively multi-tenant is not really a thing that people know.Anil: No. And also from a security posture Glitch—if you look at it as a security expert—it is a platform allowing anonymous users to execute arbitrary code at scale. That's what we do. That's our job. And so [laugh], you know, so your threat model is very different. It's very different.I mean, literally, like, you can go to Glitch and build an app, running a full-stack app, without even logging in. And the reason we enable that is because we see kids in classrooms, they're learning to code for the first time, they want to be able to remix a project and they don't even have an email address. And so that was about enabling something different, right? And then, similarly, you know, we explored Kubernetes—because of course you do; it's the default choice here—and some of the optimizations, again, if you go back several years ago, being able to suspend a project and then quickly sort of rehydrate it off disk into a running app was not a common use case, and so it was not optimized. And so we couldn't offer that experience because what we do with Glitch is, if you haven't used an app in five minutes, and you're not a paid member, who put that app to sleep. And that's just a reasonable—Corey: Uh, “Put the app to sleep,” as in toddler, or, “Put the app to sleep,” as an ill puppy.Anil: [laugh]. Hopefully, the former, but when we were at our worst and scaling the ladder. But that is that thing; it's like we had that moment that everybody does, which is that, “Oh, no. This worked.” That was a really scary moment where we started seeing app creation ramping up, and number of edits that people were making in those apps, you know, ramping up, which meant deploys for us ramping up because we automatically deploy as you edit on Glitch. And so, you know, we had that moment where just—well, as a startup, you always hope things go up into the right, and then they do and then you're not sleeping for a long time. And we've been able to get it back under control.Corey: Like, “Oh, no, I'm not succeeding.” Followed immediately by, “Oh, no, I'm succeeding.” And it's a good problem to have.Anil: Exactly. Right, right, right. The only thing worse than failing is succeeding sometimes, in terms of stress levels. And organizationally, you go through so much; technically, you go through so much. You know, we were very fortunate to have such thoughtful technical staff to navigate these things.But it was not obvious, and it was not a sort of this is what you do off the shelf. And our architecture was very different because people had looked at—like, I look at one of our inspirations was CodePen, which is a great platform and the community love them. And their front end developers are, you know, always showing off, “Here's this cool CSS thing I figured out, and it's there.” But for the most part, they're publishing static content, so architecturally, they look almost more like a content management system than an app-running platform. And so we couldn't learn anything from them about our scaling our architecture.We could learn from them on community, and they've been an inspiration there, but I think that's been very, very different. And then, conversely, if we looked at the Herokus of the world, or all those sort of easy deploy, I think Amazon has half a dozen different, like, “This will be easier,” kind of deploy tools. And we looked at those, and they were code-centric not app-centric. And that led to fundamentally different assumptions in user experience and optimization.And so, you know, we had to chart our own path and I think it was really only the last year or so that we were able to sort of turn the corner and have high degree of confidence about, we know what people build on Glitch and we know how to support and scale it. And that unlocked this, sort of, wave of creativity where there are things that people want to create on the internet but it had become too hard to do so. And the canonical example I think I was—those of us are old enough to remember FTPing up a website—Corey: Oh, yes.Anil: —right—to Geocities, or whatever your shared web host was, we remember how easy that was and how much creativity was enabled by that.Corey: Yes, “How easy it was,” quote-unquote, for those of us who spent years trying to figure out passive versus active versus ‘what is going on?' As far as FTP transfers. And it turns out that we found ways to solve for that, mostly, but it became something a bit different and a bit weird. But here we are.Anil: Yeah, there was definitely an adjustment period, but at some point, if you'd made an HTML page in notepad on your computer, and you could, you know, hurl it at a server somewhere, it would kind of run. And when you realize, you look at the coding boot camps, or even just to, like, teach kids to code efforts, and they're like, “Day three. Now, you've gotten VS Code and GitHub configured. We can start to make something.” And you're like, “The whole magic of this thing getting it to light up. You put it in your web browser, you're like, ‘That's me. I made this.'” you know, north star for us was almost, like, you go from zero to hello world in a minute. That's huge.Corey: I started participating one of those boot camps a while back to help. Like, the first thing I changed about the curriculum was, “Yeah, we're not spending time teaching people how to use VI in, at that point, the 2010s.” It was, that was a fun bit of hazing for those of us who were becoming Unix admins and knew that wherever we'd go, we'd find VI on a server, but here in the real world, there are better options for that.Anil: This is rank cruelty.Corey: Yeah, I mean, I still use it because 20 years of muscle memory doesn't go away overnight, but I don't inflict that on others.Anil: Yeah. Well, we saw the contrast. Like, we worked with, there's a group called Mouse here in New York City that creates the computer science curriculum for the public schools in the City of New York. And there's a million kids in public school in New York City, right, and they all go through at least some of this CS education. [unintelligible 00:24:49] saw a lot of work, a lot of folks in the tech community here did. It was fantastic.And yet they were still doing this sort of very conceptual, theoretical. Here's how a professional developer would set up their environment. Quote-unquote, “Professional.” And I'm like, you know what really sparks kids' interests? If you tell them, “You can make a page and it'll be live and you can send it to your friend. And you can do it right now.”And once you've sparked that creative impulse, you can't stop them from doing the rest. And I think what was wild was kids followed down that path. Some of the more advanced kids got to high school and realized they want to experiment with, like, AI and ML, right? And they started playing with TensorFlow. And, you know, there's collaboration features in Glitch where you can do real-time editing and a code with this. And they went in the forum and they were asking questions, that kind of stuff. And the people answering their questions were the TensorFlow team at Google. [laugh]. Right?Corey: I remember those days back when everything seemed smaller and more compact, [unintelligible 00:25:42] but almost felt like a balkanization of community—Anil: Yeah.Corey: —where now it's oh, have you joined that Slack team, and I'm looking at this and my machine is screaming for more RAM. It's, like, well, it has 128 gigs in it. Shouldn't that be enough? Not for Slack.Anil: Not for chat. No, no, no. Chat is demanding.Corey: Oh, yeah, that and Chrome are basically trying to out-ram each other. But if you remember the days of volunteering as network staff on Freenode when you could basically gather everyone for a given project in the entire stack on the same IRC network. And that doesn't happen anymore.Anil: And there's something magic about that, right? It's like now the conversations are closed off in a Slack or Discord or what have you, but to have a sort of open forum where people can talk about this stuff, what's wild about that is, for a beginner, a teenage creator who's learning this stuff, the idea that the people who made the AI, I can talk to, they're alive still, you know what I mean? Like, yeah, they're not even that old. But [laugh]. They think of this is something that's been carved in stone for 100 years.And so it's so inspiring to them. And then conversely, talking to the TensorFlow team, they made these JavaScript examples, like, tensorflow.js was so accessible, you know? And they're like, “This is the most heartwarming thing. Like, we think about all these enterprise use cases or whatever. But like, kids wanting to make stuff, like recognize their friends' photo, and all the vision stuff they're doing around [unintelligible 00:26:54] out there,” like, “We didn't know this is why we do it until we saw this is why we do it.”And that part about connecting the creative impulse from both, like, the most experienced, advanced coders at the most august tech companies that exist, as well as the most rank beginners in public schools, who might not even have a computer at home, saying that's there—if you put those two things together, and both of those are saying, “I'm a coder; I'm able to create; I can make something on the internet, and I can share it with somebody and be inspired by it,” like, that is… that's as good as it gets.Corey: There's something magic in being able to reach out to people who built this stuff. And honestly—you shouldn't feel this way, but you do—when I was talking to the folks who wrote the things I was working on, it really inspires you to ask better questions. Like when I'm talking to Dr. Venema, the author of Postfix and I'm trying to figure out how this thing works, well, I know for a fact that I will not be smarter than he is at basically anything in that entire universe, and maybe most beyond that, as well, however, I still want to ask a question in such a way that doesn't make me sound like a colossal dumbass. So, it really inspires you—Anil: It motivates you.Corey: Oh, yeah. It inspires you to raise your question bar up a bit, of, “I am trying to do x. I expect y to happen. Instead, z is happening as opposed to what I find the documentation that”—oh, as I read the documentation, discover exactly what I messed up, and then I delete the whole email. It's amazing how many of those things you never send because when constructing a question the right way, you can help yourself.Anil: Rubber ducking against your heroes.Corey: Exactly.Anil: I mean, early in my career, I'd gone through sort of licensing mishap on a project that later became open-source, and sort of stepped it in and as you do, and unprompted, I got an advice email from Dan Bricklin, who invented the spreadsheet, he invented VisiCalc, and he had advice and he was right. And it was… it was unreal. I was like, this guy's one of my heroes. I grew up reading about his work, and not only is he, like, a living, breathing person, he's somebody that can have the kindness to reach out and say, “Yeah, you know, have you tried this? This might work.”And it's, this isn't, like, a guy who made an app. This is the guy who made the app for which the phrase killer app was invented, right? And, you know, we've since become friends and I think a lot of his inspiration and his work. And I think it's one of the things it's like, again, if you tell somebody starting out, the people who invented the fundamental tools of the digital era, are still active, still building stuff, still have advice to share, and you can connect with them, it feels like a cheat code. It feels like a superpower, right? It feels like this impossible thing.And I think about like, even for me, the early days of the web, view source, which is still buried in our browser somewhere. And you can see the code that makes the page, it felt like getting away with something. “You mean, I can just look under the hood and see how they made this page and then I can do it too?” I think we forget how radical that is—[unintelligible 00:29:48] radical open-source in general is—and you see it when, like, you talk to young creators. I think—you know, I mean, Glitch obviously is used every day by, like, people at Microsoft and Google and the New York Timesor whatever, like, you know, the most down-the-road, enterprise developers, but I think a lot about the new creators and the people who are learning, and what they tell me a lot is the, like, “Oh, so I made this app, but what do I have to do to put it on the internet?”I'm like, “It already is.” Like, as soon as you create it, that URL was live, it all works. And their, like, “But isn't there, like, an app store I have to ask? Isn't there somebody I have to get permission to publish this from? Doesn't somebody have to approve it?”And you realize they've grown up with whether it was the app stores on their phones, or the cartridges in their Nintendo or, you know, whatever it was, they had always had this constraint on technology. It wasn't something you make; it's something that is given to you, you know, handed down from on high. And I think that's the part that animates me and the whole team, the community, is this idea of, like, I geek out about our infrastructure. I love that we're doing deploys constantly, so fast, all the time, and I love that we've taken the complexity away, but the end of the day, the reason why we do it, is you can have somebody just sort of saying, I didn't realize there was a place I could just make something put it in front of, maybe, millions of people all over the world and I don't have to ask anybody permission and my idea can matter as much as the thing that's made by the trillion-dollar company.Corey: It's really neat to see, I guess, the sense of spirit and soul that arises from a smaller, more, shall we say, soulful company. No disparagement meant toward my friends at AWS and other places. It's just, there's something that you lose when you get to a certain point of scale. Like, I don't ever have to have a meeting internally and discuss things, like, “Well, does this thing that we're toying with doing violate antitrust law?” That is never been on my roadmap of things I have to even give the slightest crap about.Anil: Right, right? You know, “What does the investor relations person at a retirement fund think about the feature that we shipped?” Is not a question that we have to answer. There's this joy in also having community that sort of has come along with us, right? So, we talk a lot internally about, like, how do we make sure Glitch stays weird? And, you know, the community sort of supports that.Like, there's no reason logically that our logo should be the emoji of two fish. But that kind of stuff of just, like, it just is. We don't question it anymore. I think that we're very lucky. But also that we are part of an ecosystem. I also am very grateful where, like… yeah, that folks at Google use Glitch as part of their daily work when they're explaining a new feature in Chrome.Like, if you go to web.dev and their dev portal teaches devs how to code, all the embedded examples go to these Glitch apps that are running, showing running code is incredible. When we see the Stripe team building examples of, like, “Do you want to use this new payment API that we made? Well, we have a Glitch for you.” And literally every day, they ship one that sort of goes and says, “Well, if you just want to use this new Stripe feature, you just remix this thing and it's instantly running on Glitch.”I mean, those things are incredible. So like, I'm very grateful that the biggest companies and most influential companies in the industry have embraced it. So, I don't—yeah, I don't disparage them at all, but I think that ability to connect to the person who'd be like, “I just want to do payments. I've never heard of Stripe.”Corey: Oh yeah.Anil: And we have this every day. They come into Glitch, and they're just like, I just wanted to take credit cards. I didn't know there's a tool to do that.Corey: “I was going to build it myself,” and everyone shrieks, “No, no. Don't do that. My God.” Yeah. Use one of their competitors, fine,k but building it yourself is something a lunatic would do.Anil: Exactly. Right, right. And I think we forget that there's only so much attention people can pay, there's only so much knowledge they have.Corey: Everything we say is new to someone. That's why I always go back to assuming no one's ever heard of me, and explain the basics of what I do and how I do it, periodically. It's, no one has done all the mandatory reading. Who knew?Anil: And it's such a healthy exercise to, right, because I think we always have that kind of beginner's mindset about what Glitch is. And in fairness, I understand why. Like, there have been very experienced developers that have said, “Well, Glitch looks too colorful. It looks like a toy.” And that we made a very intentional choice at masking—like, we're doing the work under the hood.And you can drop down into a terminal and you can do—you can run whatever build script you want. You can do all that stuff on Glitch, but that's not what we put up front and I think that's this philosophy about the role of the technology versus the people in the ecosystem.Corey: I want to thank you for taking so much time out of your day to, I guess, explain what Glitch is and how you view it. If people want to learn more about it, about your opinions, et cetera. Where can they find you?Anil: Sure. glitch.com is easiest place, and hopefully that's a something you can go and a minute later, you'll have a new app that you built that you want to share. And, you know, we're pretty active on all social media, you know, Twitter especially with Glitch: @glitch. I'm on as @anildash.And one of the things I love is I get to talk to folks like you and learn from the community, and as often as not, that's where most of the inspiration comes from is just sort of being out in all the various channels, talking to people. It's wild to be 20-plus years into this and still never get tired of that.Corey: It's why I love this podcast. Every time I talk to someone, I learn something new. It's hard to remain too ignorant after you have enough people who've shared wisdom with you as long as you can retain it.Anil: That's right.Corey: Thank you so much for taking the time to speak with me.Anil: So, glad to be here.Corey: Anil Dash, CEO of Gletch—or Glitch as he insists on calling it. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment telling me how your small team at AWS is going to crush Glitch into the dirt just as soon as they find a name that's dumb enough for the service.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

About DanDan is CISO and VP of Cybersecurity for Shipt, a Target subsidiary. He worked previously as a Distinguished Engineer on Target's cloud infrastructure. He served as CTO for Joe Biden's 2020 Presidential campaign. Prior to that Dan worked with the Hillary for America tech team through the Groundwork, and contributed as a founding developer on Spinnaker while at Netflix. Dan is an O'Reilly published author and avid public speaker.  Links: Shipt: https://www.shipt.com/ Twitter: https://twitter.com/danveloper LinkedIn: https://www.linkedin.com/in/danveloper TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: It seems like there is a new security breach every day. Are you confident that an old SSH key, or a shared admin account, isn't going to come back and bite you? If not, check out Teleport. Teleport is the easiest, most secure way to access all of your infrastructure. The open source Teleport Access Plane consolidates everything you need for secure access to your Linux and Windows servers—and I assure you there is no third option there. Kubernetes clusters, databases, and internal applications like AWS Management Console, Yankins, GitLab, Grafana, Jupyter Notebooks, and more. Teleport's unique approach is not only more secure, it also improves developer productivity. To learn more visit: goteleport.com. And not, that is not me telling you to go away, it is: goteleport.com.Corey: Writing ad copy to fit into a 30 second slot is hard, but if anyone can do it the folks at Quali can. Just like their Torque infrastructure automation platform can deliver complex application environments anytime, anywhere, in just seconds instead of hours, days or weeks. Visit Qtorque.io today and learn how you can spin up application environments in about the same amount of time it took you to listen to this ad.Corey: Welcome to Screaming in the Cloud, I'm Corey Quinn. Sometimes I talk to people who are involved in working on the nonprofit slash political side of the world. Other times I talk to folks who are deep in the throes of commercial businesses, and I obviously personally spend more of my time on one of those sides of the world than I do the other. But today's guest is a little bit different, Dan Woods is the CISO and VP of Cybersecurity at Shipt, a division of Target where he's worked for a fair number of years, but took some time off for his side project, the side hustle as the kids call it, as the CTO for the Biden campaign. Dan, thank you for joining me.Dan: Yeah. Thank you, Corey. Happy to be here.Corey: So, you have an interesting track record as far as your career goes, you've been at Target for a long time. You were a distinguished engineer—not to be confused with ‘extinguished engineer,' which is just someone who is finally—the fire has gone out. And from there you went from being a distinguished engineer to a VP slash CISO, which generally looks a lot less engineer-like, and a lot more, at least in my experience, of sitting in a whole lot of executive-level meetings, managing teams, et cetera. Was that, in fact, an individual contributor—or IC—move into a management track, or am I just misunderstanding this because these are commonly overloaded terms in our industry?Dan: Yeah, yeah, no, that's exactly right. So, IC to leadership, two distinct tracks, distinct career paths. It was something that I've spent a number of years thinking about and more or less working toward and making sure that it was the right path for me to go. The interesting thing about the break that I took in the middle of Target when I was CTO for the campaign is that that was a leadership role, right. I led the team. I managed the team.I did performance reviews and all of that kind of managerial stuff, but I also sat down and did a lot of tech. So, it was kind of like a mix of being a senior executive, but also still continuing to be a distinguished engineer. So, then the natural path out of that for me was to make a decision about do I continue to be an individual contributor or do I go into a leadership track? And I felt like for a number of reasons that my interests more aligned with being on the leadership side of the world, and so that's how I've ended up where I am.Corey: And correct me if I'm wrong because generally speaking political campaigns are not usually my target customers given the fact that they're turning the entire AWS environment off in a few months—win or lose—and yeah, that is, in fact, remains the best way to save money on your AWS bill; it's hard for me to beat that. But at that point most of the people you're working with are in large part volunteers I would imagine.So, managing in a traditional sense of, “Well, we're going to have your next quarterly review.” Well, your candidate might not be in the race then, and what we're going to put you on a PIP, and what exactly you're going to stop letting me volunteer here? You're going to dock them pay—you're not paying me for this. It becomes an interesting management challenge I would imagine just because the people you're working with are passionate and volunteering, and a lot of traditional management and career advice doesn't necessarily map one-to-one I would have to assume.Dan: That is the best way that I've heard it described yet. I try to explain this to folks sometimes and it's kind of difficult to get that message across that like there is sort of a base level organization that exists, right. There were full-time employees who were a part of the tech team, really great group of folks especially from very early on willing to join the campaign and be a part of what it was that we were doing.And then there was this whole ecosystem of folks who just wanted to volunteer, folks who wanted to be a part of it but didn't want to leave their 9:00 to 5:00 who wanted to come in. One of the most difficult things about—we rely on volunteers very heavily in the political space, and very grateful for all the folks who step up and volunteer with organizations that they feel passionate about. In fact, one of the best little tidbits of wisdom the President imparted to me at one point, we were having dinner at his house very early on in the campaign, and he said, “The greatest gift that you can give somebody is your time.” And I think that's so incredibly true. So, the folks who volunteer, it's really important, really grateful that they're all there.In particular, how it becomes difficult, is that you need somebody to manage the volunteers, right, who are there. You need somebody to come up with work and check in that work is getting done because while it's great that folks want to volunteer five, ten hours a week, or whatever it is that they can put in, we also have very real things that need to get done, and they need to get done in a timely manner.So, we had a lot of difficulty especially early on in the campaign utilizing the volunteers to the extent that we could because we were such a small and scrappy team and because everybody who was working on the campaign at the time had a lot of responsibilities that they needed to see through on their own. And so getting into this, it's quite literally a full-time job having to sit down and follow up with volunteers and make sure that they have the appropriate amount of work and make sure that we've set up our environment appropriately so that volunteers can come and go and all of that kind of stuff, so yeah.Corey: It's always an interesting joy looking at the swath of architectural decisions and how they came to be. I talked on a previous episode with Jackie Singh, who was, I believe, after your tenure as CISO, she was involved on the InfoSec side of things, and she was curious as to your thought process or rationale with a lot of the initial architectural decisions that she talked about on her episode which I'm sure she didn't intend it this way, but I am going to blatantly miscategorize as, “Justify yourself. What were you thinking?” Usually it takes years for that kind of, “I don't understand what's going on here so I'm playing data center archeologist or cloud spelunker.” This was a very short window. How did decisions get made architecturally as far as what you're going to run things on? It's been disclosed that you were on AWS, for example. Was that a hard decision?Dan: No, not at all. Not at all. We started out the campaign—I in particular I was one of the first employees hired onto the campaign and the idea all along was that we're not going to be clever, right? We're basically just going to develop what needs to be developed. And the idea with that was that a lot of the code that we were going to sit down and write or a lot of the infrastructure that we were going to build was going to be glue, it not AWS Glue, right, ideally, but just glue that would bind data streams together, right?So, data movement, vendor A produces a CSV file for you and it needs to end up in a bucket somewhere. So, somebody needs to write the code to make that happen, or you need to find a sufficient vendor who can make that happen. There's a lot more vendors today believe it or not than there were two years ago that are doing much better in that kind of space, but two years ago we had the constraints of time and money.Our idea was that the code that we were going to write was going to be for those purposes. What it actually turned into is that in other areas of the business—and I will call it a business because we had formalized roadmaps and different departments working on different things—but in other areas of the business where we didn't have enough money to purchase a solution, we had the ability to go and write software.The interesting thing about this group of technologists who came together especially early on in the campaign to build out the tech team most of them came from an enterprise software development background, right? So, we had the know-how of how to build things at scale and how to do continuous delivery and continuous deployment, and how to operate a cloud-native environment, and how to build applications for that world.So, we ended up doing things like writing an API for managing our donor vetting pipeline, right? And that turned into a complex system of Lambda functions and continuous delivery for a variety of different services that facilitated that pipeline. We also built an architecture for our mobile app which there were plenty of companies that wanted to sell us a mobile app and we just couldn't afford it so we ended up writing the mobile app ourselves.So, after some point in time, what we said was we actually have a fairly robust and complex software infrastructure. We have a number of microservices that are doing various things to facilitate the operation of the business, and something that we need to do is we need to spend a little bit of time and make sure that we're building this in a cohesive way, right? And what part of that means was that, for example, we had to take a step back and say, “Okay, we need to have a unified identity service.” We can't have a different identity—or we can't have every single individual service creating its own identity. We need to have—Corey: I really wish you could pass that lesson out on some of the AWS service teams.Dan: [laugh]. Yes, I know. I know. Yeah. So, we went through—Corey: So, there were some questionable choices you made in there, like you started that with the beginning of, “Well, we had no time which is fine and no budget. So, we chose AWS.” It's like, “Oh, that looks like the exact opposite direction of a great decision, given, you know, my view on it.” Stepping past that entirely, you are also dealing with challenges that I don't think map very well to things that exist in the corporate world. For example, you said you had to build a donor vetting pipeline.It's in the corporate world I didn't have it. It's one of those, “Why in the world would I get in the way of people trying to give me money?” And the obvious answer in your case is, federal law, and it turns out that the best outcome generally does not involve serving prison time. So, you have to address these things in ways that don't necessarily have a one-to-one analog in other spaces.Dan: That's true. That's true. Yes, correct to the federal law thing. Our more pressing reason to do this kind of thing was that we made a commitment very early on in the campaign that we wouldn't take money from executives of the gas and oil industry, for example. There were another bunch of other commitments that were made, but it was inconceivable for us to have enough people that could possibly go manually through those filings. So, for us to be able to build an automated system for doing that meant that we were literally saving thousands of human hours and still getting a beneficial result out of it.Corey: And everything you do is subject to intense scrutiny by folks who are willing to make hay out of anything. If it had leaked at the time, I would have absolutely done some ridiculous nonsense thing about, “Ah, clearly looking at this AWS bill. Joe Biden's supports managed NAT gateway data processing pricing.” And it's absolutely not, but that doesn't stop people from making hay about this because headlines are going to be headlines.And do you have to also deal with the interesting aspect—industrial espionage is always kind of a thing, but by and large most companies don't have to worry that effectively half of the population is diametrically opposed to the thing it is that they're trying to do to the point where they might very well try to get insiders there to start leaking things out. Everything you do has to be built with optics in mind, working under tight constraints, and it seems like an almost insurmountable challenge except for the fact where you actually pulled it off.Dan: Yeah. Yeah. Yeah. We kept saying that the tech was not the story, right, and we wanted to do everything within our power to keep the conversation on the candidate and not on emails or AWS bills or any of that kind of stuff. And so we were very intentional about a lot of the decisions that we ended up making with the idea that if the optics are bad, we pull away from the primary mission of what it is that we're trying to do.Corey: So, what was it that qualified you to be the CTO of a—at the time very fledgling and uncertain campaign, given that you were coming from a role where you were a distinguished engineer, which is not nothing, let's be clear, but it's an executive-level of role rather than a hands-on level of role as CTO. And then if we go back in time, you were one of the founding developers of Spinnaker over at Netflix.And I have a lot of thoughts about Netflix technology and a lot of thoughts about Spinnaker as well, and none of those thoughts are, “This seems like a reasonable architecture I should roll out for a presidential campaign.” So, please, don't take this as the insult that probably sounds like, but why were you the CTO that got tapped?Dan: Great question. And I think in some ways, right place, right time. But in other ways probably needs to speak a little bit to the journey of how I've gotten anywhere in my career. So, going back to Netflix, yeah, so I worked in Netflix. I had the opportunity to work with a lot of incredibly bright and talented folks there. One of the people in particular who I met there and became friends with was Corey Bertram who worked on the core SRE team.Corey left Netflix to go off and at the time he was just like, “I'm going to go do a political startup.” The interesting thing about Netflix at the time—this was 2013, so, this was just after the Obama for America '12 campaign. And a bunch of folks from OFA world came and worked at Netflix and a variety of other organizations in the Bay Area. Corey was not one of those people but we were very well-connected with folks in that world, and Corey said he was going off to do a political startup, and so after my non-mutual departure from Netflix, I was talking to Corey and he said, “Hey, why don't you come over and help us figure out how to do continuous delivery over on the political startup.” That political startup turned into the groundwork which turned into essentially the tech platform for the Hillary for America campaign.So, I had the opportunity working for the groundwork to work very closely with the folks in the technology organization at HFA. And that got me more exposure to what that world is and more connections into that space. And the groundwork was run by Corey, but was the CEO or head—I don't even know what he called himself, was Michael Slaby, who was President Obama's CTO in 2008 and had a bigger technical role in the 2012 campaign.And so, for his involvement in HFA '16 meant that he was a person who was very well connected for the 2020 campaign. And when we were out at a political conference in late 2018 and he said, “Hey, I think that Vice President Biden is going to run. Do you have any interest in talking with his team?” And I said, “Yes, absolutely. Please introduce me.”And I had a couple of conversations with Greg Schultz who was the campaign manager and we just hit it off. And it was a really great fit. Greg was an excellent leader. He was a real visionary, exactly the person that President Biden needed. And he brought me in to set up the tech operation and get everything to where we ultimately won the primary and won the election after that.Corey: And then, as all things do, it ended and the question then becomes, “Great, what's next?” And the answer for you was apparently, “Okay, I'm going to go back to Target-ish.” Although now you're the CISO of a Target subsidiary, Shipt and Target's relationship is—again, I imagine I have that correct as far as you are in fact a subsidiary of Target, so it wasn't exactly a new company, but rather a transition into the previous organization you were in a different role.Dan: Yeah, correct. Yeah, it's a different department inside of Target, but my paycheck still come from Target. [laugh].Corey: So, what was it that inspired you to go into the CISO role? Because obviously security is everyone's job, which is what everyone says, which is why we get away with treating it like it's nobody's job because shared responsibilities tend to work out that way.Dan: Yeah.Corey: And you've done an awful lot of stuff that was not historically deeply security-centric although there's always an element passing through it. Now, going into a CISO role as someone without a deep InfoSec background that I'm aware of, what drove that? How did that work?Dan: You know, I think the most correct answer is that security has always been in my blood. I think like most people who started out—Corey: There are medications for that now.Dan: Yeah, [laugh] good. I might need them. [laugh]. I think like most folks who are kind of my era who started seriously getting into software development and computer system administration in the late ‘90s, early thousands, cybersecurity it wasn't called cybersecurity at the time. It wasn't even called InfoSec, right, it was just called, I don't know, dabbling or something. But that was a gateway for getting into Linux system administration, network engineering, so forth and so on.And for a short period of time I became—when I was getting my RHCE certification way back in the day, I became pretty entrenched in network security and that was a really big focus area that I spent a lot of time on and I got whatever the supplemental network security certification from Red Hat was at the time. And then I realized pretty quickly that the world isn't going to need box operators for very long, and this was just before the DevOps revolution had really come around and more and more things were automated.So, we were still doing hand deployments. I was still dropping WAR files onto a file system and restarting Apache. That was our deployment process. And I saw the writing on the wall and I said, “If I don't dedicate myself to becoming first and foremost a software engineer, then I'm not going to have a very good time in technology here.” So, I jumped out of that and I got into software development, and so that's where my software engineering career evolved out of.So, when I was CTO for the campaign, I like to tell people that I was a hundred percent of CTO, I was a hundred percent a CIO, and I was a hundred percent of CISO for the first 514 days of the campaign or whatever it was. So, I was 300 percent doing all of the top-level technology jobs for the campaign, but cybersecurity was without a doubt the one that we would drop everything for every single time.And that was by necessity; we were constantly under attack on the campaign. And a lot of my headspace during that period of time was dedicated to how do we make sure that we're doing things in the most secure way? So, when I left—when I came back into Target and I came back in as a distinguished engineer there were some areas that they were hoping that I could contribute positively and help move a couple of things along.The idea always the whole time was going to be for me to jump into a leadership position. And I got a call one day from Rich Agostino who's the CISO for Target and he said, “Hey, Shipt needs a cybersecurity operation built out and you're looking for a leadership role. Would you be interested in doing this?” And believe it or not, I had missed the world of cybersecurity so much that when the opportunity came up I said, “Yes, absolutely. I'll dive in head first.” And so that was the path for getting there.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance accelerator for the Oracle MySQL Database Service. Although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLTP and OLAP, don't ask me to ever say those acronyms again, workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: My take to cybersecurity space is, a little, I think, different than most people's journeys through it. The reason I started a Thursday edition of the Last Week in AWS newsletter is the security happenings in the AWS ecosystem for folks who don't have the word security in their job titles because I used to dabble in that space a fair bit. The problem I found is that is as you move up the ladder to executives that our directors, VPs, and CISOs, the language changes significantly.And it almost becomes a dialect of corporate-speak that I find borderline impenetrable, versus the real world terminology we're talking about when, “Okay, let's make sure that we rotate credentials on a reasonable expected basis where it makes sense,” et cetera et cetera. It almost becomes much more of a box-checking compliance exercise slash layering on as much as you possibly can that for plausible deniability for the inevitable breach that one day hits and instead of actually driving towards better outcomes.And I understand that's a cynical, strange perspective, but I started talking to people about this, and I'm very far from alone in that, which is why people are subscribing to that newsletter and that's the corner of the market I wanted to start speaking to. So, given that you've been an engineer practitioner trying to build things and now a security executive as well, is my assessment of the further higher up you go the entire messaging and purpose change, or is that just someone who's been in the trenches for too long and hasn't been on that side of the world, and I have a certain lack of perspective that would make this all very clear. Which I freely accept, if that's the case.Dan: No, I think that you're right for a lot of organizations. I think that that's a hundred percent true, and it is exactly as you described: a box-checking exercise for a lot of organizations. Something that's important to remember about Target is—Target was the subject of a data breach in 2012, and that was before there were data breaches every single day, right.Now, we look at a data breach and we say that's just going to happen, right, that's the cost of doing business. But back in 2012 it was really a very big story and it was a very big deal, and there was quite a bit of activity in the Target technology world after that breach. So, it reshaped the culture quite literally, new executives were brought in, but there's this whole world of folks inside of Target who have never forgotten that, right, and work day-in and day-out to make sure that we don't have another breach.So, security at Target is a main centrally thought about kind of thing. So, it's very much something that is a part of the way that people operate inside of Target. So, coming over to Shipt, obviously, Shipt is—it is a subsidiary. It is a part of Target, but it doesn't have that long history and hasn't had that same kind of experience. The biggest thing that we really needed at Shipt is first and foremost to get the program established, right. So, I'm three or four months onto the job now and we've tripled the team size. I've been—Corey: And you've stayed out of the headlines, which is basically the biggest and most accurate breach indicator I've found so far.Dan: So far so good. Well, but the thing that we want to do though is to be able to bring that same kind of focus of importance that Target has on cybersecurity into the world of engineering at Shipt. And it's not just a compliance game, and it's not just a thing where we're just trying to say that we have it. We're actually trying to make sure that as we go forward we've got all these best practices from an organization that's been through the bad stuff that we can adopt into our day-to-day and kind of get it done.When we talk about it at an executive level, obviously we're not talking about the penetration tests done by the red team the earlier day, right. We're not calling any of that stuff out in particular. But we do try to summarize it in a way that makes it clear that the thing that we're trying to do is build a security-minded culture and not just check some boxes and make sure that we have the appropriate titles in the appropriate places so that our insurance rates go down, right. We're actually trying to keep people safe.Corey: There's a lot to be said for that. With the Target breach back in—I want to say 2012, was it?Dan: 2012. Yep.Corey: Again, it was a wake-up call and the argument that I've always seen is that everyone is vulnerable—just depends on how much work it's going to take to get there. And for, credit where due, there was a complete rotation in the executive levels which whether that's fair or not, I—people have different opinions on it; my belief has always been you own the responsibility, regardless of who's doing the work.And there's no one as fanatical as a convert, on some level, and you've clearly been doing a lot of things in the right direction. The thing that always surprises me is that when I wind up seeing these surveys in the industry that—what is it? 65% of companies say that they would be vulnerable to a breach, and everybody said, “Oh, we should definitely look at those companies.” My argument is, “Hang on a sec. I want to talk to the 35% who say, ‘oh, we're impenetrable.'” because, spoiler, you are not.No one is. Just the question of how heavy is the lift and how much work is it going to take to get there? I do know that mouthing off in public about how perfect the security of anything is, is the best way to more or less climb to the top of a mountain during a thunderstorm, a hold up a giant metal rod, and curse the name of God. It doesn't lead to positive outcomes, basically ever. In turn, this also leads to companies not talking about security openly.I find that in many cases it is easier for me to get people to talk about their AWS bills than their InfoSec posture. And I do believe, incidentally, those two things are not entirely unrelated, but how do you view it? It was surprisingly easy to get Shipt's CISO to have a conversation with me here on this podcast. It is significantly more challenging in most other companies.Dan: Well, in fairness, you've been asking me for about two-and-a-half years pretty regularly [laugh] to come.Corey: And I always say I will stop bothering you if you want. You said, “No, no. Ask me again in a few months. Ask me again, after the election. Ask me again after—I don't know, like, the one-day delivery thing gets sorted out.” Whatever it happens to be. And that's fine. I follow up religiously, and eventually I can wear people down by being polite yet persistent.Dan: So, persistence on you is actually to credit here. No, I think to your question though, I think that there's a good balance. There's a good balance in being open about what it is that you're trying to do versus over-sharing areas that maybe you're less proficient in, right. So, it wouldn't make a lot of sense for me to come on here and tell you the areas that we need to develop into security. But on the other side of things, I am very happy to come in and talk to you about how our incident response plan is evolving, right, and what our plan looks like for doing all of that kind of stuff.Some of the best security practitioners who I've worked with in the world will tell you that you're not going to prevent a breach from a motivated attacker, and your job as CISO is to make sure that your response is appropriate, right, more so than anything. So, our incident response areas where today we're dedicating quite a bit of effort to build up our proficiency, and that's a very important aspect of the cybersecurity program that we're trying to build here.Corey: And unlike the early days of a campaign, you still have to be ultra-conscious about security, but now you have the luxury of actually being able to hire security staff because it turns out that, “Please come volunteer here,” is not presumably Shipt's hiring pitch.Dan: That's correct. Yeah, exactly. We have a lot of buy-in from the rest of leadership to build out this program. Shipt's history with cybersecurity is one where there were a couple of folks who did a remarkably good job for just being two or three of them for a really long period of time who ran the cybersecurity operation very much was not a part of the engineering culture at Shipt, but there still was coverage.Those folks left earlier in the year, all of them, simultaneously, unfortunately. And that's sort of how the position became open to me in the first place. But it also meant that I was quite literally starting with next to nothing, right. And from that standpoint it made it feel a lot like the early days of the campaign because I was having to build a team from scratch and having to get people motivated to come and work on this thing that had kind of an unknown future roadmap associated with it and all of that kind of stuff.But we've been very privileged to—because we have that leadership support we're able to pay market rates and actually hire qualified and capable and competent engineers and engineering leaders to help build out the aspects of this program that we need. And like I said, we've managed to—we weren't exactly at zero when I walked in the door. So, when I say we were able to quadruple the team, it doesn't mean that we just added four zeros there, [laugh] but we've got a little bit over a dozen people focusing on all areas of security for the business that we can think of. And that's just going to continue to grow. So, it's exciting; it's a challenge. But having the support of the entire organization behind something like this really, really helps a lot.Corey: I know we're running out of time for a lot of the interview, but one more question I want to ask you about is, when you're the CISO for a nationally known politician who is running for the highest office, the risk inherent to getting it wrong is massive. This is one of those mistakes will show indelibly for the rest of, well, one would argue US history, you could arguably say that there will be consequences that go that far out.On the other side of it, once you're done on the campaign you're now the CISO at Shipt. And I am not in any way insinuating that the security of your customers, and your partners, and your data across the board is important. But it does not seem to me from the outside that it has the same, “If we get this wrong there are repercussions that will extend into my grandchildren's time.” How do you find that your ability to care as deeply about this has changed, if it has?Dan: My stress levels are a lot lower I'll say that, but—Corey: You can always spot the veterans on an SRE team because—when I say veterans I mean veterans from the armed forces because, “No one's shooting at me. We can't serve ads right now. I'm really not going to run around and scream like, ‘My hair's on fire,' because this is nothing compared to what stress can look like.” And yeah there's always a worst stressor, but, on some level, it feels like it would be an asset. And again this is not to suggest you don't take security seriously. I want to be very clear on that point.Dan: Yeah, yeah, no. The important challenge of the role is building this out in a way that we have coverage over all the areas that we really need, right, and that is actually the kind of stuff that I enjoy quite a bit. I enjoy starting a program. I enjoy seeing a program come to fruition. I enjoy helping other people build their careers out, and so I have a number of folks who are at earlier at points in their career who I'm very happy that we have them on our team because I can see them grow and I can see them understand and set up what the next thing for them to do is.And so when I look at the day-to-day here, I was motivated on the campaign by that reality of like there is some quite literal life or death stuff that is going to happen here. And that's a really strong presser to make sure that you're doing all the right stuff at the right time. In this case, my motivation is different because I actually enjoy building this kind of stuff out and making sure that we're doing all the right stuff and not having the stress of, like, this could be the end of the world if we get this wrong.Means that I can spend time focusing on making sure that the program is coming together as it should, and getting joy from seeing the program come together is where a lot of that motivation is coming from today. So, it's just different, right? It's a different thing, but at the end of the day it's very rewarding and I'm enjoying it and can see this continuing on for quite some time.Corey: And I look forward to ideally getting you back in another two-and-a-half years after I began badgering you in two hours in order to come back on the show. If—Dan: [laugh].Corey: —people want to hear more about what you're up to, how you view about these things, potentially consider working with you, where can they find you?Dan: Best place although I've not been as active because it has been very busy the last couple of months, but find me on Twitter, @danveloper, find me on LinkedIn. Those—you know, I posted a couple of blog posts about the technology choices that we made on the campaign that I think folks find interesting, and periodically I'll share out my thoughts on Twitter about whatever the most current thing is, Kubernetes or AWS about to go down or something along those lines. So, yeah, that's the best way. And I tweet out all the jobs and post all the jobs that we're hiring for on LinkedIn and all of that kind of stuff. So, usual social channels. Just not Facebook.Corey: Amen to that. And I will of course include links to those things in the [show notes 00:37:29]. Thank you so much for taking the time to speak with me. I appreciate it.Dan: Thank you, Corey.Corey: Dan Woods, CISO and VP of Cybersecurity at Shipt, also formerly of the Biden campaign because wherever he goes he clearly paints a target on his back. I'm Cloud Economist, Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast please leave a five-star review on your podcast platform of choice along with an incoherent rant that is no doubt tied to either politics or the alternate form of politics: Spinnaker.Dan: [laugh].Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

On The Cloud Pod this week, Oracle finally has some news to share. Plus Log4j is ruining everyone's lives, AWS suffers a massive outage post re:Invent, and Google CAT releases its first threat report.  A big thanks to this week's sponsors: Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. JumpCloud, which offers a complete platform for identity, access, and device management — no matter where your users and devices are located.  This week's highlights

About JakeTechnical Lead by day at the Met Office in the UK, leading a team of software developers delivering services for the UK. By night, gamer and fitness instructor, attempting to get a home cinema and gaming setup whilst coralling 3 cats, 2 rabbits, 2 fish tanks, and my wonderful girlfriend.Links: Met Office: https://www.metoffice.gov.uk Twitter: https://twitter.com/jakehendy TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: It seems like there is a new security breach every day. Are you confident that an old SSH key, or a shared admin account, isn't going to come back and bite you? If not, check out Teleport. Teleport is the easiest, most secure way to access all of your infrastructure. The open source Teleport Access Plane consolidates everything you need for secure access to your Linux and Windows servers—and I assure you there is no third option there. Kubernetes clusters, databases, and internal applications like AWS Management Console, Yankins, GitLab, Grafana, Jupyter Notebooks, and more. Teleport's unique approach is not only more secure, it also improves developer productivity. To learn more visit: goteleport.com. And not, that is not me telling you to go away, it is: goteleport.com. Corey: This episode is sponsored in part by our friends at Redis, the company behind the incredibly popular open source database that is not the bind DNS server. If you're tired of managing open source Redis on your own, or you're using one of the vanilla cloud caching services, these folks have you covered with the go to manage Redis service for global caching and primary database capabilities; Redis Enterprise. To learn more and deploy not only a cache but a single operational data platform for one Redis experience, visit redis.com/hero. Thats r-e-d-i-s.com/hero. And my thanks to my friends at Redis for sponsoring my ridiculous non-sense.  Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. It's often said that the sun never sets on the British Empire, but it's often very cloudy and hard to see the sun because many parts of it are dreary and overcast. Here to talk today about how we can predict those things in advance—in theory—is Jake Hendy, Tech Lead at the Met Office. Jake, thanks for joining me.Jake: Hey, Corey, it's lovely to be here. Thanks for inviting me on.Corey: There's a common misconception that its startups in San Francisco or the culture thereof, if you can even elevate it to being a culture above something you'd find in a petri dish, that is where cloud stuff happens, where the computer stuff is done. And I've always liked cutting against that. There are governments that are doing interesting things with Cloud; there are large companies and ‘move fast and break things' is the exact opposite of what you generally want from institutions that date back centuries. What's it like working on Cloud, something that for all intents and purposes didn't exist 20 years ago, in the context of a government office?Jake: As you can imagine, it was a bit of a foray into cloud for us when it first came around. We weren't one of the first people to jump. The Met Office, we've got our own data centers, which we've proudly sit on that contains supercomputers and mainframes as well as a plethora of x86 hardware. So, we didn't move fast at the start, but nowadays, we don't move at breakneck speeds, but we like to take advantage of those managed services. It gets out of the way of managing things for us.Corey: Let's back up a second because I tend to be stereotypically American in many ways. What is the Met Office?Jake: What is the Met Office? The Met Office is the UK's National Meteorological Service. And what does that mean? We do a lot of things though with meteorology, from weather forecasting and climate research from our Hadley Centre—which is world-renowned—down to observations, collections, and partnerships around the world. So, if you've been on a plane over Europe, the Middle East, Africa, over parts of Asia, that plane took off because the Met Office provided a forecast for that plane. There's a whole range of things we can talk about there, if you want Corey, of what the Met Office actually does.Corey: Well, let's ask some of the baseline questions. You think of a weather office in a particular country as, oh okay, it tracks the weather in the area of operations for that particular country. Are you looking at weather on a global basis, on a somewhat local basis, or—as mentioned—since due to a long many-century history it turns out that there are UK Commonwealth territories scattered around the globe, where do you start? Where do you stop?Jake: We don't start and we don't stop. The Met Office is very much a 24/7 operation. So, we've got a 24/7 operation center with staff constantly manning it, doing all sorts of things. So, we've got a defense, we work heavily with our defense colleagues from UK armed forces to NATO partners; we've got aviation, as mentioned; we've got marine shipping from—most of the listeners in the UK will have heard of the shipping forecast at one point or another. And we've got private sector as well, from transport, to energy, supermarkets, and more. We have a very heavy UK focus, for obvious reasons, but our remit goes wide. You can actually go and see some of our model data is actually on Amazon Open Data. We've got MOGREPS, which is our ensemble forecast, as well as global models and UK models, with a 24-hour time lag, but feel free to go and have a play. And you can see the wide variety of data that we produce in just those few models.Corey: Yeah, just pulling up your website now; looking at where I am here in San Francisco, it gives me a detailed hour-by-hour forecast. There are only two problems I see with it. The first is that it's using Celsius units, which I—Jake: [laugh].Corey: —as a matter of policy, don't believe in because in this country, we don't really use things that make sense in measuring context. And also, I don't believe it's a real weather site because it's not absolutely festooned with advertisements for nonsense, which is apparently—I wasn't aware—a thing that you could have on the internet. I thought that showing weather data automatically meant that you had to attempt to cater to the lowest common denominator at all times.Jake: That's an interesting point there. So, the Met Office is owned and operated by Her Majesty's Government. We are a Trading Fund with the Department for Business, Energy and Industrial Strategy. But what does that mean it's a Trading Fund?k it means that we're funded by public money. So, that's called the Public Weather Service.But we also offer a more commercial venture. So, depending on what extensions you've got going on in your browser, there are actually adverts that do run on our website, and we do this to help recover some of the cost. So, the Public Weather Service has to recover some of that. And then lots of things are funded by the Public Weather Service, from observations, to public forecasting. But then there are more those commercial ventures such as the energy markets that have more paid products, and things like that as well. So, maybe not that many adverts, but definitely more usable.Corey: Yeah, I disabled the ad blocker, and I'm reloading it and I'm not seeing any here. Maybe I'm just considered to be such a poor ad targeting prospect at this point that people have just given up in despair. Honestly, people giving up on me in despair is kind of my entire shtick.Jake: We focus heavily on user-centered design, so I was fortunate in their previous team to work in our digital area, consumer digital, which looked after our web and mobile channels. And I can heartily say that there are a lot of changes, had a lot of heavy research into them. Not just internal, getting [unintelligible 00:06:09] and having a look at it, but what does this is actually mean for members of the? Public sending people out doing guerrilla public testing, standing outside Tescos—which is one of our large superstores here—and saying, “Hey, what do you think of this?” And then you'd get a variety of opinions, and then features would be adjusted, tweaked, and so on.Corey: So, you folks have been a relatively early adopter, especially in an institutional context. And by institution, I mean, one of those things that feels like it is as permanent as the stones in a castle, on some level, something that's lasted more than 20 years here in California, what a concept. And part of me wonders, were you one of the first UK government offices to use the cloud, and is that because you do weather and someone was very confused by what Cloud meant?Jake: [laugh]. I think we were possibly one of the first; I couldn't say if we were the first. Over in the UK, we've got a very capable network of government agencies doing some wonderful, and very cloud things. And the Government Digital Service was an initiative set up—uh, I can't remember, and I—unfortunately I can't remember the name of the report that caused its creation, but they had a big hand in doing design and cloud-first deployments. In the Met Office, we didn't take a, “Ah, screw it. Let's jump in,” we took a measured step into the cloud waters.Like I said, we've been running supercomputers since the '50s, and mainframes as well, and x86. I mean, we've been around for 100 years, so we constantly adapt, and engage, and iterate, and improve. But we don't just jump in and take a risk because like you said, we are an institution; we have to provide services for the public. It's not something that you can just ignore. These are services that protect life and property, both at home and abroad.Corey: You have provided a case study historically to AWS, about your use cases of what you use, back in 2014. It was, oh, you're a heavy user of EC2, and looking at the clock, and oh, it's 2014. Surprise. But you've also focused on other services as well. I believe you personally provided a bit of a case study slash story of round your use of Pinpoint of all things, which is a wrapper around SES, their email service, in the hopes of making it a little bit more, I guess, understandable slash fully-featured for contacting people, but in my experience is a great sales device to drive business to its competitors.What's it been like working, I guess, both simultaneously with the tried and true, tested yadda, yadda, yadda, EC2 RDS style stuff, but then looking at what else you're deep into Lambda, and DynamoDB, and SQS sort of stands between both worlds give it was the first service in beta, but it also is a very modern way of thinking about services. How do you contextualize all of that? Because AWS has product strategies, clearly, “Yes.” And they build anything for anyone is more or less what it seems. How do you think about the ecosystem of services that are available and apply it to problems that you're working on?Jake: So, in my personal opinion, I think the Met Office is one of a very small handfuls of companies around the world that could use every Amazon service that's offered, even things like Ground Station. But on my first day in the office, I went and sat at my desk and was talking to my new colleagues, and I looked to the left and he said, “Oh, yeah, that's a satellite dish collecting data from a satellite passing overhead.” So, we very much pick the best tool for the job. So, we have systems which do heavy number crunching, and very intense things, we'll go for EC2.We have systems that store data that needs relationships and all sorts of things. Fine, we'll go RDS. In my space, we have over a billion observations a year coming through the system I lead on SurfaceNet. So, do we need RDS? No. What about if we use something like S3 and Glue and Athena to run queries against this?We're very fortunate that we can pick the best tool for the job, and we pride ourselves on getting the most out of our tools and getting the most value for money. Because like I said, we're funded by the taxpayer; the taxpayer wants value for money, and we are taxpayers ourselves. We don't want to see our money being wasted when we got a hundred size auto-scaling group, when we could do it with Lambda instead.Corey: It's fascinating talking about some of the forward-looking stuff, and oh, serverless and throw everything at Cloud and be all in on cloud. Cloud, cloud, cloud. Cloud is the future. But earlier this year, there was a press release where the Met Office and Microsoft are going to be joining forces to build the world's, and I quote, “Most powerful weather and climate forecasting supercomputer.” The government—your government, to be clear—is investing over a billion pounds in the project.It is slated to be online and running by the middle of next year, 2022, which for a government project as I contextualize them feels like it's underwear-on-outside-the-pants superhero speed. But that, I guess, is what happens when you start looking at these public-private partnerships in some respects. How do you contextualize that? What is the story behind, oh, we're—you're clearly investing heavily in cloud, but you're also building your own custom enormous supercomputer rather than just waiting for AWS to drop one at re:Invent. What is the decision-making process look like? What is the strategy behind it?Jake: Oh. [laugh]. So—I'll have to be careful here—supercomputing is something that we've been doing for a long time, since the '50s, and we've grown with that. When the Met Office moved offices from Bracknell in 2002, 2003, we run two supercomputers for operational resilience, at that point [unintelligible 00:12:06] building in the new building; it was ready, and they were like, “Okay, let's move a supercomputer.” So, it came hurtling down the motorway, plugged in, and congrats, we've now got two supercomputers running again. We're very fortunate—Corey: We had one. It got lonely. We wanted to make it a friend. Yeah, I get it.Jake: Yeah. It's long distance; it works. And the Met Office is actually very good at running projects. We've done many supercomputers over the years, and supercomputing our models, we run some very intense models, and we have more demands. We know we can do better.We know there's the observations in my group we collect, there's the science that's continually improving and iterating and getting better, and our limit isn't poor optimizations or poorly written code. They're scientists running some fantastic code; we have a team who go and optimize these models, and you know, in one release, they may knock down a model runtime by four minutes. And you think, okay, that's four minutes, but for example, if that's four minutes across 400 nodes, all of a sudden you've now got 400 nodes that have then got four minutes more of compute. That could be more research, that could be a different model run. You know, we're very good at running these things, and we're very fortunate with very technically capable to understand the difference between a workload that belongs on AWS, a workload that belongs on a supercomputer.And you know, a supercomputer has many benefits, which the cloud providers… are getting into, you know, we have a high performance clusters on Amazon and Azure, or with, you know, InfiniBand networking. But sometimes you really can't beat a hunking great big ton of metal and super water-cooling, sat in a data center somewhere, backed by—we're very fortunate to have one hundred percent renewable energy for the supercomputer, which is—if you look at any of the power requirements for a supercomputer is phenomenal, so we're throwing that credentials behind it for climate change as well. You can't beat a supercomputer sometimes.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance accelerator for the Oracle MySQL Database Service. Although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLTP and OLAP, don't ask me to ever say those acronyms again, workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense. Corey: I'm somewhat fortunate in the despite living in a world of web apps, these days, my business partner used to work at the Department of Energy at Oak Ridge National Lab, helping with the care and feeding of the supercomputer clusters that they had out there. And you're absolutely right; that matches my understanding with the idea that there are certain workloads you're not going to be able to beat just having this enormous purpose-built cluster sitting there ready to go. Or even if you can, certainly not economically. I have friends who are in the batch side of the world, the HPC side of the world over in the AWS organizations, and they keep—“Hey, look at this. This thing's amazing.”But so much of what they're talking about seems to distill down to, “I have this one-off giant compute task that needs to get done.” Yes, you're right. If I need to calculate the weather one time, then okay, I can make an argument for going with cloud but you're doing this on what appears to be a pretty consistent basis. You're not just assuming—as best I can tell that, “And starting next Wednesday, it will be sunny forever. The end.”Jake: I'm sure many people would love it if we could do weather on-demand.Corey: Oh, yes. [unintelligible 00:15:09] going to reserved instance weather. That would be great. Like, “All right. I'd like to schedule some rain, please.” It really seems like it's one of those areas that is one of the most commonly accepted in science fiction without any real understanding of just what it would take to do something like that. Even understanding and predicting the weather is something that is beyond an awful lot of our current capabilities.Jake: This is exactly it. So, the Met Office is world-renowned for its research capabilities and those really in-depth, very powerful models that we run. So, I mentioned earlier, something called MOGREPS, which is the Met Office's ensemble-based models. And what do we mean by ensembles? You may see in the documentation it's got 18 members.What does that mean? It means that we actually run a simulation 18 times, and we tweak the starting parameters based on these real world inputs. And then you have a number of members that iterate through and supercomputer runs all of them. And we have deterministic models, which have one set of inputs. And you know, it's not just, as you say, one time; these models must run.There are a number of models we do, models on sea state as well, and they've all got to run, so we generally tend to run our supercomputers at top capacity. It's not often you get to go on a supercomputer and there'll be some space for your job to execute right this minute. And there's all the setup as well, so it's not just okay, the supercomputer is ready to go, but there's all the things that go into it, like, those observations, whether it's from the surface, whether it's from satellite data passing overhead, we have our own lightning network, as well. We have many things, like a radar network that we own, and operate. We collaborate with the environment agency for rainfall. And all these things they feed into these models.Okay, now we produce a model, and now it's got to go out. So, it's got to come off the supercomputer, it's got to be processed, maybe the grid that we run the models on needs to be reprojected because different people feed maps in different ways. Then there's got to be cut up because not every customer wants to know what the weather is everywhere. They've got a bit they care about. And of course, these models aren't small; you know, they can be terabytes, so there's also a case of customers might not want to download terabytes; that might cost them a lot. They might only be able to process gigabytes an hour.But then there's other products that we do processing on, so weather models, it might take 40 minutes to over an hour for a model to run. Okay, that's great. You might have missed the first step. Okay, well, we can enrich it with other data that's come in, things like nowcasting, where we do very short runs for the next six-hour forecast. There's a whole number of things that run in the office. And we don't have a choice; they run operationally 24/7, around the clock.I mentioned to you before we started recording, we had an incident of ‘Beast from the East' a number of years back. Some of your listeners may remember this; in the UK, we had a front come in from the east and the UK was blanketed with snow. It was a real severe event. We pretty much kept most of our services running. We worked really hard to make sure that they continued working.And personally I say, perhaps when you go shopping for Black Friday, you might go to a retailer and it's got a queue system up because, you know, it mimics that queue thing when you're outside a store, like in Times Square, and it's raining, be like oh, I might get a deal a minute. I think possibly in the Met Office, we have almost the inverse problem. If the weather's benign, we're still there. People rely on us to go, “Yeah, okay. I can go out and have fun.” When the weather's bad, we don't have a choice. We have to be there because everybody wants us to be there, but we need to be there. It's not a case of this is an optional service.Corey: People often forget that yeah, we are living in a world in which, especially with climate change doing what it's doing, if you get this wrong, people can very easily die. That is not something to take lightly. It's not just about can I go outside and play a pickup game of basketball today?Jake: Exactly. So, you know, operationally, we have something called the National Severe Weather Warning Service, where we issue guidance and alerts across the UK, based on severe weather. And there's a number of different weather types that we issued guidance for. And the severity of that goes from yellow to amber to red. And these are manually generated products, so there's the chief meteorologist who's on shift, and he approves these.And these warnings don't just go out to the members of the public. They go out to Cabinet Office, they go out to first responders, they go out to a number of people who are interested in the weather and have a responsibility. But the other side is that we don't issue a weather warning willy-nilly. It's a measured, calculated decision by our very capable operations team. And once that weather system has passed, the weather story has changed, we'll review it. We go back and we say what could we have done differently?Could the models have predicted this earlier? Could we have new data which would have picked up on this? Some of our next generation products that are in beta, would they have spotted this earlier? There's a lot of service review that continually goes on because like I said, we are the best, and we need to stay the best. People rely on us.Corey: So, here's a question that probably betrays my own ignorance, and that's okay, that's what I'm here to do. When I was a kid, I distinctly remember—first, this is not the era wish the world was black and white; I'm a child of the '80s, let's be clear here, so this is not old-timey nonsense quite as much, but distinctly remember that it was a running gag how unreliable the weather report always was, and it was a bit hit or miss, like, “Well, the paper says it's going to be sunny today, but we're going to pack an umbrella because we know how this works.” It feels, and I could be way off base on this, but it really feels like weather forecasting has gotten significantly more accurate since I was a kid. Is that just nostalgia, and I remember my parents complaining about it, or has there been a qualitative improvement in the accuracy of weather forecasting?Jake: I wish I could tell you all the scientific improvements that we've made, but there's many groups of scientists in the office who I would more than happily shift that responsibility over to, but quite simply, yes. We have a lot of partners we work with around the world—the National Weather Service, DWD in Germany, Meteo France, just to name but a few; there are many—and we all collaborate with data. We all iterate. You know, the American Meteorological Society holds a conference every year, which we attend. And there have been absolutely leaping changes in forecast quality and accuracy over the years.And that's why we continually upgrade our supercomputers. Like I said, yeah, there's research and stuff, but we're pulling in all this science and Meteorology is generally very chaotic systems. We're still discovering many things around how the climate works and how the weather systems work. And we're going to use them to help improve quality of life, early warnings, actually, we can say, oh, in three days time, it's going to be sunny at the beach. Be great if you could know that seven days in advance. It would be great if you knew that 14 days in advance.I mean, we might not do that because at the moment, we might have an idea, but there's also the case of understanding, you know, it's a probability-based decision. And people say, “Oh, it's not going to rain.” But actually, it's a case of, well, we said there's a 20% probability is going to rain. That doesn't mean it's not going to, but it's saying, “Two times out of ten, at this time it's going to rain.” But of course, if you go out 14 days, that's a long lead time, and you know, you talk about chaos theory, and the butterfly moves and flaps its wings, and all of a sudden a [cake 00:22:50] changes color from green to pink or something like that, some other location in the world.These are real systems that have real impacts, so we have to balance out the science of pure numbers, but what do people do with it? And what can people do with it, as well? So, that's why we talk about having timely data as well. People say, “Well, you could run these simulations and all your products take longer to process them and generate them,” but for example, in SurfaceNet, we have five minutes to process an observation once it comes in. We could spend hours fine-tuning that observation to make it perfect, but it needs to be useful.Corey: As you take a look throughout all of the things that AWS is doing—and sure, not all of these are going to necessarily apply directly to empowering the accuracy of weather forecasts, let's be clear here—but you have expressed personal interest in for example, IoT, a bunch of the serverless nonsense we're seeing out there. What excites you the most? What has you the most enthusiastic about what the future the cloud might hold? Because unlike almost everyone else I talk to in this space, you are not selling anything. You don't have a position—that I'm aware of—that oh, yeah, I super want to see this particular thing win the industry because that means you get to buy a boat.You work for the Met Office; you know that in some cases, oh, that boat is not going to have a great time in that part of the world anyway. I don't need one. So, you're a little bit more objective than most people. I have pushing a corporate story. What excites you? Where do you see the future of this industry going in ways that are neat?Jake: Different parts of the office will tell you different things, you know. We worked with Google DeepMind on AI and machine learning. We work with many partners on AI and machine learning, we use it internally, as well. On a personal level, I like quality of life improvements and things that just make my life as both the developer fun and interesting. So, CDK was a big thing.I was a CloudFormation wizard—still hate writing YAML—but the CDK came along and it was [unintelligible 00:24:52] people wouldn't say, but that wasn't, like, know when Lambda launched back in, what, 2013? 2014? No, but it made our lives easier. It meant that actually, we didn't have to worry about, okay, how do we do templating with YAML? Do we have to run some pre-processes or something?It meant that we could invest a little bit of time upfront on CDK and migrating everything over, and then that freed us up to actually doing things that we need for what we call the business or the organization, delivering value, you know? It's great playing with tech but, you know, I need to deliver value. And I think, what was it, in the Google SRE book, they limit the things they do, toiling of manual tasks that don't really contribute anything, they're more like keeping the lights on. Let's get rid of that. Let's focus on delivering value.It's why Lambda is so great. I could patch an EC2, I can automate it, you know, you got AWS Systems Manager Patch Manager, or… whatever its name is, they can go and manage all those patches for you. Why when I can do it in a Lambda and I don't need to worry about it?Corey: So, one last question that I have for you is that you're a tech lead. It's easy for folks to fall into the trap of assuming, “Oh, you're a government. It's like an enterprise only bigger, slower, and way, way, way busier.” How many hundreds of thousands of engineers are working at the Met Office along with you?Jake: So, you can have a look at our public report and you can see the number of staff we have. I think there's about 1800 staff that work at the Met Office. And that includes our account manage, that includes our scientists, that includes HR and legal. And I'd say there's probably less than 300 people who work in technology, as we call it, which is managing our IT estate, managing our Linux estate, managing our storage area networks because, funnily enough, managing petabytes of data is not an easy thing. You know, managing a supercomputer, a mainframe.There really aren't that many people here at the office, but we do so much great stuff. So, as a technical lead, I'm not just a leader of services, but I lead a team of people. I'm responsible for them, for empowering them, and helping them to develop their own careers and their own training. So, it's me and a team of four that look after SurfaceNet. And it's not just SurfaceNet; we've got other systems we look after that SurfaceNet produces data for. Sending messages around the world on the World Meteorological Organization's global telecommunications system. What a mouthful. But you know, these messages go all around the world. And some people might say, “Well, I got a huge team for that.” Well, [unintelligible 00:27:27]. We have other teams that help us—I say, help us—in their own right, they transmit that data. But we're really—I personally wouldn't say we were huge, but boy, do we pack a punch.Corey: Can I just say on a personal note, it's so great to talk to someone who's focusing on building out these environments and solving these problems for a higher purpose slash calling than—and I will get letters for this—than showing ads to people on the internet. I really want to thank you for taking time out of your day to speak with me. If people want to learn more about what you're up to, how you do it, potentially consider maybe joining you if they are eligible to work at the Met Office, where can they find you?Jake: Yeah, so you do have to be a resident in the UK, but www.metoffice.gov.uk is our home on the internet. You can find me on Twitter at @jakehendy, and I could absolutely chew Corey's ear off for many more hours about many of the wonderful services that the Met Office provides. But I can tell he's got something more interesting to do. So, uh [crosstalk 00:28:29]—Corey: Oh, you'd be surprised. It's loads of fun to—no, it's always fun to talk to people who are just in different areas that I don't get to work with very often. It turns out that most of my customers are not focused on telling you what the weather is going to do. And that's fine; it takes all kinds. It's just neat to have this conversation with a different area of the industry. Thank you so much for being so generous with your time. I appreciate it.Jake: Thank you very much for inviting me on. I guess if we get some good feedback, I'll have to come on and I will have to chew your ear off after all.Corey: Don't offer if you're not serious.Jake: Oh, I am.Corey: Jake Hendy, Tech Lead at the Met Office. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with a comment yelling at one or both of us for having the temerity to rain on your parade.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.