Podcast appearances and mentions of corey quinn

Episode SummaryChris Farris, Cloud Security Nerd at Turbot, joins Corey on Screaming in the Cloud to discuss the latest events in cloud security, which leads to an interesting analysis from Chris on how legal departments obscure valuable information that could lead to fewer security failures in the name of protecting company liability, and what the future of accountability for security failures looks like. Chris and Corey also discuss the newest dangers in cloud security and billing practices, and Chris describes his upcoming cloud security conference, fwd:cloudsec. About ChrisChris Farris has been in the IT field since 1994 primarily focused on Linux, networking, and security. For the last 8 years, he has focused on public-cloud and public-cloud security. He has built and evolved multiple cloud security programs for major media companies, focusing on enabling the broader security team's objectives of secure design, incident response and vulnerability management. He has developed cloud security standards and baselines to provide risk-based guidance to development and operations teams. As a practitioner, he's architected and implemented multiple serverless and traditional cloud applications focused on deployment, security, operations, and financial modeling.Chris now does cloud security research for Turbot and evangelizes for the open source tool Steampipe. He is one of the organizers of the fwd:cloudsec conference (https://fwdcloudsec.org) and has given multiple presentations at AWS conferences and BSides events.When not building things with AWS's building blocks, he enjoys building Legos with his kid and figuring out what interesting part of the globe to travel to next. He opines on security and technology on Mastodon, Twitter and his website https://www.chrisfarris.comLinks Referenced: Turbot: https://turbot.com/ fwd:cloudsec: https://fwdcloudsec.org/ Mastodon: https://infosec.exchange/@jcfarris Personal website: https://chrisfarris.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn and we are here today to learn exciting things, steal exciting secrets, and make big trouble for Moose and Squirrel. Maybe that's the podcast; maybe that's the KGB, we're not entirely sure. But I am joined once again by Chris Farris, cloud security nerd at Turbot, which I will insist on pronouncing as ‘Turbo.' Chris, thanks for coming back.Chris: Thanks for having me.Corey: So, it's been a little while and it's been an uneventful time in cloud security with nothing particularly noteworthy happening, not a whole lot of things to point out, and honestly, we're just sort of scraping the bottom of the barrel for news… is what I wish I could say, but it isn't true. Instead, it's, “Oh, let's see what disastrous tire fire we have encountered this week.” What's top of mind for you as we record this?Chris: I think the most interesting one I thought was, you know, going back and seeing the guilty plea from Nickolas Sharp, who formerly was an employee at Ubiquiti and apparently had, like, complete access to everything there and then ran amok with it.Corey: Mm-hm.Chris: The details that were buried at the time in the indictment, but came out in the press releases were he was leveraging root keys, he was leveraging lifecycle policies to suppress the CloudTrail logs. And then of course, you know, just doing dumb things like exfiltrating all of this data from his home IP address, or exfiltrating it from his home through a VPN, which have accidentally dropped and then exposed his home IP address. Oops.Corey: There's so much to dive into there because I am not in any way shape or form, saying that what he did was good, or I endorse any of those things. And yeah, I think he belongs in prison for what he did; let's be very clear on this. But I personally did not have a business relationship with him. I am, however, Ubiquiti's customer. And after—whether it was an insider threat or whether it was someone external breaching them, Krebs On Security wound up doing a whole write-up on this and was single-sourcing some stuff from the person who it turned out, did this.And they made a lot of hay about this. They sued him at one point via some terrible law firm that's entire brand is suing media companies. And yeah, just wonderful, wonderful optics there and brilliant plan. But I don't care about the sourcing. I don't care about the exact accuracy of the reporting because what I'm seeing here is that what is not disputed is this person, who whether they were an employee or not was beside the point, deleted all of the audit logs and then as a customer of Ubiquiti, I received an email saying, “We have no indication or evidence that any customer data was misappropriated.” Yeah, you just turn off your logs and yeah, you could say that always and forever and save money on logging costs. [unintelligible 00:03:28] best practice just dropped, I guess. Clowns.Chris: So, yeah. And there's definitely, like, compliance and standards and everything else that say you turn on your logs and you protect your logs, and service control policies should have been able to detect that. If they had a security operations center, you know, the fact that somebody was using root keys should have been setting off red flags and causing escalations to occur. And that wasn't happening.Corey: My business partner and I have access to our AWS org, and when I was setting this stuff up for what we do here, at a very small company, neither of us can log in with root credentials without alarms going off that alert the other. Not that I don't trust the man; let's be very clear here. We both own the company.Chris: In business together. Yes.Corey: Ri—exactly. It is, in many ways, like a marriage in that one of us can absolutely ruin the other without a whole lot of effort. But there's still the idea of separation of duties, visibility into what's going on, and we don't use root API keys. Let me further point out that we are not pushing anything that requires you to send data to us. We're not providing a service that is software powered to people, much less one that is built around security. So, how is it that I have a better security posture than Ubiquiti?Chris: You understand AWS and in-depth cloud better. You know, it really comes down to how do you, as an AWS customer, understand all of the moving parts, all of the security tooling, all of the different ways that something can happen. And Amazon will say, “Well, it's in the documentation,” but you know, they have, what, 357 services? Are you reading the security pages of all of those? So, user education, I agree, you should have, and I have on all of my accounts, if anything pops up, if any IAM change happens, I'm getting text messages. Which is great if my account got compromised, but is really annoying when I'm actually making a change and my phone is blowing up.Corey: Yeah. It's worth pointing out as well that yes, Ubiquiti is publicly traded—that is understood and accepted—however, 93% of it is owned by their CEO-founder god-king. So, it is effectively one person's personal fiefdom. And I tend to take a very dim view as a direct result. When you're in cloud and you have suffered a breach, you have severely screwed something up somewhere. These breaches are never, “Someone stole a whole bunch of drives out of an AWS data center.” You have misconfigured something somewhere. And lashing out at people who reported on it is just a bad look.Chris: Definitely. Only error—now, of course, part of the problem here is that our legal system encourages people to not come forward and say, “I screwed up. Here's how I screwed up. Everybody come learn from my mistakes.” The legal professions are also there to manage risk for the company and they're like, “Don't say anything. Don't say anything. Don't even tell the government. Don't say anything.”Whereas we all need to learn from these errors. Which is why I think every time I do see a breach or I do see an indictment, I start diving into it to learn more. I did a blog post on some of the things that happened with Drizly and GitHub, and you know, I think the most interesting thing that came out of Drizly case was the ex-CEO of Drizly, who was CEO at the time of the breach, now has following him, for the rest of his life, an FTC order that says he must implement a security program wherever he goes and works. You know, I don't know what happens when he becomes a Starbucks barista or whatever, but that is on him. That is not on the company; that is on him.And I do think that, you know, we will start seeing more and more chief executive officers, chief security or information security officers becoming accountable to—or for the breaches and being personally accountable or professionally accountable for it. I think we kind of need it, even though, you know, there's only so much a CISO can do.Corey: One of the things that I did when I started consulting independently on AWS bills back in 2016 was, while I was looking at customer environments, I also would do a quick check for a few security baseline things. And I stopped doing it because I kept encountering a bunch of things that needed attention and it completely derailed the entire stated purpose of the engagement. And, frankly, I don't want to be running a security consultancy. There's a reason I focus on AWS bills. And people think I'm kidding, but I swear to you I'm not, when I say that the reason is in part because no one has a middle-of-the-night billing emergency. It is strictly a business-hours problem. Whereas with security, wake up.In fact, the one time I have been woken up in the middle of the night by a customer phone call, they were freaking out because it was a security incident and their bill had just pegged through the stratosphere. It's, “Cool. Fix the security problem first, then we'll worry about the bill during business hours. Bye.” And then I stopped leaving my phone off of Do Not Disturb at night.Chris: Your AWS bill is one of your indicators of compromise. Keep an eye on it.Corey: Oh, absolutely. We've had multiple engagements discover security issues on that. “So, what are these instances in Australia doing?” “We don't have anything there.” “I believe you're being sincere when you say this.”Chris: Yes.Corey: However.Chris: “Last month, you're at $1,000 and this month, you're at $50,000. And oh, by the way, it's the ninth, so you might want to go look at that.”Corey: Here's the problem that you start seeing in large-scale companies though. You or I wind up posting our IAM credentials on GitHub somewhere in public—and I do this from time to time, intentionally with absolutely no permissions attached to a thing—and I started look at the timeline of, “Okay 3, 2, 1, go,” with the push and now I start counting. What happens? At what time does the quarantine policy apply? When do I get an email alert? When do people start trying to exploit it? From where are they trying to exploit it?It's a really interesting thing to look into, just from the position of how this stuff all fits together and works. And that's great, but there's a whole ‘nother piece to it where if you or I were to do such a thing and actually give it admin credentials, okay, my, I don't know, what, $50, $100 a month account that I use for a lot of my test stuff now starts getting charged enormous piles of money that winds up looking like a mortgage in San Francisco, I'm going to notice that. But if you have a company that spending, I don't know, between ten and $20 million a month, do you have any idea how much Bitcoin you've got to be mining in that account to even make a slight dent in the overall trajectory of those accounts?Chris: In the overall bill, a lot. And in a particularly mismanaged account, my experience is you will notice it if you're monitoring billing anomalies on a per-account basis. I think it's important to note, you talked about that quarantine policy. If you look at what actually Amazon drops a deny on, it's effectively start EC2 instances and change IAM policies. It doesn't prevent anybody from listing all your buckets and exfiltrating all your data. It doesn't prevent anybody from firing up Lambdas and other less commonly used resources. Don't assume oh, Amazon dropped the quarantine policy. I'm safe.Corey: I was talking to somebody who spends $4 a month on S3 and they wound up suddenly getting $60 grand a day and Lambda charges, because max out the Lambda concurrency in every region and set it to mine crypto for 15 minutes apiece, yeah, you'll spend $60,000 a day to get, what $500 in crypto. But it's super economical as long as it's in someone else's account. And then Amazon hits them with a straight face on these things, where, “Please pay the bill.” Which is horrifying when there's several orders of magnitude difference between your normal bill and what happens post-breach. But what I did my whole post on “17 Ways to Run Containers on AWS,” followed by “17 More Ways to Run Containers on AWS,” and [unintelligible 00:12:00] about three services away from having a third one ready to go on that, the point is not, “Too many ways to run containers,” because yes, that is true and it's also amusing to me—less so to the containers team at AWS which does not have a sense of humor or sense of self-awareness of which they have been alerted—and fine, but every time you're running a container, it is a way to turn it into a crypto mining operation, in some way shape or form, which means there are almost 40-some-odd services now that can reasonably be used to spin up cryptocurrency mining. And that is the best-case breach scenario in a bunch of ways. It costs a bunch of money and things to clean up, but ‘we lost customer data.' That can destroy companies.Chris: Here's the worst part. Crypto mining is no longer profitable even when I've got stolen API keys because bitcoin's in the toilet. So, now they are going after different things. Actually, the most recent one is they look to see if your account is out of the SCS sandbox and if so, they go back to the tried-and-true way of doing internet scams, which is email spam.Corey: For me, having worked in operations for a very long time, I've been in situations where I worked at Expensify and had access to customer data there. I have worked in other finance companies—I worked at Blackrock. Where I work now, I have access to customer billing data. And let me be serious here for a second, I take all of these things seriously, but I also in all of those roles slept pretty well at night. The one that kept me up was a brief stint I did as the Director of Tech Ops at Grindr over ten years ago because unlike the stuff where I'm spending the rest of my career and my time now, it's not just money anymore.Whereas today, if I get popped, someone can get access to what a bunch of companies are paying AWS. It's scandalous, and I will be sued into oblivion and my company will not exist anymore and I will have a cloud hanging over my head forever. So, I have to be serious about it—Chris: But nobody will die.Corey: Nobody dies. Whereas, “Oh, this person is on Grindr and they're not out publicly,” or they live in a jurisdiction where that is punishable by imprisonment or death, you have blood on your hands, on some level, and I have never wanted that kind of responsibility.Chris: Yeah. It's reasonably scary. I've always been happy to say that, you know, the worst thing that I had to do was keep the Russians off CNN and my friends from downloading Rick and Morty.Corey: Exactly. It's, “Oh, heavens, you're winding up costing some giant conglomerate somewhere theoretical money on streaming subscriptions.” It's not material to the state of the world. And part of it, too, is—what's always informed my approach to things is, I'm not a data hoarder in the way that it seems our entire industry is. For the Last Week in AWS newsletter, the data that I collect and track is pretty freaking small.It's, “You want to sign up for the lastweekinaws.com newsletter. Great, I need your email address.” I don't need your name, I don't need the company you work at. You want to give me a tagged email address? Fine. You want to give me some special address that goes through some anonymizing thing? Terrific. I need to know where I'm sending the newsletter. And then I run a query on that for metrics sometimes, which is this really sophisticated database query called a count. How many subscribers do I have at any given point because that matters to our sponsors. But can we get—you give us any demographic? No, I cannot. I can't. I have people who [unintelligible 00:15:43] follow up surveys sometimes and that's it.Chris: And you're able to make money doing that. You don't have to collect, okay, you know, Chris's zip code is this and Bob's zip code is that and Frank's zip code is the other thing.Corey: Exactly.Chris: Or job titles, or you know, our mother's maiden name or anything else like that.Corey: I talk about what's going on in the world of AWS, so it sort of seems to me that if you're reading this stuff every week, either because of the humor or in spite of the humor, you probably are in a position where services and goods tied to that ecosystem would be well-received by you or one of the other 32,000 people who happen to be reading the newsletter or listening to the podcast or et cetera, et cetera, et cetera. It's an old-timey business model. It's okay, I want to wind up selling, I don't know, expensive wristwatches. Well, maybe I'll advertise in a magazine that caters to people who have an interest in wristwatches, or caters to a demographic that traditionally buys those wristwatches. And okay, we'll run an ad campaign and see if it works.Chris: It's been traditional advertising, not the micro-targeting stuff. And you know, television was the same way back in the broadcast era, you know? You watched a particular show, people of that demographic who watched that particular show had certain advertisers they wanted.Corey: That part of the challenge I've seen too, from sponsors of this show, for example, is they know it works, but they're trying to figure out how to do any form of attribution on this. And my answer—which sounds self-serving, but it's true—is, there's no effective way to do it because every time you try, like, “Enter this coupon code,” yeah, I assure you, some of these things wind up costing millions of dollars to deploy at large companies at scale and they provide value for doing it. No one's going to punch in a coupon code to get 10% off or something like that. Procurement is going to negotiate custom contracts and it's going to be brought up maybe by someone who heard the podcast ad. Maybe it just sits in the back of their mind until they hear something and it just winds of contributing to a growing awareness of these things.You're never going to do attribution that works on things like that. People try sometimes to, “Oh, you'll get $25 in credit,” or, “We'll give you a free t-shirt if you fill out the form.” Yeah, but now you're biasing for people who find that a material motivator. When I'm debating what security suite I'm going to roll out at my enterprise I don't want a free t-shirt for that. In fact, if I get a free t-shirt and I wear that shirt from the vendor around the office while I'm trying to champion bringing that thing in, I look a little compromised.Chris: Yeah. Yeah, I am—[laugh] I got no response to that [laugh].Corey: No, no. I hear you. One thing I do want to talk about is the last time we spoke, you mentioned you were involved in getting fwd:cloudsec—a conference—off the ground. Like all good cloud security conferences, it's named after an email subject line.It is co-located with re:Inforce this year in Anaheim, California. Somewhat ominously enough, I used to live a block-and-a-half away from the venue. But I don't anymore and in fact, because nobody checks the global event list when they schedule these things, I will be on the other side of the world officiating a wedding the same day. So, yet again, I will not be at re:Inforce.Chris: That is a shame because I think you would have made an excellent person to contribute to our call for papers and attend. So yes, fwd:cloudsec is deliberately actually named after a subject line because all of the other Amazon conferences seem to be that way. And we didn't want to be going backwards and thinking, you know, past tense. We were looking forward to our conference. Yeah, so we're effectively a vendor-neutral cloud security conference. We liked the idea of being able to take the talks that Amazon PR would never allow on stage at re:Inforce and run with it.Corey: I would question that. I do want to call that out because I gave a talk at re:Invent one year about a vulnerability I found and reported, with the help of two other people, Scott Piper and Brandon Sherman, to the AWS security team. And we were able to talk about that on stage with Zack Glick, who at the time, was one of basically God's own prototypes, working over in the AWS environment next to Dan [Erson 00:19:56]. Now, Dan remains the salt of the earth, and if he ever leaves basically just short the entire US economy. It's easier. He is amazing. I digress. The point being is that they were very open about talking about an awful lot of stuff that I would never have expected that they would be okay with.Chris: And last year at re:Inforce, they had an excellent, excellent chalk talk—but it was a chalk talk, not recorded—on how ransomware attacks operate. And they actually, like, revealed some internal, very anonymized patterns of how attacks are working. So, they're starting to realize what we've been saying in the cloud security community for a while, which is, we need more legitimate threat intelligence. On the other hand, they don't want to call it threat intelligence because the word threat is threatening, and therefore, you know, we're going to just call it, you know, patterns or whatever. And our conference is, again, also multi-cloud, a concept that until recently, AWS, you know, didn't really want to acknowledge that there were other clouds and that people would use both of them [crosstalk 00:21:01]—Corey: Multi-cloud security is a nightmare. It's just awful.Chris: Yeah, I don't like multi-cloud, but I've come to realize that it is a thing. That you will either start at a company that says, “We're AWS and we're uni-cloud,” and then next thing, you know, either some rogue developer out there has gone and spun up an Azure subscription or your acquire somebody who's in GCP, or heaven forbid, you have to go into some, you know, tinhorn dictator's jurisdiction and they require you to be on-prem or leverage Oracle Cloud or something. And suddenly, congratulations, you're now multi-cloud. So yes, our goal is really to be the things that aren't necessarily onstage or aren't all just, “It's great.” Even your talk was how great the incident response and vulnerability remediation process was.Corey: How great my experience with it was at the time, to be clear. Because I also have gotten to a point where I am very aware that, in many cases when dealing with AWS, my reputation precedes me. So, when I wind up tweeting about a problem or opening a support case, I do not accept as a given that my experience is what everyone is going to experience. But a lot of the things they did made a lot of sense and I was frankly, impressed that they were willing to just talk about anything that they did internally. Because previously that had not been a thing that they did in open forums like that.Chris: But you go back to the Glue incident where somebody found a bug and they literally went and went to every single CloudTrail event going back to the dawn of the service to validate that, okay, the, only two times we ever saw this happen were between the two researcher's accounts who disclosed it. And so, kudos to them for that level of forward communication to their customers because yeah, I think we still haven't heard anything out of Azure for last year's—or a year-and-a-half ago's Wiz findings.Corey: Well, they did do a broad blog post about this that they put out, which I thought, “Okay, that was great. More of this please.” Because until they start talking about security issues and culture and the remediation thereof, I don't give a shit what they have to say about almost anything else because it all comes back to security. The only things I use Azure for, which admittedly has some great stuff; their computer vision API? Brilliant—but the things I use them for are things that I start from a premise of security is not important to that service.The thing I use it for on the soon-to-be-pivoted to Mastodon Twitter thread client that I built, it writes alt-text for images that are about to be put out publicly. Yeah, there's no security issue from that perspective. I am very hard-pressed to imagine a scenario in which that were not true.Chris: I can come up with a couple, but you know—Corey: It feels really contrived. And honestly, that's the thing that concerns me, too: the fact that I finally read, somewhat recently, an AWS white paper talking about—was it a white paper or was it blog post? I forget the exact media that it took. But it was about how they are seeing ransomware attacks on S3, which was huge because before that, I assumed it was something that was being made up by vendors to sell me something.Chris: So, that was the chalk talk.Corey: Yes.Chris: They finally got the chalk talk from re:Inforce, they gave it again at re:Invent because it was so well received and now they have it as a blog post out there, so that, you know, it's not just for people who show up in the room, they can hear it; it's actually now documented out there. And so, kudos to the Amazon security team for really getting that sort of threat intelligence out there to the community.Corey: Now, it's in writing, and that's something that I can cite as opposed to, “Well, I was at re:Invent and I heard—” Yeah, we saw the drink tab. We know what you might have thought you heard or saw at re:Invent. Give us something we can take to the board.Chris: There were a lot of us on that bar tab, so it's not all you.Corey: Exactly. And it was my pleasure to do it, to be clear. But getting back to fwd:cloudsec, I'm going to do you a favor. Whether it's an actual favor or the word favor belongs in quotes, the way that I submit CFPs, or conference talks, is optimized because I don't want to build a talk that is never going to get picked up. Why bother to go through all the work until I have to give it somewhere?So, I start with a catchy title and then three to five sentences. And if people accept it, great, then I get to build the talk. This is a forcing function in some ways because if you get a little delayed, they will not move the conference for you. I've checked. But the title of a talk that I think someone should submit for fwd:cloudsec is, “I Am Smarter Than You, so Cloud Security is Easy.”And the format and the conceit of the talk is present it with sort of a stand-it-up-to-take-it-down level of approach where you are over-confident in the fact that you are smarter than everyone else and best practices don't apply to you and so much of this stuff is just security theater designed as a revenue extraction mechanism as opposed to something you should actually be doing. And talk about why none of these things matter because you use good security and you know, it's good because you came up with it and there's no way that you could come up with something that you couldn't break because you're smart. It says so right in the title and you're on stage and you have a microphone. They don't. Turn that into something. I feel like there's a great way to turn that in a bunch of different directions. I'd love to see someone give that talk.Chris: I think Nickolas Sharp thought that too.Corey: [laugh]. Exactly. In fact, that will be a great way to bring it back around at the end. And it's like, “And that's why I'm better at security than you are. If you have any questions beyond this, you can reach me at whatever correctional institute I go in on Thursday.” Exactly. There's ways to make it fun and engaging. Because from my perspective, talks have to be entertaining or people don't pay attention.Chris: They're either entertaining, or they're so new and advanced. We're definitely an advanced cloud security practice thing. They were 500 levels. Not to brag or anything, but you know, you want the two to 300-level stuff, you can go CCJ up the street. We're hitting and going above and beyond what a lot of the [unintelligible 00:27:18]—Corey: I am not as advanced on that path as you are; I want to be very clear on this. You speak, I listen. You're one of those people when it comes to security. Because again, no one's life is hanging in the balance with respect to what I do. I am confident in our security posture here, but nothing's perfect. Everything is exploitable, on some level.It's also not my core area of focus. It is yours. And if you are not better than I am at this, then I have done something sort of strange, or so of you, in the same way that it is a near certainty—but not absolute—that I am better at optimizing AWS bills than you are. Specialists exist for a reason and to discount that expertise is the peak of hubris. Put that in your talk.Chris: Yeah. So, one talk I really want to see, and I've been threatening to give it for a while, is okay, if there's seventeen ways—or sorry, seventeen times two, soon to be seventeen times three ways to run containers in AWS, there's that many ways to exfiltrate credentials from those containers. What are all of those things? Do we have a holistic way of understanding, this is how credentials can be exfiltrated so that we then as defenders can go figure out, okay, how do we build detections and mitigations for this?Corey: Yeah. I'm a huge fan of Canarytokens myself, for that exact purpose. There are many devices I have where the only credentials in plain text on disk are things that as soon as they get used, I wind up with a bunch of things screaming at me that there's been a problem and telling me where it is. I'm not saying that my posture is impenetrable. Far from it. But you're going to have to work for it a little bit harder than running some random off-the-shelf security scanner against my AWS account and finding, oops, I forgot to turn on a bucket protection.Chris: And the other area that I think is getting really interesting is, all of the things that have credentials into your Cloud account, whether it's something like CircleCI or GitHub. I was having a conversation with somebody just this morning and we were talking about Roles Anywhere, and I was like, “Roles Anywhere is great if you've got a good strong PKI solution and can keep that private certificate or that certificate you need safe.” If you just put it on a disk, like, you would have put your AKIA and secret on a desk, congratulations, you haven't really improved security. You've just gotten rid of the IAM users that are being flagged in your CSPM tool, and congratulations, you have, in fact, achieved security theater.Corey: It's obnoxious, on some level. And part of the problem is cost and security are aligned and that people care about them right after they really should have cared about them. The difference is you can beg, cry, whine, et cetera to AWS for concessions, you can raise another round of funding; there have solutions with money. But security? That ship has already sailed.Chris: Yeah. Once the data is out, the data is out. Now, I will say on the bill, you get reminded of it every month, about three or four days after. It's like, “Oh. Crap, yeah, I should have turned off that EC2 instance. I just burned $100.” Or, “Oh hey, we didn't turn off that application. I just burned $100,000.” That doesn't happen on security. Security events tend to be few and far between; they're just much bigger when they happen.Corey: I really want to thank you for taking the time to chat with me. I'm sure I'll have you back on between now and re:Inforce slash fwd:cloudsec or anything else we come up with that resembles an email subject line. If people want to learn more and follow along with your adventures—as they should—where's the best place for him to find you these days?Chris: So, I am now pretty much living on Mastodon on the InfoSec Exchange. And my website, chrisfarris.com is where you can find the link to that because it's not just at, you know, whatever. You have to give the whole big long URL in Mastodon. It's no longer—Corey: Yeah. It's like a full-on email address with weird domains.Chris: Exactly, yeah. So, find me at http colon slash slash infosec dot exchange slash at jcfarris. Or just hit Chris Farris and follow the links. For fwd:cloudsec, we are conveniently located at fwdcloudsec.org, which is F-W-D cloud sec dot org. No colons because I don't think those are valid in whois.Corey: Excellent choice. And of course, links to that go in the [show notes 00:31:32], so click the button. It's easier. Thanks again for your time. I really appreciate it.Chris: Thank you.Corey: Chris Farris, Cloud Security Nerd at Turbot slash Turbo. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment that resembles a lawsuit being filed, and then have it processed-served to me because presumably, you work at Ubiquiti.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

AWS Morning Brief for the week of March 20, 2023 with Corey Quinn. Links: jobs.lastweekinaws.com Amazon EC2 M1 Mac instances now support in-place operating system updates Announcing Amazon Linux 2023  AWS Chatbot now available in Microsoft Teams  Announcing cross-account support for Amazon S3 Multi-Region Access Points  Talk about cloud with a non-cloud audience  New – Use Amazon S3 Object Lambda with Amazon CloudFront to Tailor Content for End Users Implementing an event-driven serverless story generation application with ChatGPT and DALL-E The Future of Mining is in the Cloud 

March comes in like a lion, but goes out like a lamb. Let's explore 3 storylines that might have long-ranging implications for cloud. SHOW: 703CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotwCHECK OUT OUR NEW PODCAST - "CLOUDCAST BASICS"SHOW SPONSORS:Find "Breaking Analysis Podcast with Dave Vellante" on Apple, Google and SpotifyKeep up to data with Enterprise Tech with theCUBECloudZero – Cloud Cost Visibility and Savings​​CloudZero provides immediate and ongoing savings with 100% visibility into your total cloud spendMake Cloud Native Ubiquitous with Cloud Native Computing Foundation (CNCF)Join the foundation of doers, CNCF is the open source, vendor-neutral hub of cloud native computing, hosting projects like Kubernetes and Prometheus to make cloud native universal and sustainableKubeConEU Virtual Event Registration Code: Please use the code KCEUVCCP, while supplies last.SHOW NOTES:Microsoft brings AI-powered Co-pilot to its productivity suiteUS Gov't looks to secure the cloud (by default)AWS' hidden anti-competitive move is hidden in plain sightTHERE'S A LOT TO KEEP UP WITH RIGHT NOWAll the good TV shows are coming out with new episodes (many the last season)The banking industry may or may not be very brittle right nowThe weather is getting better - springtime is nearMarchMadness - NCAA Basketball is happeningLEVERAGE, INTEREST RATES, RISK, TIMELINES, COMMUNICATIONS and PANICSMicrosoft bringing AI to productivity tools is going to shift the focus to more strategic thinking, but also being able to know what's right vs. just “work”Are you spending time thinking about the various ways that you can make this new easy-to-use, non-data-scientists-driven AI work in your environment? The US Gov't is trying to build a cloud strategy that is security-first. Is that possible? Will this bring up the debate about the cloud as a national utility? Security speed vs. agility speed? Corey Quinn highlights an important aspect of the cloud that isn't always obvious to people - they leverage their control of the network (inter-region and external) to shape the competitive landscape. FEEDBACK?Email: show at the cloudcast dot netTwitter: @thecloudcastnet

Matty Stratton, Director of Developer Relations at Aiven, joins Corey on Screaming in the Cloud for a friendly debate on whether or not company employees can still be considered community members. Corey says no, but opens up his position to the slings and arrows of Matty in an entertaining change of pace. Matty explains why he feels company employees can still be considered community members, and also explores how that should be done in a way that is transparent and helpful to everyone in the community. Matty and Corey also explore the benefits and drawbacks of talented community members becoming employees.About MattyMatty Stratton is the Director of Developer Relations at Aiven, a well-known member of the DevOps community, founder and co-host of the popular Arrested DevOps podcast, and a global organizer of the DevOpsDays set of conferences.Matty has over 20 years of experience in IT operations and is a sought-after speaker internationally, presenting at Agile, DevOps, and cloud engineering focused events worldwide. Demonstrating his keen insight into the changing landscape of technology, he recently changed his license plate from DEVOPS to KUBECTL.He lives in Chicago and has three awesome kids, whom he loves just a little bit more than he loves Diet Coke. Links Referenced: Aiven: https://aiven.io/ Twitter: https://twitter.com/mattstratton Mastodon: hackyderm.io/@mattstratton LinkedIn: https://www.linkedin.com/in/mattstratton/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is brought to us in part by our friends at Min.ioWith more than 1.1 billion docker pulls - Most of which were not due to an unfortunate loop mistake, like the kind I like to make - and more than 37 thousand github stars, (which are admittedly harder to get wrong), MinIO has become the industry standard alternative to S3. It runs everywhere  - public clouds, private clouds, Kubernetes distributions, baremetal, raspberry's pi, colocations - even in AWS Local Zones. The reason people like it comes down to its simplicity, scalability, enterprise features and best in class throughput. Software-defined and capable of running on almost any hardware you can imagine and some you probably can't, MinIO can handle everything you can throw at it - and AWS has imagined a lot of things - from datalakes to databases.Don't take their word for it though - check it out at www.min.io and see for yourself. That's www.min.io Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I am joined today by returning guest, my friend and yours, Matty Stratton, Director of Developer Relations at Aiven. Matty, it's been a hot second. How are you?Matty: It has been a while, but been pretty good. We have to come back to something that just occurred to me when we think about the different things we've talked about. There was a point of contention about prior art of the Corey Quinn face and photos. I don't know if you saw that discourse; we may have to have a conversation. There may be some absent—Corey: I did not see—Matty: Okay.Corey: —discourse, but I also would accept freely that I am not the first person to ever come up with the idea of opening my mouth and looking ridiculous for a photograph either.Matty: That's fair, but the thing that I think was funny—and if you don't mind, I'll just go ahead and throw this out here—is that I didn't put this two and two together. So, I posted a picture on Twitter a week or so ago that was primarily to show off the fact—it was a picture of me in 1993, and the point was that my jeans were French-rolled and were pegged. But in the photo, I am doing kind of the Corey Quinn face and so people said, “Oh, is this prior art?” And I said—you know what? I actually just remembered and I've never thought about this before, but one of my friends in high school, for his senior year ID he took a picture—his picture looks like, you know, that kind of, you know, three-quarters turn with the mouth opening going, “Ah,” you know?And he loved that picture—number one, he loved that picture so much that this guy carried his senior year high school ID in his wallet until we were like 25 because it was his favorite picture of himself. But every photo—and I saw this from looking through my yearbook of my friend Jay when we are seniors, he's doing the Corey Quinn face. And he is anecdotally part of the DevOps community, now a little bit too, and I haven't pointed this out to him. But people were saying that, you know, mine was prior art on yours, I said, “Actually, I was emulating yet someone else.”Corey: I will tell you the actual story of how it started. It was at re:Invent, I want to say 2018 or so, and what happened was is someone, they were a big fan of the newsletter—sort of the start of re:Invent—they said, “Hey, can I get a selfie with you?” And I figured, sure, why not. And the problem I had is I've always looked bad in photographs. And okay, great, so if I'm going to have a photo taken of me, that's going to be ridiculous, why not as a lark, go ahead and do this for fun during the course of re:Invent this year?So, whenever I did that I just slapped—if someone asked for a selfie—I'd slap the big happy open mouth smile on my face. And people thought, “Oh, my God, this is amazing.” And I don't know that it was necessarily worth that level of enthusiasm, but okay. I'll take it. I'm not here to tell people they're wrong when they enjoy a joke that I'm putting out there.And it just sort of stuck. And I think the peak of it that I don't think I'm ever going to be able to beat is I actually managed to pull that expression on my driver's license.Matty: Wow.Corey: Yeah.Matty: That's—Corey: They don't have a sense of humor that they are aware of at the DMV.Matty: No, they really don't. And having been to the San Francisco DMV and knowing how long it takes to get in there, like, that was a bit of a risk on your part because if they decided to change their mind, you wouldn't be able to come back for another four months [laugh].Corey: It amused me to do it, so why not? What else was I going to do? I brought my iPad with me, it has cellular on it, so I just can work remotely from there. It was either that or working in my home office again, and frankly, at the height of the pandemic, I could use the break.Matty: Yes [laugh]. That's saying something when the break you can use is going to the DMV.Corey: Right.Matty: That's a little bit where we were, where we at. I think just real quick thinking about that because there's a lot to be said with that kind of idea of making a—whether it's silly or not, but having a common, especially if you do a lot of photos, do a lot of things, you don't have to think about, like, how do I look? I mean, you have to think about—you know, you can just say I just know what I do. Because if you think about it, it's about cultivating your smile, cultivating your look for your photos, and just sort of having a way so you don't—you just know what to do every time. I guess that's a, you know, maybe a model tip or something. I don't know. But you might be onto something.Corey: I joke that my entire family motto is never be the most uncomfortable person in the room. And there's something to be said for it where if you're going to present a certain way, make it your own. Find a way to at least stand out. If nothing else, it's a bit different. Most people don't do that.Remember, we've all got made fun of, generally women—for some reason—back about 15 years ago or so for duck face, where in all the pictures you're making duck face. And well, there are reasons why that is a flattering way to present your face. But if there's one thing we love as a society, it's telling women they're doing something wrong.Matty: Yeah.Corey: So yeah, there's a whole bunch of ways you're supposed to take selfies or whatnot. Honestly, I'm in no way shape or form pretty enough or young enough to care about any of them. At this point, it's what I do when someone busts out a camera and that's the end of it. Now, am I the only person to do this? Absolutely not. Do I take ownership of it? No. Someone else wants to do it, they need give no credit. The idea probably didn't come from me.Matty: And to be fair, if I'm little bit taking the mickey there or whatever about prior art, it was more than I thought it was funny because I had not even—it was this thing where it was like, this is a good friend of mine, probably some of that I've been friends with longer than anyone in my whole life, and it was a core part [laugh] of his personality when we were 18 and 19, and it just d—I just never direct—like, made that connection. And then it happened to me and went “Oh, my God. Jason and Corey did the same thing.” [laugh]. It was—Corey: No, it feels like parallel evolution.Matty: Yeah, yeah. It was more of me never having connected those dots. And again, you're making that face for your DMV photo amused you, me talking about this for the last three minutes on a podcast amused me. So.Corey: And let's also be realistic here. How many ways are there to hold your face during a selfie that is distinguishable and worthy of comment? Usually, it's like okay, well, he has this weird sardonic half-smile with an eyebrow ar—no. His mouth was wide open. We're gonna go with that.Matty: You know, there's a little—I want to kind of—because I think there's actually quite a bit to the lesson from any of this because I think about—follow me here; maybe I'll get to the right place—like me and karaoke. No one would ever accuse me of being a talented singer, right? I'm not going to sing well in a way where people are going to be moved by my talent. So instead, I have to go a different direction. I have to go funny.But what it boils down to is I can only do—I do karaoke well when it's a song where I can feel like I'm doing an impression of the singer. So, for example, the B-52s. I do a very good impression of Fred Schneider. So, I can sing a B-52 song all day long. I actually could do better with Pearl Jam than I should be able to with my terrible voice because I'm doing an Eddie Vedder impression.So, what I'm getting at is you're sort of taking this thing where you're saying, okay, to your point, you said, “Hey,”—and your words, not mine—[where 00:07:09] somebody say, “The picture is not going to be of me looking like blue steel runway model, so I might as well look goofy.” You know? And take it that way and be funny with it. And also, every time, it's the same way, so I think it's a matter of kind of owning the conversation, you know, and saying, how do you accentuate the thing that you can do. I don't know. There's something about DevOps, somehow in there.Corey: So, I am in that uncomfortable place right now between having finalized a blog post slash podcast that's going out in two days from this recording. So, it will go out before you and I have this discussion publicly, but it's also too late for me to change any of it,m so I figured I will open myself up to the slings and arrows of you, more or less. And you haven't read this thing yet, which is even better, so you're now going to be angry about an imperfect representation of what I said in writing. But the short version is this: if you work for a company as their employee, then you are no longer a part of that company's community, as it were. And yes, that's nuanced and it's an overbroad statement and there are a bunch of ways that you could poke holes in it, but I'm curious to get your take on the overall positioning of it.Matty: So, at face value, I would vehemently disagree with that statement. And by that is, that I have spent years of my life tilting at the opposite windmill, which is just because you work at this company, doesn't mean you do not participate in the community and should not consider yourself a part of the community, first and foremost. That will, again, like everything else, it depends. It depends on a lot of things and I hope we can kind of explore that a little bit because just as much as I would take umbrage if you will, or whatnot, with the statement that if you work at the company, you stop being part of the community, I would also have an issue with, you're just automatically part of the community, right? Because these things take effort.And I feel like I've been as a devreloper, or whatever, Corey—how do you say it?Corey: Yep. No, you're right on. Devreloper.Matty: As a—or I would say, as a DevRel, although people on Twitter are angry about using the word DevRel to discuss—like saying, “I'm a DevRel.” “DevRel is a department.” It's a DevOps engineer thing again, except actually—it's, like, actually wrong. But anyway, you kind of run into this, like for example—I'm going to not name names here—but, like, to say, you know, Twitter for Pets, the—what do you—by the way, Corey, what are you going to do now for your made-up company when what Twitter is not fun for this anymore? You can't have Twitter for Pets anymore.Corey: I know I'm going to have to come up with a new joke. I don't quite know what to do with myself.Matty: This is really hard. While we will pretend Twitter for Pets is still around a little bit, even though its API is getting shut down.Corey: Exactly.Matty: So okay, so we're over here at Twitter for Pets, Inc. And we've got our—Corey: Twitter for Bees, because you know it'll at least have an APIary.Matty: Yeah. Ha. We have our team of devrelopers and community managers and stuff and community engineers that work at Twitter for Pets, and we have all of our software engineers and different people. And a lot of times the assumption—and now we're going to have Twitter for Pets community something, right? We have our community, we have our area, our place that we interact, whether it's in person, it's virtual, whether it's an event, whether it's our Discord or Discourse or Slack or whatever [doodlee 00:10:33] thing we're doing these days, and a lot of times, all those engineers and people whose title does not have the word ‘community' on it are like, “Oh, good. Well, we have people that do that.”So, number one, no because now we have people whose priority is it; like, we have more intentionality. So, if I work on the community team, if I'm a dev advocate or something like that, my priority is communicating and advocating to and for that community. But it's like a little bit of the, you know, the office space, I take the requirements from the [unintelligible 00:11:07] to people, you I give them to the engineers. I've got people—so like, you shouldn't have to have a go-between, right? And there's actually quite a bit of place.So, I think, this sort of assumption that you're not part of it and you have no responsibility towards that community, first of all, you're missing a lot as a person because that's just how you end up with people building a thing they don't understand.Corey: Oh, I think you have tremendous responsibility to the community, but whether you're a part of it and having responsibility to it or not aligned in my mind.Matty: So… maybe let's take a second and what do you mean by being a part of it?Corey: Right. Where very often I'll see a certain, I don't know, very large cloud provider will have an open-source project. Great, so you go and look at the open-source project and the only people with commit access are people who work at that company. That is an easy-to-make-fun-of example of this. Another is when the people who are in a community and talking about how they perceive things and putting out content about how they've interacted with various aspects of it start to work there, you see areas where it starts to call its authenticity into question.AWS is another great example of this. As someone in the community, I can talk about how I would build something on top of AWS, but then move this thing on to Fastly instead of CloudFront because CloudFront is terrible. If you work there, you're not going to be able to say the same thing. So, even if you're not being effusive with praise, there are certain guardrails and constraints that keep you from saying what you might otherwise, just based upon the sheer self-interest that comes from the company whose product or service you're talking about is also signing your paycheck and choosing to continue to do so.Matty: And I think even less about it because that's where your paycheck is coming. It's also just a—there's a gravitational pull towards those solutions because that's just what you're spending your day with, right? You know—Corey: Yeah. And you also don't want to start and admit even to yourself, in some cases, that okay, this aspect of what our company does is terrible, so companies—people shouldn't use it. You want to sort of ignore that, on some level, psychologically because that dissonance becomes harmful.Matty: Yeah. And I think there's—so again, this is where things get nuanced and get to levels. Because if you have the right amount of psychological safety in your organization, the organization understands what it's about to that. Because even people whose job is to be a community person should be able to say, “Hey, this is my actual opinion on this. And it might be contrary to the go-to-market where that comes in.”But it's hard, especially when it gets filtered through multiple layers and now you've got a CEO who doesn't understand that nuance who goes, “Wait, why was Corey on some podcast saying that the Twitter for Pets API is not everything it could possibly be?” So, I do think—I will say this—I do think that organizations and leadership are understanding this more than they might have in the past, so we are maybe putting on ourselves this belief that we can't be as fully honest, but even if it's not about hiding the warts, even if it's just a matter of also, you're just like, hey, chances are—plus also to be quite frank, if I work at the company, I probably have access to way more shit than I would have to pay for or do whatever and I know the right way. But here's the trick, and I won't even say it's a dogfooding thing, but if you are not learning and thinking about things the way that your users do—and I will even say that that's where—it is the users, which are the community, that community or the people that use your product or are connected to it, they don't use it; they may be anecdotal—or not anecdotally, maybe tangentially connected. I will give an example. And there was a place I was working where it was very clear, like, we had a way to you know, do open-source contributions back of a type of a provider plug-in, whatever you want to call it and I worked at the company and I could barely figure out how to follow the instructions.Because it made a lot of sense to someone who built that software all day long and knew the build patterns, knew all that stuff. So, if you were an engineer at this company, “Well, yeah, of course. You just do this.” And anybody who puts the—connects the dots, this has gotten better—and this was understood relatively quickly as, “Oh, this is the problem. Let's fix it.” So, the thing is, the reason why I bring this up is because it's not something anybody does intentionally because you don't know what you don't know. And—Corey: Oh, I'm not accusing anyone of being a nefarious actor in any of this. I also wonder if part of this is comes from your background as being heavily involved in the Chef community as a Chef employee and as part of the community around that, which is inherently focused on an open-source product that a company has been built around, whereas my primary interaction with community these days is the AWS community, where it doesn't matter whether you're large or small, you are not getting much, if anything, for free from AWS; you're all their customers and you don't really have input into how something gets built, beyond begging nicely.Matty: That's definitely true. And I think we saw that and there was things, when we look at, like, how community, kind of, evolved or just sort of happened at Chef and why we can't recreate it the same way is there was a certain inflection point of the industry and the burgeoning DevOps movement, and there wasn't—you know, so a lot of that was there. But one of the big problems, too, is, as Corey said, everybody—I shouldn't say every, but I've from the A—all the way up to AWS to your smaller startups will have this problem of where you end up hiring in—whether you want to or not—all of your champions and advocates and your really strong community members, and then that ends up happening. So, number one, that's going to happen. So frankly, if you don't push towards this idea, you're actually going to have people not want to come work because you should be able to be still the member that you were before.And the other thing is that at certain size, like, at the size of a hyperscaler, or, you know, a Microsoft—well, anybody—well Microsofts not a hyperscaler, but you know what I'm saying. Like, very, very large organization, your community folks are not necessarily the ones doing that hiring away. And as much as they might—you know, and again, I may be the running the community champion program at Microsoft and see that you want—you know, but that Joe Schmo is getting hired over into engineering. Like, I'm not going to hire Joe because it hurts me, but I can't say you can't, you know? It's so this is a problem at the large size.And at the smaller size, when you're growing that community, it happens, too, because it's really exciting. When there's a place that you're part of that community, especially when there's a strong feel, like going to work for the mothership, so to speak is, like, awesome. So again, to give an example, I was a member of the Chef community, I was a user, a community person well, before, you know, I went and, you know, had a paycheck coming out of that Seattle office. And it was, like, the coolest thing in the world to get a job offer from Ch—like, I was like, “Oh, my God. I get to actually go work there now.” Right?And when I was at Pulumi, there quite a few people I could think of who I knew through the community who then get jobs at Pulumi and we're so excited, and I imagine still excited, you know? I mean, that was awesome to do. So, it's hard because when you get really excited about a technology, then being able to say, “Wait, I can work on this all the time?” That sounds awesome, right? So like, you're going to have that happen.So, I think what you have to do is rather than prevent it from happening because number one, like, you don't want to actually prevent that from happening because those people will actually be really great additions to your organization in lots of ways. Also, you're not going to stop it from happening, right? I mean, it's also just a silly way to do it. All you're going to do is piss people off, and say, like, “Hey, you're not allowed to work here because we need you in the community.” Then they're going to be like, “Great. Well, guess what I'm not a part of anymore now, jerk?” Right? You know [laugh] I mean so—Corey: Exactly.Matty: Your [unintelligible 00:18:50] stops me. So, that doesn't work. But I think to your point, you talked about, like, okay, if you have a, ostensibly this a community project, but all the maintainers are from one—are from your company, you know? Or so I'm going to point to an example of, we had—you know, this was at Pulumi, we had a Champions program called Puluminaries, and then there's something similar to like Vox Populi, but it was kind of the community that was not run by Pulumi Inc. In that case.Now, we helped fund it and helped get it started, but there was there were rules about the, you know, the membership of the leadership, steering committee or board or whatever it was called, there was a hard limit on the number of people that could be Pulumi employees who were on that board. And it actually, as I recall when I was leaving—I imagine this is not—[unintelligible 00:19:41] does sometimes have to adjust a couple of things because maybe those board members become employees and now you have to say, you can't do that anymore or we have to take someone down. But the goal was to actually, you know, basically have—you know, Pulumi Corp wanted to have a voice on that board because if for no other reason, they were funding it, but it was just one voice. It wasn't even a majority voice. And that's a hard sell in a lot of places too because you lose control over that.There's things I know with, uh—when I think about, like, running meetup communities, like, we might be—well I mean, this is not a big secret, I mean because it's been announced, but we're—you know, Aiven is helping bootstrap a bunch of data infrastructure meetups around the world. But they're not Aiven meetups. Now, we're starting them because they have to start, but pretty much our approach is, as soon as this is running and there's people, whether they work here, work with us or not, they can take it, right? Like, if that's go—you know? And being able to do that can be really hard because you have to relinquish the control of your community.And I think you don't have to relinquish a hundred percent of that control because you're helping facilitate it because if it doesn't already have its own thing—to make sure that things like code of conduct and funding of it, and there's things that come along with the okay, we as an organization, as a company that has dollars and euros is going to do stuff for this, but it's not ours. And that's the thing to remember is that your community does not belong to you, the company. You are there to facilitate it, you are there to empower it, you're there to force-multiply it, to help protect it. And yeah, you will probably slurp a whole bunch of value out of it, so this is not magnanimous, but if you want it to actually be a place it's going to work, it kind of has to be what it wants to be. But by the same token, you can't just sort of sit there and be like, “I'm going to wait for this community grow up around me without anything”—you know.So, that's why you do have to start one if there is quote-unquote—maybe if there's no shape to one. But yeah, I think that's… it is different when it's something that feels a little—I don't even want to say that it's about being open-source. It's a little bit about it less of it being a SaaS or a service, or if it's something that you—I don't know.Corey: This episode is sponsored in part by Honeycomb. I'm not going to dance around the problem. Your. Engineers. Are. Burned. Out. They're tired from pagers waking them up at 2 am for something that could have waited until after their morning coffee. Ring Ring, Who's There? It's Nagios, the original call of duty! They're fed up with relying on two or three different “monitoring tools” that still require them to manually trudge through logs to decipher what might be wrong. Simply put, there's a better way. Observability tools like Honeycomb (and very little else becau se they do admittedly set the bar) show you the patterns and outliers of how users experience your code in complex and unpredictable environments so you can spend less time firefighting and more time innovating. It's great for your business, great for your engineers, and, most importantly, great for your customers. Try FREE today at honeycomb.io/screaminginthecloud. That's honeycomb.io/screaminginthecloud.Corey: Yeah, I think you're onto something here. I think another aspect where I found it be annoying is when companies view their community as, let's hire them all. And I don't think it ever starts that way. I think that it starts as, well these are people who are super-passionate about this, and they have great ideas and they were great to work with. Could we hire them?And the answer is, “Oh, wait. You can give me money for this thing I've been doing basically for free? Yeah, sure, why not?” And that's great in the individual cases. The problem is, at some point, you start to see scenarios where it feels like, if not everyone, then a significant vocal majority of the community starts to work there.Matty: I think less often than you might think is it done strategically or on purpose. There have been exceptions to that. There's one really clear one where it feels like a certain company a few years ago, hired up all the usual suspects of the DevOps community. All of a sudden, you're like, oh, a dozen people all went to go work at this place all at once. And the fun thing is, I remember feeling a little bit—got my nose a little out of joint because I was not the hiring mana—like, I knew the people.I was like, “Well, why didn't you ask me?” And they said, “Actually, you are more important to us not working here.” Now, that might have just been a way to sell my dude-in-tech ego or not, but whether or not that was actually true for me or not, that is a thing where you say you know, your folks—but I do think that particular example of, like, okay, I'm this, that company, and I'm going to go hire up all the usual suspects, I think that's less. I think a lot of times when you see communities hire up those people, it's not done on purpose and in fact, it's probably not something they actually wanted to do in mass that way. But it happens because people who are passionate about your product, it's like I said before, it actually seems pretty cool to go work on it as your main thing.But I can think of places I've been where we had, you know—again, same thing, we had a Pulumi—we had someone who was probably our strongest, loudest, most vocal community member, and you know, I really wanted to get this person to come join us and that was sort of one of the conversations. Nobody ever said, “We won't offer this person a job if they're great.” Like, that's the thing. I think that's actually kind of would be shitty to be like, “You're a very qualified individual, but you're more important to me out in the community so I'm not going to make your job offer.” But it was like, Ooh, that's the, you know—it'd be super cool to have this person but also, not that that should be part of our calculus of decision, but then you just say, what do you do to mitigate that?Because what I'm concerned about is people hearing this the wrong way and saying, “There's this very qualified individual who wants to come work on my team at my company, but they're also really important to our community and it will hurt our community if they come work here, so sorry, person, we're not going to give you an opportunity to have an awesome job.” Like, that's also thinking about the people involved, too. But I know having talked to folks that lots of these different large organizations that have this problem, generally, those community folks, especially at those places, they don't want this [laugh] happening. They get frustrated by it. So, I mean, I'll tell you, it's you know, the—AWS is one of them, right?They're very excited about a lot of the programs and cool people coming from community builders and stuff and Heroes, you know. On one hand, it's incredibly awesome to have a Hero come work at AWS, but it hurts, right, because now they're not external anymore.Corey: And you stop being a Hero in that case, as well.Matty: Yeah. You do, yeah.Corey: Of course, they also lose the status if they go to one of their major competitors. So like, let me get this straight. You can't be a Hero if you work for AWS or one of its competitors. And okay, how are there any Heroes left at all at some point? And the answer is, they bound it via size and a relatively small list of companies. But okay.Matty: So, thinking back to your point about saying, okay, so if you work at the company, you lose some authenticity, some impartiality, some, you know… I think, rather than just saying, “Well, you're not part”—because that also, honestly, my concern is that your blog post is now going to be ammunition for all the people who don't want to act as members of the community for the company they work for now. They're going to say, well, Corey told me I don't have to. So, like I said, I've been spending the last few years tilting at the opposite windmill, which is getting people that are not on the community team to take part in community summits and discourse and things like that, like, you know, for that's—so I think the thing is, rather than saying, “Well, you can't,” or, “You aren't,” it's like, “Well, what do you do to mitigate those things?”Corey: Yeah, it's a weird thing because taking AWS as the example that I've been beating up on a lot, the vast majority of their employees don't know the community exists in any meaningful sense. Which, no fault to them. The company has so many different things, no one keeps up with at all. But it's kind of nuts to realize that there are huge communities of people out there using a thing you have built and you do not know that those users exist and talk to each other in a particular watering hole. And you of course, as a result, have no presence there. I think that's the wrong direction, too. But—Matty: Mm-hm.Corey: Observing the community and being part of the community, I think there's a difference. Are you a biologist or are you a gorilla?Matty: Okay, but [sigh] I guess that's sort of the difference, too which—and it's hard, it's very hard to not just observe. Because I think that actually even taking the mentality of, “I am here to be Jane Goodall, Dr. Jane Goodall, and observe you while I live amongst you, but I'm not going to actually”—although maybe I'm probably doing disservice—I'm remembering my Goodall is… she was actually more involved. May be a bad example.Corey: Yeah. So, that analogy does fall apart a little bit.Matty: It does fall apart a little bit—Corey: Yeah.Matty: But it's you kind of am I sitting there taking field notes or am I actually engaging with you? Because there is a difference. Even if your main reason for being there is just purely to—I mean, this is not the Prime Directive. It's not Star Trek, right? You're not going to like, hold—you don't need to hold—I mean, do you have to hold yourself aloof and say, “I don't participate in this conversation; I'm just here to take notes?”I think that's very non-genuine at that point. That's over-rotating the other way. But I think it's a matter of in those spaces—I think there's two things. I think you have to have a way to be identified as you are an employee because that's just disclosure.Corey: Oh, I'm not suggesting by any stretch of the imagination, people work somewhere but not admit that they work somewhere when talking about the company. That's called fraud.Matty: Right. No, no, and I don't think it's even—but I'm saying beyond just, if it's not, if you're a cop, you have to tell me, right?Corey: [laugh].Matty: It's like, it's not—if asked, I will tell you I work at AWS. It's like in that place, it should say, “I am an AWS em—” like, I should be badged that way, just so it's clear. I think that's actually helpful in two ways. It's also helpful because it says like, okay, maybe you have a connection you can get for me somehow. Like, you might actually have some different insight or a way to chase something that, you know, it's not necessarily just about disclosure; it's also helpful to know.But I think within those spaces, that disclosure—or not disclosure, but being an employee does not offer you any more authority. And part of that is just having to be very clear about how you're constructing that community, right? And that's sort of the way that I think about it is, like, when we did the Pulumi Community Summit about a year ago, right? It was an online, you know, thing we did, and the timing was such that we didn't have a whole lot of Pulumi engineers were able to join, but when we—and it's hard to say we're going to sit in an open space together and everybody is the same here because people also—here's the difference. You say you want this authority? People will want that authority from the people that work at the company and they will always go to them and say, like, “Well, you should have this answer. Can you tell me about this? Can you do this?”So, it's actually hard on both cases to have that two-way conversation unless you set the rules of that space such as, “Okay, I work at Aiven, but when I'm in this space, short of code of conduct or whatever, if I have to be doing that thing, I have no more authority on this than anyone else.” I'm in this space as the same way everyone else's. You can't let that be assumed.Corey: Oh, and big companies do. It's always someone else's… there's someone else's department. Like, at some level, it feels like when you work in one of those enormous orgs, it's your remit is six inches wide.Matty: Well, right. Right. So, I think it's like your authority exists only so far as it's helpful to somebody. If I'm in a space as an Aivener, I'm there just as Matty the person. But I will say I work at Aiven, so if you're like, “God, I wish that I knew who was the person to ask about this replication issue,” and then I can be like, “Aha, I actually have backchannel. Let me help you with that.” But if I can say, “You know what? This is what I think about Kafka and I think why this is whatever,” like, you can—my opinion carries just as much weight as anybody else's, so to speak. Or—Corey: Yeah. You know, it's also weird. Again, community is such a broad and diverse term, I find myself in scenarios where I will observe and talk to people inside AWS about things, but I never want to come across as gloating somehow, that oh, I know, internal people that talk to you about this and you don't. Like, that's never how I want to come across. And I also, I never see the full picture; it's impossible for me to, so I never make commitments on behalf of other people. That's a good way to get in trouble.Matty: It is. And I think in the case of, like, someone like you who's, you know, got the connections you have or whatever, it's less likely for that to be something that you would advertise for a couple of reasons. Like, nobody should be advertising to gloat, but also, part of my remit as a member of a community team is to actually help people. Like, you're doing it because you want to or because it serves you in a different way. Like, that is literally my job.So like, it shouldn't be, like—like, because same thing, if you offer up your connections, now you are taking on some work to do that. Someone who works at the company, like, yes, you should be taking on that work because this is what we do. We're already getting paid for it, you know, so to speak, so I think that's the—Corey: Yeah.Matty: —maybe a nuance, but—Corey: Every once in a while, I'll check my Twitter spam graveyard, [unintelligible 00:32:01] people asking me technical questions months ago about various things regarding AWS and whatnot. And that's all well and good; the problem I have with it is that I'm not a support vector. I don't represent for the company or work for them. Now, if I worked there, I'd feel obligated to make sure this gets handed to the right person. And that's important.The other part of it, though, is okay, now that that's been done and handed off, like do I shepherd it through the process? Eh. I don't want people to get used to asking people in DMs because again, I consider myself to be a nice guy, but if I'm some nefarious jerk, then I could lead them down a very dark path where I suddenly have access to their accounts. And oh, yeah, go ahead and sign up for this thing and I'll take over their computer or convince them to pay me in iTunes gift cards or something like that. No, no, no. Have those conversations in public or through official channels, just because I don't, I don't think you want to wind up in that scenario.Matty: So, my concern as well, with sort of taking the tack of you are just an observer of the community, not a part of it is, that actually can reinforce some pretty bad behavior from an organization towards how they treat the community. One of the things that bothers me—if we're going to go on a different rant about devrelopers like myself—is I like to say that, you know, we pride ourselves as DevRels as being very empathetic and all this stuff, but very happy to shit all over people that work in sales or marketing, based on their job title, right? And I'm like, “Wow, that's great,” right? We're painting with this broad brush. Whereas in reality, we're not separate from.And so, the thing is, when you treat your community as something separate from you, you are treating it as something separate from you. And then it becomes a lot easier also, to not treat them like people and treat them as just a bunch of numbers and treat them as something to have value extracted from rather than it—this is actually a bunch of humans, right? And if I'm part of that, then I'm in the same Dunbar number a little bit, right? I'm in the same monkey sphere as those people because me, I'm—whoever; I'm the CTO or whatever, but I'm part of this community, just like Joe Smith over there in Paducah, you know, who's just building things for the first time. We're all humans together, and it helps to not treat it as the sort of amorphous blob of value to be extracted.So, I think that's… I think all of the examples you've been giving and those are all valid concerns and things to watch out for, the broad brush if you're not part of the community if you work there, my concern is that that leads towards exacerbating already existing bad behavior. You don't have to convince most of the people that the community is separate from them. That's what I'm sort of getting at. I feel like in this work, we've been spending so much time to try to get people to realize they should be acting like part of their larger community—and also, Corey, I know you well enough to know that, you know, sensationalism to make a point [laugh] works to get somebody to join—Corey: I have my moments.Matty: Yeah, yeah, yeah. I mean, there's I think… I'll put it this way. I'm very interested to see the reaction, the response that comes out in, well now, for us a couple of days, for you the listener, a while ago [laugh] when that hits because I think it is a, I don't want to say it's controversial, but I think it's something that has a lot of, um… put it this way, anything that's simple and black and white is not good for discussion.Corey: It's nuanced. And I know that whenever I wrote in 1200 words is not going to be as nuanced of the conversation we just had, either, so I'm sure people will have opinions on it. That'd be fun. It'd be a good excuse for me to listen.Matty: Exactly [laugh]. And then we'll have to remember to go back and find—I'll have to do a little Twitter search for the dates.Corey: We'll have to do another discussion on this, if anything interesting comes out of it.Matty: Actually, that would be funny. That would be—we could do a little recap.Corey: It would. I want to thank you so much for being so generous with your time. Where can people find you if they want to learn more?Matty: Well, [sigh] for the moment, [sigh] who knows what will be the case when this comes out, but you can still find me on Twitter at @mattstratton. I'm also at hackie-derm dot io—sorry, hackyderm.io. I keep wanting to say hackie-derm, but hackyderm actually works better anyway and it's funnier. But [hackyderm.io/@mattstratton](https://hackyderm.io/@mattstratton) is my Mastodon. LinkedIn; I'm. Around there. I need to play more at that. You will—also again, I don't know when this is coming out, so you won't tell you—you don't find me out traveling as much as you might have before, but DevOpsDays Chicago is coming up August 9th and 10th in Chicago, so at the time of listening to this, I'm sure our program will have been posted. But please come and join us. It will be our ninth time of hosting a DevOpsDay Chicago. And I have decided I'm sticking around for ten, so next year will be my last DevOpsDay that I'm running. So, this is the penultimate. And we always know that the penultimate is the best.Corey: Absolutely. Thanks again for your time. It's appreciated. Matty Stratton, Director of Developer Relations at Aiven. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment talking about how I completely missed the whole point of this community and failing to disclose that you are in fact one of the producers of the show.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

Talitha A. Abell is a same race domestic adoptee born in California in 1976. She always knew she was adopted and had what she considers a positive childhood, Still she says she wondered about her beginning story and so she was always looking off and on to find her birth mom. In 2020, she was introduced to the adoptee community through a podcast which has helped to add language and growth to her understanding of her adoptee identity. Being denied access to her original birth certificate she enlisted the help of her childhood friend In January of 2021. In December of 2021 she found her birth family and has enjoyed getting to know them.Music by Corey Quinn and Invitational by MDT

Diya Wynn, Senior Practice Manager in Responsible AI for AWS Machine Learning Solutions Lab, joins Corey on Screaming in the Cloud to discuss her team's efforts to study and implement responsible practices when developing AI technology. Corey and Diya explore the ethical challenges of AI, and why it's so important to be looking ahead for potential issues before they arise. Diya explains why socially responsible AI is still a journey, and describes how her and her team at AWS are seeking to forge that path to help their customers implement the technology in a safe and ethical way. Diya also describes her approach to reducing human-caused bias in AI models. About DiyaDiya Wynn is the Senior Practice Manager in Responsible AI for AWS Machine Learning Solutions Lab. She leads the team that engages with customers globally to go from theory to practice - operationalizing standards for responsible Artificial Intelligence/Machine Learning and data. Diya leads discussions on taking intentional action to uncover potential unintended impacts, and mitigate risks related to the development, deployment and use of AI/ML systems. She leverages her more than 25 years of experience as a technologist scaling products for acquisition; driving inclusion, diversity & equity initiatives; leading operational transformation across industries and understanding of historical and systemic contexts to guide customers in establishing an AI/ML operating model that enables inclusive and responsible products. Additionally, she serves on non-profit boards including the AWS Health Equity Initiative Review Committee; mentors at Tulane University, Spelman College and GMI; was a mayoral appointee in Environment Affairs for 6 consecutive years and guest lectures regularly on responsible and inclusive technology. Diya studied Computer Science at Spelman College, the Management of Technology at New York University, and AI & Ethics at Harvard University Professional School and MIT Sloan School of Management.Links Referenced:Machine Learning is a Marvelously Executed Scam: https://www.lastweekinaws.com/blog/machine-learning-is-a-marvelously-executed-scam/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Tailscale SSH is a new, and arguably better way to SSH. Once you've enabled Tailscale SSH on your server and user devices, Tailscale takes care of the rest. So you don't need to manage, rotate, or distribute new SSH keys every time someone on your team leaves. Pretty cool, right? Tailscale gives each device in your network a node key to connect to your VPN, and uses that same key for SSH authorization and encryption. So basically you're SSHing the same way that you're already managing your network.So what's the benefit? Well, built-in key rotation, the ability to manage permissions as code, connectivity between any two devices, and reduced latency. You can even ask users to re-authenticate SSH connections for that extra bit of security to keep the compliance folks happy. Try Tailscale now - it's free forever for personal use.Corey: Kentik provides Cloud and NetOps teams with complete visibility into hybrid and multi-cloud networks. Ensure an amazing customer experience, reduce cloud and network costs, and optimize performance at scale — from internet to data center to container to cloud. Learn how you can get control of complex cloud networks at www.kentik.com, and see why companies like Zoom, Twitch, New Relic, Box, Ebay, Viasat, GoDaddy, booking.com, and many, many more choose Kentik as their network observability platform. Corey: Welcome to Screaming in the Cloud, I'm Corey Quinn. In a refreshing change of pace, I have decided to emerge from my home office cave studio thing and go to re:Invent and interview people in person. This is something of a challenge for me because it is way easier in person to punch me in the face, so we'll see how it winds up playing out. My guest today is Diya Wynn, Senior Practice Manager at AWS. Diya, what is a practice manager at AWS? What do you do?Diya: So, a practice manager, I guess you can think of it just like a manager of a team. I have a practice that's specifically focused on Responsible AI. And I mean, practices are just like you could have won in financial services or anything. It's a department, essentially. But more important than the practice in the title is actually what I get a chance to do, and that's working with our customers directly that are using and leveraging our AI/ML services to build products.And we have an opportunity to help them think about how are they using that technology in ways to have improvements or benefit individuals in society, but minimize the risk and the unintended impact or harm. And that's something that we get to do with customers over any industry as well as globally. And my team and I have been enjoying the opportunity to be able to help them along their Responsible AI journey.Corey: So, the idea of Responsible AI is… I'm going to sound old and date myself when I say this, but it feels like it's such a strange concept for me, someone who came up doing systems administration work in physical data centers. The responsible use of a server back when I was hands-on hardware was, “Well, you don't want to hit your coworker with a server no matter how obnoxious they are.” And it was fairly straightforward. It was clear: yes or no. And now it seems that whenever we talk about AI in society, in popular culture, from a technologist's point of view, the answer is always a deeply nuanced shade of gray. Help.Diya: Nuanced shade of gray. That's interesting. It is a little bit more challenging. I think that it is, you know, in one sense because of the notion of all of the data that we get to leverage, and our machine-learning models are reliant on data that has variations coming from, you know, historical sort of elements, things that are here baked with bias, all of that has to be considered. And I think when we think about some of the challenges and even the ways in which AI is being used, it means that we have to be much more mindful of its context, right?And these systems are being used in ways that we probably didn't think about servers being used in the past, but also are in the midst of some high-stakes decisions, right? Whether or not I might be identified or misidentified and inappropriately arrested or if I get the appropriate service that I was thinking about or whether or not there are associations related to my gender or my sexual preference. All of that matters, and so it does become much more of a nuanced conversation. Also because depending on the jurisdiction you're in, the region, what makes sense and what matters might differ slightly. So, it's a multidisciplinary problem or challenge that we need to think about what is the legality of this?And we have to think about social science sometimes and there's an element of ethics. And all of that plays into what becomes responsible, what is the right way in which we use the technology, what are the implications of technology? And so yes, it is a little bit more gray, but there are things that I think we have at our disposal to help us be able to respond to and put in place so that we really are doing the right things with technology.Corey: I've known Amazon across the board to be customer-obsessed, and they tell us that constantly—and I do believe it; I talk to an awful lot of Amazonians—and so much of what the company does comes directly from customer requests. I have to ask, what were customers asking that led to the creation of your group? Because it seems odd to me that you would have someone coming to you and saying, “Well, we built a ‘Hot Dog/Not A Hot Dog' image recognition app,” and, “Oopsie. It turns out our app is incredibly biased against Canadians. How do we fix this?” Like, that does not seem like a realistic conversation. What were the customer concerns? How are they articulated?Diya: No, that's really good. And you're right. They weren't asking the question in that way, but over the last five years or so, I would say, there has been an increase in interest and as well as concern about how AI is being used and the potential risks or the areas of unintended impact. And with this sort of heightened sensitivity or concern, both with our executives as well as members of common society, right—they're starting to talk about that more—they started to ask questions. They're using surfaces we want to be responsible in building.Now, some customers were saying that. And so, they would ask, “What are other customers doing? What should we be aware of? How do we or are there tools that we can use to make sure that we're minimizing bias in our systems? Are there things that we can think about in the way of privacy?”And oftentimes privacy and security are one of those areas that might come up first. And those were the kinds of questions. We actually did a survey asking a number of our customer-facing resources to find out what were customers asking so that we could begin to respond with a product or service that would actually meet that need. And I think we've done a great job in being able to respond to that in providing them assistance. And I think the other thing that we paid attention to was not just the customer requests but also what we're seeing in the marketplace. Part of our job is not only to respond to the customer need but also sometimes to see the need that they're going to have ahead of them because of the way in which the industry is moving. And I think we did a pretty good job of being able to see that and then start to provide service and respond to assist them.Corey: Yeah, it's almost like a rule that I believe it was Scott Hanselman that I stole it from where the third time that you're asked the same question, write a blog post, then that way you can do a full deep—Diya: Did he really say write a post? [laugh].Corey: Treatment of it. Yes, he did. And the idea is, write a blog post—because his blog is phenomenal—and that way, you have a really in-depth authoritative answer to that question and you don't have to ad-lib it off the cuff every time someone asks you in the future. And it feels like that's sort of an expression of what you did. You started off as a customer-facing team where they were asking you the same questions again and again and at some point it's, okay, we can either spend the rest of our lives scaling this team ad infinitum and winding up just answering the phone all day, or we can build a service that directly addresses and answers the question.Diya: Absolutely, absolutely. I think that's the way in which we scale, right, and then we have some consistency and structure in order to be able to respond and meet a need. What we were able to do was—and I think this is sort of the beauty of being at AWS and Amazon; we have this opportunity to create narratives and to see a need, and be able to identify and respond to that. And that's something that everybody can do, not just resigned to a VP or someone that's an executive, we all can do that. And that was an opportunity that I had: seeing the need, getting information and data, and being able to respond and say, “We need to come up with something.”And so, one of our first pieces of work was to actually define a framework. How would we engage? What would be that repeatable process or structure for us, framework that we can leverage with our customers every time to help them think through, look around corners, understand where there's risk, be better informed, and make better-informed decisions about how they were using the technology or what ways they could minimize bias? And so, that framework for us was important. And then we have now tools and services as well that were underway, you know, on our product side, if you will, that are complementing—or that, you know, complement the work.So, not only here's a process, here's a framework and structure, but also here are tools that in technology you can bring to bear to help you automate, to help you understand performance, or even you know, help you minimize the bias and risk.Corey: What's interesting to me, in a very different part of the world than AI, I live in AWS costing because I decided, I don't know, I should just go and try and be miserable for the rest of my life and look at bills all day. But whenever I talk to clients, they asked the same question: what are other customers doing, as you alluded to a few minutes ago? And that feels like it's a universal question. I feel like every customer, no matter in what discipline or what area they're in, is firmly convinced that somewhere out there is this utopian, platonic ideal of the perfect company that has figured all of this stuff out and we're all constantly searching for them. Like, there's got to be someone who has solved this problem the right way.And in several cases, I've had to tell clients that you are actually one of the best in the world and furthest advanced at this particular thing. That customer, the closest we've got to them is you, so we should be asking you these questions. And for whatever it's worth, no one ever likes hearing that because, “Like, oh, we're doing something wild.” It's like—Diya: [crosstalk 00:10:15] pioneers.Corey: —“Well, we got to solve this ourselves? That's terrible.”Diya: Well, it's interesting you say that because it is a common question. I think customers have an expectation that because we are AWS, we've seen a lot. And I think that's true. There are tens of thousands of customers that are using our services, we have conversations with companies all across the world, so we do have some perspective of what other customers are doing and that's certainly something that we can bring to the table. But the other part of this is that this is really a new area. This is a sort of new space, that we're focused on trustworthy and Responsible AI, and there aren't a ton of customers that are doing this—or companies at all—that have it entirely answered, that have—you know, we're all on a journey.So, these are, I would say, early stages. And we do have the benefit of being large, having a lot of customers, having some experience in building services as well as helping our customers build products, having a team that's focused on looking at standards and working with standards bodies globally, having teams that are working on our understanding what we're doing in regulation and public policy. And so, all of that we bring to bear when we start talking about, you know, this with our customers. But we don't have all the answers; we're on a journey like them. And I think that's something that we have to be comfortable with, to some degree, that this is an evolving area and we're learning. And we're investing even in research to help us continue to move forward. But there's a lot that we know, that there's a lot that we can bring to the table, and we can help our customers in that regard.Corey: Now, this might very well be old news and well understood and my understanding is laughably naive when this gets released, but as of this recording, a few hours beforehand, you released something called Service Cards. And I have to say, my initial glance at this was honestly one of disappointment when I saw what it was because what I was hoping for, with—when you ever see ‘service' and ‘cards' together, is these are going to be printable cardboard, little cards that I can slip into the Monopoly board game I have at home and game night at home is going to be so traumatic for my kids afterwards. Like, “What's a Fargate?” Says the five-year-old, and there we go. “It means that daddy is not going to passing go, going directly to jail with you. Have fun,” it's great. But I don't think that's what it is.Diya: No, not at all. Not at all. So, it is very similar to the context that people might be familiar with around model cards, being able to give definition and understanding of a model that's being used. For us, we sort of took that concept at one step beyond that in that, you know, just providing a model card isn't sufficient necessarily, especially when there are multiple services or multiple models being used for any one of our services. But what our Service Cards allow us to do is to provide a better understanding of the intended use of the service, you know, and the model that's underpinning that, give context for the performance of that service, give guidelines for our customers to be able to understand how was it best used and how does it best perform.And that's a degree of transparency that we're providing under the hood, for our customers to really help them as well be much more responsible and how they're building on top of those. And it gives them clarity because there is a growing interest in the marketplace for our customers to hold their vendors—or companies to hold their vendors responsible, right, making sure that they're doing the right things and covering off, are we building well? Do we have, like, the customer or enough of demographic covered? What the performance looks like. And this is a really big opportunity for us to be transparent with our customers about how our services are being built and give them a little bit more of that guardrail that we were talking about—guidelines—how to best use it as they look to build upon those.Corey: Not in any way, shape, or form to besmirch the importance of a lot of the areas that you're covering on this, but on some level, I'm just envious in that it would be so nice to have that for every AWS service, of this is how it is—Diya: Uh-oh [laugh].Corey: —actually intended to be used. Because to me, I look at it and all I see is database, database, really expensive database, probably a database, and, like, none of those are designed to be databases. Like, “You lack imagination,” is my approach. And no, it just turns out I'm terrible at computers, but I'm also enthusiastic and those are terrible combinations. But I would love to see breakdowns around things like that as far as intended use, potential pitfalls, and increasingly as we start seeing more and more services get machine learning mixed in, for lack of a better term, increasingly we're going to start to see areas where the ethical implications absolutely are going to be creeping in. Which is a wild thing to say about, I don't know, a service that recommends how to right-size instances having ethical concerns. But it's not that unreasonable.Diya: Well, I can't make any promises about us having those kinds of instructions or guidelines for some of our other services, but we are certainly committed to being able to provide this transparency across our AI/ML services. And again, that's something I will say that's a journey. We've released a few today; there are others that are going to come. We're going to continue to iterate and evolve so that we can get through our services. And there's a lot of work behind that, right?It's not just that we wrote up this document, but it is providing transparency. But it also means that our teams are doing a great bit in terms of the diligence to be able to provide that feedback, to be able to test their models, understand their datasets, you know, provide information about the datasets in public—you know, for the public datasets that are being tested against, and also have the structure for them to train their models appropriately. So, there's a lot going into the development of those that may not be immediately transparent, but really is core to our commitment to how we're building our services now.Corey: It's a new area in many respects because, like, to be very direct. If I wind up misusing or being surprised by a bad implementation of something in most cases in AWS context, the disaster area looks a lot closer to I get a big bill. Which—and this [unintelligible 00:16:35] is going to sound bizarre, but here we are, it's only money. Money can be fixed. I can cry and sob to support and get that fixed.With things like machine learning and AI, the stakes are significantly higher because given some of the use cases and given some of the rapid emerging technology areas in which these things are being tested and deployed, it hurts people if it gets wrong. And an AWS bill is painful, but not in a damaging to populations level. Yet. I'm sure at some point, it becomes so large it becomes its own micro-economy, I guess the way those credits are now, but it's a very different way.Diya: Right. Absolutely. So, I think that's why our work from a responsibility perspective is important. But I think it's also valuable for customers to understand, we're taking a step forward and being able to help them. Very much like what we do with well-architected, right? We have a framework, we have best practices and guidance that is being provided so that our customers who are using our cloud services really know what's the best.This is very much like those Service Cards, right? Here's the best conditions in order to be able to use and get the greatest value out of your cloud investment. The same thing is what we're doing with this approach in helping our customers in the Responsible AI way. Here's the best, sort of, best practices, guidance, guardrails, tools that are going to help you make the most out of your investment in AI and minimize where there's this unintended or potential areas of potential harm that you were describing. And you're right, there are high stakes use cases, right, that we want to make sure or want to be able to help and equip our customers to think more about intentionally and be prepared to be able to hopefully have a governance structure, people aligned, processes, technology to really be able to minimize that, right? We want to reduce the blast radius.[midroll 00:18:37]Corey: One thing I want to call out as well is that as much as we love in tech to pretend that we have invented all of these things ourselves—like, we see it all the time; like, “No one really knows how to hire, there's no real scientific study on this.” “Yes, there are. There are multi-decade longitudinal studies at places like GM and whatnot.” And, “No, no, no tech is different. There's no way to know this. La la la.”And that's great. We have to invent these things ourselves. But bias has been a thing in business decisions, even ones that are not directly caused by humans, for a long time. An easy example is in many cases, credit ratings and decisions whether to grant credit or not. Like, they were not using machine learning in the 90s to do this, but strangely, depending upon a wide variety of factors that are not actually things that are under your control as a person, you are deemed to be a good credit risk versus a bad credit risk.And as a result, I think one of the best terms I heard in the early days when machine learning started getting big, was just referring to it as bias laundering. Well, we've had versions of that for a long time. Now, at least it seems like this shines a light on it if nothing else, and gives us an opportunity to address it.Diya: Absolutely. Oh, I'd love that, right? The opportunity to address it. So, one of the things that I often share with folks is we all have bias, right? And so, like you said we've had bias in a number of cases. Now, you know, in some cases, bias is understandable. We all have it. It is the thing that often—we talk about the sort of like mental shortcuts, things that we do that help us to respond rapidly in the world in the vast array of information that we're taking in all the time. So—Corey: You're an Amazonian. You yourself bias for action.Diya: Exactly. Right? So, we have bias. Now, the intent is that we want to be able to disrupt that so that we don't make decisions, oftentimes, that could be harmful, right? So, we have proclivities, desires, interest, right, that kind of folds into our bias, but there are other things, our background, where we went to school, you know, experiences that we had, information that we've been taking that also helped to drive towards some of those biases.So, that's one element, right, understanding that. A human bias gets infiltrated into our systems. And there was a study in AI now—I think it was 2019—that talked about that, right, that our systems are often biased by—or the bias is introduced, you know, sometimes by individuals. And part of the necessity for us to be able to eliminate that is understanding that we have bias, do things to interrupt it, and then also bringing in diversity, right? Because some of our biases are just that we don't have enough of the right perspectives in the room; we don't have enough of the right people involved, right?And so, being able to sort of widen the net, making sure that we're involving the outliers, I think are important to us being able to eliminate bias as well. And then there are tools that we can use. But then you also bring up something interesting here in terms of the data, right? And there's a part that education plays a good role in helping us understand the things like what you described our institutional biases baked into our data that also can come out in decisions that are now being made. And the more that we use AI in these ways, the more there is risk for that, right?So, that's why this effort in Responsible AI, understanding how we mitigate bias, understanding how we invite the right people in, the inclusion of the right perspectives, thinking about the outliers, thinking about whether or not this is the right problem for us to solve with AI is important, right, so that we can minimize those areas where bias is just another thing that we continue to propagate.Corey: So, a year or two ago, I wrote a blog post titled Machine Learning is a Marvelously Executed Scam. And it was talking about selling digital pickaxes into a data gold rush.Diya: I [crosstalk 00:22:30] remember this one [laugh].Corey: And it was a lot of fun. In fact, the Head of Analyst Relations at AWS for Machine Learning responded by sending me a Minecraft pickaxe made out of foam, which is now in my home office hung behind my office and I get a comment at least three times a week on that. It was absolutely genius as far as rebuttal go. And I've got to find some way to wind up responding to her in kind one of these days.But it felt like it was a solution in search of a problem. And I no longer hold so closely to that particular opinion, in no small part due to the fact that, as you're discussing, this area is fraught, it's under an awful lot of scrutiny, large companies who use these things and then those tools get it wrong are going to basically wind up being castigated for it. And yet, they are clearly realizing enough value from machine learning that it is worth the risk. And these are companies whose entire business, start to finish, is managing and mitigating risk. There is something there or suddenly everyone has taken leave of their senses. I don't quite buy that second option, so I'm guessing it's the first.Diya: So, the question is, is it worth the risk? And I would say, I think some people might or some companies might have started to step into that area thinking that it is, but it's not. And that's what we're saying and that's what we're hearing in the industry [unintelligible 00:23:51], that it's not worth the risk. And you're hearing from customers, outcries from others, government officials, right, all of them are saying, like, “It's not worth the risk and we have to pay attention to that.”But I think that there's certainly value and we're seeing that, right? We're solving previously unattainable problems with AI. We want to be able to continue to do that, but give people the means to be able to sort of minimize where there is risk and recognize that this is not a risk that's worth us taking. So, the potential for reputational harm and the damage that will do is real, right? When a company is called out for the fact that they've discriminated and they're unfairly evaluating homes, for instance, for people of color in certain communities, right, that's not something that's going to be tolerated or accepted.And so, you have people really calling those things out so that we start to—organizations do the right things and not think that risk is worth the [unintelligible 00:24:52]. It is very well worth the risk to use AI, but we've got to do it responsibly. There's so much value in what we are able to accomplish. So, we're seeing, you know, even with Covid, being able to advance, like, the technology around vaccinations and how that was done and accelerated with machine learning, or being able to respond to some of the needs that small businesses and others had, you know, during Covid, being able to continuate their service because we didn't have people in businesses or in offices, a lot of that was advanced during that time as a result of AI. We want to be able to see advances like that and companies be able to continue to innovate, and so we want to be able to do that without the risk, without the sort of impact that we're talking about, the negative impact. And I think that's why the work is so important.Corey: Do you believe that societally we're closer to striking the right balance?Diya: We're on our way. I think this is certainly a journey. There is a lot of attention on this in the right ways. And my hope—and certainly, that's why I'm in a role like this—that we can actually invite the right voices into the room. One of the things—and one of my colleagues said this earlier today, and I think it was a really, really great point, right—as we are seeing—first of all, we never thought that we would have, like, ethicists roles and sort of Responsible AI folks, and chief ethics officers. That was not something that existed in the context of, sort of, machine learning, and that's something that it's evolved in the last, you know, few years.But the other thing that we're seeing is that the folks that are sitting in those roles are increasingly diverse and are helping to drive the focus on the inclusion that we need and the value of making sure that those voices are there so that we can build in inclusive and responsible ways. And that's one of the things that I think is helping us get there, right? We're not entirely there, but I think that we're on a path. And the more that we can have conversations like this, the more that companies are starting to pay attention and take intentional action, right, to build ethically and to have the trust in the technology and the products that they build, and to do that in responsible ways, we'll get there.Corey: I really want to thank you for taking so much time to talk through what you're up to with me.Diya: I am super excited and glad that you were able to have me on. I love talking about this, so it's great. And I think it's one of the ways that we get more people aware, and hopefully, it sparks the interest in companies to take their own Responsible AI journey.Corey: Thank you so much for your time.Diya: Thanks for having me.Corey: I appreciate it. Diya Wynn, Senior Practice Manager at AWS. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry insulting comment, presumably because you're Canadian.Diya: [laugh].Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

AWS Morning Brief for the week of March 13, 2023 with Corey Quinn. Links: jobs.lastweekinaws.com Amazon EC2 announces the ability to create Amazon Machine Images (AMIs) that can boot on UEFI and Legacy BIOS  AWS Application Composer is now generally available AWS CloudShell now supports the modular variant of AWS Tools for PowerShell  AWS Config now supports 18 new resource types  AWS Lambda now supports up to 10 GB of ephemeral storage for Lambda functions in 6 additional regions  AWS announces new competition structure for the 2023 Season AWS Resource Explorer supports 12 new resource types Announcing lower data warehouse base capacity configuration for Amazon Redshift Serverless Meet the Newest AWS Heroes – March 2023  Subscribe to AWS Daily Feature Updates via Amazon SNS Calculate Amazon DynamoDB reserved capacity recommendations to optimize costs How to use deletion protection to enhance your Amazon DynamoDB table protection strategy  Push notification engagement metrics tracking  Build Cloud Operations skills using the new AWS Observability Training 

On this episode of THRIVE — sponsored by E2M Solutions— Kelly and Corey Quinn discuss how saying no to non-ideal prospects is a business development strategy that allows you to scale your agency responsibly. Corey Quinn and I cover these points and more:How agencies can better serve their clients when they specialize;What to do when current clients ask for things outside of your agency's expertise;How variability reduction in your offerings allows you to scale more efficiently; andWhy saying no is a foundational strategy — regardless of economic cycles.Be sure to tune in to all the episodes of THRIVE to get practical tips on becoming a conscious leader, growing your agency, and more. Thanks for listening, and I'd love to hear your takeaways!If you enjoyed this episode, post it in your stories and tag me @agencyscaler. And don't forget to follow, rate, and review the podcast wherever you listen.Learn more about THRIVE at https://klcampbell.com/category/podcast/ and https://www.e2msolutions.com/thrive/ CONNECT WITH COREY QUINN:LinkedInInstagramWebsiteCONNECT WITH KELLY CAMPBELL:LinkedInInstagramTwitterWork with Kelly 

Emily Gorcenski, Data & AI Service Line Lead at Thoughtworks, joins Corey on Screaming in the Cloud to discuss how big data is changing our lives - both for the better, and the challenges that come with it. Emily explains how data is only important if you know what to do with it and have a plan to work with it, and why it's crucial to understand the use-by date on your data. Corey and Emily also discuss how big data problems aren't universal problems for the rest of the data community, how to address the ethics around AI, and the barriers to entry when pursuing a career in data. About EmilyEmily Gorcenski is a principal data scientist and the Data & AI Service Line Lead of ThoughtWorks Germany. Her background in computational mathematics and control systems engineering has given her the opportunity to work on data analysis and signal processing problems from a variety of complex and data intensive industries. In addition, she is a renowned data activist and has contributed to award-winning journalism through her use of data to combat extremist violence and terrorism. The opinions expressed are solely her own.Links Referenced: ThoughtWorks: https://www.thoughtworks.com/ Personal website: https://emilygorcenski.com Twitter: https://twitter.com/EmilyGorcenski Mastodon: https://mastodon.green/@emilygorcenski@indieweb.social TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. My guest today is Emily Gorcenski, who is the Data and AI Service Line Lead over at ThoughtWorks. Emily, thank you so much for joining me today. I appreciate it.Emily: Thank you for having me. I'm happy to be here.Corey: What is it you do, exactly? Take it away.Emily: Yeah, so I run the data side of our business at ThoughtWorks, Germany. That means data engineering work, data platform work, data science work. I'm a data scientist by training. And you know, we're a consulting company, so I'm working with clients and trying to help them through the, sort of, morphing landscape that data is these days. You know, should we be migrating to the cloud with our data? What can we migrate to the cloud with our data? Where should we be doing with our data scientists and how do we make our data analysts' lives easier? So, it's a lot of questions like that and trying to figure out the strategy and all of those things.Corey: You might be one of the most perfectly positioned people to ask this question to because one of the challenges that I've run into consistently and persistently—because I watch a lot of AWS keynotes—is that they always come up with the same talking point, that data is effectively the modern gold. And data is what unlocks value to your busin—“Every business agrees,” because someone who's dressed in what they think is a nice suit on stage is saying that it's, “Okay, you're trying to sell me something. What's the deal here?” Then I check my email and I discover that Amazon has sent me the same email about the same problem for every region I've deployed things to in AWS. And, “Oh, you deploy this to one of the Japanese regions. We're going to send that to you in Japanese as a result.”And it's like, okay, for a company that says data is important, they have no idea who any of their customers are at this point, is that is the takeaway here. How real is, “Data is important,” versus, “We charge by the gigabyte so you should save all of your data and then run expensive things on top of it.”Emily: I think data is very important, if you know what you're going to do with it and if you have a plan for how to work with it. I think if you look at the history of computing, of technology, if you go back 20 years to maybe the early days of the big data era, right? Everyone's like, “Oh, we've got big data. Data is going to be big.” And for some reason, we never questioned why, like, we were thinking that the ‘big' in ‘big data' meant big is in volume and not ‘big' as in ‘big pharma.'This sort of revolution never really happened for most companies. Sure, some companies got a lot of value from the, sort of, data mining and just gather everything and collect everything and if you hit it with a big computational hammer, insights will come out and somehow there's insights will make you money through magic. The reality is much more prosaic. If you want to make money with data, you have to have a plan for what you're going to do with data. You have to know what you're looking for and you have to know exactly what you're going to get when you look at your data and when you try to answer questions with it.And so, when we see somebody like Amazon not being able to correlate that the fact that you're the account owner for all of these different accounts and that the language should be English and all of these things, that's part of the operational problem because it's annoying, to try to do joins across multiple tables in multiple regions and all of those things, but it's also part—you know, nobody has figured out how this adds value for them to do that, right? There's a part of it where it's like, this is just professionalism, but there's a part of it, where it's also like… whatever. You've got Google Translate. Figure out yourself. We're just going to get through it.I think that… as time has evolved from the initial waves of the big data era into the data science era, and now we're in, you know, all sorts of different architectures and principles and all of these things, most companies still haven't figured out what to do with data, right? They're still investing a ton of money to answer the same analytics questions that they were answering 20 years ago. And for me, I think that's a disappointment in some regards because we do have better tools now. We can do so many more interesting things if you give people the opportunity.Corey: One of the things that always seemed a little odd was, back when I wielded root credentials in anger—anger,' of course, being my name for the production environment, as opposed to, “Theory,” which is what I call staging because it works in theory, but not in production. I digress—it always felt like I was getting constant pushback from folks of, “You can't delete that data. It's incredibly important because one day, we're going to find a way to unlock the magic of it.” And it's, “These are web server logs that are 15 years old, and 98% of them by volume are load balancer health checks because it turns out that back in those days, baby seals got more hits than our website did, so that's not really a thing that we wind up—that's going to add much value to it.” And then from my perspective, at least, given that I tend to live, eat, sleep, breathe cloud these days, AWS did something that was refreshingly customer-obsessed when they came out with Glacier Deep Archive.Because the economics of that are if you want to store a petabyte of data, with a 12-hour latency on request for things like archival logs and whatnot, it's $1,000 a month per petabyte, which is okay, you have now hit a price point where it is no longer worth my time to argue with you. We're just not going to delete anything ever again. Problem solved. Then came GDPR, which is neither here nor there and we actually want to get rid of those things for a variety of excellent legal reasons. And the dance continues.But my argument against getting rid of data because it's super expensive no longer holds water in the way that it wants did for anything remotely resembling a reasonable amount of data. Then again, that's getting reinvented all the time. I used to be very, I guess we'll call it, I guess, a data minimalist. I don't want to store a bunch of data, mostly because I'm not a data person. I am very bad thinking in that way.I consider SQL to be the chests of the programming world and I'm not particularly great at it. And I also unlucky and have an aura, so if I destroy a bunch of stateless web servers, okay, we can all laugh about that, but let's keep me the hell away from the data warehouse if we still want a company tomorrow morning. And that was sort of my experience. And I understand my bias in that direction. But I'm starting to see magic get unlocked.Emily: Yeah, I think, you know, you said earlier, there's, like, this mindset, like, data is the new gold or data is new oil or whatever. And I think it's actually more true that data is the new milk, right? It goes bad if you don't use it, you know, before a certain point in time. And at a certain point in time, it's not going to be very offensive if you just leave it locked in the jug, but as soon as you try to open it, you're going to have a lot of problems. Data is very, very cheap to store these days. It's very easy to hold data; it's very expensive to process data.And I think that's where the shift has gone, right? There's sort of this, like, Oracle DBA legacy of, like, “Don't let the software developers touch the prod database.” And they've kind of kept their, like, arcane witchcraft to themselves, and that mindset has persisted. But now it's sort of shifted into all of these other architectural patterns that are just abstractions on top of this, don't let the software engineers touch the data store, right? So, we have these, like, streaming-first architectures, which are great. They're great for software devs. They're great for software devs. And they're great for data engineers who like to play with big powerful technology.They're terrible if you want to answer a question, like, “How many customers that I have yesterday?” And these are the things that I think are some of the central challenges, right? A Kappa architecture—you know, streaming-first architecture—is amazing if you want to improve your application developer throughput. And it's amazing if you want to build real-time analytics or streaming analytics into your platform. But it's terrible if you want your data lake to be navigable. It's terrible if you want to find the right data that makes sense to do the more complex things. And it becomes very expensive to try to process it.Corey: One of the problems I think I have that is that if I take a look at the data volumes that I work with in my day-to-day job, I'm dealing with AWS billing data as spit out by the AWS billing system. And there isn't really a big data problem here. If you take a look at some of the larger clients, okay, maybe I'm trying to consume a CSV that's ten gigabytes. Yes, Excel is going to violently scream itself to death if I try to wind up loading it there, and then my computer smells like burning metal all afternoon. But if it fits in RAM, it doesn't really feel like it's a big data problem, on some level.And it just feels that when I look at the landscape of all the different tools you can use for things like this, they just feel like it's more or less, hmm, “I have a loose thread on my shirt. Could you pass me that chainsaw for a second?” It just seems like stupendous overkill for anything that I'm working with. Counterpoint; that the clients I'm working with have massive data farms and my default response when I meet someone who's very good at an area that I don't do a lot of work in is—counterintuitively to what a lot of people apparently do on Twitter—is not the default assumption of oh, “I don't know anything about that space. It must be worthless and they must be dumb.”No. That is not the default approach to take anything, from my perspective. So, it's clear there's something very much there that I just don't see slash understand. That is a very roundabout way of saying what could be uncharitably distilled down to, “So, is your entire career bullshit?” But no, it is clearly not.There is value being extracted from this and it's powerful. I just think that there's been an industry-wide, relatively poor job done of explaining that value in ways that don't come across as contrived or profoundly disturbing.Emily: Yeah, I think there's a ton of value in doing things right. It gets very complicated to try to explain the nuances of when and how data can actually be useful, right? Oftentimes, your historical data, you know, it really only tells you about what happened in the past. And you can throw some great mathematics at it and try to use it to predict the future in some sense, but it's not necessarily great at what happens when you hit really hard changes, right?For example, when the Coronavirus pandemic hit and purchaser and consumer behavior changed overnight. There was no data in the data set that explained that consumer behavior. And so, what you saw is a lot of these things like supply chain issues, which are very heavily data-driven on a normal circumstance, there was nothing in that data that allowed those algorithms to optimize for the reality that we were seeing at that scale, right? Even if you look at advanced logistics companies, they know what to do when there's a hurricane coming or when there's been an earthquake or things like that. They have disaster scenarios.But nobody has ever done anything like this at the global scale, right? And so, what we saw was this hard reset that we're still feeling the repercussions of today. Yes, there were people who couldn't work and we had lockdowns and all that stuff, but we also have an effect from the impact of the way that we built the systems to work with the data that we need to shuffle around. And so, I think that there is value in being able to process these really, really large datasets, but I think that actually, there's also a lot of value in being able to solve smaller, simpler problems, right? Not everything is a big data problem, not everything requires a ton of data to solve.It's more about the mindset that you use to look at the data, to explore the data, and what you're doing with it. And I think the challenge here is that, you know, everyone wants to believe that they have a big data problem because it feels like you have to have a big data problem if you—Corey: All the cool kids are having this kind of problem.Emily: You have to have big data to sit at the grownup's table. And so, what's happened is we've optimized a lot of tools around solving big data problems and oftentimes, these tools are really poor at solving normal data problems. And there's a lot of money being spent in a lot of overkill engineering in the data space.Corey: On some level, it feels like there has been a dramatic misrepresentation of this. I had an article that went out last year where I called machine-learning selling pickaxes into a digital gold rush. And someone I know at AWS responded to that and probably the best way possible—she works over on their machine-learning group—she sent me a foam Minecraft pickaxe that now is hanging on my office wall. And that gets more commentary than anything, including the customized oil painting I have of Billy the Platypus fighting an AWS Billing Dragon. No, people want to talk about the Minecraft pickaxe.It's amazing. It's first, where is this creativity in any of the marketing that this department is putting out? But two it's clearly not accurate. And what it took for me to see that was a couple of things that I built myself. I built a Twitter thread client that would create Twitter threads, back when Twitter was a place that wasn't overrun by some of the worst people in the world and turned into BirdChan.But that was great. It would automatically do OCR on images that I uploaded, it would describe the image to you using Azure's Cognitive Vision API. And that was magic. And now I see things like ChatGPT, and that's magic. But you take a look at the way that the cloud companies have been describing the power of machine learning in AI, they wind up getting someone with a doctorate whose first language is math getting on stage for 45 minutes and just yelling at you in Star Trek technobabble to the point where you have no idea what the hell they're saying.And occasionally other data scientists say, “Yeah, I think he's just shining everyone on at this point. But yeah, okay.” It still becomes unclear. It takes seeing the value of it for it to finally click. People make fun of it, but the Hot Dog, Not A Hot Dog app is the kind of valuable breakthrough that suddenly makes this intangible thing very real for people.Emily: I think there's a lot of impressive stuff and ChatGPT is fantastically impressive. I actually used ChatGPT to write a letter to some German government agency to deal with some bureaucracy. It was amazing. It did it, was grammatically correct, it got me what I needed, and it saved me a ton of time. I think that these tools are really, really powerful.Now, the thing is, not every company needs to build its own ChatGPT. Maybe they need to integrate it, maybe there's an application for it somewhere in their landscape of product, in their landscape of services, in the landscape of their interim internal tooling. And I would be thrilled actually to see some of that be brought into reality in the next couple of years. But you also have to remember that ChatGPT is not something that came because we have, like, a really great breakthrough in AI last year or something like that. It stacked upon 40 years of research.We've gone through three new waves of neural networking in that time to get to this point, and it solves one class of problem, which is honestly a fairly narrow class of problem. And so, what I see is a lot of companies that have much more mundane problems, but where data can actually still really help them. Like how do you process Cambodian driver's licenses with OCR, right? These are the types of things that if you had a training data set that was every Cambodian person's driver's license for the last ten years, you're still not going to get the data volumes that even a day worth of Amazon's marketplace generates, right? And so, you need to be able to solve these problems still with data without resorting to the cudgel that is a big data solution, right?So, there's still a niche, a valuable niche, for solving problems with data without having to necessarily resort to, we have to load the entire internet into our stream and throw GPUs at it all day long and spend hundreds of—tens of millions of dollars in training. I don't know, maybe hundreds of millions; however much ChatGPT just raised. There's an in-between that I think is vastly underserved by what people are talking about these days.Corey: There is so much attention being given to this and it feels almost like there has been a concerted and defined effort to almost talk in circles and remove people from the humanity and the human consequences of what it is that they're doing. When I was younger, in my more reckless years, I was never much of a fan of the idea of government regulation. But now it has become abundantly clear that our industry, regardless of how you want to define industry, how—describe a society—cannot self-regulate when it comes to data that has the potential to ruin people's lives. I mean, I spent a fair bit of my time in my career working in financial services in a bunch of different ways. And at least in those jobs, it was only money.The scariest thing I ever dealt with, from a data perspective is when I did a brief stint at Grindr because that was the sort of problem where if that data gets out, people will die. And I have not had to think about things like that have that level of import before or since, for which I'm eternally grateful. “It's only money,” which is a weird thing for a guy who fixes cloud bills for a living to say. And if I say that in a client call, it's not going to go very well. But it's the truth. Money is one of those things that can be fixed. It can be addressed in due course. There are always opportunities there. Someone just been outed to their friends, family, and they feel their life is now in shambles around them, you can't unring that particular bell.Emily: Yeah. And in some countries, it can lead to imprisonment, or—Corey: It can lead to death sentences, yes. It's absolutely not acceptable.Emily: There's a lot to say about the ethics of where we are. And I think that as a lot of these high profile, you know, AI tools have come out over the last year or so, so you know, Stable Diffusion and ChatGPT and all of this stuff, there's been a lot of conversation that is sort of trying to put some counterbalance on what we're seeing. And I don't know that it's going to be successful. I think that, you know, I've been speaking about ethics and technology for a long time and I think that we need to mature and get to the next level of actually addressing the ethical problems in technology. Because it's so far beyond things like, “Oh, you know, if there's a biased training data set and therefore the algorithm is biased,” right?Everyone knows that by now, right? And the people who don't know that, don't care. We need to get much beyond where, you know, these conversations about ethics and technology are going because it's a manifold problem. We have issues with the people labeling this data are paid, you know, pennies per hour to deal with some of the most horrific content you've ever seen. I mean, I'm somebody who has immersed myself in a lot of horrific content for some of the work that I have done, and this is, you know, so far beyond what I've had to deal with in my life that I can't even imagine it. You couldn't pay me enough money to do it and we're paying people in developing nations, you know, a buck-thirty-five an hour to do this. I think—Corey: But you must understand, Emily, that given the standard of living where they are, that that is perfectly normal and we wouldn't want to distort local market dynamics. So, if they make a buck-fifty a day, we are going to be generous gods and pay them a whopping dollar-seventy a day, and now we feel good about ourselves. And no, it's not about exploitation. It's about raising up an emerging market. And other happy horseshit that lies people tell themselves.Emily: Yes, it is. Yes, it is. And we've built—you know, the industry has built its back on that. It's raised itself up on this type of labor. It's raised itself up on taking texts and images without permission of the creators. And, you know, there's—I'm not a lawyer and I'm not going to play one, but I do know that derivative use is something that at least under American law, is something that can be safely done. It would be a bad world if derivative use was not something that we had freely available, I think, and on the balance.But our laws, the thing is, our laws don't account for the scale. Our laws about things like fair use, derivative use, are for if you see a picture and you want to take your own interpretation, or if you see an image and you want to make a parody, right? It's a one-to-one thing. You can't make 5 million parody images based on somebody's art, yourself. These laws were never built for this scale.And so, I think that where AI is exploiting society is it's exploiting a set of ethics, a set of laws, and a set of morals that are built around a set of behavior that is designed around normal human interaction scales, you know, one person standing in front of a lecture hall or friends talking with each other or things like that. The world was not meant for a single person to be able to speak to hundreds of thousands of people or to manipulate hundreds of thousands of images per day. It's actually—I find it terrifying. Like, the fact that me, a normal person, has a Twitter following that, you know, if I wanted to, I can have 50 million impressions in a month. This is not a normal thing for a normal human being to have.And so, I think that as we build this technology, we have to also say, we're changing the landscape of human ethics by our ability to act at scale. And yes, you're right. Regulation is possibly one way that can help this, but I think that we also need to embed cultural values in how we're using the technology and how we're shaping our businesses to use the technology. It can be used responsibly. I mean, like I said, ChatGPT helped me with a visa issue, sending an email to the immigration office in Berlin. That's a fantastic thing. That's a net positive for me; hopefully, for humanity. I wasn't about to pay a lawyer to do it. But where's the balance, right? And it's a complex topic.Corey: It is. It absolutely is. There is one last topic that I would like to talk to you about that's a little less heavy. And I've got to be direct with you that I'm not trying to be unkind, but you've disappointed me. Because you mentioned to me at one point, when I asked how things were going in your AWS universe, you said, “Well, aside from the bank heist, reasonably well.”And I thought that you were blessed as with something I always look for, which is the gift of glorious metaphor. Unfortunately, as I said, you've disappointed me. It was not a metaphor; it was the literal truth. What the hell kind of bank heist could possibly affect an AWS account? This sounds like something out of a movie. Hit me with it.Emily: Yeah, you know, I think in the SRE world, we tell people to focus on the high probability, low impact things because that's where it's going to really hurt your business, and let the experts deal with the black swan events because they're pretty unlikely. You know, a normal business doesn't have to worry about terrorists breaking into the Google data center or a gang of thieves breaking into a bank vault. Apparently, that is something that I have to worry about because I have some data in my personal life that I needed to protect, like all other people. And I decided, like a reasonable and secure and smart human being who has a little bit of extra spending cash that I would do the safer thing and take my backup hard drive and my old phones and put them in a safety deposit box at an old private bank that has, you know, a vault that's behind the meter-and-a-half thick steel door and has two guards all the time, cameras everywhere. And I said, “What is the safest possible thing that you can do to store your backups?” Obviously, you put it in a secure storage location, right? And then, you know, I don't use my AWS account, my personal AWS account so much anymore. I have work accounts. I have test accounts—Corey: Oh, yeah. It's honestly the best way to have an AWS account is just having someone else having a payment instrument attached to it because otherwise oh God, you're on the hook for that yourself and nobody wants that.Emily: Absolutely. And you know, creating new email addresses for new trial accounts is really just a pain in the ass. So, you know, I have my phone, you know, from five years ago, sitting in this bank vault and I figured that was pretty secure. Until I got an email [laugh] from the Berlin Polizei saying, “There has been a break-in.” And I went and I looked at the news and apparently, a gang of thieves has pulled off the most epic heist in recent European history.This is barely in the news. Like, unless you speak German, you're probably not going to find any news about this. But a gang of thieves broke into this bank vault and broke open the safety deposit boxes. And it turns out that this vault was also the location where a luxury watch consigner had been storing his watches. So, they made off with some, like, tens of millions of dollars of luxury watches. And then also the phone that had my 2FA for my Amazon account. So, the total value, you know, potential theft of this was probably somewhere in the $500 million range if they set up a SageMaker instance on my account, perhaps.Corey: This episode is sponsored in part by Honeycomb. I'm not going to dance around the problem. Your. Engineers. Are. Burned. Out. They're tired from pagers waking them up at 2 am for something that could have waited until after their morning coffee. Ring Ring, Who's There? It's Nagios, the original call of duty! They're fed up with relying on two or three different “monitoring tools” that still require them to manually trudge through logs to decipher what might be wrong. Simply put, there's a better way. Observability tools like Honeycomb (and very little else becau se they do admittedly set the bar) show you the patterns and outliers of how users experience your code in complex and unpredictable environments so you can spend less time firefighting and more time innovating. It's great for your business, great for your engineers, and, most importantly, great for your customers. Try FREE today at honeycomb.io/screaminginthecloud. That's honeycomb.io/screaminginthecloud.Corey: The really annoying part that you are going to kick yourself on about this—and I'm not kidding—is, I've looked up the news articles on this event and it happened, something like two or three days after AWS put out the best release of last years, or any other re:Invent—past, present, future—which is finally allowing multiple MFA devices on root accounts. So finally, we can stop having safes with these things or you can have two devices or you can have multiple people in Covid times out of remote sides of different parts of the world and still get into the thing. But until then, nope. It's either no MFA or you have to store it somewhere ridiculous like that and access becomes a freaking problem in the event that the device is lost, or in this case stolen.Emily: [laugh]. I will just beg the thieves, if you're out there, if you're secretly actually a bunch of cloud engineers who needed to break into a luxury watch consignment storage vault so that you can pay your cloud bills, please have mercy on my poor AWS account. But also I'll tell you that the credit card attached to it is expired so you won't have any luck.Corey: Yeah. Really sad part. Despite having the unexpired credit card, it just means that the charge won't go through. They're still going to hold you responsible for it. It's the worst advice I see people—Emily: [laugh].Corey: Well, intentioned—giving each other on places like Reddit where the other children hang out. And it's, “Oh, just use a prepaid gift card so it can only charge you so much.” It's yeah, and then you get exploited like someone recently was and start accruing $60,000 a day in Lambda charges on an otherwise idle account and Amazon will come after you with a straight face after a week. And, like, “Yes, we'd like our $360,000, please.”Emily: Yes.Corey: “We tried to charge the credit card and wouldn't you know, it expired. Could you get on that please? We'd like our money faster if you wouldn't mind.” And then you wind up in absolute hell. Now, credit where due, they in every case I am aware of that is not looking like fraud's close cousin, they have made it right, on some level. But it takes three weeks of back and forth and interminable waiting.And you're sitting there freaking out, especially if you're someone who does not have a spare half-million dollars sitting around. Imagine who—“You sound poor. Have you tried not being that?” And I'm firmly convinced that it a matter of time until someone does something truly tragic because they don't understand that it takes forever, but it will go away. And from my perspective, there's no bigger problem that AWS needs to fix than surprise lifelong earnings bills to some poor freaking student who is just trying to stand up a website as part of a class.Emily: All of the clouds have these missing stairs in them. And it's really easy because they make it—one of the things that a lot of the cloud providers do is they make it really easy for you to spin up things to test them. And they make it really, really hard to find where it is to shut it all down. The data science is awful at this. As a data scientist, I work with a lot of data science tools, and every cloud has, like, the spin up your magical data science computing environment so that your data scientist can, like, bang on the data with you know, high-performance compute for a while.And you know, it's one click of a button and you type in a couple of na—you know, a couple of things name, your service or whatever, name your resource. You click a couple buttons and you spin it up, but behind the scenes, it's setting up a Kubernetes cluster and it's setting up some storage bucket and it's setting up some data pipelines and it's setting up some monitoring stuff and it's setting up a VM in order to run all of this stuff. And the next thing that you know, you're burning 100, 200 euro a day, just to, like, to figure out if you can load a CSV into pandas using a Jupyter Notebook. And you're like—when you try to shut it all down, you can't. It's you have to figure, oh, there is a networking thing set up. Well, nobody told me there's a networking thing set up. You know? How do I delete that?Corey: You didn't say please, so here you go. Without for me, it's not even the giant bill going from $4 a month in S3 charges to half a million bucks because that is pretty obvious from the outside just what the hell's been happening. It's the little stuff. I am still—since last summer—waiting for a refund on $260 of ‘because we said so' SageMaker credits because of a change of their billing system, for a 45-minute experiment I had done eight months before that.Emily: Yep.Corey: Wild stuff. Wild stuff. And I have no tolerance for people saying, “Oh, you should just read the pricing page and understand it better.” Yeah, listen, jackhole. I do this for a living. If I can fall victim to it, anyone can. I promise. It is not that I don't know how the billing system works and what to do to avoid unexpected charges.And I'm just luck—because if I hadn't caught it with my systems three days into the month, it would have been a $2,000 surprise. And yeah, I run a company. I can live with that. I wouldn't be happy, but whatever. It is immaterial compared to, you know, payroll.Emily: I think it's kind of a rite of passage, you know, to have the $150 surprise Redshift bill at the end of the month from your personal test account. And it's sad, you know? I think that there's so much better that they can do and that they should do. Sort of as a tangent, one of the challenges that I see in the data space is that it's so hard to break into data because the tooling is so complex and it requires so much extra knowledge, right? If you want to become a software developer, you can develop a microservice on your machine, you can build a web app on your machine, you can set up Ruby on Rails, or Flask, or you know, .NET, or whatever you want. And you can do all of that locally.And you can learn everything you need to know about React, or Terraform, or whatever, running locally. You can't do that with data stuff. You can't do that with BigQuery. You can't do that with Redshift. The only way that you can learn this stuff is if you have an account with that setup and you're paying the money to execute on it. And that makes it a really high barrier for entry for anyone to get into this space. It makes it really hard to learn. Because if you want to learn anything by doing, like many of us in the industry have done, it's going to cost you a ton of money just to [BLEEP] around and find out.Corey: Yes. And no one likes the find out part of those stories.Emily: Nobody likes to find out when it comes to your bill.Corey: And to tie it back to the data story of it, it is clearly some form of batch processing because it tries to be an eight-hour consistency model. Yeah, I assume for everything, it's 72. But what that means is that you are significantly far removed from doing a thing and finding out what that thing costs. And that's the direct charges. There's always the oh, I'm going to set things up and it isn't going to screw you over on the bill. You're just planting a beautiful landmine you're going to stumble blindly into in three months when you do something else and didn't realize what that means.And the worst part is it feels victim-blamey. I mean, this is my pro—I guess this is one of the reasons I guess I'm so down on data, even now. It's because I contextualize it in a sense of the AWS bill. No one's happy dealing with that. You ever met a happy accountant? You have not.Emily: Nope. Nope [laugh]. Especially when it comes to clouds stuff.Corey: Oh yeah.Emily: Especially these days, when we're all looking to save energy, save money in the cloud.Corey: Ideally, save the planet. Sustainability and saving money align on the axis of ‘turn that shit off.' It's great. We can hope for a brighter tomorrow.Emily: Yep.Corey: I really want to thank you for being so generous with your time. If people want to learn more, where can they find you? Apparently filing police reports after bank heists, which you know, it's a great place to meet people.Emily: Yeah. You know, the largest criminal act in Berlin is certainly a place you want to go to get your cloud advice. You can find me, I have a website. It's my name, emilygorcenski.com.You can find me on Twitter, but I don't really post there anymore. And I'm on Mastodon at some place because Mastodon is weird and kind of a mess. But if you search me, I'm really not that hard to find. My name is harder to spell, but you'll see it in the podcast description.Corey: And we will, of course, put links to all of this in the show notes. Thank you so much for your time. I really appreciate it.Emily: Thank you for having me.Corey: Emily Gorcenski, Data and AI Service Line Lead at ThoughtWorks. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry, insipid, insulting comment, talking about why data doesn't actually matter at all. And then the comment will disappear into the ether because your podcast platform of choice feels the same way about your crappy comment.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Jessica M. Luciere is a transracial adoptee born in Bogota, Colombia who grew up on Long Island, New York. She is an Adoptee Advocate, working to create supportive spaces for adoptees and their families around the world. Currently the Manager of Community Engagements at Spence-Chapin, Jessica is also the facilitator of the Mentorship Programs, which have been in existence since 2005. Jessica is the former President of All Together Now, Inc based in Brooklyn NY, and has also worked with Adoptive and Foster Family Coalition of New York (AFFCNY). Jessica's passion for working with adoptees has always been the driving force behind her work, and is the reason she continues to create spaces for adoptee stories and voices to be heard by all.Music by Corey Quinn

AWS Morning Brief for the week of March 6, 2023 with Corey Quinn. Links: Amazon Aurora Serverless v1 now supports customer configurable maintenance windows Amazon CloudWatch Internet Monitor is now generally available AWS Lambda Powertools for .NET is now generally available Amazon Neptune Serverless now scales down to 1 NCU to save costs  AWS Control Tower announces a progress tracker for landing zone setup and upgrades In the Works – AWS Region in Malaysia  New – Amazon Lightsail for Research with All-in-One Research Environments  Announcing Amazon ECS Task Definition Deletion  Announcing the end of Windows Installer support for AWS Tools for Windows “Avatar: The Way of Water” and the future of filmmaking  A detailed overview of Trusted Advisor Organizational Dashboard 

Ed Knight graduated George Washington University with a B.A. in Journalism. (his late adoptive father studied law there.) He worked at The Washington Post for many years in the Circulation and I.T. Departments, starting as a clerk, and ending up as a database administrator.Ed took an early retirement after finding his biological family and needing time to, “find myself”. In Reunion with biological family, and still close to adoptive family Ed just consider them one big family now, from his point of view. He married Steven Frank in 2009, and in relationship since 1982. Steven was born Deaf and went blind later in life. Consequently, Ed knows ASL, and tactile ASL. He is active in his church, St. Barnabas Church of the Deaf (Episcopal), as volunteer.A member of Sons of the American Revolution after proving descent from patriot ancestor Joshua Smith, with the help of his natural mother, who is a former member of the Daughters of the American Revolution. Ed is active in his local chapter. His late natural father was also active in the SAR, having served a term as President of the DC Society of the SAR.Ed is in Year Three of the Master Artist Program at The Compass Atelier, in Rockville, MD. His website is epscottart.comEd referenced Adoptees United and Gregory Luce https://adopteesunited.org/Music by Corey Quinn and Invitational by MDT

AWS Morning Brief for the week of February 27, 2023 with Corey Quinn. Links: Amazon OpenSearch Service now lets you schedule service software updates during off-peak hours  AWS App Runner now supports HTTP to HTTPS redirect Announcing the ability to enable AWS Systems Manager by default across all EC2 instances in an account  New: AWS Telco Network Builder – Deploy and Manage Telco Networks Developing portable AWS Lambda functions Using Porting Advisor for Graviton  Query data with DynamoDB Shell – a command line interface for Amazon DynamoDB AWS and Hugging Face collaborate to make generative AI more accessible and cost efficient Branch Insurance improves hiring diversity and accelerates app development using AWS AppSync Gain compliance insights using the open source community for AWS CloudTrail  The true costs of resiliency decisions 

Valerie Naiman (pronounced naymen) is an adoptee, author, story-song writer, passionate singer, re-purposer, goat mama, apiarist, gardener, environmental activist, and eco-village founder living in Asheville, NC. As you'll discover in her soon to be release book, she's a secret to her natural family. Her personal journey throughout six decades delves into life's biggest mysteries as she searched for her identity.Join Valerie on social media; http://www.instagram.com/valnaimanauthor https://www.facebook.com/valnaimanauthorValerie's Website: http://valerienaiman.comMusic by Corey Quinn

Greg Gentry is a domestic baby scoop era adoptee, born in California in 1969. He has been in reunion with maternal family members since 2006, and in 2021 also connected with his paternal side. Greg is a facilitator and interviewer for Fireside Adoptees, a private Facebook group founded in 2021, which is committed to additional outreach through its public Facebook page and through the Fireside Adoptees Constellation private Facebook group. Greg is also an administrator and facilitator within the Adoption Trauma Network and the host of Adoptees Connect out of Derry, New Hampshire. He enjoys connecting with others in the online adoptee community, and has found these interactions to be richly rewarding and supportive. Fireside Adoptees https://www.facebook.com/groups/1411791922534076Fireside Adoptees Public Page https://www.facebook.com/fireside.adoptees.and.alliesGreg Gentry's conversation with Lori Holden https://www.youtube.com/watch?v=9W3oon6TivYNAAP presentation on Geneological Non-Facility https://www.youtube.com/watch?v=2El1j3kfYHw&t=6sMusic by Corey Quinn

To continue on the sales theme this week, we'll dissect everything you need to know about winning and retaining right-fit clients, honing in on your agency vertical, and strategizing your agency growth plan. Our guest, Corey Quinn, has a 25-year track record of extraordinary success as an entrepreneur, sales leader, and CMO for a $150M+ company. Today, he helps B2B SaaS and Agencies grow from 7 figures to 8 by doing less, not more.   In this episode, Corey will share some wisdom from his successful business career to teach us how we, as small agency owners, can implement an effective agency growth plan. He'll teach us how to keep right-fit clients on our roster and establish a vertical that gets even more customers coming to you for your expertise and fantastic customer service. A big thank you to our podcast's presenting sponsor, White Label IQ. They're an amazing resource for agencies who want to outsource their design, dev, or PPC work at wholesale prices. Check out their special offer (10 free hours!) for podcast listeners here. What You Will Learn in This Episode: Growth and retention strategies that are applicable to small agencies Honing in on your agency growth plan The power of niching and word-of-mouth connections Leveraging both inbound and outbound sales tactics The common mistakes people make when trying to increase sales How much time an agency owner should dedicate to biz dev Why specializing in specific customers and industries works so well 2 or 3 things you need to have exponential growth Sales strategies that will have new customers coming to you for work

AWS Morning Brief for the week of February 13, 2023 with Corey Quinn. Links: Amazon Chime SDK now offers a Windows client library Amazon CloudWatch now supports high resolution metric extraction from structured logs AWS SAM CLI introduces ‘sam list' command to inspect AWS SAM resources  Get cost estimates faster with AWS Pricing Calculator bulk import  New – Visualize Your VPC Resources from Amazon VPC Creation Experience  Introducing the AWS ProServe Hadoop Migration Delivery Kit TCO tool  Introducing the Amazon EKS Workshop  Using GitHub Actions with Amazon CodeCatalyst  Using Amazon CloudWatch metrics to monitor time to expiration for Reserved Instances 

Brian Stanton is a Los Angeles-based stage actor. He's currently starred in a new play titled Final Interview in Los Angeles and will immediately move on to star in Plaza Suite by Neil Simon in February in Mammoth Lakes, CA. In the world of adoption, Stanton is most known for his original solo play BLANK, the true story of his adoption and search for identity. BLANK's Hollywood premiere catapulted Stanton to perform all over the US and Canada, earning Best Solo Acting awards in New York and Los Angeles. Stanton's most recent work @ghostkingdom, originally written as a play, was filmed during the pandemic due to theatre closures. Stanton has held screenings of his original film at film festivals as well as for adoption support and educational conferences. The Love International Film Festival honored Stanton with Best Screenwriter and Best Actor awards. Stanton will be presenting @ghostkingdom in person at the NAAP/Right To Know conference in Louisville, KY late March https://untanglingourroots.org/?fbclid=IwAR2rG6DBOsuHFzPr_NqZufPHVDTCBDSyGEAgSF17AkKMXDZt3i4X7dEHmsY and a few months later in June for the Forget Me Not Family Society in Vancouver. @ghostkigdomis available for the public to rent. The direct link is: https://vimeo.com/ondemand/ghostkingdomMusic by Corey Quinn and Invitational by MDT

About CatherineCatharine brings more than fifteen years of experience building global networks and large scale data center infrastructure to the challenge of scaling quickly and safely.  She loves building engaged and curious teams, providing insightful forecasting tools, and thinking about how to build to scale in a sustainable way to preserve a humane quality of life on this swiftly tilting planet. When not trying to predict the future as a capacity planner, she's often knitting extremely complicated sweaters and coming up with ridiculous puns.TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Tailscale SSH is a new, and arguably better way to SSH. Once you've enabled Tailscale SSH on your server and user devices, Tailscale takes care of the rest. So you don't need to manage, rotate, or distribute new SSH keys every time someone on your team leaves. Pretty cool, right? Tailscale gives each device in your network a node key to connect to your VPN, and uses that same key for SSH authorization and encryption. So basically you're SSHing the same way that you're already managing your network.So what's the benefit? Well, built-in key rotation, the ability to manage permissions as code, connectivity between any two devices, and reduced latency. You can even ask users to re-authenticate SSH connections for that extra bit of security to keep the compliance folks happy. Try Tailscale now - it's free forever for personal use.Corey: Kentik provides Cloud and NetOps teams with complete visibility into hybrid and multi-cloud networks. Ensure an amazing customer experience, reduce cloud and network costs, and optimize performance at scale — from internet to data center to container to cloud. Learn how you can get control of complex cloud networks at www.kentik.com, and see why companies like Zoom, Twitch, New Relic, Box, Ebay, Viasat, GoDaddy, booking.com, and many, many more choose Kentik as their network observability platform. Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. As a cloud economist, I wind up talking to an awful lot of folks about optimizing their AWS bills. That is what it says on the tent. It's what I do. Increasingly, I'm having discussions around the idea of sustainability because the number-one rule of cloud economics is also the number-one rule for sustainability. Step one, turn that shit off. If you're not using it, turn that shit off. If it doesn't add value commensurate to what it costs, turn that shit off. Because the best way to optimize something is to get rid of it. Today, to go into bit more depth on that, my guest is Catharine Strauss. Catharine, thank you for joining me.Catharine: Thank you. I'm excited.Corey: So, you have a long and storied career of effectively running global-scale network operations in terms of capacity planning, in terms of building out world-spanning networks, and logistics of doing that. You know, the stuff that's completely invisible to most people, except when it breaks. So, it's more or less a digital plumbing-type of role. How did you go from there to thinking about sustainability in a networking context?Catharine: Yeah. Thank you. I got dropped into networking as a career option, completely from the physical side, building out global networks. And all of the constraints that we were dealing with, were largely physical, logistical, or legal. So, we would do things like ship things through customs and have items stopped because they were miscategorized as munitions because they were lasers, “Pew, pew.” We had things like contract negotiations for data centers to do trenching into them that needed easements with the railroad. Like, just weird stuff that you don't normally think of as a cloud-project constraint. So, all of these physical constraints made it just more interesting to me because they were just so tactile.Corey: There's so much that is out there in the world that is completely divorced from anything that you have to think about in terms of building out networks and software. Until, suddenly, it's very much there, and you're learning that there's an entire universe/industry/ecosystem that you know nothing about that you now need to get into. Railroad easements are a terrific example of that. It's, “Wait, what, we're building the cloud here. What the hell does the railroad have to do—is there actually a robber baron I need to go fight somewhere? How does this work?” The old saw about the cloud just being someone else's computer is not particularly helpful, but it is true. There's a tremendous amount of work that goes into building out the physical footprint for a data center—let alone a hyper-scale cloud provider's data center—that does not have to be something the vast majority of us need to think about anymore. And that's, kind of, glorious and magical. But it does mean that there are people who very much need to think about that.Increasingly, we're seeing the sustainability and climate story of cloud extend beyond those folks. There are no carbon-footprint tools and dashboards in all the major cloud providers that I'm aware of. Well, I'd say it's a good start, but in some cases, it's barely that. It feels like this is something that people are at least starting to take semi-seriously in the context of cloud. How have you seen that evolving?Catharine: So, when I think about a data center, I see it as a factory where you take heavy metals and electricity—Corey: And turn them into YAML. Sorry, sorry. Go ahead.Catharine: [laugh] you turn them into spreadsheets, cat videos, and waste heat, right? So, when I'm looking at, you know, this tremendous global network, I started to look into what's the environmental cost of that. And what I found was, kind of, surprising. Like, three percent of our total global emissions, is coming from computing and the internet, and all of these things that I spent my career building. And I started to have waves of regret. And looking at that in the context of: how can we make things better? How can we make things more efficient, and how can we operate better with the physical constraints of electricity and energy grids, and what they are struggling with doing to provide us with what we need or managing this beast of an internet?Corey: Right now, it feels like there's an awful lot of—I don't know what the term is, greenwashing, cloud washing—basically, making your problem someone else's problem. I feel like the cloud providers are in a position where they have to walk something of a tightrope. Because on the one hand, yeah, there are choices I can make as a customer that will absolutely improve the carbon footprint of what it is that I'm doing. On the other, they never invite me to have conversations to negotiate with their energy providers around a lot of these things. So, it feels like, “Oh, yeah. Make sure that the cloud you're using is green enough.” “Wasn't that what I'm paying you for?” That feels like it's a really weird dichotomy that I'm still struggling to reconcile exactly how to approach.Catharine: Yeah. I, you know, I looked at the Amazon Sustainability platform, and they've got those two parts of it. They've got sustainability in the cloud, sustainability of the cloud. And, you know, I've worked with enough Google SREs to know that they and Amazon Data Center providers and Azure, they all have a vested interest in making it as cheap as possible to operate their data centers. And that goes far beyond individual server performance. It goes to the way that they do cooling. And, like, the innovations there are tremendous.But they're not doing that out of the goodness of their heart; they're doing that because it makes business sense for them. It reduces the cost for them to provide these services. And, you know, in some cases, it really obscures things because they will sign energy contracts and then keep them super-secret. There's very little transparency because these are industry secrets, and they don't want to damage their negotiation positions for the next deal that they sign. So, Amazon, you know, will put PR releases out there about all of their solar farms that they are sustaining in Virginia. But they don't talk about what percent that is of their total energy consumption, and they don't talk about, you know, what the total footprint is because that is considered either a security risk or an economic risk if people were to find out, you know, exactly how much energy they're pulling.Corey: I am, somewhat, sympathetic, but only to the reality that the more carbon transparency that a cloud provider gives around the relative greenness of a given service that they offer in a given region, the closer they get to exposing a significant component on their per-service margins. And they're, understandably, extraordinarily reluctant to that because then people will do things like figure out exactly how much are they up-charging things like data egress and ongoing per-hour session charges for some sage-maker nonsense.There's an awful lot out there that I don't think they want to have out there just for, on the one hand, the small one that's easy to deal with is the customer uprising. But more so, they don't want to expose this to their competitors.Catharine: Yeah, I don't know that I have a ton of sympathy. If the service is cheaper because they're running off of green energy, as we have increasingly seen in the market that solar and wind are just the cheapest alternative. If it's cheaper for Amazon and Google, I, kind of, feel like they should convey that, so that people can take advantage of those savings.We've got a demand issue, where, I think, the demand for these renewable energy sources is outstripping supply. But they're planning for the next five years where that decreasingly becomes an issue. So, why not let people operate according to their values, or even, you know, their own best interests in choosing data centers that are emitting fewer emissions into the world?Corey: There seems to be a singular focus between all of these providers in what they're displaying through their tools. And that is on carbon footprint, and it is also suspiciously, tightly bounded to what looks like compute. There're a lot of other climate-impacting effects of large-scale cloud providers. It has significant disruption to local waterways. There are tremendous questions around the sustainability around manufacturing of the various components that get turned into equipment that gets sold to these providers then integrating into other things. There's an awful lot of downstream effects. And I can't shake the feeling that focusing on how renewable the energy is to power the compute, focuses on a very small part of the story. How do you land on that one?Catharine: I would agree with that. I think people will often say, “Oh, what you should do if you're managing,” you know, “Your data center resources is for efficiency, you should be updating your hardware once a year or putting out the resources that are the most powerful.” The tipping point might be later than you actually think because what happens to those resources when they go back out into the environment when you decommission them? It's so hard to resell them, especially, globally. The reuse of gear is becoming harder and harder, and so the lifetime of that gear, that equipment, those servers, routers, whatnot, all of that is becoming harder and harder to do. And the disposal of those materials has a tremendous impact.So, I do think the energy is a big part of it, and it feels like the thing that we can control the most. But, like, if you really want to change the world, go work on carbon-neutral cement or batteries made out of rust and sand to store solar energy. You know, go work on low-heat steel. Those are the things where you're really making an impact. What we need to do in the market is really transform our notion of the cloud as this infinite nebulous, weightless item into something that is physical and has a physical impact on our lives.So, when you're trying to decide what your retention policy is for your data in your company when you're trying to decide where to replicate data, how long to hold it in active storage, you're really thinking about the megawatts that it takes, and the impact of that on the full picture.Corey: Well, a question that I've had as I look across my customer base of large companies doing interesting and exciting things with cloud, is I would love—absolutely love—to see a comparative analysis done by each provider that in very human terms, says what the relative climate impact is of taking all of their different storage services, on a per-petabyte basis, where I say, “Okay, if I want to store this in their object storage, or if I want to put this on disc volumes, or I want to use their deep-archive storage that looks an awful lot like tape, I don't care so much about the cost of those things, but I want to know what is the climate impact of this,” because I think that would be revelatory on a whole bunch of different levels. But it seems it's computes where they tend to focus instead.Catharine: Yeah, it would be really nice if as businesses, we started to look at the fuller impact of our actions. And it isn't just about the money saved. But my genuine belief is that it will get cheaper to do the right thing. And it is getting cheaper every day to use fewer resources. But the market has not caught up to that, and you can see that in how many companies are still giving away free, unlimited storage, right? You know, how many Go-Pro videos of someone's backyard, how many hours of that kind of footage is there out there in the world that's never going to get viewed again, but is sitting out there taking up energy that, at the same time, that we're having brownouts, and people are suffering and having to turn off their air conditioning?Corey: I think that we would do well as a society to get rid of a heck of a lot more data just because it sits there; it burns energy; it costs money, and I'm sorry, you're going to really have to reach to convince me that the web server access logs from 2012 are in any way business valuable or relevant to, basically, anyone out there.But I want to take it one step further because now that we know that we're definitely burning the planet to wind up storing a petabyte of data here, I'm very curious as to the climate footprint of then going into your world, taking that data, and throwing it somewhere else across the internet. Because I can tell you, almost to the penny, what that's going to cost, and it's an astonishingly large number because yeah, egress fees are what they are, but I couldn't tell you what the climate footprint of that is.Catharine: Yeah. When I was working at Fastly, we did a lot of optimizations across our network to avoid peak traffic because that was how we were built. You know, we had to build out to a certain network capacity, and then we could build, essentially, the area under our diurnal curve, we can build that out. But we don't have to, necessarily, serve it from the absolute closest data center. If we could serve it from a nearby data center or a provider that was three milliseconds of ‘wait and see more,' we could potentially use resources that we have elsewhere in the cloud to serve that request more efficiently.And I think we have an opportunity to do that with data centers scattered around the globe. Why aren't we load balancing so that we're pushing traffic from the data centers that are off-peak—you know, have energy to spare to accommodate for the data centers that are reaching capacity and don't have enough energy on the grid—why aren't we using these resources more efficiently?Corey: I've often lamented, from an economic perspective, that if I want to spend less money and optimize things, I can wind up trading out my instance types. Okay, I have a super-fast, high-end processor that costs a lot of money. I can get shittier compute by spending less. The same story with storage. I can get slower storage for less money that's a lot less performing, and it has some latencies added, but, “Great,” but I can make that decision.With networking, it's all of its nothing. It's there is no option for me to say, “I want to pay half of what the normal data rates are, but in return, I really only care that this data gets to where it's going by next Tuesday.” I don't need it done in sub-second latency speeds. There's no way to turn that off or to make that election. Increasingly, I really am coming around to the idea that cloud economics and sustainability are one in the same.Catharine: Yeah. For me, it makes a lot of sense. And, you know, when I look at people in their careers, focusing on cloud economics feels like a very, very easy win if you also care about sustainability. And it feels like once you have the data and the reporting tools—and, you know, we talked about the big gaps there—but if you're reporting on both your costs and the carbon footprint, you're developing a plan for how to optimize on both of them at the same time, and you're bringing that back to your management, bringing that back to your teammates, and really making sustainability an active value in your organization.I feel like there's not only a benefit to you, the finances of your company, and your personal career, but there's also a social impact where, you know, maybe you can feel a little less guilty about eating that steak. Maybe you can offset some travel that is increasing your carbon footprint; maybe you can do a trade-off; maybe you can do everything in little bursts across a broad scope, instead of us needing, you know, some big solution that's going to save us. There's no one solution.I think that's the main thing I've discovered in my education on sustainability is it has to be 50,000 small things, the ‘magic buckshot' rather than the ‘magic bullet,' is the term that I see used a lot. Carbon removal from the sky is coming, but while we wait for it, we got to slow the pace of digging the hole, and really give our solutions a chance to work.Corey: I despair at times at the lack of corporate will, I suppose, to wind up pursuing cloud sustainability as a customer of one of the cloud providers. I get people reaching out to me, pretty frequently, to help optimize the cost of their AWS bill. That is, definitely, what I do for a living. If I don't have people reaching out on that, something is going wrong somewhere. And even then, there have been months that have been relatively slow in recent years. Because well, it turns out when money is free, you don't really care that much about saving money. Now, people are tightening their belts and have to think about it a lot more, but that is a direct incentive of if you go ahead and optimize your cloud-spend bill, you will have more money.That is, sort of, what our capitalist system is supposed to optimize for in many respects. “Great,” you can have more money. But it's still not exciting for folks, and it's not what they really wind up chasing after. I despair at getting them to think larger than money because that's the only thing that companies generally tend to think about in the abstract, and start worrying about the future and climate and to invest significant effort in doing climate optimization. I don't know that there is a business today in greening your cloud workloads that could be started the way that I have for fixing the AWS bill.Catharine: Yeah, I don't think there's a business in it; I think it's a movement. It's like accessibility; it's like security; it's like a lot of other movements that have happened recently in tech where it becomes everybody's job. And it's important to people. And it becomes part of your company's brand, and you use it for recruitment; you use it for advancing your own career; you use it for making people feel like they're making a better decision.When I look at the three big cloud providers, and I look at the ways that they are marketing their sustainability, it is so slick. You go to their sustainability page and it's all, you know, beautiful, flashy graphics and information on all these feel-good things. Because they know, if they don't do it, they're going to be passed over because somebody is going to bring this up when they're evaluating their choices. Because we want it; we all want it. We just don't quite know how to get there. And until recently, it was more expensive, and you did have a green tax made the sustainable options more expensive. We're turning the page on that. Solar is cheaper than coal. And that's all you really—all you have to say to justify some of these advancements. It's all going to flow out of that simple fact.Corey: Cloud native just means you've got more components or microservices than anyone (even a mythical 10x engineer) can keep track of. With OpsLevel, you can build a catalog in minutes and forget needing that mythical 10x engineer. Now, you'll have a 10x service catalog to accompany your 10x service count. Visit OpsLevel.com to learn how easy it is to build and manage your service catalog. Connect to your git provider and you're off to the races with service import, repo ownership, tech docs, and more. Corey: I think that there's a tremendous opportunity here to think about this. And I think you're right. It absolutely takes on aspects of looking like a movement to do that. I'm optimistic about that. The counterpoint is that individuals are often not tremendously effective at altering the behavior of trillion-dollar companies, or even the relatively small ‘only' 50 billion-dollar companies out there.I can see where it starts, and I can see the outcome that you want there. I just have no idea what it looks like in between. It's, like, “Step two, we'll figure this part out later. Step three, climate.”Catharine: Yeah. If I were going to do it at my company, I would go to HR. And I would say, “I would like to form an employee-resource group around sustainability. Do you know anyone on the executive team who is interested in sustainability?” Get them to sponsor it; talk to that sponsor, and say, “We're the co-benefits here. What do you see as things that we absolutely need to do from a corporate-strategy standpoint that are aligned with this?”And then start having meetings—open meetings—where you invite people concerned about climate change, and you start to talk cross-functionally about, “What can we do? Can we change our retention policies? Can we change the way that we bill for services? Can we individually delete data on our Wiki that hasn't been accurate for seven years?” And, you know, start to talk and share successes. Then take it out to the larger industry and start giving talks because people want to be able to do something. Climate despair is real, but we, as cloud technologists, are so powerful in the resources that we have stewardship over. But I have to think that there is a possibility of making real change here.Corey: There's a certain point of scale at which point, having a sustainability conversation becomes productive. There are further points of scale where it becomes mandatory, let's be clear here. But when I'm building something in the off hours—mostly for shit-posting purposes—it generally tends to wind up costing maybe seven cents or so, when all is said and done because I'm using Lambda functions and other things that don't take a whole lot of computer resources out there. Googling what the most climate-effective way to implement that would be, is one of those exercises where the google search has a bigger carbon footprint than the entire start-to-finish of what it is that I'm building. It's not worth me looking into that.There is some inflection point between that, and we run 500,000 servers around the world or 500,000 instances where, yeah, there's a definite on-ramp where you need to start thinking about these things. What is that, I guess, that first initial point of, “I should be thinking about this,” for a given workload?Catharine: So, I've been trying to get data on this, and my best calculation is that an average server in a hyperscale data center, where you're using the whole thing for an entire year, is one to two tons of CO2 per year. So, I think when you start to look at other initiatives that you're seeing, I think the tipping point is around ten tons per year. And for some people, that's a lot; that's a lot of resources that you need to get up to that point.Corey: That feels directionally right. I think that is absolutely around where it starts to make sense. I mean, right now, I'm also in the uncomfortable creeping-awareness position of I've run a medium-sized EC2 instance persistently. That is my developer environment. I have it running all the time because having a Linux box is, sort of, handy. And whether I need it or not, it's there. If I were to turn it off when I go to sleep at night, for example, I do not believe that would have any climate impact whatsoever from the perspective of this is a medium-size instance. There are a bunch of those on any individual server.Amazon is not going to turn off Iraq right now because my instance is there or it's not. It is well within the margin of error for anything they have as far as provisioning or de-provisioning something. So, then someone, like, steals it to the term you used of climate despair a few minutes ago, that's what this feels like. It's one of those, “Well, okay. So, if it makes no actual difference if I were to spend instrumenting that thing to turn itself off at night and turn itself back on in the morning, it doesn't change a damn thing. I'm just doing something that is effectively meaningless in order to make myself feel better.”The enormity of the problem and the task, and doing it at scale, well, I'm not going to convince customers to do that. And for some cases, maybe that's for the better; maybe it's not. But I feel like for whatever I do, there's nothing I can do to make a difference in that sense, in my small-scale personal environment.Catharine: Yeah, yeah. I definitely appreciate that. This feel to me like the same concept of—I don't know, a couple of months ago, if you remember, California had a heat wave, and there were rolling brownouts. And we got a text that said, “Energy is at a high right now. Please turn off any unnecessary devices,” trying to avoid additional impact to the energy grid. And if you go and you look at the graph, there was an immediate decrease of 1500 megawatts in that moment because enough people got the text and took a small action, and it had the necessary impact. We avoided the brownouts, and the power, generally, kept flowing because it's such a big system.You know, if we're talking about three percent of global emissions, we're talking about, you know, power that's the size of the aviation industry. We're talking about power that's, roughly, the size of Switzerland just on data centers. You, as an individual, are not going to be able to make an impact; you, as an individual talking about this to as many people as possible—as we're doing right now—that starts to move the needle. And the thing I like about forming a grass roots group inside of your company is that it's not just about the data centers. Maybe, it's also about the service that comes in and brings you food and uses disposable containers; maybe, it's about people talking about their electric cars; maybe, it's about installing a heat pump; maybe, it's about talking about solutions instead of just talking about creeping dread all the time.Like, my move into sustainability has been largely in response to I can't keep doom-scrolling. I have to find the people who are making the solutions happen. And I just got out of a program with Climatebase where that is what I did for nine weeks is talk about the solutions. And all of the people in the companies that are actually doing something, they're so much more optimistic than the people I talk about who are just reading the headlines.Corey: Doing something absolutely feels better than sitting here helplessly and more less doom-scrolling about it. I absolutely empathize there. I think the trick is to get people to start taking action on this. I am curious, getting a little bit back to where you come from, something you alluded to at one point, was how energy markets are akin to network throughput. And I definitely wanted to dive into that. What do you mean? I'm not disagreeing, but I also have a really hard time seeing that. Help?Catharine: Yeah. So, I used to do capacity planning for Fastly. And so, we would spend all day staring at the diurnal curve of our network throughput because we had to plan for the peak. Whatever our traffic throughput was, our global network needed to be able to handle it. And every day—maybe we got close to that peak; maybe we didn't—but every day it would dip down into just the doldrums as people went to sleep and weren't using the internet.So, when I moved into looking at energy markets, specifically smart grids, and the way that renewables affect the available supply of electricity, I saw that same electricity curve; it's called the duck curve in electricity markets where you have this diurnal pattern and a point every day, where the grid has electricity available but no demand.So, when I was managing costs for our network, we would be trying, as much as possible, to fill that trough every day because it was free for us because we had already built out the infrastructure to fulfill that demand.And the energy markets are same way. We have built out the infrastructure. We just need the demand to meet the timing of the day. Put another way, you have to think fourth-dimensionally. It's like Doc Brown in Back to the Future III. Marty says, “If we continue along this track, the bridge isn't built yet. We're going to plunge into the canyon and die.” And Doc Brown says, “No, no, no. You're not thinking fourth-dimensionally. When we travel through time, we will be in the future, and the bridge will be there.” So, if we can shift the load from one region where energy is being consumed at its peak and move the traffic over to a region in the Pacific Northwest or a different time zone where they haven't yet hit their energy-consumption peak, we can more efficiently use the infrastructure that is already been built out.Corey: I really wish things were a lot easier to move around in that context. Data transfer fees make that very challenging, even if you can get around the latency challenges—which for many workloads is fine; that is not a prohibitive challenge. It's the moving things around; moving data to those other regions, especially, in the sense of, “But, okay. You're making it worse because now you have the data living in two different places instead of only one. You've doubled the carbon footprint of it, too.”For some workloads, it absolutely has significant merit. I just don't know exactly what that's going to look like—actually, I take that back—the more I think about that, the more I realize that in some level, that's what SDNs do already where, “Great, if this has to be built into something; if I hit an AWS endpoint or an API Gateway or something, I want to have an option when I'm building that out to be able to have that do more or less a follow-the-sun style pattern where it's honed out of wherever energy markets are inexpensive.” And that certainly is going to break things for a lot of workloads, but not all of them, not by far.Catharine: Yeah, and I think that is where my context is coming from. You know, working at Fastly, that was the notion, you know, “We're caching your data close to your end-users, so you don't have to operate resources in that area.” And we have a certain amount of leeway to how we serve that traffic. But it is a more global-distributed model and spinning up servers only when you need them is also a model that takes advantage of not having idle services around just in case you need them, actually responding to demand in real-time.If you look at what the future holds for, you know, smart grids, energy networks, there's this tremendous ability—and I would be very surprised if the big providers are not working on this—to integrate the two—so that electricity availability and how our network traffic is served, is just built into the big providers.Corey: I really hope that one of these big providers leads the way on that. That's the kind of thing that they should really want to see come out of these folks. We are recording this before AWS reinvents. So, if they did come out with something like this, good for them, and also, I have no idea, at the time of this recording, whether they are or not. So, if I got it right, no, I'm not breaking any confidentiality agreements. I feel I need to call that out explicitly because everyone assumes that I—that I have magic insight into everything they're going to come out with. Not really; usually it's all after the fact.Catharine: What I'm really hoping is that by the time this airs, Amazon has already released version two of their carbon footprint tool, where they have per data center visibility where it's no longer three months in arrears, so that you can actually do experimentation and see how differences in the way you implement your cloud impact your carbon footprint. Rather than just, like, sort of, the receipt of, “Yep, here's your carbon footprint.” Like, “No, no, no; I want to make it better. How do I make it better?”So, I'm very much hoping they make an announcement of that kind, and then I'll come back.Corey: You're welcome to come back if and when there's anything that any of these providers release that materially changes the trajectory we're currently on. I want to thank you for being so generous with your time. If people want to learn more, where's the best place for them to find you?Catharine: Yeah. You can find me on my website, Summerstir.com. And also, I hang out an awful lot with some very smart people on ClimateAction.tech. Their Slack is a great repository for people concerned about exactly these issues.Corey: And we will, of course, put links to that in the [show notes 00:37:21]. Thank you so much for being so generous with your time. I appreciate it.Catharine: This has been delightful. Thank you.Corey: Catharine Strauss, budding digital sustainability consultant. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment that also includes the cloud sustainability metrics for that podcast platform of choice.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

About JeremyJeremy is the Director of DevRel & Community at CircleCI, formerly at Solace, Auth0, and XDA. He is active in the DevRel Community, and is a co-creator of DevOpsPartyGames.com. A lover of all things coffee, community, open source, and tech, he is also house-broken, and (generally) plays well with others.Links Referenced: CircleCI: https://circleci.com/ DevOps Party Games: https://devopspartygames.com/ Twitter: Iamjerdog LinkedIn: https://www.linkedin.com/in/jeremymeiss/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored by our friends at Logicworks. Getting to the cloud is challenging enough for many places, especially maintaining security, resiliency, cost control, agility, etc, etc, etc. Things break, configurations drift, technology advances, and organizations, frankly, need to evolve. How can you get to the cloud faster and ensure you have the right team in place to maintain success over time? Day 2 matters. Work with a partner who gets it - Logicworks combines the cloud expertise and platform automation to customize solutions to meet your unique requirements. Get started by chatting with a cloud specialist today at snark.cloud/logicworks. That's snark.cloud/logicworksCorey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I generally try to have people that I know in the ecosystem on this show from time to time, but somehow today's guest has never made it onto the show. And honestly, I have no excuse other than that, I guess I just like being contrary about it. Jeremy Meiss is the Director of DevRel and Community at CircleCI. Jeremy, thank you for finally getting on the show.Jeremy: Hey, you know what? I woke up months and months ago hoping I would be able to join and never have, so I appreciate you finally, you know, getting that celestial kick in the ass.Corey: I love the fact that this is what you lie awake at night worrying about. As all people should. So, let's get into it. You have been at CircleCI in their DevRel org—heading their DevRel org—for approximately 20 years, but in real-time and non-tech company timeframes, three years. But it feels like 20. How's that been? It's been an interesting three years, I'll say that much with the plague o'er the land.Jeremy: Yes, absolutely. No, it was definitely a time to join. I joined two weeks before the world went to shit, or shittier than it already was. And yeah, it's been a ride. Definitely see how everything's changed, but it's also been one that I couldn't be happier where I'm at and seeing the company grow.Corey: I've got to level with you. For the longest time, I kept encountering CircleCI in the same timeframes and context, as I did Travis CI. They both have CI in the name and I sort of got stuck on that. And telling one of the companies apart from the other was super tricky at the time. Now, it's way easier because Travis CI got acquired and then promptly imploded.Security issues that they tried to hide left and right, everyone I knew there long since vanished, and at this point, it is borderline negligence from my point of view to wind up using them in production. So oh, yeah, CircleCI, that's the one that's not trash. I don't know that you necessarily want to put that on a billboard somewhere, but that's my mental shortcut for it.Jeremy: You know, I'm not going to disagree with that. I think, you know, it had its place, I think there's probably only one or two companies nowadays actually propping it up as a business, and I think even they are actively trying to get out of it. So yeah, not going to argue there.Corey: I have been on record previously as talking about CI/CD—Continuous Integration slash Continuous Deployment—or for those who have not gone tumbling down that rabbit hole, the idea that when you push a commit to a particular branch on Git—or those who have not gotten to that point, push the button, suddenly code winds up deploying to different environments, occasionally production, sometimes staging, sometimes development, sometimes by accident—and there are a bunch of options in that space. AWS has a bunch of services under their CodeStar suite: CodeBuild, CodeDeploy, CodePipeline, and that's basically there as a marketing exercise by CI/CD companies that are effective because after having attempted to set those things up with the native offerings, you go scrambling to something else, anything else. GitHub Actions has also been heavily in that space because it's low friction to integrate, it's already there in GitHub, and that's awesome in some ways, terrible in others. But CircleCI has persistently been something that I see in a lot of different environments, both the open-source world, as well as among my clients, where they are using you folks to go from developer laptops to production safely and sanely.Jeremy: Absolutely, yeah. And I think that's one thing for us is, there's a niche of—you know, you can start if you're into AWS or you're into Google, or you're in—any of those big ecosystems, you can certainly use what they have, but those are always, like, add-on things, they're always like an afterthought of, “Oh, we're going to go add this,” or, “We're going to go add that.” And so, I think you adequately described it of, you know, once you start hitting scale, you're eventually going to start to want to use something, and I think that's where we generally fit in that space of, you know, you can start, but now you're going to eventually end up here and use best-in-class. I spent years Auth0 in the identity space, and it was the same kind of boat is that, you know, sure you can start with hopefully not rolling your own, but eventually you're going to end up wanting to use something best-in-class that does everything that you want it to do and does it right.Corey: The thing that just completely blows my mind is how much for all these companies, no matter who they are and how I talk to them, everyone talks about their CI/CD flow with almost a sense of embarrassment. And back in the days when I was running production environments, we use Jenkins as sort of a go-to answer for this. And that was always a giant screaming exemption to the infrastructure-as-code approach because you could configure it via the dashboard and the web interface and it would write that out as XML files. So, you wound up with bespoke thing lots of folks could interact with in different ways, and oh, by the way, it has access into development, staging, and production. Surely, there will be no disasters that happened as a result of this.And that felt terrible. And now we've gotten into a place where most folks are not doing that anymore, at least with the folks that I talk to, but I'm still amazed by how few best practices around a lot of this stuff has really emerged. Every time I see a CI/CD pipeline, it feels like it is a reimplementation locally of solving a global problem. You're the director of DevRel and have been for a few years now. Why haven't you fixed this yet?Jeremy: Primarily because I'm still stuck on the fact you mentioned, pushing a button and getting to XML. That just kind of stuck me there and sent me back that I can't come up with a solution at this point.Corey: Yeah, it's the way that you solve the gap—the schism as it were—between JSON and YAML. “Cool, we're going to use XML.” And everyone's like, “Oh, God, not that.” It's like, “Cool, now you're going to settle your differences or I'm going to implement other things, too.”Jeremy: That's right, yeah. I mean, then we're going to go use some bespoke company's own way of doing IAC. No, I think there's an element here where—I mean, it goes back to still using best-in-class. I think Hudson, which eventually became Jenkins, after you know, Cisco—was it Cisco? No, it was Sun—after Sun, you know, got their hands all over it, it was the thing. It's kind of, well, we're just going to spin this up and do it ourselves.But as the industry changes, we do more and more things on the cloud and we do it primarily because we're relocating the things that we don't want to have to manage ourselves with all of the overhead and all of the other stuff. We're going to go spit it over to the cloud for that. And so, I think there's been this shift in the industry that they still do, like you said, look at their pipelines with a little bit of embarrassment [laugh], I think, yeah. I chuckle when I think about that, but there is a piece where more and more people are recognizing that there is a better way and that you can—you don't have to look at your pipelines as this thing you hate and you can start to look at what better options there are than something you have to host yourself.Corey: What I'm wondering about now, though, because you've been fairly active in the space for a long time, which is a polite way of saying you have opinions—and you should hear the capital O and ‘Opinions' when I say it that way—let's fight about DevRel. What does DevRel mean to you? Or as I refer to it, ‘devrelopers?'Jeremy: Uh, devrelopers. Yes. You know, not to take from the standard DevOps answer, but I think it depends.Corey: That's the standard lawyer answer to anything up to and including, is it legal for me to murder someone? And it's also the senior consultant answer, to anything, too, because it turns out the world is baked and nuanced and doesn't lend itself to being resolved in 280 characters or less. That's what threads are for.Jeremy: Right [laugh]. Trademark. That is ultimately the answer, I think, with DevRel. For me, it is depending on what your company is trying to do. You ultimately want to start with building relationships with your developers because they're the ones using your product, and if you can get them excited about what they're doing with your product—or get excited about your product with what they're doing—then you have something to stand on.But you also have to have a product fit. You have to actually know what the hell your product is doing and is it going to integrate with whatever your developers want. And so, DevRel kind of stands in that gap that says, “Okay, here's what the community wants,” and advocates for the community, and then you have—it's going to advocate for the company back to the community. And hopefully, at the end of the day, they all shake hands. But also I've been around enough to recognize that there comes that point where you either a have to say, “Hey, our product for that thing is probably not the best thing for what you're trying to do. Here, you should maybe start at this other point.”And also understanding to take that even, to the next step to finish up the answer, like, my biggest piece now is all the fights that we have constantly around DevRel in the space of what is it and what is it not, DevRel is marketing. DevRel is sales. DevRel is product. And each of those, if you're not doing those things as a member of the company, you're not doing your job. Everybody in the company is the product. Everybody in the company is sales. Everybody in the company is marketing.Corey: Not everyone in the company realizes this, but I agree—Jeremy: Yes.Corey: Wholeheartedly.Jeremy: Yes. And so, that's where it's like yes, DevRel is marketing. Yes, it is sales. Because if you're not out there, spreading whatever the news is about your product and you're not actually, you know, showing people how to use it and making things easier for people, you're not going to have a job. And too often, these companies that—or too often I think a lot of DevRel teams find themselves in places where they're the first that get dropped when the company goes through things because sometimes it is just the fact that the company has not figured out what they really want, but also, sometimes it's the team hasn't really figured out how to position themselves inside the business.Corey: One of the biggest, I'll call it challenges that I see in the DevRel space comes down to defining what it is, first and foremost. I think that it is collectively a mistake for an awful lot of practitioners of developer relations, to wind up saying first and foremost that we're not marketing. Well, what is it that you believe that marketing is? In fact, I'll take it a step beyond that. I think that marketing is inherently the only place in most companies where we know that doing these things leads to good results, but it's very difficult to attribute or define that value, so how do we make sure that we're not first up on the chopping block?That has been marketing's entire existence. It's, you know that doing a whole bunch of things in marketing will go well for you, but as the old chestnut says, half your marketing budget is wasted and you'll go broke figuring out which half it is.Jeremy: Yeah. And whenever you have to make cuts, generally, they always, you know, always come to the marketing teams because hey, they're the ones spending, you know, millions of dollars a quarter on ads, or whatever it is. And so yeah, marketing has, in many ways figured this out. They're also the team that spends the most money in a company. So, I don't really know where to go with that isn't completely off the rails, but it is the reality. Like, that's where things happen, and they are the most in touch with what the direction of the company is going to ultimately be received as, and how it's going to be spoken about. And DevRel has great opportunities there.Corey: I find that when people are particularly militant about not liking sales or marketing or any other business function out there, one of the ways to get through them is to ask, “Great. In your own words, describe to me what you believe that department does. What is that?” And people will talk about marketing in a bunch of tropes—or sales in a bunch of tropes—where it is the worst examples of that.It's, “Terrific, great. Do you want me to wind up describing what you do as an engineer—in many cases—in the most toxic stereotype of Uber and 2015-style engineer I can come up with?” I think, in most cases if we're having a conversation and I haven't ended it by now, you would be horrified by that descriptor. Yeah. Not every salesperson is the skeezy used car salesman trying to trick you into something awful. Actual selling comes down to how do we wind up taking your pain away. One of my lines is, “I'm a consultant. You have problems and money. I will take both.”Jeremy: That's right [laugh]. Yeah, that's right.Corey: If you don't have a painful problem, I have nothing to sell you and all I'm doing is wasting my breath trying.Jeremy: Yeah, exactly. And that's where—I'll say it two ways—the difference between good marketing teams are, is understanding that pain point of the people that they're trying to sell to. And it's also a difference between, like, good and bad, even, DevRel teams is understanding what are the challenges that your users are having you're trying to express to, you're trying to fix? Figure that out because if you can't figure that out, then you or your marketing team are probably soon to be on the block and they're going to bring someone else in.Corey: I'm going to fight you a little bit, I suspect, in that a line I've heard is that, “Oh, DevRel is part of product because we are the voice of the community back into the development cycle of what product is building.” And the reason that I question that is I think that it glosses over an awful lot of what makes product competent as a department and not just a function done by other people. It's, “Oh, you're part of the product. Well, great. How much formal training have you had as part of your job on conducting user research and interviews with users and the rest?”And the answer invariably rounds to zero and, okay, in other words, you're just giving feedback in a drive-by fashion that not structured in any way and your product people are polite enough not to call you out on it. And that's when the fighting and slapping begins.Jeremy: Yeah. I don't think we're going to disagree too much there. I think one of the challenges, though, is for the very reason you just mentioned, that the product teams tend to hear your product sucks. And we've heard all the people telling us that, like, people in the community say that, they hear that so much and they've been so conditioned to it that it just rolls off their back, like, “Okay, whatever.” So, for DevRel teams, even if you're in product, which we can come back to that, regardless of where you're at, like, bringing any type of feedback you bring should have a person, a name associated with it with, like, Corey at Duckbill Group hates this product.Corey: Uh-oh [laugh]. Whenever my name is tied to feedback, it never goes well for me, but that will teach me eventually, ideally, to keep my mouth shut.Jeremy: Yeah. Well, how's that working for you?Corey: I'll let you know if it ever happens.Jeremy: Good. But once you start making the feedback like an actual person, it changes the conversation. Because now it's like, oh, it's not this nebulous, like, thing I can not listen to. It's now oh, it's actually a person at a specific company. So, that's one of the challenges in working with product that you have to overcome.When I think about DevRel in product, while I don't think that's a great spot for it, I think DevRel is an extension of product. That's part of where that, like, the big developer experience craze comes from, and why it is a valuable place for DevRel to be able to have input into is because you tend to be the closest to the people actually using the product. So, you have a lot of opportunities and a big surface area to have some impact.Corey: This episode is sponsored in part by our friends at Strata. Are you struggling to keep up with the demands of managing and securing identity in your distributed enterprise IT environment? You're not alone, but you shouldn't let that hold you back. With Strata's Identity Orchestration Platform, you can secure all your apps on any cloud with any IDP, so your IT teams will never have to refactor for identity again. Imagine modernizing app identity in minutes instead of months, deploying passwordless on any tricky old app, and achieving business resilience with always-on identity, all from one lightweight and flexible platform.Want to see it in action? Share your identity challenge with them on a discovery call and they'll hook you up with a complimentary pair of AirPods Pro. Don't miss out, visit Strata.io/ScreamingCloud. That's Strata dot io slash ScreamingCloud.Corey: I think that that is a deceptively nuanced statement. One of the things I learned from an earlier episode I had with Dr. Christina Maslach, is contributors to occupational burnout, so much of it really distills down—using [unintelligible 00:16:35] crappy layman's terms—to a lack of, I guess what I'm going to call relevance or a lack—a feeling like you are not significant to what the company is actually doing in any meaningful way. And I will confess to having had a number of those challenges in my career when I was working in production environments because, yeah, I kept the servers up and the applications up, but if you really think about it, one of the benefits of working in the system space—or the production engineers base, or DevOps, or platform engineering, or don't even start with me these days—is that what you do conveys almost seamlessly from company to company. Like, the same reason that I can do what I do now, I don't care what your company does, necessarily, I just know that the AWS bill is a bounded problem space and I can reason about it almost regardless of what you do.And if I'm keeping the site up, okay, it doesn't matter if we're streaming movies or selling widgets or doing anything, just so long as I don't find that it contradicts my own values. And that's great, but it also is isolating because you feel like you're not really relevant to the direction of what the company actually does. It's, “Okay, so what does this company do?” “We make rubber bands,” and well, I'm not really a rubber band connoisseur, but I could make sure that the website stays up. But it just feels like there's a disconnect element happening.Jeremy: That is real. It is very real. And one of the ways that I tried to kind of combat that, and I help my team kind of really try and keep this in mind, is we try to meet as much as possible with the people that are actually doing the direction, whether it be product marketing, or whether it's in product managers, or it's even, you know, in engineering is have some regular conversations with what we do as a company. How are we going to fit with that in what we do and what we say and all of our objectives, and making sure that everything we do ties to something that helps other teams and that fits within the business and where it's going so that we grow our understanding of what the company is trying to do so that we don't kind of feel like a ship that's without a sail and just floating wherever things go.Corey: On some level, I am curious as to what you're seeing as we navigate this—I don't know if it's a recession,' I don't know if it's a correction; I'm not sure what to call it—but my gut tells me that a lot of things that were aimed at, let's call it developer quality of life, they were something of a necessity in the unprecedented bull market that we've seen for the last decade at some point because most companies cannot afford to compete with the giant tech company compensation packages, so you have to instead talk about quality of life and what work-life balance looks like, and here's why all of the tools and processes here won't drive you to madness. And now it feels like, “Oh, we don't actually have to invest in a lot of those things, just because oh yeah, like, the benefits here are you're still going to be employed next week. So, how about that?” And I don't think that's a particularly healthy way to interact with people—it's certainly not how I do—but it does seem that worrying about keeping developers absolutely thrilled with every aspect of their jobs has taken something of a backseat during the downturn.Jeremy: I don't know. I feel like developer satisfaction is still an important piece, even though, you know, we have a changing market. And as you described, if you're not happy with the tool you're using, you're not going to be as productive than using the tool or using—you know, whether it's an actual developer productivity tool, or it's even just the fact that you might need two monitors, you're not going to be as productive if you're not enjoying what you're doing. So, there is a piece of it, I think, the companies are recognizing that there are some tools that do ultimately benefit and there's some things that they can say, we're not going to invest in that area right now. We're good with where we're at.Corey: On some level, being able to say, “No, we're not going to invest in that right now,” is the right decision. It is challenging, in some cases, to wind up talking to some team members in some orgs, who do not have the context that is required to understand why that decision is being made. Because without context, it looks like, “Mmm, no. I'm just canceling Christmas for you personally this year. Sorry, doesn't it suck to be you? [singing] Dut, dut.” And that is very rarely how executives make decisions, except apparently if they're Elon Musk.Jeremy: Right. Well, the [Muskrat 00:21:23] can, you know, sink any company—Corey: [laugh].Jeremy: — and get away with it. And that's one thing I've really been happy with where I'm at now, is you have a leadership team that says, “Hey, here's where things are, and here's what it looks like. And here's how we're all contributing to where we're going, and here's the decisions we're going to make, and here's how—” they're very open with what's going on. And it's not a surprise to anybody that the economic time means that we maybe can't go to 65 events next year. Like, that's just reality.But at the end of the day, we still have to go and do a job and help grow the company. So, how can we do that more efficiently? Which means that we—it leaves it better to try and figure that out than to be so nebulous, with like, “Yep, nope. You can't go do that.” That's where true leadership comes to is, like, laying it out there, and just, you know, getting people alongside with you.Corey: How do you see DevRel evolving? Because I think we had a giant evolution over the past few years. Because suddenly, the old vision of DevRel—at least in some quarters, which I admit I fell a little too deeply into—was, I'm going to go to all the conferences and give all of the talks, even though most of them are not related to the core of what I do. And maybe that's a viable strategy; maybe it's not. I think it depends on what your business does.And I don't disagree with the assertion that going and doing something in public can have excellent downstream effects, even if the connection is not obvious. But suddenly, we weren't able to do that, and people were forced to almost reinvent how a lot of that works. Now, that the world is, for better or worse, starting to open up again, how do you see it evolving? Are we going right back to a different DevOps days in a different city every week?Jeremy: I think it's a lot more strategic now. I think generally, there is less mountains of money that you can pull from to go and do whatever the hell you want. You have to be more strategic. I said that a few times. Like, there's looking at it and making sure, like, yeah, it would be great to go and, you know, get in front of 50,000 people this quarter or this year, whatever you want to do, but is that really going to move the bottom line for us? Is that really going to help the business, or is that just helping your Delta miles?What is really the best bang for the buck? So, I think DevRel as it evolves, in the next few years, has to come to a good recognition moment of we need to be a little bit more prescriptive in how we do things within our company and not so willy-nilly return to you know, what we generally used to get away with. That means you're going to see a lot more people have to be held to account within their companies of, is what you're doing actually match up to our business goal here? How does that fit? And having to explain more of that, and that's, I think, for some people will be easy. Some people are going to have to stretch that muscle, and others are going to be in a real tough pickle.Corey: One last topic I want to get into with you is devopspartygames.com, an online more or less DevOps, quote-unquote, “Personality” assortment of folks who wind up playing online games. I was invited once and promptly never invited back ever again. So first, was it something I said—obviously—and two what is that and how—is that still going in this post-pandemic-ish era?Jeremy: I like how you answered your own question first; that way I don't have to answer it. The second one, the way it came about was just, you know, Matty and I had started missing that interaction that we would tend to have in person. And so, one of the ways we started realizing is we play these, you know, Jackbox games, and why can't we just do this with DevOps tech prompts? So, that's kind of how it kicked off. We started playing around doing it for fun and then I was like, “You know, we should—we could do this as a big, big deal for foreseeable future.”Where's that now is, we actually have not done one online for—what is it? So probably, like, eight, nine months, primarily because it's harder and harder to do so as everybody [laugh]—we're now doing a little bit more travel, and it's hard to do those—as you know, doing podcasts, it takes a lot of work. It's not an easy kind of thing. And so, we've kind of put that on pause. But we actually did our first in-person DevOps Party Games at DevOpsDays Chicago recently, and that was a big hit, I think, and opportunity to kind of take what we're doing virtually, and the fun and excitement that we generally would have—relatively half-drunk—to actually doing it actually in-person at an event. And in the different—like, just as giving talks in person was a different level of interaction with the crowd, the same thing is doing it in person. So, it was just kind of a fun thing and an opportunity maybe to continue to do it in person.Corey: I think we all got a hell of a lot better very quickly at speaking to cameras instead of audiences and the rest. It also forced us to be more focused because the camera gives you nothing in a way that the audience absolutely does.Jeremy: They say make love to the camera, but it doesn't work anyways.Corey: I really want to thank you for spending as much time as you have talking to me. If people want to learn more about who you are and what you're up to, where should they go?Jeremy: Well, for the foreseeable future, or at least what we can guess, you can find me on the Twitters at @Iamjerdog. You can find me there or you can find me at, you know, LinkedIn, at jeremymeiss, LinkedIn. And you know, probably come into your local DevOpsDays or other conference as well.Corey: Of course. And we will, of course, put links to that in the show notes.Jeremy: Excellent.Corey: Thank you so much for being so generous with your time. It is always appreciated. And I do love talking with you.Jeremy: And I appreciate it, Corey. It was great beyond, finally. I won't hold it against you anymore.Corey: Jeremy Meiss, Director of DevRel at CircleCI. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry, irritated comment talking about how CI/CD is nonsense and the correct way to deploy to production is via the tried-and-true method of copying and pasting.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

About EmilyEmily Ruppe is a Solutions Engineer at Jeli.io whose greatest accomplishment was once being referred to as “the Bob Ross of incident reviews.” Previously Emily has written hundreds of status posts, incident timelines and analyses at SendGrid, and was a founding member of the Incident Command team at Twilio. She's written on human centered incident management and facilitating incident reviews. Emily believes the most important thing in both life and incidents is having enough snacks.Links Referenced: Jeli.io: https://jeli.io Twitter: https://twitter.com/themortalemily Howie Guide: https://www.jeli.io/howie/welcome TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored by our friends at Logicworks. Getting to the cloud is challenging enough for many places, especially maintaining security, resiliency, cost control, agility, etc, etc, etc. Things break, configurations drift, technology advances, and organizations, frankly, need to evolve. How can you get to the cloud faster and ensure you have the right team in place to maintain success over time? Day 2 matters. Work with a partner who gets it - Logicworks combines the cloud expertise and platform automation to customize solutions to meet your unique requirements. Get started by chatting with a cloud specialist today at snark.cloud/logicworks. That's snark.cloud/logicworksCorey: Cloud native just means you've got more components or microservices than anyone (even a mythical 10x engineer) can keep track of. With OpsLevel, you can build a catalog in minutes and forget needing that mythical 10x engineer. Now, you'll have a 10x service catalog to accompany your 10x service count. Visit OpsLevel.com to learn how easy it is to build and manage your service catalog. Connect to your git provider and you're off to the races with service import, repo ownership, tech docs, and more. Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. My guest today is Emily Ruppe, who's a solutions engineer over at Jeli.io, but her entire career has generally focused around incident management. So, I sort of view her as being my eternal nemesis, just because I like to cause problems by and large and then I make incidents for other people to wind up solving. Emily, thank you for joining me and agreeing to suffer my slings and arrows here.Emily: Yeah. Hey, I like causing problems too. I am a solutions engineer, but sometimes we like to call ourselves problems engineers. So.Corey: Yeah, I'm a problems architect is generally how I tend to view it. But doing the work, ah, one wonders. So, you are a Jeli, where as of this recording, you've been for a year now. And before that, you spent some time over at Twilio slash SendGrid—spoiler, it's kind of the same company, given the way acquisitions tend to work and all. And—Emily: Now, it is.Corey: Yeah. Oh, yeah. You were there during the acquisition.Emily: Mm-hm. Yes, they acquired me and that's why they bought SendGrid.Corey: Indeed. It's a good reason to acquire a company. That one person I want to bring in. Absolutely. So, you started with email and then effectively continued in that general direction, given the Twilio now has eaten that business whole. And that's where I started my career.The one thing I've learned about email systems is that they love to cause problems because it's either completely invisible and no one knows, or suddenly an email didn't go through and everyone's screaming at you. And there's no upside, only down. So, let me ask the obvious question I suspect I know the answer to here. What made you decide to get into incident management?Emily: [laugh]. Well, I joined SendGrid actually, I've, I love mess. I run towards problems. I'm someone who really enjoys that. My ADHD, I hyperfocus, incidents are like that perfect environment of just, like, all of the problems are laying themselves out right in front of you, the distraction is the focus. It's kind of a wonderful place where I really enjoy the flow of that.But I've started in customer support. I've been in technical support and customer—I used to work at the Apple Store, I worked at the Genius Bar for a long time, moved into technical support over the phone, and whenever things broke really bad, I really enjoyed that process and kind of getting involved in incidents. And I came, I was one of two weekend support people at SendGrid, came in during a time of change and growth. And everyone knows that growth, usually exponential growth, usually happens very smoothly and nothing breaks during that time. So… no, there was a lot of incidents.And because I was on the weekend, one of the only people on the weekend, I kind of had to very quickly find my way and learn when do I escalate this. How do I make the determination that this is something that is an incident? And you know, is this worth paging engineers that are on their weekend? And getting involved in incidents and being kind of a core communication between our customers and engineers.Corey: For those who might not have been involved in sufficiently scaled-out environments, that sounds counterintuitive, but one of the things that you learn—very often the hard way—has been that as you continue down the path of building a site out and scaling it, it stops being an issue relatively quickly of, “Is the site up or down?” And instead becomes a question of, “How up is it?” So, it's it doesn't sound obvious until you've lived it, but declaring what is an incident versus what isn't an incident is incredibly nuanced and it's not the sort of thing that lends itself to casual solutions. Because every time a customer gets an error, we should open an incident on that. Well, I've worked at companies that throw dozens of 500 errors every second at their scale. You will never hire enough people to solve that if you do an incident process on even 10% of them.Emily: Yeah. So, I mean, it actually became something that when you join Twilio, they have you create a project using Twilio's API to earn your track jacket, essentially. It's kind of like an onboarding thing. And as they absorbed SendGrid, we all did that onboarding process. And mine was a number for support people to text and it would ask them six questions and if they answered yes to more than two of them, it would text back, “Okay, maybe you should escalate this.”And the questions were pretty simple of, “Can emails be sent?” [laugh]. Can customers log into their website? Are you able to view this particular part of the website? Because it is—with email in particular, at SendGrid in particular—the bulk of it is the email API. So, like, the site being up or down was the easiest type of incident, the easiest thing to flex on because that's so much easier to see.Being able to determine, like, what percentage or what level, like, how many emails are not processing? Are they getting stuck or is this, like, the correct amount of things that should be bouncing because of IP reput—there's, like, a thousand different things. We had kind of this visualization of this mail pipeline that was just a mess of all of these different pipes kind of connected together. And mail could get stuck in a lot of different places, so it was a lot of spending time trying to find that and segwayed into project management. I was a QA for a little while doing QA work.Became a project manager and learned a lot about imposing process because you're supposed to and that sometimes imposing process on teams that are working well can actually destroy them [laugh]. So, I learned a lot of interesting things about process the hard way. And during all of that time that I was doing project management, I kind of accidentally started owning the incident response process because a lot of people left, I had been a part of the incident analysis group as well, and so I kind of became the sole owner of that. And when Twilio purchase SendGrid, I found out they were creating an incident commander team and I just reached out and said, “Here's all of SendGrids incident response stuff. We just created a new Slackbot, I just retrained the entire team on how to talk to each other and recognize when something might be an incident. Please don't rewrite all of this to be Twillio's response process.”And Terry, the person who was putting together that team said, “Excellent. You're going to be [laugh] welcome to Twilio Incident Command. This is your problem and it's a lot worse than you thought because here's all the rest of it.” So yeah, it was really interesting experience coming into technically the same company, but an entirely different company and finding out—like, really trying to learn and understand all of the differences, and you know, the different problems, the different organizational history, the, like, fascia that has been built up between some of these parts of the organization to understand why things are the way that they are within process. It's very interesting.And I kind of get to do it now as my job. I get to learn about the full organizational subtext of [laugh] all of these different companies to understand how incident response works, how incident analysis works, and maybe some of the whys. Like, what are the places where there was a very bad incident, so we put in very specific, very strange process pieces in order to navigate that, or teams that are difficult to work with, so we've built up interesting process around them. So yeah.Corey: It feels like that can almost become ossified if you're not careful because you wind up with a release process that's two thousand steps long, and each one of them is there to wind up avoiding a specific type of failure that had happened previously. And this gets into a world where, in so many cases, there needs to be a level of dynamism to how you wind up going about your work. It feels almost like companies have this idealized vision of the future where if they can distill every task that happens within the company down to a series of inputs and responses—scripts almost—you can either wind up replacing your staff with a bunch of folks who just work from a runbook and cost way less money or computers in the ultimate sense of things. But that's been teased for generations now and I have a very hard time seeing a path where you're ever going to be able to replace the contextually informed level of human judgment that, honestly, has fixed every incident I've ever seen.Emily: Yeah. The problem comes down to in my opinion, the fact that humans wrote this code, people with specific context and specific understanding of how the thing needs to work in a specific way and the shortcomings and limitations they have for the libraries they're using or the different things are trying to integrate in, a human being is who's writing the code. Code is not being written by computers, it's being written by people who have understanding and subtext. And so, when you have that code written and then maybe that person leaves or that person joins a different team and they focus and priorities on something else, there is still human subtests that exists within the services that have been written. We have it call in this specific way and timeout in this specific amount of time because when we were writing it, there was this ancient service that we had to integrate with.Like, there's always just these little pieces of we had to do things because we were people trying to make connections with lines of code. We're trying to connect a bunch of things to do some sort of task, and we have a human understanding of how to get from A to B, and probably if A computer wrote this code, it would work in an entirely different way, so in order to debug a problem, the humans usually need some sort of context, like, why did we do this the way that we did this? And I think it's a really interesting thing that we're finding that it is very hard to replace humans around computers, even though intellectually we think, like, this is all computers. But it's not. It's people convincing computers to do things that maybe they shouldn't necessarily be doing. Sometimes they're things that computers shouldn't be doing, maybe, but a lot of the times, it's kind of a miracle [laugh] that any of these things continue to work on it on a given basis. And I think that it's very interesting when we, I think, we think that we can take people out of it.Corey: The problem I keep running into though, the more I think about this and the more I see it out there is I don't think that it necessarily did incident management any favors when it was originally cast as the idea of blamelessness and blameless postmortems. Just because it seems an awful lot to me like the people who are the most advocate champions of approaching things from a blameless perspective and having a blameless culture are the people who would otherwise have been blamed themselves. So, it really kind of feels on some broader level, like, “Oh, was this entire movement really just about being self-serving so that people don't themselves get in trouble?” Because if you're not going to blame no one, you're going to blame me instead. I think that, on some level, set up a framing that was not usually helpful for folks with only a limited understanding of what the incident lifecycle looks like.Emily: Mmm. Yeah, I think we've evolved, right? I think, from the blameless, I think there was good intentions there, but I think that we actually missed the really big part of that boat that a lot of folks glossed over because then, as it is now, it's a little bit harder to sell. When we're talking about being blameless, we have to talk about circumventing blame in order to get people to talk candidly about their experiences. And really, it's less about blaming someone and what they've done because we as humans blame—there's a great Brené Brown talk that she gives, I think it's a TED talk about blame and how we as humans cannot physically avoid blaming, placing blame on things.It's about understanding where that's coming from, and working through it that is actually how we grow. And I think that we're starting to kind of shift into this more blame-aware culture. But I think the hard pill to swallow about blamelessness is that we actually need to talk about the way that this stuff makes us feel as people. Like feelings, like emotions [laugh]. Talk about emotions during a technical incident review is not really an easy thing to get some tech executives to swallow.Or even engineers. There's a lot of engineers who are just kind of like, “Why do you care about how I felt about this problem?” But in reality, you can't measure emotions as easily as you can measure Mean Time to Resolution. But Mean Time to Resolution is impacted really heavily by, like, were we freaking out? Did we feel like we had absolutely no idea what we were trying to solve, or did we understand this problem, and we were confident that we could solve it; we just couldn't find the specific place where this bug was happening. All of that is really interesting and important context about how we work together and how our processes work for us, but it's hard because we have to talk about our feelings.Corey: I think that you're onto something here because I look back at the key outages that really define my perspective on things over the course of my career, and most of the early ones were beset by a sense of panic of am I going to get fired for this? Because at the time, I was firmly convinced that well, root cause is me. I am the person that did the thing that blew up production. And while I am certainly not blameless in some of those things, I was never setting out with an intent to wind up tiering things down. So, it was not that I was a bad actor subverting internal controls because, in many companies, you don't need that level of rigor.This was a combination of factors that made it easy or possible to wind up tiering things down when I did not mean to. So, there were absolutely systemic issues there. But I still remember that rising tide of panic. Like, should I be focused on getting the site backup or updating my resume? Which of these is going to be the better longer-term outcome? And now that I've been in this industry long enough and I've seen enough of these, it's, you almost don't feel the blood pressure rise anymore when you wind up having something gets panicky. But it takes time and nuance to get there.Emily: Yeah. Well, and it's also, in order to best understand how you got in that situation, like, were you willing to tell people that you were absolutely panicked? Would you have felt comfortable, like, if someone was saying like, “Okay, so what happened? How did—walk me through what you were experiencing?” Would you have said like, “I was scared out of my goddamn mind?”Were you absolutely panicking or did you feel like you had some, like, grasping at some straws? Like, where were you? Because uncovering that for the person who is experiencing that in the issue, in the incident can help understand, what resources did they feel like they knew where to go to. Or where did they go to? Like, what resource did they decide in the middle of this panicked haze to grasp for? Is that something that we should start using as, “Hey, if it's your first time on call, this is a great thing to pull into,” because that's where instinctively you went?Like, there's so much that we can learn from the people who are experiencing [laugh] this massive amount of panic during the incident. But sometimes we will, if we're being quote-unquote, “Blameless,” gloss over your entire, like, your involvement in that entirely. Because we don't want to blame Corey for this thing happening. Instead, we'll say, “An engineer made a decision and that's fine. We'll move past that.” But there's so much wealth of information there.Corey: Well, I wound up in postmortems later when I ran teams, I said, “Okay, so an engineer made a mistake.” It's like, “Well, hang on. There's always more to it than that”—Emily: Uh-huh.Corey: —“Because we don't hire malicious people and the people we have are competent for their role.” So, that goes a bit beyond that. We will never get into a scenario people do not make mistakes in a variety of different ways. So, that's not a helpful framing, it's a question of what—if they made a mistake, sure, what was it that brought them to that place because that's where it gets really interesting. The problem is when you're trying to figure out in a business context why a customer is super upset—if they're a major partner, for example—and there's a sense of, “All right, we're looking for a sacrificial lamb or someone that we can blame for this because we tend to think in relatively straight lines.”And in those scenarios, often, a nuanced understanding of the systemic failure modes within your organization that might wind up being useful in the mid to long-term are not helpful for the crisis there. So, trying to stuff too much into a given incident response might be a symptom there. I'm thinking of one or two incidents in the course of my later career that really had that stink to them, for lack of a better term. What's your take on the idea?Emily: I've been in a lot of incidents where it's the desire to be able to point and say a person made this mistake is high, it's definitely something that the, “organization”—and I put the organization in quotes there—and say technical leadership, or maybe PR or the comms team said like, “We're going to say, like, a person made this mistake,” when in reality, I mean, nine times out of ten, calling it a mistake is hindsight, right? Usually people—sometimes we know that we make a mistake and it's the recovery from that, that is response. But a lot of times we are making an informed decision, you know? An engineer has the information that they have available to them at the time and they're making an informed decision, and oh, no [laugh], it does not go as we planned, things in the system that we didn't fully understand are coexisting, it's a perfect storm of these events in order to lead to impact to this important customer.For me, I've been customer-facing for a very long time and I feel like from my observation, customers tend to—like if you say, like, “This person did something wrong,” versus, “We learned more about how the system works together and we understand how these kind of different pieces and mechanisms within our system are not necessarily single points of failure, but points at which they interact that we didn't understand could cause impact before, and now we have a better understanding of how our system works and we're making some changes to some pieces,” I feel like personally, as someone who has had to say that kind of stuff to customers a thousand times, saying, “It was a person who did this thing,” it shows so much less understanding of the event and understanding of the system than actually talking through the different components and different kind of contributing factors that were wrong. So, I feel like there's a lot of growth that we as an industry can could go from blaming things on an intern to actually saying, “No, we invested time and understanding how a single person could perform these actions that would lead to this impact, and now we have a deeper understanding of our system,” is in my opinion, builds a little bit more confidence from the customer side.Corey: This episode is sponsored in part by Honeycomb. I'm not going to dance around the problem. Your. Engineers. Are. Burned. Out. They're tired from pagers waking them up at 2 am for something that could have waited until after their morning coffee. They're fed up with relying on two or three different “monitoring tools” that still require them to manually trudge through logs to decipher what might be wrong. Simply put, there's a better way. Observability tools like Honeycomb show you the patterns and outliers of how users experience your code in complex and unpredictable environments so you can spend less time firefighting and more time innovating. It's great for your business, great for your engineers, and, most importantly, great for your customers. Try FREE today at honeycomb.io/screaminginthecloud. That's honeycomb.io/screaminginthecloud.Corey: I think so much of this is—I mean, it gets back to your question to me that I sort of dodged was I willing to talk about how my emotional state in these moments? And yeah, I was visibly sweating and very nervous and I've always been relatively okay with calling out the fact that I'm not in a great place at the moment, and I'm panicking. And it wasn't helped in some cases by, in those early days, the CEO of the company standing over my shoulder, coming down from the upstairs building to know what was going on, and everything had broken. And in that case, I was only coming in to do mop-up I wasn't one of the factors contributing to this, at least not by a primary or secondary degree, and it still was incredibly stress-inducing. So, from that perspective, it feels odd.But you also talk about ‘we,' in the sense of as an industry, as a culture, and the rest. I'm going to push back on that a little bit because there are still companies today in the closing days of 2022 that are extraordinarily far behind where many of us are at the companies we work for. And they're still stuck in the relative Dark Ages technically, were, “Well, are VMs okay, or should we stay on bare metal?” Is still the era that they're in, let alone cloud, let alone containerization, let alone infrastructure as code, et cetera, et cetera. I'm unconvinced that they have meaningfully progressed on the interpersonal aspects of incident management when they've been effectively frozen in amber from a technical basis.Emily: Mmm, I don't think that's fair [laugh].Corey: No. Excellent. Let's talk about that.Emily: [laugh]. I think just because an organization is still, like, maybe in DCs and using hardware and maybe hasn't advanced so thoroughly within the technical aspect of things, that doesn't necessarily mean that they haven't adopted new—Corey: Ah, very fair. Let me add one point of clarification, then, on this because what I'm talking about here is the fact there are companies who are that far behind on a technical basis, they are not necessarily one and the same, too—Emily: Correct.Corey: Because you're using older technology, that means your processes are stuck in the past, too.Emily: Right.Corey: But rather, just as there are companies that are anxious on the technology basis, there are also companies who will be 20 years behind in learnings—Emily: Yes.Corey: —compared to how the more progressive folks have already internalized some of these things ages ago. Blamelessness is still in the future for them. They haven't gotten there yet.Emily: I mean, yeah, there's still places that are doing root cause analysis, that are doing the five whys. And I think that we're doing our best [laugh]. I mean, I think it really takes—that's a cultural change. A lot of the actual change in approach of incident analysis and incident response is a cultural change. And I can speak from firsthand experience that that's really hard to do, especially from the inside it's very hard to do.So luckily, with the role that I'm in now at Jeli.io, I get to kind of support those folks who are trying to champion a change like that internally. And right now, my perspective is just trying to generate as much material for those folks to send internally, to say like, “Hey, there's a better way. Hey, there's a different approach for this that can maybe get us around these things that are difficult.” I do think that there's this tendency—and I've used this analogy before—is for us to think that our junk drawers are better than somebody else's junk drawers.I see an organization as just a junk drawer, a drawer full of weird odds and ends and spilled glue and, like, a broken box of tacks. And when you pull out somebody else's junk drawer, you're like, “This is a mess. This is an absolute mess. How can anyone live like this?” But when you pull out your own junk drawer, like, I know there are 17 rubber bands in this drawer, somehow. I am going to just completely rifle through this drawer until I find those things that I know are in here.Just a difference of knowing where our mess is, knowing where the bodies are buried, or the skeletons are in each closet, whatever analogy works best. But I think that some organizations have this thought process that—by organizations, I mean, executive leadership organizations are not an entity with an opinion, they're made up of a bunch of individuals doing [laugh] the work that they need to do—but they think that their problems are harder or more unique than at other organizations. And so, it's a lot harder to kind of help them see that, yes, there is a very unique situation, the way that your people work together with their technology is unique to every single different organization, but it's not that those problems cannot be solved in new and different ways. Just because we've always done something in this way does not mean that is the way that is serving us the best in this moment. So, we can experiment and we can make some changes.Especially with process, especially with the human aspect of things of how we talk to each other during incidents and how we communicate externally during incidents. Those aren't hard-coded. We don't have to do a bunch of code reviews and make sure it's working with existing integrations to be able to make those changes. We can experiment with that kind of stuff and I really would like to try to encourage folks to do that even though it seems scary because incidents are… [unintelligible 00:24:33] people think they're scary. They're not. They're [unintelligible 00:24:35].Corey: They seem to be. For a lot of folks, they are. Let's not be too dismissive on that.Emily: But we were both talking about panic [laugh] and the panic that we have felt during incidents. And I don't want to dismiss that and say that it's not real. But I also think that we feel that way because we're worried about how we're going to be judged for our involvement in them. We're panicking because, “Oh no, we have contributed to this in some way, and the fact that I don't know what to do, or the fact that I did something is going to reflect poorly on me, or maybe I'm going to get fired.” And I think that the panic associated with incidents also very often has to do with the environment in which you are experiencing that incident and how that is going to be accepted and discussed. Are you going to be blamed regardless of how, quote-unquote, “Blameless,” your organization is?Corey: I wish there was a better awareness of a lot of these things, but I don't think that we are at a point yet where we're there.Emily: No.Corey: How does this map what you do, day-to-day over at Jeli.io?Emily: It is what I do every single day. So, I mean, I do a ton of different things. We're a very small startup, so I'm doing a lot, but the main thing that I'm doing is working with our customers to tackle these hurdles within each of their organizations. Our customers vary from very small organizations to very, very large organizations, and working with them to find how to make movement, how to sell this internally, sell this idea of let's talk about our incidents a little bit differently, let's maybe dial back some of the hard-coded automation that we're doing around response and change that to speaking to each other, as opposed to, we need 11 emails sent automatically upon the creation of an incident that will automatically map to these three PagerDuty schedules, and a lot more of it can be us working through the issue together and then talking about it afterwards, not just in reference to the root cause, but in how we interfaced: how did it go, how did response work, as well as how did we solve the problem of the technical problem that occurred?So, I kind of pinch myself. I feel very lucky that I get to work with a lot of different companies to understand these human aspects and the technical aspects of how to do these experiments and make some change within organizations to help make incidents easier. That's the whole feeling, right? We were talking about the panic. It doesn't need to be as hard as it feels, sometimes. And I think that it can be easier than we let ourselves think.Corey: That's a good way of framing it. It just feels on so many levels like this is one of the hardest areas to build a company in because you're not really talking about fixing technical, broken systems out there. You're talking about solving people problems. And I have some software that solves your people problems, I'm not sure if that's ever been true.Emily: Yeah, it's not the software that's going to solve the people problems. It's building the skills. A lot of what we do is we have software that helps you immensely in the analysis process and build out a story as opposed to just building out a timeline, trying to tell, kind of, the narrative of the incident because that's what works. Like anthropologically, we've been conveying information through folklore, through tales, telling tales of things that happened in order to help teach people lessons is kind of how we've—oral history has worked for [laugh] thousands of years. And we aren't better than that just because we have technology, so it's really about helping people uncover those things by using the technology we have: pulling in Slack transcripts, and PagerDuty alerts, and Zoom transcripts, and all of this different information that we have available to us, and help people tell that story and convey that story to the folks that were involved in it, as well as other peoples in your organization who might have similar things come up in the future.And that's how we learn. That's how we teach. But that's what we learn. I feel like there's a big difference—I'm understanding, there's a big difference between being taught something and learning something because you usually have to earn that knowledge when you learn it. You can be taught something a thousand times and then you've learned that once.And so, we're trying to use those moments that we actually learn it where we earn that hard-earned information through an incident and tell those stories and convey that, and our team—the solutions team—is in there, helping people build these skills, teaching people how to talk to each other [laugh] and really find out this information during incidents, not after them.Corey: I really want to thank you for being as generous with your time as you have been. And if people want to learn more, where's the best place to find you?Emily: Oh. I was going to say Twitter, but… [laugh].Corey: Yeah, that's a big open question these days, isn't it? Assuming it's still there by the time this episode airs, it might be a few days between now and then. Where should they find you on Twitter, with a big asterisk next to it?Emily: It's at @themortalemily. Which, I started this by saying I like mess and I'm someone who loves incidents, so I'll be on Twitter [laugh].Corey: We're there to watch it all burn.Emily: Oh, I feel terrible saying that. Actually, if any Twitter engineers are listening to this, someone is found that the TLS certificate is going to expire at the end of this year. Please check Twitter for where that TLS certificate lives so that you all can renew that. Also, Jeli.io, we have a blog that a lot of us write, our solutions team, we—and honestly a lot of us, we tend to hire folks who have a lot of experience in incident response and analysis.I've never been a solutions engineer before in my life, but I've done a lot of incident response. So, we put up a lot of stuff and our goal is to build resources that are available to folks who are trying to make these changes happen, who are in those organizations where they're still doing five whys, and RCAs, and are trying to convince people to experiment and change. We have our Howie Guide, which is available for free. It's ‘How We Got Here' which is, like, a full, free incident analysis guide and a lot of cool blogs and stuff there. So, if you can't find me on Twitter, we're writing… things… there [laugh].Corey: We will, of course, put links to all of that in the [show notes 00:30:46]. Thank you so much for your time today. It's appreciated.Emily: Thank you, Corey. This was great.Corey: Emily Ruppe, solutions engineer at Jeli.io. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this episode, please leave a five-star review on your podcast platform of choice, along with an angry comment talking about how we've gotten it wrong and it is always someone's fault.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

Lora K. Joy is an adult adoptee. She is a closed, same race, domestic adoptee. She has self-published 3 illustrated books about her healing journey. This past year, she legally changed her first name back to her birth name and was legally reclaimed by her biological mother. Lora has been in reunion with her maternal side since 2008. After facing secondary rejection from her biological father, she has been in reunion with her paternal siblings and cousins since 2020 and is fully estranged from her adoptive parents. Lora's website and blog are at www.myadopteetruth.com. Books published by Lora K. Joy: Nobody Looks Like Me: An Adoptee Experience https://www.amazon.com/NoBODY-Looks-Like-Me-Experience/dp/1736990039/ref=sr_1_1?keywords=Lora+K.+Joy&qid=1674087157&sr=8-1 Self Attunement: An Adoptee Superpower https://www.amazon.com/Self-Attunement-Superpower-Lora-Joy/dp/1736990047/ref=sr_1_2?keywords=Lora+K.+Joy&qid=1674087157&sr=8-2 Goodbye Hypervilgilance: Healing Adoptee Worry https://www.amazon.com/Goodbye-Hypervigilance-Healing-Adoptee-Worry/dp/1736990012/ref=sr_1_3?keywords=Lora+K.+Joy&qid=1674087157&sr=8-3The Flourish Experience: The Power of Adoptee Healing in Community https://www.amazon.com/Flourish-Experience-Adoptee-Healing-Community/dp/B0B4TY22JC/ref=sr_1_1?crid=3TZ6F33JZCOCN&keywords=the+flourish+experience&qid=1674087538&sprefix=the+flourish+experience%2Caps%2C119&sr=8-1https://adoptee-voices.com/ Music by Corey Quinn and Invitational by MDT

About AerinAerin is a Cloud Sustainability Advocate and neurodiverse founder in tech on a mission to help developers understand the real impact that cloud computing has on the world and reduce their carbon emissions in the cloud. Did you know that internet and cloud computing contribute over 4% of annual carbon emissions? Twice that of the airline industry!Aerin also hosts "Public Cloud for Public Good," a podcast targeted towards developers and senior leaders in tech. Every episode, they also donate £500 to charities and highlight organisations that are working towards a better future. Listen and learn how you can contribute towards making the world a better place through the use of public cloud services.Links Referenced: Twitter: https://twitter.com/aerincloud LinkedIn: https://www.linkedin.com/in/aerinb/ Public Cloud for Public Good: https://publicgood.cloud/ duckbillgroup.com: https://duckbillgroup.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Uptycs, because they believe that many of you are looking to bolster your security posture with CNAPP and XDR solutions. They offer both cloud and endpoint security in a single UI and data model. Listeners can get Uptycs for up to 1,000 assets through the end of 2023 (that is next year) for $1. But this offer is only available for a limited time on UptycsSecretMenu.com. That's U-P-T-Y-C-S Secret Menu dot com.Corey: Cloud native just means you've got more components or microservices than anyone (even a mythical 10x engineer) can keep track of. With OpsLevel, you can build a catalog in minutes and forget needing that mythical 10x engineer. Now, you'll have a 10x service catalog to accompany your 10x service count. Visit OpsLevel.com to learn how easy it is to build and manage your service catalog. Connect to your git provider and you're off to the races with service import, repo ownership, tech docs, and more. Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn and I am joined what feels like roughly a year later by a returning guest, Aerin Booth. How long have you been?Aerin: I've been really great. You know, it's been a journey of a year, I think, since we sort of did this podcast even, like, you know, a year and a bit since we met, and, like, I'm doing so much and I think it's doing, like, a big difference. And yeah, I can't wait for everything else. It's just yeah, a lot of work right now, but I'm really enjoying it. So, I'm really well, thank you.Corey: Normally, I like to introduce people by giving their job title and the company in which they work because again, that's a big deal for an awful lot of people. But a year ago, you were independent. And now you still are. And back when I was doing my own consulting independently, it felt very weird to do that, so I'm just going to call you the Ted Lasso of cloud at this point.Aerin: [laugh].Corey: You've got the mustache, you've got the, I would say, obnoxiously sunny disposition. It's really, there's a certain affinity right there. So, there we go. I feel like that is the best descriptor for what you have become.Aerin: I—do know what, I only just watched Ted Lasso over Christmas and I really found it so motivational in some ways because wow, like, it's not just who we'd want to be in a lot of ways? And I think, you know, for the work that I do, which is focused on sustainability, like, I want to present a positive future, I want to encourage people to achieve more and collaborate, and yeah, basically work on all these problems that we need to be worked on. And yeah, I think that's [laugh] [crosstalk 00:02:02]—Corey: One of the challenges of talking to you sometimes is you talk about these depressing things, but there's such a—you take such an upbeat, positive approach to it that I, by comparison, invariably come away from our conversations during, like, I'm Surly McBastard over here.Aerin: [laugh]. Yeah, you can be the bad cop of cloud computing and I'll try and be the good cop. Do you know, you say that the stuff I talk about is depressing, and it is true and people do worry about climate change. Like I did an online conference recently, it's focused on FinOps, and we had a survey, “Do you worry about climate change?” 70% of the people that responded said they worry about it.So, we all know, it's something we worry about and we care about. And, you know, I guess what I'm really trying to do is encourage people to care a bit more and start taking action and look after yourself. Because you know, when you do start taking action towards it, when you join those communities that are also working on it, it is good, it is helpful. And, you know, I've gone through some ups and downs and some of this, like, just do I throw in the towel because no one cares about it? Like, we spoke last year; I had attended re:Invent for the first time.This year, I was able to speak at re:Invent. So, I did a talk on being ethical in tech. And it was fun, it was good. I enjoyed what I delivered, but I had about 35 people sign up to that. I'm pretty sure if I talked about serverless or the next Web3 blockchain product, I would have got hundreds more. But what I'm starting to realize is that I think people just aren't ready to, sort of, want to do this yet. And yeah, I'm hoping that'll change.Corey: Let's first talk about, I guess, something that is more temporally pressing than some other things. Not that it is more important than climate change, mind you, but it feels like it's on a shorter timeline which is, relatively soon after this recording, there is a conference that you are kicking off called The State of Open. Ajar, Aerin. The State of Open is ajar. What is this conference? Is it in person? Is it virtual? Is it something where you and three friends are going to show up and basically talk to each other? How big? How small? What is it? What's it about? Tell me more, please. I'm riveted.Aerin: So, State of Open conference is a conference that's been in the works now for maybe about two weeks, a little bit longer in the planning, but the work we've been putting in over the last two weeks. It'll be on the seventh and eighth of February in London as a physical event in the QEII Conference Centre, but it will also be available online. And you know, when we talk about the State of Open, it's that question: what is the State of Open? The state of open-source, the state of open hardware, and the state of open data. And it is going to be probably the first and hopefully the biggest open-source conference in the UK.We already have over 100 confirmed guest speakers from Jimmy Wales, the co-founder of Wikipedia, to many of our great guests and headliners who haven't even announced yet for the plenary. So, I'm really excited. And the reason why I wanted to get involved with this is because one of the coolest things about this conference—compared to some others like re:Invent, for example—is that sustainability and diversity run through every single thing that we do. So, as the content director, I reviewed every single CFP for both of these things. I mean, you couldn't get a better person than someone like me, who's the queer person who won't shut up about sustainability to sort of do this thing.So, you know, I looked after those scorings for the CFPs in support of the CFP chairs. And now, as I'm working with those individual speakers on their content and making sure that diversity is included in the content. It's not just the diversity of the speaker, for example it's, who were the other people whose voice you're raising? What other people if you worked on this? Are there anyone that you've mentored, like, you know, actually, you know, let's have this as a wider conversation?Corey: Thank God. I thought you were about to say diversity of thought, and I was about to reach through the screen to strangle you.Aerin: [laugh]. No, no. I mean, we're doing really well, so of the announced speakers online, we are 40% non-male and about 18% non-white, which to be honest, for a fair sheer conference, when we didn't really do that much to specifically call this out, but I would probably raise this to Amanda Brock, who is the CEO of OpenUK, you know, she has built a community in the UK and around the world over the last few years which has been putting women forward and building these links. And that's why we've had such a great response for our first-year conferences, the work she's put in. It's hard.Like, this isn't easy. You know, we've had to do a lot of work to make sure that it is representative, at least better than other conferences, at least. So, I'm really excited. And like, there's so much, like, open-source is probably going to be the thing that saves the world. If we're going to end up looking at two different futures with monopolies and closed systems and all the money going towards cloud providers versus a fair and equitable society, open-source is the thing that's going to get us closer to that. So yeah, this conference will be a great event.Corey: Is it all in person? Is it being live-streamed as well? What is the deal here?Aerin: So, in person, we have loads of different things going on, but what will be streamed online if you sign up for virtual ticket is five different tracks. So, our platform engineering track, our security track, government law and policy, open data, and open hardware. And of course, the keynote and plenaries. But one of the things I'm also really proud about this conference is that we're really focusing on the developer experience, like, you know, what is your experience at the conference? So, we also have an unconference, we have a sub-conference run by Sustain OSS focused on workshops related to climate change and sustainability.We have loads of developer experience halls in the event itself. And throughout the day, over the two days, we have two one-hour blocks with no speaking content at all so that we can really make sure that people have that hardware track and are out there meeting each other and having a good time. And obviously, of course, like any good conference, the all-hands party on the first night. So, it really is a conference that's doing things differently from diversity to sustainability to that experience. So, it's awesome.Corey: One of the challenges that I've seen historically around things aiming at the idea of open conferences—and when we talk open-source, et cetera, et cetera—open' seems like it is a direction parallel to, we haven't any money, where it's, “Yes, we're a free software foundation,” and it turns out conferences themselves are not free. And you wind up with a whole bunch of folks showing up to it who are, in many cases, around the fringes of things. There are individual hobbyists who are very passionate about a thing but do not have the position in the corporate world. I'm looking through the lengthy list of speakers you have here and that is very much not this. These are serious people at serious companies. Not that there are not folks who are individual practitioners and passionate advocates and hobbyists than the rest. This is, by virtually any way you look at it, a remarkably diverse conference.Aerin: Mmm. You know, you are right about, like, that problem in open-source. It's like, you know, we look at open and whether we want to do open and we just go, “Well, it won't make me any money. I can't do that. I don't have the time. I need to bring in some money.”And one of the really unique things, again, about this conference is—I have not even mentioned it yet—we have an entrepreneurship room. So, we have 20 tables filled with entrepreneurs and CEOs and founders of open-source companies throughout the two days where you can book in time to sit at that table and have conversations with them. Ask them the questions that you want to ask about, whether it's something that you want to work on, or a company you want to found, and you'll be able to get that time. I had a very similar experience in some ways. It was re:Invent.I was a peer talk expert and you know, I had 15 or so conversations with some really interesting people just because they were able put that time in and they were able to find me on the website. So, that's something we are replicating to get those 20 also entrepreneurs and co-founders out to everyone else. They want to be able to help you and support you.Corey: That is an excellent segue if I do say so myself. Let's talk about re:Invent. It's the one time of the year you and I get to spend time in the same room. One thing that I got wrong is that I overbooked myself as I often do, and I didn't have time to do anything on their peer talk expert program, which is, you more or less a way that any rando can book time to sit down and chat with you. Now, in my case, I have assassination concerns because it turns out Amazon employees can read that thing too and some of them might work on billing. One wonders.So yeah, I have to be a little careful for personal reasons but for most people, it's a non-issue. I didn't get as much time as I wanted to talk to folks in the community. That is not going to repeat itself at the end of this year. But what was your take on re:Invent, because I was in meetings for most of them?Aerin: So, comparing this re:Invent to the re:Invent I went to, my first re:Invent when we met in 2021, you know, that was the re:Invent that inspired me to get into sustainability. They'd announced stuff to do with the shared responsibility model. A few months later, they released their carbon calculator, and I was like, “Yeah, this is the problem. This is the thing I want to work on and it will make me happy.” And a lot of that goes into, you know, finding a passion that keeps me motivated when things aren't that great.When maybe not a lot of money is coming in, at least I know, I'm doing everything I can to help save the world. So, re:Invent 2021 really inspired me to get involved with sustainability. When I look at re:Invent 2022, you might have Adam Selipsky on the main stage saying that sustainability is the problem of our generation, but that is just talk and bluster compared to what they were putting out in terms of content and their experience of, like, let's say the sustainability—I don't know what to call it—tiny little square in the back of the MGM Grand compared to the paid hall in the expo. Like, you know, that's the sort of thing where you can already see the prioritization of money. Let's put the biggest sponsors and all the money that we can bring it in the big hall where everyone is, and then put the thing we care about the most, apparently—sustainability—in the back of the MGM.And that in itself was annoying, but then you get there in the content, and it was like a massive Rivian van, like, an advert for, “Oh, Amazon has done all this to electrify Rivian and deliver you Prime.” But where was the people working on sustainability in the cloud? You know, we had a couple of teams who were talking about the customer carbon footprint tool, but there was just not much. And I spoke to a lot of people and they were saying similar things, like, “Where are the announcements? Where are the actual interesting things?” Rather than just—which is kind of what I'm starting to realize is that a lot of the conversations about sustainability is about selling yourself as sustainable.Use me rather than my competitors because we're 88% more, kind of, carbon neutral when it comes to traditional data centers, not because we are really going to solve these problems. And not to say that Amazon isn't doing innovative, amazing things that no one else can't do, because that is true, and cloud as part of the solution, but you know, sustainability shouldn't be about making more sales and growing your business, it should be about making the world a better place, not just in terms of carbon emissions, but you know, our life, the tech that we can access. Three billion people on this planet have never accessed the internet. And as we continue to grow all of our services like AI and machine learning and new Web3, bloody managed services come online, that's going to be more carbon, more compute power going towards the already rich and the already westernized people, rather than solving the problems we need to solve in the face of climate change.So, I was a little bit disappointed. And I did put a tweet thread out about it afterwards. And I just hope it can be different next year and I hope more people will start to ask for this. And that also what I'm starting to realize is that until more Amazon customers put this as their number one priority and say, “I'm not going to do business with you because of this issue,” or, you know, “This is what we really care about,” they're not going to make a change. Unless it starts to impact their bottom lines and people start to choose other cloud providers, they're not going to prioritize it.And I think up until this point, we're not seeing that from customers. We're kind of getting some people like me shouting about it, but across the board, sustainability isn't the number one priority right now. It's, like what Amazon says, security or resiliency or something else.Corey: And I think that, at least from where I set, the challenge is that if you asked me what I got out of re:Invent, and what the conversations I had—going into it, what are my expectations, and what do I hope to get and how's it going to end up, and then you ask you that same question—though maybe you are a poor example of this—and then you ask someone who works out as an engineer at a company that uses AWS and their two or three years into their career, why don't you talk to a manager or director or someone else? And the problem is if you start polling the entire audience, you'll find that this becomes—you're going to wind up with 20 different answers, at least. The conference doesn't seem like it has any idea of what it wants to be and to whom and in that vacuum, it tries to be all things to all people. And surprise, just like the shooting multifunction printer some of us have in our homes, it doesn't do well with any of those things because it's trying to stand in too many worlds at the same time.Aerin: You know, let's not, like, look at this from a way that you know, re:Invent is crap and, like, do all the work that everyone puts it is wasted because it is a really great event for a lot of different things for a lot of different people. And to be honest, the work that the Amazon staff put into it is pretty out of this world. I feel sorry though because you know, the rush for AWS sell more and do this massive event, they put people through the grinder. And I feel like, I don't know, we could see the cracks in some of that, the way that works. But, you know, there's so many people that I speak to who were like, “Yeah, I'm definitely not going again. I'm not even going to go anywhere near submitting a talk.”And, sort of, the thing is, like, I can imagine if the conference was something different; it was focused at sustainability at number one, it was about making the world a better place from everything that they do, it was about bringing diverse communities together. Like, you know, bringing these things up the list would make the whole thing a lot better. And to be honest, it would probably make it a lot more enjoyable [laugh] for the Amazon staff who end up talking at it. Because, you know, I guess it can feel a bit soulless over time is all you're doing is making money for someone else and selling more things. And, yeah, I think there's a lot more… different things we can do and a lot more things we can talk about if people just start to talk about, like you know, if you care about this as well and you work at Amazon, then start saying that as well.It'll really make a difference if you say we want re:Invent to look different. I mean, even Amazon staff, [laugh] and we've not even mentioned this one because I got Covid straight after re:Invent, nine days and staring at a wall in hotel room in Vegas was not my idea of a good time post-conference. So, that was a horrible, horrible experience. But, you know, I've had people call it re:Infect. Like, where are the Covid support?Like, there was hardly any conversation about that. It was sort of like, “Don't mention it because oh, s”—whatever else. But imagine if you just did something a little bit differently to look like you care about your customers. Just say, “We recommend people mask or take a test,” or even provide tests and masks. Like, even if it's not mandatory, they could have done a lot more to make it safer for everyone. Because, yeah, imagine having the reputation of re:Infect rather than re:Invent?Corey: I can only imagine how that would play out.Aerin: Only imagine.Corey: Yeah, it's it feels like we're all collectively decided to pretend that the pandemic is over. Because yeah, that's a bummer. I don't want to think about it. You know, kind of like we approach climate change.Aerin: Yeah. At the end of the day, like, and I keep coming across this more and more, you know, my thinking has changed over the last year because, like, you know, initially it was like a hyperactive puppy. Why are we caring about this? Like, yeah, if I say it, people will come, but the reality is, we have to blinker ourselves in order to deal with a lot of this stuff. We can't always worry about all of this stuff all of the time. And that's fine. That's acceptable. We do that in so many different parts of our life.But there comes to a point when you kind of think, “How much do I care about this?” And for a lot of people, it's because they have kids. Like, anyone who has kids right now must have to think, “Wow, what's the future going to look like?” And if you worry about what the future is going to look like, make sure you're taking steps to make the world a better place and make it the future you want it to look like. You know, I made the decision a long time ago not to have kids because I don't think I'd want to bring anyone into the world on what it might actually end up being, but you know, when I speak to people who are older in the 60s and they're like, “Oh, you've got 100 years. You don't need to worry about it.” Like, “Maybe you can say that because you're closer to dying than I am.” But yeah, I have to worry about this now because I'll still be eighty when all this shit is kicking off [laugh].Corey: This episode is sponsored in part by our friends at Strata. Are you struggling to keep up with the demands of managing and securing identity in your distributed enterprise IT environment? You're not alone, but you shouldn't let that hold you back. With Strata's Identity Orchestration Platform, you can secure all your apps on any cloud with any IDP, so your IT teams will never have to refactor for identity again. Imagine modernizing app identity in minutes instead of months, deploying passwordless on any tricky old app, and achieving business resilience with always-on identity, all from one lightweight and flexible platform.Want to see it in action? Share your identity challenge with them on a discovery call and they'll hook you up with a complimentary pair of AirPods Pro. Don't miss out, visit Strata.io/ScreamingCloud. That's Strata dot io slash ScreamingCloud.Corey: That I guess is one of the big fears I have—and I think it's somewhat unfounded—is that every year starts to look too much like the year before it. Because it's one of those ideas where we start to see the pace of innovation is slowing at AWS—and I'm not saying that to piss people at Amazon off and have them come after me with pitchforks and torches again—but they're not launching new services at the rate they once did, which is good for customers, but it starts to feel like oh, have we hit peak cloud this is what it's going to look like? Absolutely not. I don't get the sense that the world is like, “Well, everything's been invented. Time to shut down the patent office,” anytime soon.And in the short term, it feels like oh, there's not a lot exciting going on, but you look back the last five years even and look at how far we've come even in that period of time and—what is it? “The days are long, but the years are short.” It becomes a very macro thing of as things ebb and flow, you start to see the differences but the micro basis on a year-to-year perspective, it seems harder to detect. So longer term, I think we're going to see what the story looks like. And it's going to be satisfying one. Just right now, it's like, well, this wasn't as entertaining as I would have hoped, so I'm annoyed. Which I am because it wasn't, but that's not the biggest problem in the world.Aerin: It's not. And, you know, you look at okay, cool, there wasn't all these new flashy services. There was a few things are announced, I mean, hopefully that are going to contribute towards climate change. One of them is called AWS Supply Chain. And the irony of seeing sort of like AWS Supply Chain where a company that already has issues with data and conversations around competition, saying to everyone, “Hey, trust us and give all of your supply chain information and put it into one of our AWS products,” while at the same time their customer carbon footprint tool won't even show the full scope for their emissions of their own supply chain is not lost on me.And you do say, “Maybe we should start seeing things at a macro level,” but unless Amazon and other cloud hyperscalers start pulling the finger out and showing us how they have got a vision between now and 2040, and now in 2050, of how they're going to get there, it kind of just feels like they're saying, “It'll all be fine as long as we continue to grow, as long as we keep sucking up the market.” And, you know, an interesting thing that just kicked off in the UK back in November was the Competition and Markets Authority have started an investigation into the cloud providers on how they are basically sucking up all these markets, and how the growth of things that are not hyperscale is going. So, in the UK, the percentage of cloud has obviously gone up—more and more cloud spending has gone up—but kind of usage across non-hyperscalers has gone down over that same period. And they really are at risk of sucking up the world. Like, I have got involved in a lot of different things.I'm an AWS community builder; like, I do promote AWS. And, you know, the reason why I promote cloud, for example is serverless. We need serverless as the way we run our IT because that's the only way we'll do things like time shifting or demand shifting. So, when we look at renewable energy on the grid if that really high, the wind is blowing and the sun is shining, we want more workloads to be running then and when they're tiny, and they're [unintelligible 00:21:03], and what's the call it serverless generally, uh—Corey: Hype?Aerin: Function as a Code?Corey: Function—yeah, Function as a Service and all kinds of other nonsense. But I have to ask, when you're talking about serverless, in this context, is a necessary prerequisite of serverless that scale to zero when it's [unintelligible 00:21:19].Aerin: [laugh]. I kind of go back to marketing. What Amazon releasing these days when it relates to serverless that isn't just marketing and saying, “Oh, it's serverless.” Because yeah, there was a few products this year that is not scaled to zero is it? It's a 100-pound minimum. And when you're looking at number of accounts that you have, that can add up really quickly and it excludes people from using it.Corey: It's worse than that because it's not number of accounts. I consider DynamoDB to be serverless, by any definition of the term. Because it is. And what I like about it is I can have a separate table for every developer, for every service or microservice or project that they have, and in fact, each branch can have its own stuff like that. I look at some of the stuff that I build with multi-branch testing and whatnot, and, “Oh, wow. That would cost more than the engineer if they were to do that with some of the serverless offerings that AWS has put out.”Which makes that entire philosophy a complete non-starter, which means that invariably as soon as you start developing down that path, you are making significant trade-offs. That's just from a economics slash developer ergonomics slash best practices point of view. But there's a sustainability story to it as well.Aerin: Yeah. I mean, this sustainability thing is like, if you're not going to encourage this new way of working, like, if you're not going to move everyone to this point of view and this is how we need to do things, then you kind of just propagating the old world, putting it into your data center. For every managed service that VMware migrated piece of crap, just that land in the cloud, it's not making a real difference in the world because that's still going to exist. And we mentioned this just before the podcast and, you know, a lot of focus these days and for a lot of people is, “Okay, green energy is the problem. We need to solve green energy.”And Amazon is the biggest purchaser of power purchase agreements in renewable energy around the world, more than most governments. Or I think that the biggest corporate purchaser of it anyway. And that all might sound great, like, “Oh, the cloud is going to solve this problem for me and Amazon is going to solve it for me even better because they're bigger.” But at the end of the day, when we think about a data center, it exists in the real world.It's made of concrete. You know, when you pour concrete and when you make concrete, it releases CO2. It's got racks of servers that all are running. So, those individual servers had to be made by whoever it is in Asia or mined from rare earth metals and end up in the supply chain and then transported into the data centers in us-east-1. And then things go wrong. You have to repair you have to replace and you have to maintain them.Unless we get these circular economies going in a closed system, we can't just continue to grow like this. Because carbon emissions related to Scope 3, all those things I've just been talking about, basically anything that isn't the energy, is about 80 to 90% of all the carbon emissions. So, when Amazon says, “Oh, we're going to go green and get energy done by 2030”—which is seven years away—they've then got ten years to solve 90% of the problem. And we cannot all just continue to grow and think of tech as neutral and better for the world if we still got that 90% problem, which we do right now. And it really frustrates me when you look at the world and the way we've jumped on technology just go on, “Oh, it must be good.”Like Bitcoin, for example. Bitcoin has released 200 million metric tons of CO2 since its inception. And for something that is basically a glorified Ponzi scheme, I can't see how that is making the world a better place. So, when cloud providers are making managed services for Web3 and for blockchain, and they're selling more and more AI and machine learning, basically so they can keep on selling GPU access, I do worry about whether our path to infinite growth with all of these hyperscalers is probably the wrong way of looking at things. So, linking back to, you know, the conference, open-source and, you know, thinking about things differently is really important in tech right now.And not just for your own well-being and being able to sleep at night, but this is how we're going to solve our problems. When all companies on the planet want people to be sustainable and we have to start tackling this because there's a financial cost related to it, then you're going to be in the vogue. If you're really good developer, thinking about things differently can be efficient, then yeah, you're the developer that's going to win in the future. You might be assisted by ChatGPT three or whatever else, but yeah, sustainability and efficiency can really be the number one priority because it's a win, win, win. We save the world, we make ourselves better, we sleep better at night, and you just become a better developer.I keep monologuing at this point, but you know, when it comes to stuff like games design, we look at things like Quake and Pokemon and all these things when there's like, “How did they get these amazing games and these amazing experiences in such small sizes,” they had boundaries. They had boundaries to innovate within because they had to. They couldn't release the game if they couldn't fit into the cartridge, therefore, they made it work. When the cloud is sold as infinitely scalable and horizontally scalable and no one needs to worry about this stuff because you can get your credit card out, people stop caring about being innovative and being more efficient. So yeah, let's get some more boundaries in the cloud.Corey: What I find that is super helpful, has been, like, if I can, like, descri—like, Instagram is down. Describe your lunch to me style meme description, like, the epic handshake where you have two people clasping hands, and one side is labeled in this case, ‘sustainability advocates,' and the other side should be labeled ‘cloud economists,' and in the middle, it's, “Turn that shit off.” Because it's not burning carbon if it's not running, and it's not costing you anything—ideally—if it's not running, so it's one of those ideas where we meet in the middle. And that's important, not just because it makes both of us independently happy because it's both good for the world and you'll get companies on board with this because, “Wait. We can do this thing and it saves us money?” Suddenly, you're getting them aligned because that is their religion.If companies could be said to have a religion, it is money. That's the way it works. So, you have to make it worth money for them to do the right thing or you're always going to be swimming upstream like a depressed salmon.Aerin: I mean, look at why [unintelligible 00:27:11] security is near the top: because there's so many big fines related to security breaches. It will cost them money not to be secure. Right now, it doesn't cost companies money to be inefficient or to release all this carbon, so they get away with it or they choose to do it. And I think that's going to change. We see in regulations across you're coming out.So, you know, if you work for a big multinational that operates in Europe, by next year, you'll have to report on all of your Scope 3 carbon emissions. If you're a customer of AWS right now, you have no ability to do that. So, you know, this is going to be crunch time over the next 18 months to two years for a lot of big businesses, for Amazon and the other hyperscalers, to really start demonstrating that they can do this. And I guess that's my big push. And, you know, I want to work with anyone, and it's funny because I have been running this business for about, you know, a couple of years now, it's been going really well, I did my podcast, I'm on this path.But I did, last year, take some time, and I applied into AWS. And you know, I was like, “Okay, maybe I'll apply for this big tech company and help Amazon out.” And because I'll take that salary and I'll do something really good with it afterwards, I'll do my time for three years and attend re:Invent and deliver 12 talks and never sleep, but you know, at the end of it, I'll say, “Okay, I've done that and now I can do something really good.” Unfortunately, I didn't get the role—or fortunately—but you know, when I applied for that role, what I said to them is, “I really care about sustainability. I want to make the world a better place. I want to help your customers be more sustainable.”And they didn't want me to join. So, I'm just going to continue doing that but from the outside. And whether that means working with politicians or developers or anyone else to try and make the world better and to kind of help fight against climate change, then, yeah, that's definitely what I'm doing.Corey: So, one last question before we wind up calling it an episode. How do we get there? What is the best next step that folks can take? Because it's easy to look at this as a grand problem and realize it's too big to solve. Well, great. You don't need to solve the entire problem. You need take the first step. What is that first step?Aerin: Individuals, I would say it's just realizing that you do care about it and you want to take action. And you're going to say to yourself, “Even if I do little things, I'm going to move forward towards that point.” So, if that is being a more sustainable engineer or getting more conversations about climate change or even just doing other things in your community to make the world a better place than it is, taking that action. But one thing that I can definitely help about and talk a bit more of is that at the conference itself, I'll be running a panel with some great experts called the, “Next Generation of Cloud Education.” So, I really think we need to—like I said earlier in the podcast—to think differently about the cloud and IT.So, I am doing this panel and I'm bringing together someone like Simon Wardley to help people do Wardley Mapping. Like, that is a tool that allows you to see the landscape that you're operating in. You know, if you use that sort of tool to understand the real-world impact of what you're doing, then you can start caring about it a bit more. I'm bringing in somebody called Anne Currie, who is a tech ethicist and speaker and lecturer, and she's actually written some [laugh] really great nonfiction books, which I'd recommend everyone reads. It starts with Utopia Five.And that's about asking, “Well, is this ethical? Can we continue to do these things?” Can't—talks about things about sustainability. If it's not sustainable for everyone, it's not ethical. So, when I mentioned 3 billion people currently don't use the internet, it's like, can we continue to just keep on doing things the same way?And then John Booth, who is a data center expert, to help us really understand what the reality is on the ground. What are these data centers really look like? And then Amanda Brock, from OpenUK in the conference will joining as well to talk about, kind of, open-source and how we can make the world kind of a better place by getting involved in these communities. So, that'll be a really great panel.But what I'm also doing is releasing this as an online course. So, for people who want to get involved, it will be very intimate, about 15 seats on each core, so three weeks for you to actually work and talk directly with some of these experts and me to figure out what you want to do in the world of climate change and how you can take those first steps. So, it'll be a journey that even starts with an ecotherapist to help us deal with climate grief and wonder about the things we can do as individuals to feel better ourselves and be happier. So, I think that'd be a really great thing for a lot of people. And, yeah, not only that, but… it'll be great for you, but it also goes towards making the world a better place.So, 50% of the course fees will be donated, 25%, to charity, and 25% supporting open-source projects. So, I think it kind of just win, win, win. And that's the story of sustainability in general. It's a win, win, win for everyone. If you start seeing the world through a lens of sustainability, you'll save money, you'll sleep better at night, you'll get involved with some really great communities, and meet some really great people who care about this as well. And yeah, it'll be a brighter future.Corey: If people want to learn more, where can they find you?Aerin: So, if you want to learn more about what I'm up to, I'm on Twitter under @aerincloud, that A-E-R-I-N cloud. And then you can also find me on LinkedIn. But I also run my own podcast that was inspired by Corey, called Public Cloud for Public Good talking about cloud sustainability and how to make the world a better place for the use of public cloud services.Corey: And we will, of course, put a link to that in the [show notes 00:32:32]. Thank you so much for your time. I appreciate it, as always.Aerin: Thank you.Corey: Aerin Booth, the Ted Lasso of cloud. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this episode, please leave a five-star review on your podcast platform of choice, along with an angry and insulting comment that I will immediately scale to zero in true serverless fashion.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Matt Ray (@mattray, Senior Community Manager for OpenCost) talks about the evolution of FinOps, alignment between Eng and Finance, and FinOps best practices. SHOW: 688CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotwNEW TO CLOUD? CHECK OUT OUR OTHER PODCAST - "CLOUDCAST BASICS"SHOW SPONSORS:Datadog Security Solution: Modern Monitoring and SecurityStart investigating security threats before it affects your customers with a free 14 day Datadog trial. Listeners of The Cloudcast will also receive a free Datadog T-shirt.Eaton HomepageEaton and Tripp Lite have joined forces to bring more sanity to IT pros days, every day. Visit www.eaton.com/audio to learn more!SHOW NOTES:FinOps FoundationOpenCost (open source project)5 Strategies to Control Cloud CostsTopic 1 - Welcome to the show. Tell us a little bit about your background, and where you focus on Cloud Cost Management today.Topic 2 - The cloud has been around for more than a decade. Why does it seem like Cloud Cost Management has suddenly become such a big topic of discussion over the last couple years - is it mostly the pandemic and economy (interest rates), or more driven by end-user behaviors….or both? Topic 3 - What do best practices for Cost Management look like today? It is mostly about having the right monitoring tools, or is it how groups are organized, or something else?Topic 4 - How in the world can any finance person understand the nuances of all the services that are available in the cloud, and how they might impact both architecture and ultimately the bill? Who needs to be educating them? Topic 5 - How do most companies engage around cost management? Does it start with cost management tools and then bring in consultants (e.g. Corey Quinn) if they can't figure things out? Or is it the CFO getting heavy-handed and setting strict policies? Topic 6 - Let's talk about OpenCost. Topic 7 - Given all the variables involved, can you be useful in the cloud cost management space if they don't have a strong background in building applications or architecture? Topic 8 - What are some of the areas of cloud cost management that are most interesting to you today? FEEDBACK?Email: show at the cloudcast dot netTwitter: @thecloudcastnet

Isaac Etter is an activist and social entrepreneur. He was transracially adopted at the age of two. He is the founder of Identity, a startup focused on using technology to help foster and adoptive families thrive. Isaac has worked in adoption through his consulting firm Etter Consulting for the past 5 years. At Etter Consulting, Isaac led trainings for families and adoption agencies on transracial adoption. He is passionate about making sure adoptees have parents that can support them. Isaac is excited to chat about his work in transracial adoption.Isaac mentions interracial/transracial adoptees on Instagram Jessica Luciere @jmluciere and Hannah Matthews @HannahjacksonmatthewsIsaac Etter can be reached at isaac@identitylearning.co Website: https://www.identitylearning.co/ https://isaacetter.com/etterconsultingMusic by Corey Quinn

About Chris Chris Farris has been in the IT field since 1994 primarily focused on Linux, networking, and security. For the last 8 years, he has focused on public-cloud and public-cloud security. He has built and evolved multiple cloud security programs for major media companies, focusing on enabling the broader security team's objectives of secure design, incident response and vulnerability management. He has developed cloud security standards and baselines to provide risk-based guidance to development and operations teams. As a practitioner, he's architected and implemented multiple serverless and traditional cloud applications focused on deployment, security, operations, and financial modeling.Chris now does cloud security research for Turbot and evangelizes for the open source tool Steampipe. He is one if the organizers of the fwd:cloudsec conference (https://fwdcloudsec.org) and has given multiple presentations at AWS conferences and BSides events.When not building things with AWS's building blocks, he enjoys building Legos with his kid and figuring out what interesting part of the globe to travel to next. He opines on security and technology on Twitter and his website https://www.chrisfarris.comLinks Referenced: Turbot: https://turbot.com/ fwd:cloudsec: https://fwdcloudsec.org/ Steampipe: https://steampipe.io/ Steampipe block: https://steampipe.io/blog TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Tailscale SSH is a new, and arguably better way to SSH. Once you've enabled Tailscale SSH on your server and user devices, Tailscale takes care of the rest. So you don't need to manage, rotate, or distribute new SSH keys every time someone on your team leaves. Pretty cool, right? Tailscale gives each device in your network a node key to connect to your VPN, and uses that same key for SSH authorization and encryption. So basically you're SSHing the same way that you're already managing your network.So what's the benefit? Well, built-in key rotation, the ability to manage permissions as code, connectivity between any two devices, and reduced latency. You can even ask users to re-authenticate SSH connections for that extra bit of security to keep the compliance folks happy. Try Tailscale now - it's free forever for personal use.Corey: This episode is sponsored by our friends at Logicworks. Getting to the cloud is challenging enough for many places, especially maintaining security, resiliency, cost control, agility, etc, etc, etc. Things break, configurations drift, technology advances, and organizations, frankly, need to evolve. How can you get to the cloud faster and ensure you have the right team in place to maintain success over time? Day 2 matters. Work with a partner who gets it - Logicworks combines the cloud expertise and platform automation to customize solutions to meet your unique requirements. Get started by chatting with a cloud specialist today at snark.cloud/logicworks. That's snark.cloud/logicworksCorey: Welcome to Screaming in the Cloud. I'm Corey Quinn. My guest today is someone that I have been meaning to invite slash drag onto this show for a number of years. We first met at re:Inforce the first year that they had such a thing, Amazon's security conference for cloud, as is Amazon's tradition, named after an email subject line. Chris Farris is a cloud security nerd at Turbot. He's also one of the organizers for fwd:cloudsec, another security conference named after an email subject line with a lot more self-awareness than any of Amazon's stuff. Chris, thank you for joining me.Chris: Oh, thank you for dragging me on. You can let go of my hair now.Corey: Wonderful, wonderful. That's why we're all having the thinning hair going on. People just use it to drag us to and fro, it seems. So, you've been doing something that I'm only going to describe as weird lately because your background—not that dissimilar from mine—is as a practitioner. You've been heavily involved in the security space for a while and lately, I keep seeing an awful lot of things with your name on them getting sucked up by the giant app surveillance apparatus deployed to the internet, looking for basically any mention of AWS that I wind up using to write my newsletter and feed the content grist mill every year. What are you doing and how'd you get there?Chris: So, what am I doing right now is, I'm in marketing. It's kind of a, you know, “Oops, I'm sorry I did that.”Corey: Oh, the running gag is, you work in DevRel; that means, “Oh, you're in marketing, but they're scared to tell you that.” You're self-aware.Chris: Yeah.Corey: Good for you.Chris: I'm willing to address that I'm in marketing now. And I've been a cloud practitioner since probably 2014, cloud security since about 2017. And then just decided, the problem that we have in the cloud security community is a lot of us are just kind of sitting in a corner in our companies and solving problems for our companies, but we're not solving the problems at scale. So, I wanted a job that would allow me to reach a broader audience and help a broader audience. Where I see cloud security having—you know, or cloud in general falling down is Amazon makes it really hard for you to do your side of shared responsibility, and so we need to be out there helping customers understand what they need to be doing. So, I am now at a company called Turbot and we're really trying to promote cloud security.Corey: One of the first promoted guest episodes of this show was David Boeke, your CTO, and one of the things that I regret is that I've sort of lost track of Turbot over the past few years because, yeah, one or two things might have been going on during that timeline as I look back at having kids in the middle of a pandemic and the deadly plague o'er land. And suddenly, every conversation takes place over Zoom, which is like, “Oh, good, it's like a happy hour only instead, now it's just like a conference call for work.” It's like, ‘Conference Calls: The Drinking Game' is never the great direction to go in. But it seems the world is recovering. We're going to be able to spend some time together at re:Invent by all accounts that I'm actively looking forward to.As of this recording, you're relatively new to Turbot, and I figured out that you were going there because, once again, content hits my filters. You wrote a fascinating blog post that hits on an interest of mine that I don't usually talk about much because it's off-putting to some folk, and these days, I don't want to get yelled at and more than I have to about the experience of traveling, I believe it was to an all-hands on the other side of the world.Chris: Yep. So, my first day on the job at Turbot, I was landing in Kuala Lumpur, Malaysia, having left the United States 24 hours—or was it 48? It's hard to tell when you go to the other side of the planet and the time zones have also shifted—and then having left my prior company day before that. But yeah, so Turbot about traditionally has an annual event where we all get together in person. We're a completely remote company, but once a year, we all get together in person in our integrate event.And so, that was my first day on the job. And then you know, it was basically two weeks of reasonably intense hackathons, building out a lot of stuff that hopefully will show up open-source shortly. And then yeah, meeting all of my coworkers. And that was nice.Corey: You've always had a focus through all the time that I've known you and all the public content that you've put out there that has come across my desk that seems to center around security. It's sort of an area that I give a nod to more often than I would like, on some level, but that tends to be your bread and butter. Your focus seems to be almost overwhelmingly on I would call it AWS security. Is that fair to say or is that a mischaracterization of how you view it slash what you actually do? Because, again, we have these parasocial relationships with voices on the internet. And it's like, “Oh, yeah, I know all about that person.” Yeah, you've met them once and all you know other than that is what they put on Twitter.Chris: You follow me on Twitter. Yeah, I would argue that yes, a lot of what I do is AWS-related security because in the past, a lot of what I've been responsible for is cloud security in AWS. But I've always worked for companies that were multi-cloud; it's just that 90% of everything was Amazon and so therefore 90% of my time, 90% of my problems, 90% of my risk was all in AWS. I've been trying to break out of that. I've been trying to understand the other clouds.One of the nice aspects of this role and working on Steampipe is I am now experimenting with other clouds. The whole goal here is to be able to scale our ability as an industry and as security practitioners to support multiple clouds. Because whether we want to or not, we've got it. And so, even though 90% of my spend, 90% of my resources, 90% of my applications may be in AWS, that 10% that I'm ignoring is probably more than 10% of my risk, and we really do need to understand and support major clouds equally.Corey: One post you had recently that I find myself in wholehearted agreement with is on the adoption of Tailscale in the enterprise. I use it for all of my personal nonsense and it is transformative. I like the idea of what that portends for a multi-cloud, or poly-cloud, or whatever the hell we're calling it this week, sort of architectures were historically one of the biggest problems in getting to clouds two speak to one another and manage them in an intelligent way is the security models are different, the user identity stuff is different as well, and the network stuff has always been nightmarish. Well, with Tailscale, you don't have to worry about that in the same way at all. You can, more or less, ignore it, turn on host-based firewalls for everything and just allow Tailscale. And suddenly, okay, I don't really have to think about this in the same way.Chris: Yeah. And you get the micro-segmentation out of it, too, which is really nice. I will agree that I had not looked at Tailscale until I was asked to look at Tailscale, and then it was just like, “Oh, I am completely redoing my home network on that.” But looking at it, it's going to scare some old-school network engineers, it's going to impact their livelihoods and that is going to make them very defensive. And so, what I wanted to do in that post was kind of address, as a practitioner, if I was looking at this with an enterprise lens, what are the concerns you would have on deploying Tailscale in your environment?A lot of those were, you know, around user management. I think the big one that is—it's a new thing in enterprise security, but kind of this host profiling, which is hey, before I let your laptop on the network, I'm going to go make sure that you have antivirus and some kind of EDR, XDR, blah-DR agents so that you know we have a reasonable thing that you're not going to just go and drop [unintelligible 00:09:01] on the network and next thing you know, we're Maersk. Tailscale, that's going to be their biggest thing that they are going to have to figure out is how do they work with some of these enterprise concerns and things along those lines. But I think it's an excellent technology, it was super easy to set up. And the ability to fine-tune and microsegment is great.Corey: Wildly so. They occasionally sponsor my nonsense. I have no earthly idea whether this episode is one of them because we have an editorial firewall—they're not paying me to set any of this stuff, like, “And this is brought to you by whatever.” Yeah, that's the sponsored ad part. This is just, I'm in love with the product.One of the most annoying things about it to me is that I haven't found a reason to give them money yet because the free tier for my personal stuff is very comfortably sized and I don't have a traditional enterprise network or anything like that people would benefit from over here. For one area in cloud security that I think I have potentially been misunderstood around, so I want to take at least this opportunity to clear the air on it a little bit has been that, by all accounts, I've spent the last, mmm, few months or so just absolutely beating the crap out of Azure. Before I wind up adding a little nuance and context to that, I'd love to get your take on what, by all accounts, has been a pretty disastrous year-and-a-half for Azure security.Chris: I think it's been a disastrous year-and-a-half for Azure security. Um—[laugh].Corey: [laugh]. That was something of a leading question, wasn't it?Chris: Yeah, no, I mean, it is. And if you think, though, back, Microsoft's repeatedly had these the ebb and flow of security disasters. You know, Code Red back in whatever the 2000s, NT 4.0 patching back in the '90s. So, I think we're just hitting one of those peaks again, or hopefully, we're hitting the peak and not [laugh] just starting the uptick. A lot of what Azure has built is stuff that they already had, commercial off-the-shelf software, they wrapped multi-tenancy around it, gave it a new SKU under the Azure name, and called is cloud. So, am I super-surprised that somebody figured out how to leverage a Jupyter notebook to find the back-end credentials to drop the firewall tables to go find the next guy over's Cosmos DB? No, I'm not.Corey: I find their failures to be less egregious on a technical basis because let's face it, let's be very clear here, this stuff is hard. I am not pretending for even a slight second that I'm a better security engineer than the very capable, very competent people who work there. This stuff is incredibly hard. And I'm not—Chris: And very well-funded people.Corey: Oh, absolutely, yeah. They make more than I do, presumably. But it's one of those areas where I'm not sitting here trying to dunk on them, their work, their efforts, et cetera, and I don't do a good enough job of clarifying that. My problem is the complete radio silence coming out of Microsoft on this. If AWS had a series of issues like this, I'm hard-pressed to imagine a scenario where they would not have much more transparent communications, they might very well trot out a number of their execs to go on a tour to wind up talking about these things and what they're doing systemically to change it.Because six of these in, it's like, okay, this is now a cultural problem. It's not one rando engineer wandering around the company screwing things up on a rotational basis. It's, what are you going to do? It's unlikely that firing Steven is going to be your fix for these things. So, that is part of it.And then most recently, they wound up having a blog post on the MSRC, the Microsoft Security Resource Center is I believe that acronym? The [mrsth], whatever; and it sounds like a virus you pick up in a hospital—but the problem that I have with it is that they spent most of that being overly defensive and dunking on SOCRadar, the vulnerability researcher who found this and reported it to them. And they had all kinds of quibbles with how it was done, what they did with it, et cetera, et cetera. It's, “Excuse me, you're the ones that left customer data sitting out there in the Azure equivalent of an S3 bucket and you're calling other people out for basically doing your job for you? Excuse me?”Chris: But it wasn't sensitive customer data. It was only the contract information, so therefore it was okay.Corey: Yeah, if I put my contract information out there and try and claim it's not sensitive information, my clients will laugh and laugh as they sue me into the Stone Age.Chris: Yeah well, clearly, you don't have the same level of clickthrough terms that Microsoft is able to negotiate because, you know, [laugh].Corey: It's awful as well, it doesn't even work because, “Oh, it's okay, I lost some of your data, but that's okay because it wasn't particularly sensitive.” Isn't that kind of up to you?Chris: Yes. And if A, I'm actually, you know, a big AWS shop and then I'm looking at Azure and I've got my negotiations in there and Amazon gets wind that I'm negotiating with Azure, that's not going to do well for me and my business. So no, this kind of material is incredibly sensitive. And that was an incredibly tone-deaf response on their part. But you know, to some extent, it was more of a response than we've seen from some of the other Azure multi-tenancy breakdowns.Corey: Yeah, at least they actually said something. I mean, there is that. It's just—it's wild to me. And again, I say this as an Azure customer myself. Their computer vision API is basically just this side of magic, as best I can tell, and none of the other providers have anything like it.That's what I want. But, you know, it almost feels like that service is under NDA because no one talks about it when they're using this service. I did a whole blog post singing its praises and no one from that team reached out to me to say, “Hey, glad you liked it.” Not that they owe me anything, but at the same time it's incredible. Why am I getting shut out? It's like, does this company just have an entire policy of not saying anything ever to anyone at any time? It seems it.Chris: So, a long time ago, I came to this realization that even if you just look at the terminology of the three providers, Amazon has accounts. Why does Amazon have Amazon—or AWS accounts? Because they're a retail company and that's what you signed up with to buy your underwear. Google has projects because they were, I guess, a developer-first thing and that was how they thought about it is, “Oh, you're going to go build something. Here's your project.”What does Microsoft have? Microsoft Azure Subscriptions. Because they are still about the corporate enterprise IT model of it's really about how much we're charging you, not really about what you're getting. So, given that you're not a big enterprise IT customer, you don't—I presume—do lots and lots of golfing at expensive golf resorts, you're probably not fitting their demographic.Corey: You're absolutely not. And that's wild to me. And yet, here we are.Chris: Now, what's scary is they are doing so many interesting things with artificial intelligence… that if… their multi-tenancy boundaries are as bad as we're starting to see, then what else is out there? And more and more, we is carbon-based life forms are relying on Microsoft and other cloud providers to build AI, that's kind of a scary thing. Go watch Satya's keynote at Microsoft Ignite and he's showing you all sorts of ways that AI is going to start replacing the gig economy. You know, it's not just Tesla and self-driving cars at this point. Dali is going to replace the independent graphics designer.They've got things coming out in their office suite that are going to replace the mom-and-pop marketing shops that are generating menus and doing marketing plans for your local restaurants or whatever. There's a whole slew of things where they're really trying to replace people.Corey: That is a wild thing to me. And part of the problem I have in covering AWS is that I have to differentiate in a bunch of different ways between AWS and its Amazon corporate parent. And they have that problem, too, internally. Part of the challenge they have, in many cases, is that perks you give to employees have to scale to one-and-a-half million people, many of them in fulfillment center warehouse things. And that is a different type of problem that a comp