Podcasts about last week in aws

  • 12PODCASTS
  • 932EPISODES
  • 22mAVG DURATION
  • 5WEEKLY NEW EPISODES
  • Feb 1, 2023LATEST

POPULARITY

20152016201720182019202020212022

Categories



Best podcasts about last week in aws

Latest podcast episodes about last week in aws

AWS Morning Brief
S3 Encryption at Rest Does NOT Solve for Bucket Negligence

AWS Morning Brief

Play Episode Listen Later Feb 1, 2023 8:31


Want to give your ears a break and read this as an article? You're looking for this link.https://www.lastweekinaws.com/blog/s3-encryption-at-rest-does-not-solve-for-bucket-negligence/Never miss an episode Join the Last Week in AWS newsletter Subscribe wherever you get your podcasts Help the show Leave a review Share your feedback Subscribe wherever you get your podcasts Buy our merch https://store.lastweekinaws.comWhat's Corey up to? Follow Corey on Twitter (@quinnypig) See our recent work at the Duckbill Group Apply to work with Corey and the Duckbill Group to help lower your AWS bill

Screaming in the Cloud
The Art of Effective Incident Response with Emily Ruppe

Screaming in the Cloud

Play Episode Listen Later Jan 31, 2023 34:22


About EmilyEmily Ruppe is a Solutions Engineer at Jeli.io whose greatest accomplishment was once being referred to as “the Bob Ross of incident reviews.” Previously Emily has written hundreds of status posts, incident timelines and analyses at SendGrid, and was a founding member of the Incident Command team at Twilio. She's written on human centered incident management and facilitating incident reviews. Emily believes the most important thing in both life and incidents is having enough snacks.Links Referenced: Jeli.io: https://jeli.io Twitter: https://twitter.com/themortalemily Howie Guide: https://www.jeli.io/howie/welcome TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored by our friends at Logicworks. Getting to the cloud is challenging enough for many places, especially maintaining security, resiliency, cost control, agility, etc, etc, etc. Things break, configurations drift, technology advances, and organizations, frankly, need to evolve. How can you get to the cloud faster and ensure you have the right team in place to maintain success over time? Day 2 matters. Work with a partner who gets it - Logicworks combines the cloud expertise and platform automation to customize solutions to meet your unique requirements. Get started by chatting with a cloud specialist today at snark.cloud/logicworks. That's snark.cloud/logicworksCorey: Cloud native just means you've got more components or microservices than anyone (even a mythical 10x engineer) can keep track of. With OpsLevel, you can build a catalog in minutes and forget needing that mythical 10x engineer. Now, you'll have a 10x service catalog to accompany your 10x service count. Visit OpsLevel.com to learn how easy it is to build and manage your service catalog. Connect to your git provider and you're off to the races with service import, repo ownership, tech docs, and more. Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. My guest today is Emily Ruppe, who's a solutions engineer over at Jeli.io, but her entire career has generally focused around incident management. So, I sort of view her as being my eternal nemesis, just because I like to cause problems by and large and then I make incidents for other people to wind up solving. Emily, thank you for joining me and agreeing to suffer my slings and arrows here.Emily: Yeah. Hey, I like causing problems too. I am a solutions engineer, but sometimes we like to call ourselves problems engineers. So.Corey: Yeah, I'm a problems architect is generally how I tend to view it. But doing the work, ah, one wonders. So, you are a Jeli, where as of this recording, you've been for a year now. And before that, you spent some time over at Twilio slash SendGrid—spoiler, it's kind of the same company, given the way acquisitions tend to work and all. And—Emily: Now, it is.Corey: Yeah. Oh, yeah. You were there during the acquisition.Emily: Mm-hm. Yes, they acquired me and that's why they bought SendGrid.Corey: Indeed. It's a good reason to acquire a company. That one person I want to bring in. Absolutely. So, you started with email and then effectively continued in that general direction, given the Twilio now has eaten that business whole. And that's where I started my career.The one thing I've learned about email systems is that they love to cause problems because it's either completely invisible and no one knows, or suddenly an email didn't go through and everyone's screaming at you. And there's no upside, only down. So, let me ask the obvious question I suspect I know the answer to here. What made you decide to get into incident management?Emily: [laugh]. Well, I joined SendGrid actually, I've, I love mess. I run towards problems. I'm someone who really enjoys that. My ADHD, I hyperfocus, incidents are like that perfect environment of just, like, all of the problems are laying themselves out right in front of you, the distraction is the focus. It's kind of a wonderful place where I really enjoy the flow of that.But I've started in customer support. I've been in technical support and customer—I used to work at the Apple Store, I worked at the Genius Bar for a long time, moved into technical support over the phone, and whenever things broke really bad, I really enjoyed that process and kind of getting involved in incidents. And I came, I was one of two weekend support people at SendGrid, came in during a time of change and growth. And everyone knows that growth, usually exponential growth, usually happens very smoothly and nothing breaks during that time. So… no, there was a lot of incidents.And because I was on the weekend, one of the only people on the weekend, I kind of had to very quickly find my way and learn when do I escalate this. How do I make the determination that this is something that is an incident? And you know, is this worth paging engineers that are on their weekend? And getting involved in incidents and being kind of a core communication between our customers and engineers.Corey: For those who might not have been involved in sufficiently scaled-out environments, that sounds counterintuitive, but one of the things that you learn—very often the hard way—has been that as you continue down the path of building a site out and scaling it, it stops being an issue relatively quickly of, “Is the site up or down?” And instead becomes a question of, “How up is it?” So, it's it doesn't sound obvious until you've lived it, but declaring what is an incident versus what isn't an incident is incredibly nuanced and it's not the sort of thing that lends itself to casual solutions. Because every time a customer gets an error, we should open an incident on that. Well, I've worked at companies that throw dozens of 500 errors every second at their scale. You will never hire enough people to solve that if you do an incident process on even 10% of them.Emily: Yeah. So, I mean, it actually became something that when you join Twilio, they have you create a project using Twilio's API to earn your track jacket, essentially. It's kind of like an onboarding thing. And as they absorbed SendGrid, we all did that onboarding process. And mine was a number for support people to text and it would ask them six questions and if they answered yes to more than two of them, it would text back, “Okay, maybe you should escalate this.”And the questions were pretty simple of, “Can emails be sent?” [laugh]. Can customers log into their website? Are you able to view this particular part of the website? Because it is—with email in particular, at SendGrid in particular—the bulk of it is the email API. So, like, the site being up or down was the easiest type of incident, the easiest thing to flex on because that's so much easier to see.Being able to determine, like, what percentage or what level, like, how many emails are not processing? Are they getting stuck or is this, like, the correct amount of things that should be bouncing because of IP reput—there's, like, a thousand different things. We had kind of this visualization of this mail pipeline that was just a mess of all of these different pipes kind of connected together. And mail could get stuck in a lot of different places, so it was a lot of spending time trying to find that and segwayed into project management. I was a QA for a little while doing QA work.Became a project manager and learned a lot about imposing process because you're supposed to and that sometimes imposing process on teams that are working well can actually destroy them [laugh]. So, I learned a lot of interesting things about process the hard way. And during all of that time that I was doing project management, I kind of accidentally started owning the incident response process because a lot of people left, I had been a part of the incident analysis group as well, and so I kind of became the sole owner of that. And when Twilio purchase SendGrid, I found out they were creating an incident commander team and I just reached out and said, “Here's all of SendGrids incident response stuff. We just created a new Slackbot, I just retrained the entire team on how to talk to each other and recognize when something might be an incident. Please don't rewrite all of this to be Twillio's response process.”And Terry, the person who was putting together that team said, “Excellent. You're going to be [laugh] welcome to Twilio Incident Command. This is your problem and it's a lot worse than you thought because here's all the rest of it.” So yeah, it was really interesting experience coming into technically the same company, but an entirely different company and finding out—like, really trying to learn and understand all of the differences, and you know, the different problems, the different organizational history, the, like, fascia that has been built up between some of these parts of the organization to understand why things are the way that they are within process. It's very interesting.And I kind of get to do it now as my job. I get to learn about the full organizational subtext of [laugh] all of these different companies to understand how incident response works, how incident analysis works, and maybe some of the whys. Like, what are the places where there was a very bad incident, so we put in very specific, very strange process pieces in order to navigate that, or teams that are difficult to work with, so we've built up interesting process around them. So yeah.Corey: It feels like that can almost become ossified if you're not careful because you wind up with a release process that's two thousand steps long, and each one of them is there to wind up avoiding a specific type of failure that had happened previously. And this gets into a world where, in so many cases, there needs to be a level of dynamism to how you wind up going about your work. It feels almost like companies have this idealized vision of the future where if they can distill every task that happens within the company down to a series of inputs and responses—scripts almost—you can either wind up replacing your staff with a bunch of folks who just work from a runbook and cost way less money or computers in the ultimate sense of things. But that's been teased for generations now and I have a very hard time seeing a path where you're ever going to be able to replace the contextually informed level of human judgment that, honestly, has fixed every incident I've ever seen.Emily: Yeah. The problem comes down to in my opinion, the fact that humans wrote this code, people with specific context and specific understanding of how the thing needs to work in a specific way and the shortcomings and limitations they have for the libraries they're using or the different things are trying to integrate in, a human being is who's writing the code. Code is not being written by computers, it's being written by people who have understanding and subtext. And so, when you have that code written and then maybe that person leaves or that person joins a different team and they focus and priorities on something else, there is still human subtests that exists within the services that have been written. We have it call in this specific way and timeout in this specific amount of time because when we were writing it, there was this ancient service that we had to integrate with.Like, there's always just these little pieces of we had to do things because we were people trying to make connections with lines of code. We're trying to connect a bunch of things to do some sort of task, and we have a human understanding of how to get from A to B, and probably if A computer wrote this code, it would work in an entirely different way, so in order to debug a problem, the humans usually need some sort of context, like, why did we do this the way that we did this? And I think it's a really interesting thing that we're finding that it is very hard to replace humans around computers, even though intellectually we think, like, this is all computers. But it's not. It's people convincing computers to do things that maybe they shouldn't necessarily be doing. Sometimes they're things that computers shouldn't be doing, maybe, but a lot of the times, it's kind of a miracle [laugh] that any of these things continue to work on it on a given basis. And I think that it's very interesting when we, I think, we think that we can take people out of it.Corey: The problem I keep running into though, the more I think about this and the more I see it out there is I don't think that it necessarily did incident management any favors when it was originally cast as the idea of blamelessness and blameless postmortems. Just because it seems an awful lot to me like the people who are the most advocate champions of approaching things from a blameless perspective and having a blameless culture are the people who would otherwise have been blamed themselves. So, it really kind of feels on some broader level, like, “Oh, was this entire movement really just about being self-serving so that people don't themselves get in trouble?” Because if you're not going to blame no one, you're going to blame me instead. I think that, on some level, set up a framing that was not usually helpful for folks with only a limited understanding of what the incident lifecycle looks like.Emily: Mmm. Yeah, I think we've evolved, right? I think, from the blameless, I think there was good intentions there, but I think that we actually missed the really big part of that boat that a lot of folks glossed over because then, as it is now, it's a little bit harder to sell. When we're talking about being blameless, we have to talk about circumventing blame in order to get people to talk candidly about their experiences. And really, it's less about blaming someone and what they've done because we as humans blame—there's a great Brené Brown talk that she gives, I think it's a TED talk about blame and how we as humans cannot physically avoid blaming, placing blame on things.It's about understanding where that's coming from, and working through it that is actually how we grow. And I think that we're starting to kind of shift into this more blame-aware culture. But I think the hard pill to swallow about blamelessness is that we actually need to talk about the way that this stuff makes us feel as people. Like feelings, like emotions [laugh]. Talk about emotions during a technical incident review is not really an easy thing to get some tech executives to swallow.Or even engineers. There's a lot of engineers who are just kind of like, “Why do you care about how I felt about this problem?” But in reality, you can't measure emotions as easily as you can measure Mean Time to Resolution. But Mean Time to Resolution is impacted really heavily by, like, were we freaking out? Did we feel like we had absolutely no idea what we were trying to solve, or did we understand this problem, and we were confident that we could solve it; we just couldn't find the specific place where this bug was happening. All of that is really interesting and important context about how we work together and how our processes work for us, but it's hard because we have to talk about our feelings.Corey: I think that you're onto something here because I look back at the key outages that really define my perspective on things over the course of my career, and most of the early ones were beset by a sense of panic of am I going to get fired for this? Because at the time, I was firmly convinced that well, root cause is me. I am the person that did the thing that blew up production. And while I am certainly not blameless in some of those things, I was never setting out with an intent to wind up tiering things down. So, it was not that I was a bad actor subverting internal controls because, in many companies, you don't need that level of rigor.This was a combination of factors that made it easy or possible to wind up tiering things down when I did not mean to. So, there were absolutely systemic issues there. But I still remember that rising tide of panic. Like, should I be focused on getting the site backup or updating my resume? Which of these is going to be the better longer-term outcome? And now that I've been in this industry long enough and I've seen enough of these, it's, you almost don't feel the blood pressure rise anymore when you wind up having something gets panicky. But it takes time and nuance to get there.Emily: Yeah. Well, and it's also, in order to best understand how you got in that situation, like, were you willing to tell people that you were absolutely panicked? Would you have felt comfortable, like, if someone was saying like, “Okay, so what happened? How did—walk me through what you were experiencing?” Would you have said like, “I was scared out of my goddamn mind?”Were you absolutely panicking or did you feel like you had some, like, grasping at some straws? Like, where were you? Because uncovering that for the person who is experiencing that in the issue, in the incident can help understand, what resources did they feel like they knew where to go to. Or where did they go to? Like, what resource did they decide in the middle of this panicked haze to grasp for? Is that something that we should start using as, “Hey, if it's your first time on call, this is a great thing to pull into,” because that's where instinctively you went?Like, there's so much that we can learn from the people who are experiencing [laugh] this massive amount of panic during the incident. But sometimes we will, if we're being quote-unquote, “Blameless,” gloss over your entire, like, your involvement in that entirely. Because we don't want to blame Corey for this thing happening. Instead, we'll say, “An engineer made a decision and that's fine. We'll move past that.” But there's so much wealth of information there.Corey: Well, I wound up in postmortems later when I ran teams, I said, “Okay, so an engineer made a mistake.” It's like, “Well, hang on. There's always more to it than that”—Emily: Uh-huh.Corey: —“Because we don't hire malicious people and the people we have are competent for their role.” So, that goes a bit beyond that. We will never get into a scenario people do not make mistakes in a variety of different ways. So, that's not a helpful framing, it's a question of what—if they made a mistake, sure, what was it that brought them to that place because that's where it gets really interesting. The problem is when you're trying to figure out in a business context why a customer is super upset—if they're a major partner, for example—and there's a sense of, “All right, we're looking for a sacrificial lamb or someone that we can blame for this because we tend to think in relatively straight lines.”And in those scenarios, often, a nuanced understanding of the systemic failure modes within your organization that might wind up being useful in the mid to long-term are not helpful for the crisis there. So, trying to stuff too much into a given incident response might be a symptom there. I'm thinking of one or two incidents in the course of my later career that really had that stink to them, for lack of a better term. What's your take on the idea?Emily: I've been in a lot of incidents where it's the desire to be able to point and say a person made this mistake is high, it's definitely something that the, “organization”—and I put the organization in quotes there—and say technical leadership, or maybe PR or the comms team said like, “We're going to say, like, a person made this mistake,” when in reality, I mean, nine times out of ten, calling it a mistake is hindsight, right? Usually people—sometimes we know that we make a mistake and it's the recovery from that, that is response. But a lot of times we are making an informed decision, you know? An engineer has the information that they have available to them at the time and they're making an informed decision, and oh, no [laugh], it does not go as we planned, things in the system that we didn't fully understand are coexisting, it's a perfect storm of these events in order to lead to impact to this important customer.For me, I've been customer-facing for a very long time and I feel like from my observation, customers tend to—like if you say, like, “This person did something wrong,” versus, “We learned more about how the system works together and we understand how these kind of different pieces and mechanisms within our system are not necessarily single points of failure, but points at which they interact that we didn't understand could cause impact before, and now we have a better understanding of how our system works and we're making some changes to some pieces,” I feel like personally, as someone who has had to say that kind of stuff to customers a thousand times, saying, “It was a person who did this thing,” it shows so much less understanding of the event and understanding of the system than actually talking through the different components and different kind of contributing factors that were wrong. So, I feel like there's a lot of growth that we as an industry can could go from blaming things on an intern to actually saying, “No, we invested time and understanding how a single person could perform these actions that would lead to this impact, and now we have a deeper understanding of our system,” is in my opinion, builds a little bit more confidence from the customer side.Corey: This episode is sponsored in part by Honeycomb. I'm not going to dance around the problem. Your. Engineers. Are. Burned. Out. They're tired from pagers waking them up at 2 am for something that could have waited until after their morning coffee. They're fed up with relying on two or three different “monitoring tools” that still require them to manually trudge through logs to decipher what might be wrong. Simply put, there's a better way. Observability tools like Honeycomb show you the patterns and outliers of how users experience your code in complex and unpredictable environments so you can spend less time firefighting and more time innovating. It's great for your business, great for your engineers, and, most importantly, great for your customers. Try FREE today at honeycomb.io/screaminginthecloud. That's honeycomb.io/screaminginthecloud.Corey: I think so much of this is—I mean, it gets back to your question to me that I sort of dodged was I willing to talk about how my emotional state in these moments? And yeah, I was visibly sweating and very nervous and I've always been relatively okay with calling out the fact that I'm not in a great place at the moment, and I'm panicking. And it wasn't helped in some cases by, in those early days, the CEO of the company standing over my shoulder, coming down from the upstairs building to know what was going on, and everything had broken. And in that case, I was only coming in to do mop-up I wasn't one of the factors contributing to this, at least not by a primary or secondary degree, and it still was incredibly stress-inducing. So, from that perspective, it feels odd.But you also talk about ‘we,' in the sense of as an industry, as a culture, and the rest. I'm going to push back on that a little bit because there are still companies today in the closing days of 2022 that are extraordinarily far behind where many of us are at the companies we work for. And they're still stuck in the relative Dark Ages technically, were, “Well, are VMs okay, or should we stay on bare metal?” Is still the era that they're in, let alone cloud, let alone containerization, let alone infrastructure as code, et cetera, et cetera. I'm unconvinced that they have meaningfully progressed on the interpersonal aspects of incident management when they've been effectively frozen in amber from a technical basis.Emily: Mmm, I don't think that's fair [laugh].Corey: No. Excellent. Let's talk about that.Emily: [laugh]. I think just because an organization is still, like, maybe in DCs and using hardware and maybe hasn't advanced so thoroughly within the technical aspect of things, that doesn't necessarily mean that they haven't adopted new—Corey: Ah, very fair. Let me add one point of clarification, then, on this because what I'm talking about here is the fact there are companies who are that far behind on a technical basis, they are not necessarily one and the same, too—Emily: Correct.Corey: Because you're using older technology, that means your processes are stuck in the past, too.Emily: Right.Corey: But rather, just as there are companies that are anxious on the technology basis, there are also companies who will be 20 years behind in learnings—Emily: Yes.Corey: —compared to how the more progressive folks have already internalized some of these things ages ago. Blamelessness is still in the future for them. They haven't gotten there yet.Emily: I mean, yeah, there's still places that are doing root cause analysis, that are doing the five whys. And I think that we're doing our best [laugh]. I mean, I think it really takes—that's a cultural change. A lot of the actual change in approach of incident analysis and incident response is a cultural change. And I can speak from firsthand experience that that's really hard to do, especially from the inside it's very hard to do.So luckily, with the role that I'm in now at Jeli.io, I get to kind of support those folks who are trying to champion a change like that internally. And right now, my perspective is just trying to generate as much material for those folks to send internally, to say like, “Hey, there's a better way. Hey, there's a different approach for this that can maybe get us around these things that are difficult.” I do think that there's this tendency—and I've used this analogy before—is for us to think that our junk drawers are better than somebody else's junk drawers.I see an organization as just a junk drawer, a drawer full of weird odds and ends and spilled glue and, like, a broken box of tacks. And when you pull out somebody else's junk drawer, you're like, “This is a mess. This is an absolute mess. How can anyone live like this?” But when you pull out your own junk drawer, like, I know there are 17 rubber bands in this drawer, somehow. I am going to just completely rifle through this drawer until I find those things that I know are in here.Just a difference of knowing where our mess is, knowing where the bodies are buried, or the skeletons are in each closet, whatever analogy works best. But I think that some organizations have this thought process that—by organizations, I mean, executive leadership organizations are not an entity with an opinion, they're made up of a bunch of individuals doing [laugh] the work that they need to do—but they think that their problems are harder or more unique than at other organizations. And so, it's a lot harder to kind of help them see that, yes, there is a very unique situation, the way that your people work together with their technology is unique to every single different organization, but it's not that those problems cannot be solved in new and different ways. Just because we've always done something in this way does not mean that is the way that is serving us the best in this moment. So, we can experiment and we can make some changes.Especially with process, especially with the human aspect of things of how we talk to each other during incidents and how we communicate externally during incidents. Those aren't hard-coded. We don't have to do a bunch of code reviews and make sure it's working with existing integrations to be able to make those changes. We can experiment with that kind of stuff and I really would like to try to encourage folks to do that even though it seems scary because incidents are… [unintelligible 00:24:33] people think they're scary. They're not. They're [unintelligible 00:24:35].Corey: They seem to be. For a lot of folks, they are. Let's not be too dismissive on that.Emily: But we were both talking about panic [laugh] and the panic that we have felt during incidents. And I don't want to dismiss that and say that it's not real. But I also think that we feel that way because we're worried about how we're going to be judged for our involvement in them. We're panicking because, “Oh no, we have contributed to this in some way, and the fact that I don't know what to do, or the fact that I did something is going to reflect poorly on me, or maybe I'm going to get fired.” And I think that the panic associated with incidents also very often has to do with the environment in which you are experiencing that incident and how that is going to be accepted and discussed. Are you going to be blamed regardless of how, quote-unquote, “Blameless,” your organization is?Corey: I wish there was a better awareness of a lot of these things, but I don't think that we are at a point yet where we're there.Emily: No.Corey: How does this map what you do, day-to-day over at Jeli.io?Emily: It is what I do every single day. So, I mean, I do a ton of different things. We're a very small startup, so I'm doing a lot, but the main thing that I'm doing is working with our customers to tackle these hurdles within each of their organizations. Our customers vary from very small organizations to very, very large organizations, and working with them to find how to make movement, how to sell this internally, sell this idea of let's talk about our incidents a little bit differently, let's maybe dial back some of the hard-coded automation that we're doing around response and change that to speaking to each other, as opposed to, we need 11 emails sent automatically upon the creation of an incident that will automatically map to these three PagerDuty schedules, and a lot more of it can be us working through the issue together and then talking about it afterwards, not just in reference to the root cause, but in how we interfaced: how did it go, how did response work, as well as how did we solve the problem of the technical problem that occurred?So, I kind of pinch myself. I feel very lucky that I get to work with a lot of different companies to understand these human aspects and the technical aspects of how to do these experiments and make some change within organizations to help make incidents easier. That's the whole feeling, right? We were talking about the panic. It doesn't need to be as hard as it feels, sometimes. And I think that it can be easier than we let ourselves think.Corey: That's a good way of framing it. It just feels on so many levels like this is one of the hardest areas to build a company in because you're not really talking about fixing technical, broken systems out there. You're talking about solving people problems. And I have some software that solves your people problems, I'm not sure if that's ever been true.Emily: Yeah, it's not the software that's going to solve the people problems. It's building the skills. A lot of what we do is we have software that helps you immensely in the analysis process and build out a story as opposed to just building out a timeline, trying to tell, kind of, the narrative of the incident because that's what works. Like anthropologically, we've been conveying information through folklore, through tales, telling tales of things that happened in order to help teach people lessons is kind of how we've—oral history has worked for [laugh] thousands of years. And we aren't better than that just because we have technology, so it's really about helping people uncover those things by using the technology we have: pulling in Slack transcripts, and PagerDuty alerts, and Zoom transcripts, and all of this different information that we have available to us, and help people tell that story and convey that story to the folks that were involved in it, as well as other peoples in your organization who might have similar things come up in the future.And that's how we learn. That's how we teach. But that's what we learn. I feel like there's a big difference—I'm understanding, there's a big difference between being taught something and learning something because you usually have to earn that knowledge when you learn it. You can be taught something a thousand times and then you've learned that once.And so, we're trying to use those moments that we actually learn it where we earn that hard-earned information through an incident and tell those stories and convey that, and our team—the solutions team—is in there, helping people build these skills, teaching people how to talk to each other [laugh] and really find out this information during incidents, not after them.Corey: I really want to thank you for being as generous with your time as you have been. And if people want to learn more, where's the best place to find you?Emily: Oh. I was going to say Twitter, but… [laugh].Corey: Yeah, that's a big open question these days, isn't it? Assuming it's still there by the time this episode airs, it might be a few days between now and then. Where should they find you on Twitter, with a big asterisk next to it?Emily: It's at @themortalemily. Which, I started this by saying I like mess and I'm someone who loves incidents, so I'll be on Twitter [laugh].Corey: We're there to watch it all burn.Emily: Oh, I feel terrible saying that. Actually, if any Twitter engineers are listening to this, someone is found that the TLS certificate is going to expire at the end of this year. Please check Twitter for where that TLS certificate lives so that you all can renew that. Also, Jeli.io, we have a blog that a lot of us write, our solutions team, we—and honestly a lot of us, we tend to hire folks who have a lot of experience in incident response and analysis.I've never been a solutions engineer before in my life, but I've done a lot of incident response. So, we put up a lot of stuff and our goal is to build resources that are available to folks who are trying to make these changes happen, who are in those organizations where they're still doing five whys, and RCAs, and are trying to convince people to experiment and change. We have our Howie Guide, which is available for free. It's ‘How We Got Here' which is, like, a full, free incident analysis guide and a lot of cool blogs and stuff there. So, if you can't find me on Twitter, we're writing… things… there [laugh].Corey: We will, of course, put links to all of that in the [show notes 00:30:46]. Thank you so much for your time today. It's appreciated.Emily: Thank you, Corey. This was great.Corey: Emily Ruppe, solutions engineer at Jeli.io. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this episode, please leave a five-star review on your podcast platform of choice, along with an angry comment talking about how we've gotten it wrong and it is always someone's fault.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

AWS Morning Brief
Timecode Burn-In, Employee Burn-Out

AWS Morning Brief

Play Episode Listen Later Jan 30, 2023 6:14


Links: AWS Purity Test  Amazon Detective adds Amazon VPC Flow Logs visualizations for Amazon EKS workloads  AWS Elemental MediaLive adds timecode burn-in  AWS Pricing Calculator now supports optimized pricing estimation for EC2 Dedicated Hosts  Announcing Porting Advisor for Graviton  Now Open — AWS Asia Pacific (Melbourne) Region in Australia  Amazon OpenSearch Serverless is now generally available! AWS Lambda: Resilience under-the-hood  VPC Routing Enhancements and GWLB Deployment Patterns Introducing AWS Lambda runtime management controls 

Screaming in the Cloud
Saving the World though Cloud Sustainability with Aerin Booth

Screaming in the Cloud

Play Episode Listen Later Jan 26, 2023 35:56


About AerinAerin is a Cloud Sustainability Advocate and neurodiverse founder in tech on a mission to help developers understand the real impact that cloud computing has on the world and reduce their carbon emissions in the cloud. Did you know that internet and cloud computing contribute over 4% of annual carbon emissions? Twice that of the airline industry!Aerin also hosts "Public Cloud for Public Good," a podcast targeted towards developers and senior leaders in tech. Every episode, they also donate £500 to charities and highlight organisations that are working towards a better future. Listen and learn how you can contribute towards making the world a better place through the use of public cloud services.Links Referenced: Twitter: https://twitter.com/aerincloud LinkedIn: https://www.linkedin.com/in/aerinb/ Public Cloud for Public Good: https://publicgood.cloud/ duckbillgroup.com: https://duckbillgroup.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Uptycs, because they believe that many of you are looking to bolster your security posture with CNAPP and XDR solutions. They offer both cloud and endpoint security in a single UI and data model. Listeners can get Uptycs for up to 1,000 assets through the end of 2023 (that is next year) for $1. But this offer is only available for a limited time on UptycsSecretMenu.com. That's U-P-T-Y-C-S Secret Menu dot com.Corey: Cloud native just means you've got more components or microservices than anyone (even a mythical 10x engineer) can keep track of. With OpsLevel, you can build a catalog in minutes and forget needing that mythical 10x engineer. Now, you'll have a 10x service catalog to accompany your 10x service count. Visit OpsLevel.com to learn how easy it is to build and manage your service catalog. Connect to your git provider and you're off to the races with service import, repo ownership, tech docs, and more. Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn and I am joined what feels like roughly a year later by a returning guest, Aerin Booth. How long have you been?Aerin: I've been really great. You know, it's been a journey of a year, I think, since we sort of did this podcast even, like, you know, a year and a bit since we met, and, like, I'm doing so much and I think it's doing, like, a big difference. And yeah, I can't wait for everything else. It's just yeah, a lot of work right now, but I'm really enjoying it. So, I'm really well, thank you.Corey: Normally, I like to introduce people by giving their job title and the company in which they work because again, that's a big deal for an awful lot of people. But a year ago, you were independent. And now you still are. And back when I was doing my own consulting independently, it felt very weird to do that, so I'm just going to call you the Ted Lasso of cloud at this point.Aerin: [laugh].Corey: You've got the mustache, you've got the, I would say, obnoxiously sunny disposition. It's really, there's a certain affinity right there. So, there we go. I feel like that is the best descriptor for what you have become.Aerin: I—do know what, I only just watched Ted Lasso over Christmas and I really found it so motivational in some ways because wow, like, it's not just who we'd want to be in a lot of ways? And I think, you know, for the work that I do, which is focused on sustainability, like, I want to present a positive future, I want to encourage people to achieve more and collaborate, and yeah, basically work on all these problems that we need to be worked on. And yeah, I think that's [laugh] [crosstalk 00:02:02]—Corey: One of the challenges of talking to you sometimes is you talk about these depressing things, but there's such a—you take such an upbeat, positive approach to it that I, by comparison, invariably come away from our conversations during, like, I'm Surly McBastard over here.Aerin: [laugh]. Yeah, you can be the bad cop of cloud computing and I'll try and be the good cop. Do you know, you say that the stuff I talk about is depressing, and it is true and people do worry about climate change. Like I did an online conference recently, it's focused on FinOps, and we had a survey, “Do you worry about climate change?” 70% of the people that responded said they worry about it.So, we all know, it's something we worry about and we care about. And, you know, I guess what I'm really trying to do is encourage people to care a bit more and start taking action and look after yourself. Because you know, when you do start taking action towards it, when you join those communities that are also working on it, it is good, it is helpful. And, you know, I've gone through some ups and downs and some of this, like, just do I throw in the towel because no one cares about it? Like, we spoke last year; I had attended re:Invent for the first time.This year, I was able to speak at re:Invent. So, I did a talk on being ethical in tech. And it was fun, it was good. I enjoyed what I delivered, but I had about 35 people sign up to that. I'm pretty sure if I talked about serverless or the next Web3 blockchain product, I would have got hundreds more. But what I'm starting to realize is that I think people just aren't ready to, sort of, want to do this yet. And yeah, I'm hoping that'll change.Corey: Let's first talk about, I guess, something that is more temporally pressing than some other things. Not that it is more important than climate change, mind you, but it feels like it's on a shorter timeline which is, relatively soon after this recording, there is a conference that you are kicking off called The State of Open. Ajar, Aerin. The State of Open is ajar. What is this conference? Is it in person? Is it virtual? Is it something where you and three friends are going to show up and basically talk to each other? How big? How small? What is it? What's it about? Tell me more, please. I'm riveted.Aerin: So, State of Open conference is a conference that's been in the works now for maybe about two weeks, a little bit longer in the planning, but the work we've been putting in over the last two weeks. It'll be on the seventh and eighth of February in London as a physical event in the QEII Conference Centre, but it will also be available online. And you know, when we talk about the State of Open, it's that question: what is the State of Open? The state of open-source, the state of open hardware, and the state of open data. And it is going to be probably the first and hopefully the biggest open-source conference in the UK.We already have over 100 confirmed guest speakers from Jimmy Wales, the co-founder of Wikipedia, to many of our great guests and headliners who haven't even announced yet for the plenary. So, I'm really excited. And the reason why I wanted to get involved with this is because one of the coolest things about this conference—compared to some others like re:Invent, for example—is that sustainability and diversity run through every single thing that we do. So, as the content director, I reviewed every single CFP for both of these things. I mean, you couldn't get a better person than someone like me, who's the queer person who won't shut up about sustainability to sort of do this thing.So, you know, I looked after those scorings for the CFPs in support of the CFP chairs. And now, as I'm working with those individual speakers on their content and making sure that diversity is included in the content. It's not just the diversity of the speaker, for example it's, who were the other people whose voice you're raising? What other people if you worked on this? Are there anyone that you've mentored, like, you know, actually, you know, let's have this as a wider conversation?Corey: Thank God. I thought you were about to say diversity of thought, and I was about to reach through the screen to strangle you.Aerin: [laugh]. No, no. I mean, we're doing really well, so of the announced speakers online, we are 40% non-male and about 18% non-white, which to be honest, for a fair sheer conference, when we didn't really do that much to specifically call this out, but I would probably raise this to Amanda Brock, who is the CEO of OpenUK, you know, she has built a community in the UK and around the world over the last few years which has been putting women forward and building these links. And that's why we've had such a great response for our first-year conferences, the work she's put in. It's hard.Like, this isn't easy. You know, we've had to do a lot of work to make sure that it is representative, at least better than other conferences, at least. So, I'm really excited. And like, there's so much, like, open-source is probably going to be the thing that saves the world. If we're going to end up looking at two different futures with monopolies and closed systems and all the money going towards cloud providers versus a fair and equitable society, open-source is the thing that's going to get us closer to that. So yeah, this conference will be a great event.Corey: Is it all in person? Is it being live-streamed as well? What is the deal here?Aerin: So, in person, we have loads of different things going on, but what will be streamed online if you sign up for virtual ticket is five different tracks. So, our platform engineering track, our security track, government law and policy, open data, and open hardware. And of course, the keynote and plenaries. But one of the things I'm also really proud about this conference is that we're really focusing on the developer experience, like, you know, what is your experience at the conference? So, we also have an unconference, we have a sub-conference run by Sustain OSS focused on workshops related to climate change and sustainability.We have loads of developer experience halls in the event itself. And throughout the day, over the two days, we have two one-hour blocks with no speaking content at all so that we can really make sure that people have that hardware track and are out there meeting each other and having a good time. And obviously, of course, like any good conference, the all-hands party on the first night. So, it really is a conference that's doing things differently from diversity to sustainability to that experience. So, it's awesome.Corey: One of the challenges that I've seen historically around things aiming at the idea of open conferences—and when we talk open-source, et cetera, et cetera—open' seems like it is a direction parallel to, we haven't any money, where it's, “Yes, we're a free software foundation,” and it turns out conferences themselves are not free. And you wind up with a whole bunch of folks showing up to it who are, in many cases, around the fringes of things. There are individual hobbyists who are very passionate about a thing but do not have the position in the corporate world. I'm looking through the lengthy list of speakers you have here and that is very much not this. These are serious people at serious companies. Not that there are not folks who are individual practitioners and passionate advocates and hobbyists than the rest. This is, by virtually any way you look at it, a remarkably diverse conference.Aerin: Mmm. You know, you are right about, like, that problem in open-source. It's like, you know, we look at open and whether we want to do open and we just go, “Well, it won't make me any money. I can't do that. I don't have the time. I need to bring in some money.”And one of the really unique things, again, about this conference is—I have not even mentioned it yet—we have an entrepreneurship room. So, we have 20 tables filled with entrepreneurs and CEOs and founders of open-source companies throughout the two days where you can book in time to sit at that table and have conversations with them. Ask them the questions that you want to ask about, whether it's something that you want to work on, or a company you want to found, and you'll be able to get that time. I had a very similar experience in some ways. It was re:Invent.I was a peer talk expert and you know, I had 15 or so conversations with some really interesting people just because they were able put that time in and they were able to find me on the website. So, that's something we are replicating to get those 20 also entrepreneurs and co-founders out to everyone else. They want to be able to help you and support you.Corey: That is an excellent segue if I do say so myself. Let's talk about re:Invent. It's the one time of the year you and I get to spend time in the same room. One thing that I got wrong is that I overbooked myself as I often do, and I didn't have time to do anything on their peer talk expert program, which is, you more or less a way that any rando can book time to sit down and chat with you. Now, in my case, I have assassination concerns because it turns out Amazon employees can read that thing too and some of them might work on billing. One wonders.So yeah, I have to be a little careful for personal reasons but for most people, it's a non-issue. I didn't get as much time as I wanted to talk to folks in the community. That is not going to repeat itself at the end of this year. But what was your take on re:Invent, because I was in meetings for most of them?Aerin: So, comparing this re:Invent to the re:Invent I went to, my first re:Invent when we met in 2021, you know, that was the re:Invent that inspired me to get into sustainability. They'd announced stuff to do with the shared responsibility model. A few months later, they released their carbon calculator, and I was like, “Yeah, this is the problem. This is the thing I want to work on and it will make me happy.” And a lot of that goes into, you know, finding a passion that keeps me motivated when things aren't that great.When maybe not a lot of money is coming in, at least I know, I'm doing everything I can to help save the world. So, re:Invent 2021 really inspired me to get involved with sustainability. When I look at re:Invent 2022, you might have Adam Selipsky on the main stage saying that sustainability is the problem of our generation, but that is just talk and bluster compared to what they were putting out in terms of content and their experience of, like, let's say the sustainability—I don't know what to call it—tiny little square in the back of the MGM Grand compared to the paid hall in the expo. Like, you know, that's the sort of thing where you can already see the prioritization of money. Let's put the biggest sponsors and all the money that we can bring it in the big hall where everyone is, and then put the thing we care about the most, apparently—sustainability—in the back of the MGM.And that in itself was annoying, but then you get there in the content, and it was like a massive Rivian van, like, an advert for, “Oh, Amazon has done all this to electrify Rivian and deliver you Prime.” But where was the people working on sustainability in the cloud? You know, we had a couple of teams who were talking about the customer carbon footprint tool, but there was just not much. And I spoke to a lot of people and they were saying similar things, like, “Where are the announcements? Where are the actual interesting things?” Rather than just—which is kind of what I'm starting to realize is that a lot of the conversations about sustainability is about selling yourself as sustainable.Use me rather than my competitors because we're 88% more, kind of, carbon neutral when it comes to traditional data centers, not because we are really going to solve these problems. And not to say that Amazon isn't doing innovative, amazing things that no one else can't do, because that is true, and cloud as part of the solution, but you know, sustainability shouldn't be about making more sales and growing your business, it should be about making the world a better place, not just in terms of carbon emissions, but you know, our life, the tech that we can access. Three billion people on this planet have never accessed the internet. And as we continue to grow all of our services like AI and machine learning and new Web3, bloody managed services come online, that's going to be more carbon, more compute power going towards the already rich and the already westernized people, rather than solving the problems we need to solve in the face of climate change.So, I was a little bit disappointed. And I did put a tweet thread out about it afterwards. And I just hope it can be different next year and I hope more people will start to ask for this. And that also what I'm starting to realize is that until more Amazon customers put this as their number one priority and say, “I'm not going to do business with you because of this issue,” or, you know, “This is what we really care about,” they're not going to make a change. Unless it starts to impact their bottom lines and people start to choose other cloud providers, they're not going to prioritize it.And I think up until this point, we're not seeing that from customers. We're kind of getting some people like me shouting about it, but across the board, sustainability isn't the number one priority right now. It's, like what Amazon says, security or resiliency or something else.Corey: And I think that, at least from where I set, the challenge is that if you asked me what I got out of re:Invent, and what the conversations I had—going into it, what are my expectations, and what do I hope to get and how's it going to end up, and then you ask you that same question—though maybe you are a poor example of this—and then you ask someone who works out as an engineer at a company that uses AWS and their two or three years into their career, why don't you talk to a manager or director or someone else? And the problem is if you start polling the entire audience, you'll find that this becomes—you're going to wind up with 20 different answers, at least. The conference doesn't seem like it has any idea of what it wants to be and to whom and in that vacuum, it tries to be all things to all people. And surprise, just like the shooting multifunction printer some of us have in our homes, it doesn't do well with any of those things because it's trying to stand in too many worlds at the same time.Aerin: You know, let's not, like, look at this from a way that you know, re:Invent is crap and, like, do all the work that everyone puts it is wasted because it is a really great event for a lot of different things for a lot of different people. And to be honest, the work that the Amazon staff put into it is pretty out of this world. I feel sorry though because you know, the rush for AWS sell more and do this massive event, they put people through the grinder. And I feel like, I don't know, we could see the cracks in some of that, the way that works. But, you know, there's so many people that I speak to who were like, “Yeah, I'm definitely not going again. I'm not even going to go anywhere near submitting a talk.”And, sort of, the thing is, like, I can imagine if the conference was something different; it was focused at sustainability at number one, it was about making the world a better place from everything that they do, it was about bringing diverse communities together. Like, you know, bringing these things up the list would make the whole thing a lot better. And to be honest, it would probably make it a lot more enjoyable [laugh] for the Amazon staff who end up talking at it. Because, you know, I guess it can feel a bit soulless over time is all you're doing is making money for someone else and selling more things. And, yeah, I think there's a lot more… different things we can do and a lot more things we can talk about if people just start to talk about, like you know, if you care about this as well and you work at Amazon, then start saying that as well.It'll really make a difference if you say we want re:Invent to look different. I mean, even Amazon staff, [laugh] and we've not even mentioned this one because I got Covid straight after re:Invent, nine days and staring at a wall in hotel room in Vegas was not my idea of a good time post-conference. So, that was a horrible, horrible experience. But, you know, I've had people call it re:Infect. Like, where are the Covid support?Like, there was hardly any conversation about that. It was sort of like, “Don't mention it because oh, s”—whatever else. But imagine if you just did something a little bit differently to look like you care about your customers. Just say, “We recommend people mask or take a test,” or even provide tests and masks. Like, even if it's not mandatory, they could have done a lot more to make it safer for everyone. Because, yeah, imagine having the reputation of re:Infect rather than re:Invent?Corey: I can only imagine how that would play out.Aerin: Only imagine.Corey: Yeah, it's it feels like we're all collectively decided to pretend that the pandemic is over. Because yeah, that's a bummer. I don't want to think about it. You know, kind of like we approach climate change.Aerin: Yeah. At the end of the day, like, and I keep coming across this more and more, you know, my thinking has changed over the last year because, like, you know, initially it was like a hyperactive puppy. Why are we caring about this? Like, yeah, if I say it, people will come, but the reality is, we have to blinker ourselves in order to deal with a lot of this stuff. We can't always worry about all of this stuff all of the time. And that's fine. That's acceptable. We do that in so many different parts of our life.But there comes to a point when you kind of think, “How much do I care about this?” And for a lot of people, it's because they have kids. Like, anyone who has kids right now must have to think, “Wow, what's the future going to look like?” And if you worry about what the future is going to look like, make sure you're taking steps to make the world a better place and make it the future you want it to look like. You know, I made the decision a long time ago not to have kids because I don't think I'd want to bring anyone into the world on what it might actually end up being, but you know, when I speak to people who are older in the 60s and they're like, “Oh, you've got 100 years. You don't need to worry about it.” Like, “Maybe you can say that because you're closer to dying than I am.” But yeah, I have to worry about this now because I'll still be eighty when all this shit is kicking off [laugh].Corey: This episode is sponsored in part by our friends at Strata. Are you struggling to keep up with the demands of managing and securing identity in your distributed enterprise IT environment? You're not alone, but you shouldn't let that hold you back. With Strata's Identity Orchestration Platform, you can secure all your apps on any cloud with any IDP, so your IT teams will never have to refactor for identity again. Imagine modernizing app identity in minutes instead of months, deploying passwordless on any tricky old app, and achieving business resilience with always-on identity, all from one lightweight and flexible platform.Want to see it in action? Share your identity challenge with them on a discovery call and they'll hook you up with a complimentary pair of AirPods Pro. Don't miss out, visit Strata.io/ScreamingCloud. That's Strata dot io slash ScreamingCloud.Corey: That I guess is one of the big fears I have—and I think it's somewhat unfounded—is that every year starts to look too much like the year before it. Because it's one of those ideas where we start to see the pace of innovation is slowing at AWS—and I'm not saying that to piss people at Amazon off and have them come after me with pitchforks and torches again—but they're not launching new services at the rate they once did, which is good for customers, but it starts to feel like oh, have we hit peak cloud this is what it's going to look like? Absolutely not. I don't get the sense that the world is like, “Well, everything's been invented. Time to shut down the patent office,” anytime soon.And in the short term, it feels like oh, there's not a lot exciting going on, but you look back the last five years even and look at how far we've come even in that period of time and—what is it? “The days are long, but the years are short.” It becomes a very macro thing of as things ebb and flow, you start to see the differences but the micro basis on a year-to-year perspective, it seems harder to detect. So longer term, I think we're going to see what the story looks like. And it's going to be satisfying one. Just right now, it's like, well, this wasn't as entertaining as I would have hoped, so I'm annoyed. Which I am because it wasn't, but that's not the biggest problem in the world.Aerin: It's not. And, you know, you look at okay, cool, there wasn't all these new flashy services. There was a few things are announced, I mean, hopefully that are going to contribute towards climate change. One of them is called AWS Supply Chain. And the irony of seeing sort of like AWS Supply Chain where a company that already has issues with data and conversations around competition, saying to everyone, “Hey, trust us and give all of your supply chain information and put it into one of our AWS products,” while at the same time their customer carbon footprint tool won't even show the full scope for their emissions of their own supply chain is not lost on me.And you do say, “Maybe we should start seeing things at a macro level,” but unless Amazon and other cloud hyperscalers start pulling the finger out and showing us how they have got a vision between now and 2040, and now in 2050, of how they're going to get there, it kind of just feels like they're saying, “It'll all be fine as long as we continue to grow, as long as we keep sucking up the market.” And, you know, an interesting thing that just kicked off in the UK back in November was the Competition and Markets Authority have started an investigation into the cloud providers on how they are basically sucking up all these markets, and how the growth of things that are not hyperscale is going. So, in the UK, the percentage of cloud has obviously gone up—more and more cloud spending has gone up—but kind of usage across non-hyperscalers has gone down over that same period. And they really are at risk of sucking up the world. Like, I have got involved in a lot of different things.I'm an AWS community builder; like, I do promote AWS. And, you know, the reason why I promote cloud, for example is serverless. We need serverless as the way we run our IT because that's the only way we'll do things like time shifting or demand shifting. So, when we look at renewable energy on the grid if that really high, the wind is blowing and the sun is shining, we want more workloads to be running then and when they're tiny, and they're [unintelligible 00:21:03], and what's the call it serverless generally, uh—Corey: Hype?Aerin: Function as a Code?Corey: Function—yeah, Function as a Service and all kinds of other nonsense. But I have to ask, when you're talking about serverless, in this context, is a necessary prerequisite of serverless that scale to zero when it's [unintelligible 00:21:19].Aerin: [laugh]. I kind of go back to marketing. What Amazon releasing these days when it relates to serverless that isn't just marketing and saying, “Oh, it's serverless.” Because yeah, there was a few products this year that is not scaled to zero is it? It's a 100-pound minimum. And when you're looking at number of accounts that you have, that can add up really quickly and it excludes people from using it.Corey: It's worse than that because it's not number of accounts. I consider DynamoDB to be serverless, by any definition of the term. Because it is. And what I like about it is I can have a separate table for every developer, for every service or microservice or project that they have, and in fact, each branch can have its own stuff like that. I look at some of the stuff that I build with multi-branch testing and whatnot, and, “Oh, wow. That would cost more than the engineer if they were to do that with some of the serverless offerings that AWS has put out.”Which makes that entire philosophy a complete non-starter, which means that invariably as soon as you start developing down that path, you are making significant trade-offs. That's just from a economics slash developer ergonomics slash best practices point of view. But there's a sustainability story to it as well.Aerin: Yeah. I mean, this sustainability thing is like, if you're not going to encourage this new way of working, like, if you're not going to move everyone to this point of view and this is how we need to do things, then you kind of just propagating the old world, putting it into your data center. For every managed service that VMware migrated piece of crap, just that land in the cloud, it's not making a real difference in the world because that's still going to exist. And we mentioned this just before the podcast and, you know, a lot of focus these days and for a lot of people is, “Okay, green energy is the problem. We need to solve green energy.”And Amazon is the biggest purchaser of power purchase agreements in renewable energy around the world, more than most governments. Or I think that the biggest corporate purchaser of it anyway. And that all might sound great, like, “Oh, the cloud is going to solve this problem for me and Amazon is going to solve it for me even better because they're bigger.” But at the end of the day, when we think about a data center, it exists in the real world.It's made of concrete. You know, when you pour concrete and when you make concrete, it releases CO2. It's got racks of servers that all are running. So, those individual servers had to be made by whoever it is in Asia or mined from rare earth metals and end up in the supply chain and then transported into the data centers in us-east-1. And then things go wrong. You have to repair you have to replace and you have to maintain them.Unless we get these circular economies going in a closed system, we can't just continue to grow like this. Because carbon emissions related to Scope 3, all those things I've just been talking about, basically anything that isn't the energy, is about 80 to 90% of all the carbon emissions. So, when Amazon says, “Oh, we're going to go green and get energy done by 2030”—which is seven years away—they've then got ten years to solve 90% of the problem. And we cannot all just continue to grow and think of tech as neutral and better for the world if we still got that 90% problem, which we do right now. And it really frustrates me when you look at the world and the way we've jumped on technology just go on, “Oh, it must be good.”Like Bitcoin, for example. Bitcoin has released 200 million metric tons of CO2 since its inception. And for something that is basically a glorified Ponzi scheme, I can't see how that is making the world a better place. So, when cloud providers are making managed services for Web3 and for blockchain, and they're selling more and more AI and machine learning, basically so they can keep on selling GPU access, I do worry about whether our path to infinite growth with all of these hyperscalers is probably the wrong way of looking at things. So, linking back to, you know, the conference, open-source and, you know, thinking about things differently is really important in tech right now.And not just for your own well-being and being able to sleep at night, but this is how we're going to solve our problems. When all companies on the planet want people to be sustainable and we have to start tackling this because there's a financial cost related to it, then you're going to be in the vogue. If you're really good developer, thinking about things differently can be efficient, then yeah, you're the developer that's going to win in the future. You might be assisted by ChatGPT three or whatever else, but yeah, sustainability and efficiency can really be the number one priority because it's a win, win, win. We save the world, we make ourselves better, we sleep better at night, and you just become a better developer.I keep monologuing at this point, but you know, when it comes to stuff like games design, we look at things like Quake and Pokemon and all these things when there's like, “How did they get these amazing games and these amazing experiences in such small sizes,” they had boundaries. They had boundaries to innovate within because they had to. They couldn't release the game if they couldn't fit into the cartridge, therefore, they made it work. When the cloud is sold as infinitely scalable and horizontally scalable and no one needs to worry about this stuff because you can get your credit card out, people stop caring about being innovative and being more efficient. So yeah, let's get some more boundaries in the cloud.Corey: What I find that is super helpful, has been, like, if I can, like, descri—like, Instagram is down. Describe your lunch to me style meme description, like, the epic handshake where you have two people clasping hands, and one side is labeled in this case, ‘sustainability advocates,' and the other side should be labeled ‘cloud economists,' and in the middle, it's, “Turn that shit off.” Because it's not burning carbon if it's not running, and it's not costing you anything—ideally—if it's not running, so it's one of those ideas where we meet in the middle. And that's important, not just because it makes both of us independently happy because it's both good for the world and you'll get companies on board with this because, “Wait. We can do this thing and it saves us money?” Suddenly, you're getting them aligned because that is their religion.If companies could be said to have a religion, it is money. That's the way it works. So, you have to make it worth money for them to do the right thing or you're always going to be swimming upstream like a depressed salmon.Aerin: I mean, look at why [unintelligible 00:27:11] security is near the top: because there's so many big fines related to security breaches. It will cost them money not to be secure. Right now, it doesn't cost companies money to be inefficient or to release all this carbon, so they get away with it or they choose to do it. And I think that's going to change. We see in regulations across you're coming out.So, you know, if you work for a big multinational that operates in Europe, by next year, you'll have to report on all of your Scope 3 carbon emissions. If you're a customer of AWS right now, you have no ability to do that. So, you know, this is going to be crunch time over the next 18 months to two years for a lot of big businesses, for Amazon and the other hyperscalers, to really start demonstrating that they can do this. And I guess that's my big push. And, you know, I want to work with anyone, and it's funny because I have been running this business for about, you know, a couple of years now, it's been going really well, I did my podcast, I'm on this path.But I did, last year, take some time, and I applied into AWS. And you know, I was like, “Okay, maybe I'll apply for this big tech company and help Amazon out.” And because I'll take that salary and I'll do something really good with it afterwards, I'll do my time for three years and attend re:Invent and deliver 12 talks and never sleep, but you know, at the end of it, I'll say, “Okay, I've done that and now I can do something really good.” Unfortunately, I didn't get the role—or fortunately—but you know, when I applied for that role, what I said to them is, “I really care about sustainability. I want to make the world a better place. I want to help your customers be more sustainable.”And they didn't want me to join. So, I'm just going to continue doing that but from the outside. And whether that means working with politicians or developers or anyone else to try and make the world better and to kind of help fight against climate change, then, yeah, that's definitely what I'm doing.Corey: So, one last question before we wind up calling it an episode. How do we get there? What is the best next step that folks can take? Because it's easy to look at this as a grand problem and realize it's too big to solve. Well, great. You don't need to solve the entire problem. You need take the first step. What is that first step?Aerin: Individuals, I would say it's just realizing that you do care about it and you want to take action. And you're going to say to yourself, “Even if I do little things, I'm going to move forward towards that point.” So, if that is being a more sustainable engineer or getting more conversations about climate change or even just doing other things in your community to make the world a better place than it is, taking that action. But one thing that I can definitely help about and talk a bit more of is that at the conference itself, I'll be running a panel with some great experts called the, “Next Generation of Cloud Education.” So, I really think we need to—like I said earlier in the podcast—to think differently about the cloud and IT.So, I am doing this panel and I'm bringing together someone like Simon Wardley to help people do Wardley Mapping. Like, that is a tool that allows you to see the landscape that you're operating in. You know, if you use that sort of tool to understand the real-world impact of what you're doing, then you can start caring about it a bit more. I'm bringing in somebody called Anne Currie, who is a tech ethicist and speaker and lecturer, and she's actually written some [laugh] really great nonfiction books, which I'd recommend everyone reads. It starts with Utopia Five.And that's about asking, “Well, is this ethical? Can we continue to do these things?” Can't—talks about things about sustainability. If it's not sustainable for everyone, it's not ethical. So, when I mentioned 3 billion people currently don't use the internet, it's like, can we continue to just keep on doing things the same way?And then John Booth, who is a data center expert, to help us really understand what the reality is on the ground. What are these data centers really look like? And then Amanda Brock, from OpenUK in the conference will joining as well to talk about, kind of, open-source and how we can make the world kind of a better place by getting involved in these communities. So, that'll be a really great panel.But what I'm also doing is releasing this as an online course. So, for people who want to get involved, it will be very intimate, about 15 seats on each core, so three weeks for you to actually work and talk directly with some of these experts and me to figure out what you want to do in the world of climate change and how you can take those first steps. So, it'll be a journey that even starts with an ecotherapist to help us deal with climate grief and wonder about the things we can do as individuals to feel better ourselves and be happier. So, I think that'd be a really great thing for a lot of people. And, yeah, not only that, but… it'll be great for you, but it also goes towards making the world a better place.So, 50% of the course fees will be donated, 25%, to charity, and 25% supporting open-source projects. So, I think it kind of just win, win, win. And that's the story of sustainability in general. It's a win, win, win for everyone. If you start seeing the world through a lens of sustainability, you'll save money, you'll sleep better at night, you'll get involved with some really great communities, and meet some really great people who care about this as well. And yeah, it'll be a brighter future.Corey: If people want to learn more, where can they find you?Aerin: So, if you want to learn more about what I'm up to, I'm on Twitter under @aerincloud, that A-E-R-I-N cloud. And then you can also find me on LinkedIn. But I also run my own podcast that was inspired by Corey, called Public Cloud for Public Good talking about cloud sustainability and how to make the world a better place for the use of public cloud services.Corey: And we will, of course, put a link to that in the [show notes 00:32:32]. Thank you so much for your time. I appreciate it, as always.Aerin: Thank you.Corey: Aerin Booth, the Ted Lasso of cloud. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this episode, please leave a five-star review on your podcast platform of choice, along with an angry and insulting comment that I will immediately scale to zero in true serverless fashion.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

AWS Morning Brief
Aspirational Audit Logs

AWS Morning Brief

Play Episode Listen Later Jan 26, 2023 5:25


Links: Datadog reports that an undocumented API allowed CloudTrail bypass MailChimp was breached and had customer data exposed Folks can use GitHub Codespaces to host and deliver malware. How to revoke federated users' active AWS sessions The worst backup software known to humankind

Screaming in the Cloud
Solving for Cloud Security at Scale with Chris Farris

Screaming in the Cloud

Play Episode Listen Later Jan 24, 2023 35:39


About Chris Chris Farris has been in the IT field since 1994 primarily focused on Linux, networking, and security. For the last 8 years, he has focused on public-cloud and public-cloud security. He has built and evolved multiple cloud security programs for major media companies, focusing on enabling the broader security team's objectives of secure design, incident response and vulnerability management. He has developed cloud security standards and baselines to provide risk-based guidance to development and operations teams. As a practitioner, he's architected and implemented multiple serverless and traditional cloud applications focused on deployment, security, operations, and financial modeling.Chris now does cloud security research for Turbot and evangelizes for the open source tool Steampipe. He is one if the organizers of the fwd:cloudsec conference (https://fwdcloudsec.org) and has given multiple presentations at AWS conferences and BSides events.When not building things with AWS's building blocks, he enjoys building Legos with his kid and figuring out what interesting part of the globe to travel to next. He opines on security and technology on Twitter and his website https://www.chrisfarris.comLinks Referenced: Turbot: https://turbot.com/ fwd:cloudsec: https://fwdcloudsec.org/ Steampipe: https://steampipe.io/ Steampipe block: https://steampipe.io/blog TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Tailscale SSH is a new, and arguably better way to SSH. Once you've enabled Tailscale SSH on your server and user devices, Tailscale takes care of the rest. So you don't need to manage, rotate, or distribute new SSH keys every time someone on your team leaves. Pretty cool, right? Tailscale gives each device in your network a node key to connect to your VPN, and uses that same key for SSH authorization and encryption. So basically you're SSHing the same way that you're already managing your network.So what's the benefit? Well, built-in key rotation, the ability to manage permissions as code, connectivity between any two devices, and reduced latency. You can even ask users to re-authenticate SSH connections for that extra bit of security to keep the compliance folks happy. Try Tailscale now - it's free forever for personal use.Corey: This episode is sponsored by our friends at Logicworks. Getting to the cloud is challenging enough for many places, especially maintaining security, resiliency, cost control, agility, etc, etc, etc. Things break, configurations drift, technology advances, and organizations, frankly, need to evolve. How can you get to the cloud faster and ensure you have the right team in place to maintain success over time? Day 2 matters. Work with a partner who gets it - Logicworks combines the cloud expertise and platform automation to customize solutions to meet your unique requirements. Get started by chatting with a cloud specialist today at snark.cloud/logicworks. That's snark.cloud/logicworksCorey: Welcome to Screaming in the Cloud. I'm Corey Quinn. My guest today is someone that I have been meaning to invite slash drag onto this show for a number of years. We first met at re:Inforce the first year that they had such a thing, Amazon's security conference for cloud, as is Amazon's tradition, named after an email subject line. Chris Farris is a cloud security nerd at Turbot. He's also one of the organizers for fwd:cloudsec, another security conference named after an email subject line with a lot more self-awareness than any of Amazon's stuff. Chris, thank you for joining me.Chris: Oh, thank you for dragging me on. You can let go of my hair now.Corey: Wonderful, wonderful. That's why we're all having the thinning hair going on. People just use it to drag us to and fro, it seems. So, you've been doing something that I'm only going to describe as weird lately because your background—not that dissimilar from mine—is as a practitioner. You've been heavily involved in the security space for a while and lately, I keep seeing an awful lot of things with your name on them getting sucked up by the giant app surveillance apparatus deployed to the internet, looking for basically any mention of AWS that I wind up using to write my newsletter and feed the content grist mill every year. What are you doing and how'd you get there?Chris: So, what am I doing right now is, I'm in marketing. It's kind of a, you know, “Oops, I'm sorry I did that.”Corey: Oh, the running gag is, you work in DevRel; that means, “Oh, you're in marketing, but they're scared to tell you that.” You're self-aware.Chris: Yeah.Corey: Good for you.Chris: I'm willing to address that I'm in marketing now. And I've been a cloud practitioner since probably 2014, cloud security since about 2017. And then just decided, the problem that we have in the cloud security community is a lot of us are just kind of sitting in a corner in our companies and solving problems for our companies, but we're not solving the problems at scale. So, I wanted a job that would allow me to reach a broader audience and help a broader audience. Where I see cloud security having—you know, or cloud in general falling down is Amazon makes it really hard for you to do your side of shared responsibility, and so we need to be out there helping customers understand what they need to be doing. So, I am now at a company called Turbot and we're really trying to promote cloud security.Corey: One of the first promoted guest episodes of this show was David Boeke, your CTO, and one of the things that I regret is that I've sort of lost track of Turbot over the past few years because, yeah, one or two things might have been going on during that timeline as I look back at having kids in the middle of a pandemic and the deadly plague o'er land. And suddenly, every conversation takes place over Zoom, which is like, “Oh, good, it's like a happy hour only instead, now it's just like a conference call for work.” It's like, ‘Conference Calls: The Drinking Game' is never the great direction to go in. But it seems the world is recovering. We're going to be able to spend some time together at re:Invent by all accounts that I'm actively looking forward to.As of this recording, you're relatively new to Turbot, and I figured out that you were going there because, once again, content hits my filters. You wrote a fascinating blog post that hits on an interest of mine that I don't usually talk about much because it's off-putting to some folk, and these days, I don't want to get yelled at and more than I have to about the experience of traveling, I believe it was to an all-hands on the other side of the world.Chris: Yep. So, my first day on the job at Turbot, I was landing in Kuala Lumpur, Malaysia, having left the United States 24 hours—or was it 48? It's hard to tell when you go to the other side of the planet and the time zones have also shifted—and then having left my prior company day before that. But yeah, so Turbot about traditionally has an annual event where we all get together in person. We're a completely remote company, but once a year, we all get together in person in our integrate event.And so, that was my first day on the job. And then you know, it was basically two weeks of reasonably intense hackathons, building out a lot of stuff that hopefully will show up open-source shortly. And then yeah, meeting all of my coworkers. And that was nice.Corey: You've always had a focus through all the time that I've known you and all the public content that you've put out there that has come across my desk that seems to center around security. It's sort of an area that I give a nod to more often than I would like, on some level, but that tends to be your bread and butter. Your focus seems to be almost overwhelmingly on I would call it AWS security. Is that fair to say or is that a mischaracterization of how you view it slash what you actually do? Because, again, we have these parasocial relationships with voices on the internet. And it's like, “Oh, yeah, I know all about that person.” Yeah, you've met them once and all you know other than that is what they put on Twitter.Chris: You follow me on Twitter. Yeah, I would argue that yes, a lot of what I do is AWS-related security because in the past, a lot of what I've been responsible for is cloud security in AWS. But I've always worked for companies that were multi-cloud; it's just that 90% of everything was Amazon and so therefore 90% of my time, 90% of my problems, 90% of my risk was all in AWS. I've been trying to break out of that. I've been trying to understand the other clouds.One of the nice aspects of this role and working on Steampipe is I am now experimenting with other clouds. The whole goal here is to be able to scale our ability as an industry and as security practitioners to support multiple clouds. Because whether we want to or not, we've got it. And so, even though 90% of my spend, 90% of my resources, 90% of my applications may be in AWS, that 10% that I'm ignoring is probably more than 10% of my risk, and we really do need to understand and support major clouds equally.Corey: One post you had recently that I find myself in wholehearted agreement with is on the adoption of Tailscale in the enterprise. I use it for all of my personal nonsense and it is transformative. I like the idea of what that portends for a multi-cloud, or poly-cloud, or whatever the hell we're calling it this week, sort of architectures were historically one of the biggest problems in getting to clouds two speak to one another and manage them in an intelligent way is the security models are different, the user identity stuff is different as well, and the network stuff has always been nightmarish. Well, with Tailscale, you don't have to worry about that in the same way at all. You can, more or less, ignore it, turn on host-based firewalls for everything and just allow Tailscale. And suddenly, okay, I don't really have to think about this in the same way.Chris: Yeah. And you get the micro-segmentation out of it, too, which is really nice. I will agree that I had not looked at Tailscale until I was asked to look at Tailscale, and then it was just like, “Oh, I am completely redoing my home network on that.” But looking at it, it's going to scare some old-school network engineers, it's going to impact their livelihoods and that is going to make them very defensive. And so, what I wanted to do in that post was kind of address, as a practitioner, if I was looking at this with an enterprise lens, what are the concerns you would have on deploying Tailscale in your environment?A lot of those were, you know, around user management. I think the big one that is—it's a new thing in enterprise security, but kind of this host profiling, which is hey, before I let your laptop on the network, I'm going to go make sure that you have antivirus and some kind of EDR, XDR, blah-DR agents so that you know we have a reasonable thing that you're not going to just go and drop [unintelligible 00:09:01] on the network and next thing you know, we're Maersk. Tailscale, that's going to be their biggest thing that they are going to have to figure out is how do they work with some of these enterprise concerns and things along those lines. But I think it's an excellent technology, it was super easy to set up. And the ability to fine-tune and microsegment is great.Corey: Wildly so. They occasionally sponsor my nonsense. I have no earthly idea whether this episode is one of them because we have an editorial firewall—they're not paying me to set any of this stuff, like, “And this is brought to you by whatever.” Yeah, that's the sponsored ad part. This is just, I'm in love with the product.One of the most annoying things about it to me is that I haven't found a reason to give them money yet because the free tier for my personal stuff is very comfortably sized and I don't have a traditional enterprise network or anything like that people would benefit from over here. For one area in cloud security that I think I have potentially been misunderstood around, so I want to take at least this opportunity to clear the air on it a little bit has been that, by all accounts, I've spent the last, mmm, few months or so just absolutely beating the crap out of Azure. Before I wind up adding a little nuance and context to that, I'd love to get your take on what, by all accounts, has been a pretty disastrous year-and-a-half for Azure security.Chris: I think it's been a disastrous year-and-a-half for Azure security. Um—[laugh].Corey: [laugh]. That was something of a leading question, wasn't it?Chris: Yeah, no, I mean, it is. And if you think, though, back, Microsoft's repeatedly had these the ebb and flow of security disasters. You know, Code Red back in whatever the 2000s, NT 4.0 patching back in the '90s. So, I think we're just hitting one of those peaks again, or hopefully, we're hitting the peak and not [laugh] just starting the uptick. A lot of what Azure has built is stuff that they already had, commercial off-the-shelf software, they wrapped multi-tenancy around it, gave it a new SKU under the Azure name, and called is cloud. So, am I super-surprised that somebody figured out how to leverage a Jupyter notebook to find the back-end credentials to drop the firewall tables to go find the next guy over's Cosmos DB? No, I'm not.Corey: I find their failures to be less egregious on a technical basis because let's face it, let's be very clear here, this stuff is hard. I am not pretending for even a slight second that I'm a better security engineer than the very capable, very competent people who work there. This stuff is incredibly hard. And I'm not—Chris: And very well-funded people.Corey: Oh, absolutely, yeah. They make more than I do, presumably. But it's one of those areas where I'm not sitting here trying to dunk on them, their work, their efforts, et cetera, and I don't do a good enough job of clarifying that. My problem is the complete radio silence coming out of Microsoft on this. If AWS had a series of issues like this, I'm hard-pressed to imagine a scenario where they would not have much more transparent communications, they might very well trot out a number of their execs to go on a tour to wind up talking about these things and what they're doing systemically to change it.Because six of these in, it's like, okay, this is now a cultural problem. It's not one rando engineer wandering around the company screwing things up on a rotational basis. It's, what are you going to do? It's unlikely that firing Steven is going to be your fix for these things. So, that is part of it.And then most recently, they wound up having a blog post on the MSRC, the Microsoft Security Resource Center is I believe that acronym? The [mrsth], whatever; and it sounds like a virus you pick up in a hospital—but the problem that I have with it is that they spent most of that being overly defensive and dunking on SOCRadar, the vulnerability researcher who found this and reported it to them. And they had all kinds of quibbles with how it was done, what they did with it, et cetera, et cetera. It's, “Excuse me, you're the ones that left customer data sitting out there in the Azure equivalent of an S3 bucket and you're calling other people out for basically doing your job for you? Excuse me?”Chris: But it wasn't sensitive customer data. It was only the contract information, so therefore it was okay.Corey: Yeah, if I put my contract information out there and try and claim it's not sensitive information, my clients will laugh and laugh as they sue me into the Stone Age.Chris: Yeah well, clearly, you don't have the same level of clickthrough terms that Microsoft is able to negotiate because, you know, [laugh].Corey: It's awful as well, it doesn't even work because, “Oh, it's okay, I lost some of your data, but that's okay because it wasn't particularly sensitive.” Isn't that kind of up to you?Chris: Yes. And if A, I'm actually, you know, a big AWS shop and then I'm looking at Azure and I've got my negotiations in there and Amazon gets wind that I'm negotiating with Azure, that's not going to do well for me and my business. So no, this kind of material is incredibly sensitive. And that was an incredibly tone-deaf response on their part. But you know, to some extent, it was more of a response than we've seen from some of the other Azure multi-tenancy breakdowns.Corey: Yeah, at least they actually said something. I mean, there is that. It's just—it's wild to me. And again, I say this as an Azure customer myself. Their computer vision API is basically just this side of magic, as best I can tell, and none of the other providers have anything like it.That's what I want. But, you know, it almost feels like that service is under NDA because no one talks about it when they're using this service. I did a whole blog post singing its praises and no one from that team reached out to me to say, “Hey, glad you liked it.” Not that they owe me anything, but at the same time it's incredible. Why am I getting shut out? It's like, does this company just have an entire policy of not saying anything ever to anyone at any time? It seems it.Chris: So, a long time ago, I came to this realization that even if you just look at the terminology of the three providers, Amazon has accounts. Why does Amazon have Amazon—or AWS accounts? Because they're a retail company and that's what you signed up with to buy your underwear. Google has projects because they were, I guess, a developer-first thing and that was how they thought about it is, “Oh, you're going to go build something. Here's your project.”What does Microsoft have? Microsoft Azure Subscriptions. Because they are still about the corporate enterprise IT model of it's really about how much we're charging you, not really about what you're getting. So, given that you're not a big enterprise IT customer, you don't—I presume—do lots and lots of golfing at expensive golf resorts, you're probably not fitting their demographic.Corey: You're absolutely not. And that's wild to me. And yet, here we are.Chris: Now, what's scary is they are doing so many interesting things with artificial intelligence… that if… their multi-tenancy boundaries are as bad as we're starting to see, then what else is out there? And more and more, we is carbon-based life forms are relying on Microsoft and other cloud providers to build AI, that's kind of a scary thing. Go watch Satya's keynote at Microsoft Ignite and he's showing you all sorts of ways that AI is going to start replacing the gig economy. You know, it's not just Tesla and self-driving cars at this point. Dali is going to replace the independent graphics designer.They've got things coming out in their office suite that are going to replace the mom-and-pop marketing shops that are generating menus and doing marketing plans for your local restaurants or whatever. There's a whole slew of things where they're really trying to replace people.Corey: That is a wild thing to me. And part of the problem I have in covering AWS is that I have to differentiate in a bunch of different ways between AWS and its Amazon corporate parent. And they have that problem, too, internally. Part of the challenge they have, in many cases, is that perks you give to employees have to scale to one-and-a-half million people, many of them in fulfillment center warehouse things. And that is a different type of problem that a company, like for example, Google, where most of their employees tend to be in office job-style environments.That's a weird thing and I don't know how to even start conceptualizing things operating at that scale. Everything that they do is definitionally a very hard problem when you have to make it scale to that point. What all of the hyperscale cloud providers do is, from where I sit, complete freaking magic. The fact that it works as well as it does is nothing short of a modern-day miracle.Chris: Yeah, and it is more than just throwing hardware at the problem, which was my on-prem solution to most of the things. “Oh, hey. We need higher availability? Okay, we're going to buy two of everything.” We called it the Noah's Ark model, and we have an A side and a B side.And, “Oh, you know what? Just in case we're going to buy some extra capacity and put it in a different city so that, you know, we can just fail from our primary city to our secondary city.” That doesn't work at the cloud provider scale. And really, we haven't seen a major cloud outage—I mean, like, a bad one—in quite a while.Corey: This episode is sponsored in part by Honeycomb. When production is running slow, it's hard to know where problems originate. Is it your application code, users, or the underlying systems? I've got five bucks on DNS, personally. Why scroll through endless dashboards while dealing with alert floods, going from tool to tool to tool that you employ, guessing at which puzzle pieces matter? Context switching and tool sprawl are slowly killing both your team and your business. You should care more about one of those than the other; which one is up to you. Drop the separate pillars and enter a world of getting one unified understanding of the one thing driving your business: production. With Honeycomb, you guess less and know more. Try it for free at honeycomb.io/screaminginthecloud. Observability: it's more than just hipster monitoring.Corey: The outages are always fascinating, just from the way that they are reported in the mainstream media. And again, this is hard, I get it. I am not here to crap on journalists. They, for some ungodly, unknowable reason, have decided not to spend their entire career focusing on the nuances of one very specific, very deep industry. I don't know why.But as [laugh] a result, they wind up getting a lot of their baseline facts wrong about these things. And that's fair. I'm not here to necessarily act as an Amazon spokesperson when these things happen. They have an awful lot of very well-paid people who can do that. But it is interesting just watching the blowback and the reaction of whatever there's an outage, the conversation is never “Does Amazon or Azure or Google suck?” It's, “Does cloud suck as a whole?”That's part of the reason I care so much about Azure getting their act together. If it were just torpedoing Microsoft's reputation, then well, that's sad, but okay. But it extends far beyond that to a point where it's almost where the enterprise groundhog sees the shadow of a data breach and then we get six more years of data center build-outs instead of moving things to a cloud. I spent too many years working in data centers and I have the scars from the cage nuts and crimping patch cables frantically in the middle of the night to prove it. I am thrilled at the fact that I don't believe I will ever again have to frantically drive across town in the middle of the night to replace a hard drive before the rest of the array degrades. Cloud has solved those problems beautifully. I don't want to go back to the Dark Ages.Chris: Yeah, and I think that there's a general potential that we could start seeing this big push towards going back on-prem for effectively sovereign data reasons, whether it's this country has said, “You cannot store your data about our citizens outside of our borders,” and either they're doing that because they do not trust the US Silicon Valley privacy or whatever, or because if it's outside of our borders, then our secret police agents can come knocking on the door at two in the morning to go find out what some dissidents' viewings habits might have been, I see sovereign cloud as this thing that may be a back step from this ubiquitous thing that we have right now in Amazon, Azure, and Google. And so, as we start getting to the point in the history books where we start seeing maps with lots of flags, I think we're going to start seeing a bifurcation of cloud as just a whole thing. We see it already right now. The AWS China partition is not owned by Amazon, it is not run by Amazon, it is not controlled by Amazon. It is controlled by the communist government of China. And nobody is doing business in Russia right now, but if they had not done what they had done earlier this year, we might very well see somebody spinning up a cloud provider that is completely controlled by and in the Russian government.Corey: Well, yes or no, but I want to challenge that assessment for a second because I've had conversations with a number of folks about this where people say, “Okay, great. Like, is the alt-right, for example, going to have better options now that there might be a cloud provider spinning up there?” Or, “Well, okay, what about a new cloud provider to challenge the dominance of the big three?” And there are all these edge cases, either geopolitically or politically based upo—or folks wanting to wind up approaching it from a particular angle, but if we were hired to build out an MVP of a hyperscale cloud provider, like, the budget for that MVP would look like one 100 billion at this point to get started and just get up to a point of critical mass before you could actually see if this thing has legs. And we'd probably burn through almost all of that before doing a single dime in revenue.Chris: Right. And then you're doing that in small markets. Outside of the China partition, these are not massively large markets. I think Oracle is going down an interesting path with its idea of Dedicated Cloud and Oracle Alloy [unintelligible 00:22:52].Corey: I like a lot of what Oracle's doing, and if younger me heard me say that, I don't know how hard I'd hit myself, but here we are. Their free tier for Oracle Cloud is amazing, their data transfer prices are great, and their entire approach of, “We'll build an entire feature complete region in your facility and charge you what, from what I can tell, is a very reasonable amount of money,” works. And it is feature complete, not, “Well, here are the three services that we're going to put in here and everything else is well… it's just sort of a toehold there so you can start migrating it into our big cloud.” No. They're doing it right from that perspective.The biggest problem they've got is the word Oracle at the front end and their, I would say borderline addiction to big-E enterprise markets. I think the future of cloud looks a lot more like cloud-native companies being founded because those big enterprises are starting to describe themselves in similar terminology. And as we've seen in the developer ecosystem, as go startups, so do big companies a few years later. Walk around any big company that's undergoing a digital transformation, you'll see a lot more Macs on desktops, for example. You'll see CI/CD processes in place as opposed to, “Well, oh, you want something new, it's going to be eight weeks to get a server rack downstairs and accounting is going to have 18 pages of forms for you to fill out.” No, it's “click the button,” or—Chris: Don't forget the six months of just getting the financial CapEx approvals.Corey: Exactly.Chris: You have to go through the finance thing before you even get to start talking to techies about when you get your server. I think Oracle is in an interesting place though because it is embracing the fact that it is number four, and so therefore, it's like we are going to work with AWS, we are going to work with Azure, our database can run in AWS or it can run in our cloud, we can interconnect directly, natively, seamlessly with Azure. If I were building a consumer-based thing and I was moving into one of these markets where one of these governments was demanding something like a sovereign cloud, Oracle is a great place to go and throw—okay, all of our front-end consumer whatever is all going to sit in AWS because that's what we do for all other countries. For this one country, we're just going to go and build this thing in Oracle and we're going to leverage Oracle Alloy or whatever, and now suddenly, okay, their data is in their country and it's subject to their laws but I don't have to re-architect to go into one of these, you know, little countries with tin horn dictators.Corey: It's the way to do multi-cloud right, from my perspective. I'll use a component service in a different cloud, I'm under no illusions, though, in doing that I'm increasing my resiliency. I'm not removing single points of failure; I'm adding them. And I make that trade-off on a case-by-case basis, knowingly. But there is a case for some workloads—probably not yours if you're listening to this; assume not, but when you have more context, maybe so—where, okay, we need to be across multiple providers for a variety of strategic or contextual reasons for this workload.That does not mean everything you build needs to be able to do that. It means you're going to make trade-offs for that workload, and understanding the boundaries of where that starts and where that stops is going to be important. That is not the worst idea in the world for a given appropriate workload, that you can optimize stuff into a container and then can run, more or less, anywhere that can take a container. But that is also not the majority of most people's workloads.Chris: Yeah. And I think what that comes back to from the security practitioner standpoint is you have to support not just your primary cloud, your favorite cloud, the one you know, you have to support any cloud. And whether that's, you know, hey, congratulations. Your developers want to use Tailscale because it bypasses a ton of complexity in getting these remote island VPCs from this recent acquisition integrated into your network or because you're going into a new market and you have to support Oracle Cloud in Saudi Arabia, then you as a practitioner have to kind of support any cloud.And so, one of the reasons that I've joined and I'm working on, and so excited about Steampipe is it kind of does give you that. It is a uniform interface to not just AWS, Azure, and Google, but all sorts of clouds, whether it's GitHub or Oracle, or Tailscale. So, that's kind of the message I have for security practitioners at this point is, I tried, I fought, I screamed and yelled and ranted on Twitter, against, you know, doing multi-cloud, but at the end of the day, we were still multi-cloud.Corey: When I see these things evolving, is that, yeah, as a practitioner, we're increasingly having to work across multiple providers, but not to a stupendous depth that's the intimidating thing that scares the hell out of people. I still remember my first time with the AWS console, being so overwhelmed with a number of services, and there were 12. Now, there are hundreds, and I still feel that same sense of being overwhelmed, but I also have the context now to realize that over half of all customer spend globally is on EC2. That's one service. Yes, you need, like, five more to get it to work, but okay.And once you go through learning that to get started, and there's a lot of moving parts around it, like, “Oh, God, I have to do this for every service?” No, take Route 53—my favorite database, but most people use it as a DNS service—you can go start to finish on basically everything that service does that a human being is going to use in less than four hours, and then you're more or less ready to go. Everything is not the hairy beast that is EC2. And most of those services are not for you, whoever you are, whatever you do, most AWS services are not for you. Full stop.Chris: Yes and no. I mean, as a security practitioner, you need to know what your developers are doing, and I've worked in large organizations with lots of things and I would joke that, oh, yeah, I'm sure we're using every service but the IoT, and then I go and I look at our bill, and I was like, “Oh, why are we dropping that much on IoT?” Oh, because they wanted to use the Managed MQTT service.Corey: Ah, I start with the bill because the bill is the source of truth.Chris: Yes, they wanted to use the Managed MQTT service. Okay, great. So, we're now in IoT. But how many of those things have resource policies, how many of those things can be made public, and how many of those things are your CSPM actually checking for and telling you that, hey, a developer has gone out somewhere and made this SageMaker notebook public, or this MQTT topic public. And so, that's where you know, you need to have that level of depth and then you've got to have that level of depth in each cloud. To some extent, if the cloud is just the core basic VMs, object storage, maybe some networking, and a managed relational database, super simple to understand what all you need to do to build a baseline to secure that. As soon as you start adding in on all of the fancy services that AWS has. I re—Corey: Yeah, migrating your Step Functions workflow to other cloud is going to be a living goddamn nightmare. Migrating something that you stuffed into a container and run on EC2 or Fargate is probably going to be a lot simpler. But there are always nuances.Chris: Yep. But the security profile of a Step Function is significantly different. So, you know, there's not much you can do there wrong, yet.Corey: You say that now, but wait for their next security breach, and then we start calling them Stumble Functions instead.Chris: Yeah. I say that. And the next thing, you know, we're going to have something like Lambda [unintelligible 00:30:31] show up and I'm just going to be able to put my Step Function on the internet unauthenticated. Because, you know, that's what Amazon does: they innovate, but they don't necessarily warn security practitioners ahead of their innovation that, hey, you're we're about to release this thing. You might want to prepare for it and adjust your baselines, or talk to your developers, or here's a service control policy that you can drop in place to, you know, like, suppress it for a little bit. No, it's like, “Hey, these things are there,” and by the time you see the tweets or read the documentation, you've got some developer who's put it in production somewhere. And then it becomes a lot more difficult for you as a security practitioner to put the brakes on it.Corey: I really want to thank you for spending so much time talking to me. If people want to learn more and follow your exploits—as they should—where can they find you?Chris: They can find me at steampipe.io/blog. That is where all of my latest rants, raves, research, and how-tos show up.Corey: And we will, of course, put a link to that in the [show notes 00:31:37]. Thank you so much for being so generous with your time. I appreciate it.Chris: Perfect, thank you. You have a good one.Corey: Chris Farris, cloud security nerd at Turbot. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry insulting comment, and be sure to mention exactly which Azure communications team you work on.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

AWS Morning Brief
1000 Access Points of Light

AWS Morning Brief

Play Episode Listen Later Jan 23, 2023 5:41


Links: Amazon CloudFront now supports the request header order and header count headers Amazon ECS announces the new default console experience  Amazon EFS Supports 1,000 Access Points per File System AWS Nitro Enclaves announces support for multiple enclaves AWS Network Optimization Tips  Introducing multi-function packager, allowing more than one function per event trigger on Amazon CloudFront  Winning the Cat-and-Mouse Race: Staying One Step Ahead of Streaming Free-Riders with GeoGuard and AWS

AWS Morning Brief
Wait Did You Say Root API Keys?

AWS Morning Brief

Play Episode Listen Later Jan 19, 2023 4:53


Links: Join Corey in Phoenix next Sunday at 1PM at Zuzu for a community meet-up. Rackspace continues to trickle the truth out; it's now admitting that attackers accessed customer data  Tom Forbes scanned--wait, holy hell, he scanned every package on PyPi and found 57 live AWS keys.  In one year we're going to come back and see how accurate the heads of AWS security are with their predictions for cybersecurity in 2023 Today's tip of the week is to go fire up your important AWS account(s) and validate that the root user doesn't have API credentials assigned.

Screaming in the Cloud
Becoming a Rural Remote Worker with Chris Vermilion

Screaming in the Cloud

Play Episode Listen Later Jan 19, 2023 33:01


About ChrisChris is a mostly-backend mostly-engineer at Remix Labs, working on visual app development. He has been in software startups for ten years, but his first and unrequited love was particle physics.  Before joining Remix Labs, he wrote numerical simulation and analysis tools for the Large Hadron Collider, then co-founded Roobiq, a clean and powerful mobile client for Salesforce back when the official ones were neither.Links Referenced: Remix Labs: https://remixlabs.com/ Twitter: https://twitter.com/chrisvermilion TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Tailscale SSH is a new, and arguably better way to SSH. Once you've enabled Tailscale SSH on your server and user devices, Tailscale takes care of the rest. So you don't need to manage, rotate, or distribute new SSH keys every time someone on your team leaves. Pretty cool, right? Tailscale gives each device in your network a node key to connect to your VPN, and uses that same key for SSH authorization and encryption. So basically you're SSHing the same way that you're already managing your network. So what's the benefit? Well, built-in key rotation, the ability to manage permissions as code, connectivity between any two devices, and reduced latency. You can even ask users to re-authenticate SSH connections for that extra bit of security to keep the compliance folks happy. Try Tailscale now - it's free forever for personal use.Corey: This episode is sponsored by our friends at Logicworks. Getting to the cloud is challenging enough for many places, especially maintaining security, resiliency, cost control, agility, etc, etc, etc. Things break, configurations drift, technology advances, and organizations, frankly, need to evolve. How can you get to the cloud faster and ensure you have the right team in place to maintain success over time? Day 2 matters. Work with a partner who gets it - Logicworks combines the cloud expertise and platform automation to customize solutions to meet your unique requirements. Get started by chatting with a cloud specialist today at snark.cloud/logicworks. That's snark.cloud/logicworksCorey: Welcome to Screaming in the Cloud. I'm Corey Quinn. When I was nine years old, one of the worst tragedies that can ever befall a boy happened to me. That's right, my parents moved me to Maine. And I spent the next ten years desperately trying to get out of the state.Once I succeeded and moved to California, I found myself in a position where almost nothing can drag me back there. One of the exceptions—basically, the only exception—is Monktoberfest, a conference put on every year by the fine folks at RedMonk. It is unquestionably the best conference that I have ever been to, and it continually amazes me every time I go. The last time I was out there, I met today's guest. Chris Vermilion is a Senior Software Developer at Remix Labs. Chris, now that I finished insulting the state that you call home, how are you?Chris: I'm great. I'm happy to be in a state that's not California.Corey: I hear you. It's, uh—I talk a lot of smack about Maine. But to be perfectly direct, my problem with it is that I grew up there and that was a difficult time in my life because I, really I guess, never finished growing up according to most people. And all right, we'll accept it. No one can hate a place in the same way that you can hate it if you grew up there and didn't enjoy the experience.So, it's not Maine that's the problem; it's me. I feel like I should clarify that I'm going to get letters and people in Maine will write those letters and then have to ride their horses to Massachusetts to mail them. But we know how that works.Chris: [laugh].Corey: So, what is Remix Labs? Let's start there. Because Remix sounds like… well, it sounds like a term that is overused. I see it everywhere in the business space. I know there was a Remix thing that recently got sold to I think it was at Shopify or Spotify; I keep getting those two confused. And—Chris: One of the two, yeah.Corey: Yeah, exactly one of them plays music and one of them sells me things except now I think they both do both, and everything has gone wonky and confusing. But what do you folks do over there?Chris: So, we work on visual app development for everybody. So, the goal is to have kind of a spreadsheet-on-steroids-like development environment where you can build interactively, you have live coding, you have a responsive experience in building interactive apps, websites, mobile apps, a little bit of everything, and providing an experience where you can build systems of engagement. So tools, mobile apps, that kind of work with whatever back-end resources you're trying to do, you can collaborate across different people, pass things around, and you can do that all with a nice kind of visual app developer, where you can sort of drop nodes around and wire them together and built in a way that's it's hopefully accessible to non-developers, to project managers, to domain experts, to you know, whatever stakeholders are interested in modifying that final product.Corey: I would say that I count as one of those. I use something similar to build the tool that assembles my newsletter every week, and that was solving a difficult problem for me. I can write back-ends reasonably well, using my primary tool, which is sheer brute force. I am not much of a developer, but it turns out that with enough enthusiasm, you can overcome most limitations. And that's great, but I know nothing about front end; it does not make sense to me, it does not click in the way that other things have clicked.So, I was fourth and inches from just retaining a contractor to build out a barely serviceable internal app. And I discovered, oh, use this low-code tool to drag and drop things and that basically was Visual Basic for internal apps. And that was awesome, but they're still positioned squarely in the space of internal apps only. There's no mobile app story, there's—and it works well enough for what I do, but I have other projects, I want to wind up getting out the door that are not strictly for internal use that would benefit from being able to have a serviceable interface slapped onto. It doesn't need to be gorgeous, it doesn't need to win awards, it just needs to be, “Cool, it can display the output of a table in a variety of different ways. It has a button and when I click a button, it does a thing, generally represented as an API call to something.”And doesn't take much, but being able to have something like that, even for an internal app, has been absolutely transformative just for workflow stuff internally, for making things accessible to people that are not otherwise going to be able to do those sorts of things, by which I mean me.Chris: Yeah. I mean, exactly, I think that is the kind of use case that we are aiming for is making this accessible to everybody, building tools that work for people that aren't necessarily software developers, they don't want to dive into code—although they can if they want, it's extensible in that way—that aren't necessarily front-end developers or designers, although it's accessible to designers and if you want to start from that end, you can do it. And it's amenable to collaboration, so you can have somebody that understands the problem build something that works, you can have somebody that understands design build something that works well and looks nice, and you can have somebody that understands the code or is more of a back-end developer, then go back in and maybe fine-tune the API calls because they realize that you're doing the same thing over and over again and so there's a better way to structure the lower parts of things. But you can pass around that experience between all these different stakeholders and you can construct something that everybody can modify to sort of suit their own needs and desires.Corey: Many years ago, Bill Clinton wound up coining the phrase, ‘The Digital Divide' to talk about people who had basically internet access and who didn't—those who got it or did not—and I feel like we have a modern form of that, the technology haves and have nots. Easy example of this for a different part of my workflow here: this podcast, as anyone listening to it is probably aware by now, is sponsored by awesome folks who wind up wanting to tell you about the exciting services or tools or products that they are building. And sometimes some of those sponsors will say things like, “Okay, here's the URL I want you to read into the microphone during the ad read,” and my response is a polite form of, “Are you serious?” It's seven different subdirectories on the web server, followed by a UTM series of tracking codes that, yeah, I promise, none of you are going to type that in. I'm not even going to wind up reading into the microphone because my attention span trips out a third of the way through.So, I needed a URL shortener. So, I set up snark.cloud for this. For a long time, that was relatively straightforward because I just used an S3 bucket with redirect objects inside of it. But then you have sort of the problem being a victim of your own success, to some extent, and I was at a point where, oh, I can have people control some of these things that aren't me; I don't need to be the person that sets up the link redirection work.Yeah, the challenge is now that you have a business user who is extraordinarily good at what he does, but he's also not someone who has deep experience in writing code, and trying to sit here and explain to him, here's how to set up a redirect object in an S3 bucket, like, why didn't I save time and tell him to go screw himself? It's awful. So, I've looked for a lot of different answers for this, and the one that I found lurking on GitHub—and I've talked about it a couple of times, now—runs on Google Cloud Run, and the front-end for that of the business user—which sounds ridiculous, but it's also kind of clever, is a Google Sheet. Because every business user knows how to work a Google Sheet. There's one column labeled ‘slug' and the other one labeled ‘URL' that it points to.And every time someone visits a snark.cloud slash whatever the hell the slug happens to be, it automatically does a redirect. And it's glorious. But I shouldn't have to go digging into the depths of GitHub to find stuff like that. This feels like a perfect use case for a no-code, low-code tool.Chris: Yeah. No, I agree. I mean, that's a cool use case. And I… as always, our competitor is Google Sheets. I think everybody in software development in enterprise software's only real competitor is the spreadsheet.Corey: Oh, God, yes, I wind up fixing AWS bills for a living and my biggest competitor is always Microsoft Excel. It's, “Yeah, we're going to do it ourselves internally,” is what most people do. It seems like no matter what business line I've worked in, I've companies that did Robo-advising for retirement planning; yeah, some people do it themselves in Microsoft Excel. I worked for an expense reporting company; everyone does that in Microsoft Excel. And so, on and so forth.There are really very few verticals where that's not an option. It's like, but what about a dating site? Oh, there are certain people who absolutely will use Microsoft Excel for that. Personally, I think it's a bad idea to hook up where you VLOOKUP but what do I know?Chris: [laugh]. Right, right.Corey: Before you wound up going into the wide world of low-code development over at Remix, you—well, a lot of people have different backstories when I talk to them on this show. Yours is definitely one of the more esoteric because the common case and most people talk about is oh, “I went to Stanford and then became a software engineer.” “Great. What did you study?” “Computer Science,” or something like it. Alternately, they drop out of school and go do things in their backyard. You have a PhD in particle physics, is it?Chris: That's right. Yeah.Corey: Which first, is wild in his own right, but we'll get back to that. How did you get here from there?Chris: Ah. Well, it's kind of the age-old story of academia. So, I started in electrical engineering and ended up double majoring in physics because that you had to take a lot of physics to be an engineer, and I said, you know, this is more fun. This is interesting. Building things is great, but sitting around reading papers is really where my heart's at.And ended up going to graduate school, which is about the best gig you can ever get. You get paid to sit in an office and read and write papers, and occasionally go out drinking with other grad students, and that's really about it.Corey: I only just now for the first time in my life, realized how much some aspects of my career resemble being a [laugh] grad student. Please, continue.Chris: It doesn't pay very well is the catch, you know? It's very hard to support a lifestyle that exists outside of your office, or, you know, involves a family and children, which is certainly one downside. But it's a lot of fun and it's very low stress, as long as you are, let's say, not trying to get a job afterward. Because where this all breaks down is that, you know, as I recall, the time I was a graduate student, there were roughly as many people graduating as graduate students every year as there were professors total in the field of physics, at least in the United States. That was something like the scale of the relationship.And so, if you do the math, and unfortunately, we were relatively good at doing math, you could see, you know, most of us were not going to go on, you know? This was the path to becoming a professor, but—Corey: You look at number of students and the number of professorships available in the industry, I guess we'll call it, and yeah, it's hmm, basic arithmetic does not seem like something that anyone in that department is not capable of doing.Chris: Exactly. So, you're right, we were all I think, more or less qualified to be an academic professor, certainly at research institutions, where the only qualification, really, is to be good at doing research and you have to tolerate teaching students sometimes. But there tends to be very little training on how to do that, or a meaningful evaluation of whether you're doing it well.Corey: I want to dive into that a bit because I think that's something we see a lot in this industry, where there's no training on how to do a lot of different things. Teaching is one very clear example, another one is interviewing people for jobs, so people are making it up as they go along, despite there being decades and decades of longitudinal studies of people figuring out what works and what doesn't, tech his always loved to just sort of throw it all out and start over. It's odd to me that academia would follow in similar patterns around not having a clear structure for, “Oh, so you're a grad student. You're going to be teaching a class. Here's how to be reasonably effective at it.” Given that higher education was not the place for me, I have very little insight into this. Is that how it plays out?Chris: I don't want to be too unfair to academia as a whole, and actually, I was quite lucky, I was a student at the University of Washington and we had a really great physics education group, so we did actually spend a fair amount of time thinking about effective ways to teach undergraduates and doing this great tutorial system they had there. But my sense was in the field as a whole, for people on the track to become professors at research institutions, there was typically not much in the way of training as a teacher, there was not really a lot of thought about pedagogy or the mechanics of delivering lectures. You know, you're sort of given a box full of chalk and a classroom and said, you know, “You have freshman physics this quarter. The last teacher used this textbook and it seems to be okay,” tended to be the sort of preparation that you would get. You know, and I think it varies institution to institution what kind of support you get, you know, the level of graduate students helping you out, but I think in lots of places in academia, the role of professors as teachers was the second thought, you know, if it was indeed thought at all.And similarly, the role of professors as mentors to graduate students, which, you know, if anything, is sort of their primary job is guiding graduate students through their early career. And again, I mean, much like in software, that was all very ad hoc. You know, and I think there are some similarities in terms of how academics and how tech workers think of themselves as sort of inventing the universe, we're at the forefront, the bleeding edge of human knowledge, and therefore because I'm being innovative in this one particular aspect, I can justify being innovative in all of them. I mean, that's the disruptive thing to do, right?Corey: And it's a shame that you're such a nice person because you would be phenomenal at basically being the most condescending person in all of tech if you wanted to. Because think about this, you have people saying, “Oh, what do you do?” “I'm a full-stack engineer.” And then some of the worst people in the world, of which I admit I used to be one, are, “Oh, full-stack. Really? When's the last time you wrote a device driver?”And you can keep on going at that. You work in particle physics, so you're all, “That's adorable. Hold my tea. When's the last time you created matter from energy?” And yeah, and then it becomes this the—it's very hard to wind up beating you in that particular game of [who'd 00:15:07] wore it better.Chris: Right. One of my fond memories of being a student is back when I got to spend more time thinking about these things and actually still remembered them, you know, in my electoral engineering days and physics days, I really had studied all the way down from the particle physics to semiconductor physics to how to lay out silicon chips and, you know, how to build ALUs and CPUs and whatnot from basic transistor gates. Yeah, and then all the way up to, you know, writing compilers and programming languages. And it really did seem like you could understand all those parts. I couldn't tell you how any of those things work anymore. Sadly, that part of my brain has now taken up with Go's lexical scoping rules and borrow checker fights with Rust. But there was a time when I was a smart person and knew those things.Corey: This episode is sponsored in part by our friends at Strata. Are you struggling to keep up with the demands of managing and securing identity in your distributed enterprise IT environment? You're not alone, but you shouldn't let that hold you back. With Strata's Identity Orchestration Platform, you can secure all your apps on any cloud with any IDP, so your IT teams will never have to refactor for identity again. Imagine modernizing app identity in minutes instead of months, deploying passwordless on any tricky old app, and achieving business resilience with always-on identity, all from one lightweight and flexible platform.Want to see it in action? Share your identity challenge with them on a discovery call and they'll hook you up with a complimentary pair of AirPods Pro. Don't miss out, visit Strata.io/ScreamingCloud. That's Strata dot io slash ScreamingCloud.Corey: I want to go back to what sounded like a throwaway joke at the start of the episode. In seriousness, one of the reasons—at least that I told myself at the time—that I left Maine was that it was pretty clear that there was no significant, lasting opportunity in industry when I was in Maine. In fact, the girl that I was dating at the time in college graduated college, and the paper of record for the state, The Maine Sunday Telegram, which during the week is called The Portland Press Herald, did a front-page story on her about how she went to school on a pulp and paper scholarship, she was valedictorian in her chemical engineering class at the University of Maine and had to leave the state to get a job. And every year they would roll out the governor, whoever that happened to be, to the University of Maine to give a commencement speech that's, “Don't leave Maine, don't leave Maine, don't leave Maine,” but without any real answer to, “Well, for what jobs?”Now, that Covid has been this plague o'er the land that has been devastating society for a while, work-from-home has become much more of a cohesive thing. And an awful lot of companies are fully embracing it. How have you seen Maine change based upon that for one, and for another, how have you found that community has been developed in the local sense because there was none of that in Maine when I was there? Even the brief time where I was visiting for a conference for a week, I saw definite signs of a strong local community in the tech space. What happened? I love it.Chris: It's great. Yeah, so I moved to Maine eight years ago, in 2014. And yeah, I was lucky enough to pretty early on, meet up with a few of the local nerds, and we have a long-running Slack group that I just saw was about to turn nine, so I guess I was there in the early days, called Computers Anonymous. It was a spinoff, I think, from a project somebody else had started in a few other cities. The joke was it was a sort of a confessional group of, you know, we're here to commiserate over our relationships with technology, which all of us have our complaints.Corey: Honestly, tech community is more of a support group than most other areas, I think.Chris: Absolutely. All you have to do is just have name and technology and somebody will pipe up. “Okay, you know, I've a horror story about that one.” But it has over the years turned into, you know, a very active Slack group of people that meet up once a month for beers and chats with each other, and you know, we all know each other's kids. And when the pandemic hit, it was absolutely a lifeline that we were all sort of still talking to each other every day and passing tips of, you know, which restaurants were doing takeout, and you know which ones were doing takeout and takeout booze, and all kinds of local knowledge was being spread around that way.So, it was a lucky thing to have when that hit, we had this community. Because it existed already as this community of, you know, people that were remote workers. And I think over the time that I've been here, I've really seen a growth in people coming here to work somewhere else because it's a lovely place to live, it's a much cheaper place to live than almost anywhere else I've ever been, you know, I think it's pretty attractive to the folks come up from Boston or New York or Connecticut for the summer, and they say, “Ah, you know, this doesn't seem so bad to live.” And then they come here for a winter, and then they think, “Well, okay, maybe I was wrong,” and go back. But I've really enjoyed my time here, and the tools for communicating and working remotely, have really taken off.You know, a decade ago, my first startup—actually, you know, in kind of a similar situation, similar story, we were starting a company in Louisville, Kentucky. It was where we happen to live. We had a tech community there that were asking those same questions. “Why is anybody leaving? Why is everybody leaving?”And we started this company, and we did an accelerator in San Francisco, and every single person we talked to—and this is 2012—said, you have to bring the company to San Francisco. It's the only way you'll ever hire anybody, it's the only way you'll ever raise any money, this is the only place in the world that you could ever possibly run a tech company. And you know, we tried and failed.Corey: Oh, we're one of those innovative industries in the world. We've taken a job that can be done from literally anywhere that has internet access and created a land crunch on eight square miles, located in an earthquake zone.Chris: Exactly. We're going to take a ton of VC money and where to spend 90% of it on rent in the Bay Area. The rent paid back to the LPs of our VC funds, and the circle of life continues.Corey: Oh, yeah. When I started this place as an independent consultant six years ago, I looked around, okay, should I rent space in an office so I have a place where I go and work? And I saw how much it costs to sublet even, like, a closed-door office in an existing tech startup's office space, saw the price tag, laughed myself silly, and nope, nope, nope. Instead installed a door on my home office and got this place set up as a—in my spare room now is transformed into my home office slash recording studio. And yeah, “Well, wasn't it expensive to do that kind of stuff?” Not compared to the first three days of rent in a place like that it wasn't. I feel like that's what's driving a lot of the return to office stories is the sort of, I guess, an expression of the sunk cost fallacy.Chris: Exactly. And it's a variation of nobody ever got fired for choosing IBM, you know? Nobody ever got fired for saying we should work in the office. It's the way we've always done things, people are used to it, and there really are difficulties to collaborating effectively remotely, you know? You do lose something with the lack of day-to-day contact, a lack of in-person contact, people really do get kind of burned out on interacting over screens. But I think there are ways around that and the benefits, in my mind, my experience, you know, working remotely for the last ten years or so, tend to outweigh the costs.Corey: Oh, yeah. If I were 20 years younger, I would absolutely have been much more amenable to staying in the state. There's a lot of things that recommend it. I mean, I don't want people listening to this to think I actually hate Maine. It's become a running joke, but it's also, there was remarkably little opportunity in tech back when I lived there.And now globally, I think we're seeing the rise of opportunity. And that is a line I heard in a talk once that stuck with me that talent is evenly distributed, but opportunity isn't. And there are paths forward now for folks who—I'm told—somehow don't live in that same eight-square miles of the world, where they too can build tech companies and do interesting things and work intelligently with other folks. I mean, the thing that always struck me as so odd before the pandemic was this insistence on, “Oh, we don't allow remote work.” It's, “Well, hang on a minute. Aren't we all telecommuting in from wherever offices happen to be to AWS?” Because I've checked thoroughly, they will not let you work from us-east-1. In fact, they're very strict on that rule.Chris: [laugh]. Yeah. And it's remarkable how long I think the attitude persisted that we can solve any problem except how to work somewhere other than SoMa.Corey: Part of the problem too in the startup space, and one of the things I'm so excited about seeing what you're doing over at Remix Labs, is so many of the tech startups for a long time felt like they were built almost entirely around problems that young, usually single men had in their 20s when they worked in tech and didn't want to deal with the inconveniences of having to take care of themselves. Think food delivery, think laundry services, think dating apps, et cetera, et cetera. It feels like now we're getting into an era where there's a lot of development and focus and funding being aimed at things that are a lot more substantial, like how would we make it possible for someone to build an app internally or externally without making them go to through a trial-by-fire hazing ritual of going to a boot camp for a year first?Chris: Yeah. No, I think that's right. I think there's been an evolution toward building tools for broader problems, for building tools that work for everybody. I think there was a definite startup ouroboros in the, kind of, early days of this past tech boom of so much money being thrown at early-stage startups with a couple of young people building them, and they solved a zillion of their own problems. And there was so much money being thrown at them that they were happy to spend lots of money on the problems that they had, and so it looked like there was this huge market for startups to solve those problems.And I think we'll probably see that dry up a little bit. So, it's nice to get back to what are the problems that the rest of us have. You know, or maybe the rest of you. I can't pretend that I'm not one of those startup people that wants on-demand laundry. But.Corey: Yet you wake up one day and realize, oh, yeah. That does change things a bit. Honestly, one of the weirdest things for me about moving to California from Maine was just the sheer level of convenience in different areas.Chris: Yes.Corey: And part of it is city living, true, but Maine is one those places where if you're traveling somewhere, you're taking a car, full stop. And living in a number of cities like San Francisco, it's, oh great, if I want to order food, there's not, “The restaurant that delivers,” it's, I can have basically anything that I want showing up here within the hour. Just that alone was a weird, transformative moment. I know, I still feel like 20 years in, that I'm “Country Boy Discovers City for the First Time; Loses Goddamn Mind.” Like, that is where I still am. It's still magic. I became an urban creature just by not being one for my formative years.Chris: Yeah. No, I mean, absolutely. I grew up in Ann Arbor, which is sort of a smallish college town, and certainly more urban than the areas around it, but visiting the big city of Detroit or Lansing, it was exciting. And, you know, I got older, I really sort of thought of myself as a city person. And I lived in San Francisco for a while and loved it, and Seattle for a while and loved it.Portland has been a great balance of, there's city; it's a five minute drive from my house that has amazing restaurants and concerts and a great art scene and places to eat and roughly 8000 microbreweries, but it's still a relatively small community. I know a lot of the people here. I sort of drive across town from one end to the other in 20 minutes, pick up my kids from school pretty easily. So, it makes for a nice balance here.Corey: I am very enthused on, well, the idea of growing community in localized places. One thing that I think we did lose a bit during the pandemic was, every conference became online, so therefore, every conference becomes the same and it's all the same crappy Zoom-esque experience. It's oh, it's like work with a slightly different topic, and for once the people on this call can't fire me… directly. So, it's one of those areas of just there's not enough differentiation.I didn't realize until I went back to Monktoberfest a month or so ago at the time at this call recording just how much I'd missed that sense of local community.Chris: Yeah.Corey: Because before that, the only conferences I'd been to since the pandemic hit were big corporate affairs, and yeah, you find community there, but it also is very different element to it, it has a different feeling. It's impossible to describe unless you've been to some of these community conferences, I think.Chris: Yeah. I mean, I think a smallish conference like that where you see a lot of the same people every year—credit to Steven, the whole RedMonk team for Monktoberfest—that they put on such a great show that every year, you see lots and lots of faces that you've seen the last several because everybody knows it's such a great conference, they come right back. And so, it becomes kind of a community. As I've gotten older a year between meetings doesn't seem like that long time anymore, so these are the friends I see from time to time, and you know, we have a Slack who chat from time to time. So, finding those ways to sort of cultivate small groups that are in regular contact and have that kind of specific environment and culture to them within the broader industry, I think has been super valuable, I think. To me, certainly.Corey: I really enjoyed so much of what has come out of the pandemic in some ways, which sounds like a weird thing to say, but I'm trying to find the silver linings where I can. I recently met someone who'd worked here with me for a year-and-a-half that I'd never met in person. Other people that I'd spoken to at length for the last few years in various capacity, I finally meet them in person and, “Huh. Somehow it never came up in conversation that they're six foot eight.” Like, “Yeah, okay/ that definitely is one of those things that you notice about them in person.” Ah, but here we are.I really want to thank you for spending as much time as you have to talk about what you're up to, what your experiences have been like. If people want to learn more, where's the best place for them to find you? And please don't say Maine.Chris: [laugh]. Well, as of this recording, you can find me on Twitter at @chrisvermilion, V-E-R-M-I-L-I-O-N. That's probably easiest.Corey: And we will, of course, put links to that in the [show notes 00:28:53]. Thank you so much for being so generous with your time. I appreciate it.Chris: No, thanks for having me on. This was fun.Corey: Chris Vermilion, Senior Software Developer at Remix Labs. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment, and since you're presumably from Maine when writing that comment, be sure to ask a grown-up to help you with the more difficult spellings of some of the words.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Screaming in the Cloud
Defining and Nurturing a Self-Supporting Community with Alyss Noland

Screaming in the Cloud

Play Episode Listen Later Jan 17, 2023 33:48


About AlyssAlyss Noland is the head of Developer Relations Relations and Product Marketing at Common Room, an intelligent community-led growth platform. She previously led product marketing for Developer Experience at GitHub where she focused on open source community investment and helping engineering teams find success through development metrics and developer-focused research. She's been working in tech since 2012 in various roles from Sales Engineering and Developer Advocacy to Product Marketing with companies such as GitHub, Box, Atlassian, and BigCommerce, as well as being an advisor at Heavybit. Links Referenced: Common Room: https://www.commonroom.io/ Heavybit: https://www.heavybit.com/ Twitter: https://twitter.com/PreciselyAlyss Twitch: https://www.twitch.tv/PreciselyAlyss TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Tailscale SSH is a new, and arguably better way to SSH. Once you've enabled Tailscale SSH on your server and user devices, Tailscale takes care of the rest. So you don't need to manage, rotate, or distribute new SSH keys every time someone on your team leaves. Pretty cool, right? Tailscale gives each device in your network a node key to connect to your VPN, and uses that same key for SSH authorization and encryption. So basically you're SSHing the same way that you're managing your network.So what's the benefit? You'll get built-in key rotation, the ability to manage permissions as code, connectivity between two devices, and reduced latency. You can even ask users to re-authenticate SSH connections for that extra bit of security. Try Tailscale now - it's free forever for personal use forever.Corey: This episode is sponsored by our friends at Logicworks. Getting to the cloud is challenging enough for many places, especially maintaining security, resiliency, cost control, agility, etc, etc, etc. Things break, configurations drift, technology advances, and organizations, frankly, need to evolve. How can you get to the cloud faster and ensure you have the right team in place to maintain success over time? Day 2 matters. Work with a partner who gets it - Logicworks combines the cloud expertise and platform automation to customize solutions to meet your unique requirements. Get started by chatting with a cloud specialist today at snark.cloud/logicworks. That's snark.cloud/logicworksCorey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I often wonder how to start these conversations, but sometimes it's just handed to me and I don't even have to do a whole lot of work. My guest today is Alyss Noland, who's the Head of Developer Relations Relations and Product Marketing at Common Room. Alyss, thank you for joining me.Alyss: Thanks for having me, Corey. I'm really excited to be here.Corey: So, developer relations relations. It feels like an abstraction that has been forced to be built on top of another abstraction that has gotten too complicated, so as best I can tell, you are walking around as a human equivalent of Kubernetes.Alyss: Oh, gosh, I would really hope not to be a human equivalent of Kubernetes. I think that would make me an octopus. But—Corey: Yeah, “What did you say about me?” Yeah.Alyss: [laugh].Corey: “I didn't come here to be insulted, Quinn.” Yeah.Alyss: No, like listen, I love octopodes. Which [tattoo 00:01:24] is which? So, developer relations relations. Yes, it's an abstraction on an abstraction. A really critical level, it is how do I relate? Can I relate to people that are in the developer relations profession at large?We are at the point at which this is a somewhat poorly-defined area that is continuing to grow. And there's a lot of debates in that space and so I'm really excited to be at an organization that will give me a platform to try and move the industry forward.Corey: Your relatively recent career history is honestly fascinating to me. You spent about a year and a half as a senior developer advocate at Box. And as anyone who's ever tried it knows, it's very hard to beat Box [beatboxing noises]. But you tried and went to GitHub, in which case, you basically transitioned pretty quickly from a Senior Product Marketing Manager to Director of Product Marketing, where you were the go-to-market lead for GitHub Copilot.Alyss: Yeah, that was a really interesting project to be on. I started off at the technical preview back in 2021, launching that too—it ended up being with about a little over a million, two million folks in technical preview. And it's fairly new to the market. There was nothing else—or at the time, there had been nothing else that was using a descendant of GPT-3. There was nothing else using a descendant of GPT-3 to generate suggestions for code to—there were a couple that were using GPT-2, but the amount of language coverage they had was a little bit limited, what they were suggesting was a little bit limited.And it's hard to say, like, highlight of my career, but at that point in time, I would say probably, highlight of my career to be able to work on something with that opportunity for impact.Corey: As someone who was in the technical preview and now tried to be a paying customer of it, but I can't because of my open-source work, it wound up giving it to me for free. I found it to be absolutely transformative. And I know I'm going to get letters and I don't even slightly care because it's not, “I'm going to tab-complete my application.” If a tool can do that, your application is probably not that complex. No, for me, what I find incredibly valuable is the ability to tab-complete through obnoxious boilerplate. CloudFormation, I am not subtweeting you; I am calling you out directly. You are wordy and obnoxious. Fix yourself.And especially in languages that I don't deal with day-to-day—because I'm not a full-time developer—I forget certain parameters or argument order or things like that and being able to effectively tab-complete is awesome for that use case. It's not doing my job; it's automating the crappy part of my job. And I absolutely love it for that.Alyss: Yeah, and was really interesting working on a common portion of product marketing work is that we build messaging houses. We try to identify where's the value to the user, to the organization at large, depending on, like, who it is we're trying to sell to, how does that ladder up from, like, an IoT to a manager. And so, one of the things that I got really excited about as we started to see it—and there's some great work that Dr. Eirini Kallaimvakou has published that I would definitely refer to if you're interested in diving deeper into it—is the way in which Copilot and this, like, ability to improve the boilerplate experience, improve the boring shit—automate the boring shit, if you will—is about developer satisfaction. It's not about making you build your commits faster or about having more lines of code that you like get deployed out; it's about making your jobs suck less.Corey: Well, if you spent, what was it roughly two years, give or take, at GitHub between your various roles—and yes, I'm going to pronounce it ‘GIF-ub' because that's my brand of obnoxious, so I'm going to go for it—you went to Common Room. Let's begin there. What does Common Room do, exactly?Alyss: So, Common Room is an intelligent community-led growth platform. And there's a few things kind of packed into that really short description, but the idea is that we've seen all of these product-lead grows businesses. But at a critical point, and something we've seen at GitHub, which is a product-led growth company, it's something that we've seen at Atlassian, Asana, you name half a dozen different, like, SaaS companies, self-hosted software, open-source, community is at the heart of it. And so, how do you nurture that community? How do you measure that community? How do you prove that the work that you're doing is valuable?And that's what Common Room is setting out to do. And so, when I saw—like, they're not the only person or organization in the market that's doing this, but I think they're doing it exceptionally well, and with really great goals in mind. And so, I'm enthused to try and facilitate that investment in community for more organizations.Corey: One of the challenges that I have seen of products in the community space is it tended, historically, to go in really, I guess I'll call them uncomfortable directions. In the before times, I used to host dinner parties near constantly here, and someone confide into me once—after, you know, six beers or so, because that's when people get the excitingly honest—they mentioned that, “Yeah, I'm supposed to wind up putting these dinners into Salesforce”—or whatever the hell it was—“To track the contacts we have with influencers in this space.” And that made me feel so profoundly uncomfortable. It's, you're invited here to spend time with my friends and my family. You're meeting my kids, it's, yeah, this is just a go-to-market motion and you can [BLEEP] on out of here and never come back.And I did not get that sense to be clear and I'm told the company wound up canceling that horrifying program, but it does feel like it's very easy to turn an authentic relationship into something that feels remarkably sleazy. That said, Common Room has been around for a while and I have yet to hear a single accusation that you folks have come within a thousand miles of doing that. How do you avoid the trap?Alyss: It's a slippery slope, and I can't say that Common Room creates any kind of like enforcement or silos or prevents organizations from falling into this trap. Fundamentally, the way in which community can be abused, the way in which these relationships can be taken advantage of, at least from the perception of the parties that initially built the relationship, is to take the context out of them, to take the empathy out of them, take the people out of them. And so, that is fundamentally left to the organization's principles, it's left to how much authority does community have within the business relative to a sales team. And so first, being able to elevate community in such a way to show that they are having that impact already without having to turn the community into a prospect pool is, I think, one of the critical first steps, and it's something that we've been able to break through initially by connecting things like Slack, Discord, Twitter to show, here's all these people talking about you, here's all the things that they're saying, here's the sentiment analysis, and also, now we're going to push that into Salesforce. So, you can see that this started out in community and it was fostered there. Now, you can see the ROI, you don't need to go hitting up our community contacts to try and sell to them because we're doing it on your behalf in a very real way.Corey: Part of the challenge, I think, is that—and you've talked to me about this in previous conversations we've had—that so much of community is distilled down to a sales motion, which let's be direct, it kind of sucks at, in some levels, because it's okay, great, I'm here to talk to you about how community works. Well, in the AWS community, for example, the reason that formed and is as broad and fast as it is because AWS's documentation is Byzantine and there's a sort of shared suffering that we all get to commiserate over. And whenever AWS tries to take, “Ownership,” quote-unquote, of its community, right, that doesn't actually work that way. They have community watering holes, but to my understanding, the largest AWS-centric Slack team is the Open Guide to AWS's Slack team, which now has, at last count, 15,000 people in it. I'm lucky enough to be the community lead for that project.But it was pre-existing before I got there and it's great to be able to go and talk to people who are using these things. It doesn't feel like it is owned, run, or controlled—because it's not—by AWS themselves. It's clear from the way that your product has evolved, that you feel similarly around that where it's about being aware of the community rather than controlling the community. And that's important.Alyss: Absolutely. And one of the ways in which we, like, highlight this as soon as you're in the product, is being able to show community responsiveness and then what percentage of those responses are coming from my team members. And frankly, as someone who's previously set strategy for developer relations teams, for developer communities, what I want to see is community members responding to each other, community members knowing what's the right place to look, what's the right answer, how am I ensuring that they have the resources that they need, the answers that they need. Because at the end of the day, I can't scale one-to-one; no one can. And so, the community being able to support itself is at the heart of the definition of community.Corey: One of the other problems that I've seen historically, and I'll call it the Chef problem because Chef had an incredibly strong community, and as someone who is deep in the configuration management space myself, but never use Chef, it was the one that I avoided for a variety of reasons at the time, it was phenomenal. I wound up going to ChefConf, despite not being a Chef user, just to spend time with some of the great people that were involved. The blunder that they made before they were acquired into irrelevance by progress—and to be fair, the industry changed direction toward immutable infrastructure in ways that were hard to foresee—but the problem is, they made was hiring their entire community. And it doesn't sound like that would be a bad thing, but suddenly, everyone who was talking about the product had a Chef email address, and that hits very differently.Alyss: It does. And it goes back to that point of trying to maintain those authentic relationships. And if we're to step outside of tech, I have a background prior to tech in the video game industry, and that was a similar problem. Nearly every single community-made application, extension ends up getting acquired by some organization, like Curse, and then piped full of ads, or the person that you thought you could ask or to see build some other better experience of version control software, or a Git client ends up getting consumed into a large business and then the project never sees the light of day. And frankly, that's not how you run community in my estimation.My estimation is, if the community is doing things better than you are, take notes. Product management, pay attention. That's something that is another aspect of doing developer relations is about checking in with those teams, about showing them evidence. And like, it so often ends up being qualitative in a way that doesn't change people's minds or their feelings, where people want to see quantitative numbers in order to say, “Oh, this is the business justification. Like, this is the ROI. This proves that this is the thing we should invest in.” And frankly, no. Like, sometimes it is a little bit more about stepping back and letting the organic empathy and participation happen without having to own it.Corey: There's a sense, I think that a lot of companies feel the need to own every conversation that happens around them, their product, et cetera, and you can't. You just can't, unless—to be direct—your company is failing. Just because if no one's talking about you, then great, you're the only ones talking about you. And you can see this from time to time and it's depressing as hell when you have people who work for a company all tweeting the same cookie-cutter statement, and they get zero interaction except from a bot account. It's sad.Alyss: Yeah. And I've unfortunately seen this more times than I can count in community Slacks where people just, like, copy-paste whatever marketing handed to them, and I would be shocked if they got any engagement at all. Because that's… cool. What do I know about you? Why do I care about this event? Have you personalized it to me?And yeah, you don't want the organization to be the only one talking about you. If you are then you've already failed in this, you know, product-led growth motion. You've kind of—if we want to get into the murky water of NPS, like, nobody's going and telling their friends about your product [laugh]. And the thing that's so valuable is the authentic voice. It's the, “I'm excited to talk about this and I like it enough to tell you what I like about it.” I like it enough to tell you about this use case that might never seen the light of day, but because we're having a conversation between ourselves, it can all be personalized. It can all be about what's going on between us and about our shared experiences. And that is ten times more powerful than most Twitter-promoted ads you'll ever see.Corey: So, I want to unpack a little bit about not developer relations as such, but developer relations relations because I can mostly understand—badly—what product marketing is, but developer relations relations—or as you'd like to call it developer relations squared—that's something new. I've always called DevRel to be devrelopers, and people get annoyed enough at that. What is that newfound layer of abstraction on top of it?Alyss: Well, there's several things that I'm going to end up—and I say end up; I'm six weeks into the role, so I have a lot of high hopes for where I hope this goes. And one of those is things, like, we don't have a very shared understanding and shared definition of what developer advocacy even is, what is developer relations? Does developer marketing belong under that umbrella? How should organizations approach developer relations? How should they value it? Where should it, you know, belong in terms of business strategy?And there's an opportunity for a company whose business it is to elevate this industry, this career path, if you will, where we can spend the time, we can spend the money to say, here's what success looks like. We've interviewed all these groups, we've talked with the leaders in this space that are making it their jobs to think about this. Here's a set of group-developed recommendations for how the industry should mature. Or here's an open-source set of job descriptions and requirements. And like, let's get to some level of shared understanding.So, as an example of, kind of, where I'm leading to with all of this, and some of the challenges that developer relations faces is the State of Developer Relations report that just came out. There's a significant number of people that are coming into developer advocate, developer relations roles for the first time, they have one to two years of experience, they're coming into programs that have been around for one to two years, and so what does that tell you? That tells you you're bringing in people with no experience to try to establish brand new programs, that they're being asked to by their business, and they don't have the vocabulary, the tools, the frameworks in which to establish that for themselves. And so, they're going to be swayed by, you know, the tides of business, by the influences of their leadership without having their own pre-built notions. And so, how do we give them that equipment and how do we elevate the practice?Corey: Cloud native just means you've got more components or microservices than anyone (even a mythical 10x engineer) can keep track of. With OpsLevel, you can build a catalog in minutes and forget needing that mythical 10x engineer. Now, you'll have a 10x service catalog to accompany your 10x service count. Visit OpsLevel.com to learn how easy it is to build and manage your service catalog. Connect to your git provider and you're off to the races with service import, repo ownership, tech docs, and more. Corey: It feels like so much of the DevRel discourse has turned into, one, we define it by what is not, and two, it doesn't matter how you're measuring it, you're measuring it wrong. I feel like that is, I guess we'll call it counterproductive, for lack of a better descriptor. It feels like there's such a short-sighted perspective on all of this, but at the same time, you've absolutely got to find ways to articulate the value of DevRel slash community to the business otherwise, it turns into a really uncomfortable moment when, okay, time to cut costs. Why should we keep your function over a different function? If there's not a revenue or upside or time to market or some form of value story tied to that, that the business can understand that isn't just touchy-feely, it's a very difficult path forward from there. How do you see it?Alyss: I agree with you and I've, frankly, run into this problem several times in my career, and every time I've been a developer advocate. It's, you know—and where I've found the most success is not in saying, “Here's exactly the numbers that I'm going to be constantly looking at. I'm going to try to produce this many pieces of content, or I'm absolutely not speaking at events. And that's not my job. Or I'm not writing code. That's not my job.”It's about understanding what is driving the business forward. Who do I need participation and buy-in from and where am I hoping to go? Like, what does a year out from this look like? What does three years out from this look like? At Box, we do not want to be the API governance standard. That is not our job. That's not where we sit within engineering.That's frankly, if you really want to get into it, internal developer advocacy because it can influence the impact on the community. It is not the core focus and there are probably people better equipped and better educated on the core application. Big commerce, platform ecosystem, platform flywheel developers are fundamentally a part of continuing to grow the business and how do I go make that point to sales, how do I go make that point to partners, how do I go make that point to customer success, so that I can build a function that has more than one person. And so, I think to kind of bring it back to the larger question, that is where I see our greatest challenge is that we haven't given ourselves the vocabulary or the framework to understand the level of complexity that DevRel has become in being across so many industries, and being in B2B, and being in business to developer, and being in business to consumer. No one size fits all and we need to stop trying to treat it as though it can be.Corey: I think that there is a, how to put it, a problem in terms of how Twitter views a lot of these things. Someone wound up finally distilling it down for me in relatively recent times with a very resonant quote, which was simply put, that Twitter is not where you go for nuance. Twitter is where you go to be righteous. And I realized, oh, my God, that describes a good 80% of the things I've put up there. Like when I talk about how when companies do this thing to their staff and it's crappy, I am not necessarily for a nuanced debate, although of course there's always nuance and edge cases in the rest.As a counterpoint, whenever I wind up talking about things on Twitter and speak in generalities, I get a whole bunch of people pushing back with a, “Well, what about this edge case? That renders your entire point invalid.” And, ugh, not really. It feels like one of the casualties of the pandemic has been a sense of community in a sense of humans relating to other humans. I think we're all tired of the Zoom calls from hell I got to see you a couple of weeks before this recording at Monktoberfest in Portland, Maine, and oh, my God, dealing with people face to face, it was so much richer, at least from my perspective, compared to everything that we've been able to do during the pandemic. Am I alone on that? Are you seeing this across the board? Where companies are talking about this?Alyss: I will say with confidence, you're not alone in this. Whether or not companies are talking about it is also across the board. How rich are those understandings? How rich are those conversations? Because trying to step back as a brand is not really a way.Like, having nuance, being real, been community members, like that's not a way in which I think companies can participate in a way that feels truly authentic. That's why you need faces. That's why you need people. That's why you need folks whose job it is to do this. But in terms of things are lost, like, Twitter is not the right place to be having these conversations. It's not the right place in which to necessarily relate to people, absolutely.When you get distilled down all of your interactions into oh, I've got a notification. Oh, I have a checkmark, and so I have, like, better moderation tools. Oh, like, I made a statement and I don't want to hear a solution for it. We get all of these, uncurated experiences that are so dissatisfying that it does make us miss being around people who can read body language, that can understand my immediate relationship to them in spaces that we choose to be in, whereas Twitter is this big panopticon where we can just get yelled at and yell at each other. And it loves to amplify those conversations far more than any of the touchy-feely, good news success stories.Corey: When you take a look across the entire landscape of managing DevRel programs and ensuring that companies are receiving value for it, and—by which I mean, nurturing the long-term health of communities because yes, I am much more interested in that than I am in next quarter's numbers, how do you see that evolving, particularly with the recent economic recession or correction or drawback or everything's on fire, depending upon who it is you talk to? How do you see that evolving?Alyss: It goes back to what I said earlier about, I can speak in generalities, there will be specifics to various organizations, but at a fundamental part, like, I'll kind of take a step back and maybe make some very strong statements about what I think DevRel is, in a regard, which is, without documentation, without support, you don't have a product. And if you don't have folks going out and understanding what it is your customers need, and especially when those customers are maybe all the time or sometimes developers, and understanding what it is that they're saying and truly how having empathy for what's going on in their day-to-day, what task are they trying to complete, how relevant is this to them, if you don't invest in that, when that happens, you've lost the plot. And so, in those instances, unfortunately, that's a conversation with leadership team. Your leadership doesn't fundamentally understand the value and maybe it's worth it to make the argument in favor of to illustrate that without this feedback loop, without this investment in the educational journey of developers, without the investment in what is going on in our product, and where have we allowed ourselves to remain ignorant of what is happening in the day-to-day of our users. We need those folks.Product managers are in sprints, they're in standups. They're doing, like, strategic planning and their yearly planning. We need a group who is rewarded to care about this but also is innately driven to do so as well. And that's not something that you can make. And it's not something that we otherwise see. It's part of why we have such an absence in good developer marketing is because marketers aren't paid well enough to ever have learned the skills to be developers, and so there's no skills transfer.Corey: One last topic that I want to get into something you've only been doing for a short while, but you've become an advisor at Heavybit, which is a VC firm. How did that come about and what do you do?Alyss: So currently, I—I'll do the super-high level. What I do right now is I host office hours with seed startups and Series A that are in the dev tool space. And we generally talk about developer relations, a little bit in developer marketing go-to-market strategies. And it's super enriching for me because I love hearing about different experiences and problems and, like, areas of practice. But it was really interesting, and a little bit of a make-your-own-luck-and-opportunity type deal.Where I live in Austin, Texas; I do not live in the Bay Area, I don't have all those connections, I've been a bit distant from it. And I saw someone who had accepted a role that I had interviewed for, end up in some of their content. And I was like, “They're doing a great job. They definitely deserve to be there, but I also had similar qualifications, so why should I also be there?” And I found someone, his name's Tim, on LinkedIn, who runs their events. And I reached out and I said, “Hey, Tim, how would you like a new advisor?” And so, Tim responded back and we—Corey: Knock knock. Who's there? It's me.Alyss: Yeah, exactly. It's—and it was just, I want this thing to happen. How do I make it happen? I ask.Corey: And what does it day-to-day that look like? How much time does it take? What do you do exactly?Alyss: Yeah. I mean, right now, it's about five hours every quarter. So, I spend anywhere between 30 minutes to an hour with various organizations that are a part of Heavybit's portfolio, talking with them through their motion to go general availability, or they want to start participating in events, or they want to discover what are the right events for them to—or, like, DevOpsDays, should we participate in that? Should we hire a DevRel person? Should we hire a product marketing person? Just helping them sort wheat from chaff in terms of, like, how to proceed.And so, it's relatively, for me, lightweight. And Heavybit also gives us the opportunity to contribute back in blog posts, participate in podcasts and be able to have some of those richer conversations. So, I have a set of bookmarks, so there's over 100, bookmarks long, that is fully curated across several different categories. That was my first blog post was diving into a few of those where I think are critical areas of developer relations. What are some of the conversations on DevRel metrics? How do I think about setting a DevRel strategy for the first time? How do I do my first DevRel hire? And so, I wouldn't even call it a second job. It's more of a getting to, again, enrich my own experience, see a wider variety of different problems in this space and expand my own understanding.Corey: I really want to thank you for being so generous with your time. If people want to learn more about what you're up to, how you view the world, and basically just come along for the ride as you continue to demonstrate a side of tech that I don't think we get to see very often, where can they find you?Alyss: I am@PreciselyAlyss on Twitter, as well as Twitch. Aside from that, I would not recommend looking for me.Corey: Excellent. Always a good decision. I will put links to that in the [show notes 00:30:00]. Thank you so much for your time. I appreciate it.Alyss: Thanks, Corey.Corey: Alyss Noland, Head of Developer Relations Relations and Product Marketing at Common Room. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry, insulting comment belittling community and letting the rest of us know by observation just why you've been thrown out of every community to which you've ever been a part.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

AWS Morning Brief
Four Announcements of the Boring Apocalypse

AWS Morning Brief

Play Episode Listen Later Jan 17, 2023 6:06


Links: Join Corey in Phoenix next Sunday at 1PM at Zuzu for a community meet-up. AWS Config supports 22 new resource types  Changes to AWS Billing, Cost Management, and Account Consoles Permissions Run a popular benchmark on Amazon Redshift Serverless easily with AWS Data Exchange How to optimize costs for grant-based research projects with AWS

AWS Morning Brief
Computers Checking Compliance Boxes

AWS Morning Brief

Play Episode Listen Later Jan 12, 2023 5:32


Links: CircleCI came out with a security alert urging you to rotate any secrets stored in CircleCI. Another bite at the craptastic LastPass breach response, this article parses their weak-sauce PR statement  Over the holidays Slack had some private GitHub code repositories stolen. ACSESSED is another Azure vulnerability Amazon S3 Encrypts New Objects By Default  Updated whitepaper available: AWS Security Incident Response Guide iamfast analyzes your application code to generate a least-privilege IAM policy. Wiz has come up with and open sourced PEACH, a tenant isolation framework for cloud applications.

Screaming in the Cloud
The Return of re:Invent with Pete Cheslock

Screaming in the Cloud

Play Episode Listen Later Jan 12, 2023 41:45


About PetePete is currently the Head of Growth And Community for AppMap, the open source dynamic runtime code analyzer. Pete also works with early stage startups, helping them navigate the complex world of early stage new product development.Pete also fully acknowledges his profile pic is slightly out of date, but has been too lazy to update it to reflect current hair growth trends.Links:AppMap: https://appmap.io/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: If you asked me to rank which cloud provider has the best developer experience, I'd be hard-pressed to choose a platform that isn't Google Cloud. Their developer experience is unparalleled and, in the early stages of building something great, that translates directly into velocity. Try it yourself with the Google for Startups Cloud Program over at cloud.google.com/startup. It'll give you up to $100k a year for each of the first two years in Google Cloud credits for companies that range from bootstrapped all the way on up to Series A. Go build something, and then tell me about it. My thanks to Google Cloud for sponsoring this ridiculous podcast.Corey: Cloud native just means you've got more components or microservices than anyone (even a mythical 10x engineer) can keep track of. With OpsLevel, you can build a catalog in minutes and forget needing that mythical 10x engineer. Now, you'll have a 10x service catalog to accompany your 10x service count. Visit OpsLevel.com to learn how easy it is to build and manage your service catalog. Connect to your git provider and you're off to the races with service import, repo ownership, tech docs, and more. Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn and this is probably my favorite recurring episode slash tradition, every year. I drag Pete Cheslock on who talks with me about his experience at re:Invent. Last year, Pete, you didn't go. The year before, none of us went because it was all virtual, but it feels like we're finally getting back into the swing of things. How are you, Pete?Pete: I am doing great. It is always a pleasure. It was amazing to see other humans in person at a industry event. As weird as it sounds to say that, you know, it was great to be in Vegas [laugh], it was mostly great, just because there were other humans there too that I wanted to see.Corey: Because this is going to confuse folks who haven't been following our various adventures, these days, you are the Head of Growth and Community at AppMap. But you and I have been talking for years and you did a stint working at The Duckbill Group here with us as a cloud economist. Ah, I miss those days. It was fun working with you and being able to bother you every day as opposed to just on special occasions like this.Pete: Yeah, I know. I got to slide into your Slack DMs in addition, and then when I didn't get a response, I would slide into your Twitter DMs. It worked out perfectly. So yeah, it's been a wild ride. I mean, I took an interlude from my startup journey by continually working at tech startups.And yeah, I got to join onboard the Duckbill and have, you know, a really wonderful time cutting bills and diving into all of the amazing parts of people's Amazon usage. But I am also equally broken in my brain, and continually said to myself, “Maybe I'll do another startup.” [laugh].Corey: Right. And it turns out that we're not a startup. Everyone likes to think we are. It's like, oh, okay—like Amazon, for example, has us historically in their startup division as far as how they—the buckets as they put different accounts into. And if you look at us through that lens, it's yeah, we're a specific kind of startups, specifically a failing startup—or failed—because to us growth is maybe we'll hire one or two people next year, as opposed to, “Oh, yeah, we're going to TEDx this place.” No, yeah, we're building a lifestyle business very much by design.Pete: I'd be very curious how many account managers actually Duckbill has kind of churned through because usually, you get to keep your account manager if you're growing at a pretty incredible clip. And it's kind of a bellwether for, like, how fast are we—are we growing so fast that we have kept our account manager for multiple years?Corey: Your timing is apt. We're a six-year-old company and I just met our fourth account manager last week.Pete: [laugh].Corey: No it's, honestly, what happens with AWS account managers is the only way you get to keep them is if your spend trajectory on AWS matches their career trajectory inside of AWS. Because if you outpace them, they'll give you to someone that they view as being more senior, whereas if they outpace you, they're going to stop dealing with the small accounts and move on to the bigger ones. Honestly, at this point, I've mostly stopped dealing with my account managers. I had one that was just spectacular. It was sad to see him get promoted; good for him.But I get tired of trying to explain the lunacy that is me to someone on the AWS side every year. It just doesn't make sense because my accounts are super weird and when they try and suggest the usual things that work for 99.995% of AWS customers and things they care about, it falls to custard when it comes to me specifically. And that's not on them; it's because I'm weird and broken.Pete: I'm remembering now one of the best account managers that I ever worked with at a startup, years and years ago. She was with us for a couple of years, pretty solidly. And then, you know, because careers are long and jobs are short, when I was at The Duckbill Group again, doing work, turns out she was the account manager on this other thing, you know? Which, like, looking at the company she was account manager for was like 500x [laugh] my previous company, so I was like, “Oh, yeah. You're clearly moving up in the world because my company did not 500x.” So, sometimes you got to chase the ones who are.Corey: So, let's talk about re:Invent. This felt like the first re:Invent post-pandemic. And let's be clear, I wound up getting Covid by the end, so I don't recommend that to everyone. But let's be clear, this was not a re:Invent were anyone officially accepted that Covid existed. I was one of the only people wearing masks to most of the events I was at. Great load of good that did me.But it was big. It was the usual sixty-some-odd-thousand people that had been in previous years, as opposed to the hard cap of 30 or so that they had last year so it felt smaller and more accessible. Nope. Right back to bizarre numbers of people. But fewer sponsors than most years, so it felt like their budget was severely constrained. And they were trying to have not as many sponsors, but still an overwhelming crush of attendees. It felt odd, but definitely very large scale.Pete: Yeah, I can echo that a hundred percent. I'm sure we've talked about this in previous ones, but I've had the pleasure—well, I don't know, some might call it not a pleasure, but it's been a pleasure to watch re:Invent grow over so many years. I went to the first re:Invent. A company I was at actually sponsored it. And remembering back to that first re:Invent, it was kind of quaint by comparison.There were 4000 people at the first re:Invent, which again, it's a big conference, especially when a lot of the conferences that I think I was really attending at the time were like, you know, 600, 1000, maybe tops. To go to a 4000-person event in Vegas especially, it's again, in the same Expo Hall it's been since that first one, it still felt big. But every person stayed in the Venetian. Pretty much everyone was in the same hotel, all of the attendees that year. All the talks were there.There was, you know, a lot [laugh]—I mean, a lot less of everything that was there. And so, watching it grow over time, not only as a sponsor because I've actually been—kind of worked re:Invent as a, like, a booth person for many of these years for multiple different sponsors and had to coordinate that aspect of it, but then also a couple of times just being more, like, attendee, right, just someone who could go and kind of consume the content. This year was more on the side of being more of an attendee where I got to just kind of experience the Expo Hall. You know, I actually spent a lot of time in the Expo Hall because a big part of why I was there was—Corey: To get t-shirts.Pete: Yeah, we'll get to—I was running low on not only t-shirts but socks. My socks were really worse for wear the last few years. I had to, like, re-up that, right [laugh]?Corey: Yeah, you look around. It's like, “Well, none of you people have, like, logoed pants? What's the deal here? Like, I have to actually buy those myself. I don't—I'm not here to spend money.”Pete: Yeah, I know. So. And so yeah, this year, it felt—it was like Covid wasn't a thing. It wasn't in anyone's mind. Just walking around—Vegas in general, obviously, it's kind of in its own little bubble, but, you know, I've been to other events this year that were much more controlled and had a lot more cautious attendees and this was definitely not like that at all. It felt very much, like the last one I was at. The last one I was at was 2019 and it was a big huge event with probably 50,000-plus people. And this one felt like to me at least, attendee-wise, it definitely felt bigger than that one in a lot of ways.Corey: I think that when all is said and done, it was a good event, but it wasn't necessarily what a lot of folks were expecting. What was your take on the content and how the week played out?Pete: Yeah, so I do, in many ways, kind of miss [laugh] the event of yore that was a little bit more of a targeted, focused event. And I understand that it will never be that kind of event anymore. Maybe they start splitting it off to be, you know, there's—just felt much more like a builder event in previous years. The content in the keynotes, you know, the big keynotes and things like that would be far more, these big, iterative improvements to the cloud. That's something that always felt kind of amazing to see. I mean, for years and years, it was like, “Who's ready for another re:Invent price drop?” Right? It was all about, like, what's the next big price drop going to be?Corey: Was it though because I never was approaching with an eye toward, “Oh, great. What are they going to cut prices on now?” That feels like the least interesting things that ever came out of re:Invent, at least for me. It's, what are they doing architecturally that lets me save money, yes. Or at least do something interesting architecturally, great. I didn't see Lambda when it first came out, for example, as a cost opportunity, although, of course, it became one. I saw it as this is a neat capability that I'm looking forward to exploring.Pete: Yeah, and I think that's what was really cool about some of those early ones is these, like, big things would get released. Like, Lambda was a big thing that got released. There was just these larger types of services coming out. And I think it's one of your quotes, right? Like, there's a service for everyone, but every service isn't for everyone.Corey: Yeah.Pete: And I feel like, you know, again, years ago, looking back, it felt like more of the services were more geared towards the operational, the early adopters of Amazon, a lot of those services was for those people. And it makes sense. They got to spread out further, they've got to have kind of a wider reach to grow into all of these different areas. And so, when they come out with things that, yeah, to me, I'm like, “This is ridiculous feature. Who would ever use this?” Like, there's probably a dozen other people at different companies that are obscenely excited because they're at some enterprise that has been ignored for years and now finally they're getting the exact tooling that they need, right?Corey: That made sense for a long time. I think that now, the idea that we're going to go and see an Andy Jassy-era style feature drop of, “Here's five new databases and a whole new compute platform and 17,000 more ways to run containers,” is not necessarily what is good for the platform, certainly not good for customers. I think that we're seeing an era of consolidation where, okay, you have all these services to do a thing. How do I pick which one to use? How do I get onto a golden path that I can also deviate from without having to rebuild everything? That's where customers seem to be. And it feels like AWS has been relatively slow to acknowledge or embrace that to me.Pete: Yeah, a lot of the services, you know, are services they're probably building for just their own internal purposes, as well. You know, I know, they are for a while very motivated to get off anything Oracle-related, so they started building these services that would help migrate, you know, away from Oracle because they were trying to do it themselves. But also, it's like, there's still—I mean, I talk with friends of mine who have worked at Amazon for many years and I'm always fascinated by how excited they are still to be there because they're operating at a scale that just doesn't exist anywhere else, right? It's like, they're off on their lone island that go into work somewhere else is almost going backwards because you've already solved problems at this lower level of scale. That's obviously not what you want to be doing anymore.And at the scale that they're at for some of these services, even like the core services, the small improvements they're making, they seem so simple and basic, like a tiny EBS improvement, you're like, “Ugh, that's so boring.” But at their level of scale for, like, something like an EBS, like one of those top five services, the impact of that tiny little change is probably even so amazingly impactful. Like it's just so huge [laugh], you know, inside that scope of the business that is just—that's what—if you really start pulling the thread, you're like, “Wow, actually, that is a massive improvement.” It just doesn't feel that way because it's just oh, it's just this tiny little thing [laugh]. It's like, just almost—it's too simple. It's too simple to be complex, except at massive scale.Corey: Exactly. The big problem I ran into is, I should have noticed it last year, but it was Adam Selipsky's first re:Invent and I didn't want to draw too many conclusions on it, but now we have enough dots to make a line—specifically two—where he is not going to do the Andy Jassy thing of getting on stage and reading off of a giant 200 item list of new feature and service announcements, which in AWS parlance, are invariably the same thing, and they wind up rolling all of that out. And me planning my content schedule for re:Quinnvent around that was a mistake. I had to cancel a last-minute rebuttal to his keynote because there was so little there of any substance that all would have been a half-hour-long personal attack and I try not to do that.Pete: Mmm. Yeah, the online discussion, I feel like, around the keynote was really, like, lackluster. It was yeah, like you said, very devoid of… not value; it's not really the right word, but just substance and heft to it. And maybe, look, we were just blessed with many, many years of these dense, really, really dense, full keynotes that were yeah, just massive feature drops, where here's a thing and here's a thing, and it was almost that, like, Apple-esque style kind of keynote where it was like, we're just going to bombard you with so many amazing things that kind of is in a cohesive storyline. I think that's the thing that they were always very good about in the past was having a cohesive story to tell about all of these crazy features.All of these features that they were just coming out with at this incredible velocity, they could weave the story around it. And you felt like, yeah, keynote was whatever hour, two hours long, but it would go by—it always felt like it would go by quickly because they were just they had down kind of really tight messaging and kept your attention the whole way through because you were kind of like, “Well, what's next? There's always—there's more. There's got to be more.” And there would be, right? There would be that payoff.Corey: I'm glad that they recognized that what got them here won't get them there, but I do wish that they had done a better job of signaling that to us in more effective ways. Does that make any sense?Pete: Yeah, that's an interesting… it's kind of an interesting thought exercise. I mean, you kind of mentioned before earlier, before we started recording, the CMO job is still available, it's still open [laugh] at AWS. So, if this was a good way to attract a top-tier CMO, I'd almost feel like if you were that person to come in and be like, “Hey, this did not work. Here are the following reasons and here's what you need to do to improve it.” Like, you might have a pretty solid shot of landing that role [laugh].Corey: Yeah, I'm not trying to make people feel intentionally bad over it. This stuff is very hard, particularly at scale. The problem I had with his keynote was not in fact that he was a bad speaker, far from it. He was good a year ago, he's clearly put work and energy into becoming better over the past year. From a technical analysis of how is Adam Selipsky as a public speaker, straight A's as far as I'm concerned, and I spent a lot of time focusing on this stuff myself as a professional speaker myself. I have no problems with how he wound up delivering any of the content. My problem was with the content itself. It feels like he was let down by the content team.Pete: Yeah, it definitely felt not as dense or as rich as we had come to expect in previous years. I don't think it was that the content didn't exist. It's not like they didn't build just as much, if not way, way more than they have in previous years. It just seemed to just not be part of the talk.I don't know. I always kind of wonder, too, is this just an audience thing? Which is, like, maybe I'm just not the audience for his talk, right? Was there someone else in that Expo Hall, someone else watching the stream, that was just kept on the edge of their seat hearing these stories? I don't know. I'm really kind of curious. Like, you know, are we only representing this one slice of the pie, basically?Corey: I think part of the problem is that re:Invent has grown so big, that it doesn't know what it wants to be anymore. Is it a sales event? By the size of the Expo Hall, yeah, it kind of is. Is it a partner expo where they talk about how they're going to milk various companies? Possibly. There's certainly one of those going on.There was an analyst summit that I attended for a number of days during re:Invent this year. They have a whole separate thing for press. The community has always been a thriving component of re:Invent, at least for me, and seeing those folks is always terrific. Is it supposed to be where they dump a whole bunch of features and roadmap information? Is it time for them to wind up having executive meetings with their customers? It tries to be all of those things and as a result, at this scale it feels like it is failing to be any of them, at least in an effective, cohesive way.Pete: Yeah, and you really nailed each of the personas, like, of a re:Invent attendee. I've talked to many people who are considering going to re:Invent, and they're, “I don't really know if I want to go, but I really want to go to some sessions, and I really want to do this.” And I always have to kind of push back and say, “Look, if you're only going there to attend talks,” like, just don't bother, right? As everyone knows, the talks are all recorded, you can watch them later. I did have conversations with some engineer, principal engineer level software folks that were there and the prevailing consensus from chatting with those folks, kind of anecdotally, is that, like, they had actually a lot of struggles even getting into some of these sessions, which for anyone who has been to re:Invent in the last, I don't know, four or five years, like, it's still a challenge, right?There's—you got to register for a lot of these talks way far in advance, there'll be a standby list, there'll be a standby line. It's a lot of a lot. And so, there's not usually a ton of value there. And so, I always try to say, like, “If you're going to re:Invent your, kind of, main purpose to go would be more for networking,” or just you're going because of the human interaction that you hope to get out of it, right, the high bandwidth conversations that are really hard to do in other areas. And I think you've nailed a bunch of those, right? Like, an analyst briefing is really efficient if you can get all the analysts in a room versus doing one-off analyst meetings.Meeting with big enterprises and hearing their thoughts and feelings and needs and requirements, you can get a lot of those conversations. And especially, too, if, like, talking to an enterprise and they got a dozen people all spread over the world, well you can get them all in one room, like, that's pretty amazing in this world. And then on the sales side, I feel like granted, I spent most of my time in the Expo Hall, but that was probably the area that I think you said earlier which I really picked up on, which was the balance between sponsors and attendees felt out of whack. Like it felt like there were way more attendees than the sponsors that I would have kind of expect to see.Now, there were a lot of sponsors on that Expo Hall and it took days. I mean, I was on the Expo Hall for days walking around and chatting with different companies and people. But one of the things that I saw that I have never seen before was a number of sponsorship booths, right—and these are booths that are, like, prebuilt, ten by ten-foot size or the smaller ones—that were blanks. They were like, you know, like, in a low-quality car where you have blank buttons that, like, if you paid more you get that feature. Walking around, there was a nonzero number of just straight-up empty booth, blank booths around which, I don't know, like, that felt kind of telling. Like, did they not sell all their sponsorships? Has that ever happened? I don't even know. But this was I felt like the first I've had—Corey: Or did companies buy the sponsorship and then realize that it was so expensive to go on top of it, throwing bad money after good might not have made sense. Because again, when people—Pete: Right.Corey: —brought out these sponsorships, in many cases, it was in the very early days of the growing recession we've seen now. And they may have been more optimistic, their circumstances may certainly have changed. I do know that pricing for re:Invent sponsorships was lower this past year than it had been in previous years. In 2019, for example, they had two Expo Halls, one at the ARIA Quad and the other at the Venetian. They had just one this year, which made less running around on my part, but still.Pete: Yeah, I do remember that, that they had so many sponsors. What I would say about the sponsors that there's two parts of this that were actually interesting. One, you're definitely right. As someone who has sponsored re:Invent before and has had to navigate that world, you are likely going to commit to the sponsorship as early as June, you know, could even be earlier than June depending on how big of a thing that you're doing. But it's early. It's usually in the summertime that you're—if you haven't made a decision by the summertime, like, you could actually not get a booth, right?And this was, I remember, the last one that I had sponsored was maybe 2018, 2019. And, like, you don't want those last few booths. Like, they put you in the back and not a good way. But going there, were a lot of—I did notice a lot of booths that had pretty massive layoffs who still had the booths, you know, and again, large booths, large companies, which again, same thing. I kind of am like, “Wow, like, how many employees did that booth cost you, right?”Because like [laugh], some of these booths are hundreds of thousands of dollars to sponsor. And then the other thing that I actually noticed, too, which I was honestly a little surprised by, with the exception of the Datadog booth; I love my friends at Datadog, they have the most amazingly aggressive booth BDRs who are always just, they'll get you if you're, like, hovering near them. And there's always someone to talk to over there. Like, they staff it really, really well. But there were some other booths that I was actually really interested in talking to some of the people to learn about their technology, that I actually waited to talk to someone. Like, I waited for someone to talk to, and then finally I'm like, “You know what? I'm going to come back.”And then I came back and waited again. So, it's like, how many of these sponsors obviously spent a lot of money to go there, then months later, they start looking at the people that they have to support this, they've already had some layoffs and probably sent a much smaller audience there to actually, like, operate the booth.Corey: This episode is sponsored in part by Honeycomb. When production is running slow, it's hard to know where problems originate. Is it your application code, users, or the underlying systems? I've got five bucks on DNS, personally. Why scroll through endless dashboards while dealing with alert floods, going from tool to tool to tool that you employ, guessing at which puzzle pieces matter? Context switching and tool sprawl are slowly killing both your team and your business. You should care more about one of those than the other; which one is up to you. Drop the separate pillars and enter a world of getting one unified understanding of the one thing driving your business: production. With Honeycomb, you guess less and know more. Try it for free at honeycomb.io/screaminginthecloud. Observability: it's more than just hipster monitoring.Corey: One bright light that I did enjoy and I always enjoy, though I'm not certain how actionable it is in the direct sense, was Peter DeSantis' Monday Night Live keynote. It was great. I mean, the one criticism I had of it—on Twitter at the time, before that thing melted and exploded—was that it was a bit mislabeled because it really should have been what it turned into midway through of surprise computer science lecture with Professor DeSantis. And I was totally there for it. But it was fun just watching some of the deep magic that makes this all work be presented in a way that most of us normally don't get to see.Pete: Well, this was the first year they did not do their Midnight Madness over-the-top kind of thing. And I also I don't recall that I saw them doing one of the other things I feel like is at night is they're, like, giant wing-eating competition. Am I wrong? Did they do that this year and I just missed hearing about it?Corey: They did not. Turns out that competitive Gluttony is not as compelling as it once was. But they also canceled their Midnight Madness event a month or two before re:Invent itself. What was super weird to me was that there was no event—community or otherwise—that sprung forth to seize that mantle. So, you had a whole bunch of people who were used to going for several hours that night to a big event with nothing to do.And at 9 p.m. they started just dumping a whole bunch of service releases in their blog and RSS feeds and the rest, and it just felt very weird and discordant. Like, do they think that we have nothing better to do than sit here and read through this on a Sunday night where we would have otherwise been at a party? Well yeah, in my case, I'm super sad and of course, I had nothing better to do that night. But most people had things going on.Pete: Yeah. Yeah, exactly. I think also, if you—maybe it's a little bit better now but I don't know when you have to buy that many chicken wings in advance, but with supply chains being what they are and the cost of chicken wings, I mean, not that I track the cost of chicken wings, except I absolutely do every time I go to Costco, they're up substantially. So, that was probably a contributing factor to the wing-eating contest: supply chain pain and suffering. But yeah, it's really interesting that just even in what some of the sponsors kind of were doing this year over previous years, I doubt they did this in 2021—but maybe, I don't know—but definitely not in 2019, something that I don't recall to this level was the sponsors essentially booking out entire restaurants near the venue every single day of the conference.And so again, if you were at this event like we were, and you at the end of the thing, were just like, I just want to sit and I've got a handful of friends, I want to sit and, like, have a drink, and just, like, chat and catch up and hear how the day went and everything else, finding a place to actually go to do that was very, very hard to do. And the thing that I noticed was—again, seemed like it was new this year; I don't recall it in 2019 to this level is, there were a lot of the big sponsors that had just booked a whole restaurant, breakfast, lunch, and dinner, like, from open to close, fully booked it, which was honestly, brilliant.Corey: Oh, yeah. If you bring 200, 300 people to an event, you've got to feed him somehow. And, “Hey, can we just rent out your restaurant for the entirety of this week?” Is not out of the question compared to what you'd even spend just reimbursing that sea of people to go and eat somewhere else.Pete: Exactly. The reason—I'm approaching this from, like, a business perspective—if I had a large group of enterprise salespeople and they need a place to book meetings, well, it's super compelling if I'm being courted by one of these salespeople and they're like, “Hey, come and have breakfast. Come and grab a coffee.” You know, and there's a place where you can sit down and quietly enjoy that meal or coffee while having a sale. Like, I'll have that sales conversation and I'm going to be way more motivated to show up to it because you're telling me it's like, this is where we're going to meet.Versus some of my friends were trying to, like, coordinate a lunch or a coffee and it's like, do we want to go to the Starbucks that has 500 people in line or do we want to walk four hotels, you know, down the street to find a bar that has video poker that no one will be sitting at and that we can just sit down and talk, right? It kind of felt like those were your two options.Corey: One thing that MongoDB did is rented out the Sugarcane restaurant. And they did this a couple of years in a row and they wound up effectively making it available to community leaders, heroes, and whatnot, for meetings or just a place to sit down and catch your breath. And I think that was a brilliant approach. You've gone to the trouble of setting this thing up for meetings for your execs and whatnot. Why not broaden it out?You can't necessarily do it for everyone, for obvious reasons, but it was nice to just reach out to folks in your orbit and say, “Yeah, this is something available to you.” I thought that was genius. And I—Pete: Oh yeah.Corey: —wish I thought of doing something like that. Let's be clear. I also wish I had rent-out-Sugarcane-for-a-week budget. But you know—Pete: [laugh].Corey: —we take what we can get.Pete: Yeah. That'll be a slight increase to the Spite Budget to support that move.Corey: Just a skosh, yeah.Pete: Yeah, the MongoDB, they were one that I do remember had done it similarly. I don't know if they had done it, kind of, full-time before, but a friend of mine work there, had invited me over and said, “Hey, like, come by, let's grab a drink. You know, we've got this hotel, you know, this restaurant kind of booked out.” And that was back in 2019. Really enjoyed it.And yeah, I noticed it was like, you know, basically, they had this area available, again, a place to sit down, to open your laptop, to respond to some emails, making it available to community people should have been a no-brainer to, really, all of these other sponsors that may have times of less kind of attendance, right? So obviously, at any of the big meals, maybe that's when you can't make it available for all the people you want to, but there's going to be off hours in between times that making that available and offering that up generates a supreme amount of goodwill, you know, in the community because you know, you're just looking for a place to sit out and drink some water [laugh].Corey: Yeah, that was one challenge that I saw across the board. There were very few places to just sit and work on something. And I'm not talking a lounge everywhere around every corner was needed necessarily or even advisable. No, the problem I've got was that I just wanted to sit down for two or three minutes and just type up an email quickly, or a tweet or something, and nope, you're walking and moving the whole time.Pete: Yeah. Now honestly, this would be a—this was a big missed opportunity for the Amazon event planning folks. There was a lot of unnecessary space usage that I understand why they had it. Here's an area you could play Foursquare, here's an area that had seesaws that you could sit on. Like, just, I don't know, kitschy stuff like that, and it was kind of off to the side or whatever.Those areas honestly, like, we're kind of off to the side, they were a little bit quieter. Would have been a great spot to just, like, load up some chairs and couches and little coffee tables and just having places that people could sit down because what ended up happening—and I'm sure you saw it just like I did—is that any hallway that had somewhere that you could lean your back against had a line of people just sitting there on their laptops because again, a lot of us are at this event, but we're also have jobs that we're working at, too, and at some point during the day, you need to check in, you need to check some stuff out. It felt like a lack of that kind of casual space that you can just relax in. And when you add on top of all the restaurants nearby being essentially fully booked, it really, really leaves you hanging for any sort of area to sit and relax and just check a thing or talk to a person or anything like that.Corey: Yeah, I can't wait to see what lessons get learned from this and how it was a mapping to next year, across the board. Like, I have a laundry list of things that I'm going to do differently at re:Invent next year. I do every year. And sometimes it works out; sometimes it really doesn't. And it's a constant process of improvement.I mean, one of the key breakthroughs for me was when I finally internalized the idea that, yeah, this isn't going to be like most jobs where I get fired in the next six months, where when I'm planning to go to re:Invent this is not the last re:Invent I will be at in my current capacity, doing what I do professionally. And that was no small thing. Where oh, yeah. So, I'm already making plans, not just for next re:Invent, but laying the groundwork for the re:Invent after that.Pete: Yeah, I mean, that's smart way to do it. And especially, too, when you don't consider yourself an analyst, even though you obviously are an analyst. Maybe you do consider yourself an analyst, but you're [laugh] more, you know, you're also the analyst who will go and actually use the product and start being like, “Why does this work the way it does?” But you're kind of a little bit the re:Invent target audience in a lot of ways, right? You're kind of equal parts on the analyst expert and user as well. It's like you kind of touch in a bunch of those areas.But yeah, I mean, I would say the one part that I definitely enjoyed was the nature walk that you did. And just seeing the amount of people that also enjoyed that and came by, it was kind of surreal to watch you in, like, full safari garb, basically meandering through the Expo Hall with this, like, trail of, like, backpacks [laugh] following you around. It was a lot of fun. And, you know, it's like stuff like that, where people are looking for interesting takes on, kind of, the state of something that is its own organism. Like, the Expo Hall is kind of its own thing that is outside of the re:Invent control. It's kind of whatever is made up by the people who are actually sponsoring it.Corey: Yeah, it was neat to see it play out. I'm curious to see how it winds up continuing to evolve in future years. Like right now, the Nature Walk is a blast, but it was basically at the top and I had something like 50 people following me around at one point. And that is too big for the Expo Hall. And I'm not there to cause a problem for AWS. Truly, I'm not. So, I need to find ways to embrace that in ways that don't kill the mojo or the energy but also don't create problems for, you know, the company whose backup I am perched upon, yelling more or less ridiculous things.Pete: [laugh]. I think it was particularly interested in how many people I'd be walking by and every once a while I would see, like, a friend of mine, someone actually working one of the booths and just be like, “What's going on here?” Like, I know one of my friends even said, “Yeah, like, nothing draws a crowd like a crowd.” And you can almost see more people [laugh] just, like, connecting themselves onto this safari train moving their way through. Yeah, it's a sight to see, that's for sure [laugh].Corey: Yeah, I'll miss aspects of this. Again, nothing can ever stay the same, on some level. You've got to wind up continuing to evolve and grow or you wind up more or less just frozen in place[ and nothing great ever happens for you.Pete: Yeah, I mean, again, Expo Hall has gone through these different iterations, and I—you know, when it does come to the event, as I kind of think back, I probably have spent most of my time actually in the Expo Hall, usually just related to the fact that, like, when you're a sponsor, like, you're just—that's where you're at. For better or worse, you're going to be in there. And especially if you're a sponsor, you want to check out what other sponsors are doing because you want to get ideas around things that you might want to try in later years. I mentioned Datadog before because Datadog to this day continues to have the best-designed booth ever, right? Like when it comes to a product that is highly demoable, I've been myself as a sponsor, it has always been a struggle to have a very effective demo setup.And I actually remember, kind of, recommending to a startup that I was at years ago, I'm doing a demo setup that was very, very similar to how Datadog did it because it was brilliant, where you have this, like, octagon around a main area of tables, and having double-sided demo stations. A lot more people are doing this now, but again, as I walked by I was again reminded just how effective that setup is because not only do you have people that just they don't want to talk, they just want to look, and they can kind of safely stand there and look, but you also have enough people staffing the booth for conversations that for, like me who actually might want to ask for questions, I don't have to wait and I can get an answer and be taken care of right away, versus some other booths. This year, one of the areas that I actually really enjoyed—and I don't even know the details of, like, how it all came about—but it looked like some sort of like Builder's Expo. I don't know if you remember walking by there, but there was a whole area of different people who had these little IoT or various powered things. One of them was, like, a marble sorting thing that was set up with a bunch of AWS services. I think there was like the Simple Beer Service V… four or five at this point. I had one of those iterations.It was some sort of mixture between Amazon software services that were powering these, like, physical things that you can interact with. But what was interesting is like, I have no idea, like, how it was set up, and who—I'm assuming it was Amazon specific—but each of these little booths were like chocked up with information about who they were and what they built, which gave it a feel of, like, this was like a last-minute builder event thing. It didn't feel like it was a highly produced thing. It had a much more casual feel to it, which honestly got me more interested to spend time there and check out the different booths.Corey: It was really nice to be able to go and I feel like you got to see all of the booths and whatnot. I know in previous years, it feels like you go looking for specific companies and you never find them. And you thought, “Oh”—Pete: [laugh].Corey: —“They must not have been here.” You find out after the fact, oh no, just you were looking in the wrong direction because there was so much to see.Pete: There were definitely still a couple of those. I had a list of a handful of booths I wanted to stop by, either to say hi to someone I knew who was going to be there or just to chat with them in general, there was a couple that I had to do a couple loops to really track them down. But yeah, I mean, it didn't feel as overly huge as a previous one, or as previous ones. I don't know, maybe it was like the way they designed it, the layout was maybe a little bit more efficient so that you could do loops through, like, an outer loop and an inner loop and actually see everything, or if it just was they just didn't have enough sponsors to truly fill it out and maybe that's why it felt like it was a little bit more approachable.I mean, it was still massive. I mean, it was still completely over the top, and loud and shiny lights and flashing things and millions of people. But it is kind of funny that, like, if you do enough of these, you can start to say, “Oh well, I don't know, it's still felt… a little bit less [laugh] for some reason.”Corey: Yeah, just a smidgen. Yeah. Pete, it is always a pleasure to get your take on re:Invent and see what you saw that I didn't and vice versa. And same time next year, same place?Pete: Yeah. I mean, like I said, one of my favorite parts of re:Invent is, you know, we always try to schedule, like, an end-of-event breakfast when we're both just supremely exhausted. Most of us don't even have a voice by the end. But just being able to, like, catch up and do our quick little recap and then obviously to be able to get on a podcast and talk about it is always a lot of fun. And yeah, thanks again for having me. This is—it's always, it's always a blast.Corey: It really is. Pete Cheslock, Head of Growth and Community at AppMap. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice and then put something insulting about me in the next keynote because you probably work on that content team.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Screaming in the Cloud
Exposing Vulnerabilities in the World of Cloud Security with Tim Gonda

Screaming in the Cloud

Play Episode Listen Later Jan 10, 2023 33:23


About TimTim Gonda is a Cloud Security professional who has spent the last eight years securing and building Cloud workloads for commercial, non-profit, government, and national defense organizations. Tim currently serves as the Technical Director of Cloud at Praetorian, influencing the direction of its offensive-security-focused Cloud Security practice and the Cloud features of Praetorian's flagship product, Chariot. He considers himself lucky to have the privilege of working with the talented cyber operators at Praetorian and considers it the highlight of his career.Tim is highly passionate about helping organizations fix Cloud Security problems, as they are found, the first time, and most importantly, the People/Process/Technology challenges that cause them in the first place. In his spare time, he embarks on adventures with his wife and ensures that their two feline bundles of joy have the best playtime and dining experiences possible.Links Referenced: Praetorian: https://www.praetorian.com/ LinkedIn: https://www.linkedin.com/in/timgondajr/ Praetorian Blog: https://www.praetorian.com/blog/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Thinkst Canary. Most Companies find out way too late that they've been breached. Thinkst Canary changes this. Deploy Canaries and Canarytokens in minutes and then forget about them. Attackers tip their hand by touching 'em giving you the one alert, when it matters. With 0 admin overhead and almost no false-positives, Canaries are deployed (and loved) on all 7 continents. Check out what people are saying at canary.love today!Corey: Kentik provides Cloud and NetOps teams with complete visibility into hybrid and multi-cloud networks. Ensure an amazing customer experience, reduce cloud and network costs, and optimize performance at scale — from internet to data center to container to cloud. Learn how you can get control of complex cloud networks at www.kentik.com, and see why companies like Zoom, Twitch, New Relic, Box, Ebay, Viasat, GoDaddy, booking.com, and many, many more choose Kentik as their network observability platform. Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Every once in a while, I like to branch out into new and exciting territory that I've never visited before. But today, no, I'd much rather go back to complaining about cloud security, something that I tend to do an awful lot about. Here to do it with me is Tim Gonda, Technical Director of Cloud at Praetorian. Tim, thank you for joining me on this sojourn down what feels like an increasingly well-worn path.Tim: Thank you, Corey, for having me today.Corey: So, you are the Technical Director of Cloud, which I'm sort of short-handing to okay, everything that happens on the computer is henceforth going to be your fault. How accurate is that in the grand scheme of things?Tim: It's not too far off. But we like to call it Praetorian for nebula. The nebula meaning that it's Schrödinger's problem: it both is and is not the problem. Here's why. We have a couple key focuses at Praetorian, some of them focusing on more traditional pen testing, where we're looking at hardware, hit System A, hit System B, branch out, get to goal.On the other side, we have hitting web applications and [unintelligible 00:01:40]. This insecure app leads to this XYZ vulnerability, or this medical appliance is insecure and therefore we're able to do XYZ item. One of the things that frequently comes up is that more and more organizations are no longer putting their applications or infrastructure on-prem anymore, so therefore, some part of the assessment ends up being in the cloud. And that is the unique rub that I'm in. And that I'm responsible for leading the direction of the cloud security focus group, who may not dive into a specific specialty that some of these other teams might dig into, but may have similar responsibilities or similar engagement style.And in this case, if we discover something in the cloud as an issue, or even in your own organization where you have a cloud security team, you'll have a web application security team, you'll have your core information security team that defends your environment in many different methods, many different means, you'll frequently find that the cloud security team is the hot button for hey, the server was misconfigured at one certain level, however the cloud security team didn't quite know that this web application was vulnerable. We did know that it was exposed to the internet but we can't necessarily turn off all web applications from the internet because that would no longer serve the purpose of a web application. And we also may not know that a particular underlying host's patch is out of date. Because technically, that would be siloed off into another problem.So, what ends up happening is that on almost every single incident that involves a cloud infrastructure item, you might find that cloud security will be right there alongside the incident responders. And yep, this [unintelligible 00:03:20] is here, it's exposed to the internet via here, and it might have the following application on it. And they get cross-exposure with other teams that say, “Hey, your web application is vulnerable. We didn't quite inform the cloud security team about it, otherwise this wouldn't be allowed to go to the public internet,” or on the infrastructure side, “Yeah, we didn't know that there was a patch underneath it, we figured that we would let the team handle it at a later date, and therefore this is also vulnerable.” And what ends up happening sometimes, is that the cloud security team might be the onus or might be the hot button in the room of saying, “Hey, it's broken. This is now your problem. Please fix it with changing cloud configurations or directing a team to make this change on our behalf.”So, in essence, sometimes cloud becomes—it both is and is not your problem when a system is either vulnerable or exposed or at some point, worst case scenario, ends up being breached and you're performing incident response. That's one of the cases why it's important to know—or important to involve others in the cloud security problem, or to be very specific about what the role of a cloud security team is, or where cloud security has to have certain boundaries or has to involve certain extra parties have to be involved in the process. Or when it does its own threat modeling process, say that, okay, we have to take a look at certain cloud findings or findings that's within our security realm and say that these misconfigurations or these items, we have to treat the underlying components as if they are vulnerable, whether or not they are and we have to report on them as if they are vulnerable, even if it means that a certain component of the infrastructure has to already be assumed to either have a vulnerability, have some sort of misconfiguration that allows an outside attacker to execute attacks against whatever the [unintelligible 00:05:06] is. And we have to treat and respond our security posture accordingly.Corey: One of the problems that I keep running into, and I swear it's not intentional, but people would be forgiven for understanding or believing otherwise, is that I will periodically inadvertently point out security problems via Twitter. And that was never my intention because, “Huh, that's funny, this thing isn't working the way that I would expect that it would,” or, “I'm seeing something weird in the logs in my test account. What is that?” And, “Oh, you found a security vulnerability or something akin to one in our environment. Oops. Next time, just reach out to us directly at the security contact form.” That's great. If I'd known I was stumbling blindly into a security approach, but it feels like the discovery of these things is not heralded by an, “Aha, I found it.” But, “Huh, that's funny.”Tim: Of course. Absolutely. And that's where some of the best vulnerabilities come where you accidentally stumble on something that says, “Wait, does this work how—what I think it is?” Click click. Like, “Oh, boy, it does.”Now, I will admit that certain cloud providers are really great about with proactive security reach outs. If you either just file a ticket or file some other form of notification, just even flag your account rep and say, “Hey, when I was working on this particular cloud environment, the following occurred. Does this work the way I think it is? Is this is a problem?” And they usually get back to you with reporting it to their internal team, so on and so forth. But let's say applications are open-source frameworks or even just organizations at large where you might have stumbled upon something, the best thing to do was either look up, do they have a public bug bounty program, do they have a security contact or form reach out that you can email them, or do you know, someone that the organization that you just send a quick email saying, “Hey, I found this.”And through some combination of those is usually the best way to go. And to be able to provide context of the organization being, “Hey, the following exists.” And the most important things to consider when you're sending this sort of information is that they get these sorts of emails almost daily.Corey: One of my favorite genre of tweet is when Tavis Ormandy and Google's Project Zero winds up doing a tweet like, “Hey, do I know anyone over at the security apparatus at insert company here?” It's like, “All right. I'm sure people are shorting stocks now [laugh], based upon whatever he winds up doing that.”Tim: Of course.Corey: It's kind of fun to watch. But there's no cohesive way of getting in touch with companies on these things because as soon as you'd have something like that, it feels like it's subject to abuse, where Comcast hasn't fixed my internet for three days, now I'm going to email their security contact, instead of going through the normal preferred process of wait in the customer queue so they can ignore you.Tim: Of course. And that's something else you want to consider. If you broadcast that a security vulnerability exists without letting the entity or company know, you're also almost causing a green light, where other security researchers are going to go dive in on this and see, like, one, does this work how you described. But that actually is a positive thing at some point, where either you're unable to get the company's attention, or maybe it's an open-source organization, or maybe you're not being fully sure that something is the case. However, when you do submit something to the customer and you want it to take it seriously, here's a couple of key things that you should consider.One, provide evidence that whatever you're talking about has actually occurred, two, provide repeatable steps that the layman's term, even IT support person can attempt to follow in your process, that they can repeat the same vulnerability or repeat the same security condition, and three, most importantly, detail why this matters. Is this something where I can adjust a user's password? Is this something where I can extract data? Is this something where I'm able to extract content from your website I otherwise shouldn't be able to? And that's important for the following reason.You need to inform the business what is the financial value of why leaving this unpatched becomes an issue for them. And if you do that, that's how those security vulnerabilities get prioritized. It's not necessarily because the coolest vulnerability exists, it's because it costs the company money, and therefore the security team is going to immediately jump on it and try to contain it before it costs them any more.Corey: One of my least favorite genres of security report are the ones that I get where I found a vulnerability. It's like, that's interesting. I wasn't aware that I read any public-facing services, but all right, I'm game; what have you got? And it's usually something along the lines of, “You haven't enabled SPF to hard fail an email that doesn't wind up originating explicitly from this list of IP addresses. Bug bounty, please.” And it's, “No genius. That is very much an intentional choice. Thank you for playing.”It comes down to also an idea of whenever I have reported security vulnerabilities in the past, the pattern I always take is, “I'm seeing something that I don't fully understand. I suspect this might have security implications, but I'm also more than willing to be proven wrong.” Because showing up with, “You folks are idiots and have a security problem,” is a terrific invitation to be proven wrong and look like an idiot. Because the first time you get that wrong, no one will take you seriously again.Tim: Of course. And as you'll find that most bug bounty programs are, if you participate in those, the first couple that you might have submitted, the customer might even tell you, “Yeah, we're aware that that vulnerability exists, however, we don't view it as a core issue and it cannot affect the functionality of our site in any meaningful way, therefore we're electing to ignore it.” Fair.Corey: Very fair. But then when people write up about those things, well, they've they decided this is not an issue, so I'm going to do a write-up on it. Like, “You can't do that. The NDA doesn't let you expose that.” “Really? Because you just said it's a non-issue. Which is it?”Tim: And the key to that, I guess, would also be that is there an underlying technology that doesn't necessarily have to be attributed to said organization? Can you also say that, if I provide a write-up or if I put up my own personal blog post—let's say, we go back to some of the OpenSSL vulnerabilities including OpenSSL 3.0, that came out not too long ago, but since that's an open-source project, it's fair game—let's just say that if there was a technology such as that, or maybe there's a wrapper around it that another organization could be using or could be implementing a certain way, you don't necessarily have to call the company up by name, or rather just say, here's the core technology reason, and here's the core technology risk, and here's the way I've demoed exploiting this. And if you publish an open-source blog like that and then you tweet about that, you can actually gain security support around such issue and then fight for the research.An example would be that I know a couple of pen testers who have reported things in the past, and while the first time they reported it, the company was like, “Yeah, we'll fix it eventually.” But later, when another researcher report this exact same finding, the company is like, “We should probably take this seriously and jump on it.” It sometimes it's just getting in front of that and providing frequency or providing enough people around to say that, “Hey, this really is an issue in the security community and we should probably fix this item,” and keep pushing others organizations on it. A lot of times, they just need additional feedback. Because as you said, somebody runs an automated scanner against your email and says that, “Oh, you're not checking SPF as strictly as the scanner would have liked because it's a benchmarking tool.” It's not necessarily a security vulnerability rather than it's just how you've chosen to configure something and if it works for you, it works for you.Corey: How does cloud change this? Because a lot of what we talked about so far could apply to anything. Go back in time to 1995 and a lot of what we're talking about mostly holds true. It feels like cloud acts as a significant level of complexity on top of all of this. How do you view the differentiation there?Tim: So, I think it differentiated two things. One, certain services or certain vulnerability classes that are handled by the shared service model—for the most part—are probably secure better than you might be able to do yourself. Just because there's a lot of research, the team is [experimented 00:13:03] a lot of time on this. An example of if there's a particular, like, spoofing or network interception vulnerability that you might see on a local LAN network, you probably are not going to have the same level access to be able to execute that on a virtual private cloud or VNet, or some other virtual network within cloud environment. Now, something that does change with the paradigm of cloud is the fact that if you accidentally publicly expose something or something that you've created expo—or don't set a setting to be private or only specific to your resources, there is a couple of things that could happen. The vulnerabilities exploitability based on where increases to something that used to be just, “Hey, I left a port open on my own network. Somebody from HR or somebody from it could possibly interact with it.”However, in the cloud, you've now set this up to the entire world with people that might have resources or motivations to go after this product, and using services like Shodan—which are continually mapping the internet for open resources—and they can quickly grab that, say, “Okay, I'm going to attack these targets today,” might continue to poke a little bit further, maybe an internal person that might be bored at work or a pen tester just on one specific engagement. Especially in the case of let's say, what you're working on has sparked the interest of a nation-state and they want to dig into a little bit further, they have the resources to be able to dedicate time, people, and maybe tools and tactics against whatever this vulnerability that you've given previously the example of—maybe there's a specific ID and a URL that just needs to be guessed right to give them access to something—they might spend the time trying to brute force that URL, brute force that value, and eventually try to go after what you have.The main paradigm shift here is that there are certain things that we might consider less of a priority because the cloud has already taken care of them with the shared service model, and rightfully so, and there's other times that we have to take heightened awareness on is, one, we either dispose something to the entire internet or all cloud accounts within creations. And that's actually something that we see commonly. In fact, one thing I would like to say we see very common is, all AWS users, regardless if it's in your account or somewhere else, might have access to your SNS topic or SQS Queue. Which doesn't seem like that big of vulnerability, but I changed the messages, I delete messages, I viewed your messages, but rather what's connected to those? Let's talk database Lambda functions where I've got source code that a developer has written to handle that source code and may not have built in logic to handle—maybe there was a piece of code that could be abused as part of this message that might allow an attacker to send something to your Lambda function and then execute something on that attacker's behalf.You weren't aware of it, you weren't thinking about it, and now you've exposed it to almost the entire internet. And since anyone can go sign up for an AWS account—or Azure or GCP account—and then they're able to start poking at that same piece of code that you might have developed thinking, “Well, this is just for internal use. It's not a big deal. That one static code analysis tool isn't probably too relevant.” Now, it becomes hyper-relevant and something you have to consider with a little more attention and dedicated time to making sure that these things that you've written or deploying, are in fact, safe because misconfigured or mis-exposed, and suddenly the entire world is starts knocking at it, and increases the risk of, it may really well be a problem. The severity of that issue could increase dramatically.Corey: As you take a look across, let's call it the hyperscale clouds, the big three—which presumably I don't need to define out—how do you wind up ranking them in terms of security from top to bottom? I have my own rankings that I like to dole out and basically, this is the, let's offend someone at every one of these companies, no matter how we wind up playing it. Because I will argue with you just on principle on them. How do you view them stacking up against each other?Tim: So, an interesting view on that is based on who's been around longest and who is encountered of the most technical debt. A lot of these security vulnerabilities or security concerns may have had to deal with a decision made long ago that might have made sense at the time and now the company has kind of stuck with that particular technology or decision or framework, and are now having to build or apply security Band-Aids to that process until it gets resolved. I would say, ironically, AWS is actually at the top of having that technical debt, and actually has so many different types of access policies that are very complex to configure and not very user intuitive unless you speak intuitively JSON or YAML or some other markdown language, to be able to tell you whether or not something was actually set up correctly. Now, there are a lot of security experts who make their money based on knowing how to configure or be able to assess whether or not these are actually the issue. I would actually bring them as, by default, by design, between the big three, they're actually on the lower end of certain—based on complexity and easy-to-configure-wise.The next one that would also go into that pile, I would say is probably Microsoft Azure, who [sigh] admittedly, decided to say that, “Okay, let's take something that was very complicated and everyone really loved to use as an identity provider, Active Directory, and try to use that as a model for.” Even though they made it extensively different. It is not the same as on-prem directory, but use that as the framework for how people wanted to configure their identity provider for a new cloud provider. The one that actually I would say, comes out on top, just based on use and based on complexity might be Google Cloud. They came to a lot of these security features first.They're acquiring new companies on a regular basis with the acquisition of Mandiant, the creation of their own security tooling, their own unique security approaches. In fact, they probably wrote the book on Kubernetes Security. Would be on top, I guess, from usability, such as saying that I don't want to have to manage all these different types of policies. Here are some buttons I would like to flip and I'd like my resources, for the most part by default, to be configured correctly. And Google does a pretty good job of that.Also, one of the things they do really well is entity-based role assumption, which inside of AWS, you can provide access keys by default or I have to provide a role ID after—or in Azure, I'm going to say, “Here's a [unintelligible 00:19:34] policy for something specific that I want to grant access to a specific resource.” Google does a pretty good job of saying that okay, everything is treated as an email address. This email address can be associated in a couple of different ways. It can be given the following permissions, it can have access to the following things, but for example, if I want to remove access to something, I just take that email address off of whatever access policy I had somewhere, and then it's taken care of. But they do have some other items such as their design of least privilege is something to be expected when you consider their hierarchy.I'm not going to say that they're not without fault in that area—in case—until they had something more recently, as far as finding certain key pieces of, like say, tags or something within a specific sub-project or in our hierarchy, there were cases where you might have granted access at a higher level and that same level of access came all the way down. And where at least privilege is required to be enforced, otherwise, you break their security model. So, I like them for how simple it is to set up security at times, however, they've also made it unnecessarily complex at other times so they don't have the flexibility that the other cloud service providers have. On the flip side of that, the level of flexibility also leads to complexity at times, which I also view as a problem where customers think they've done something correctly based on their best knowledge, the best of documentation, the best and Medium articles they've been researching, and what they have done is they've inadvertently made assumptions that led to core anti-patterns, like, [unintelligible 00:21:06] what they've deployed.Corey: This episode is sponsored in part by our friends at Uptycs, because they believe that many of you are looking to bolster your security posture with CNAPP and XDR solutions. They offer both cloud and endpoint security in a single UI and data model. Listeners can get Uptycs for up to 1,000 assets through the end of 2023 (that is next year) for $1. But this offer is only available for a limited time on UptycsSecretMenu.com. That's U-P-T-Y-C-S Secret Menu dot com.Corey: I think you're onto something here, specifically in—well, when I've been asked historically and personally to rank security, I have viewed Google Cloud as number one, and AWS is number two. And my reasoning behind that has been from an absolute security of their platform and a pure, let's call it math perspective, it really comes down to which of the two of them had what for breakfast on any given day there, they're so close on there. But in a project that I spin up in Google Cloud, everything inside of it can talk to each other by default and I can scope that down relatively easily, whereas over an AWS land, by default, nothing can talk to anything. And that means that every permission needs to be explicitly granted, which in an absolutist sense and in a vacuum, yeah, that makes sense, but here in reality, people don't do that. We've seen a number of AWS blog posts over the last 15 years—they don't do this anymore—but it started off with, “Oh, yeah, we're just going to grant [* on * 00:22:04] for the purposes of this demo.”“Well, that's horrible. Why would you do that?” “Well, if we wanted to specify the IAM policy, it would take up the first third of the blog post.” How about that? Because customers go through that exact same thing. I'm trying to build something and ship.I mean, the biggest lie in any environment or any codebase ever, is the comment that starts with, “To do.” Yeah, that is load-bearing. You will retire with that to do still exactly where it is. You have to make doing things the right way at least the least frictionful path because no one is ever going to come back and fix this after the fact. It's never going to happen, as much as we wish that it did.Tim: At least until after the week of the breach when it was highlighted by the security team to say that, “Hey, this was the core issue.” Then it will be fixed in short order. Usually. Or a Band-Aid is applied to say that this can no longer be exploited in this specific way again.Corey: My personal favorite thing that, like, I wouldn't say it's a lie. But the favorite thing that I see in all of these announcements right after the, “Your security is very important to us,” right after it very clearly has not been sufficiently important to them, and they say, “We show no signs of this data being accessed.” Well, that can mean a couple different things. It can mean, “We have looked through the audit logs for a service going back to its launch and have verified that nothing has ever done this except the security researcher who found it.” Great. Or it can mean, “What even are logs, exactly? We're just going to close our eyes and assume things are great.” No, no.Tim: So, one thing to consider there is in that communication, that entire communication has probably been vetted by the legal department to make sure that the company is not opening itself up for liability. I can say from personal experience, when that usually has occurred, unless it can be proven that breach was attributable to your user specifically, the default response is, “We have determined that the security response of XYZ item or XYZ organization has determined that your data was not at risk at any point during this incident.” Which might be true—and we're quoting Star Wars on this one—from a certain point of view. And unfortunately, in the case of a post-breach, their security, at least from a regulation standpoint where they might be facing a really large fine, is absolutely probably their top priority at this very moment, but has not come to surface because, for most organizations, until this becomes something that is a financial reason to where they have to act, where their reputation is on the line, they're not necessarily incentivized to fix it. They're incentivized to push more products, push more features, keep the clients happy.And a lot of the time going back and saying, “Hey, we have this piece of technical debt,” it doesn't really excite our user base or doesn't really help us gain a competitive edge in the market is considered an afterthought until the crisis occurs and the information security team rejoices because this is the time they actually get to see their stuff fixed, even though it might be a super painful time for them in the short run because they get to see these things fixed, they get to see it put to bed. And if there's ever a happy medium, where, hey, maybe there was a legacy feature that wasn't being very well taken care of, or maybe this feature was also causing the security team a lot of pain, we get to see both that feature, that item, that service, get better, as well as security teams not have to be woken up on a regular basis because XYZ incident happened, XYZ item keeps coming up in a vulnerability scan. If it finally is put to bed, we consider that a win for all. And one thing to consider in security as well as kind of, like, we talk about the relationship between the developers and security and/or product managers and security is if we can make it a win, win, win situation for all, that's the happy path that we really want to be getting to. If there's a way that we can make sure that experience is better for customers, the security team doesn't have to be broken up on a regular basis because an incident happened, and the developers receive less friction when they want to go implement something, you find that that secure feature, function, whatever tends to be the happy path forward and the path of least resistance for everyone around it. And those are sometimes the happiest stories that can come out of some of these incidents.Corey: It's weird to think of there being any happy stories coming out of these things, but it's definitely one of those areas that there are learnings there to be had if we're willing to examine them. The biggest problem I see so often is that so many companies just try and hide these things. They give the minimum possible amount of information so the rest of us can't learn by it. Honestly, some of the moments where I've gained the most respect for the technical prowess of some of these cloud providers has been after there's been a security issue and they have disclosed either their response or why it was a non-issue because they took a defense-in-depth approach. It's really one of those transformative moments that I think is an opportunity if companies are bold enough to chase them down.Tim: Absolutely. And in a similar vein, when we think of certain cloud providers outages and we're exposed, like, the major core flaw of their design, and if it kept happening—and again, these outages could be similar and analogous to an incident or a security flaw, meaning that it affected us. It was something that actually happened. In the case of let's say, the S3 outage of, I don't know, it was like 2017, 2018, where it turns out that there was a core DNS system that inside of us-east-1, which is actually very close to where I live, apparently was the core crux of, for whatever reason, the system malfunctioned and caused a major outage. Outside of that, in this specific example, they had to look at ways of how do we not have a single point of failure, even if it is a very robust system, to make sure this doesn't happen again.And there was a lot of learnings to be had, a lot of in-depth investigation that happened, probably a lot of development, a lot of research, and sometimes on the outside of an incident, you really get to understand why a system was built a certain way or why a condition exists in the first place. And it sometimes can be fascinating to kind of dig into that very deeper and really understand what the core problem is. And now that we know what's an issue, we can actually really work to address it. And sometimes that's actually one of the best parts about working at Praetorian in some cases is that a lot of the items we find, we get to find them early before it becomes one of these issues, but the most important thing is we get to learn so much about, like, why a particular issue is such a big problem. And you have to really solve the core business problem, or maybe even help inform, “Hey, this is an issue for it like this.”However, this isn't necessarily all bad in that if you make these adjustments of these items, you get to retain this really cool feature, this really cool thing that you built, but also, you have to say like, here's some extra, added benefits to the customers that you weren't really there. And—such as the old adage of, “It's not a bug, it's a feature,” sometimes it's exactly what you pointed out. It's not necessarily all bad in an incident. It's also a learning experience.Corey: Ideally, we can all learn from these things. I want to thank you for being so generous with your time and talking about how you view this increasingly complicated emerging space. If people want to learn more, where's the best place to find you?Tim: You can find me on LinkedIn which will be included in this podcast description. You can also go look at articles that the team is putting together at praetorian.com. Unfortunately, I'm not very big on Twitter.Corey: Oh, well, you must be so happy. My God, what a better decision you're making than the rest of us.Tim: Well, I like to, like, run a little bit under the radar, except on opportunities like this where I can talk about something I'm truly passionate about. But I try not to pollute the airwaves too much, but LinkedIn is a great place to find me. Praetorian blog for stuff the team is building. And if anyone wants to reach out, feel free to hit the contact page up in praetorian.com. That's one of the best places to get my attention.Corey: And we will, of course, put links to that in the [show notes 00:30:19]. Thank you so much for your time. I appreciate it. Tim Gonda, Technical Director of Cloud at Praetorian. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment talking about how no one disagrees with you based upon a careful examination of your logs.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

AWS Morning Brief
The Work of Sober Minds

AWS Morning Brief

Play Episode Listen Later Jan 9, 2023 4:54


Links: Amazon CloudFront now supports the removal of response headers  Amazon SageMaker is now available in AWS Middle East (UAE) Region Amazon Neptune announces graph-explorer, an open-source visual exploration tool for low-code users An elastic deployment of Stable Diffusion with Discord on AWS  Measure the Business Impact of Personalize Recommendations  How Heineken's Connected Brewery Ecosystem fuels automation 

Screaming in the Cloud
Life of a Fellow Niche Internet Micro Celebrity with Matt Margolis

Screaming in the Cloud

Play Episode Listen Later Jan 5, 2023 36:36


About MattMatt is the head of community at Lawtrades, a legal tech startup that connects busy in-house legal departments with flexible on-demand legal talent. Prior to this role, Matt was the director of legal and risk management at a private equity group down in Miami, Florida. Links Referenced: Lawtrades: https://www.lawtrades.com/ Instagram: https://www.instagram.com/itsmattslaw/ TikTok: https://www.tiktok.com/@itsmattslaw Twitter: https://twitter.com/ItsMattsLaw LinkedIn: https://www.linkedin.com/in/flattorney/ duckbillgroup.com: https://duckbillgroup.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: If you asked me to rank which cloud provider has the best developer experience, I'd be hard-pressed to choose a platform that isn't Google Cloud. Their developer experience is unparalleled and, in the early stages of building something great, that translates directly into velocity. Try it yourself with the Google for Startups Cloud Program over at cloud.google.com/startup. It'll give you up to $100k a year for each of the first two years in Google Cloud credits for companies that range from bootstrapped all the way on up to Series A. Go build something, and then tell me about it. My thanks to Google Cloud for sponsoring this ridiculous podcast.Corey: This episode is sponsored by our friends at Logicworks. Getting to the cloud is challenging enough for many places, especially maintaining security, resiliency, cost control, agility, etc, etc, etc. Things break, configurations drift, technology advances, and organizations, frankly, need to evolve. How can you get to the cloud faster and ensure you have the right team in place to maintain success over time? Day 2 matters. Work with a partner who gets it - Logicworks combines the cloud expertise and platform automation to customize solutions to meet your unique requirements. Get started by chatting with a cloud specialist today at snark.cloud/logicworks. That's snark.cloud/logicworksCorey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Something that I've learned in my career as a borderline full-time shitposter is that as the audience grows, people tend to lose sight of the fact that no, no, the reason that I have a career is because I'm actually good at one or two specific things, and that empowers the rest of the shitposting, gives me a basis from which to stand. Today's guest is Matt Margolis, Head of Community at Lawtrades. And I would say he is also a superior shitposter, but instead of working in the cloud space, he works in the legal field. Matt, thank you for joining me.Matt: That was the nicest intro I've ever received in my entire career.Corey: Well, yes, usually because people realize it's you and slam the door in your face, I assume, just based upon some of your TikToks. My God. Which is—I should point out—where I first encountered you.Matt: You found me on TikTok?Corey: I believe so. It sends me down these really weird rabbit holes, and at first, I was highly suspicious of the entire experience. Like, it's showing ADHD videos all the time, and as far as advertisements go, and it's, “Oh, my God, they're doing this really weird tracking,” and like, no, no, they just realize I'm on TikTok. It's that dopamine hit that works out super well. For a while, it drifted me into lesbian TikTok—which is great—because apparently, I follow a lot of creators who are not men, but I also don't go for the whole thirst trap things. Like, who does that? That's right. Must be lesbians. Which, great, I'm in good company. And it really doesn't know what to make of me. But you show up on my feed with fairly consistent frequency. Good work.Matt: That is fac—I appreciate that. I don't know if that's a compliment, though. But I [laugh]—no, I appreciate it. You know, for me, I get… not to plug a friend but I get—Alex Su's TikToks are probably like, one in two and then the other person is—maybe I'm also on lesbian TikTok as well. I think maybe we have earned the similar vote here.Corey: In fact, there's cohorts that they slot people into and I feel like we're right there together. Though Ales Su, who has been on the show as well, talk about source of frustration. I mentioned in passing that I was going to be chatting with him to my wife, who's an attorney. And she lit up. Like, “Oh, my God, you know him? My girlfriends and I talk about him all the time.”And I was sitting there going, well, there better damn well be a subculture out there that talks about me and those glowing terms because he's funny, yes, but he's not that funny. My God. And don't tell him that. It'll go to his head.Matt: I say the same thing. I got a good one for you. I was once in the sales call, and I remember speaking with—I was like, “You know, I'm like, pretty decent on Twitter. I'm pretty decent on LinkedIn”—which I don't think anyone brags about that, but I do—“And I'm okay on, like, Instagram and TikTok.” And he goes, “That's cool. That's really cool. So, are you kind of like Alex? Like, Alex Su?” And I go? “Uh, yeah,” he goes, “Yeah, because he's really funny. He's probably the best lawyer out there that, you know, shitposts and post funny things on the internet.” And I just sat there—and I love Alex; he's a good friend—I just sat there, and I'm like, “All right. All right. This is a conversation about Alex. This isn't a conversation about Matt.” And I took him to stride. I called Alex immediately after. I'm like, “Hey, you want to hear something funny.” And he got a kick out of it. He certainly got a kick out of it.Corey: It's always odd to me, just watching my own reputation come back to me filtered through other people's perceptions whenever I wind up encountering people in the wild, and they say, oh, you're Corey Quinn at—which is usually my clue to look at them very carefully with my full attention because if their next words are, “I work at Amazon,” that's my cue to duck before I get punched in the face. Whereas in other cases, they're like, “Oh, yeah, you're hilarious on the Twitters.” Or, “I saw you give a conference talk years ago,” or whatever it is. But no one ever says the stuff that's actually intellectually rigorous. No one ever says, “Yeah, I read some of your work on AWS contract negotiation,” or, “In-depth bill analysis as mapped to architecture.” Yeah, yeah. That is not the stuff that sticks in people's head. It's, “No, no, the funny guy with his mouth wide open on the internet.” It's, “Yep, that's me. The human flytrap.”Matt: Yeah, I feel that. I've been described, I think, is a party clown. That comes up from time to time. And to your point, Corey, like, I get that all the time where someone will say, “Matt I really enjoyed that meme you posted, the TikTok, the funny humor.” And then every so often, I'll post, gosh, like, an article about something we're doing, maybe a white paper on commercial contracting, or some sort of topic that really fits into my wheelhouse, and people were like, “That's… I guess that's cool. I just thought you were a party clown.” And you know, I make the balloon animals but… not all the time.Corey: That's the weirdest part to me of all of this is just this weird experience where we become the party clowns and that is what people view us as, but peeling away the humor and the jokes and the things we do for engagement, as we're like, we're sitting here each trying to figure out the best way to light ourselves on fire and survive the experience because the views would be enormous, you do have a legal background. You are an attorney yourself—still are, if I understand the process properly. Personally have an eighth-grade education, so basically, what I know of bars is a little bit of a different context.Matt: I also know those bars. I'm definitely a fan of those bars as well. I am still an attorney. I was in private practice, I worked in the government. I then went in-house in private equity down in Miami, Florida. And now, though I am shitposter, you are right, I am still a licensed attorney in the state of Florida. Could not take a bar exam anywhere else because I probably would light myself on fire. But yeah, I am. I am still an attorney.Corey: It's wild to me just to see how much of this world winds up continuing to, I guess, just evolve in strange and different ways. Because you take a look at the legal profession, it's—what is it, the world's second oldest profession? Because they say that the oldest profession was prostitution and then immediately someone, of course, had a problem with this, so they needed to have someone to defend them and hence, lawyers; the second oldest profession. And it seems like it's a field steeped in traditionalism, and with the bar, yes, a bit of gatekeeping. And now it's trying to deal with a highly dynamic, extraordinarily irreverent society.And it feels like an awful lot of, shall we say, more buttoned-down attorney types tend to not be reacting to any of that super well. I mean, most of my interaction with lawyers in a professional context when it comes to content takes a lot more of the form of a cease and desist than it does conversations like this. Thanks for not sending one of those, by the way, so far. It's appreciated.Matt: [laugh]. No worries, no worries. The day is not over yet. First off, Corey, I'm going to do a thing that attorneys love doing is I'm going to steal what you just said and I'm going to use it later because that was stellar.Corey: They're going to license it, remember?Matt: License it.Corey: That's how this works.Matt: Copy and paste it. I'm going to re—its precedent now. I agree with you wholeheartedly. I see it online, I see it on Link—LinkedIn is probably the best example of it; I sometimes see it on Twitter—older attorneys, attorneys that are part of that old guard, see what we're doing, what we're saying, the jokes we're making—because behind every joke is a real issue a real thing, right? The reason why we laugh, at least for some of these jokes, is we commiserate over it. We're like, “That's funny because it hurts.”And a lot of these old-guard attorneys hate it. Do not want to talk about it. They've been living good for years. They've been living under this regime for years and they don't want to deal with it. And attorneys like myself who are making these jokes, who are shitposting, who are bringing light to these kinds of things are really, I would say dis—I hate to call myself a disrupter, but are disrupting the traditional buttoned-up attorney lifestyle and world.Corey: It's wild to me, just to see how much of this winds up echoing my own experiences in dealing with, shall we say, some of the more I don't use legacy, which is a condescending engineering term for ‘it makes money,' but some of the older enterprise companies that had the temerity to found themselves before five years ago in somewhere that wasn't San Francisco and build things on computers that weren't rented by the gigabyte-month from various folks in Seattle. It's odd talking to some of those folks, and I've heard from a number of people, incidentally, that they considered working with my company, but decided not to because I seem a little too lighthearted and that's not how they tend to approach things. One of the nice things about being a boutique consultant is that you get to build things like this to let the clients that are not likely to be a good fit self-select out of working with you.Matt: It's identical to law.Corey: Yeah. “Aren't you worried you're losing business?” Like, “Oh, don't worry. It's not business I would want.”Matt: I'm okay with it. I'll survive. Yeah, like, the clients that are great clients, you're right, will be attracted to it. The clients that you never wanted to approach, they probably were never going to approach you anyways, are not [laugh] going to approach you. So, I agree wholeheartedly. I was always told lawyers are not funny. I've been told that jobs, conferences, events—Corey: Who are you hanging out with doctors?Matt: [laugh]. Dentists. The funniest of doctors. And I've been told that just lawyers aren't funny, right? So, lawyers shouldn't be funny; that's not how they should present themselves.You're never going to attract clients. You're ever going to engage in business development. And then I did. And then I did because people are attracted by funny. People like the personality. Just like you Corey, people enjoy you, enjoy your company, enjoy what you have to do because they enjoy being around you and they want to continue via, you know, like, business relationship.Corey: That's part of the weird thing from where I sit, where it's this—no matter what you do or where you sit, people remain people. And one of the big eye-openers for me that happened, fortunately early in my career, was discovering that a number of execs at name brand, publicly traded companies—not all of them, but a good number; the ones you'd want to spend time with—are in fact, human beings. I know, it sounds wild to admit that, but it's true. And they laugh, they tell stories themselves, they enjoy ridiculous levels of nonsense that tends to come out every second time I opened my mouth. But there's so much that I think people lose sight of. “Oh, they're executives. They only do boring and their love language is PowerPoint.” Mmm, not really. Not all of them.Matt: It's true. Their love language sometimes is Excel. So, I agree [laugh].Corey: That's my business partner.Matt: I'm not good at Excel, I'll tell you that. But I hear that as well. I hear that in my own business. So, I'm currently at a place called Lawtrades, and for the listeners out there, if you don't know who Lawtrades is, this is the—I'm not a salesperson, but this is my sales spiel.Corey: It's a dating site for lawyers, as best I can tell.Matt: [laugh]. It is. Well, I guess close. I mean, we are a marketplace. If you're a company and you need an attorney on a fractional basis, right—five hours, ten hours, 15 hours, 20 hours, 40 hours—I don't care, you connect.And what we're doing is we're empowering these freelance attorneys and legal professionals to kind of live their life, right, away from the old guard, having to work at these big firms to work at big clients. So, that's what we do. And when I'm in these conversations with general counsels, deputy general counsels, heads of legal at these companies, they don't want to talk like you're describing, this boring, nonsense conversation. We commiserate, we talk about the practice, we talk about stories, war stories, funny things about the practice that we enjoy. It's not a conversation about business; it's a conversation about being a human being in the legal space. It's always a good time, and it always results in a long-lasting relationship that I personally appreciate more than—probably more than they do. But [laugh].Corey: It really comes down to finding the watering holes where your humor works. I mean, I made the interesting choice one year to go and attend a conference for CFOs and the big selling point of this conference was that it counts as continuing professional education, which as you're well aware, in regulated professions, you need to attend a certain number of those every so often, or you lose your registration slash license slash whatever it is. My jokes did not work there. Let's put it that way.Matt: [laugh]. That's unfortunate because I'm having trouble keeping a straight face as we do this podcast.Corey: It was definitely odd. I'm like, “Oh, so what do you do?” Like, “Oh, I'm an accountant.” “Well, that's good. I mean, assume you don't bring your work home with you and vice versa. I mean, it's never a good idea to hook up where you VLOOKUP.”And instead of laughing—because I thought as Excel jokes go, that one's not half bad—instead, they just stared at me and then walked away. All right. Sorry, buddy, I didn't mean to accidentally tell a joke in your presence.Matt: [laugh]. You're setting up all of my content for Twitter. I like that one, too. That was really good.Corey: No, no, it comes down to just being a human being. And one of the nice things about doing what I've done—I'm curious to get your take on this, is that for the first time in my career doing what I do now, I feel like I get to bring my whole self to work. That is not what it means that a lot of ways it's commonly used. It doesn't mean I get to be problematic and make people feel bad as individuals. That's just being an asshole; that's not bringing your whole self to work.But it also means I feel like I don't have to hide, I can bring my personality with me, front and center. And people are always amazed by how much like my Twitter personality I am in real life. And yeah because I can't do a bit for this long. I don't have that kind of attention span for one. But the other side of that, too, is does exaggerate certain elements and it's always my highs, never my lows.I'm curious to know how you wind up viewing how you present online with who you are as a person.Matt: That is a really good question. Similar. Very similar. I do some sort of exaggeration. The character I like to play is ‘Bad Associate.' It's, like, one of my favorite characters to play where it's like, if I was the worst version of myself, in practice, what would I look like?And those jokes to me always make me laugh because I always—you know, you have a lot of anxiety when you practice. That's just an aspect of the law. So, for me, I get to make jokes about things that I thought I was going to do or sound like or be like, so it honestly makes me feel a little better. But for the humor itself and how I present online, especially on Twitter, my boss, one of my co-founders, put it perfectly. And we had met for a conference, and—first time in person—and he goes, “You're no different than Twitter, are you?” I go, “Nope.” And he goes, “That's great.”And he really appreciated that. And you're right. I felt like I presented my whole personality, my whole self, where in the legal profession, in private practice, it was not the case. Definitely not the case.Corey: Yeah, and sometimes I talk in sentences that are more than 280 characters, which is, you know, a bad habit.Matt: Sometimes. I have a habit from private practice that I can't get rid of, and I ask very aggressive depo questions like I'm deposing somebody. If you're listening in, can you write me on Twitter and tell me if you're a litigator and you do the same thing? Because, like, I will talk to folks, and they're like, “This isn't an interview or like a deposition.” I'm like, “Why? Why isn't it?” And it [laugh] gets really awkward really quickly. But I'm trying to break that habit.Corey: I married a litigator. That pattern tracks, let's be clear. Not that she doesn't so much, but her litigator friends, if litigators could be said to have friends, yeah, absolutely.Matt: My wife is a former litigator. Transactional attorney.Corey: Yes. Much the same. She's grown out of the habit, thankfully.Matt: Oh, yeah. But when we were in the thick of litigation, we were actually at competing law firms. It was very much so, you come home, and it's hard to take—right, it's hard to not take your work home, so there was definitely occasions where we would talk to each other and I thought the judge had to weigh in, right, because there were some objections thrown, some of the questions were leading, a little bit of compound questions. So, all right, that's my lawyer joke of the day. I'm sorry, Corey. I won't continue on the schtick.Corey: It works, though. It's badgering the witness, witnessing the badger, et cetera. Like, all kinds of ridiculous nonsense and getting it wrong, just to be, I guess, intentionally obtuse, works out well. Something you said a minute ago does tie into what you do professionally, where you mentioned that your wife was a litigator and now is a transactional attorney. One thing they never tell you when you start a business is how many lawyers you're going to be working with.And that's assuming everything goes well. I mean, we haven't been involved in litigation, so that's a whole subset of lawyer we haven't had to deal with yet. But we've worked with approximately six—if memory serves—so far, not because we're doing anything egregious, just because—rather because so many different aspects of the business require different areas of specialty. We also, to my understanding—and I'm sure my business partner will correct me slash slit my throat if I'm wrong—I've not had to deal with criminal attorneys in any interesting ways. Sorry, criminal defense attorneys, criminal attorneys is a separate setup for a separate story.But once I understood that, realizing, oh, yeah, Lawtrades. You can find specialist attorneys to augment your existing staff. That is basically how I view that. Is that directionally accurate?Matt: Yeah. So like, common issue I run into, right is, like, a general counsel, is a corporate attorney, right? That's their background. And they're very aware that they're not an employment attorney. They're not a privacy attorney. Maybe they're not an IP attorney or a patent attorney.And because they realize that, because they're not like that old school attorney that thinks they can do everything and solve everyone's problems, they come to Lawtrades and they say, “Look, I don't need an employment attorney for 40 hours a week. I just need ten hours. That's all I need, right? That's the amount of work that I have.” Or, “I don't have the budget for an attorney for 40 hours, but I need somebody. I need somebody here because that's not my specialty.”And that happens all the time where all of a sudden, a solo general counsel becomes a five or six-attorney legal department, right, because you're right, attorneys add up very quickly. We're like rabbits. So, that's where Lawtrades comes in to help out these folks, and help out freelance attorneys, right, that also are like, “Hey, listen, I know employment law. I can help.”Corey: Do you find that the vast slash entire constituency of your customers pretend to be attorneys themselves, or is this one of those areas where, “I'm a business owner. I don't know how these law things work. I had a firm handshake and now they're not paying as agreed. What do I do?” Do you wind up providing, effectively, introduction services—since I do view you as, you know, match.com for dating with slightly fewer STDs—do you wind up then effectively acting as an—[unintelligible 00:18:47] go to talk to find a lawyer in general? Or does it presuppose that I know which end of a brief is up?Matt: There's so many parts of what you just said I want to take as well. I also liked that you didn't just say no STDs. That was very lawyerly of you. It's always, like, likely, right?Corey: Oh, yes. So, the answer to any particular level of seniority and every aspect of being an attorney is, “It depends.”Matt: That's right. That's right. It triggers me for you to say it. Ugh. So, our client base, generally speaking, our companies ranging from, like, an A round company that has a solo GC all the way up to a publicly traded company that has super robust legal department that maybe needs a bunch of paralegals, bunch of legal operations professionals, contract managers, attorneys for very niche topics, niche issues, that they're just, that is not what they want to do.So, generally speaking, that's who we service. We used to be in the SMB space. There was a very public story—my founders are really cool because they built in public and we almost went broke, actually in that space. Which, Corey, I'm happy to share that article with you. I think you'll get a kick out of it.Corey: I would absolutely look forward to seeing that article. In fact, if you send me the link, we will definitely make it a point to throw it into the [show notes 00:19:58].Matt: Awesome. Happy to do it. Happy to do it. But it's cool. The clients, I tell you what, when I was in private practice when I was in-house, I would always deal with an adverse attorney. That was always what I was dealing with.No one was ever—or a business person internally that maybe wasn't thrilled to be on the phone. I tell you what, now, when I get to talk to some of these folks, they're happy to talk to me; it's a good conversation. It really has changed my mentality from being a very adverse litigator attorney to—I mean it kind of lends itself to a shitposter, to a mean guy, to a party clown. It's a lot of fun.Corey: This episode is sponsored in part by our friends at Uptycs, because they believe that many of you are looking to bolster your security posture with CNAPP and XDR solutions. They offer both cloud and endpoint security in a single UI and data model. Listeners can get Uptycs for up to 1,000 assets through the end of 2023 (that is next year) for $1. But this offer is only available for a limited time on UptycsSecretMenu.com. That's U-P-T-Y-C-S Secret Menu dot com.Corey: One area that I think is going to be a point of commonality between us is in what the in-and-out of our day jobs look like. Because looking at it from a very naive perspective, why on earth does what is effectively an attorney referral service—yes, which may or may not run afoul of how you describe yourselves; I know, lawyers are very particular about wording—Matt: Staffing [laugh].Corey: Exactly. Legal staffing. There we are. It doesn't seem to lend itself to having a, “Head of Community,” quote-unquote, which really translates into, “I shitpost on the internet.” The same story could be said to apply to someone who fixes AWS bills because in my part of the industry, obviously, there is a significant problem with people who have large surprise bills from their cloud provider, but they generally don't talk about them in public as soon as they become an even slightly serious company.You don't find someone at a Fortune 500 complaining on Twitter about how big their AWS bill is because that does horrifying things to their stock price as well as them personally, once the SEC gets involved. So, for me, it was always I'm going to be loud and noisy and have fun in the space so that people hear about me, and then when they have this problem, in the come. Is that your approach to this, or is it more or less the retconning story that I just told, and it really had its origins in, “I'm just going to shitpost. I feel like good things will happen.”Matt: Funnily enough, it's both. That's how it started. So, when I was in private practice, I was posting like crazy on—I'm going to say LinkedIn for the third time—and again, I hope somebody sends a nasty message to me about how bad LinkedIn is, which I don't think it's that bad. I think it's okay—so I was shitposting on LinkedIn before probably many folks were shitposting on LinkedIn, again like Alex, and I was doing it just because I was tired of attorneys being what we described, this old guard, buttoned up, just obnoxiously perfect version of themselves. And it eventually led itself into this career. The whole journey was wild, how I got here. Best way to describe it was a crazy trip.Corey: It really is. You also have a very different audience in some ways. I mean, for example, when you work in the legal field, to my understanding from the—or being near to it, but not within it, where you go to school is absolutely one of those things that people still bring up as a credential decades later; it's the first thing people scroll to on LinkedIn. And in tech, we have nothing like that at all. I mean, just ask anyone of the random engineers who talk about where they used to work in their Twitter bio: ex-Google, ex-Uber, et cetera.Not quite as bad as the VC space where it's, “Oh, early investor in,” like, they list their companies, which of course to my mind, just translates directly into, the most interesting thing about you is that once upon a time, you wrote a check. Which yeah, and with some VCs that definitely tracks.Matt: That's right. That's a hundred percent right. It's still like that. I actually saw a Twitter post, not necessarily about education, but about big law, about working in big law where folks were saying, “Hey, I've heard a rumor that you cannot go in-house at a company unless you worked in big law.” And I immediately—I have such a chip on my shoulder because I am not a big law attorney—I immediately jumped to it to say, “Listen, I talk to in-house attorneys all the time. I'm a former in-house attorney. You don't have to work with big law. You don't have to go to a T-14 law school.” I didn't. I went to Florida State University in Tallahassee.But I hear that to this day. And you're right, it drives me nuts because that is a hallmark of the legal industry, bragging about credentials, bragging about where I came from. Because it also goes back to that old guard of, “Oh, I came from Harvard, and I did this, and I did that,” because we love to show how great and special we are not by our actual merits, but where we came from.Corey: When someone introduces themselves to me at a party—which has happened to me before—and in their introduction, they mention where they went to law school, I make it a point to ask them about it and screw it up as many times in the rest of the evening as I can work in to. It's like they went to Harvard. Like so, “Tell me about your time at Yale.” “Oh, sorry. I must have forgotten about that.” Or, “What was the worst part about living in DC when you went to law school?” “Oh, I'm sorry. I missed that. You went to Harvard. How silly of me.”Matt: There's a law school at Dartmouth [laugh]?Corey: I know. I'm as surprised as anyone to discover these things. Yeah. I mean, again, on the one hand, it does make people feel a little off and that's not really what I like doing. But on the other, ideally, it's a little bit of a judgment nudge as far as this may not sound the way that you think it sounds when you introduce yourself to people that way.Matt: All the time. I hear that all the time. Every so often, I'll have someone—and I think a lot of the industry, maybe just the industry where I'm in, it's not brought up anymore. I usually will ask, right? “Hey, where do you come from?” Just as a conversation starter, “What firm did you practice at? Did you practice in big law? Small law?”Someone once called it insignificant law to me, which hurts because I'm part of insignificant law. I get those and it's just to start a conversation, but when it's presented to me initially, “Hey, yeah, I was at Harvard,” unprompted. Or, “I went to Yale,” or went to whatever in the T-14, you're right, it's very off-putting. At least it's off-putting to me. Maybe if someone wants to tell me otherwise, online if you went to Harvard, and someone said, “Hey, I went to Harvard,” and that's how they started the conversation, and you enjoy it, then… so be it. But I'll tell you, it's a bit off-putting to me, Corey.Corey: It definitely seems it. I guess, on some level, I think it's probably rooted in some form of insecurity. Hmm, it's easy to think, “Oh, they're just completely full of themselves,” but that stuff doesn't spring fully formed from nowhere, like the forehead of some God. That stuff gets built into people. Like, the constant pressure of you are not good enough.Or if you've managed to go to one of those schools and graduate from it, great. The constant, like, “Not everyone can go here. You should feel honored.” It becomes, like, a cornerstone of their personality. For better or worse. Like, it made me more interesting adult if it made my 20s challenging. I don't have any big-name companies on my resume. Well, I do now because I make fun of one, but that's a separate problem entirely. It just isn't something I ever got to leverage, so I didn't.Matt: I feel that completely. I come from—again, someone once told me I worked in insignificant law. And if I ever write a book, that's what I'm going to call it is Insignificant Law. But I worked the small law firms, regional law firms, and these in Tallahassee and I worked in South Florida and nothing that anyone would probably recognize in conversation, right? So, it never became something I bring up.I just say, “I'm an attorney. I do these things,” if you ask me what I do. So, I think honestly, my personality, and probably the shitposting sprung out of that as well, where I just had a different thing to talk about. I didn't talk about the prestige. I talked about the practice, I talked about what I didn't like about the practice, I didn't talk about being on Wall Street doing these crazy deals, I talked about getting my ass kicked in Ponce, Florida, up in the panhandle. For me, I've got a chip on my shoulder, but a different kind of chip.Corey: It's amazing to me how many—well, let's calls this what we are: shitposters—I talk to where their brand and the way that they talk about their space is, I don't want to say rooted in trauma, but definitely built from a place of having some very specific chips on their shoulder. I mean, when I was running DevOps teams and as an engineer myself, I wound up continually tripping over the AWS bill of, “Ha, ha. Now, you get to pay your tax for not reading this voluminous documentation, and the fine print, and with all of the appendices, and the bibliography, and tracked down those references. Doesn't it suck to be you? Da da.” And finally, it was all right, I snapped. Okay. You want to play? Let's play.Matt: That's exactly right. There's, like, a meme going around. I think it actually saw from the accounting meme account, TB4—which is stellar—and it was like, “Ha, I'm laughing because it hurts.” And it's true. That's why we all laugh at the jokes, right?I'll make jokes about origination credit, which is always an issue in the legal industry. I make jokes about the toxic work environment, the partner saying, “Please fix,” at three o'clock in the morning. And we make fun of it because everyone's had to deal with it. Everyone's had to deal with it. And I will say that making fun of it brings light to it and hopefully changes the industry because we all can see how ridiculous it is. But at least at the very beginning, we all look at it and we say, “That's funny because it hurts.”Corey: There's an esprit de corps of shared suffering that I think emerges from folks who are in the trenches, and I think that the rise of—I mean some places called the micro-influencers, but that makes me want to just spit a rat when I hear it; I hate the term—but the rise of these niche personalities are because there are a bunch of in-jokes that you don't have to be very far in to appreciate and enjoy, but if you aren't in the space at all, they just make zero sense. Like when I go to family reunions and start ranting about EC2 instance pricing, I don't get to talk to too many people anymore because oh my God, I've become the drunk uncle I always wanted to be. Goal achieved.Matt: [laugh].Corey: You have to find the right audience.Matt: That's right. There is a term, I think coin—I think it was coined by Taylor Lorenz at Washington Post and it's called a nimcel, which is, like, a niche micro-influencer. It's the worst term I've ever heard in my entire life. The nimcel [laugh]. Sorry, Taylor, it's terrible.But so I don't want to call myself a nimcel. I guess I have a group of people that enjoy the content, but you are so right that the group of people, once you get it, you get it. And if you don't get it, you may think some parts of it—like, you can kind of piece things together, but it's not as funny. But there's plenty of litigation jokes I'll make—like, where I'm talking to the judge. It's always these hypothetical scenarios—and you can maybe find it funny.But if you're a litigator who's gotten their ass kicked by a judge in a state court that just does not like you, you are not a local, they don't like the way you're presenting yourself, they don't like your argument, and they just dig you into the ground, you laugh. You laugh because you're, like, I've been there. I've had—or on the flip, you're the attorney that watched your opposing counsel go through it, you're like, “I remember that.” And you're right, it really you get such a great reaction from these folks, such great feedback, and they love it. They absolutely love it. But you're right, if you're outside, you're like, “Eh, it's kind of funny, but I don't really get all of it.”Corey: My mother approaches it this way whenever she talks to me like I have no idea what you're talking about, but you seem to really know what you're talking about, so I'm proud of you. It's like, “No, Mom, that is, like, the worst combination of everything.” It's like, “Well, are you any good at this thing?” “No. But I'm a white man, so I'm going to assume yes and the world will agree with me until proven otherwise.” So yeah, maybe nuclear physics ain't for you in that scenario.But yeah, the idea of finding your people, finding your audience, before the rise of the internet, none of this stuff would have worked just because you live in a town; how many attorneys are really going to be within the sound of your voice, hearing these stories? Not to mention the fact that everyone knows everyone's business in some of those places, and oh, you can't really subtweet the one person because they're also in the room. The world changes.Matt: The world changes. I've never had this happen. So, when I really started to get aggressive on, like, Twitter, I had already left private practice; I was in-house at that point. And I've always envisioned, I've always, I always want to, like, go back to private practice for one case: to go into a courtroom in, like, Miami, Florida, and sit there and commiserate and tell the stories of people again like I used to do—just like what you're saying—and see what everyone says. Say, “Hey, I saw you on Twitter. Hey, I saw this story on Twitter.”But in the same breath, like, you can't talk like you talk online in person, to some degree, right? Like, I can't make fun of opposing counsel because the judge is right there and opposing counsel was right there, and I'm honestly, knowing my luck, I'm about to get my ass kicked by opposing counsel. So, I probably should watch myself in that courtroom.Corey: But I'm going to revise the shit out of this history when it comes time to do my tweet after the fact. “And then everybody clapped.”Matt: [laugh]. I found five dollars outside the courtroom.Corey: Exactly. I really want to thank you for spending so much time chatting with me. If people want to learn more and follow your amazing shitpost antics on the internet, where's the best place for them to do it?Matt: Corey it's been an absolute pleasure. Instagram, TikTok, Twitter, LinkedIn. For everything but LinkedIn: @ItsMattsLaw. LinkedIn, just find me by my name: Matt Margolis.Corey: And we will put links to all of it in the [show notes 00:33:04]. Thank you so much for being so generous with your time. It's appreciated.Matt: I have not laughed as hard in a very, very long time. Corey, thank you so much.Corey: Matt Margolis, Head of Community at Lawtrades. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry, insulting comment that you've drafted the first time realized, oh wait, you're not literate, and then hired someone off of Lawtrades to help you write in an articulate fashion.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Screaming in the Cloud
Building Trust in the World of DevRel with Taylor Barnett

Screaming in the Cloud

Play Episode Listen Later Jan 3, 2023 29:41


About TaylorTaylor Barnett is a Staff Developer Advocate at PlanetScale. She is passionate about building great developer experiences emphasizing empathy within product, documentation, and other developer-facing projects. For the past decade, Taylor has worked at various data and API-focused startups in software development and developer relations. In her free time, as a firm believer in "touching grass," she's either gardening, taking long walks, climbing rocks with friends, trying to find the funkiest sour beers, or hanging out with her corgi, Yoda, and spouse in Austin, Texas.Links Referenced: PlanetScale: https://planetscale.com/ Twitter: https://twitter.com/taylor_atx Personal website: https://taylorbar.net TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: If you asked me to rank which cloud provider has the best developer experience, I'd be hard-pressed to choose a platform that isn't Google Cloud. Their developer experience is unparalleled and, in the early stages of building something great, that translates directly into velocity. Try it yourself with the Google for Startups Cloud Program over at cloud.google.com/startup. It'll give you up to $100k a year for each of the first two years in Google Cloud credits for companies that range from bootstrapped all the way on up to Series A. Go build something, and then tell me about it. My thanks to Google Cloud for sponsoring this ridiculous podcast.Corey: This episode is sponsored by our friends at Logicworks. Getting to the cloud is challenging enough for many places, especially maintaining security, resiliency, cost control, agility, etc, etc, etc. Things break, configurations drift, technology advances, and organizations, frankly, need to evolve. How can you get to the cloud faster and ensure you have the right team in place to maintain success over time? Day 2 matters. Work with a partner who gets it - Logicworks combines the cloud expertise and platform automation to customize solutions to meet your unique requirements. Get started by chatting with a cloud specialist today at snark.cloud/logicworks. That's snark.cloud/logicworksCorey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I'm joined this week by Taylor Barnett, Staff Developer Advocate at PlanetScale. Taylor, you're one of those people that I'm ashamed I haven't had on the show before now. Thanks for joining me.Taylor: You're welcome. Yeah, I'm glad to be here.Corey: We've been traveling in similar circles for a while now. And I lost track of a lot of those areas when the pandemic hit, you know, the global plague o'er the land. And during that time, it seemed like there was a lot of question that folks had about what is developer advocacy. What does DevRel become now? And now that we're largely on the other side of it—at least business is pretending that we're behind it—do we have an answer yet?Taylor: I hope so. I mean, I have an answer. Not sure if other businesses have figured that out yet. But no, I mean, to me, advocacy is still just that glue between company and a community. But I think one of the things that the pandemic has really, like, pushed that, you know, when there were no in-person events, was that it questioned what activities that actually looks like.You know, I see advocacy as a ton of different levers and you can tweak those levers to different levels. Before, it was largely a lot of in-person stuff—I will say I was doing less in-person, actually before than most; I was doing a little bit more content—then it had to become so content-focused. And I think now we're in this awkward place where in-person events have come back and we're still, like, figuring out, like, how do we do those? What does that look like? And we've actually—I think part of it is we've over-indexed now on content.I think part of that is because it is visible and it is measurable, and that's always a big topic [laugh] in developer relations is metrics. But also, I think we've lost track of the actual advocacy part: how do we actually advocate for users internally? It's just disappeared a little bit because we were so content-focused during the pandemic.Corey: I would say that's been a recurring theme with every DevRel person that I've spoken to that metrics are the bane of their existence. And I want to be clear, I'm not just talking about developer advocates, I'm talking about people who manage and run developer advocacy teams, I'm talking about executives who are trying to bring the appropriate context to strategic-level discussions around these things. All of the metrics that I have been able to uncover are wrong. But it's like the, ‘all models are wrong, but some models are useful' type of approach, where—Taylor: Yeah.Corey: Every time you start putting a metric around it and measuring people based upon the outcome of that metric, it ends in disaster. My one and only interview for a DevRel job in my past was my question for them was how do you measure success? “Well, we want to see you have talks accepted at some of the big tier-one conferences.” And they list a few examples, and it's, yeah, “I've spoken at for the ones you just listed in the past year, so… do I get a raise?” It's one of those areas where there's no right answer, but a lot of wrong ones.Taylor: Yeah. And one of the other troubling patterns that I've started to see also more is that in these cloud startups, they have DevRel programs now that are fairly young, we're talking not even a year old. Some in the recent DevRel survey results, it was about, like, 29% of programs are less than a year old. Within those programs, 43% of those people have not even been in a DevRel role for more than a year. So, not only do we have folks that haven't done this before, the startup has not done this before.And so, the metrics conversation is basically a shit show. People with the right experiences aren't in these roles and so they're not able to craft strategies and actually look at good metrics. And so, then we then over-index on the things like, “Oh, you wrote a blog post.” Great, you know, that's, like, some kind of metric. “It got X number of page views.” Great, that's some kind of metric.And it often incentivizes some of the wrong things. And so, then it just incentivizes more and more of this content creation, to just get those pageviews up. And it's scary to me because then we're just going back to the more evangelist type of developer relations and less of the advocacy type stuff where we're actually advocating for users internally.Corey: I would agree. I'd say that there's a problem where we have a, almost across the board, lack of understanding about—let's even start at the very beginning of when DevRel is required or when it's not. I mean, take where you work now at PlanetScale. You're effectively managed Vitess-as-a-service. That's a little on the technical side and is not the sort of thing that's going to necessarily lend itself to a mass-market marketing approach.This is not something to put on billboards outside of most highways, for example, but it does require engaging with people on a technical level. I keep joking but also serious when I refer to DevRel as meaning you work in marketing, but they're scared to tell you.Taylor: Yeah. No, I mean, I actually sometimes say, “Well, like, I'm secretly probably a pretty good product marketer, but I don't want developers to know that because then I'll lose my street cred from my actual development and engineering background.” And I have a computer science degree and, like, I'm actually, like [laugh], very, very technical. But the reality is, like, you know, somebody's got to write the words, sometimes.Corey: The words are harder when they go into people then they are into computers. At least with computers—Taylor: Exactly.Corey: It's pretty—it's a bounded problem space to some extent. With people, oh no, no, there's no consistency at all.Taylor: Yeah. And like, words mean different things to different people, especially, like, my favorite one lately is, like, what does edge mean? Nobody actually has one [laugh] definition of that word.Corey: Oh, I think most of them do. Edge always means, “Oh, it's a way of describing the thing that we've been doing for 15 years, but now want to sell into a hype cycle.”Taylor: Yeah, yeah. I mean, CDNs have been around for a while. You know, and that's really—like, what PlanetScale, it is, in some ways, we're challenging what people expect from their database. We think you can actually expect more from your database platform, and so there are things you know, to teach people about some of these newer ways of working with a database. And that requires needing to think about how we present that to users, but also hearing back from users how do we work within their applications, their stacks.We're MySQL. That's, you know, a trusted standard. It's been around for a while, so it works with many, but also, we're in this whole new paradigm of how to use a database. These are all new ideas and they require both a two-way street of both putting things out there—so content, not bad; it's still needed—but also things coming in and taking that, making it actionable, and talking about it internally.Corey: When you take a look at the DevRel world, what do you think that most organizations are missing or getting wrong about it? And yes, I understand that I'm basically asking you to start beef with a whole bunch of companies out there, but that's all right. It's what we do here.Taylor: Yeah, one of the things I love, [Matty 00:07:44] Stratton had this thing where I tweeted out a few months ago that we've over-indexed on content, and matty's reply was that we've over-indexed on being able to do cool shit that isn't connected to revenue because that somehow is dirty for DevRel to somehow be connected to revenue. I think, you know, a lot of times, there are ways that we can look at how do users actually get value from our products. Like, are they actually getting value? One way they express that is by paying for it. So therefore, we are then somehow connected to revenue.I mean, I want to build things, I want to work on platforms that deliver value, that people actually want to pay for because they see this is makes my life easier, somehow. But to do that, and again, we've got to talk to our users. We've got to figure out where do they actually value. What are the things that are just fluff? There's a lot of fluff out there.Sometimes if we don't listen to them, then we don't have to find out that what we're building is fluff. So, that's probably the part that could start some beefs. But it's the reality of lots of VC money and tooling and being able to build things super easily, it's a bunch of different factors coming together in this time.Corey: One of the things that I don't pretend to understand, but I'm going to roll with it anyway, is there's been a lot of discourse on where DevRel does not belong in an org chart. I don't have a terrific answer at this, but I do know that most of the answers I get from practitioners in the space are deeply dissatisfying. It seems that—not to be unkind or cast aspersions where they don't belong, but whenever I ask the question, everyone has a whole laundry list of wrong answers and very few right ones.Taylor: I honestly will say I don't care [laugh]. I mean, that's the reality.Corey: Corporate IT. Got it.Taylor: Do I want to be on a team that makes me directly responsible for qualified leads? No. That does not necessarily say anything about the team itself. That is just a metric. That is—you know, and that team exists in a larger system that has put certain pressures on it.Like, you know, there's, like, things, like, it's more about how a team looks at just doing the DevRel stuff and doing marketing in general, or how they do sales. You know, I know lots of developers hate to hate on sales—marketing, too—and I don't necessarily think sales and marketing are a bad thing, I think is the way we incentivize those roles create bad behaviors, and so maybe we should look at how we incentivize them. And so, I don't care what team I'm honestly on most of the time. I've been on a few different ones. As long as I get to do the developer advocacy work that I actually think is impactful for developers and actually making developers' lives better, I'm cool.Corey: It's my belief, on some level, that it's very easy to internalize a bad expression of it. You can have phenomenally well-empowered DevRel teams working in marketing—Taylor: Yep.Corey: —at some companies, and in other places, it can be an absolute disaster because they start putting metrics like number of qualified leads around you. And I can't shake the feeling that people internalize, “Well, we've reported marketing once and it was terrible,” without realizing the context of yeah, but in a terrible way, and an org that didn't really understand what you do. That doesn't necessarily mean that you should throw that whole baby out with the bathwater.Taylor: Yeah, I mean, we've all had bad managers. So, we're not going to say we're just never going to have a manager.Corey: Some people try that.Taylor: Is that what you've done [laugh]?Corey: Indirectly. No, I was talking about more about the holacracy companies where oh yeah, no one reports to anyone. It's really? Because everyone makes different amounts of money, so one wonders about that.Taylor: Yeah. But by far, we just go find better managers is what we often do, you know? And there's the whole phrase that, like, people don't leave companies, they leave managers. It's very true in my experience. And we don't just say, “All marketing teams bad, so I'm never going to join a marketing team.” We should say, “Let's just go find one that fits better.”Corey: I was very frustrated in my last couple of real jobs because so much of what I was doing was DevRel-like, but this was before that was an established and accepted thing in the places that I worked, so there were questions like, “Well, what is the value of you going to give a keynote at this conference?” And the honest answer was, “Yeah, I have no idea how to quantify it, but I know that if I do it, good things come out of it.” And that was a difficult battle to fight, whereas now when I decided to go work for myself, it's, “Yeah, I'm going to go speak there. I don't know what the ROI is. I know good things and maybe some useful things will come out of it. Maybe I'll learn something, but this is how we experiment and learn.” And that looks an awful lot to most traditional management types. Like I'm trying to justify a trip somewhere.Taylor: Yeah. And I think, you know, what's been also interesting, as I noticed, some people are starting to notice a lot of more junior people wanting to get into developer relations. And we sometimes actually are wondering, some of us in developer relations, if we've not always shown like the negative parts of that. What happens when you go do that keynote? What does that mean for your week leading up to that keynote? What does travel look like? What is, like, running across an airport wearing a mask and carrying your luggage look like?I think we don't always get to see that and so it looks a little bit less glamorous when people see that. And maybe they would be slightly less interested in the role or just, like, how do you handle working with, like, five different teams across a company to try to be like that glue piece between all of them to get something done? Like, there's a lot less glamorous parts that I'm hoping more people talk about because, like you said, it just looks like you're trying to go get a trip somewhere. I think the other thing is, like, even if you are having a keynote, I think one of the things that some people—they think one keynote is going to just wreck a budget. The reality is for our business, it will not do that, so why can't we, like, have a better balance of extremes?Like, you're not going to be giving ten of those keynotes in a year, maybe experiment doing two and see what comes out of doing two of them. But the other thing is, it's a long-term game and so you're not going to see something maybe the week after. It could be six months later. I had this one experience where someone actually told me—it was probably, like, a whole year after I had given a talk—that him and his teammates—this was back when people you know, went into offices—sat in an office and watched one of my old talks together. And I was just like, what, like, y'all, like, got together and did that?Corey: Yeah, you could have invited me and I could have delivered it for you in person and answered questions, but all right.Taylor: Yeah. It was like, what I was just like, oh my gosh, that is literally never happened to me. This was a few years ago. And then, too, I was like, that just made it worth it. If you asked a CEO, would you like to have an advocate go give a talk for a whole team at a company, they'd be like, “Yes, I want you—” especially if that's a big company and the name is shiny and they would love to have that as a customer, they would be, like, a hundred percent, “Go give that talk.”And so, I think many times, leadership needs to actually kind of check in on, like, is this really that much of a cost if it's just, like, one keynote? I've seen battles over really feels like stupid things sometimes. But everything in moderation is kind of the way I approach it.[midroll 00:15:17]Corey: One problem that I tended to see and I don't know how closely your experience mirrors my own, but it seemed, especially in the before times, right before the pandemic hit, that we were almost trapped in a downward spiral at a lot of the conferences because it felt like it was mostly becoming DevRels speaking to DevRel. And that wasn't the most inclusive thing for folks who used to wind up going to a lot of local conferences to learn from their local community and see how other people were solving the problems that they were solving. Instead, it felt like a bunch of DevRel types getting up there, in most cases giving a talk that was heavily alluding to why you should buy their product, if not an outright sales pitch for it. And it just felt like we're losing something. Do you think that's something that we've avoided, that we've pressed pause on, with the pandemic and now the recession, or do you think there's something else afoot?Taylor: I think that's still happening today, especially with, like, engineers wanting sometimes to travel less, you know, some people still have personal and family reasons for not traveling, so even less of them are wanting to speak. I don't think I saw, like, a huge swath of engineers, like, really excited to speak once conferences started in person again. They thought, “Oh, my gosh, I have to go talk to people in person again?” And so, it's still happening. I've seen it from an organizer's perspective.I used to organize the API specifications conference. There's tons of DevRel submissions in there, so you know, we really tried to spend time reaching out to companies that were member companies of the OpenAPI Initiative and get them to actually have member engineers from their teams come speak. I think DevRel has a role to internally advocate for engineers who are doing the day-to-day work, go speak at conferences. You know, I think many times engineers feel like, “Oh, what I have to talk about is not very interesting.” And I have to tell them, it is very interesting, and I would love to have you speak, and I'm here to help you, and you know, need help writing a CFP? I'm there. You need help putting together slides, practicing talks? I'm there.And I think DevRel can be kind of like these coaches for folks to go speak at conferences because the reality is attendees want to hear from them. They want to hear engineers from especially major companies or companies just doing really interesting engineering challenges speaking. And I think DevRel has a part in helping that happen. I've personally backed away from speaking the last six months, partially because I'm kind of not seeing as much value for myself doing it before I was doing a lot more, so I'm using that effort to try to advocate internally to help people CFPs. Last week, I helped a bunch of people KubeCon submissions, and then next week, I have other conferences I would love to—I have engineers that I've kind of picked out that I would love to have speak. And yeah, I'm glad to play a part in trying to improve that. And I think other advocates should, too.Corey: Where do you think that we're going as an industry? Because it became pretty clear for a couple of years that so much of what we were doing and how we were discussing it, it felt like there was a brief moment in time that we could really transform what we were doing and start to have a broader awareness that DevRel was more than giving talks on stage at conferences. And it feels like we squandered that opportunity and it mostly turned into, oh, now we're going to give the same talks, we're just going to do it to webcams, either pre-recorded—which was the better approach—or we're going to do it live, even though there's no interactive component to it, just introduce a whole bunch of different failure modes. I was disappointed. I liked some of the early stuff I saw coming out, like Desert Island DevOps, where they did it inside of Animal Crossing. Like I wanted to see more stuff like that, but it just seems like we didn't.Taylor: Yeah, I mean, the reality is, I think a lot of the online events have disappeared a lot in the last three or four months. And we're also seeing events trying to be hybrid. To me, a hybrid event is, like, throwing two events. Do you have an organizing team that can actually handle two concurrent events? It's hard.And API Specifications Conference, we did two years in person. Pretty niche conference. It's like the API nerds of the API nerds. And so, we still had pretty engaged attendees because there weren't any other sources of this, but then when everyone was starting to do the same content, attendees started checking out. They got tired of sitting in front of their monitors and watching talks.You know, we're seeing things coming back in person. I think it's going to be very interesting for the Spring because the Fall for me, it was probably one of my busiest conference seasons in terms of us just also sponsoring things. And I'm unsure of the return on investment today. We will see over time how that return on investment comes out, but I think it's going to change the way we look at the Spring, it's going to change the way we look at next Fall, and I think other companies are having the same conversations, too. And so, it's going to be like, okay, what do we do instead if we don't focus on conferences? I don't know. For me, that's focusing on the actual advocacy part, the user feedback, talking to users, building a product that people find value in. But for other teams, their team might not be in the place to do that. They might be expected to still produce this content in different ways, in-person, written, online.Corey: So, one of the burning questions that I think is not asked or addressed particularly well in the space has been, how do you get users to trust you? And to be clear, I am not saying you personally. It's like, “Well, given your history of flagrant lying and misleading people and scam after scam after scam, that is honestly impressive—” No, no, no, none of that. It's how do you—the indefinite you—build user trust?Taylor: Yeah, I think this is something we've seen, lots of companies of all sizes really struggle with. You know, the obvious thing I think many times companies think of is like, oh, if I'm open and transparent and have great docs, users will trust me. You know, I think that's part of it. I think the other thing that many often forget is that you need to listen to them, you need to take their feedback that they give you when you ask questions—and there's a whole, like, asking questions; I'm learning myself, like, how to ask better questions—how do you then make that actionable internally?You know, you have to understand who makes product decisions. Who do I need to talk to about this feature versus this other feature, and there's all these internal dynamics that you're then wading into. So, you have to get good at that. And then when you finally actually get some kind of change, whether that be some small paper cut of a thing related to a feature, or a big feature that you release, you actually go back to the user and you tell them, “Hey, look, we did this.” And what blows my mind is I do this, I take notes on who told me what feedback, and when that issue gets closed out, I go back to them and they're just shocked that I replied. They are shocked that I actually followed up. And to me, it's like such a basic thing, just following up. Doesn't seem, like, that hard.But it actually is hard but also useful. And you know, I think we've seen this so many times. We see—this is one example that I think about a lot, and I think you're familiar with this one too, Corey, the Aurora Serverless Data API in V1, people loved that. Then they came out with V2. There was no data API.And if you search that people are upset everywhere. And AWS keeps on telling them, “Nope, it's not going to happen.” And it's like, it's such an easy win if they actually listened to the user base. But there's countless examples of this, you know? There's things that we do at PlanetScale that we could improve on, you know, that users are telling us.There's only so much time in the day, but I think part of an advocate's job to wade through this feedback and figure out where can we bring the biggest value and the most impact. And, you know, I think all companies could benefit just from listening more and doing something about it.Corey: I wish that were a message that would get in front of the right people to make them a little bit more receptive. It feels like that's a message that is bandied around—to be direct—in DevRel circles an awful lot, but it doesn't seem to gain traction outside of that.Taylor: This kind of goes back to what we were talking about earlier with what team you're on. Sometimes that makes a huge difference, especially in larger companies. If you were siloed away in a marketing org—nothing bad about marketing, to be clear, but internally, you're seen as marketing—engineers, developers, see you as marketing. When you come with product feedback, they're kind of, “That's not your box. Go back to your box. Go back to your silo.”And you know, I think the reality is, we can't look at advocacy like that. I have users tell me things that they would never tell salesperson, they would never tell someone on our leadership team, they might tell someone in support. They tell me things. They send me emails that are multiple paragraphs long, giving positive and negative feedback. Many times it's positive, but I'm just shocked they'll even write that much, you know, positive. Like, they actually took out the time to do it.And they trusted that it was worth their time. I've done something right there if they're willing to do that at that point. And I, you know, I make sure I respond to every single one of those emails. I had someone ask me like, “Oh, do you want us to forward you all of them?” And I'm like, “Yes. Every single one. No matter what it says, I'm going to reply to this email.”Because then if I lose that trust, it's everything for me as an advocate. It's how I can help them, you know, see the value in the product, and help them with adoption, and bring them along to eventually paying, potentially—dirty word, revenue—but otherwise, I wouldn't have a job. So, you know, I think it's really something that startups, they think they see DevRel advocacy as content farms and not enough of the part that actually helps them make money.Corey: I really want to thank you for being so generous with your time. If people want to learn more, where's the best place for them to find you?Taylor: So, for now, I'm on Twitter as @taylor_atx. But if anything happens with that, as we know right now, you can also find me at taylorbar.net is my website. I'll always try to keep links of where I am on there. Trying to write more. We'll see if I accomplish that over the holidays. But yeah, that's the two places you can find me.Corey: And we will, of course, include links to that in the [show notes 00:26:27]. Thank you so much for your time. I appreciate it.Taylor: Yeah, thanks, Corey, for letting me rant, ramble, kind of have all these thoughts about advocacy. I'm hoping we can have a good 2023 in the world of DevRel and advocacy and make progress on some of these things.Corey: I sure hope you're right.Taylor: [laugh]. I hope I'm right, too, for the happiness of my job [laugh].Corey: Taylor Barnett, Staff Developer Advocate at PlanetScale. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment channeling a late Christmas spirit to tell us what the true meaning of DevRel was all along. Which will be wrong. Because it includes metrics.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

AWS Morning Brief
LastStrawPass

AWS Morning Brief

Play Episode Listen Later Dec 29, 2022 4:28 Very Popular


inks: AWS Lambda Security Threats and Mitigations LastPass now admits that hackers stole customers' password vaults. Google WordPress Plug-in Bug  McGraw Hill earned this week's S3 Bucket Negligence Award for exposing 100K students' grades Announcing the new security widget on AWS Console Home  Introducing the Security Design of the AWS Nitro System whitepaper  Please +1 my request to add support for an ~/.aws/config.d/ directory to the AWS cli. 

Screaming in the Cloud
Holiday Replay Edition - Inside the Mind of a DevOps Novelist with Gene Kim

Screaming in the Cloud

Play Episode Listen Later Dec 29, 2022 30:49


About GeneGene Kim is a multiple award-winning CTO, researcher and author, and has been studying high-performing technology organizations since 1999. He was founder and CTO of Tripwire for 13 years. He has written six books, including The Unicorn Project (2019), The Phoenix Project (2013), The DevOps Handbook (2016), the Shingo Publication Award winning Accelerate (2018), and The Visible Ops Handbook (2004-2006) series. Since 2014, he has been the founder and organizer of DevOps Enterprise Summit, studying the technology transformations of large, complex organizations.Links: The Phoenix Project: https://www.amazon.com/Phoenix-Project-DevOps-Helping-Business/dp/1942788290/ The Unicorn Project: https://www.amazon.com/Unicorn-Project-Developers-Disruption-Thriving/dp/B0812C82T9 The DevOps Enterprise Summit: https://events.itrevolution.com/ @RealGeneKim TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Cloud Economist Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: If you asked me to rank which cloud provider has the best developer experience, I'd be hard-pressed to choose a platform that isn't Google Cloud. Their developer experience is unparalleled and, in the early stages of building something great, that translates directly into velocity. Try it yourself with the Google for Startups Cloud Program over at cloud.google.com/startup. It'll give you up to $100k a year for each of the first two years in Google Cloud credits for companies that range from bootstrapped all the way on up to Series A. Go build something, and then tell me about it. My thanks to Google Cloud for sponsoring this ridiculous podcast.Corey: This episode is brought to us by our friends at Pinecone. They believe that all anyone really wants is to be understood, and that includes your users. AI models combined with the Pinecone vector database let your applications understand and act on what your users want… without making them spell it out. Make your search application find results by meaning instead of just keywords, your personalization system make picks based on relevance instead of just tags, and your security applications match threats by resemblance instead of just regular expressions. Pinecone provides the cloud infrastructure that makes this easy, fast, and scalable. Thanks to my friends at Pinecone for sponsoring this episode. Visit Pinecone.io to understand more.Corey Quinn: Welcome to Screaming in the Cloud. I'm Corey Quinn. I'm joined this week by a man who needs no introduction but gets one anyway. Gene Kim, most famously known for writing The Phoenix Project, but now the Wall Street Journal best-selling author of The Unicorn Project, six years later. Gene, welcome to the show.Gene Kim: Corey so great to be on. I was just mentioning before how delightful it is to be on the other side of the podcast. And it's so much smaller in here than I had thought it would be.Corey Quinn: Excellent. It's always nice to wind up finally meeting people whose work was seminal and foundational. Once upon a time, when I was a young, angry Unix systems administrator—because it's not like there's a second type of Unix administrator—[laughing] The Phoenix Project was one of those texts that was transformational, as far as changing the way I tended to view a lot of what I was working on and gave a glimpse into what could have been a realistic outcome for the world, or the company I was at, but somehow was simultaneously uplifting and incredibly depressing all at the same time. Now, The Unicorn Project does that exact same thing only aimed at developers instead of traditional crusty ops folks.Gene Kim: [laughing] Yeah, yeah. Very much so. Yeah, The Phoenix Project was very much aimed at ops leadership. So, Bill Palmer, the protagonist of that book was the VP of Operations at Parts Unlimited, and the protagonist in The Unicorn Project is Maxine Chambers, Senior Architect, and Developer, and I love the fact that it's told in the same timeline as The Phoenix Project, and in the first scene, she is unfairly blamed for causing the payroll outage and is exiled to The Phoenix Project, where she recoils in existential horror and then finds that she can't do anything herself. She can't do a build, she can't run her own tests. She can't, God forbid, do her own deploys. And I just love the opening third of the book where it really does paint that tundra that many developers find themselves in where they're just caught in decades of built-up technical debt, unable to do even the simplest things independently, let alone be able to independently develop tests or create value for customers. So, it was fun, very much fun, to revisit the Parts Unlimited universe.Corey Quinn: What I found that was fun about—there are few things in there I want to unpack. The first is that it really was the, shall we say, retelling of the same story in, quote/unquote, “the same timeframe”, but these books were written six years apart.Gene Kim: Yeah, and by the way, I want to first acknowledge all the help that you gave me during the editing process. Some of your comments are just so spot on with exactly the feedback I needed at the time and led to the most significant lift to jam a whole bunch of changes in it right before it got turned over to production. Yeah, so The Phoenix Project is told, quote, “in the present day,” and in the same way, The Unicorn Project is also told—takes place in the present day. In fact, they even start, plus or minus, on the same day. And there is a little bit of suspension of disbelief needed, just because there are certain things that are in the common vernacular, very much in zeitgeist now, that weren't six years ago, like “digital disruption”, even things like Uber and Lyft that feature prominently in the book that were just never mentioned in The Phoenix Project, but yeah, I think it was the story very much told in the same vein as like Ender's Shadow, where it takes place in the same timeline, but from a different perspective.Corey Quinn: So, something else that—again, I understand it's an allegory, and trying to tell an allegorical story while also working it into the form of a fictional work is incredibly complicated. That's something that I don't think people can really appreciate until they've tried to do something like it. But I still found myself, at various times, reading through the book and wondering, asking myself questions that, I guess, say more about me than they do about anyone else. But it's, “Wow, she's at a company that is pretty much scapegoating her and blaming her for all of us. Why isn't she quitting? Why isn't she screaming at people? Why isn't she punching the boss right in their stupid, condescending face and storming out of the office?” And I'm wondering how much of that is my own challenges as far as how life goes, as well as how much of it is just there for, I guess, narrative devices. It needed to wind up being someone who would not storm out when push came to shove.Gene Kim: But yeah, I think she actually does the last of the third thing that you mentioned where she does slam the sheet of paper down and say, “Man, you said the outage is caused by a technical failure and a human error, and now you're telling me I'm the human error?” And just cannot believe that she's been put in that position. Yeah, so thanks to your feedback and the others, she actually does shop her resume around. And starts putting out feelers, because this is no longer feeling like the great place to work that attracted her, eight years prior. The reality is for most people, is that it's sometimes difficult to get a new job overnight, even if you want to. But I think that Maxine stays because she believes in the mission. She takes a great deal of pride of what she's created over the years, and I think like most great brands, they do create a sense of mission and there's a deep sense of the customers they serve. And, there's something very satisfying about the work to her. And yeah, I think she is very much, for a couple of weeks, very much always thinking about, she won't be here for long, one way or another, but by the time she stumbles into the rebellion, the crazy group of misfits, the ragtag bunch of misfits, who are trying to find better ways of working and willing to break whatever rules it takes to take over the very ancient powerful order, she falls in love with a group. She found a group of kindred spirits who very much, like her, believe that developer productivity is one of the most important things that we can do as an organization. So, by the time that she looks up with that group, I mean, I think she's all thoughts of leaving are gone.Corey Quinn: Right. And the idea of, if you stick around, you can theoretically change things for the better is extraordinarily compelling. The challenge I've seen is that as I navigate the world, I've met a number of very gifted employees who, frankly wind up demonstrating that same level of loyalty and same kind of loyalty to companies that are absolutely not worthy of them. So my question has always been, when do I stick around versus when do I leave? I'm very far on the bailout as early as humanly possible side of that spectrum. It's why I'm a great consultant but an absolutely terrible employee.Gene Kim: [laughing] Well, so we were honored to have you at the DevOps Enterprise Summit. And you've probably seen that The Unicorn Project book is really dedicated to the achievements of the DevOps Enterprise community. It's certainly inspired by and dedicated to their efforts. And I think what was so inspirational to me were all these courageous leaders who are—they know what the mission is. I mean, they viscerally understand what the mission is and understand that the ways of working aren't working so well and are doing whatever they can to create better ways of working that are safer, faster, and happier. And I think what is so magnificent about so many of their journeys is that their organization in response says, “Thank you. That's amazing. Can we put you in a position of even more authority that will allow you to even make a more material, more impactful contribution to the organization?” And so it's been my observation, having run the conference for, now, six years, going on seven years is that this is a population that is being out promoted—has been promoted at a rate far higher than the population at large. And so for me, that's just an incredible story of grit and determination. And so yeah, where does grit and determination becomes sort of blind loyalty? That's ultimately self-punishing? That's a deep question that I've never really studied. But I certainly do understand that there is a time when no amount of perseverance and grit will get from here to there, and that's a fact.Corey Quinn: I think that it's a really interesting narrative, just to see it, how it tends to evolve, but also, I guess, for lack of a better term, and please don't hold this against me, it seems in many ways to speak to a very academic perspective, and I don't mean that as an insult. Now, the real interesting question is why I would think, well—why would accusing someone of being academic ever be considered as an insult, but my academic career was fascinating. It feels like it aligns very well with The Five Ideals, which is something that you have been talking about significantly for a long time. And in an academic setting that seems to make sense, but I don't see it thought of or spoken of in the same way on the ground. So first, can you start off by giving us an intro to what The Five Ideals are, and I guess maybe disambiguate the theory from the practice?Gene Kim: Oh for sure, yeah. So The Five Ideals are— oh, let's go back one step. So The Phoenix Project had The Three Ways, which were the principles for which you can derive all the observed DevOps practices from and The Four Types of Work. And so in The Five Ideals I used the concept of The Five Ideals and they are—the first—Corey Quinn: And the next version of The Nine whatever you call them at that point, I'm sure. It's a geometric progression.Gene Kim: Right or actually, isn't it the pri—oh, no. four isn't, four isn't prime. Yeah, yeah, I don't know. So, The Five Ideals is a nice small number and it was just really meant to verbalize things that I thought were very important, things I just gravitate towards. One is Locality and Simplicity. And briefly, that's just, to what degree can teams do what they need to do independently without having to coordinate, communicate, prioritize, sequence, marshal, deconflict, with scores of other teams. The Second Ideal is what I think the outcomes are when you have that, which is Focus, Flow and Joy. And so, Dr. Mihaly Csikszentmihalyi, he describes flow as a state when we are so engrossed in the work we love that we lose track of time and even sense of self. And that's been very much my experience, coding ever since I learned Clojure, this functional programming language. Third Ideal is Improvement of Daily Work, which shows up in The Phoenix Project to say that improvement daily work is even more important than daily work itself. Fourth Ideal is Psychological Safety, which shows up in the State of DevOps Report, but showed up prominently in Google's Project Oxygen, and even in the Toyota production process where clearly it has to be—in order for someone to pull the andon cord that potentially stops the assembly line, you have to have an environment where it's psychologically safe to do so. And then Fifth Ideal is Customer Focus, really focus on core competencies that create enduring, durable business value that customers are willing to pay for, versus context, which is everything else. And yeah, to answer your question, Where did it come from? Why do I think it is important? Why do I focus on that? For me, it's really coming from the State of DevOps Report, that I did with Dr. Nicole Forsgren and Jez Humble. And so, beyond all the numbers and the metrics and the technical practices and the architectural practices and the cultural norms, for me, what that really tells the story of is of The Five Ideals, as to what one of them is very much a need for architecture that allows teams to work independently, having a higher predictor of even, continuous delivery. I love that. And that from the individual perspective, the ideal being, that allows us to focus on the work we want to do to help achieve the mission with a sense of flow and joy. And then really elevating the notion that greatness isn't free, we need to improve daily work, we have to make it psychologically safe to talk about problems. And then the last one really being, can we really unflinchingly look at the work we do on an everyday basis and ask, what the customers care about it? And if customers don't care about it, can we question whether that work really should be done or not. So that's where for me, it's really meant to speak to some more visceral emotions that were concretized and validated through the State of DevOps Report. But these notions I am just very attracted to.Corey Quinn: I like the idea of it. The question, of course, is always how to put these into daily practice. How do you take these from an idealized—well, let's not call it a textbook, but something very similar to that—and apply it to the I guess, uncontrolled chaos that is the day-to-day life of an awful lot of people in their daily jobs.Gene Kim: Yeah. Right. So, the protagonist is Maxine and her role in the story, in the beginning, is just to recognize what not great looks like. She's lived and created greatness for all of her career. And then she gets exiled to this terrible Phoenix project that chews up developers and spits them out and they leave these husks of people they used to be. And so, she's not doing a lot of problem-solving. Instead, it's this recoiling from the inability for people to do builds or do their own tests or be able to do work without having to open up 20 different tickets or not being able to do their own deploys. She just recoil from this spending five days watching people do code merges, and for me, I'm hoping that what this will do, and after people read the book, will see this all around them, hopefully, will have a similar kind of recoiling reaction where they say, “Oh my gosh, this is terrible. I should feel as bad about this as Maxine does, and then maybe even find my fellow rebels and see if we can create a pocket of greatness that can become like the sublimation event in Dr. Thomas Kuhn's book, The Structure of Scientific Revolutions.” Create that kernel of greatness, of which then greatness then finds itself surrounded by even more greatness.Corey Quinn: What I always found to be fascinating about your work is how you wind up tying so many different concepts together in ways you wouldn't necessarily expect. For example, when I was reviewing one of your manuscripts before this went to print, you did reject one of my suggestions, which was just, retitle the entire thing. Instead of calling it The Unicorn Project. Instead, call it Gene Kim's Love Letter to Functional Programming. So what is up with that?Gene Kim: Yeah, to put that into context, for 25 years or more, I've self-identified as an ops person. The Phoenix Project was really an ops book. And that was despite getting my graduate degree in compiler design and high-speed networking in 1995. And the reason why I gravitated towards ops, because that was my observation, that that's where the saves were made. It was ops who saved the customer from horrendous, terrible developers who just kept on putting things into production that would then blow up and take everyone with it. It was ops protecting us from the bad adversaries who were trying to steal data because security people were so ineffective. But four years ago, I learned a functional programming language called Clojure and, without a doubt, it reintroduced the joy of coding back into my life and now, in a good month, I spend half the time—in the ideal—writing, half the time hanging out with the best in the game, of which I would consider this to be a part of, and then 20% of time coding. And I find for the first time in my career, in over 30 years of coding, I can write something for years on end, without it collapsing in on itself, like a house of cards. And that is an amazing feeling, to say that maybe it wasn't my inability, or my lack of experience, or my lack of sensibilities, but maybe it was just that I was sort of using the wrong tool to think with. That comes from the French philosopher Claude Lévi-Strauss. He said of certain things, “Is it a good tool to think with?” And I just find functional programming is such a better tool to think with, that notions like composability, like immutability, what I find so exciting is that these things aren't just for programming languages. And some other programming languages that follow the same vein are, OCaml, Lisp, ML, Elixir, Haskell. These all languages that are sort of popularizing functional programming, but what I find so exciting is that we see it in infrastructure and operations, too. So Docker is fundamentally immutable. So if you want to change a container, we have to make a new one. Kubernetes composes these containers together at the level of system of systems. Kafka is amazing because it usually reveals the desire to have this immutable data model where you can't change the past. Version control is immutable. So, I think it's no surprise that as our systems get more and more complex and distributed, we're relying on things like immutability, just to make it so that we can reason about them. So, it is something I love addressing in the book, and it's something I decided to double down on after you mentioned it. I'm just saying, all kidding aside is this a book for—Corey Quinn: Oh good, I got to make it worse. Always excited when that happens.Gene Kim: Yeah, I mean, your suggestion really brought to the forefront a very critical decision, which was, is this a book for technology leaders, or even business leaders, or is this a book developers? And, after a lot of soul searching, I decided no, this is a book for developers, because I think the sensibilities that we need to instill and the awareness we need to create these things around are the developers and then you just hope and pray that the book will be good enough that if enough engineers like it, then engineering leaders will like it. And if enough engineering leaders like it, then maybe some business leaders will read it as well. So that's something I'm eagerly seeing what will happen as the weeks, months, and years go by. Corey Quinn: This episode is sponsored in part by DataStax. The NoSQL event of the year is DataStax Accelerate in San Diego this May from the 11th through the 13th. I've given a talk previously called the myth of multi-cloud, and it's time for me to revisit that with... A sequel! Which is funny given that it's a NoSQL conference, but there you have it. To learn more, visit datastax.com that's D-A-T-A-S-T-A-X.com and I hope to see you in San Diego. This May.Corey Quinn: One thing that I always admired about your writing is that you can start off trying to make a point about one particular aspect of things. And along the way you tie in so many different things, and the functional programming is just one aspect of this. At some point, by the end of it, I half expected you to just pick a fight over vi versus Emacs, just for the sheer joy you get in effectively drawing interesting and, I guess, shall we say, the right level of conflict into it, where it seems very clear that what you're talking about is something thing that has the potential to be transformative and by throwing things like that in you're, on some level, roping people in who otherwise wouldn't weigh in at all. But it's really neat to watch once you have people's attention, just almost in spite of what they want, you teach them something. I don't know if that's a fair accusation or not, but it's very much I'm left with the sense that what you're doing has definite impact and reverberations throughout larger industries.Gene Kim: Yeah, I hope so. In fact, just to reveal this kind of insecurity is, there's an author I've read a lot of and she actually read this blog post that she wrote about the worst novel to write, and she called it The Yeomans Tour of the Starship Enterprise. And she says, “The book begins like this: it's a Yeoman on the Starship Enterprise, and all he does is admire the dilithium crystals, and the phaser, and talk about the specifications of the engine room.” And I sometimes worry that that's what I've done in The Unicorn Project, but hopefully—I did want to have that technical detail there and share some things that I love about technology and the things I hate about technology, like YAML files, and integrate that into the narrative because I think it is important. And I would like to think that people reading it appreciate things like our mutual distaste of YAML files, that we've all struggled trying to escape spaces and file names inside of make files. I mean, these are the things that are puzzles we have to solve, but they're so far removed from the business problem we're trying to solve that really, the purpose of that was trying to show the mistake of solving puzzles in our daily work instead of solving real problems.Corey Quinn: One thing that I found was really a one-two punch, for me at least, was first I read and give feedback on the book and then relatively quickly thereafter, I found myself at my first DevOps Enterprise Summit, and I feel like on some level, I may have been misinterpreted when I was doing my live-tweeting/shitposting-with-style during a lot of the opening keynotes, and the rest, where I was focusing on how different of a conference it was. Unlike a typical DevOps Days or big cloud event, it wasn't a whole bunch of relatively recent software startups. There were serious institutions coming out to have conversations. We're talking USAA, we're talking to US Air Force, we're talking large banks, we're talking companies that have a 200-year history, where you don't get to just throw everything away and start over. These are companies that by and large, have, in many ways, felt excluded to some extent, from the modern discussions of, well, we're going to write some stuff late at night, and by the following morning, it's in production. You don't get to do that when you're a 200-year-old insurance company. And I feel like that was on some level interpreted as me making fun of startups for quote/unquote, “not being serious,” which was never my intention. It's just this was a different conversation series for a different audience who has vastly different constraints. And I found it incredibly compelling and I intend to go back.Gene Kim: Well, that's wonderful. And, in fact, we have plans for you, Mr. Quinn.Corey Quinn: Uh-oh.Gene Kim: Yeah. I think when I say I admire the DevOps Enterprise community. I mean that I'm just so many different dimensions. The fact that these, leaders and—it's not leaders just in terms of seniority on the organization chart—these are people who are leading technology efforts to survive and win in the marketplace. In organizations that have been around sometimes for centuries, Barclays Bank was founded in the year 1634. That predates the invention of paper cash. HMRC, the UK version of the IRS was founded in the year 1200. And, so there's probably no code that goes that far back, but there's certainly values and—Corey Quinn: Well, you'd like to hope not. Gene Kim: Yeah, right. You never know. But there are certainly values and traditions and maybe even processes that go back centuries. And so that's what's helped these organizations be successful. And here are a next generation of leaders, trying to make sure that these organizations see another century of greatness. So I think that's, in my mind, deeply admirable.Corey Quinn: Very much so. And my only concern was, I was just hoping that people didn't misinterpret my snark and sarcasm as aimed at, “Oh, look at these crappy—these companies are real companies and all those crappy SAS companies are just flashes in the pan.” No, I don't believe that members of the Fortune 500 are flash in the pan companies, with a couple notable exceptions who I will not name now, because I might want some of them on this podcast someday. The concern that I have is that everyone's work is valuable. Everyone's work is important. And what I'm seeing historically, and something that you've nailed, is a certain lack of stories that apply to some of those organizations that are, for lack of a better term, ossified into their current process model, where they there's no clear path for them to break into, quote/unquote, “doing the DevOps.”Gene Kim: Yeah. And the business frame and the imperative for it is incredible. Tesla is now offering auto insurance bundled into the car. Banks are now having to compete with Apple. I mean, it is just breathtaking to see how competitive the marketplaces and the need to understand the customer and deliver value to them quickly and to be able to experiment and innovate and out-innovate the competition. I don't think there's any business leader on the planet who doesn't understand that software is eating the world and they have to that any level of investment they do involves software at some level. And so the question is, for them, is how do they get educated enough to invest and manage and lead competently? So, to me it really is like the sleeping giant awakening. And it's my genuine belief is that the next 50 years, as much value as the tech giants have created: Facebook, Amazon, Netflix, Google, Microsoft, they've generated trillions of dollars of economic value. When we can get eighteen million developers, as productive as an engineer at a tech giant is, that will generate tens of trillions of dollars of economic value per year. And so, when you generate that much economic activity, all problems become solvable, you look at climate change, you take a look at the disparity between rich and poor. All things can be fixed when you significantly change the economic economy in this way. So, I'm extremely hopeful and I know that the need for things like DevOps are urgent and important.Corey Quinn: I guess that that's probably the best way of framing this. So you wrote one version that was aimed at operators back in 2013, this one was aimed at developers, and effectively retails and clarifies an awful lot of the same points. As a historical ops person, I didn't feel left behind by The Unicorn Project, despite not being its target market. So I guess the question on everyone's mind, are you planning on doing a third iteration, and if so, for what demographic?Gene Kim: Yeah, nothing at this point, but there is one thing that I'm interested in which is the role of business leaders. And Sarah is an interesting villain. One of my favorite pieces of feedback during the review process was, “I didn't think I could ever hate Sarah more. And yet, I did find her even to be more loathsome than before.” She's actually based on a real person, someone that I worked with.Corey Quinn: That's the best part, is these characters are relatable enough that everyone can map people they know onto various aspects of them, but can't ever disclose the entire list in public because that apparently has career consequences.Gene Kim: That's right. Yes, I will not say who the character is based on but there's, in the last scene of the book that went to print, Sarah has an interesting interaction with Maxine, where they meet for lunch. And, I think the line was, “And it wasn't what Maxine had thought, and she's actually looking forward to the next meeting.” I think that leaves room for it. So one of the things I want to do with some friends and colleagues is just understand, why does Sarah act the way she does? I think we've all worked with someone like her. And there are some that are genuinely bad actors, but I think a lot of them are doing something, based on genuine, real motives. And it would be fun, I thought, to do something with Elizabeth Henderson, who we decided to start having a conversation like, what does she read? What is her background? What is she good at? What does her resume look like? And what caused her to—who in technology treated her so badly that she treats technology so badly? And why does she behave the way she does? And so I think she reads a lot of strategy books. I think she is not a great people manager, I think she maybe has come from the mergers and acquisition route that viewed people as fungible. And yeah, I think she is definitely a creature of economics, was lured by an external investor, about how good it can be if you can extract value out of the company, squeeze every bit of—sweat every asset and sell the company for parts. So I would just love to have a better understanding of, when people say they work with someone like a Sarah, is there a commonality to that? And can we better understand Sarah so that we can both work with her and also, compete better against her, in our own organizations?Corey Quinn: I think that's probably a question best left for people to figure out on their own, in a circumstance where I can't possibly be blamed for it.Gene Kim: [laughing].That can be arranged, Mr. Quinn.Corey Quinn: All right. Well, if people want to learn more about your thoughts, ideas, feelings around these things, or of course to buy the book, where can they find you?Gene Kim: If you're interested in the ideas that are in The Unicorn Project, I would point you to all of the freely available videos on YouTube. Just Google DevOps Enterprise Summit and anything that's on the plenary stage are specifically chosen stories that very much informed The Unicorn Project. And the best way to reach me is probably on Twitter. I'm @RealGeneKim on Twitter, and feel free to just @ mention me, or DM me. Happy to be reached out in whatever way you can find me. Corey Quinn: You know where the hate mail goes then. Gene, thank you so much for taking the time to speak with me, I appreciate it.Gene Kim: And Corey, likewise, and again, thank you so much for your unflinching feedback on the book and I hope you see your fingerprints all over it and I'm just so delighted with the way it came out. So thanks to you, Corey. Corey Quinn: As soon as my signed copy shows up, you'll be the first to know.Gene Kim: Consider it done. Corey Quinn: Excellent, excellent. That's the trick, is to ask people for something in a scenario in which they cannot possibly say no. Gene Kim, multiple award-winning CTO, researcher, and author. Pick up his new book, The Wall Street Journal best-selling The Unicorn Project. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on Apple Podcasts. If you hated this podcast, please leave a five-star review on Apple Podcasts and leave a compelling comment.Announcer: This has been this week's episode of Screaming in the Cloud. You can also find more Corey at ScreamingintheCloud.com or wherever fine snark is sold.This has been a HumblePod production. Stay humble.