POPULARITY
Robert Ross, CEO and Co-Founder at FireHydrant, joins Corey on Screaming in the Cloud to discuss how being an on-call engineer fighting incidents inspired him to start his own company. Robert explains how FireHydrant does more than just notify engineers of an incident, but also helps them to be able to effectively put out the fire. Robert tells the story of how he “accidentally” started a company as a result of a particularly critical late-night incident, and why his end goal at FireHydrant has been and will continue to be solving the problem, not simply choosing an exit strategy. Corey and Robert also discuss the value and pricing models of other incident-reporting solutions and Robert shares why he feels surprised that nobody else has taken the same approach FireHydrant has. About RobertRobert Ross is a recovering on-call engineer, and the CEO and co-founder at FireHydrant. As the co-founder of FireHydrant, Robert plays a central role in optimizing incident response and ensuring software system reliability for customers. Prior to founding FireHydrant, Robert previously contributed his expertise to renowned companies like Namely and Digital Ocean. Links Referenced: FireHydrant: https://firehydrant.com/ Twitter: https://twitter.com/bobbytables TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Developers are responsible for more than ever these days. Not just the code they write, but also the containers and cloud infrastructure their apps run on. And a big part of that responsibility is app security — from code to cloud. That's where Snyk comes in. Snyk is a frictionless security platform that meets teams where they are, automating application security controls across their existing tools, workflows, and the AWS application stack — including seamless integrations with AWS CodePipeline, Amazon EKS, Amazon Inspector and several others. I'm a customer myself. Deploy on AWS. Secure with Snyk. Learn more at snyk.co/scream. That's S-N-Y-K-dot-C-O/scream.Corey: Welcome to Screaming in the Cloud, I'm Corey Quinn. And this featured guest episode is brought to us by our friends at FireHydrant and for better or worse, they've also brought us their CEO and co-founder, Robert Ross, better known online as Bobby Tables. Robert, thank you for joining us.Robert: Super happy to be here. Thanks for having me.Corey: Now, this is the problem that I tend to have when I've been tracking companies for a while, where you were one of the only people that I knew of at FireHydrant. And you kind of still are, so it's easy for me to imagine that, oh, it's basically your own side project that turned into a real job, sort of, side hustle that's basically you and maybe a virtual assistant or someone. I have it on good authority—and it was also signaled by your Series B—that there might be more than just you over there now.Robert: Yes, that's true. There's a little over 60 people now at the company, which is a little mind-boggling for me, starting from side projects, building this in Starbucks to actually having people using the thing and being on payroll. So, a little bit of a crazy thing for me. But yes, over 60.Corey: So, I have to ask, what is it you folks do? When you say ‘fire hydrant,' the first thing that I think I was when I was a kid getting yelled at by the firefighter for messing around with something I probably shouldn't have been messing around with.Robert: So, it's actually very similar where I started it because I was messing around with software in ways I probably shouldn't have and needed a fire hydrant to help put out all the fires that I was fighting as an on-call engineer. So, the name kind of comes from what do you need when you're putting out a fire? A fire hydrant. So, what we do is we help people respond to incidents really quickly, manage them from ring to retro. So, the moment you declare an incident, we'll do all the timeline tracking and eventually help you create a retrospective at the very end. And it's been a labor of love because all of that was really painful for me as an engineer.Corey: One of the things that I used to believe was that every company did something like this—and maybe they do, maybe they don't—I'm noticing these days an increasing number of public companies will never admit to an incident that very clearly ruined things for their customers. I'm not sure if they're going to talk privately to customers under NDAs and whatnot, but it feels like we're leaving an era where it was an expectation that when you had a big issue, you would do an entire public postmortem explaining what had happened. Is that just because I'm not paying attention to the right folks anymore, or are you seeing a downturn in that?Robert: I think that people are skittish of talking about how much reliability they—or issues they may have because we're having this weird moment where people want to open more incidents like the engineers actually want to say we have more incidents and officially declare those, and in the past, we had these, like, shadow incidents that we weren't officially going to say it was an incident, but was a pretty big deal, but we're not going to have a retro on it so it's like it didn't happen. And kind of splitting the line between what's a SEV1, when should we actually talk about this publicly, I think companies are still trying to figure that out. And then I think there's also opposing forces. We talk to folks and it's, you know, public relations will sometimes get involved. My general advice is, like, you should be probably talking about it no matter what. That's how you build trust.It's trust, with incidences, lost in buckets and gained back in drops, so you should be more public about it. And I think my favorite example is a major CDN had a major incident and it took down, like, the UK government website. And folks can probably figure out who I'm talking about, but their stock went up the next day. You would think that a major incident taking down a large portion of the internet would cause your stock to go down. Not the case. They were on it like crazy, they communicated about it like crazy, and lo and behold, you know, people were actually pretty okay with it as far as they could be at the end of the day.Corey: The honest thing that really struck me about that was I didn't realize that CDN that you're referencing was as broadly deployed as it was. Amazon.com took some downtime as a result of this.Robert: Yeah.Corey: It's, “Oh, wow. If they're in that many places, I should be taking them more seriously,” was my takeaway. And again, I don't tend to shame folks for incidents because as soon as you do that, they stopped talking about them. They still have them, but then we all lose the ability to learn from them. I couldn't help but notice that the week that we're recording this, so there was an incident report put out by AWS for a Lambda service event in Northern Virginia.It happened back in June, we're recording this late in October. So, it took them a little bit of time to wind up getting it out the door, but it's very thorough, very interesting as far as what it talks about as far as their own approach to things. Because otherwise, I have to say, it is easy as a spectator slash frustrated customer to assume the absolute worst. Like, you're sitting around there and like, “Well, we have a 15-minute SLA on this, so I'm going to sit around for 12 minutes and finish my game of solitaire before I answer the phone.” No, it does not work that way. People are scrambling behind the scenes because as systems get more complicated, understanding the interdependencies of your own system becomes monstrous.I still remember some of the very early production engineering jobs that I had where—to what you said a few minutes ago—oh, yeah, we'll just open an incident for every alert that goes off. Then we dropped a [core switch 00:05:47] and Nagio sent something like 8000 messages inside of two minutes. And we would still, 15 years later, not be done working through that incident backlog had we done such a thing. All of this stuff gets way harder than you would expect as soon as your application or environment becomes somewhat complicated. And that happens before you realize it.Robert: Yeah, much faster. I think that, in my experience, there's a moment that happens for companies where maybe it's the number of customers you have, number of servers you're running in production, that you have this, like, “Oh, we're running a big workload right now in a very complex system that impacts people's lives, frankly.” And the moment that companies realize that is when you start to see, like, oh, process change, you build it, you own it, now we have an SRE team. Like, there's this catalyst that happens in all of these companies that triggers this. And it's—I don't know, from my perspective, it's coming at a faster rate than people probably realize.Corey: From my perspective, I have to ask you this question, and my apologies in advance if it's one of those irreverent ones, but do you consider yourself to be an observability company?Robert: Oh, great question. No. No, actually. We think that we are the baton handoff between an observability tool and our platform. So, for example, we think that that's a good way to kind of, you know, as they say, monitor the system, give reports on that system, and we are the tool that based on that monitor may be going off, you need to do something about it.So, for example, I think of it as like a smoke detector in some cases. Like, in our world, like that's—the smoke detector is the thing that's kind of watching the system and if something's wrong, it's going to tell you. But at that point, it doesn't really do anything that's going to help you in the next phase, which is managing the incident, calling 911, driving to the scene of the fire, whatever analogies you want to use. But I think the value-add for the observability tools and what they're delivering for businesses is different than ours, but we touch each other, like, very much so.Corey: Managing an incident when something happens and diagnosing what is the actual root cause of it, so to speak—quote-unquote, “Root cause.” I know people have very strong opinions on—Robert: Yeah, say the word [laugh].Corey: —that phrase—exactly—it just doesn't sound that hard. It is not that complicated. It's, more or less, a bunch of engineers who don't know what they're actually doing, and why are they running around chasing this stuff down is often the philosophy of a lot of folks who have never been in the trenches dealing with these incidents themselves. I know this because before I was exposed to scale, that's what I thought and then, oh, this is way harder than you would believe. Now, for better or worse, an awful lot of your customers and the executives at those customers did, for some strange reason, not come up through production engineering as the thing that they've done. They are executives, so it feels like it would be a challenging conversation to have with them, but one thing that you've got in your back pocket, which I always love talking to folks about, is before this, you were an engineer and then you became a CEO of a reasonably-sized company. That is a very difficult transition. Tell me about it.Robert: Yeah. Yeah, so a little of that background. I mean, I started writing code—I've been writing code for two-thirds of my life. So, I'm 32 now; I'm relatively young. And my first job out of high school—skipping college entirely—was writing code. I was 18, I was working in a web dev shop, I was making good enough money and I said, you know what? I don't want to go to college. That sounds—I'm making money. Why would I go to college?And I think it was a good decision because I got to be able—I was right kind of in the centerpiece of when a lot of really cool software things were happening. Like, DevOps was becoming a really cool term and we were seeing the cloud kind of emerge at this time and become much more popular. And it was a good opportunity to see all this confluence of technology and people and processes emerge into what is, kind of like, the base plate for a lot of how we build software today, starting in 2008 and 2009. And because I was an on-call engineer during a lot of that, and building the systems as well, that I was on call for, it meant that I had a front-row seat to being an engineer that was building things that was then breaking, and then literally merging on GitHub and then five minutes later [laugh], seeing my phone light up with an alert from our alerting tool. Like, I got to feel the entire process.And I think that that was nice because eventually one day, I snapped. And it was after a major incident, I snapped and I said, “There's no tool that helps me during this incident. There's no tool that kind of helps me run a process for me.” Because the only thing I care about in the middle of the night is going back to bed. I don't have any other priority [laugh] at 2 a.m.So, I wanted to solve the problem of getting to the fire faster and extinguishing it by automating as much as I possibly could. The process that was given to me in an outdated Confluence page or Google Doc, whatever it was, I wanted to automate that part so I could do the thing that I was good at as an engineer: put out the fire, take some notes, and then go back to bed, and then do a retrospective sometime next day or in that week. And it was a good way to kind of feel the problem, try to build a solution for it, tweak a little bit, and then it kind of became a company. I joke and I say on accident, actually.Corey: I'll never forget one of the first big, hairy incidents that I had to deal with in 2009, where my coworker had just finished migrating the production environment over to LDAP on a Thursday afternoon and then stepped out for a three-day weekend, and half an hour later, everything started exploding because LDAP will do that. And I only had the vaguest idea of how LDAP worked at all. This was a year into my first Linux admin job; I'd been a Unix admin before that. And I suddenly have the literal CEO of the company breathing down my neck behind me trying to figure out what's going on and I have no freaking idea of myself. And it was… feels like there's got to be a better way to handle these things.We got through. We wound up getting it back online, no one lost their job over it, but it was definitely a touch-and-go series of hours there. And that was a painful thing. And you and I went in very different directions based upon experiences like that. I took a few more jobs where I had even worse on-call schedules than I would have believed possible until I started this place, which very intentionally is centered around a business problem that only exists during business hours. There is no 2 a.m. AWS billing emergency.There might be a security issue masquerading as one of those, but you don't need to reach me out of business hours because anything that is a billing problem will be solved in Seattle's timeline over a period of weeks. You leaned into it and decided, oh, I'm going to start a company to fix all of this. And okay, on some level, some wit that used to work here, wound up once remarking that when an SRE doesn't have a better idea, they start a monitoring company.Robert: [laugh].Corey: And, on some level, there's some validity to it because this is the problem that I know, and I want to fix it. But you've differentiated yourself in a few key ways. As you said earlier, you're not an observability company. Good for you.Robert: Yeah. That's a funny quote.Corey: Pete Cheslock. He has a certain way with words.Robert: Yeah [laugh]. I think that when we started the company, it was—we kind of accidentally secured funding five years ago. And it was because this genuinely was something I just, I bought a laptop for because I wanted to own the IP. I always made sure I was on a different network, if I was going to work on the company and the tool. And I was just writing code because I just wanted to solve the problem.And then some crazy situation happened where, like, an investor somehow found FireHydrant because they were like, “Oh, this SRE thing is a big space and incidents is a big part of it.” And we got to talking and they were like, “Hey, we think what you're building is valuable and we think you should build a company here.” And I was—like, you know, the Jim Carrey movie, Yes Man? Like, that was kind of me in that moment. I was like, “Sure.” And here we are five years later. But I think the way that we approached the problem was let's just solve our own problem and let's just build a company that we want to work at.And you know, I had two co-founders join me in late 2018 and that's what we told ourselves. We said, like, “Let's build a company that we want to work for, that solves problems that we have had, that we care about solving.” And I think it's worked out, you know? We work with amazing companies that use our tool—much to their chagrin [laugh]—multiple times a day. It's kind of a problem when you build an incident response tool is that it's a good thing when people are using it, but a bad thing for them.Corey: I have to ask of all of the different angles to approach this from, you went with incident management as opposed to focusing on something that is more purely technical. And I don't say that in any way that is intended to be sounding insulting, but it's easier from an engineering mind to—having been one myself—to come up with, “Here's how I make one computer talk to his other computer when the following event happens.” That's a much easier problem by orders of magnitude than here's how I corral the humans interacting with that computer's failure to talk to another computer in just the right way. How did you get onto this path?Robert: Yeah. The problem that we were trying to solve for it was the getting the right people in the room problem. We think that building services that people own is the right way to build applications that are reliable and stable and easier to iterate on. Put the right people that build that software, give them, like, the skin in the game of also being on call. And what that meant for us is that we could build a tool that allowed people to do that a lot easier where allowing people to corral the right people by saying, “This service is broken, which powers this functionality, which means that these are the people that should get involved in this incident as fast as possible.”And the way we approached that is we just built up part of our functionality called Runbooks, where you can say, “When this happens, do this.” And it's catered for incidents. So, there's other tools out there, you can kind of think of as, like, we're a workflow tool, like Zapier, or just things that, like, fire webhooks at services you build and that ends up being your incident process. But for us, we wanted to make it, like, a really easy way that a project manager could help define the process in our tool. And when you click the button and say, “Declare Incident: LDAP is Broken,” and I have a CEO standing behind me, our tool just would corral the people for you.It was kind of like a bat signal in the air, where it was like, “Hey, there's this issue. I've run all the other process. I just need you to arrive at and help solve this problem.” And we think of it as, like, how can FireHydrant be a mech suit for the team that owns incidents and is responsible for resolving them?Corey: There are a few easier ways to make a product sound absolutely ridiculous than to try and pitch it to a problem that it is not designed to scale to. What is the ‘you must be at least this tall to ride' envisioning for FireHydrant? How large slash complex of an organization do you need to be before this starts to make sense? Because I promise, as one person with a single website that gets no hits, that is probably not the best place for—Robert: Probably not.Corey: To imagine your ideal user persona.Robert: Well, I'm sure you get way more hits than that. Come on [laugh].Corey: It depends on how controversial I'm being in a given week.Robert: Yeah [laugh].Corey: Also, I have several ridiculous, nonsense apps out there, but honestly, those are for fun. I don't charge people for them, so they can deal with my downtime till I get around to it. That's the way it works.Robert: Or, like, spite-visiting your website. No it's—for us, we think that the ‘must be this tall' is when do you have, like, sufficiently complicated incidents? We tell folks, like, if you're a ten-person shop and you have incidents, you know, just use our free tier. Like, you need something that opens a Slack channel? Fine. Use our free tier or build something that hits the Slack API [unintelligible 00:18:18] channel. That's fine.But when you start to have a lot of people in the room and multiple pieces of functionality that can break and multiple people on call, that's when you probably need to start to invest in incident management. Because it is a return on investment, but there is, like, a minimum amount of incidents and process challenges that you need to have before that return on investment actually, I would say, comes to fruition. Because if you do think of, like, an incident that takes downtime, or you know, you're a retail company and you go down for, let's say, ten minutes, and your number of sales per hour is X, it's actually relatively simple for that type of company to understand, okay, this is how much impact we would need to have from an incident management tool for it to be valuable. And that waterline is actually way—it's way lower than I think a lot of people realize, but like you said, you know, if you have a few 100 visitors a day, it's probably not worth it. And I'll be honest there, you can use our free tier. That's fine.Corey: Which makes sense. It's challenging to wind up-sizing things appropriately. Whenever I look at a pricing page, there are two things that I look for. And incidentally, when I pull up someone's website, I first make a beeline for pricing because that is the best way I found for a lot of the marketing nonsense words to drop away and it get down to brass tacks. And the two things I want are free tier or zero-dollar trial that I can get started with right now because often it's two in the morning and I'm trying to see if this might solve a problem that I'm having.And I also look for the enterprise tier ‘contact us' because there are big companies that do not do anything that is not custom nor do they know how to sign a check that doesn't have two commas in it. And whatever is between those two, okay, that's good to look at to figure out what dimensions I'm expected to grow on and how to think about it, but those are the two tent poles. And you've got that, but pricing is always going to be a dark art. What I've been seeing across the industry. And if we put it under the broad realm of things that watch your site and alert you and help manage those things, there are an increasing number of, I guess what I want to call component vendors, where you'll wind up bolting together a couple dozen of these things together into an observability pipeline-style thing, and each component seems to be getting extortionately expensive.Most of the wake-up-in-the-middle-of-the-night services that will page you—and there are a number of them out there—at a spot check of these, they all cost more per month per user than Slack, the thing that most of us to end up living within. This stuff gets fiendishly expensive, fiendishly quickly, and at some point, you're looking at this going, “The outage is cheaper than avoiding the outage through all of these things. What are we doing here?” What's going on in the industry, other than ‘money printing machine stopped going brrr' in quite the same way?Robert: Yeah, I think that for alerting specifically, this is a big part of, like, the journey that we wanted to have in FireHydrant was like, we also want to help folks with the alerting piece. So, I'll focus on that, which is, I think that the industry around notifying people for incidents—texts, call, push notifications, emails, there's a bunch of different ways to do it—I think where it gets really crazy expensive as in this per-seat model that most of them seem to have landed on. And we're per-seat for, like, the core platform of FireHydrant—so you know, before people spite-visit FireHydrant, look at our pricing pitch—but we're per-seat there because the value there is, like, we're the full platform for the service catalog retrospectives, Runbooks, like, there's a whole other component of FireHydrant—status pages—but when it comes to alerting, like, in my opinion, that should be active user for a few reasons. I think that if you're going to have people responding to incidents and the value from us is making sure they get to that incident very quickly because we wake them up in the middle of the night, we text them, we call them we make their Hue lights turn red, whatever it is, then that's, like, the value that we're delivering at that moment in time, so that's how we should probably invoice you.And I think that what's happened is that the pricing for these companies, they haven't innovated on the product in a way that allows them to package that any differently. So, what's happened, I think, is that the packaging of these products has been almost restrictive in the way that they could change their pricing models because there's nothing much more to package on. It's like, cool there's an alerting aspect to this, but that's what people want to buy those tools for. They want to buy the tool so it wakes them up. But that tool is getting more expensive.There was even a price increase announced today for a big one [laugh] that I've been publicly critical of. That is crazy expensive for a tool that texts you and call you. And what peo—what's going on now are people are looking, they're looking at the pricing sheet for Twilio and going, “What the heck is going on?” Like, I—to send a text on Twilio in the United States is fractions of a penny and here we are paying $40 a user for that person to receive six texts that month because of a webhook that hit an HCP server and, like, it's supposed to call that person? That's kind of a crazy model if you think about it. Like, engineers are kind of going, “Wait a minute. What's up here?” Like, and when engineers start thinking, “I could build this on a weekend,” like, something's wrong, like, with that model. And I think that people are starting to think that way.Corey: Well engineers, to be fair, will think that about an awful lot of stuff.Robert: Anything. Yeah, they [laugh]—Corey: I've heard it said about Dropbox, Facebook, the internet—Robert: Oh, Dropbox is such a good one.Corey: BGP. Yeah okay, great. Let me know how that works out for you.Robert: What was that Dropbox comment on Hacker News years ago? Like, “Just set up NFS and host it that way and it's easy.” Right?Corey: Or rsync. Yeah—Robert: Yeah, it was rsync.Corey: What are you going to make with that? Like, who's going to buy that? Like, basically everyone for at least a time.Robert: And whether or not the engineers are right, I think is a different point.Corey: It's the condescension dismissal of everything that isn't writing the code that really galls, on some level.Robert: But I think when engineers are thinking about, like, “I could build this on a weekend,” like, that's a moment that you have an opportunity to provide the value in an innovative, maybe consolidated way. We want to be a tool that's your incident management ring to retro, right? You get paged in the middle of the night, we're going to wake you up, and when you open up your laptop, groggy-eyed, and like, you're about to start fighting this fire, FireHydrant's already done a lot of work. That's what we think is, like, the right model do this. And candidly, I have no idea why the other alerting tools in this space haven't done this. I've said that and people tend to nod in agreement and say like, “Yeah, it's been—it's kind of crazy how they haven't approached this problem yet.” And… I don't know, I want to solve that problem for folks.Corey: So, one thing that I have to ask, you've been teasing on the internet for a little bit now is something called Signals where you are expanding your product into the component that wakes people up in the middle of the night, which in isolation, fine, great, awesome. But there was a company whose sole stated purpose was to wake people up in the middle of the night, and then once they started doing some business things such as, oh I don't know, going public, they needed to expand beyond that to do a whole bunch of other things. But as a customer, no, no, no, you are the thing that wakes me up in the middle of the night. I don't want you to sprawl and grow into everything else because if you're going to have to pick a vendor that claims to do everything, well, I'll just stay with AWS because they already do that and it's one less throat to choke. What is that pressure that is driving companies that are spectacular at the one thing to expand into things that frankly, they don't have the chops to pull off? And why is this not you doing the same thing?Robert: Oh, man. The end of that question is such a good one and I like that. I'm not an economist. I'm not—like, that's… I don't know if I have a great comment on, like, why are people expanding into things that they don't know how to do. It seems to be, like, a common thing across the industry at a certain point—Corey: Especially particularly generative AI. “Oh, we've been experts in this for a long time.” “Yeah, I'm not that great at dodgeball, but you also don't see me mouthing off about how I've been great at it and doing it for 30 years, either.”Robert: Yeah. I mean, there was a couple ads during football games I watched. I'm like, “What is this AI thing that you just, like, tacked on the letter X to the end of your product line and now all of a sudden, it's AI?” I have plenty of rants that are good for a cocktail at some point, but as for us, I mean, we knew that we wanted to do alerting a long time ago, but it does have complications. Like, the problem with alerting is that it does have to be able to take a brutal punch to the face the moment that AWS us-east-2 goes down.Because at that moment in time, a lot of webhooks are coming your way to wake somebody up, right, for thousands of different companies. So, you do have to be able to take a very, very sufficient amount of volume instantaneously. So, that was one thing that kind of stopped us. In 2019 even, we wrote a product document about building an alerting tool and we kind of paused. And then we got really deep into incident management, and the thing that makes us feel very qualified now is that people are actually already integrating their alerting tools into FireHydrant today. This is a very common thing.In fact, most people are paying for a FireHydrant and an alerting tool. So, you can imagine that gets a little expensive when you have both. So, we said, well, let's help folks consolidate, let's help folks have a modern version of alerting, and let's build on top of something we've been doing very well already, which is incident management. And we ended up calling it Signals because we think that we should be able to receive a lot of signals in, do something correct with them, and then put a signal out and then transfer you into incident management. And yeah, we're are excited for it actually. It's been really cool to see it come together.Corey: There's something to be said for keeping it in a certain area of expertise. And people find it very strange when they reach out to my business partner and me asking, okay, so are you going to expand into Google Cloud or Azure or—increasingly, lately—Datadog—which has become a Fortune 500 board-level expense concern, which is kind of wild to me, but here we are—and asking if we're going to focus on that, and our answer is no because it's very… well, not very, but it is relatively easy to be the subject matter expert in a very specific, expensive, painful problem, but as soon as you start expanding that your messaging loses focus and it doesn't take long—since we do you view this as an inherent architectural problem—where we're saying, “We're the best cloud engineers and cloud architects in the world,” and then we're competing against basically everyone out there. And it costs more money a year for Accenture or Deloitte's marketing budget than we'll ever earn as a company in our entire lifetime, just because we are not externally boosted, we're not putting hundreds of people into the field. It's a lifestyle business that solves an expensive, painful problem for our customers. And that focus lends clarity. I don't like the current market pressure toward expansion and consolidation at the cost of everything, including it seems, customer trust.Robert: Yeah. That's a good point. I mean, I agree. I mean, when you see a company—and it's almost getting hard to think about what a company does based on their name as well. Like, names don't even mean anything for companies anymore. Like Datadog has expanded into a whole lot of things beyond data and if you think about some of the alerting tools out there that have names of, like, old devices that used to attach to our hips, that's just a different company name than what represents what they do.And I think for us, like, incidents, that's what we care about. That's what I know. I know how to help people manage incidents. I built software that broke—sometimes I was an arsonist—sometimes I was a firefighter, it really depends, but that's the thing that we're going to be good at and we're just going to keep building in that sphere.Corey: I think that there's a tipping point that starts to become pretty clear when companies focus away from innovating and growing and serving customers into revenue protection mode. And I think this is a cyclical force that is very hard to resist. But I can tell even having conversations like this with folks, when the way that a company goes about setting up one of these conversations with me, you came by yourself, not with a squadron of PR people, not with a whole giant list of talking points you wanted to go to, just, “Let's talk about this stuff. I'm interested in it.”As a company grows, that becomes more and more uncommon. Often, I'll see it at companies a third the size of yours, just because there's so much fear around everything we say must be spoken in such a way that it could never be taken in a negative way against us. That's not the failure mode. The failure mode is that no one listens to you or cares what you have to say. At some point, yeah, I get the shift, but damned if it doesn't always feel like it's depressing.Robert: Yeah. This is such great questions because I think that the way I think about it is, I care about the problem and if we solve the problem and we solve it well and people agree with us on our solution being a good way to solve that problem, then the revenue, like, happens because of that. I've gotten asked from, like, from VCs and customers, like, “What's your end goal with FireHydrant as the CEO of the company?” And what they're really asking is, like, “Do you want to IPO or be acquired?” That's always a question every single time.And my answer is, maybe, I don't know, philosophical, but it's, I think if we solve the problem, like, one of those will happen, but that's not the end goal. Because if I aim at that, we're going to come up short. It's like how they tell you to throw a ball, right? Like they don't say, aim at the glove. They say, like, aim behind the person.And that's what we want to do. We just want to aim at solving a problem and then the revenue will come. You have to be smart about it, right? It's not a field of dreams, like, if you build it, like, revenue arrives, but—so you do have to be conscious of the business and the operations and the model that you work within, but it should all be in service of building something that's valuable.Corey: I really want to thank you for taking the time to speak with me. If people want to learn more, where should they go to find you, other than, you know, to their most recent incident page?Robert: [laugh]. No, thanks for having me. So, to learn more about me, I mean, you can find me on Twitter on—or X. What do we call it now?Corey: I call it Twitter because I don't believe in deadnaming except when it's companies.Robert: Yeah [laugh]. twitter.com/bobbytables if you want to find me there. If you want to learn more about FireHydrant and what we're doing to help folks with incidents and incident response and all the fun things in there, it's firehydrant.com or firehydrant.io, but we'll redirect you to dot com.Corey: And we will, of course, put a link to all of that in the [show notes 00:33:10]. Thank you so much for taking the time to speak with me. It's deeply appreciated.Robert: Thank you for having me.Corey: Robert Ross, CEO and co-founder of FireHydrant. This featured guest episode has been brought to us by our friends at FireHydrant, and I'm Corey Quinn. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an insulting comment that will never see the light of day because that crappy platform you're using is having an incident that they absolutely do not know how to manage effectively.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.
Nickolas Means, VP Engineering at Sym, joins Corey on Screaming in the Cloud to discuss how Sym is looking to solve the most common and most frustrating elements of compliance. Nick reveals why he finds it valuable to focus on making it easy for people to do the right thing over preventing them from doing the wrong thing, and why he feels the true spirit of compliance involves helping teams collaboratively come up with mutually beneficial solutions. Corey and Nick also dive into the common problems that engineers experience as a result of traditional compliance methods, and why historically the compliance industry has gotten a bad rap. About NickolasNickolas Means loves nothing more than a story of engineering triumph (except maybe a story of engineering disaster). When he's not stuck in a Wikipedia loop reading about plane crashes, he leads the engineering team at Sym, helping create the building blocks engineering teams need to build delightful developer access and approval workflows.Nick has been leading software engineering teams for more than a decade in the healthtech and devtools spaces. His focus is on building distributed organizations defined by their cultures of high trust and autonomy. He's also an international keynote speaker, having shared his unique brand of storytelling with audiences around the world. He works remotely from Austin, TX, and spends his spare time going on adventures with his wife and kids, running very slowly, and trying to brew the perfect cup of coffee.Links Referenced: symops.com: https://symops.com Twitter: https://twitter.com/nmeans TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Developers are responsible for more than ever these days. Not just the code they write, but also the containers and cloud infrastructure their apps run on. And probably the billing on top of that - which is neither here nor there. And a big part of that responsibility is app security — from code to cloud.That's where Snyk comes in. Snyk is a frictionless security platform that meets teams where they are, automating application security controls across their existing tools, workflows, and the AWS application stack — including seamless integrations with AWS CodePipeline, Amazon EKS, Amazon Inspector and several others.Deploy on AWS. Secure with Snyk. Learn more at snyk.co/scream. That's S-N-Y-K-dot-C-O/scream. And my thanks to them for sponsoring this ridiculous nonsense!Corey: LANs of the late 90's and early 2000's were a magical place to learn about computers, hang out with your friends, and do cool stuff like share files, run websites & game servers, and occasionally bring the whole thing down with some ill-conceived software or network configuration. That's not how things are done anymore, but what if we could have a 90's style LAN experience along with the best parts of the 21st century internet? (Most of which are very hard to find these days.) Tailscale thinks we can, and I'm inclined to agree. With Tailscale I can use trusted identity providers like Google, or Okta, or GitHub to authenticate users, and automatically generate & rotate keys to authenticate devices I've added to my network. I can also share access to those devices with friends and teammates, or tag devices to give my team broader access. And that's the magic of it, your data is protected by the simple yet powerful social dynamics of small groups that you trust.Try now - it's free forever for personal use. I've been using it for almost two years personally, and am moderately annoyed that they haven't attempted to charge me for what's become an essential-to-my-workflow service.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. This promoted guest episode is brought to us by our friends over at Sym, and into my verbal grist mill, they have thrown their VP of Engineering, Nickolas Means. Nickolas, thank you for joining me.Nickolas: Thank you so much for having me, Corey. And feel free to call me Nick.Corey: I certainly shall. So, let's begin at a high level. When you're starting a company and trying to, sort of, bootstrap and raise initial rounds of funding and the rest, you're trying to save money in a bunch of places. And one of the most expensive things you can buy when starting a company is, of course, a vowel. You wound up not naming the company—or the vowel, really—the y is sometimes a vowel, sometimes not. It's S-Y-M. What is it you folks do exactly? What do you folks start? Where do you stop?Nickolas: So, the name of the company comes from the idea of helping humans and machines work together more effectively. And that's really nice and high level; it doesn't tell you any information about what we do.Corey: It feels like we're—we'd assume that most startups pivot at some point; we're just going to set—Nickolas: [laugh].Corey: —[crosstalk 00:01:33] seeds for that nice and early on, and dive on in.Nickolas: So, what we actually do, the two co-founders and myself all have a background in highly compliant industries. I've done VPN stints at a couple of health tech startups; they've done similarly. And all three of us ended up building sort of a certain set of things every time we were at one of these companies. Because you have to be compliant with things, and in order to be compliant with things, you have to have a set of controls, you have to restrict certain things: how people get to production, how people access customer data. And those controls, by and large, all suck. They're all painful and every company ends up building something from scratch at some point to make them not suck quite so bad. And it seemed like there was a product opportunity there.Corey: I would argue there absolutely is. One of the big problems that I've found throughout the time that I've been fixing AWS bills on a consultancy basis has been, we're really talking about cloud governance. But even now, by using the phrase cloud governance, three-quarters of the audience immediately wound up skipping to the next podcast over on their playlist because it sounds like it is one of those incredibly boring things. And to be fair, usually, when it comes to compliance, you want some of the most boring, least creative people in the world overseeing that. Like, when you wind up talking to someone at a company and they have a great sense of humor and they are constantly cracking jokes constantly, it's like, “What do you do?” Like, “Oh, I'm the CFO.” All you hear from that is, “Oh, I'm about to go to prison. Awesome.”Like, you want the wild, cutting-loose CEO to have three drinks and then confide, “I really like typing the number six.” You want them [laugh] to be predictable in a whole bunch of ways. And it always feels like compliance takes that entire mindset of, it's always about risk management, it's about wanting to make sure that people don't go off script in a bunch of weird ways, but as an engineer, what I always heard from that is slow down, don't be creative, go ahead and do things in very predictable ways. Only release things once a quarter, et cetera, et cetera. And yes, that's one way to meet compliance goals, but it's a crappy way, in my experience. I'm going to guess, though, that you have a lot more experience with the compliance world than I do because having worked a few times now, for big regulated finance companies, I wanted to get the hell out of the compliance universe.Nickolas: Yeah, I mean, you used an interesting turn of phrase there. You used the phrase, “Avoid going off script,” and I think there's a subtle turn there that actually makes all of this work a lot better. Instead of focusing on keeping people from going off script, you focus on keeping them on script. You focus on making it easier to do the right thing than to do the wrong thing. And that takes away a significant amount of the pain involved in compliance stuff.You look at implementing controls—and everybody has the exact same reaction you just brought up about governance—because there's so much FUD around this stuff. Everybody has been slowed down by one of these silly rules that makes no sense, that's checking a box and not actually meeting the spirit of any kind of meaningful improvement.Corey: Oh, cloud has absolutely doubled our speed of iteration because it used to take six weeks to get a server racked in the data center and we moved our processes to cloud and now to spin up an EC2 instance, it only takes three weeks of approvals. And at that point, it's what are you really doing? You wind up with people building on shadow IT. It's part of what contributed to the rise of cloud in the first place. Well, I can go through the annoying thing that this company wants me to do, or I have a corporate credit card and by the time it raises the level of spend to a point where it gets scrutiny, it's in production serving customers and what are they going to do?Some of the very early AWS sales conversations with customers started off as, “Well, why should we build on top of your cloud?” asks the exec, and they say, “Oh, sorry, you have 87 different accounts throughout your organization currently with us. We're just trying to give you some unified view into it and possibly some discounting if you want.” Yeah, these days, that's a fast track to getting yourself fired in some companies, if you wind up deviating from that story. But also, people are not doing this out of malfeasance; they're trying to get their job done.And as soon as guardrails start increasing friction, making it harder to do things the right way than to go around it, people will not comply. I strongly believe that, whether it's cost—which is my universe, and frankly, only a business hours problem—or actual governance issues with some compliance regimes, which get those wrong and hope you enjoy some time in prison.Nickolas: Yeah, exactly. I mean, you know, if you look at SOC 2, for example, there's a lot of companies out there that are willing to sell you a program that will help you become SOC 2 compliant. They show you all the steps you need to take, all the programs you need to put in place. The thing they don't do is help you establish the controls that are required. They'll tell you that you have to have somebody formally approving before software goes out to production. They won't give you any guidance whatsoever on how to put that control in place. And so, it's really easy for a compliance person that's not looking to collaborate with engineering just to go, “Okay, I need you to put a button in the deploy process and I need the CTO to click that button.”Corey: Yes. We've always seen that as reactions to different things. I was at a company once where there were some outages caused by bad deploys, so they decided that a VP had to sign off on every deploy. Now, I come from the sysadmin ops world, which explains so much about my cynical perspective on life, so the way we got that overturned within two days is we did the malicious compliance thing, where oh, we need to deploy this. Great, we are walking into the middle of a senior leadership team meeting to get them to—with a tablet or comput—laptop—“I need you to click the button right now.”And doing that out of hours and all kinds of other things, it's oh. Yeah. How about we wind up only doing that for significant large changes? How about that? Maybe you don't need to wake someone up at home in the middle of the night when there's a deploy going out that fixes a typo on the marketing page; little things like that.And at some point, you're always felt like the goal of governance was either ossified scar tissue around all the ways that things have failed before, or through a, frankly, misguided belief that if we wind up distilling everything down to processes and procedures, eventually, someday, we can have a bunch of trained monkeys doing this job instead of people who are expensive and, you know, cynical, and difficult to please. I feel like that is not the right way to think about these things.Nickolas: Well, I mean, the thing about those controls, you know, it's exactly what you just said. Nowhere in SOC 2 does it say that your VPN [unintelligible 00:07:56] or CTO has to approve all code deploys, that's not in there. But that's the reality of life at a bunch of companies. In reality, if you just follow a software development life cycle that has multiple people looking at code before it gets deployed, multiple people signing off on that code being okay to deploy and you have a staging environment before you hit production, you've met the control. And SOC 2 gives you so much flexibility in how you write the control.So, I think the thing that I've seen that makes compliance so much less painful, is when you have somebody that is 95% the boring persona like you're talking about, but 5% creative. 5% willing to kind of get their hands dirty, empathize with the engineering team, collaborate with the engineering team, and find a way to put some of these controls in place that doesn't just bring things to a grinding halt.Corey: I have to assume that, given that you've built an entire product slash company around this idea, that you have some opinions other than doing what I do, which is sitting in my lofty ivory tower and oh, you should, in this idealized case, do things a little bit differently. But it's going to be bespoke and the answer to any complex question, the more senior you get is, “It depends.” You, of course, have built something that scales out in a bunch of different ways. How do you view that in a way that makes it not either completely useless or overly prescriptive?Nickolas: We focus on giving the power to engineering teams and giving the security complexity [unintelligible 00:09:23] the power to oversee those things. You know, it would be easy to give somebody, like, a clickbox UI, let them design controls for SOC 2 or whatever, end-user interface, but that's not how engineers think; engineers think and express ideas and code. So, we've made the rather controversial decision in the face of a bunch of no-code tools to go low-code instead. So, to build a compliance workflow in Sym, you're going to write some Terraform, you're going to write a little bit of Python—a lot less than if you were building it from scratch—but you're going to end up with something that perfectly fits the way that you already work versus having to shift your work practices around to fit the tool.Corey: If you have inadvertently stumbled upon one of my hot buttons. There's a lot of people that take a perspective around low code. And I just want to say that that perspective is often garbage. Like, oh, that's not a real program—great. Hypothetically, if you have an idea for a business or a product or something, and involve software as most things seem to these days, maybe having to go to a boot camp for six months first as a prerequisite is not the best path forward.“Well, you're never going to build something hyperscale in a low-code environment.” Great, how many things that we built that actually need to be hyperscale that don't go through 16 different architectural iterations between ridiculous idea one day and thing that is actually hyperscale? It's an early optimization. I have an entire production pipeline in Retool that I built using low code. I think that that is a very powerful thing. And this idea that, “Oh, that's not real code.” Cool. What's your point?Nickolas: Well, and for us, one of the things that we're trying to enable is for software engineering teams, ops teams, whoever is building these controls, to interact with a security person or a compliance person, for them to be able to read the code, understand what it does, understand the way that the control has been implemented. And so, we provide a bunch of frameworks around that and a bunch of things. Like, you don't have to go and build a Slack workflow from scratch and nobody has to understand that code because it's buried in the platform. The only thing that the security or compliance person has to understand is the business logic that's been put into place. Who can approve it? Who can't approve it? How does that change after hours? How does that change if there's an incident? All of that is in very simple Python that you don't have to be an experienced programmer to be able to read.Corey: One of the big powerful things behind that is it really reduces the interrupt volume of someone coming by to an engineer who is deep in the middle of something else, and, “Hey, guess what I have? A surprise context switch for something that's going to take you probably 30 seconds, but then you're going to be distracted by all of this.” If you give people the ability to self-serve, everything tends to work a lot more smoothly.Nickolas: Yeah, absolutely. And, you know, that's one of the ways we use Sym at Sym: we've got it in front of our AWS production environment, so if you need to go and do anything in production, you just have to get approval from any other engineer that happens to be in the approval channel, sort of a two-keys-to-launch-a-missile model. And that works fine for our compliance needs and it avoids there being a single point of failure that every time you need to go and get into production, you have to go and say, “Mother, may I?”Corey: Exactly. It's one of those things where every time you wind up with something that injects friction, people are going to find ways around it. And in some cases, this leads to positive outcomes where, when you're subject to PCI, which is a lot more prescriptive than a number of other compliance regimes, it's, great; this is a lot of things that don't necessarily reflect how we work, how we want to work, et cetera. We can ignore it, which is not a great plan, we can wind up having to slow everything down, which is the common case, or the right answer is, we're going to build the PCI environment that is very self-contained, just the critical stuff that needs to be in there is going to be in there, and then we can build everything that touches it around it in ways that are a lot more aligned with how we believe software should be built.Nickolas: Yeah, absolutely. I mean, you silo off those high-control places, but there are controls that have to extend into the rest of the business. And one of the things that I'm a very firm believer in is, if you're going to impose a control upon somebody, they need to have the agency to shape and to change that control so that it lets them work the way that they want to work.Corey: I just want to call out how wonderful that is because I had a belief that looked borderline heretical, 12 years ago, when I said that, “Okay, simple rule. If you want me on call, I am empowered to change the thing that wakes me up.” Whether that is the code itself, the system itself, the paging threshold and frequency, or ultimately, I'm turning the physical pager off. It's one of those things where I decide what's an emergency outside of hours on that point. If it's going to wake me up, I need the power to make sure it never does. Otherwise, you have no agency. It just feels like you're being victimized by the stuff.Nickolas: Yeah, absolutely. I mean, there was a wave of on-call regimes that ran through large companies for a while where there would be a centralized on-call team that would be responsible for responding to hundreds of services. And thankfully, we are maturing past that; we're distributing on-call rotations so that teams that actually build services are responsible for them. And it's the same mindset, right? If you're going to be participating, if you're going to be working with a system or working with a control, then you need to be able to change it, you need to be able to make it work the way that you think that it ought to work.And in the context of compliance, you need to bring somebody along with you. You need to bring the person that's responsible for the controls that actually has to sign on the dotted line at the end of the audit period, saying that we do all of these things. So, you have to be able to explain what you're doing to them. But you have to be able to iterate.Corey: I have to ask, given that what you are building is going to have heavy involvement from engineering, how do you respond to the probably most common engineering objection I imagine you get, which is, “Well, this doesn't look hard. I could build this in a weekend.”Nickolas: You know, it's funny. We joke that our biggest competitor is build in-house, right? It's pretty easy to start looking at what it takes to build a from scratch workflow in Slack to build a Slack app, to understand the cost of building it in-house. Because nothing about building an elegant user interface in Slack is easy or cheap. That API is difficult to work with and hard to get good user experience out of.And we've spent a lot of time polishing a lot of places in the platform: we've got good documentation, we've got a good SDK, we've got good integration with third-party services that make all of this stuff easy to do. And it does look easy on the surface, it does look like ‘I can build it,' but we've had customers that have had that objection gone and tried to build it and come back. Because it's not as easy as anybody thinks.Corey: My biggest competitor for fixing AWS bills has always been Microsoft Excel. It's the, we're going to do it ourselves—badly—internally. Okay, great. If that works for you, terrific—Nickolas: Yeah.Corey: —but very often it doesn't. I mean, I think a classic case study of this is, in the terms of something that is well designed but is almost mind-bogglingly complex—and we're getting a case study in it this year—is Twitter because it looks from the outside, very simple. I wind up writing a thing and I hit the post button and it shows up in a timeline. And then other people can subscribe to it or not, and they see it themselves. That sounds like something you can build on a weekend. And we look at all the ways it's now exploding and collapsing and having weird bugs that no one anticipated, to realize, oh, this is a very challenging, very sophisticated application. But because it was well designed at one point, it looks easy.Nickolas: Yeah. Yeah, it continues to run despite the fact that it's having less than a quarter of the staff that originally maintained it, maintaining it because the services were well designed in the first place. They're resilient on their own and they're self-healing in a lot of cases. It's the same thing with Sym. You can build these tools in-house, you can build them yourself, but then you've got more software to maintain. Because once you build something, you own it, forever. And the cheapest code is no code; the cheapest code is code that you don't have to write.It's easy to look at a simple use case and understand a little bit of the cost of this. If you want a Slack workflow that gives you access to production in AWS, you can wire that up fairly quickly. Those APIs are not all that difficult. Now, let's say you want to add an integration where if you're on-call in PagerDuty, you can get to production without having to get an approval. Okay, well, now you've got a new API that you need to wire in.And let's say that every time that happens, you want to open a Jira ticket so that you can record that that's happened. Well, there's another API that you've got to wire in.j, whereas with Sym, it's just, it's right there. It's a few lines of code to wire it all together. And it deploys in Terraform alongside the rest of your infrastructure, so you manage it the same way you're used to managing things.Corey: It reminds me of my earlier career when I was deep in the configuration management weeds with Puppet and SaltStack, where the biggest competitor we had any of those projects was always someone writing a bash script to do it themselves. And yes, you can do that, but then the requirements change, or you're going to hit a point of scale that was surprising. And one of the valuable parts of it is that when the future is uncertain, as it always is—Nickolas: Always.Corey: Having folks who work in environments that aren't just yours who encounter a lot of those edge cases you're going to stumble into and can build things in is incredibly valuable. I don't think I've ever met anyone who ran an infrastructure that said, “I would build it the same way if I had to start over again.” They always want to, “I would fix these annoying things.” Well, by having a product focused on a space like this, it's yeah, today, you can have that VP click the approve button inside the GitHub Actions workflow. Good for you.But when you get just a little bit further down the path, you aren't going to want to do that anymore. There needs to be some decision-making it builds into it, and for certain high-risk changes, maybe a second person and so on. How do you build that logic engine? How do you build that workflow approach? How do you have a break glass thing for middle of the night when the site is down? Et cetera, et cetera, et cetera.And that's exactly the sort of thing that I would expect something like Sym to get very right, just because there's always a bigger fish. You've seen this [unintelligible 00:19:17] before in other shops. And more to the point, if there's something I want to do as a part of this that Sym doesn't support and you are looking at me strangely if I asked how to do it, that's usually a good early warning sign that maybe there's something I'm not thinking about here. Because whatever the problem space is, I'm probably not the only person that has to do this. How are other companies solving for this? And it turns out that all my copy of our SOC 2 report has a typo on it. That would explain a lot. That's a ‘can' instead of ‘can't.' Nevermind. Or something like that.Nickolas: Well, and the flip side of that is also true. I mean, the interesting thing about working on something that is sort of wide open with what you can wire up and build with it is we're always learning from our customers. We're always learning from the things that they're doing. And so, you know, when somebody approaches us of, “Hey, we need to solve this particular problem,” if we don't have a ready answer, we brainstorm and help figure that out. And to your point, that always extrapolates to other customers finding the same sort of thing useful.The other bit of this that's really interesting beyond the durability and the ability to kind of rapidly evolve these workflows is the audibility. It's helpful in a lot of these compliance regimes to have a third-party tracking this data for you. So, when somebody accesses AWS production, who approved that access? When somebody deploys code, who approved those deploys? Well, we sit there as kind of a third party on the side, observing all of this, taking all these notes for you, and piping them into whatever audit tool that you want.So, you've got that data long-term and when it comes time to audit, you've got all the evidence you need; it's already there, already collected. You don't have to go through and write a regex to parse a bunch of logs to get the information you need.Corey: And invariably, that regex is always going to be different, depending upon the log stuff. It's great having a unified central approach that is the trusted repository for this stuff. As you've been going to market and talking to your earlier customers and seeing the problems that you folks solve, what have you learned about the market space since you've gone into this direction? Because I feel like this is one of those products where you start designing and thinking you know a lot about the space, and you learn so much more just from the customer conversations and seeing that you can build the most finely crafted torque wrench in the world and the customer complains because it turns out, you built a crappy hammer.Nickolas: So, I think what's been really interesting to me is how much use our Lambda integration gets. We have a lot of first-party integrations with things like IAM and IAM identity center and Aptible and a bunch of tools that you can interact with, but a lot of our customers have wanted to do very specific things inside their infrastructure and put those things behind an approval. And the Lambda integration turns out to be a great Swiss army knife to do that because you can wire it up—it runs inside your firewall—to take essentially whatever action that you need it to. And that gets a ton of use. Probably more than half of our customers have at least one Lambda workflow in production, and I would not have expected that going in.Corey: It's wild to me just how pervasive Lambda has become. And even from a compliance perspective, it's great because unlike, “Well, it's a script that runs on a server somewhere,” yeah, it's immutable. It's versioned. There's a way to conclusively prove that at invocation, this is the code that ran, the end, with the following parameters. Done.There's no, “Well, looking at the timestamp on the file”—like, no. None of that nonsense. It's arguable that something that I have seen has been that Lambda is one of those rare technologies where you're seeing faster adoption in the enterprise and you are in startup land.Nickolas: Yeah, I would say that's true. I mean, it's so great for running undifferentiated workloads. I just need this one thing to happen really quickly and I don't want to mess with standing up a server to run this thing that runs once a week. Okay, well, here's a computer that will run just long enough for you to run this thing and then go away. It tracks exactly what ran, exactly when it ran, exactly how it got kicked off.And in our case, it has access to all of the internal AWS APIs that we wall off in our platform because we obviously don't want you using those things in the Sym runtime. But you can do anything that you want to your AWS environment from your own Lambda and we will gladly provide the approval step ahead of kicking that job off.Corey: Are you seeing people use Lambda-based workflows to manage on-premises things or is it more heavily in environments that are already within the AWS boundary?Nickolas: The Lambda stuff that we see is almost entirely—I think it is entirely for things that are within the AWS boundary. I can't think of an instance when somebody is managing something on-prem with it.Corey: I am increasingly discovering, through the magic of Tailscale—among a few other things—that I can use that for things on-premises that talk directly and interface with my Raspberry Pi in the spare room, et cetera. Which is—I think some people call it hybrid, which is the business enterprise term for ‘horrifying—Nickolas: Yep.Corey: —because it's a terrible pattern in some ways. But it's so convenient and it's so nice not to have to worry about some of these things, just an infrastructure point of view. One thing that I think that AWS has done very well at, as they've evolved, has been with AWS Artifact, which ties directly to their own compliance reports, where in the early days when I was responsible for SOC 2 controls at a company, I found myself answering security questionnaires from vendors as if I was running in a data center. And sure enough, they wanted to tour us-east-1. And it turns out, you can't really do that.So now, just pointing them to the stuff that comes out of Artifact, it's written by auditors for auditors and they go away and leave you alone without having to explain your bespoke artisanal nonsense to them. There's something very pleasant about being able to throw the lion's share of the work over to someone who already knows how to do it.Nickolas: Our audit period is ending here shortly and I have recently been and spending time in Artifact. So yes, a hundred percent.Corey: It used to be that you would only be able to get those things under explicit NDAs, you'd have to talk to your account manager for every one, it was a back-and-forth process, and you didn't really know if what you were going to get was going to answer the questions that they had. Now it's, you show up, you click things three times, and you're done. The hardest part is sorting out which ones you need from the hundreds of things available within Artifact.It's like, okay, that's great, but this one is in Spanish for some reason. And that's awesome, but on some level, it feels like that should be an easy filter option. But yeah, no one ever accused AWS of building a good user interface. But once you get the thing you need and can pass it off, great. Job over. It's one of my favorite services that most people who are what we know as ‘happy' don't know exist.Nickolas: Yeah well, and that, it points to a larger industry trend, right, that companies are getting SOC 2 specifically earlier and earlier because it is becoming table stakes to be able to sell into other companies. They want to see your SOC 2 report before they're willing to work with you before they're willing to let your software touch their infrastructure. And there is a lot of value in these compliance programs as essentially a stamp of approval that you're taking these things seriously, even with as much flexibility as SOC 2 has, just the stamp that we've thought about these things and we have serious answers to them is a pretty important signal to be able to send to somebody that's wanting to buy your software.Corey: We've toyed with the idea of going through the process ourselves because we get asked about it all the time, but it feels like the procurement processes that ask us for it expect us to come in with a whole software suite and the rest. And yeah, if that's the world we're operating in, it makes a lot of sense. We're a services-based consultancy; we come in as individuals, we have conversations with people, and we talk about this and we have no write access to anything in your environment and give you scoped-down permissions for what we talk to because we don't want the responsibility of that stuff.And a lot of companies get that intrinsically, but there's occasionally a few you have to go round and round and round with. It just it feels like it's one of those, okay, you're not quite there yet. You're trying to view everything through this very specific worldview. Maybe it works for your constraints and requirements, but I've never understood it. And I've learned the older I get, the more time I spend around this, I used to have such a negative perspective on compliance.And now it's, you know, everything's nuanced. There's a reason that these things are there. It's not just a make-work project for an industry that wants to slow everyone else down. It's, there are risks here; these things exist for a reason. There's a reason that you can go start Twitter for Pets tonight and not be regulated, but the same is not true of First Bank of Twitter Pets.It's okay, yeah, one of those things is going to require a fair bit of regulatory scrutiny, and as a society, we want that. Now, the counterargument that I don't necessarily want to get too far into is, should Twitter for Pets be regulated?Nickolas: [laugh].Corey: And that's a can of worms that I think we'll leave for another episode.Nickolas: Yeah, I mean, that's—you know, the people that hate compliance the most are the people that are on the sharp end of compliance, people that are having to actually deal with the controls that are imposed upon them by these compliance regimes and by somebody who's taking a very literal view in interpreting the things that some of these compliance programs say that you'd have to put in place. And I think, you know, that's—kind of bring the conversation full circle—that's the thing that we want to change more than anything. If we can wave a magic wand and change the compliance universe, the thing that I most want is to help compliance and security people collaborate with their engineering teams and come up with mutually beneficial solutions. Things that actually—the spirit of compliance.Corey: Oh, yeah. My first PCI audit was a little bit of a challenge, just because the auditor wasn't really conversant with anything that wasn't a large company. So, they show up at our twelve-person start off, and, “Okay, where's the Active Directory?” It's like, “We don't have one of those.” “Okay, well how do you authenticate to the WiFi?” It's like, “Oh, the password's on the wall.”It's, “Well, what happens if I get on that WiFi?” It's, “What can I do that I couldn't do from anywhere else?” Like, “Use that printer over there. That's it.” Because everything else was the idea of the security boundary was built on identity, not on what blessed network you happened to be on; there was no special permissioning that didn't apply to the Starbucks WiFi next town over.But that was one of those things where at first they thought this was a horrifying problem and they were not going to be able to certify us, and it turned into no, we had significantly advanced culture of security compliance, oversight, separation of duties, all the things you really care about. We just didn't have the trappings that usually came across with when you're thinking about this or starting—or having the temerity to start a company, you know, longer than 18 months ago at a place that wasn't San Francisco on the latest version of a MacBook Pro running the bleeding edge version of Chrome. It turns out that there's a big universe out there. And not that there's anything wrong with either side of it, until they start forgetting that not everyone operates the way that they do.Nickolas: Yeah. I mean, you know, we talked about checkbox compliance a lot and I think that's probably the biggest problem is there is a lot of checkbox compliance out there. And people have seen it not actually solve anything and just make everything harder. And so, compliance gets a bad rap.Corey: Oh, for me, the one that I've been picking fights on social media about for a few years now is encryption-at-rest in the cloud. Like, yes, you want full-disk encryption turned on your laptops, your phones, your tablets, et cetera. Someone steals it from the coffee shop, you want to be out the cost the hardware. The end. But if you can get a hard drive intact out of an AWS facility and then reassemble it with the right number of drives in the right places, without… and hasn't been encrypted. Congratulations, you earned it. As far as I'm concerned, that's yours. You can keep it.Because AWS employees aren't able to do that, let alone third parties. But it is easier by far to click the box to enable encryption-at-rest and not spend half an hour arguing with the auditor… and just get on with your day. And recently in S3, for example, they wound up making that a default. Good for them. It's just, can we please focus on the part of the story that's relevant and germane to our business? Because that is not the threat model of modern attacks.Nickolas: Yeah, I mean, for a long time, how much of the internet ran on unencrypted HTTP, but it was being served off of an encrypted disk? Great. What have we solved?Corey: Oh, absolutely. It's wild to me. Even now, I still we feel like there should be a reasonable way to handle—to [unintelligible 00:31:17] basically encryption between two points that doesn't depend on the third-party CA's with expiring certs and the rest. Drives me up a wall every time because it's always the worst possible time. It causes the strangest issues and there is something deeply and profoundly wrong with the fact that the failure mode from the user perspective between, “Your connection is being intercepted by a third party,” and, “Holy shit. This certificate expired two hours ago.” Like, those are very different use cases, but the scary warnings have trained people to treat them the same way.Nickolas: Yep. Yep, exactly the same. Ugh.Corey: I really want to thank you for being so generous with your time. If people want to learn more, where's the best place for them to find you?Nickolas: Yeah, so the best place to find out more about Sym is our website, symops.com, SYMOPS dot com. And I should mention that Sym is completely free for teams of up to ten people. If any of you out there listening check it out, please reach out. We'd love to hear about your experiences, help any way we can. And if you want to get in touch with me directly, the best place to do that for now, while it lasts is still Twitter. I'm on there as @nmeans.Corey: And we will, of course, include a link to that in the [show notes 00:32:27]. Thank you so much for agreeing to talk to me about all this stuff. I really appreciate it.Nickolas: Yeah. Thanks so much for having me on, Corey. It's been a lot of fun.Corey: Nick Means, VP of Engineering at Sym. I'm Cloud Economist Corey Quinn, and this has been a promoted guest episode, brought to us by our friends at Sym. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry bitter comment that will get posted in six weeks, after you track down your elusive VP to click the approve button.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.
Josh Doody, Owner of Fearless Salary Negotiation, joins Corey on Screaming in the Cloud to discuss how to successfully negotiate your salary, and why it's important to do so even in times of economic uncertainty. Corey and Josh chat about some of the hidden reasons why salary negotiation is critical to job seekers, and what goes into determining salary bands behind the scenes. Josh also reveals why he feels there's some stagnancy in the big tech job market, and why it's critical for job seekers to have a balanced view of the value that they provide to employers when negotiating salary. Josh also describes some of the unexpected ways salary negotiations can come up throughout the interview process, and how to best handle the discomfort of negotiation. About JoshJosh is a salary negotiation coach who works with senior software engineers and engineering managers to negotiate job offers with big tech companies. He also wrote Fearless Salary Negotiation: A Step-by-Step Guide to Getting Paid What You're Worth, and recently launched Salary Negotiation Mastery to help folks who aren't able to work with him 1-on-1.Links Referenced: Company website: https://fearlesssalarynegotiation.com Twitter: https://twitter.com/joshdoody LinkedIn: https://www.linkedin.com/in/joshdoody/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Developers are responsible for more than ever these days. Not just the code they write, but also the containers and cloud infrastructure their apps run on. And probably the billing on top of that - which is neither here nor there. And a big part of that responsibility is app security — from code to cloud.That's where Snyk comes in. Snyk is a frictionless security platform that meets teams where they are, automating application security controls across their existing tools, workflows, and the AWS application stack — including seamless integrations with AWS CodePipeline, Amazon EKS, Amazon Inspector and several others.Deploy on AWS. Secure with Snyk. Learn more at snyk.co/scream. That's S-N-Y-K-dot-C-O/scream. And my thanks to them for sponsoring this ridiculous nonsense!Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. I have a returning guest today who hasn't been on for a couple of years, at least. Josh Doody is the owner of fearlesssalarynegotiation.com and focuses on a problem that's near and dear to my heart from my previous life as an employee, salary negotiation, specifically emphasizing software engineers, if I have that right. Josh, thanks for joining me.Josh: Yeah. You have it exactly right. It's great to be here, and good to talk to you again, Corey.Corey: I used to be practiced at doing salary negotiations, which is a very roundabout way of saying I got fired a lot, so I got lots of practice at doing it. And I found that it was a very strange experience that was completely orthogonal to anything else that I did in the course of my day-to-day. Now, of course, I you know, negotiate AWS bills for a living among many other things, and do a lot of sales work, and yeah okay, now it's a lot more germane. But back in my engineering life, it was the one time I got to really negotiate that wasn't, you know, haggling with some vendor somewhere when I'm trying to buy a burrito, was salary negotiation, and I felt utterly unprepared for it.Josh: Yeah, I think most people feel that way and you summarized pretty well why that is. And that's, you know, let's say you have a really robust career, you're around for decades, you know, you're working in lots of companies, you might have, I don't know, let's say ten, a dozen job offers that you negotiate, you know, give or take. And that's not very many reps for doing something that's as consequential as, you know, negotiating your actual pay. Which, depending on how senior you are, could be literally negotiating, like, you know, multiple cars' worth of value per year that you're going to [laugh] that you're going to earn. But you don't get the reps, so most people just kind of—I think they kind of don't even think about it until they have to think about it until it's directly in front of them. And then they just kind of power through it, get it over with or even totally ignore it and just get back to the thing that they're doing in their career, which is why you show up to work.Corey: It feels, on some level, like it's one of those areas where people wind up thinking about it long after they really should have. These days, it feels like salary negotiation process, more or less should start when you start debating, huh, maybe I'll change jobs. Like, it feels like it's really that early, not when you have an offer sitting in your inbox that needs a response by the end of the week. Right, wrong, or am I just thinking about this in ridiculous ways?Josh: No, I think you're right. So, you can start thinking about it, you know, you get a job offer in your inbox and you can start thinking how do I negotiate this now, but you know, you're going to be in a less secure position to do a strong negotiation at that moment than you would have had you begun thinking about it when you mentioned, which is, like, you're actually thinking about changing jobs, or, you know, maybe you just got a cold call from a recruiter and they're at a company that you're kind of interested in working with. So, maybe I will talk to this recruiter instead of just blocking them or whatever. And so, the whole process can begin at that moment when they say, “Hey, you know, we have this opportunity that we think you might be interested in. What do you think?”And then, you know, early on, they'll even kind of officially start the negotiation, at least in my mind, where they'll say, “But before we really go too far on this, like, what are you hoping to make here? You know, what are your salary expectations if you come work here?” And you're kind of off to the races at that moment. But even if they don't say that out loud, that's something that you should be thinking about from the beginning, which is, you know, maybe most broadly, how do I position myself to get the best possible version of the job offer that they're willing to give and to leave myself the most latitude to improve that job offer to be the maximum that they can afford to pay me or the maximum that their budget allows or however you want to frame that. So, short answer, yeah, I think you're right that most people think about it as sort of an afterthought, either after they've already started a job and they go, “Huh, I wonder if that guy over there is making more than I am?” Or, you know, “Shoot. I think I moved too fast there. Maybe I should have done something a little bit better.” When they could think about it way earlier in the process than that.Corey: Since I was last on the job market, there have been some changes, at least here in California, that have had a somewhat significant impact, to my understanding. First, job salary changes need to be posted in job ads, which I think is great—and that's occurring in a number of different states—and also it is now against the law, in California—or at least against public policy—to ask what someone's current compensation is and your salary history and dive into that. Now, that's all well and good, but I also have been asked a number of questions that are not exactly… green, when it comes to being in the middle of an interview. And, “You're not legally allowed to ask me that question,” is that a heck of a pushback.Josh: Yeah, I think that I've had a couple conversations about this recently, but also over the past few years, especially on the—you know, you mentioned the two prongs of that idea: what's your current salary, what are you making now? And, you know, what is the salary that you expect to make? And so, kind of one by one, states are outlawing potential employers' ability to ask about what you're currently making. And then I've also heard some agitation lately that there might be some federal legislation that's coming down that might just kind of take that off the table. As you mentioned, recruiters, companies, organizations, however you want to model them are very clever, and so there are always ways, you know, even if they're indirect questions, you know, you don't ask them what they're currently making, but you ask them something that gives you some insight into what they currently might be thinking.Also, if you're in the big tech world—which you mentioned you negotiate AWS contracts—in the big tech world, they don't necessarily have to ask you what you're currently making if they know that you're an L4 software engineer at Google. They can probably approximate it pretty well. And of course, they know that because you're going to have to tell them, you know, with a resume or when you're interviewing, that's kind of how you get in the door. So, that's an interesting thing. But I still say, avoid that. Try to avoid giving him as much information as possible.And I think the most important thing with the current salary idea is, you just don't want to say it out loud. You want to make sure that they can't quite grab onto that because you make it too easy for them when they know what your current salary is to just do sort of a cost-plus version of offering you a job, which is, “Well, you're making this much now. We'll just add 10% to that,” and that's your new job offer, when you know, that's not how you level up quickly and in big ways.And then you mentioned the salary expectations. I do think it's great that a lot of job offers now will have a salary range in them. That's a question that I see a lot is, like, how do I know that I'm not going to waste 25 hours of my personal time and maybe a trip across the country for a job, where when they finally make the offer, it's just laughably low? And the answer is, you know, hopefully, they have something that you can grab onto in the actual job description that says, here's what the range looks like. But even then, you'll notice if you look carefully—I saw one yesterday, and I don't remember where I saw it, but it was like, “Yeah, our range of salary is, you know, 120k up to 290k, depending on geographic region.”And it's like… I mean, technically, that's a salary range, but they don't tell you what the regions are, how they map, and all that stuff. So, you're not getting a lot of information there; you're just getting sort of an approximate number. But it's still helpful to know that information. And it's also helpful to not disclose that information. If you have a number in mind that you're hoping for, it's not in your best interest to share that with the company.So, I think at least what you can do is look at the job description. If they have some kind of a range, take a look at it, see if it feels like, okay, this is something I can work with or if it's just, you know, there's no way that that would ever work for me and you can just pass on it and save yourself some time.Corey: For me, one of the things that always frustrated me was that at the start of looking into a job, there's always the big question that they ask that has been the socially acceptable paths at screwing you over, and the knowing how to answer that is important. But I still bungled it a number of times whenever I was out of practice, which is quite simply, “Okay, what is it that you expect to make in your next role? What are your compensation requirements?” And it feels like answering that at the beginning of the process just completely sets your course for how the rest of that process is going to go.Josh: It does and it's something that's very subtle and clever because most people will not perceive that to be a negotiation tactic when it is. And also you mentioned earlier in the context of, like, asking you what your current salary is that it can be perceived as sort of a gatekeeping question. Like you mentioned, you know, you're in the middle of an interview and somebody pops a question at you like, “What is your [laugh] what's your current salary?” And you're looking at an interviewer and you're thinking, “If I don't give him this information, then I'm saying no to an interviewer, and how's that going to go over?”This is the same kind of thing. When, you know, at any point in the process, they might ask you what your salary expectations are, it could be on the first screening call, it could be right before, they like to hold this till right before they make an offer where you go through the whole interview process and then right before they're going to extend an offer, they say, “Hey, you know, I'm going to go to the hiring committee and make a recommendation that we hire. But before I do that, you know, what are you hoping to make if we actually do extend an offer and I go talk to the comp team?” And it can feel like, well, gosh, I better tell them the answer to that question because they literally just said, basically, like, “I have an offer for you, but first, I need this information from you.” And it can feel really kind of daunting to say, “No, I'm not going to give you that.”So, the question is, you know, should you give them that and how? You shouldn't, as I mentioned earlier. Giving them salary expectations, I'll give kind of a brief summary of why it's not a good idea. I think a good way to reframe that question, you know, what are you what are you hoping to make if you join our team, is, you know, “Hey, you know, we have a giant company here. We've got tens of thousands of employees. We've got thousands of engineers that are at your level and doing your kind of work. We have salary surveys that we run once a quarter, or once a month, that are super expensive. We know what everybody else in the industry is doing. We know what the value of this role is to our company. We know how many other people are applying for this job. We know how many open seats we have. You don't know any of that stuff, but even though you don't know any of that stuff, why don't you take a wild guess what we would pay you to do this job at this company at this moment?”Corey: And then of course, we're going to use it against you later, when you wind up having what you view as a negotiation, like, “Ah, but you said at the beginning of the process that this would be sufficient.”Josh: Yeah. So, that's the problem, right? As you take that wild guess and you're going to do one of two things. It's basically 0% that you're going to hit the nail on the head in terms of you guessed the actual maximum compensation that they would pay you to do the job, it's very unlikely.Corey: You're either going to guess too high and then basically get yourself disqualified—Josh: Yeah.Corey: —you're going to get too low and leave money on the table, or you're going to get it exactly right, but you'll never know whether you got it exactly right or whether you guessed low.Josh: Right. Even if you do guess exactly right, you won't know that you did. And so, of course, if you guess low, like you said, you leave money on the table. And the really pernicious thing is, you could guess low and still feel great about the result and never know it or not find out until the next time you get a job, which is to say, you know, you say a number that's well below the bottom of the minimum that they could pay you and so you say, I don't know, to use round numbers, you say $100,000. And they go, “Great, how about 120?”And you say, “Wow, they must really like me. They're going to pay, I just said 100 and they said 120. That's amazing.” And really what's going on is they're looking at, you know, their internal pay structure and they're like, we can't pay less than 120, like, the pay structure starts at 120. So, we'll pay 120, which is the literal bottom that you could make.You feel like you got a huge win of a 20% bump, but the reality is, you're probably not anywhere near the middle of that pay range and you're way behind the eight ball already. And of course, you could overshoot. And the worst-case scenario is you overshoot so far that you basically disqualify yourself from the process early. So, it's like, if it's on that first screening call, and you say—Corey: And they view you as being fundamentally unserious, where it's a, okay, the compensation for this role is 100 to 130, for example—to use made-up numbers—and you come in asking for 340. It's… okay like, there's no point in even doing a counter and having a negotiation at this point. We are so far apart, that it doesn't work out that way.Josh: Right, which on the surface, seems like oh, well, I just saved a bunch of time. But in reality, what you may have done is sort of like knocked yourself out of the entire hiring funnel for them, when what could have happened is perhaps you could have as you interviewed, you could have aligned better with a more senior role that would have had a higher pay range that you would have been a better fit for, you could have changed their budget based on the way that you present in your interviews and what they perceive from you. And who knows, maybe you actually do get an offer that looks like 340 because they say, “Oh, wow, we had you leveled as a, you know, an L6 and really should be, like, at an L7. So, how about this, you know, this senior or principal or lead role over here that we've been trying to fill for six months, we now realize you might be a good fit for that role. Why don't you go talk to that hiring manager, and if we have to, we'll just put you into that hiring stream?” Instead of, you set a giant number and we got to kick you out because there's no room for you here.Corey: This is all well and good and we're talking about effectively cash comp and salaries, but so many companies these days seem to tie a fair part of their compensation to the equity portion of it. And because remember, everything's up and to the right. Always. The end. Until one day, it's very much not.And now we're taking a look and seeing that, for example, Amazon stock has largely been in the toilet for a couple of years. It's what, 50% off of what it was at the peak.Josh: Yeah.Corey: So it's, on some level, when you're negotiating comp, it feels like you're being asked to predict the future of how well the company does. And at these multibillion-dollar company scales, are you really going to be in a position personally to meaningfully impact the stock price? Like, well, not positively anyway. And it just feels like it's a bit of a shell game where if you can't spot the sucker, it's probably you. Because I wanted to be an engineer, not a stockbroker.Josh: Yeah, I mean, first of all, you're right, that no individual engineer is really going to be impacting the bottom line of Google.Corey: Unless I take the site down.Josh: Right. Well, I was just [laugh]—man, you beat me to the punch on that one. Yeah. So, there is a possibility that one engineer could have a dramatic impact, but not the kind that you would hope if [laugh] you're also tied to their stock price, right? So, there's a couple of ways that I think about this.One of them, you mentioned the Amazon stock going down. So, one thing that's really interesting about that is really what Amazon is doing is they're targeting a total annual compensation number with their stock. And so, they start with their current known stock value—I don't know if they're doing this now, but for many years, they were just kind of building in a year-over-year growth number of 10 to 15%. So, we're going to give you this much total comp and we're targeting 300k total comp per year. And if you kind of map it out based on the base salary and the equity that vests and the signup bonuses they give you in years one and two, then it looks like a pretty flat, like, 300k a year when you build in that stock growth.So, the magic question that I started talking to—and had a couple of internal recruiter friends, like, last year, mid-year last year when things were looking pretty bad, and the question that I don't think that they had an answer to at the time and now they have answered is, well, what do we do when the stock doesn't grow 10 to 15% and actually kind of collapses, like, takes a huge nosedive? And the answer is that Amazon is still targeting a total comp of 300k a year. And they go back and they say, “Well, here's some more RSUs at the current value to kind of makeup for that. Here's your new vesting schedule on these.” They essentially are giving refreshers, and here's the new vesting schedule.And so, at least in Amazon's case, they did kind of try to right the ship. But the reason is that something you alluded to, you're not really getting equity in the company because you impact the company; you're getting equity in the company because it's another way for them to kind of generate, quote-unquote, “Cash flow” of some kind or comp, that isn't, you know, dollars coming off the books. So, this is something I think that's kind of a TBD is, Amazon has now answered this, which is we're going to give him—because otherwise, they're going to have a mass exodus, right, like, if you thought you're going to make 300k a year and you're actually going to make 180k a year, that's a huge dropoff, and you're probably going to be looking elsewhere. So, they say, “Well, here's some more RSUs.”The question is, you know, what will other companies do? All of this is, you know, we're talking about public companies here. So, there's a big difference between, like, Amazon stock, Google stock, whatever—or GSUs, whatever you want to call them—and then private, pre-IPO equity, and all these different things. I see those as much more in the category of what you described, which is, you know, if you're getting stock options on, like, an early stage, you know, like, an early stage startup, right, they're raising, like, their first or second or third round, you are going to have maybe kind of a large impact on the trajectory of the company, but on the price of that you have almost no agency whatsoever because of all the options that they have for dilution and all that other stuff that can go on and whether you even have shares that are going to be liquid at some point and all that stuff. So, I see that as much more like, you've just got to look at the company, the cash that they're paying you, how you feel about that, how you feel about the mission of the company, and understand that you've got, you know, you've got some lotto tickets in that company and who knows, maybe it goes to the moon and you get to go along for the ride, but much less certain than, you know, like I said, like, an Amazon-type situation where they actually will give you even more RSUs if the stock tanks over the course of the year.Corey: What are you seeing these days in terms of the macroeconomic conditions as a result? Like, some wit on Twitter said that the correction in the market has identified the grim reality that there are more engineers making $600,000 a year than there are engineering problems that need $600,000 engineers to fix them. So, there's a certain, are people being overcompensated? Is there a correction in the market? Is that changing the world of salary negotiation and peoples' job mobility?Josh: I think—working backwards—yes, job mobility is affected right now. I mean, I've seen you know, even in my own business, there are just fewer people reaching out and saying, “Hey, I have an offer at a big tech company.” Which is, you know, all over the news, layoffs. First, it was hiring freezes, right? This is late last year, October last year-ish, Q3, Q4, last year. They kind of said, “Oh, we're going to hire—we're going to slow down for a little bit on this hiring.”And then it was layoffs. And so, the last several months have been layoff after a layoff, you know, 5% here, 10% there at lots of different companies. Paradoxically, a lot of those companies are still up into the right, if you're looking at their stock price, lately. And I think a lot of that is back to the first thing that you said, which is, you know, do we have more engineers that are kind of sitting around looking for problems to solve than there are problems to solve? And I think the answer was probably, yes.Certainly, the pandemic, interest rates where they were, and all these other kind of macro-economic things, which I won't opine on too much because I'm not super-educated on them, but I understand them well enough to understand that basically, it was a better investment for a big company to hire an engineer, than necessarily to try to find somewhere to invest that money because interest rates were so low, so it's hard to find a nice quote-unquote, “Risk-free” return on the investment, so they said, “Why not? We'll just hire some engineers and maybe we'll get a bigger ROI there. We'll try a bunch of different projects, we'll put a bunch of people and maybe we'll go to the moon.”Corey: A lot of speculative or strategic hiring—Josh: Yeah.Corey: —and then okay, then you have—something that companies do when they have extra money is they greenlit additional projects. And when things get tight, they wind up effectively removing some of those projects from the table. And what I think people misunderstand in many cases is that compensation of employees is always more expensive than the infrastructure they work on, with very rare exceptions. So, the AWS bill is always secondary to payroll expenses, and fixing AWS cost takes time, effort, and engineering work, whereas laying people off requires a couple of difficult conversations—that companies increasingly seem to be bungling—and that's the end.Josh: Yeah. I think you're right about that. I mean, payroll, it's an old saw in businesses is that payroll is the biggest expense, right? Like, it's very expensive to hire people. But it could be the kind of thing, like you said, “We'll just fire up a bunch of these projects. We've been thinking about them anyway. We can't really invest this money anywhere else for a good return, so we'll take some shots here.” Right?But then interest rates go up and oh, there are places that I can get a nice return on this investment of cash, so maybe, you know, some of these projects that aren't going so well, we're going to shut them down. We're going to lean up a little bit. We're going to increase our margins, reduce our payroll costs, and just kind of ride this economic turmoil out and see how it goes. And who knows, maybe they'll fire some of those projects up later. But yes, it's much easier to say we're laying off 10% of our workforce tomorrow than it is to make a lot of other changes, especially on the expenses side.That's one of the few expenses I think that a company has direct control over and can simply reduce if they choose to. And that's kind of where we are right now, I think. And so, you mentioned economic mobility or job mobility. It's definitely way down. And I think the reason is that, you know, I mean, if I'd been through layoffs at companies that I worked at before, right?It's a really uncomfortable feeling, where the person that was sitting next to you in the office next to you gets laid off and you're sitting there wondering, “Am I going to be next?” And the last thing that you're going to do is start kind of poking your head up and looking for jobs and making it known that you're shopping, or even go ask for a raise or something because you're just trying to keep your head down and maybe the scythe will pass over me [laugh], right? Maybe they're going to miss me in this next round of layoffs if I just keep my mouth shut and I keep typing away here on my keyboard. So, I think a lot of that is going on where people are, if they're still employed, they're happy to be there and they're just going to kind of hunker down. And then if they're not employed, there's not a lot of them, you know, especially if you're coming from big tech, you would want to go most of the time to another big tech company.Like, that's why you're there, a lot of people aspire to work for big tech, they want to be in that ecosystem. But if all the big tech companies are laying people off or freezing hiring, there's nowhere to go. And so, there's nowhere to move if they want to. They don't want to make it known that they're looking to move because they don't want to draw attention to themselves if they're still employed. And if they're unemployed, the options for them to go somewhere are slim, but they probably have a severance package that they're kind of going to milk for a little bit and see if things kind of warm up again and they can go find somewhere to move to. So, everything feels, in the big tech level, there's a lot of inertia right now. People are just kind of sitting back, and there's a lot of friction, and they're just kind of hanging back to see what happens.Corey: And also, at least from my somewhat naive perspective, it feels like when people do get offers and they have made the decision to move on, there's an increasing sense of they should be thankful for what they get and not rock the boat by asking for more. But I vehemently disagree, to be very clear on this. I think that negotiate for the best package you can get. Do it in good faith and be responsible about it, but money that is life-changing to you is a rounding error at best for a lot of these companies. You will always be more invested in this than the counterparty that you're negotiating against. But it just really throws me and on some level, makes me sad watching people take less than they could be getting.Josh: Yeah. I mean, I think that's just the nature of people who are spooked when the economy is doing weird stuff. And it's an understandable reaction to it, but I agree with you. Just yesterday—you know, I'm in a bunch of [laugh] a bunch of different developer Slacks. I don't know which one this was, but I was in a developer Slack—and somebody was saying exactly that.They're like, “Yeah, I got this offer, it seems pretty good. I don't know if I should bother negotiating it, you know? Like, I, I—shouldn't I just be, you know, pretty satisfied with this thing that I got?” And I wrote a long response, which was, the short version of it was basically, “No.” And the reason is, think about all the costs that the company has incurred just to get to the point where they made you an offer?It was expensive for them. Believe me, a lot of money has been spent. They've gotten all the way to the finish line with you. I mean, the number is at least in the thousands of dollars; it's probably in the tens of thousands of dollars, especially if they flew out for an onside or something. If you went through an interview loop, just do the math on, well, I talked to six people for about an hour apiece. That's six hours right there of really expensive time probably at, like [laugh], you know, senior manager and above pay rates.So, they put a lot of money into trying to fill this role. They want to fill the role, especially in this environment. If you're that deep in the process, they've got a role that they probably feel is pretty crucial to be filled. So, you've got a lot of reasons that you should be optimistic about the value that you're bringing to that role and I think it's a mistake to not see what the maximum value is that you can get in return for the work that you're going to provide for them. So, I do think that being scared is not the right response there, again because they've made a significant investment to get to the point of making an offer.And remember their fallback, right, if you negotiate with them and they don't want to give you any more, I have never seen—and I underline the word ‘never—I've never seen that a big tech company, somebody negotiates, and the big tech company says, “Nevermind. Get out of here.” Job offer went away. I've never seen it.Corey: I was about to ask that because I've heard about it at startups. And back in years when I was on Twitter a lot more than I am now, I periodically have people messaging me saying that this happened to them. What should they do? Do I want to put the company on blast and the rest? It's something I learned relatively early on in that process was before I go off half-cocked—which I'm thrilled to do—can I get a screenshot of that email exchange back and forth?Because it hasn't happened often, but once or twice, what I have clearly seen is that the company makes an offer in good faith and the person comes back with what they believe is the professional way to negotiate for more money and it is such a screaming red flag that is basically fists-of-ham-powered here that companies are like, “Oh, thank God. We just learned this giant red flag. We can get out of this super easy by rescinding the offer because of the negotiation, rather than asking them who they think they're speaking to like that.” And that is the way of getting out of it in those cases. I don't think that's particularly common, and as you say, I don't suspect that happens at big tech companies.Josh: I mean, it's not a good look, right? There was a period last year where a big tech company… [laugh] I don't know if this is privileged information or not, but they were actually resending offers, and it's because they had gotten out over their skis. They were hiring way ahead of where they should have been, and then of course, everything turned and they had to start reducing headcount. So, they did, and then they started actually res—Corey: I can think of at least three companies off the top of my head that would qualify for that story. A lot of it came, but no one made an announcement that we're rescinding offers, but it doesn't take much on Twitter when you start seeing wow, 15 people all popped up at the same time claiming that. I wonder if they're telling—Josh: Weird.Corey: —the truth, given they've never—Josh: It's a pattern.Corey: —interacted with each other?Josh: Yes.Corey: Yeah…Josh: So, without putting them on blast, obviously, the reason I'm not saying their names is I would be putting them on bla—it's not a good look, right? Nobody wants to know that they're in the interview process for a company who is known for rescinding offers. And so, you know it wasn't a decision they took lightly. And so, to your point, companies are not just going to willy-nilly start pulling back offers because that's really terrible PR. I mean, it's just not a good idea.So, it's either what you said, which is—and this is something, like when I say, “I've never seen it; underline the word never,” right, what I mean is I work with people one-on-one for a living; that's what I do. None of my clients have ever had a job offer rescinded from big tech company. That's not to say it hasn't happened for reasons like you mentioned.Corey: Yeah, I have to imagine that the emails you help them craft to respond to these things don't start off with, “Now, listen here, asshole…”Josh: Right.Corey: Like, I sort of get the sense that that's not quite the negotiating tone that you take, most days.Josh: [laugh]. No. There's no, like, you know, “I've CC'd my lawyer on this email… and blah, blah—” you know, that's not how I negotiate; it's not a good way to negotiate if you want to get good results and build rapport with people. So, in general, if you follow what I would call, like, kind of good negotiating practices—which is self-serving because I would say that I've created a lot of them for salary negotiations, right—and if you're following the best practices there, everybody's understanding that we're having a professional, business conversation among, you know, [unintelligible 00:26:52] professionals. We're trying to find the best result, that's good for everybody and we're going to get there.And so, as long as you're not—you know, you mentioned, you know [laugh], I say, you know, pounding your fist on the desk and making ultimatums and stuff, like, that's not how I negotiate; you can hear it in the way I talk. You're going to be fine. They're not going to be rescinding offers and therefore, you have pretty much carte blanche to, in good faith, negotiate with them to see if there's more room to negotiate. And how aggressive you're being and what you're asking for, these are all things that are dependent on the situation, right? There's some cases where asking for another half a million a year would be completely absurd; in some cases where it's totally appropriate [laugh] and it just depends on what your situation is.Corey: For some roles, if you just accept the offer as given, you will lose status in their eyes, on some level. For example, one of the challenges we've had with contract negotiation has been when we hire folks to work on negotiations. It's one of those, like, “Okay, do we want somebody who accepts the first offer or do we want someone who really fights us tooth and nail over every aspect of it?” And it's, on some level, it's an extension to the interviewing process there.Josh: Yeah.Corey: I don't know what the right answer is on that I mostly shrug and make that my business partner's problem.Josh: I think it's a good metric to see, especially in your business, like, you want to know not only, like, can they negotiate contracts and all this stuff, but you want to know, like, how savvy are they in terms of business? And I think, in general, a person who just accepts the first offer they get in business, I will not say that they are not savvy because I don't know that, but it's not a signal of savviness, I think, to just outright accept the first thing that comes your way in business, in general.Corey: Oh, when I wind up interviewing people in person and telling them about offers and whatnot, in years past, it was always a, would you like me to sign it right now? It's… to be honest, I'm actually starting to reconsider having given it to you at all because only someone who is deranged is going to sign a contract they haven't read, and we don't try to hire for that.Josh: Right. Yeah, I mean, that's just not—especially when your job is negotiating—you want to know that this person is running a number of filters when they're considering, you know, what is probably a kind of a life-altering decision for them, right? And so, one of those filters is, “Are the terms of this contract good for me? Is there anything dangerous in here?” And one of the filters is, “Am I being appropriately compensated for the value that I'm going to bring?” That's the big one that I focus on, right?And there's a number of those filters and I think—you know, when I'm coaching someone, the first thing that we always say when a job offer comes in is, “Hey, thanks for the offer. I appreciate it. If you wouldn't mind, I'd like”—Corey: Yeah, acknowledge receipt.Josh: Yeah, yeah, “Thank you. I got the offer. Thank you for that.” And also acknowledge it and be thankful. Like, you know, “Hey, I appreciate it.” Like, “We have now made a significant step forward in this whole process that we're going through. I appreciate what you've done to get us here. I appreciate the fact that you're giving me an offer. That demonstrates a lot of trust and all these things. And if you don't mind, I'd like to take a day or two to think it over.” And then the last thing is, “Would you mind sending me a bullet-point summary in email of the numbers that you said, so I make sure I don't mess them up?”Because you're trying to avoid the very unlikely chance that they said numbers and you heard different numbers and then you start negotiating based on the different numbers and everything just kind of go sideways. So, that's the first three things: “Thank you for the offer.” “Can I have a day or two?” “Would you send me a bullet-point summary?” It doesn't have to be formal; just bullet points is fine.Corey: Would always irked me—and I you tend to see this a lot more with early career folks, but there's also this is a common failure mode as well among people who have been in one job for a while where they have gotten completely rusty at doing the interview dance. And they tend to view jobs as being this benevolent gift bestowed upon them by the employer and they become falling over themselves, just thanking them for the opportunity and the rest. And no, no, no, no, no. A job is a mutual exchange of value. You are solving a problem that the company has, and in turn, they are bringing you in and giving you a not inconsiderable amount of money—presumably—to wind up solving that problem for them, you both come out better than you were independently. That is what a job is. Confusing the power dynamic for something else feels, to me at least, like it's the wrong way to view things.Josh: Yeah. I've always not liked even the meta sort of way that we talk about jobs as, like, jobs created, jobs destroyed, somebody gave me a job. I don't know when that term—I would be curious actually, to kind of know the etymology of that term, but like, when we started describing jobs is the thing that was given or taken or—and instead, what it is, is it's a verbal contract or written contract. It's like, “Hey, I'm going to do work for you because I bring value. You're going to pay me because I'm creating value and because it's valuable to you. And we're going to figure out, you know, what's the meet-in-the-middle number, basically, that makes us both feel good about that business transaction.”You as a company can't do what you're doing without people like me. And I as a person have found a good place to flex that particular muscle at your company. That's great for both of us. Let's figure out how, you know, we can both be happy with it. So, it's definitely not that, you know, nobody's really doing anybody favors there. You're both entering into a mutual exchange of value for business reasons.And of course, your business reasons are different than theirs, but that's what they are. So yeah, I like the way that you frame that and you think about it. And I do think it can be a little harmful for people to have that perspective, especially like if they're in a position where they're thinking, “Oh, I'm so thankful that this company is willing to give me this job.” You know, “They're gifting me with this job and they're creating this job for me.” That's actually not what's happening.Corey: Something that I want to talk about, just because I've gone through this process myself as an employee, who interviewed a lot, negotiated a lot, and got hired a lot. Then I started this place and I've been on the other side of the table. And it turns out that it's not that hard to be a human being when you're the hiring manager and making these decisions. And understand the fact that yeah, you may be hiring five people this month, but these people aren't accepting five job offers a month—you hope—and going through that entire process themselves. And extending grace is just not that hard.Like, one thing that we've done since day one here has always been to put our salary compensation for the job in the job posting so we don't waste anyone's time. Where, like, “Well, what do you want to make?” It's like, if someone walks in to buy a car, the salesperson doesn't say, “Well, how much do you want to pay for it?” It doesn't work that way. It's, “This is the thing we're offering. This is the compensation we can build here. We don't do equity, so there's no funny money stories.”And yeah, I know you'd like to make three times more. So, would we, but without growth, that doesn't become sustainable. So, let's talk about how to get there. And being a responsible, decent human being is not that hard in the hiring space, but no one tells those stories because it's more fun, and outrage goes around the internet three times while the truth is still putting its boots on, where the idea of these horrible companies with people who don't know what they're doing just completely kicking themselves.Josh: Yeah, you know, it's funny, I thought, two things kind of flashed in my mind while you were talking. And the first one was, you know, I was a hiring manager for a while. And a lot of the sort of philosophy that I built around, like, asking for raises and promotions, right? Like, I have a process for that that's different than negotiating job offers, but the way that I developed it was as a hiring manager, my employees would say, you know—in their one-on-one or something—like, “Hey, you know, I feel like, you know, here's what happened when I started the company. For these reasons, I feel like I'm like, way behind where I should be in pay. Can you help?”And so, the way that I kind of approached that was, yes, I want to help them, but I cannot really do that on my own. I need a lot of information that I don't have for them. So, what information do I need from them to have them help me help them to get them a raise or get them or promotion, right? And so, I started thinking about it from the manager side of, like, essentially, kind of like a compassionate approach to, like, I need you to give me information and I will do what I can for you. And that was like, my whole philosophy with that, which is, I think I agree with you, but I need you to kind of prove to me that you should be paid more. Not because I don't believe you, but because I can't get you more money if I can't make that case, and I'm not able to make that case on my own, right?And so, I think that there is room for hiring managers to be compassionate in terms of like you said, just putting numbers in a job description, just so the person knows, like, yeah, this seems like it's probably approximately for me. Or you know, like I said, as a hiring manager saying, “Hey listen, I need you to bring me these three things. If you bring me those three things, that'd be the information that I need to go to finance or to HR or whoever and see if we can get you a raise or get you a promotion.” And if we can't, then I'll figure out, like, what are the next steps for you to get there to do that thing. And I think that in general, that's just removing friction from, like, forward-moving business processes and that's a good way to go.I think for you, right, you're saving yourself time, by putting those numbers in the job description, you're saving your applicants time by putting the numbers in the job description, and you're also kind of setting the terms for, like, the conversation that you're going to have, in addition to the abilities that they're bringing, the skills that they're going to bring, the things they're going to do for your company, you're also saving time on what the pay is going to be, what the compensation is going to look like approximately for that role, so that you can say, “Are we having a conversation whose parameters are known to us and that we agree upon to start with? Yes? Okay, great. Let's keep talking.” Otherwise, no, and maybe they should go somewhere else or maybe you need to rework your job listings because nobody is [laugh] applying for that job, right?But it's all data. It's a feedback loop. And it can be done compassionately. It doesn't have to be this, kind of, aggressive, you know, Shark Tank-style, like, I'm going to beat you over the head with this thing and get my result that I want, regardless of how you feel about it or, you know, how it makes you feel as a person.Corey: One last thing that I want to comment on this is that I've done this a fair bit, but if I wound up finding myself on the job market, I would absolutely reach out to you for coaching on the salary piece of it, just because you are a dispassionate third-party who is very aware of what the current state of the market is, you have a bunch of different offerings these days that range from a bunch of free articles on your website all the way up to individualized personalized coaching. I have bullied friends of mine into becoming your clients with a, “If he doesn't justify his fee, I will pay it instead of you.”Josh: [laugh]. Thank you for that, by the way.Corey: Of course. And I've never had to do it because you know what you're doing and the results absolutely speak for themselves. But my question is, what are you doing these days that's between the everything free on a website if you read it and, individualized one-on-one coaching? Are there now points in between those two extremes?Josh: Yeah. I think you actually summarized the whole spectrum pretty well. I mean, I've made—since I started my business seven-and-a-half years ago, one of the primary things that I did to start was, I'm going to create as much free content as I can and make it publicly available, just so that people can find it. Because there's no way that I can talk to tens of thousands, hundreds of thousands of people one-on-one. And so, that's there on fearlesssalarynegotiation.com.The other end is my one-on-one coaching which I developed because, frankly, people were reaching out and saying, “Hey, will you coach me through this?” And I said, “Sure.” And I developed that business. And then in between is, I created a program… three or four months ago, I launched it. It's called Salary Negotiation Mastery, but it is essentially me sitting down late last year with an instructional designer and asking the question, how can I teach the methodology I use in my one-on-one coaching to people who can't afford to hire me or just aren't inclined to hire a consultant to help them do something? And how can I teach that to them in a way that they can execute it on their own to get a good result, or possibly, you know, they're just at an income bracket right now where it doesn't make sense for them to hire me?And so, that's kind of the middle ground there is it's a coaching program, but it's wrapped in a do-it-yourself thing, where you have, you know, worksheets and workbooks and things that you can use to do it yourself using exactly the methodology, even the templates and things that I use with my clients. And the only thing, of course, that you're missing is my brain, but I've put as much of that as I can into the program as possible. So, that's the spectrum is: free articles, Salary Negotiation Mastery in the middle, and then the top tier offering that I have is, like you said, one-on-one bespoke coaching, where I work with somebody one-on-one. And I don't do a lot of that, just because it requires a lot of time and I like to give a ton of focus to everybody that I work with.Corey: Which makes sense because it also feels like it's a very time-sensitive issue as well. Like with AWS bill, great people want it fixed now, but then procurement can slow things down. But that's okay; there's another bill coming next month. Job offers, speaking as a hiring manager, if you accept the job, terrific, that's great. If you don't. Then okay, that's unfortunate, but it happens. But either way, let us know so we can either continue speaking to other people or begin planning for you to show up. So, it feels like there's very much a strong sense of urgency around the entirety of what you do.Josh: Yeah, especially for the coaching. And the whole offering for my coaching offering is really designed to make sure that I have enough bandwidth available for someone to call me. I mean, literally, as we're in this recording right now, I could have gotten an application in the email that would say, “Hey, I have a job offer in hand from Google. It's for this much money, it's for this level, can you help?”Corey: “And they're on the other line. Please respond immediately.”Josh: Yes. And their recruiter is pressuring me for an answer. They want to get back to the hiring manager. And so, I need to be able to respond quickly, get back to that person, have an intro call, get to know them, see if I can help in their situation, kickoff, you know, this afternoon or tomorrow morning, get a counteroffer over in the next 24 to 36 hours, that kind of thing. And so, in order to do that, I've got to build an offering that allows me to have enough bandwidth and, kind of, agency over my schedule so that I can just sort of jump in immediately into the middle of a process that's ongoing and help the person get the best result possible.So, I enjoy that to be honest with you. I like, kind of, being called on in emergency situations like that. It's really good. But of course, I had to structure the offering so that it facilitates it so that I'm not, you know, already booked on the phone eight hours a day and unable to even look at my email until tomorrow or something because it just wouldn't work.Corey: Yeah. There's something to be said for being able to take a vacation.Josh: Yes. Which takes some planning, but can be done [laugh]. And it means I just have to turn off the application sometimes [laugh].Corey: Glad to see things are still going well for you. You started your business a few months before I started mine and it's great to see that we're both still failing to go out of business every month.Josh: [laugh]. That's how I see it, too. I'm still here. Now, what [laugh]? That's every month on the first when I do the books.Corey: [laugh]. I hear you. I really want to thank you for taking the time to speak with me. If people want to learn more—and if they're changing jobs, they absolutely should—where should they go to find out?Josh: fearlesssalarynegotiation.com is the first place to go. I'm also on Twitter. I don't tweet a lot kind of actively, I probably should do better on that, but I'm at @joshdoody on Twitter and I'm very responsive on there. So, you could ping me on there or, you know, connect with me on LinkedIn if you wanted to; I'm also joshdoody there. But fearlesssalarynegotiation.com is the best place to go, especially if you're kind of in a time crunch. Everything is just right there for you to jump in and kind of grab, you know, the free resource that you might need or apply to work with me as a coaching client.Corey: Oh, the template emails are glorious.Josh: Yes. Those are one of my favorite things on the site. They don't look like other emails that people write, and something I take a lot of pride in is communicating well and creating good email templates that help a lot of people.Corey: Oh, in TextExpander, for a decade now, I've had a fill-in-the-blanks templated resignation letter, which it turns out, most people don't have. But I don't need it much these days, but it is useful to wind up giving to people from time to time. Like, “So, how do I tell my boss to take this job and shove it?” It's like—Josh: Well—Corey: —life is long and the industry is small. Go vent to your friends over beers. But there's very little upside and huge potential downside, so write the formal thing. Here you go. And it turns out that it's sort of cathartic, just filling that out. And it's like, oh, that's what this [unintelligible 00:42:05]. And it often helps people step back from the ledge sometimes. Or pushes them right off, depending.Josh: I think that's a useful service.Corey: But yeah, the [unintelligible 00:42:11] template emails are way better than mine.Josh: [laugh]. Well thanks, I appreciate it. It means a lot to me.Corey: [laugh]. Josh Doody the owner of fearlesssalarynegotiation.com. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment and be sure to include your salary expectations.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.
If you are thinking of using an external CICD tool to deploy to AWS you are probably wondering how to securely connect your pipelines to your AWS account. You could create a user for your CICD tool of choice and copy some hard coded credentials into it, but, let's face it: this doesn't feel like the right - or at least the most secure - approach! In the previous episode we discussed how AWS and GitHub solved this problem by using OIDC identity providers and this seems to be a good solution to the problem. In this episode of AWS Bites we will try to demystify the secrets of OIDC identity providers and explain how they work and what's the trust model between AWS and an OIDC provider like GitHub actions. We will also explain all the steps required to integrate AWS with GitHub, how JWT works in this particular scenario and other use cases where you could use OIDC providers. In this episode, we mentioned the following resources: - GitHub docs explaining how to integrate with AWS as an OIDC provider: https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect - Article “What's in a JWT” https://loige.co/whats-in-a-jwt - jwtinfo, CLI tool to inspect JWT: https://github.com/lmammino/jwtinfo - AWS action to assume a role from a GitHub Pipeline: https://github.com/aws-actions/configure-aws-credentials#assuming-a-role - Great post by Elias Brange detailing how to setup GitHub OIDC integration for AWS: https://www.eliasbrange.dev/posts/secure-aws-deploys-from-github-actions-with-oidc/ - Previous episode on why you should consider GitHub Actions rather than AWS CodePipeline: https://awsbites.com/44-do-you-use-codepipeline-or-github-actions/ This episode is also available on YouTube: https://www.youtube.com/AWSBites You can listen to AWS Bites wherever you get your podcasts. See https://awsbites.com for all the links. Do you have any AWS questions you would like us to address? Connect with us on Twitter: - https://twitter.com/eoins - https://twitter.com/loige
Automated, Continuous Build and Continuous Delivery are must-haves when building modern applications on AWS. To achieve this, you have numerous options, including third party providers like GitHub Actions and Circle CI, and the AWS services, CodePipeline and CodeBuild. In this episode we focus on GitHub Actions and we compare it with the native AWS features offered by services like CodePipeline and Code Build. In particular we discuss what CodePipeline offers and how to set it up, what the tradeoffs are and when to choose one over the other. We also discuss when you should look outside AWS to a third-party provider and highlight when GitHub Actions can be a great fit for your AWS CI/CD needs! In this episode, we mentioned the following resources: - Example pipeline for a serverless mono repo using CDK is available in SLIC Starter: https://github.com/fourTheorem/slic-starter/tree/main/packages/cicd - 50+ official actions provided by GitHub themselves: https://github.com/actions - How to configure OIDC integrations with AWS and other services like GitHub: https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services - GitHub Actions billing details: https://docs.github.com/en/billing/managing-billing-for-github-actions/about-billing-for-github-actions - Workshop illustrating how to create CodeBuild and CodePipeline resources using CDK: https://cdkworkshop.com/20-typescript/70-advanced-topics/200-pipelines/3000-new-pipeline.html - Paul Swail's article “Why I switched from AWS CodePipeline to GitHub Actions”: https://serverlessfirst.com/switch-codepipeline-to-github-actions/ - A tutorial article by AWS showing how to authenticate and use GitHub actions to build & deploy a web app to an EC2 instance https://aws.amazon.com/blogs/devops/integrating-with-github-actions-ci-cd-pipeline-to-deploy-a-web-app-to-amazon-ec2/ - Other examples of when it is OK to ditch AWS services for third party (previous podcast episode): https://awsbites.com/43-when-is-it-ok-to-cheat-on-aws/ This episode is also available on YouTube: https://www.youtube.com/AWSBites You can listen to AWS Bites wherever you get your podcasts. See https://awsbites.com Leave a comment here or connect with us on Twitter: - https://twitter.com/eoins - https://twitter.com/loige
On The Cloud Pod this week, with the first half of the year full of less-than-ideal events, the team is looking forward to another next six months of less-than-ideal events. Also, everyone is excited to see how they can manipulate the AWS BugBust Challenge for a free ticket to re:Invent. A big thanks to this week's sponsors: Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. JumpCloud, which offers a complete platform for identity, access, and device management — no matter where your users and devices are located. This week's highlights AWS launches the BugBust Challenge in the hopes of finding and fixing 1 million bugs. The challenge aims to help developers improve code quality, eliminate bugs and boost application performance while saving millions of dollars in application resource costs. Google has announced new features for Cloud Monitoring Grafana plugins. The new features include popular dashboard samples, more effective troubleshooting with deep links, better visualizations through precalculated metrics and more powerful analysis capabilities. Azure VM Image Builder service is now generally available. Image Builder will make it easier to build custom Linux or Windows virtual machine images. Amazon Web Services: Does Not Have Bugs AWS announces the world's first global competition to find and fix 1 million software bugs. We don't think they're referring to Amazon bugs, just software bugs in general. AWS launches customized images for Amazon EMR on Amazon Elastic Kubernetes Service. If you're looking to reduce the time it takes to build images, that's a good thing: otherwise it's a fully managed service, so we're not sure that users will care. Amazon announces new Java Detectors and CI/CD Integration with GitHub Actions for CodeGuru Reviewer. We're amazed by how quickly GitHub Actions is being adopted. AWS acquires communication technology company Wickr. We want to know why Amazon is buying this: maybe they're trying to enhance their enterprise and public sector application suites. AWS now supports container images to simplify continuous integration tasks. Continuing to build the ecosystem around serverless applications is a smart move by AWS. Google Cloud Platform: Smart Player Google announces that a new public dataset for Google Trends is now available for preview. This is really cool. Google introduces a new Tau VM family that extends Compute Engine's VM offerings. If you're using some of the older VM classes, this is a reminder to check out the new ones that could save you money. Google announces a new version of Transfer Appliance for the US, EU and Singapore regions. It's new and improved — they just haven't told us how. Google announces new features for Cloud Monitoring Grafana plugins. Grafana is one of our favourite visualization tools so this is great. Google launches three security and scalability improvements for Cloud SQL for SQL Server. This is a smart play: these capabilities will help differentiate Google's product offering through improved performance. Azure: Gives You Ingestion Azure introduces the Ingestion Client for Azure Speech. Getting a full-blown scalable and secure transcription pipeline is great, but we really don't like the name. Azure VM image builder service is now generally available. We've found a customer who is able to pay Hashicorp to update to Go. Azure has built a cloud adoption framework for retail. We hope they extend this beyond the Azure lens: it should tie back into the much larger digital transformation story for the sector. Azure has partnered with Red Hat to offer Red Hat JBoss EAP on the Azure App Service. It's nice to see digital app services available from Microsoft. TCP Lightning Round Justin stuns everyone with his multimedia power move so takes this week's point, leaving scores at Justin (11), Ryan (5), Jonathan (8), Peter (1). Other Headlines Mentioned: Expansion of the public preview of on-demand disk bursting for Premium SSD to more regions AWS DevOps Monitoring Dashboard solution v1.1 adds support for AWS CodeBuild and AWS CodePipeline related metrics CloudWatch adds 14 new Metric math functions Amazon DocumentDB (with MongoDB compatibility) Now Supports r5.8xlarge and r5.16xlarge Instances Configure GitHub Actions workflows with a new GitHub Action for building serverless applications AWS Control Tower announces accessibility, console and performance improvements AWS Client VPN launches desktop client for Linux AWS Lambda now supports SASL/PLAIN authentication for functions triggered from self-managed Apache Kafka Google joins the O-RAN Alliance to advance telecommunication networks Things Coming Up Announcing Google Cloud 2021 Summits [frequently updated] State of FinOps Update — July 8 (virtual) Amazon re:Inforce — August 24–25 — Houston, TX Google Cloud Next 2021 — October 12–14, 2021 AWS re:Invent — November 29–December 3 — Las Vegas Oracle Open World (no details yet)
最新情報を "ながら" でキャッチアップ! ラジオ感覚放送 「毎日AWS」 おはようございます、水曜日担当パーソナリティの福島です。 今日は 3/2 に出たアップデートをピックアップしてご紹介。 感想は Twitter にて「#サバワ」をつけて投稿してください! ■ アンケートのご協力お願いします! https://forms.gle/tapCvDfCbjXHjSMf8 ■ UPDATE PICKUP AWS Gateway Load Balancer が東京リージョンで利用可能に Amazon Connect は履歴メトリクスを 15 分間隔で表示できるように Amazon Connect は Customer Profiles のデータソースに AmazonS3 をサポート Amazon EKS は既存クラスターの機密情報を KMS キーを利用して暗号化できるように Amazon CloudWatch Synthetics はクロスリージョンバケットアクセスとアップグレードされた依存関係をサポートするように 2021 AWS DeepRacer リーグシーズンの開催 AWS CodePipeline はアカウントごとに 1000 のパイプラインをサポートするように ■ サーバーワークスSNS Twitter / Facebook ■ サーバーワークスブログ サーバーワークスエンジニアブログ
最新情報を "ながら" でキャッチアップ! ラジオ感覚放送 「毎日AWS!」 おはようございます、サーバーワークスの加藤です。 今日は 11/11 に出たアップデート9件をご紹介。 感想は Twitter にて「#サバワ」をつけて投稿してください! ■ UPDATE ラインナップ 新しいビジュアルデータ準備ツール AWS Glue DataBrew がリリース Amazon ElastiCache が memcached 1.6.6 に対応 Amazon S3 の新コンソールが一般利用可能に Amazon Redshift が TIME および TIMETZ 型をサポート FreeRTOS が IoT および AWS ライブラリを含むように AWS Systems Manager エクスプローラーが複数アカウント・複数リージョンの AWS Config コンプライアンスの概要を表示できるように AWS CodePipeline のソースアクションが AWS CodeCommit の git clone をサポート 新しいデジタルコースが追加 - レガシーデータベースからの脱却 セキュリティと IoT に関する新しいデジタルコースが edX と Coursera に追加 ■ サーバーワークスSNS Twitter / Facebook ■ サーバーワークスブログ サーバーワークスエンジニアブログ
Hoy nos visita Carlos Afonso, un Solutions Architect para hablar de DevOps y como los servicios de AWS pueden ayudar a los desarrolladores.Carlos Afonso Basado en Madrid, España, Carlos es un Solutions Architect que ayuda a Startups en España y Portugal construyendo aplicaciones robustas, tolerantes a fallas y optimizadas en costes en AWS. Cuando no esta hablando de AWS, lo podemos encontrar haciendo código como entretenimiento o intentando crear su propia cerveza (con éxitos relativos).Rodrigo Asensio - @rasensioBasado en Barcelona, España, Rodrigo es responsable de un equipo de Solution Architecture del segmento Enterprise que ayuda a grandes clientes en Iberia a moverse al cloud y aprovechar sus beneficios.LinksAWS CodeCommit: https://aws.amazon.com/codecommit/ AWS CodeCommit es un servicio completamente administrado de control de código fuente que aloja repositorios basados en Git seguros. Simplifica la colaboración en el código por parte de los equipos, en un ecosistema seguro y con alta escalabilidad. Con CodeCommit no necesita utilizar su propio sistema de control de código fuente ni preocuparse por el escalado de la infraestructura de dicho sistema. CodeCommit, que funciona perfectamente con las herramientas de Git existentes, se puede utilizar para almacenar de forma segura cualquier elemento, ya sea código fuente o binario.AWS CodeBuild: https://aws.amazon.com/codebuild/AWS CodeBuild es un servicio de integración continua completamente administrado que compila código fuente, ejecuta pruebas y produce paquetes de software listos para su implementación. Con CodeBuild, no es necesario aprovisionar, administrar y escalar sus propios servidores de compilación. CodeBuild se escala constantemente y procesa numerosas compilaciones a la vez, de manera que estas no permanecen a la espera en una cola. Puede comenzar con rapidez mediante entornos de compilación preempaquetados, o crear entornos de compilación propios personalizados que utilicen sus herramientas de compilación. Con CodeBuild, se le cobra por cada minuto de recursos informáticos que utilice.AWS CodeDeploy: https://aws.amazon.com/codedeploy/AWS CodeDeploy es un servicio de implementación completamente administrado que automatiza las implementaciones de software en diferentes servicios informáticos, como Amazon EC2, AWS Fargate, AWS Lambda y sus servidores locales. AWS CodeDeploy facilita el lanzamiento rápido de nuevas características, ayuda a evitar tiempos de inactividad durante la implementación de una aplicación y administra la compleja actualización de las aplicaciones. Puede usar AWS CodeDeploy para automatizar implementaciones de software, lo que elimina la necesidad de realizar operaciones manuales propensas a errores. El servicio se adapta a sus necesidades de implementación.AWS CodePipeline: https://aws.amazon.com/codepipeline/AWS CodePipeline es un servicio de entrega continua completamente administrado que permite automatizar canalizaciones de lanzamiento para lograr actualizaciones de infraestructura y aplicaciones rápidas y fiables. CodePipeline automatiza las fases de compilación, prueba e implementación del proceso de lanzamiento cada vez que se realiza una modificación en el código, en función del modelo de lanzamiento que defina. Esto le permite entregar características y actualizaciones de forma rápida y fiable. Puede integrar fácilmente AWS CodePipeline con servicios de terceros, como GitHub o su propio complemento personalizado. Con AWS CodePipeline solo paga por lo que utiliza. No es necesario pagar cuotas iniciales ni asumir compromisos a largo plazo.AWS CodeStar: https://aws.amazon.com/codestar/AWS CodeStar le permite desarrollar, compilar e implementar rápidamente aplicaciones en AWS. AWS CodeStar proporciona una interfaz de usuario unificada que permite administrar fácilmente actividades de desarrollo de software en un solo lugar. Con AWS CodeStar puede configurar en cuestión de minutos toda su cadena de herramientas de entrega continua, lo que permite comenzar a publicar código más rápido. AWS CodeStar facilita que todo su equipo trabaje junto de forma segura, lo que permite administrar fácilmente el acceso a sus proyectos, así como agregar propietarios, contribuyentes y espectadores de forma sencilla. Cada proyecto AWS CodeStar incorpora un panel de administración de proyectos que incluye la funcionalidad integrada de seguimiento de incidencias con tecnología de Atlassian JIRA Software. Con el panel de proyectos de AWS CodeStar puede realizar un seguimiento del progreso en todo el proceso de desarrollo de software, desde su lista de tareas pendientes hasta las implementaciones de código recientes de los equipos.Amazon CodeGuru: https://aws.amazon.com/codeguru/Amazon CodeGuru es una herramienta para desarrolladores basada en aprendizaje automático que brinda recomendaciones inteligentes para mejorar la calidad del código e identificar las líneas de código más costosas de una aplicación. Integre Amazon CodeGuru en el flujo de trabajo de desarrollo de software existente, en el que tendrá revisiones de código integradas para detectar y optimizar las líneas de código más costosas a fin de reducir los costos.AWS CloudFormation: https://aws.amazon.com/cloudformation AWS CloudFormation proporciona un lenguaje común para que modele y aprovisione recursos de aplicación de AWS y de terceros en su entorno de nube. AWS CloudFormation permite utilizar lenguajes de programación o un archivo de texto simple para modelar y aprovisionar, de una manera segura y automatizada, todos los recursos necesarios para las aplicaciones en todas las regiones y cuentas. Esto proporciona una única fuente de confianza para los recursos de AWS y de terceros.CDK: https://aws.amazon.com/cdk/ El kit de desarrollo de la nube de AWS (AWS CDK) es un marco de desarrollo de software de código abierto que sirve para modelar y aprovisionar sus recursos destinados a aplicaciones en la nube mediante lenguajes de programación conocidos.Aprovisionar aplicaciones en la nube puede resultar un proceso desafiante que implica realizar acciones manuales, escribir secuencias de comandos personalizadas, mantener plantillas o aprender lenguajes para dominios específicos. AWS CDK usa la familiaridad con los lenguajes de programación y la capacidad expresiva de estos para modelar aplicaciones. Provee componentes de alto nivel que preconfiguran recursos en la nube con valores predeterminados fiables. Esto le permite crear aplicaciones en la nube sin necesidad de ser un experto. AWS CDK aprovisiona sus recursos de una manera segura y repetible mediante AWS CloudFormation. También posibilita crear y compartir componentes personalizados propios que incorporen los requisitos de su organización, proceso que lo ayuda a iniciar proyectos nuevos con mayor rapidez.
最新情報を "ながら" でキャッチアップ! ラジオ感覚放送 「毎日AWS!」 おはようございます、サーバーワークスの加藤です。 今日は 9/30 に出たアップデート12件をご紹介。 感想は Twitter にて「#サバワ」をつけて投稿してください! ■ UPDATE ラインナップ Amazon Timestream が一般利用可能に Amazon SageMaker Processing がビッグデータ処理のためのビルトインSparkコンテナをサポート Amazon Pinpoint がイベントトリガージャーニーを発表 AWS CodePipeline がソースアクションに git clone をサポート AWS CodePipeline が GitHub Enterprise Server に対応 AWS Client VPN がクライアント間接続をサポート Amazon QLDB がインデックスの改善を発表 AWS SDK for Java 2.x に AWS CRT HTTPクライアントがプレビューリリース Amazon MSK がクラスタストレージを自動で拡張できるように 新しいソリューションが発表 - AWS WAF と VPC セキュリティグループの管理中央化 AWS Elemental MediaConnect で予約済みのアウトバウンド帯域幅を利用可能に Amazon S3 on Outposts が一般利用可能に ■ サーバーワークスSNS Twitter / Facebook ■ サーバーワークスブログ サーバーワークスエンジニアブログ
最新情報を "ながら" でキャッチアップ! ラジオ感覚放送 「毎日AWS!」 おはようございます、サーバーワークスの加藤です。 今日は 6/26 に出た 6件のアップデートをご紹介。 感想は Twitter にて「#サバワ」をつけて投稿してください! ■ UPDATE ラインナップ AWS PrivateLink 経由で Amazon Transcribeリアルタイムストリーミングにアクセスできるように AWS CodePipeline が 新しいデプロイアクションタイプとしてAWS AppConfig をサポート Amazon AppStream 2.0 がペンタブレットをサポート Amazon MSK のブローカーで追加インスタンスサイズが利用可能に Amazon Athena がパーティションプロジェクションをサポート AWS IoT Device Tester v3.1.0 for AWS IoT Greengrassが利用可能に ■ サーバーワークスSNS Twitter / Facebook ■ サーバーワークスブログ サーバーワークスエンジニアブログ
A month passed before we could blink, and once again Arjen is joined by Jean-Manuel and Guy to discuss the highlights of the April announcements. Co-starring interrupted chatbots and terrifying music. The News Finally in Sydney Sellers, consulting partners, and data providers from Australia and New Zealand now available in AWS Marketplace and AWS Data Exchange AWS Ground Station is now available in the Asia Pacific (Sydney) Region in Australia AWS Transit Gateway now Supports Inter-Region Peering in 11 additional regions EKS Adds Fargate Support in Frankfurt, Oregon, Singapore, and Sydney AWS Regions Amazon Aurora with PostgreSQL Compatibility for PostgreSQL 11 is available in all commercial AWS Regions Serverless Amazon RDS Proxy with PostgreSQL Compatibility (Preview) (not in Sydney) Exporting HTTP APIs as OpenAPI 3.0 Now Supported by Amazon API Gateway AWS Lambda now supports .NET Core 3.1 The AWS Toolkit for Visual Studio Code now supports AWS Step Functions Amplify CLI adds support for additional Lambda runtimes (Java, Go, .NET and Python) and Lambda cron jobs AWS X-Ray SDK for Go is now generally available Containers Amazon ECS and AWS Fargate support for Amazon EFS File Systems now generally available AWS App Mesh adds support to connect services deployed in multiple AWS accounts into a shared mesh Amazon EKS Now Supports Service-Linked Roles Amazon EKS managed node groups allow fully private cluster networking Databases Amazon Keyspaces (for Apache Cassandra) is now generally available Amazon RDS Now Supports PostgreSQL 12 Amazon RDS now supports MariaDB 10.4 AWS Database Migration Service now supports replicating data to Apache Kafka streaming platform (Keyspaces) Amazon Neptune now supports the T3.medium instance type Dev & Ops AWS Chatbot Now Generally Available Receive Notifications for AWS CodeBuild, AWS CodeCommit, AWS CodeDeploy, and AWS CodePipeline in Slack EC2 Image Builder adds support for Ubuntu, RHEL, CentOS, and SLES Amazon CloudWatch Synthetics is now generally available Amazon CloudWatch Synthetics now supports monitoring private endpoints in a VPC Security Amazon Detective is now generally available Review and remediate unintended access allowed on your AWS resources from outside your AWS organization Amazon Cognito Identity Pools now supports Sign in with Apple Track changes to secrets stored in AWS Secrets Manager using AWS Config and AWS Config Rules AWS Security Hub launches the Foundational Security Best Practices standard VPC & EC2 Amazon Elastic File System announces 400% increase in read operations for General Purpose mode file systems AWS Elastic Beanstalk Launches support for AWS PrivateLink AWS Elastic Beanstalk adds API support for listing platform branches AWS Elastic Beanstalk Announces General Availability of Amazon Linux 2 Based Docker, Corretto, and Python Platforms New AWS Elastic Beanstalk console now available AI & ML AWS DeepComposer is now generally available Introducing Amazon Augmented AI (A2I) for human reviews of machine learning predictions Introducing TorchServe: a PyTorch model serving framework Amazon Transcribe Medical now supports batch transcription of medical audio files Amazon Personalize now provides scores for recommended items Other Cool Stuff You can now use AWS Control Tower to set up new multi-account AWS environments in AWS Organizations Announcing the new AWS Africa (Cape Town) Region AWS Canada (Central) Region Adds Third Availability Zone Introducing AWS Cost Categories Amazon CloudWatch Contributor Insights is now generally available Introducing the AWS Transfer Family with fully managed support for SFTP, FTPS, and FTP Announcing general availability of Amazon Pinpoint Custom Channels Amazon Kinesis Data Firehose adds support for streaming data delivery to an Amazon Elasticsearch Service domain in an Amazon Virtual Private Cloud (VPC) AWS IQ waives fees until June 30, 2020 Amazon Connect adds custom terminating keypress for DTMF Amazon Connect now enables customers to interrupt Amazon Lex Chatbots Introducing Amazon Chime Proxy Phone Sessions AWS Snowball Edge Storage Optimized now delivers 25% faster data transfer performance AWS Snowball adds task automation with AWS Systems Manager AWS Snowball now supports local AWS IAM Introducing AWS OpsHub for Snow Family, a graphical user interface to manage AWS Snowball devices Other links AWS DeepComposer - Oasis - Wonderwall - Experiment #001 by The Dirk I Think Breath Noise is an Interesting One | Ambassador Lounge Podcast Episode #4 AWS Inside the Region | ig.nore.me Sponsors Gold Sponsor Innablr Silver Sponsors AC3 CMD Solutions
In this Episode of AWS TechChat, Shane and Pete embark on a different style of the show and share with you a lot of updates - over 30 updates and we tackle it like speed dating. We start the show with some updates, there are now an additional 2 AWS regions, Milan in Italy and Cape Town in South Africa. This brings the region count to 24 Regions and 76 Availability Zones. Amazon Guard Duty has a price reduction for the customers who are consuming it on the upper end of the scale, VPC flow log scanning is now 40% cheaper when your logs are more than 10,000GB. Lots of Database engine updates: • Database engine version updates across almost all engines. Microsoft SSAS (SQL Server Analysis Studio) is now available on Amazon Relational Database Service (Amazon RDS) for SQL Server now. • If you are currently running SSAS on Amazon Elastic Compute Cloud (Amazon EC2), you can now save costs by running SSAS directly on the same Amazon RDS DB instance as your SQL Server database. SSAS is currently available on Amazon RDS for SQL Server 2016 and SQL Server 2017 in the single-AZ configuration on both the Standard and Enterprise edition. • NoSQL Workbench for Amazon DynamoDB is now is now generally available. NoSQL Workbench is a client-side application, available for Windows and macOS that helps developers build scalable, high-performance data models, and simplifies query development and testing. • Apache Kafka is an option for AWS Database Migration Service and Amazon Managed Apache Cassandra Service is now available in public preview. Microsoft SQL Server on RDS now supports Read Replicas. Storage updates: • More nitro based Amazon EC2 systems receive IO performance updates. • Amazon FSx for Windows File Server is now has a Magnetic HDD option which brings storage down to 1.3cents per GB. • Amazon Elastic File System (Amazon EFS) announces 400% increase in read operations for General Purpose mode file systems. On Development front: • AWS Lambda@Edge now supports Node 12.x and Python 3.8. • Amplify CLI add support for additional AWS Lambda runtimes (Java, Go, .NET and Python) and Lambda cron jobs. • AWS Lambda now supports .NET Core 3.1. • Receive notifications for AWS CodeBuild, AWS CodeCommit, AWS CodeDeploy, and AWS CodePipeline in Slack, no need to use Amazon Simple Notification Service (SNS) and AWS Glue. • Amazon MSK adds support for Apache Kafka version 2.4.1 • Updates to AWS Deep Learning Containers for PyTorch 1.4.0 and MXNet 1.6.0 Containers updates: • AWS Fargate launches platform version 1.4 which brings a raft of improvements. • Amazon Elastic Kubernetes Service (Amazon EKS) updates service level agreement to 99.95%. • Amazon EKS now supports service-linked roles. • Amazon EKS adds envelope encryption for secrets with AWS Key Management Service (KMS). • Amazon EKS now supports Kubernetes version 1.15 • Amazon ECS supports in preview updating placement strategy and constraints for existing Amazon ECS Services without recreating the service. Connect your managed call centre in the cloud: • Introducing Voicemail for Amazon Connect. • Amazon Connect adds custom terminating keypress for DTMF. Other updates: • New versions of Elastic Search available for Amazon Elastic Search. • AWS DeepComposer is now shipping from Amazon.com Speakers: Shane Baldacchino - Solutions Architect, ANZ, AWS Peter Stanski - Head of Solution Architecture, AWS AWS Events: AWS Summit Online https://aws.amazon.com/events/summits/online/ AWSome Day Online Conference https://aws.amazon.com/events/awsome-day/awsome-day-online/ AWS Innovate AIML Edition on-demand https://aws.amazon.com/events/aws-innovate/machine-learning/ AWS Events and Webinars https://aws.amazon.com/events/
CI/CD techniques enable teams to increase agility and quickly release a high-quality product. In this talk, we walk you through the best practices for building CI/CD workflows to enable you to manage your containerized applications. We cover infrastructure as code application models with the AWS Cloud Development Kit, and we show you how to set up CI/CD release pipelines with AWS CodePipeline and AWS CodeBuild. Finally, we discuss automating safer deployments using AWS CodeDeploy.
There are many options available when you are looking for ways to implement a deployment pipeline. You might have heard about Jenkins, CircleCi, BitBucket Pipelines, GitLab Pipelines, and many others. AWS, on the other hand, offers services for CI/CD itself: CodeBuild and CodePipeline. AWS CodePipeline orchestrates deployment pipelines. Unfortunately, the learning curve is steep and the implementation is often complicated. Therefore, I recommend a more simple approach: use CodeBuild. In general, CodeBuild feels like CircleCI or GitLab Pipelines. However, CodePipeline offers tighter security controls and excellent integration into your AWS infrastructure.
Simon and Nicki run through some interesting new AWS capabilities for customers as well as a look at the upcoming re:MARS conference (https://remars.amazon.com/). 0:29 - Databases 1:20 - Analytics 1:52 - Compute 3:22 - IoT 4:05 - Customer Engagement 5:07 - Networking 5:34 - Developer Tools 7:46 - Application Integration 8:20 - Game Tech 8:42 - Media Services 9:24 - Management and Governance 12:41 - re:MARS Topic || Databases Amazon DynamoDB adds support for switching encryption keys to encrypt your data at rest | https://aws.amazon.com/about-aws/whats-new/2019/02/amazon-dynamodb-adds-support-for-switching-encryption-keys-to-encrypt-your-data-at-rest/ Amazon ElastiCache for Redis adds support for Redis 5.0.3 and the ability to change Redis command names | https://aws.amazon.com/about-aws/whats-new/2019/02/amazon-elasticache-for-redis-adds-support-for-redis-503-and-the-ability-to-change-redis-command-names/ Performance Insights is Generally Available on Amazon RDS for SQL Server | https://aws.amazon.com/about-aws/whats-new/2019/03/performance-insights-is-generally-available-for-sql-server/ Topic || Analytics Amazon QuickSight Supports Row Level Security Enabled Email Reports, New Analytical Capabilities and More | https://aws.amazon.com/about-aws/whats-new/2019/03/amazon-quickSight-supports-row-level-security-enabled-email-reports-new-analytical-capabilities-and-more/ Topic || Compute AWS Step Functions Adds Tag-Based Permissions | https://aws.amazon.com/about-aws/whats-new/2019/03/aws-step-functions-adds-tag-based-permissions/ AWS ParallelCluster support for Amazon FSx Lustre | https://aws.amazon.com/about-aws/whats-new/2019/03/aws-parallelcluster-support-for-amazon-fsx-lustre/ Announcing the Preupgrade Assistant to Migrate to Amazon Linux 2 From Amazon Linux AMI | https://aws.amazon.com/about-aws/whats-new/2019/03/announcing_the_amazon_linux_2_preupgrade_assistant/ Topic || IoT AWS IoT Greengrass Introduces New Networking Configurations and Group Permission Settings | https://aws.amazon.com/about-aws/whats-new/2019/03/aws-iot-greengrass-introduces-new-networking-configurations-group-permission-settings/ Topic || Customer Engagement Amazon Connect Simplifies Adding AWS Lambda Functions to Contact Flows | https://aws.amazon.com/about-aws/whats-new/2019/02/amazon-connect-simplifies-adding-aws-lambda-functions-to-contact-flows/ Introducing new AWS Digital Customer Experience Competency Partner Solutions | https://aws.amazon.com/about-aws/whats-new/2019/03/introducing-new-aws-digital-customer-experience-competency/ Topic || Networking Announcing the new AWS Direct Connect Console | https://aws.amazon.com/about-aws/whats-new/2019/03/announcing-the-new-aws-direct-connect-console/ Topic || Developer Tools Amazon Corretto 11 is Now Available as a Release Candidate | https://aws.amazon.com/about-aws/whats-new/2019/03/amazon-corretto-11-is-now-available-as-a-release-candidate/ AWS Amplify Console Adds Support for Instant CDN Cache Invalidation and Delta Deployments | https://aws.amazon.com/about-aws/whats-new/2019/03/aws-amplify-console-adds-support-for-instant-cdn-cache-invalidation-and-delta-deployments/ AWS CodeCommit Supports VPC Endpoints | https://aws.amazon.com/about-aws/whats-new/2019/03/aws-codecommit-supports-vpc-endpoints/ Automate Releases to the AWS Serverless Application Repository using AWS CodePipeline | https://aws.amazon.com/about-aws/whats-new/2019/03/automate-releases-to-the-aws-serverless-application-repository-using-aws-codepipeline/ Topic || Application Integration New Amazon SNS Console Now Available | https://aws.amazon.com/about-aws/whats-new/2019/03/new-amazon-sns-console-now-available/ Topic || Game Tech Identity and Access Management (IAM) Roles Now Available for Amazon GameLift | https://aws.amazon.com/about-aws/whats-new/2019/03/identity-and-access-management--iam--roles-now-available-for-ama/ Topic || Media Services AWS Elemental MediaLive Adds Support for Encrypted HLS and VPC Inputs | https://aws.amazon.com/about-aws/whats-new/2019/03/aws-elemental-medialive-adds-supports-for-encrypted-hls-and-vpc-inputs/ AWS Elemental MediaLive Now Supports Pausing Channel Delivery on a Schedule | https://aws.amazon.com/about-aws/whats-new/2019/03/aws-elemental-medialive-now-supports-pausing-channel-delivery-on-a-schedule/ AWS Elemental MediaLive Simplifies Sending Live Streams to AWS Elemental MediaPackage | https://aws.amazon.com/about-aws/whats-new/2019/03/aws-elemental-medialive-simplifies-sending-live-streams-to-aws-elemental-mediapackage/ Topic || Management and Governance AWS Systems Manager now supports on-premises instance management for large hybrid environments | https://aws.amazon.com/about-aws/whats-new/2019/03/AWS_Systems_Manager_on-premises_instance_management_for_large_hybrid_environments/ AWS CloudFormation Coverage Updates for AWS RAM, AWS Robomaker, Amazon ApiGateway, and more | https://aws.amazon.com/about-aws/whats-new/2019/02/aws-cloudformation-coverage-updates-for-aws-ram--aws-robomaker--/ whats-new/2019/02/amazon-elasticache-for-redis-adds-support-for-redis-503-and-the-ability-to-change-redis-command-names/ AWS License Manager adds new capabilities to track on premises usage, number of instances, and vCPUs based on Optimize CPU settings | https://aws.amazon.com/about-aws/whats-new/2019/02/NewLicenseManagervCPU/ AWS License Manager enhances support for tracking instances on premises | https://aws.amazon.com/about-aws/whats-new/2019/03/LicenseManagerOnPremises/
Cloud engineering teams at Corteva Agriscience, Agriculture Division of DowDuPont, have a challenge: how to support a global business of research scientists and software developers in building a world-class innovation organization. Modern agriculture produces larger and more varied data types, so their approach must be not only scalable and flexible, but also commit to operational excellence while remaining adoptable. This session will walk through how Corteva Agriscience builds container-based infrastructures with CI/CD pipelines that remove undifferentiated heavy lifting and allow teams to empower developers. Members of the cloud engineering team will discuss problems they face, solutions they implement, and show an example of how they leverage AWS services (AWS CodeCommit, AWS CodePipeline, AWS CloudFormation, AWS Fargate) to deploy a novel machine learning algorithm for scoring genetic markers.
In this session, we detail how Thomson Reuters hosted its critical enterprise .NET Framework application on Amazon ECS using Windows containers. We also dive into the company's decision-making process in choosing the right hosting platform, technology, and so on. We describe the unique custom solution Thomson Reuters developed using AWS CodePipeline, AWS CodeBuild, and Amazon Elastic Container Registry (Amazon ECR) that helped it create an end-to-end CI/CD pipeline for its environment. Complete Title: AWS re:Invent 2018: [REPEAT] Thomson Reuters Shows How It Hosted a .NET App on Amazon ECS Using Windows Containers (CON314-R)
To get the most out of the agility afforded by serverless and containers, it is essential to build CI/CD pipelines that help teams iterate on code and quickly release features. In this talk, we demonstrate how developers can build effective CI/CD release workflows to manage their serverless or containerized deployments on AWS. We cover infrastructure-as-code (IaC) application models, such as AWS Serverless Application Model (AWS SAM) and new imperative IaC tools. We also demonstrate how to set up CI/CD release pipelines with AWS CodePipeline and AWS CodeBuild, and we show you how to automate safer deployments with AWS CodeDeploy.
As financial institutions look to accelerate and scale their use of machine learning, they need to address questions related to specific results, such as the version of the code and the data that lead to a particular inference. The use of disparate and increasingly non-traditional data sources for activities such as targeted marketing, fraud detection, and improved returns is driving a need for structured development of machine learning models. In this session, we'll discuss how we can use a combination of AWS services including Amazon SageMaker, AWS CodeCommit, AWS CodeBuild, and AWS CodePipeline to create a workflow that will help financial institutions meet their requirements and drive business results.
In this talk, Junaid Kapadia, DevOps manager and staff software engineer for Aetion Systems, speaks about the company's journey from a Chef, Jenkins, and EC2-based architecture to a fault-tolerant, highly available, continuously provisioned, and deployed architecture via AWS CodePipeline, AWS CodeBuild, AWS CloudFormation, AWS Systems Manager Parameter Store, and Amazon Elastic Container Service (Amazon ECS). Kapadia also discusses the history of Aetion's original architecture, explains how it transitioned its movement to the new architecture, and shares specific caveats of the purpose of each service and why it was chosen. He also talks about future developments of the overall architecture.
Continuous delivery (CD) enables teams to be more agile and quickens the pace of innovation. Too often, however, teams adopt CD without putting the right safety mechanisms in place. In this talk, we discuss opportunities for you to transform your software release process into a safer one. We explore various DevOps best practices, showcasing sample applications and code with AWS CodePipeline and AWS CodeDeploy. We discuss how to set up delivery pipelines with nonproduction testing stages, failure cases, rollbacks, redundancy, canary testing and blue/green deployments, and monitoring. We discuss continuous delivery practices for deploying to Amazon EC2, AWS Lambda, and containers such as Amazon ECS or AWS Fargate.
In this session, learn how to architect, configure, and deploy an ASP.NET Core microservices application running in containerized AWS Fargate tasks. We cover how to use Amazon DynamoDB for session state and how to use Amazon Cognito for identity management. We also discuss using Amazon ECS for service discovery and AWS CodePipeline to create CI/CD pipelines for each microservice so that each one is individually deployed when an AWS CodeCommit repository is updated. Join us, and learn everything you need to know to start designing and deploying containerized ASP.NET Core applications on AWS.
Simon takes you through a great list of new services, functions and capabilities - hopefully something for everyone! Shownotes: AWS Global Infrastructure: https://aws.amazon.com/about-aws/global-infrastructure/ Amazon EFS Now Supports Provisioned Throughput | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-efs-now-supports-provisioned-throughput/ Amazon EFS Achieves PCI DSS Compliance | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-efs-achieves-pci-dss-compliance/ Amazon EC2 P3 instances, one of the most powerful GPU instances in the cloud, now available in 6 additional regions | https://aws.amazon.com/about-aws/whats-new/2018/08/amazon-ec2-p3-instances-now-available-in-6-additional-regions/ New SBE1 Amazon EC2 instances for AWS Snowball Edge | https://aws.amazon.com/about-aws/whats-new/2018/07/new-sbe1-instances-for-snowball-edge/ Introducing Amazon EC2 R5 Instances, the next generation of memory-optimized instances | https://aws.amazon.com/about-aws/whats-new/2018/07/introducing-amazon-ec2-r5-instances/ Introducing Amazon EC2 z1d Instances with a sustained all core frequency of up to 4.0 GHz | https://aws.amazon.com/about-aws/whats-new/2018/07/introducing-amazon-ec2-z1d-instances/ Amazon EC2 M5d Instances are Now Available in Additional Regions | https://aws.amazon.com/about-aws/whats-new/2018/08/amazon-ec2-m5d-instances-are-now-available-in-additional-regions/ Amazon EC2 C5d Instances are Now Available in Additional Regions | https://aws.amazon.com/about-aws/whats-new/2018/08/amazon-ec2-c5d-instances-are-now-available-in-additional-regions/ Amazon EC2 F1 Instances Adds New Features and Performance Improvements | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-ec2-f1-instances-adds-new-features-and-performance-improvements/ Amazon EC2 Fleet Now Supports Two New Allocation Strategies: On-Demand Prioritized List, and Lowest Price | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-ec2-fleet-now-supports-two-new-allocation-strategies/ Amazon EC2 Nitro System Based Instances Now Support Faster Amazon EBS-Optimized Instance Performance | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-ec2-nitro-system-based-instances-now-support-faster-ebs-optimized-performance/ Access Reserved Instance (RI) Purchase Recommendations for your Amazon Redshift, Amazon ElastiCache, and Amazon Elasticsearch Reservations using AWS Cost Explorer | https://aws.amazon.com/about-aws/whats-new/2018/07/reserved-instance-purchase-recommendations-redshift-elasticache-elasticsearch-reservations/ AWS Systems Manager Run Command Now Streams Output to Amazon CloudWatch Logs | https://aws.amazon.com/about-aws/whats-new/2018/07/aws-systems-manager-run-command-streams-output-to-amazon-cloudwatch-logs/ AWS Systems Manager Automation Conditional Branching for Step Failure | https://aws.amazon.com/about-aws/whats-new/2018/07/aws-systems-manager-automation-conditional-branching-for-step-failure/ Amazon EKS AMI Build Scripts Available on GitHub | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-eks-ami-build-scripts-available-on-github/ Add Scaling to Services You Build on AWS | https://aws.amazon.com/about-aws/whats-new/2018/07/add-scaling-to-services-you-build-on-aws/ Announcing Bring Your Own IP for Amazon Virtual Private Cloud (Preview) | https://aws.amazon.com/about-aws/whats-new/2018/07/announcing-bring-your-own-ip-for-amazon-virtual-private-cloud-preview/ Introducing Amazon Data Lifecycle Manager for EBS Snapshots | https://aws.amazon.com/about-aws/whats-new/2018/07/introducing-amazon-data-lifecycle-manager-for-ebs-snapshots/ Amazon S3 Announces Increased Request Rate Performance | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-s3-announces-increased-request-rate-performance/ Amazon CloudFront announces four new Edge locations, including its first location in Cape Town, South Africa | https://aws.amazon.com/about-aws/whats-new/2018/07/cloudfront-capetown-launch/ Amazon CloudFront announces nine new Edge locations globally across major cities in North America, Europe, and Asia | https://aws.amazon.com/about-aws/whats-new/2018/07/cloudfront-nine-edge-locations-july2018/ Amazon Route 53 Expands Into Africa With New Edge Locations in Cape Town and Johannesburg | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-route-53-expands-into-africa-with-new-edge-locations-in-cape-town-and-johannesburg/ Amazon API Gateway Increases API Limits | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-api-gateway-increases-api-limits/ Amazon API Gateway Usage Plans Now Support Method Level Throttling | https://aws.amazon.com/about-aws/whats-new/2018/07/api-gateway-usage-plans-support-method-level-throttling/ Amazon API Gateway Supports Request/Response Parameters and Status Overrides | https://aws.amazon.com/about-aws/whats-new/2018/07/api-gateway-supports-request-response-parameters-and-status-overrides/ Automate Amazon GuardDuty Provisioning Over Multiple Accounts and Regions with AWS CloudFormation StackSets Integration | https://aws.amazon.com/about-aws/whats-new/2018/07/automate-amazon-guardduty-provisioning-over-multiple-accounts-and-regions-with-aws-cloudformation-stacksets-integration/ AWS Secrets Manager Now Supports AWS PrivateLink | https://aws.amazon.com/about-aws/whats-new/2018/07/aws-secrets-manager-now-supports-aws-privatelink/ AWS Systems Manager Parameter Store integrates with AWS Secrets Manager, and adds labeling for easy configuration updates | https://aws.amazon.com/about-aws/whats-new/2018/07/aws-systems-manager-parameter-store-integrates-with-aws-secrets-manager-and-adds-parameter-version-labeling/ Delegate Permission Management to Employees by Using IAM Permissions Boundaries | https://aws.amazon.com/about-aws/whats-new/2018/07/delegate-permission-management-to-employees-by-using-IAM-permissions-boundaries/ AWS Lambda Supports .NET Core 2.1 | https://aws.amazon.com/about-aws/whats-new/2018/06/lambda-supports-dotnetcore-twopointone/ AWS Glue now provides additional ETL job metrics | https://aws.amazon.com/about-aws/whats-new/2018/07/aws-glue-now-provides-additional-ETL-job-metrics/ AWS Glue now supports reading from Amazon DynamoDB tables | https://aws.amazon.com/about-aws/whats-new/2018/07/aws-glue-now-supports-reading-from-amazon-dynamodb-tables/ The Data Lake Solution Now Transforms and Analyzes Data | https://aws.amazon.com/about-aws/whats-new/2018/07/the-data-lake-solution-now-transforms-and-analyzes-data/ AWS Marketplace Helps Customers Quickly Map Products in Their Existing Software Inventory | https://aws.amazon.com/about-aws/whats-new/2018/07/aws-marketplace-helps-customers-quickly-map-products-in-their-existing-software-inventory/ Amazon SageMaker Now Supports Resource Tags for More Efficient Access Control | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-sagemaker-now-supports-resource-tags-for-more-efficient-access-control/ Amazon SageMaker Supports High Throughput Batch Transform Jobs for Non-Real Time Inferencing | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-sagemaker-supports-high-throughput-batch-transform-jobs-for-non-real-time-inferencing/ Amazon SageMaker Now Supports Pipe Input Mode for Built-In TensorFlow Containers | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-sagemaker-supports-pipe-input-mode-for-built-in-tensorflow-containers/ Amazon SageMaker Now Supports k-Nearest-Neighbor and Object Detection Algorithms | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-sagemaker-supports-knn-and-object-detection-algorithms/ Amazon SageMaker Announces Several Enhancements to Built-in Algorithms and Frameworks | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-sagemaker-announces-enhancements-for-built-in-algorithms-and-frameworks/ AWS Service Catalog Now Supports Service Catalog Resources in CloudFormation | https://aws.amazon.com/about-aws/whats-new/2018/07/aws-service-catalog-now-supports-service-catalog-resources-in-cloudformation/ Kinesis Video Streams now supports HTTP Live Streaming (HLS) to playback live and recorded video from devices | https://aws.amazon.com/about-aws/whats-new/2018/07/kinesis-video-adds-hls-support/ Amazon Polly Now Lets You Define the Maximum Amount of Time for Speech to Complete | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-polly-now-lets-you-define-the-maximum-amount-of-time-for-speech-to-complete/ Amazon Polly Now Supports Input Character Limit of 100K and Stores Output Files in S3 | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-polly-now-supports-input-character-limit-of-100k-and-stores-output-files-in-s3/ Amazon Polly Adds Bilingual Indian English/Hindi Language Support | https://aws.amazon.com/about-aws/whats-new/2018/08/amazon-polly-adds-bilingual-indian-english-hindi-language-support/ Amazon Translate Adds Six New Languages | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-translate-adds-six-new-languages/ Amazon Transcribe Now Lets You Designate Your Own Amazon S3 Buckets to Store Transcription Outputs | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-transcribe-now-lets-you-designate-your-own-amazon-s3-buckets-to-store-transcription-outputs/ Amazon Comprehend Now Supports Syntax Analysis | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-comprehend-now-supports-syntax-analysis/ Amazon Rekognition Increases Accuracy of Text-in-Image | https://aws.amazon.com/about-aws/whats-new/2018/08/amazon-rekognition-increases-accuracy-of-text-in-image/ AWS AppSync releases enhanced no-code GraphQL API builder, HTTP resolvers, and new built-in scalar types | https://aws.amazon.com/about-aws/whats-new/2018/07/aws-appsync-releases-enhanced-capabilities-nocode-graphql/ Introducing the Serverless Bot Framework | https://aws.amazon.com/about-aws/whats-new/2018/04/introducing-the-serverless-bot-framework/ AWS SAM CLI Launches New Commands to Simplify Testing and Debugging Serverless Applications | https://aws.amazon.com/about-aws/whats-new/2018/04/aws-sam-cli-launches-new-commands/ AWS Device Farm Adds Integration with AWS CodePipeline | https://aws.amazon.com/about-aws/whats-new/2018/07/aws-device-farm-adds-integration-with-aws-codepipeline/ Amazon Aurora Serverless Brings Serverless Computing to Relational Databases | https://aws.amazon.com/about-aws/whats-new/2018/08/amazon-aurora-serverless-brings-serverless-computing-to-relational-databases/ Amazon RDS now Provides Best Practice Recommendations | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-rds-recommendations/ Copying Amazon RDS Encrypted Snapshots across Regions now Completes Faster with Less Storage | https://aws.amazon.com/about-aws/whats-new/2018/07/rds-crossregion-incremental-encrypted-snapshots/ Amazon RDS Performance Insights on RDS for PostgreSQL | https://aws.amazon.com/about-aws/whats-new/2018/04/rds-performance-insights-on-rds-for-postgresql/ Performance Insights is Available for Amazon Aurora with MySQL Compatibility | https://aws.amazon.com/about-aws/whats-new/2018/08/performance-insights-is-available-for-amazon-aurora-with-mysql-compatibility/ Amazon DynamoDB Accelerator (DAX) SDK Enhancements | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-dynamodb-accelerator--dax--sdk-enhancements/ Amazon DynamoDB Accelerator (DAX) Adds Support for Encryption at Rest | https://aws.amazon.com/about-aws/whats-new/2018/08/amazon-dynamodb-accelerator--dax--adds-support-for-encryption-at/ Amazon DynamoDB Global Tables Now Available in Three Additional Asia Pacific Regions | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-dynamodb-global-tables-regional-expansion/ Amazon Redshift announces free upgrade for DC1 Reserved Instances to DC2 | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon_redshift_announces_free_upgrade_for_dc1_reserved_instances_to_dc2/ Amazon Redshift now provides customized best practice recommendations with Advisor | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-redshift-now-provides-customized-best-practice-recommendations-with-advisor/ Amazon Redshift now supports current and trailing tracks for release updates | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-redshift-now-supports-current-and-trailing-tracks-for-release-updates/ Amazon Redshift announces new metrics to help optimize cluster performance | https://aws.amazon.com/about-aws/whats-new/2018/07/amazon-redshift-announces-new-metrics-to-help-optimize-cluster-performance/ Amazon Redshift announces support for lateral column alias reference | https://aws.amazon.com/about-aws/whats-new/2018/08/amazon-redshift-announces-support-for-lateral-column-alias-reference/ Amazon Redshift automatically enables short query acceleration | https://aws.amazon.com/about-aws/whats-new/2018/08/amazon-redshift-automatically-enables-short-query-acceleration/ Amazon Redshift announces support for nested data with Redshift Spectrum | https://aws.amazon.com/about-aws/whats-new/2018/08/amazon-redshift-announces-support-for-nested-data-with-redshift-spectrum/ Elastic Load Balancing Announces Support for Redirects and Fixed Responses for Application Load Balancer | https://aws.amazon.com/about-aws/whats-new/2018/07/elastic-load-balancing-announces-support-for-redirects-and-fixed-responses-for-application-load-balancer/ AWS IoT Device Defender - Now Generally Available | https://aws.amazon.com/about-aws/whats-new/2018/08/aws-iot-device-defender-now-generally-available/ AWS IoT Rules Engine Now Supports Step Functions Action | https://aws.amazon.com/about-aws/whats-new/2018/07/aws-iot-rules-engine-now-supports-step-functions-action/ Stream data 65% faster with 5x higher fan-out using new Kinesis Data Streams features | https://aws.amazon.com/about-aws/whats-new/2018/08/stream_data_65_faster_with_5x_higher_fan_out_using_new_kinesis_data_streams_features/ Amazon Elasticsearch Service now supports zero downtime, in-place version upgrades | https://aws.amazon.com/about-aws/whats-new/2018/08/amazon_elasticsearch_service_now_supports_zero_downtime_in-place_version_upgrades/ Announcing the New AWS Free Tier Widget on the AWS Billing Dashboard | https://aws.amazon.com/about-aws/whats-new/2018/07/aws-billing-dashboard-free-tier-widget/ New AWS Public Datasets Available from Allen Institute for Brain Science, NOAA, Hubble Space Telescope, and Others | https://aws.amazon.com/about-aws/whats-new/2018/07/new-aws-public-datasets-available/
Welcome Gabe Hollombe on-board AWS TechChat in this latest episode. Hosts Dean and Gabe start the episode with the latest AWS stats, general availability of Amazon Neptune, Amazon EKS and Amazon Sumerian. They then go into the latest from Amazon Cognito, AWS AppSync, AWS MobileHub, AWS CodeBuild, AWS CodePipeline, Amazon SQS, Application Load Balancer and Amazon SageMaker.
There is a constant tension between empowering teams to be agile through autonomy and enforcing governance policies to maintain regulatory compliance. Hear from Nathan Scott, Senior Consultant at AWS and James Martin, Automation Engineering Manager at 3M on how they have achieved both autonomy and governance through self-service automation tools on AWS. Learn how to avoid pitfalls with building the CI/CD team, right sizing and how to address. This session will also feature a demo from Casey Lee, Chief Architect at Stelligent on the tools used to accomplish this for 3M, including AWS Service Catalog, AWS CloudFormation, AWS CodePipeline and Cloud Custodian, an open source tool for managing AWS accounts.
Continuous delivery (CD) enables teams to be more agile and quickens the pace of innovation. Too often, however, teams adopt CD without putting the right safety mechanisms in place. In this talk, we discuss opportunities for you to transform your software release process into a safer one. We explore various DevOps best practices, showcasing sample applications and code. We discuss how to set up delivery pipelines with nonproduction testing stages, failure cases, rollbacks, machine and Availability Zone redundancy, canary testing and deployments, and monitoring. We'll use AWS Lambda, AWS CloudFormation, AWS CodePipeline, AWS CodeDeploy, and both Amazon CloudWatch alarms and events.
Today, small software teams have the ability to disrupt big markets as more and more businesses start to deliver their products as-a-service. The ability for teams to respond to customers and innovate quickly is their key differentiator. In this session, we will cover how you can begin your DevOps journey by sharing best practices used by the "two pizza" engineering teams at Amazon. We will showcase how you can accelerate developer productivity by implementing continuous Integration and delivery workflows using AWS Developer tools including AWS CodeStar, AWS CodeCommit, AWS CodeBuild, AWS CodePipeline and AWS CodeDeploy. Finally, we will demonstrate how to build an end-to-end CICD pipeline with CodeStar in minutes.
We speak with Jim Rohrer - DevOps Automation Engineer from Stelligent - about automating the provisioning of AWS Workspace environments using AWS CodePipeline, AWS CodeBuild, and Chef.
Simon takes a walk through LOTS of the updates that have been happening for AWS Customers. Shownotes New – Per-Second Billing for EC2 Instances and EBS Volumes - AWS Blog | https://aws.amazon.com/blogs/aws/new-per-second-billing-for-ec2-instances-and-ebs-volumes/ Amazon Virtual Private Cloud (VPC) now allows customers to expand their existing VPCs | https://aws.amazon.com/about-aws/whats-new/2017/08/amazon-virtual-private-cloud-vpc-now-allows-customers-to-expand-their-existing-vpcs/ New – Descriptions for Security Group Rules - AWS Blog | https://aws.amazon.com/blogs/aws/new-descriptions-for-security-group-rules/ New – Stop & Resume Workloads on EC2 Spot Instances - AWS Blog | https://aws.amazon.com/blogs/aws/new-stop-resume-workloads-on-ec2-spot-instances/ Amazon VPC NAT Gateways now support Amazon CloudWatch Monitoring and Resource Tagging | https://aws.amazon.com/about-aws/whats-new/2017/09/amazon-vpc-nat-gateways-now-support-amazon-cloudwatch-monitoring-and-resource-tagging/ AWS VPN Update – Custom PSK, Inside Tunnel IP, and SDK update | https://aws.amazon.com/about-aws/whats-new/2017/10/aws-vpn-update-custom-psk-inside-tunnel-ip-and-sdk-update/ Elasticsearch 5.5 now available on Amazon Elasticsearch Service | https://aws.amazon.com/about-aws/whats-new/2017/09/elasticsearch-5_5-now-available-on-amazon-elasticsearch-service/ Amazon Route 53 Traffic Flow Announces Support For Geoproximity Routing With Traffic Biasing | https://aws.amazon.com/about-aws/whats-new/2017/09/amazon-route-53-traffic-flow-announces-support-for-geoproximity-routing-with-traffic-biasing/ New Network Load Balancer – Effortless Scaling to Millions of Requests per Second - AWS Blog | https://aws.amazon.com/blogs/aws/new-network-load-balancer-effortless-scaling-to-millions-of-requests-per-second/ Now Available – EC2 Instances with 4 TB of Memory - AWS Blog | https://aws.amazon.com/blogs/aws/now-available-ec2-instances-with-4-tb-of-memory/ Use OpenCL Development Environment with Amazon EC2 F1 FPGA Instances to accelerate your C/C++ applications, also F1 instances are now available in US West (Oregon) and EU (Ireland) Regions | https://aws.amazon.com/about-aws/whats-new/2017/09/use-opencl-development-environment-with-amazon-ec2-f1-fpga-instances-to-accelerate-your-c-c-plus-plus-applications-also-f1-instances-are-now-available-in-us-west-oregon-and-eu-ireland-regions/ Announcing: React Native Starter Project with One-Click AWS Deployment and Serverless Infrastructure - AWS Mobile Blog | https://aws.amazon.com/blogs/mobile/announcing-react-native-starter-project-with-one-click-aws-deployment-and-serverless-infrastructure/ Announcing enhancements to the Amazon Lex test console | https://aws.amazon.com/about-aws/whats-new/2017/09/announcing-enhancements-to-the-amazon-lex-test-console/ Announcing support for synonyms and slot value validation on Amazon Lex | https://aws.amazon.com/about-aws/whats-new/2017/08/announcing-support-for-synonyms-and-slot-value-validation-on-amazon-lex/ Now Specify Request Level Attributes with Amazon Lex | https://aws.amazon.com/about-aws/whats-new/2017/09/now-specify-request-level-attributes-with-amazon-lex/ New Amazon Lex Built-in Slot Types for Phone numbers, Speed, and Weight, Available in Preview | https://aws.amazon.com/about-aws/whats-new/2017/09/new-amazon-lex-built-in-slot-types-for-phone-numbers-speed-and-weight-available-in-preview/ Export your Amazon Lex chatbot to the Alexa Skills Kit | https://aws.amazon.com/about-aws/whats-new/2017/09/export-your-amazon-lex-chatbot-to-the-alexa-skills-kit/ Apple Core ML and Keras Support Now Available for Apache MXNet - AWS AI Blog | https://aws.amazon.com/blogs/ai/apple-core-ml-and-keras-support-now-available-for-apache-mxnet/ AWS CodePipeline now provides notifications on pipeline, stage, and action status changes | https://aws.amazon.com/about-aws/whats-new/2017/09/aws-codepipeline-now-provides-notifications-on-pipeline-stage-and-action-status-changes/ Amazon Pinpoint Introduces Two-Way Text Messaging | https://aws.amazon.com/about-aws/whats-new/2017/09/amazon-pinpoint-introduces-two-way-text-messaging/ Amazon Cognito Integrates with Amazon Pinpoint to Add Analytics for User Pools and Enrich Pinpoint Campaigns | https://aws.amazon.com/about-aws/whats-new/2017/09/amazon-cognito-integrates-with-amazon-pinpoint-to-add-analytics-for-user-pools-and-enrich-pinpoint-campaigns/ Amazon Redshift now supports late-binding views referencing Amazon Redshift and Redshift Spectrum external tables | https://aws.amazon.com/about-aws/whats-new/2017/09/amazon-redshift-now-supports-late-binding-views-referencing-amazon-redshift-and-redshift-spectrum-external-tables/ Custom Artifacts on AWS Device Farm - AWS Mobile Blog | https://aws.amazon.com/blogs/mobile/custom-artifacts-on-aws-device-farm/ Amazon Aurora Can Migrate Encrypted Databases from Amazon RDS for MySQL | https://aws.amazon.com/about-aws/whats-new/2017/09/amazon-aurora-can-migrate-encrypted-databases-from-amazon-rds-for-mysql/ Amazon EC2 Systems Manager Adds Raspbian OS and Raspberry Pi Support | https://aws.amazon.com/about-aws/whats-new/2017/09/amazon-ec2-systems-manager-adds-raspbian-os-and-raspberry-pi-support/ AWS Greengrass is available in Asia Pacific (Tokyo) Region. | https://aws.amazon.com/about-aws/whats-new/2017/09/aws-greengrass-is-available-in-asia-pacific-tokyo-region/ AWS CloudTrail Enables Option to Add All Amazon S3 Buckets to Data Events | https://aws.amazon.com/about-aws/whats-new/2017/09/aws-cloudtrail-enables-option-to-add-all-amazon-s3-buckets-to-data-events/ Amazon Kinesis Analytics improves application performance for high volume data streams | https://aws.amazon.com/about-aws/whats-new/2017/09/amazon-kinesis-analytics-improves-application-performance-for-high-volume-data-streams/ New Kinesis Analytics stream processing functions for time series analytics, real time sessionization, and more | https://aws.amazon.com/about-aws/whats-new/2017/09/new-kinesis-analytics-stream-processing-functions-for-time-series-analytics-real-time-sessionization-and-more/ AWS CloudFormation provides Stack Termination Protection | https://aws.amazon.com/about-aws/whats-new/2017/09/aws-cloudformation-provides-stack-termination-protection/
In this episode, we cover recent DevOps in AWS news and Paul Duvall talks about using AWS CodePipeline to Deploy Amazon Alexa Skills. For more info, go to https://stelligent.com/2017/07/25/use-aws-codepipeline-to-deploy-amazon-alexa-skill/
Systems Manager Parameter Store is a managed service (part of AWS EC2 Systems Manager (SSM)) that provides a convenient way to efficiently and securely get and set commonly used configuration data across multiple resources in your software delivery lifecycle. In this episode, we will be focusing on the basic usage of Parameter Store and how to effectively use it as part of a continuous delivery pipeline using AWS CodePipeline along with DevOps in AWS news.
HAPPY NEW YEARDaniel Freij (@DanielFreij) – Senior Performance Engineer and Community Manager at Apica – has been doing hundreds of load tests in his career. 5-10 years ago performance engineers used the “well known” load testing tools such as Load Runner. But things have changed as we have seen both a Shift-Left and a Shift-Right of performance engineering away from the classical performance and load testing teams. Tools became easier, automatable and cloud ready. In this session we discuss these changes that happened in the recent years, what it means for today’s engineering teams and also what might happen in 5-10 years from now. We also want to do a shout out to a performance clinic Daniel and Andi are doing on January 25th 2017 where they walk you through a modern cloud based pipeline using AWS CodePipeline, Jenkins, Apica and Dynatrace. Registration link can be found here: http://bit.ly/onlineperfclinicRelated Link:ZebraTester Community: https://community.zebratester.com/
HAPPY NEW YEARDaniel Freij (@DanielFreij) – Senior Performance Engineer and Community Manager at Apica – has been doing hundreds of load tests in his career. 5-10 years ago performance engineers used the “well known” load testing tools such as Load Runner. But things have changed as we have seen both a Shift-Left and a Shift-Right of performance engineering away from the classical performance and load testing teams. Tools became easier, automatable and cloud ready. In this session we discuss these changes that happened in the recent years, what it means for today’s engineering teams and also what might happen in 5-10 years from now. We also want to do a shout out to a performance clinic Daniel and Andi are doing on January 25th 2017 where they walk you through a modern cloud based pipeline using AWS CodePipeline, Jenkins, Apica and Dynatrace. Registration link can be found here: http://bit.ly/onlineperfclinicRelated Link:ZebraTester Community: https://community.zebratester.com/
Amazon API Gateway and AWS Lambda provide a new way of building applications by removing servers from the picture. But what does the removal of servers mean to tasks like deployment, monitoring, and debugging? How should you set up blue-green deployments or set alarms? Come learn all this and more, including how to use AWS services and tools like AWS CodePipeline, AWS CloudFormation, and Amazon CloudWatch to manage your serverless applications at high quality.
GxP is an acronym that refers to the regulations and guidelines applicable to life sciences organizations that make food and medical products such as drugs, medical devices, and medical software applications. The overall intent of GxP requirements is to ensure that food and medical products are safe for consumers and to ensure the integrity of data used to make product-related safety decisions. The term GxP encompasses a broad range of compliance-related activities such as Good Laboratory Practices (GLP), Good Clinical Practices (GCP), Good Manufacturing Practices (GMP), and others, each of which has product-specific requirements that life sciences organizations must implement based on the 1) type of products they make and 2) country in which their products are sold. When life sciences organizations use computerized systems to perform certain GxP activities, they must ensure that the computerized GxP system is developed, validated, and operated appropriately for the intended use of the system. For this session, co-presented with Merck, services such as Amazon EC2, Amazon CloudWatch Logs, AWS CloudTrail, AWS CodeCommit, Amazon Simple Storage Service (S3), and AWS CodePipeline will be discussed with an emphasis on implementing GxP-compliant systems in the AWS Cloud.
Continuous delivery makes teams more agile and quickens the pace of innovation. Too often, though, teams adopt continuous delivery without putting the right safety mechanisms in place. In this talk, we'll transform a simple but typical software release process into one that is safe. We'll use DevOps techniques like continuous integration, a variety of non-production testing stages, rollbacks, machine redundancy, Availability Zone redundancy, canary deployments, canary tests, and dashboards. We'll use AWS Lambda, AWS CloudFormation, AWS CodePipeline, AWS CodeDeploy, Amazon CloudWatch alarms and dashboards, and AWS Elastic Beanstalk.
To ensure that your application operates in a predictable manner in both your test and production environments, you must vigilantly maintain the configuration of your resources. By leveraging configuration management solutions, Dev and Ops engineers can define the state of their resources across their entire lifecycle. In this session, we will show you how to use AWS OpsWorks, AWS CodeDeploy, and AWS CodePipeline to build a reliable and consistent development pipeline that assures your production workloads behave in a predictable manner.
In this session, we’ll look at the AWS services that customers are using to build and deploy Microsoft-based solutions that use technologies like Windows, .NET, SQL Server, and PowerShell. We’ll start by showing you how to build a Windows-based CI/CD pipeline on AWS using AWS CodeDeploy, AWS CodePipeline, AWS CloudFormation, and PowerShell using an AWS Quick Start. We’ll also cover best practices for how you can create templates that let you automatically deploy ready-to-use Windows products by leveraging services and tools like AWS CloudFormation, PowerShell, and Git. Woot, an online retailer for electronics, will share how it moved from using a complex mix of custom PowerShell code for its DevOps processes to using services like Amazon EC2 Simple Systems Manager (SSM), AWS CodeDeploy, and AWS Directory Service. This migration eliminated the need for complex PowerShell scripts and reduced the operational complexity of performing operational tasks like renaming servers, joining domains, and securely handling keys.
Today’s cutting edge companies have software release cycles measured in days instead of months. This agility is enabled by the DevOps practice of continuous delivery, which automates building, testing, and deploying all code changes. This automation helps you catch bugs sooner and accelerates developer productivity. In this session, we’ll share the processes followed by Amazon engineers and discuss how you can bring them to your company by using AWS CodePipeline and AWS CodeDeploy, services inspired by Amazon's internal developer tools and DevOps culture.
Keeping consistent environments across your development, test, and production systems can be a complex task. Docker containers offer a way to develop and test your application in the same environment in which it runs in production. You can use tools such as the ECS CLI and Docker Compose for local testing of applications; Jenkins and AWS CodePipeline for building and workflow orchestration; Amazon EC2 Container Registry to store your container images; and Amazon EC2 Container Service to manage and scale containers. In this session, you will learn how to build containers into your development workflow and orchestrate container deployments using Amazon ECS. You will hear how Okta runs 30,000 tests per developer commit and releases 10,000 new lines of code each week to production with a CI system based on 100% AWS services. We'll also discuss how Okta uses ECS for parallelized testing in CI and for production microservices in a multi-region, always on cloud service.
We discuss using Docker containers with the AWS EC2 Container Service (ECS), EC2 Container Registry (ECR), AWS CodePipeline and HashiCorp's Consul to create a simple, efficient and cost effective platform for delivering applications and services to users.