Podcasts about vpc

  • 101PODCASTS
  • 243EPISODES
  • 44mAVG DURATION
  • 1WEEKLY EPISODE
  • Nov 21, 2022LATEST

POPULARITY

20152016201720182019202020212022


Best podcasts about vpc

Latest podcast episodes about vpc

The Quack Shack
Best Lanyards in the game VPC

The Quack Shack

Play Episode Play 30 sec Highlight Listen Later Nov 21, 2022 79:32


We sit down with Wes Owner of VPC to talk about what's new and waterfowl. Follow us on Instagram https://www.instagram.com/thequackshackpodcast/?hl=en:Huge thanks to our sponsors be sure to check them out and take advantage of the discount codes!sponsors:www.gatorwaders.com Quackshack10 at check out.www.retayusa.com www.retaynation.comwww.slayercalls.com Quackshack25 at check out.Wesley Vaughan 919 NC (@vpclanyards) • Instagram photos and videos on Instagram and mention The Quack Shack for a discount.Follow us on Instagram https://www.instagram.com/thequackshackpodcast/?hl=en:check out all of our socials here > linktr.ee/Takeemfowlco

Screaming in the Cloud
Snyk and the Complex World of Vulnerability Intelligence with Clinton Herget

Screaming in the Cloud

Play Episode Listen Later Nov 17, 2022 38:39


About ClintonClinton Herget is Field CTO at Snyk, the leader is Developer Security. He focuses on helping Snyk's strategic customers on their journey to DevSecOps maturity. A seasoned technnologist, Cliton spent his 20-year career prior to Snyk as a web software developer, DevOps consultant, cloud solutions architect, and engineering director. Cluinton is passionate about empowering software engineering to do their best work in the chaotic cloud-native world, and is a frequent conference speaker, developer advocate, and technical thought leader.Links Referenced: Snyk: https://snyk.io/ duckbillgroup.com: https://duckbillgroup.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is brought to us in part by our friends at Pinecone. They believe that all anyone really wants is to be understood, and that includes your users. AI models combined with the Pinecone vector database let your applications understand and act on what your users want… without making them spell it out.Make your search application find results by meaning instead of just keywords, your personalization system make picks based on relevance instead of just tags, and your security applications match threats by resemblance instead of just regular expressions. Pinecone provides the cloud infrastructure that makes this easy, fast, and scalable. Thanks to my friends at Pinecone for sponsoring this episode. Visit Pinecone.io to understand more.Corey: This episode is bought to you in part by our friends at Veeam. Do you care about backups? Of course you don't. Nobody cares about backups. Stop lying to yourselves! You care about restores, usually right after you didn't care enough about backups.  If you're tired of the vulnerabilities, costs and slow recoveries when using snapshots to restore your data, assuming you even have them at all living in AWS-land, there is an alternative for you. Check out Veeam, thats V-E-E-A-M for secure, zero-fuss AWS backup that won't leave you high and dry when it's time to restore. Stop taking chances with your data. Talk to Veeam. My thanks to them for sponsoring this ridiculous podcast.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. One of the fun things about establishing traditions is that the first time you do it, you don't really know that that's what's happening. Almost exactly a year ago, I sat down for a previous promoted guest episode much like this one, With Clinton Herget at Snyk—or Synic; however you want to pronounce that. He is apparently a scarecrow of some sorts because when last we spoke, he was a principal solutions engineer, but like any good scarecrow, he was outstanding in his field, and now, as a result, is a Field CTO. Clinton, Thanks for coming back, and let me start by congratulating you on the promotion. Or consoling you depending upon how good or bad it is.Clinton: You know, Corey, a little bit of column A, a little bit of column B. But very glad to be here again, and frankly, I think it's because you insist on mispronouncing Snyk as Synic, and so you get me again.Corey: Yeah, you could add a couple of new letters to it and just call the company [Synack 00:01:27]. Now, it's a hard pivot to a networking company. So, there's always options.Clinton: I acknowledge what you did there, Corey.Corey: I like that quite a bit. I wasn't sure you'd get it.Clinton: I'm a nerd going way, way back, so we'll have to go pretty deep in the stack for you to stump me on some of this stuff.Corey: As we did with the, “I wasn't sure you'd get it.” See that one sailed right past you. And I win. Chalk another one up for me and the networking pun wars. Great, we'll loop back for that later.Clinton: I don't even know where I am right now.Corey: [laugh]. So, let's go back to a question that one would think that I'd already established a year ago, but I have the attention span of basically a goldfish, let's not kid ourselves. So, as I'm visiting the Snyk website, I find that it says different words than it did a year ago, which is generally a sign that is positive; when nothing's been updated including the copyright date, things are going really well or really badly. One wonders. But no, now you're talking about Snyk Cloud, you're talking about several other offerings as well, and my understanding of what it is you folks do no longer appears to be completely accurate. So, let me be direct. What the hell do you folks do over there?Clinton: It's a really great question. Glad you asked me on a year later to answer it. I would say at a very high level, what we do hasn't changed. However, I think the industry has certainly come a long way in the past couple years and our job is to adapt to that Snyk—again, pronounced like a pair of sneakers are sneaking around—it's a developer security platform. So, we focus on enabling the people who build applications—which as of today, means modern applications built in the cloud—to have better visibility, and ultimately a better chance of mitigating the risk that goes into those applications when it matters most, which is actually in their workflow.Now, you're exactly right. Things have certainly expanded in that remit because the job of a software engineer is very different, I think this year than it even was last year, and that's continually evolving over time. As a developer now, I'm doing a lot more than I was doing a few years ago. And one of the things I'm doing is building infrastructure in the cloud, I'm writing YAML files, I'm writing CloudFormation templates to deploy things out to AWS. And what happens in the cloud has a lot to do with the risk to my organization associated with those applications that I'm building.So, I'd love to talk a little bit more about why we decided to make that move, but I don't think that represents a watering down of what we're trying to do at Snyk. I think it recognizes that developer security vision fundamentally can't exist without some understanding of what's happening in the cloud.Corey: One of the things that always scares me is—and sets the spidey sense tingling—is when I see a company who has a product, and I'm familiar—ish—with what they do. And then they take their product name and slap the word cloud at the end, which is almost always codes to, “Okay, so we took the thing that we sold in boxes in data centers, and now we're making a shitty hosted version available because it turns out you rubes will absolutely pay a subscription for it.” Yeah, I don't get the sense that at all is what you're doing. In fact, I don't believe that you're offering a hosted managed service at the moment, are you?Clinton: No, the cloud part, that fundamentally refers to a new product, an offering that looks at the security or potentially the risks being introduced into cloud infrastructure, by now the engineers who were doing it who are writing infrastructure as code. We previously had an infrastructure-as-code security product, and that served alongside our static analysis tool which is Snyk Code, our open-source tool, our container scanner, recognizing that the kinds of vulnerabilities you can potentially introduce in writing cloud infrastructure are not only bad to the organization on their own—I mean, nobody wants to create an S3 bucket that's wide open to the world—but also, those misconfigurations can increase the blast radius of other kinds of vulnerabilities in the stack. So, I think what it does is it recognizes that, as you and I think your listeners well know, Corey, there's no such thing as the cloud, right? The cloud is just a bunch of fancy software designed to abstract away from the fact that you're running stuff on somebody else's computer, right?Corey: Unfortunately, in this case, the fact that you're calling it Snyk Cloud does not mean that you're doing what so many other companies in that same space do it would have led to a really short interview because I have no faith that it's the right path forward, especially for you folks, where it's, “Oh, you want to be secure? You've got to host your stuff on our stuff instead. That's why we called it cloud.” That's the direction that I've seen a lot of folks try and pivot in, and I always find it disastrous. It's, “Yeah, well, at Snyk if we run your code or your shitty applications here in our environment, it's going to be safer than if you run it yourself on something untested like AWS.” And yeah, those stories hold absolutely no water. And may I just say, I'm gratified that's not what you're doing?Clinton: Absolutely not. No, I would say we have no interest in running anyone's applications. We do want to scan them though, right? We do want to give the developers insight into the potential misconfigurations, the risks, the vulnerabilities that you're introducing. What sets Snyk apart, I think, from others in that application security testing space is we focus on the experience of the developer, rather than just being another tool that runs and generates a bunch of PDFs and then throws them back to say, “Here's everything you did wrong.”We want to say to developers, “Here's what you could do better. Here's how that default in a CloudFormation template that leads to your bucket being, you know, wide open on the internet could be changed. Here's the remediation that you could introduce.” And if we do that at the right moment, which is inside that developer workflow, inside the IDE, on their local machine, before that gets deployed, there's a much greater chance that remediation is going to be implemented and it's going to happen much more cheaply, right? Because you no longer have to do the round trip all the way out to the cloud and back.So, the cloud part of it fundamentally means completing that story, recognizing that once things do get deployed, there's a lot of valuable context that's happening out there that a developer can really take advantage of. They can say, “Wait a minute. Not only do I have a Log4Shell vulnerability, right, in one of my open-source dependencies, but that artifact, that application is actually getting deployed to a VPC that has ingress from the internet,” right? So, not only do I have remote code execution in my application, but it's being put in an enclave that actually allows it to be exploited. You can only know that if you're actually looking at what's really happening in the cloud, right?So, not only does Snyk cloud allows us to provide an additional layer of security by looking at what's misconfigured in that cloud environment and help your developers make remediations by saying, “Here's the actual IAC file that caused that infrastructure to come into existence,” but we can also say, here's how that affects the risk of other kinds of vulnerabilities at different layers in the stack, right? Because it's all software; it's all connected. Very rarely does a vulnerability translate one-to-one into risk, right? They're compound because modern software is compound. And I think what developers lack is the tooling that fits into their workflow that understands what it means to be a software engineer and actually helps them make better choices rather than punishing them after the fact for guessing and making bad ones.Corey: That sounds awesome at a very high level. It is very aligned with how executives and decision-makers think about a lot of these things. Let's get down to brass tacks for a second. Assume that I am the type of developer that I am in real life, by which I mean shitty. What am I going to wind up attempting to do that Snyk will flag and, in other words, protect me from myself and warn me that I'm about to commit a dumb?Clinton: First of all, I would say, look, there's no such thing as a non-shitty developer, right? And I built software for 20 years and I decided that's really hard. What's a lot easier is talking about building software for a living. So, that's what I do now. But fundamentally, the reason I'm at Snyk, is I want to help people who are in the kinds of jobs that I had for a very long time, which is to say, you have a tremendous amount of anxiety because you recognize that the success of the organization rests on your shoulders, and you're making hundreds, if not thousands of decisions every day without the right context to understand fully how the results of that decision is going to affect the organization that you work for.So, I think every developer in the world has to deal with this constant cognitive dissonance of saying, “I don't know that this is right, but I have to do it anyway because I need to clear that ticket because that release needs to get into production.” And it becomes really easy to short-sightedly do things like pull an open-source dependency without checking whether it has any CVEs associated with it because that's the version that's easiest to implement with your code that already exists. So, that's one piece. Snyk Open Source, designed to traverse that entire tree of dependencies in open-source all the way down, all the hundreds and thousands of packages that you're pulling in to say, not only, here's a vulnerability that you should really know is going to end up in your application when it's built, but also here's what you can do about it, right? Here's the upgrade you can make, here's the minimum viable change that actually gets you out of this problem, and to do so when it's in the right context, which is in you know, as you're making that decision for the first time, right, inside your developer environment.That also applies to things like container vulnerabilities, right? I have even less visibility into what's happening inside a container than I do inside my application. Because I know, say, I'm using an Ubuntu or a Red Hat base image. I have no idea, what are all the Linux packages that are on it, let alone what are the vulnerabilities associated with them, right? So, being able to detect, I've got a version of OpenSSL 3.0 that has a potentially serious vulnerability associated with it before I've actually deployed that container out into the cloud very much helps me as a developer.Because I'm limiting the rework or the refactoring I would have to do by otherwise assuming I'm making a safe choice or guessing at it, and then only finding out after I've written a bunch more code that relies on that decision, that I have to go back and change it, and then rewrite all of the things that I wrote on top of it, right? So, it's the identifying the layer in the stack where that risk could be introduced, and then also seeing how it's affected by all of those other layers because modern software is inherently complex. And that complexity is what drives both the risk associated with it, and also things like efficiency, which I know your audience is, for good reason, very concerned about.Corey: I'm going to challenge you on aspect of this because on the tin, the way you describe it, it sounds like, “Oh, I already have something that does that. It's the GitHub Dependabot story where it winds up sending me a litany of complaints every week.” And we are talking, if I did nothing other than read this email in that day, that would be a tremendously efficient processing of that entire thing because so much of it is stuff that is ancient and archived, and specific aspects of the vulnerabilities are just not relevant. And you talk about the OpenSSL 3.0 issues that just recently came out.I have no doubt that somewhere in the most recent email I've gotten from that thing, it's buried two-thirds of the way down, like all the complaints like the dishwasher isn't loaded, you forgot to take the trash out, that baby needs a change, the kitchen is on fire, and the vacuuming, and the r—wait, wait. What was that thing about the kitchen? Seems like one of those things is not like the others. And it just gets lost in the noise. Now, I will admit to putting my thumb a little bit on the scale here because I've used Snyk before myself and I know that you don't do that. How do you avoid that trap?Clinton: Great question. And I think really, the key to the story here is, developers need to be able to prioritize, and in order to prioritize effectively, you need to understand the context of what happens to that application after it gets deployed. And so, this is a key part of why getting the data out of the cloud and bringing it back into the code is so important. So, for example, take an OpenSSL vulnerability. Do you have it on a container image you're using, right? So, that's question number one.Question two is, is there actually a way that code can be accessed from the outside? Is it included or is it called? Is the method activated by some other package that you have running on that container? Is that container image actually used in a production deployment? Or does it just go sit in a registry and no one ever touches it?What are the conditions required to make that vulnerability exploitable? You look at something like Spring Shell, for example, yes, you need a certain version of spring-beans in a JAR file somewhere, but you also need to be running a certain version of Tomcat, and you need to be packaging those JARs inside a WAR in a certain way.Corey: Exactly. I have a whole bunch of Lambda functions that provide the pipeline system that I use to build my newsletter every week, and I get screaming concerns about issues in, for example, a version of the markdown parser that I've subverted. Yeah, sure. I get that, on some level, if I were just giving it random untrusted input from the internet and random ad hoc users, but I'm not. It's just me when I write things for that particular Lambda function.And I'm not going to be actively attempting to subvert the thing that I built myself and no one else should have access to. And looking through the details of some of these things, it doesn't even apply to the way that I'm calling the libraries, so it's just noise, for lack of a better term. It is not something that basically ever needs to be adjusted or fixed.Clinton: Exactly. And I think cutting through that noise is so key to creating developer trust in any kind of tool that scanning an asset and providing you what, in theory, are a list of actionable steps, right? I need to be able to understand what is the thing, first of all. There's a lot of tools that do that, right, and we tend to mock them by saying things like, “Oh, it's just another PDF generator. It's just another thousand pages that you're never going to read.”So, getting the information in the right place is a big part of it, but filtering out all of the noise by saying, we looked at not just one layer of the stack, but multiple layers, right? We know that you're using this open-source dependency and we also know that the method that contains the vulnerability is actively called by your application in your first-party code because we ran our static analysis tool against that. Furthermore, we know because we looked at your cloud context, we connected to your AWS API—we're big partners with AWS and very proud of that relationship—but we can tell that there's inbound internet access available to that service, right? So, you start to build a compound case that maybe this is something that should be prioritized, right? Because there's a way into the asset from the outside world, there's a way into the vulnerable functions through the labyrinthine, you know, spaghetti of my code to get there, and the conditions required to exploit it actually exist in the wild.But you can't just run a single tool; you can't just run Dependabot to get that prioritization. You actually have to look at the entire holistic application context, which includes not just your dependencies, but what's happening in the container, what's happening in your first-party, your proprietary code, what's happening in your IAC, and I think most importantly for modern applications, what's actually happening in the cloud once it gets deployed, right? And that's sort of the holy grail of completing that loop to bring the right context back from the cloud into code to understand what change needs to be made, and where, and most importantly why. Because it's a priority that actually translates into organizational risk to get a developer to pay attention, right? I mean, that is the key to I think any security concern is how do you get engineering mindshare and trust that this is actually what you should be paying attention to and not a bunch of rework that doesn't actually make your software more secure?Corey: One of the challenges that I see across the board is that—well, let's back up a bit here. I have in previous episodes talked in some depth about my position that when it comes to the security of various cloud providers, Google is number one, and AWS is number two. Azure is a distant third because it figures out what Crayons tastes the best; I don't know. But the reason is not because of any inherent attribute of their security models, but rather that Google massively simplifies an awful lot of what happens. It automatically assumes that resources in the same project should be able to talk to one another, so I don't have to painstakingly configure that.In AWS-land, all of this must be done explicitly; no one has time for that, so we over-scope permissions massively and never go back and rein them in. It's a configuration vulnerability more than an underlying inherent weakness of the platform. Because complexity is the enemy of security in many respects. If you can't fit it all in your head to reason about it, how can you understand the security ramifications of it? AWS offers a tremendous number of security services. Many of them, when taken in some totality of their pricing, cost more than any breach, they could be expected to prevent. Adding more stuff that adds more complexity in the form of Snyk sounds like it's the exact opposite of what I would want to do. Change my mind.Clinton: I would love to. I would say, fundamentally, I think you and I—and by ‘I,' I mean Snyk and you know, Corey Quinn Enterprises Limited—I think we fundamentally have the same enemy here, right, which is the cyclomatic complexity of software, right, which is how many different pathways do the bits have to travel down to reach the same endpoint, right, the same goal. The more pathways there are, the more risk is introduced into your software, and the more inefficiency is introduced, right? And then I know you'd love to talk about how many different ways is there to run a container on AWS, right? It's either 30 or 400 or eleventy-million.I think you're exactly right that that complexity, it is great for, first of all, selling cloud resources, but also, I think, for innovating, right, for building new kinds of technology on top of that platform. The cost that comes along with that is a lack of visibility. And I think we are just now, as we approach the end of 2022 here, coming to recognize that fundamentally, the complexity of modern software is beyond the ability of a single engineer to understand. And that is really important from a security perspective, from a cost control perspective, especially because software now creates its own infrastructure, right? You can't just now secure the artifact and secure the perimeter that it gets deployed into and say, “I've done my job. Nobody can breach the perimeter and there's no vulnerabilities in the thing because we scanned it and that thing is immutable forever because it's pets, not cattle.”Where I think the complexity story comes in is to recognize like, “Hey, I'm deploying this based on a quickstart or CloudFormation template that is making certain assumptions that make my job easier,” right, in a very similar way that choosing an open-source dependency makes my job easier as a developer because I don't have to write all of that code myself. But what it does mean is I lack the visibility into, well hold on. How many different pathways are there for getting things done inside this dependency? How many other dependencies are brought on board? In the same way that when I create an EKS cluster, for example, from a CloudFormation template, what is it creating in the background? How many VPCs are involved? What are the subnets, right? How are they connected to each other? Where are the potential ingress points?So, I think fundamentally, getting visibility into that complexity is step number one, but understanding those pathways and how they could potentially translate into risk is critically important. But that prioritization has to involve looking at the software holistically and not just individual layers, right? I think we lose when we say, “We ran a static analysis tool and an open-source dependency scanner and a container scanner and a cloud config checker, and they all came up green, therefore the software doesn't have any risks,” right? That ignores the fundamental complexity in that all of these layers are connected together. And from an adversaries perspective, if my job is to go in and exploit software that's hosted in the cloud, I absolutely do not see the application model that way.I see it as it is inherently complex and that's a good thing for me because it means I can rely on the fact that those engineers had tremendous anxiety, we're making a lot of guesses, and crossing their fingers and hoping something would work and not be exploitable by me, right? So, the only way I think we get around that is to recognize that our engineers are critical stakeholders in that security process and you fundamentally lack that visibility if you don't do your scanning until after the fact. If you take that traditional audit-based approach that assumes a very waterfall, legacy approach to building software, and recognize that, hey, we're all on this infinite loop race track now. We're deploying every three-and-a-half seconds, everything's automated, it's all built at scale, but the ability to do that inherently implies all of this additional complexity that ultimately will, you know, end up haunting me, right? If I don't do anything about it, to make my engineer stakeholders in, you know, what actually gets deployed and what risks it brings on board.Corey: This episode is sponsored in part by our friends at Uptycs. Attackers don't think in silos, so why would you have siloed solutions protecting cloud, containers, and laptops distinctly? Meet Uptycs - the first unified solution that prioritizes risk across your modern attack surface—all from a single platform, UI, and data model. Stop by booth 3352 at AWS re:Invent in Las Vegas to see for yourself and visit uptycs.com. That's U-P-T-Y-C-S.com. My thanks to them for sponsoring my ridiculous nonsense.Corey: When I wind up hearing you talk about this—I'm going to divert us a little bit because you're dancing around something that it took me a long time to learn. When I first started fixing AWS bills for a living, I thought that it would be mostly math, by which I mean arithmetic. That's the great secret of cloud economics. It's addition, subtraction, and occasionally multiplication and division. No, turns out it's much more psychology than it is math. You're talking in many aspects about, I guess, what I'd call the psychology of a modern cloud engineer and how they think about these things. It's not a technology problem. It's a people problem, isn't it?Clinton: Oh, absolutely. I think it's the people that create the technology. And I think the longer you persist in what we would call the legacy viewpoint, right, not recognizing what the cloud is—which is fundamentally just software all the way down, right? It is abstraction layers that allow you to ignore the fact that you're running stuff on somebody else's computer—once you recognize that, you realize, oh, if it's all software, then the problems that it introduces are software problems that need software solutions, which means that it must involve activity by the people who write software, right? So, now that you're in that developer world, it unlocks, I think, a lot of potential to say, well, why don't developers tend to trust the security tools they've been provided with, right?I think a lot of it comes down to the question you asked earlier in terms of the noise, the lack of understanding of how those pieces are connected together, or the lack of context, or not even frankly, caring about looking beyond the single-point solution of the problem that solution was designed to solve. But more importantly than that, not recognizing what it's like to build modern software, right, all of the decisions that have to be made on a daily basis with very limited information, right? I might not even understand where that container image I'm building is going in the universe, let alone what's being built on top of it and how much critical customer data is being touched by the database, that that container now has the credentials to access, right? So, I think in order to change anything, we have to back way up and say, problems in the cloud or software problems and we have to treat them that way.Because if we don't if we continue to represent the cloud as some evolution of the old environment where you just have this perimeter that's pre-existing infrastructure that you're deploying things onto, and there's a guy with a neckbeard in the basement who is unplugging cables from a switch and plugging them back in and that's how networking problems are solved, I think you missed the idea that all of these abstraction layers introduced the very complexity that needs to be solved back in the build space. But that requires visibility into what actually happens when it gets deployed. The way I tend to think of it is, there's this firewall in place. Everybody wants to say, you know, we're doing DevOps or we're doing DevSecOps, right? And that's a lie a hundred percent of the time, right? No one is actually, I think, adhering completely to those principles.Corey: That's why one of the core tenets of ClickOps is lying about doing anything in the console.Clinton: Absolutely, right? And that's why shadow IT becomes more and more prevalent the deeper you get into modern development, not less and less prevalent because it's fundamentally hard to recognize the entirety of the potential implications, right, of a decision that you're making. So, it's a lot easier to just go in the console and say, “Okay, I'm going to deploy one EC2 to do this. I'm going to get it right at some point.” And that's why every application that's ever been produced by human hands has a comment in it that says something like, “I don't know why this works but it does. Please don't change it.”And then three years later because that developer has moved on to another job, someone else comes along and looks at that comment and says, “That should really work. I'm going to change it.” And they do and everything fails, and they have to go back and fix it the original way and then add another comment saying, “Hey, this person above me, they were right. Please don't change this line.” I think every engineer listening right now knows exactly where that weak spot is in the applications that they've written and they're terrified of that.And I think any tool that's designed to help developers fundamentally has to get into the mindset, get into the psychology of what that is, like, of not fundamentally being able to understand what those applications are doing all of the time, but having to write code against them anyway, right? And that's what leads to, I think, the fear that you're going to get woken up because your pager is going to go off at 3 a.m. because the building is literally on fire and it's because of code that you wrote. We have to solve that problem and it has to be those people who's psychology we get into to understand, how are you working and how can we make your life better, right? And I really do think it comes with that the noise reduction, the understanding of complexity, and really just being humble and saying, like, “We get that this job is really hard and that the only way it gets better is to begin admitting that to each other.”Corey: I really wish that there were a better way to articulate a lot of these things. This the reason that I started doing a security newsletter; it's because cost and security are deeply aligned in a few ways. One of them is that you care about them a lot right after you failed to care about them sufficiently, but the other is that you've got to build guardrails in such a way that doing the right thing is easier than doing it the wrong way, or you're never going to gain any traction.Clinton: I think that's absolutely right. And you use the key term there, which is guardrails. And I think that's where in their heart of hearts, that's where every security professional wants to be, right? They want to be defining policy, they want to be understanding the risk posture of the organization and nudging it in a better direction, right? They want to be talking up to the board, to the executive team, and creating confidence in that risk posture, rather than talking down or off to the side—depending on how that org chart looks—to the engineers and saying, “Fix this, fix that, and then fix this other thing.” A, B, and C, right?I think the problem is that everyone in a security role or an organization of any size at this point, is doing 90% of the latter and only about 10% of the former, right? They're acting as gatekeepers, not as guardrails. They're not defining policy, they're spending all of their time creating Jira tickets and all of their time tracking down who owns the piece of code that got deployed to this pod on EKS that's throwing all these errors on my console, and how can I get the person to make a decision to actually take an action that stops these notifications from happening, right? So, all they're doing is throwing footballs down the field without knowing if there's a receiver there, right, and I think that takes away from the job that our security analysts really shouldn't be doing, which is creating those guardrails, which is having confidence that the policy they set is readily understood by the developers making decisions, and that's happening in an automated way without them having to create friction by bothering people all the time. I don't think security people want to be [laugh] hated by the development teams that they work with, but they are. And the reason they are is I think, fundamentally, we lack the tooling, we lack—Corey: They are the barrier method.Clinton: Exactly. And we lacked the processes to get the right intelligence in a way that's consumable by the engineers when they're doing their job, and not after the fact, which is typically when the security people have done their jobs.Corey: It's sad but true. I wish that there were a better way to address these things, and yet here we are.Clinton: If only there were better way to address these things.Corey: [laugh].Clinton: Look, I wouldn't be here at Snyk if I didn't think there were a better way, and I wouldn't be coming on shows like yours to talk to the engineering communities, right, people who have walked the walk, right, who have built those Terraform files that contain these misconfigurations, not because they're bad people or because they're lazy, or because they don't do their jobs well, but because they lacked the visibility, they didn't have the understanding that that default is actually insecure. Because how would I know that otherwise, right? I'm building software; I don't see myself as an expert on infrastructure, right, or on Linux packages or on cyclomatic complexity or on any of these other things. I'm just trying to stay in my lane and do my job. It's not my fault that the software has become too complex for me to understand, right?But my management doesn't understand that and so I constantly have white knuckles worrying that, you know, the next breach is going to be my fault. So, I think the way forward really has to be, how do we make our developers stakeholders in the risk being introduced by the software they write to the organization? And that means everything we've been talking about: it means prioritization; it means understanding how the different layers of the stack affect each other, especially the cloud pieces; it means an extensible platform that lets me write code against it to inject my own reasoning, right? The piece that we haven't talked about here is that risk calculation doesn't just involve technical aspects, there's also business intelligence that's involved, right? What are my critical applications, right, what actually causes me to lose significant amounts of money if those services go offline?We at Snyk can't tell that. We can't run a scanner to say these are your crown jewel services that can't ever go down, but you can know that as an organization. So, where we're going with the platform is opening up the extensible process, creating APIs for you to be able to affect that risk triage, right, so that as the creators have guardrails as the security team, you are saying, “Here's how we want our developers to prioritize. Here are all of the factors that go into that decision-making.” And then you can be confident that in their environment, back over in developer-land, when I'm looking at IntelliJ, or, you know, or on my local command line, I am seeing the guardrails that my security team has set for me and I am confident that I'm fixing the right thing, and frankly, I'm grateful because I'm fixing it at the right time and I'm doing it in such a way and with a toolset that actually is helping me fix it rather than just telling me I've done something wrong, right, because everything we do at Snyk focuses on identifying the solution, not necessarily identifying the problem.It's great to know that I've got an unencrypted S3 bucket, but it's a whole lot better if you give me the line of code and tell me exactly where I have to copy and paste it so I can go on to the next thing, rather than spending an hour trying to figure out, you know, where I put that line and what I actually have to change it to, right? I often say that the most valuable currency for a developer, for a software engineer, it's not money, it's not time, it's not compute power or anything like that, it's the right context, right? I actually have to understand what are the implications of the decision that I'm making, and I need that to be in my own environment, not after the fact because that's what creates friction within an organization is when I could have known earlier and I could have known better, but instead, I had to guess I had to write a bunch of code that relies on the thing that was wrong, and now I have to redo it all for no good reason other than the tooling just hadn't adapted to the way modern software is built.Corey: So, one last question before we wind up calling it a day here. We are now heavily into what I will term pre:Invent where we're starting to see a whole bunch of announcements come out of the AWS universe in preparation for what I'm calling Crappy Cloud Hanukkah this year because I'm spending eight nights in Las Vegas. What are you doing these days with AWS specifically? I know I keep seeing your name in conjunction with their announcements, so there's something going on over there.Clinton: Absolutely. No, we're extremely excited about the partnership between Snyk and AWS. Our vulnerability intelligence is utilized as one of the data sources for AWS Inspector, particularly around open-source packages. We're doing a lot of work around things like the code suite, building Snyk into code pipeline, for example, to give developers using that code suite earlier visibility into those vulnerabilities. And really, I think the story kind of expands from there, right?So, we're moving forward with Amazon, recognizing that it is, you know, sort of the de facto. When we say cloud, very often we mean AWS. So, we're going to have a tremendous presence at re:Invent this year, I'm going to be there as well. I think we're actually going to have a bunch of handouts with your face on them is my understanding. So, please stop by the booth; would love to talk to folks, especially because we've now released the Snyk Cloud product and really completed that story. So, anything we can do to talk about how that additional context of the cloud helps engineers because it's all software all the way down, those are absolutely conversations we want to be having.Corey: Excellent. And we will, of course, put links to all of these things in the [show notes 00:35:00] so people can simply click, and there they are. Thank you so much for taking all this time to speak with me. I appreciate it.Clinton: All right. Thank you so much, Corey. Hope to do it again next year.Corey: Clinton Herget, Field CTO at Snyk. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment telling me that I'm being completely unfair to Azure, along with your favorite tasting color of Crayon.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

High Velocity Radio
Cheri’ Benjamin with Village Premier Collection

High Velocity Radio

Play Episode Listen Later Nov 14, 2022


Cheri' Benjamin is the CEO of The Benjamin Group, Village Premier Collection (VPC), and Weinsure Jag companies. She began her journey in Real Estate as a Loan Officer in 2000 and became a real estate agent in 2006. Since then, Cheri' has worked diligently to have VPC recognized as one of the top 5 commercial […] The post Cheri’ Benjamin with Village Premier Collection appeared first on Business RadioX ®.

Packet Pushers - Fat Pipe
Heavy Networking 655: On-Prem VPC Networking With Netris (Sponsored)

Packet Pushers - Fat Pipe

Play Episode Listen Later Nov 11, 2022 45:22


Heavy Networking welcomes sponsor Netris to the show with a special episode for you network nerds who are really getting into automation, infrastructure as code, pipelines, and so on. Netris is all about bringing that public cloud VPC experience to the network you've already got. Imagine being able to consume your existing network with APIs and being able to stand up VLANs, VXLANs, elastic load balancers, firewalls, Internet gateways, and more the same way you do in the cloud, but on premises. The post Heavy Networking 655: On-Prem VPC Networking With Netris (Sponsored) appeared first on Packet Pushers.

Packet Pushers - Heavy Networking
Heavy Networking 655: On-Prem VPC Networking With Netris (Sponsored)

Packet Pushers - Heavy Networking

Play Episode Listen Later Nov 11, 2022 45:22


Heavy Networking welcomes sponsor Netris to the show with a special episode for you network nerds who are really getting into automation, infrastructure as code, pipelines, and so on. Netris is all about bringing that public cloud VPC experience to the network you've already got. Imagine being able to consume your existing network with APIs and being able to stand up VLANs, VXLANs, elastic load balancers, firewalls, Internet gateways, and more the same way you do in the cloud, but on premises. The post Heavy Networking 655: On-Prem VPC Networking With Netris (Sponsored) appeared first on Packet Pushers.

Packet Pushers - Full Podcast Feed
Heavy Networking 655: On-Prem VPC Networking With Netris (Sponsored)

Packet Pushers - Full Podcast Feed

Play Episode Listen Later Nov 11, 2022 45:22


Heavy Networking welcomes sponsor Netris to the show with a special episode for you network nerds who are really getting into automation, infrastructure as code, pipelines, and so on. Netris is all about bringing that public cloud VPC experience to the network you've already got. Imagine being able to consume your existing network with APIs and being able to stand up VLANs, VXLANs, elastic load balancers, firewalls, Internet gateways, and more the same way you do in the cloud, but on premises. The post Heavy Networking 655: On-Prem VPC Networking With Netris (Sponsored) appeared first on Packet Pushers.

Screaming in the Cloud
A Cloud Economist is Born - The AlterNAT Origin Story

Screaming in the Cloud

Play Episode Listen Later Nov 9, 2022 34:45


About BenBen Whaley is a staff software engineer at Chime. Ben is co-author of the UNIX and Linux System Administration Handbook, the de facto standard text on Linux administration, and is the author of two educational videos: Linux Web Operations and Linux System Administration. He is an AWS Community Hero since 2014. Ben has held Red Hat Certified Engineer (RHCE) and Certified Information Systems Security Professional (CISSP) certifications. He earned a B.S. in Computer Science from Univ. of Colorado, Boulder.Links Referenced: Chime Financial: https://www.chime.com/ alternat.cloud: https://alternat.cloud Twitter: https://twitter.com/iamthewhaley LinkedIn: https://www.linkedin.com/in/benwhaley/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Forget everything you know about SSH and try Tailscale. Imagine if you didn't need to manage PKI or rotate SSH keys every time someone leaves. That'd be pretty sweet, wouldn't it? With Tailscale SSH, you can do exactly that. Tailscale gives each server and user device a node key to connect to its VPN, and it uses the same node key to authorize and authenticate SSH.Basically you're SSHing the same way you manage access to your app. What's the benefit here? Built-in key rotation, permissions as code, connectivity between any two devices, reduce latency, and there's a lot more, but there's a time limit here. You can also ask users to reauthenticate for that extra bit of security. Sounds expensive?Nope, I wish it were. Tailscale is completely free for personal use on up to 20 devices. To learn more, visit snark.cloud/tailscale. Again, that's snark.cloud/tailscaleCorey: Welcome to Screaming in the Cloud. I'm Corey Quinn and this is an episode unlike any other that has yet been released on this august podcast. Let's begin by introducing my first-time guest somehow because apparently an invitation got lost in the mail somewhere. Ben Whaley is a staff software engineer at Chime Financial and has been an AWS Community Hero since Andy Jassy was basically in diapers, to my level of understanding. Ben, welcome to the show.Ben: Corey, so good to be here. Thanks for having me on.Corey: I'm embarrassed that you haven't been on the show before. You're one of those people that slipped through the cracks and somehow I was very bad at following up slash hounding you into finally agreeing to be here. But you certainly waited until you had something auspicious to talk about.Ben: Well, you know, I'm the one that really should be embarrassed here. You did extend the invitation and I guess I just didn't feel like I had something to drop. But I think today we have something that will interest most of the listeners without a doubt.Corey: So, folks who have listened to this podcast before, or read my newsletter, or follow me on Twitter, or have shared an elevator with me, or at any point have passed me on the street, have heard me complain about the Managed NAT Gateway and it's egregious data processing fee of four-and-a-half cents per gigabyte. And I have complained about this for small customers because they're in the free tier; why is this thing charging them 32 bucks a month? And I have complained about this on behalf of large customers who are paying the GDP of the nation of Belize in data processing fees as they wind up shoving very large workloads to and fro, which is I think part of the prerequisite requirements for having a data warehouse. And you are no different than the rest of these people who have those challenges, with the singular exception that you have done something about it, and what you have done is so, in retrospect, blindingly obvious that I am embarrassed the rest of us never thought of it.Ben: It's interesting because when you are doing engineering, it's often the simplest solution that is the best. I've seen this repeatedly. And it's a little surprising that it didn't come up before, but I think it's in some way, just a matter of timing. But what we came up with—and is this the right time to get into it, do you want to just kind of name the solution, here?Corey: Oh, by all means. I'm not going to steal your thunder. Please, tell us what you have wrought.Ben: We're calling it AlterNAT and it's an alternative solution to a high-availability NAT solution. As everybody knows, NAT Gateway is sort of the default choice; it certainly is what AWS pushes everybody towards. But there is, in fact, a legacy solution: NAT instances. These were around long before NAT Gateway made an appearance. And like I said they're considered legacy, but with the help of lots of modern AWS innovations and technologies like Lambdas and auto-scaling groups with max instance lifetimes and the latest generation of networking improved or enhanced instances, it turns out that we can maybe not quite get as effective as a NAT Gateway, but we can save a lot of money and skip those data processing charges entirely by having a NAT instance solution with a failover NAT Gateway, which I think is kind of the key point behind the solution. So, are you interested in diving into the technical details?Corey: That is very much the missing piece right there. You're right. What we used to use was NAT instances. That was the thing that we used because we didn't really have another option. And they had an interface in the public subnet where they lived and an interface hanging out in the private subnet, and they had to be configured to wind up passing traffic to and fro.Well, okay, that's great and all but isn't that kind of brittle and dangerous? I basically have a single instance as a single point of failure and these are the days early on when individual instances did not have the level of availability and durability they do now. Yeah, it's kind of awful, but here you go. I mean, the most galling part of the Managed NAT Gateway service is not that it's expensive; it's that it's expensive, but also incredibly good at what it does. You don't have to think about this whole problem anymore, and as of recently, it also supports ipv4 to ipv6 translation as well.It's not that the service is bad. It's that the service is stonkingly expensive, particularly at scale. And everything that we've seen before is either oh, run your own NAT instances or bend your knee and pays your money. And a number of folks have come up with different options where this is ridiculous. Just go ahead and run your own NAT instances.Yeah, but what happens when I have to take it down for maintenance or replace it? It's like, well, I guess you're not going to the internet today. This has the, in hindsight, obvious solution, well, we just—we run the Managed NAT Gateway because the 32 bucks a year in instance-hour charges don't actually matter at any point of scale when you're doing this, but you wind up using that for day in, day out traffic, and the failover mode is simply you'll use the expensive Managed NAT Gateway until the instance is healthy again and then automatically change the route table back and forth.Ben: Yep. That's exactly it. So, the auto-scaling NAT instance solution has been around for a long time well, before even NAT Gateway was released. You could have NAT instances in an auto-scaling group where the size of the group was one, and if the NAT instance failed, it would just replace itself. But this left a period in which you'd have no internet connectivity during that, you know, when the NAT instance was swapped out.So, the solution here is that when auto-scaling terminates an instance, it fails over the route table to a standby NAT Gateway, rerouting the traffic. So, there's never a point at which there's no internet connectivity, right? The NAT instance is running, processing traffic, gets terminated after a certain period of time, configurable, 14 days, 30 days, whatever makes sense for your security strategy could be never, right? You could choose that you want to have your own maintenance window in which to do it.Corey: And let's face it, this thing is more or less sitting there as a network traffic router, for lack of a better term. There is no need to ever log into the thing and make changes to it until and unless there's a vulnerability that you can exploit via somehow just talking to the TCP stack when nothing's actually listening on the host.Ben: You know, you can run your own AMI that has been pared down to almost nothing, and that instance doesn't do much. It's using just a Linux kernel to sit on two networks and pass traffic back and forth. It has a translation table that kind of keeps track of the state of connections and so you don't need to have any service running. To manage the system, we have SSM so you can use Session Manager to log in, but frankly, you can just disable that. You almost never even need to get a shell. And that is, in fact, an option we have in the solution is to disable SSM entirely.Corey: One of the things I love about this approach is that it is turnkey. You throw this thing in there and it's good to go. And in the event that the instance becomes unhealthy, great, it fails traffic over to the Managed NAT Gateway while it terminates the old node and replaces it with a healthy one and then fails traffic back. Now, I do need to ask, what is the story of network connections during that failover and failback scenario?Ben: Right, that's the primary drawback, I would say, of the solution is that any established TCP connections that are on the NAT instance at the time of a route change will be lost. So, say you have—Corey: TCP now terminates on the floor.Ben: Pretty much. The connections are dropped. If you have an open SSH connection from a host in the private network to a host on the internet and the instance fails over to the NAT Gateway, the NAT Gateway doesn't have the translation table that the NAT instance had. And not to mention, the public IP address also changes because you have an Elastic IP assigned to the NAT instance, a different Elastic IP assigned to the NAT Gateway, and so because that upstream IP is different, the remote host is, like, tracking the wrong IP. So, those connections, they're going to be lost.So, there are some use cases where this may not be suitable. We do have some ideas on how you might mitigate that, for example, with the use of a maintenance window to schedule the replacement, replaced less often so it doesn't have to affect your workflow as much, but frankly, for many use cases, my belief is that it's actually fine. In our use case at Chime, we found that it's completely fine and we didn't actually experience any errors or failures. But there might be some use cases that are more sensitive or less resilient to failure in the first place.Corey: I would also point out that a lot of how software is going to behave is going to be a reflection of the era in which it was moved to cloud. Back in the early days of EC2, you had no real sense of reliability around any individual instance, so everything was written in a very defensive manner. These days, with instances automatically being able to flow among different hardware so we don't get instance interrupt notifications the way we once did on a semi-constant basis, it more or less has become what presents is bulletproof, so a lot of people are writing software that's a bit more brittle. But it's always been a best practice that when a connection fails okay, what happens at failure? Do you just give up and throw your hands in the air and shriek for help or do you attempt to retry a few times, ideally backing off exponentially?In this scenario, those retries will work. So, it's a question of how well have you built your software. Okay, let's say that you made the worst decisions imaginable, and okay, if that connection dies, the entire workload dies. Okay, you have the option to refactor it to be a little bit better behaved, or alternately, you can keep paying the Manage NAT Gateway tax of four-and-a-half cents per gigabyte in perpetuity forever. I'm not going to tell you what decision to make, but I know which one I'm making.Ben: Yeah, exactly. The cost savings potential of it far outweighs the potential maintenance troubles, I guess, that you could encounter. But the fact is, if you're relying on Managed NAT Gateway and paying the price for doing so, it's not as if there's no chance for connection failure. NAT Gateway could also fail. I will admit that I think it's an extremely robust and resilient solution. I've been really impressed with it, especially so after having worked on this project, but it doesn't mean it can't fail.And beyond that, upstream of the NAT Gateway, something could in fact go wrong. Like, internet connections are unreliable, kind of by design. So, if your system is not resilient to connection failures, like, there's a problem to solve there anyway; you're kind of relying on hope. So, it's a kind of a forcing function in some ways to build architectural best practices, in my view.Corey: I can't stress enough that I have zero problem with the capabilities and the stability of the Managed NAT Gateway solution. My complaints about it start and stop entirely with the price. Back when you first showed me the blog post that is releasing at the same time as this podcast—and you can visit that at alternat.cloud—you sent me an early draft of this and what I loved the most was that your math was off because of a not complete understanding of the gloriousness that is just how egregious the NAT Gateway charges are.Your initial analysis said, “All right, if you're throwing half a terabyte out to the internet, this has the potential of cutting the bill by”—I think it was $10,000 or something like that. It's, “Oh no, no. It has the potential to cut the bill by an entire twenty-two-and-a-half thousand dollars.” Because this processing fee does not replace any egress fees whatsoever. It's purely additive. If you forget to have a free S3 Gateway endpoint in a private subnet, every time you put something into or take something out of S3, you're paying four-and-a-half cents per gigabyte on that, despite the fact there's no internet transitory work, it's not crossing availability zones. It is simply a four-and-a-half cent fee to retrieve something that has only cost you—at most—2.3 cents per month to store in the first place. Flip that switch, that becomes completely free.Ben: Yeah. I'm not embarrassed at all to talk about the lack of education I had around this topic. The fact is I'm an engineer primarily and I came across the cost stuff because it kind of seemed like a problem that needed to be solved within my organization. And if you don't mind, I might just linger on this point and kind of think back a few months. I looked at the AWS bill and I saw this egregious ‘EC2 Other' category. It was taking up the majority of our bill. Like, the single biggest line item was EC2 Other. And I was like, “What could this be?”Corey: I want to wind up flagging that just because that bears repeating because I often get people pushing back of, “Well, how bad—it's one Managed NAT Gateway. How much could it possibly cost? $10?” No, it is the majority of your monthly bill. I cannot stress that enough.And that's not because the people who work there are doing anything that they should not be doing or didn't understand all the nuances of this. It's because for the security posture that is required for what you do—you are at Chime Financial, let's be clear here—putting everything in public subnets was not really a possibility for you folks.Ben: Yeah. And not only that but there are plenty of services that have to be on private subnets. For example, AWS Glue services must run in private VPC subnets if you want them to be able to talk to other systems in your VPC; like, they cannot live in public subnet. So essentially, if you want to talk to the internet from those jobs, you're forced into some kind of NAT solution. So, I dug into the EC2 Other category and I started trying to figure out what was going on there.There's no way—natively—to look at what traffic is transiting the NAT Gateway. There's not an interface that shows you what's going on, what's the biggest talkers over that network. Instead, you have to have flow logs enabled and have to parse those flow logs. So, I dug into that.Corey: Well, you're missing a step first because in a lot of environments, people have more than one of these things, so you get to first do the scavenger hunt of, okay, I have a whole bunch of Managed NAT Gateways and first I need to go diving into CloudWatch metrics and figure out which are the heavy talkers. Is usually one or two followed by a whole bunch of small stuff, but not always, so figuring out which VPC you're even talking about is a necessary prerequisite.Ben: Yeah, exactly. The data around it is almost missing entirely. Once you come to the conclusion that it is a particular NAT Gateway—like, that's a set of problems to solve on its own—but first, you have to go to the flow logs, you have to figure out what are the biggest upstream IPs that it's talking to. Once you have the IP, it still isn't apparent what that host is. In our case, we had all sorts of outside parties that we were talking to a lot and it's a matter of sorting by volume and figuring out well, this IP, what is the reverse IP? Who is potentially the host there?I actually had some wrong answers at first. I set up VPC endpoints to S3 and DynamoDB and SQS because those were some top talkers and that was a nice way to gain some security and some resilience and save some money. And then I found, well, Datadog; that's another top talker for us, so I ended up creating a nice private link to Datadog, which they offer for free, by the way, which is more than I can say for some other vendors. But then I found some outside parties, there wasn't a nice private link solution available to us, and yet, it was by far the largest volume. So, that's what kind of started me down this track is analyzing the NAT Gateway myself by looking at VPC flow logs. Like, it's shocking that there isn't a better way to find that traffic.Corey: It's worse than that because VPC flow logs tell you where the traffic is going and in what volumes, sure, on an IP address and port basis, but okay, now you have a Kubernetes cluster that spans two availability zones. Okay, great. What is actually passing through that? So, you have one big application that just seems awfully chatty, you have multiple workloads running on the thing. What's the expensive thing talking back and forth? The only way that you can reliably get the answer to that I found is to talk to people about what those workloads are actually doing, and failing that you're going code spelunking.Ben: Yep. You're exactly right about that. In our case, it ended up being apparent because we have a set of subnets where only one particular project runs. And when I saw the source IP, I could immediately figure that part out. But if it's a K8s cluster in the private subnets, yeah, how are you going to find it out? You're going to have to ask everybody that has workloads running there.Corey: And we're talking about in some cases, millions of dollars a month. Yeah, it starts to feel a little bit predatory as far as how it's priced and the amount of work you have to put in to track this stuff down. I've done this a handful of times myself, and it's always painful unless you discover something pretty early on, like, oh, it's talking to S3 because that's pretty obvious when you see that. It's, yeah, flip switch and this entire engagement just paid for itself a hundred times over. Now, let's see what else we can discover.That is always one of those fun moments because, first, customers are super grateful to learn that, oh, my God, I flipped that switch. And I'm saving a whole bunch of money. Because it starts with gratitude. “Thank you so much. This is great.” And it doesn't take a whole lot of time for that to alchemize into anger of, “Wait. You mean, I've been being ridden like a pony for this long and no one bothered to mention that if I click a button, this whole thing just goes away?”And when you mention this to your AWS account team, like, they're solicitous, but they either have to present as, “I didn't know that existed either,” which is not a good look, or, “Yeah, you caught us,” which is worse. There's no positive story on this. It just feels like a tax on not knowing trivia about AWS. I think that's what really winds me up about it so much.Ben: Yeah, I think you're right on about that as well. My misunderstanding about the NAT pricing was data processing is additive to data transfer. I expected when I replaced NAT Gateway with NAT instance, that I would be substituting data transfer costs for NAT Gateway costs, NAT Gateway data processing costs. But in fact, NAT Gateway incurs both data processing and data transfer. NAT instances only incur data transfer costs. And so, this is a big difference between the two solutions.Not only that, but if you're in the same region, if you're egressing out of your say us-east-1 region and talking to another hosted service also within us-east-1—never leaving the AWS network—you don't actually even incur data transfer costs. So, if you're using a NAT Gateway, you're paying data processing.Corey: To be clear you do, but it is cross-AZ in most cases billed at one penny egressing, and on the other side, that hosted service generally pays one penny ingressing as well. Don't feel bad about that one. That was extraordinarily unclear and the only reason I know the answer to that is that I got tired of getting stonewalled by people that later turned out didn't know the answer, so I ran a series of experiments designed explicitly to find this out.Ben: Right. As opposed to the five cents to nine cents that is data transfer to the internet. Which, add that to data processing on a NAT Gateway and you're paying between thirteen-and-a-half cents to nine-and-a-half cents for every gigabyte egressed. And this is a phenomenal cost. And at any kind of volume, if you're doing terabytes to petabytes, this becomes a significant portion of your bill. And this is why people hate the NAT Gateway so much.Corey: I am going to short-circuit an angry comment I can already see coming on this where people are going to say, “Well, yes. But it's a multi-petabyte scale. Nobody's paying on-demand retail price.” And they're right. Most people who are transmitting that kind of data, have a specific discount rate applied to what they're doing that varies depending upon usage and use case.Sure, great. But I'm more concerned with the people who are sitting around dreaming up ideas for a company where I want to wind up doing some sort of streaming service. I talked to one of those companies very early on in my tenure as a consultant around the billing piece and they wanted me to check their napkin math because they thought that at their numbers when they wound up scaling up, if their projections were right, that they were going to be spending $65,000 a minute, and what did they not understand? And the answer was, well, you didn't understand this other thing, so it's going to be more than that, but no, you're directionally correct. So, that idea that started off on a napkin, of course, they didn't build it on top of AWS; they went elsewhere.And last time I checked, they'd raised well over a quarter-billion dollars in funding. So, that's a business that AWS would love to have on a variety of different levels, but they're never going to even be considered because by the time someone is at scale, they either have built this somewhere else or they went broke trying.Ben: Yep, absolutely. And we might just make the point there that while you can get discounts on data transfer, you really can't—or it's very rare—to get discounts on data processing for the NAT Gateway. So, any kind of savings you can get on data transfer would apply to a NAT instance solution, you know, saving you four-and-a-half cents per gigabyte inbound and outbound over the NAT Gateway equivalent solution. So, you're paying a lot for the benefit of a fully-managed service there. Very robust, nicely engineered fully-managed service as we've already acknowledged, but an extremely expensive solution for what it is, which is really just a proxy in the end. It doesn't add any value to you.Corey: The only way to make that more expensive would be to route it through something like Splunk or whatnot. And Splunk does an awful lot for what they charge per gigabyte, but it just feels like it's rent-seeking in some of the worst ways possible. And what I love about this is that you've solved the problem in a way that is open-source, you have already released it in Terraform code. I think one of the first to-dos on this for someone is going to be, okay now also make it CloudFormation and also make it CDK so you can drop it in however you want.And anyone can use this. I think the biggest mistake people might make in glancing at this is well, I'm looking at the hourly charge for the NAT Gateways and that's 32-and-a-half bucks a month and the instances that you recommend are hundreds of dollars a month for the big network-optimized stuff. Yeah, if you care about the hourly rate of either of those two things, this is not for you. That is not the problem that it solves. If you're an independent learner annoyed about the $30 charge you got for a Managed NAT Gateway, don't do this. This will only add to your billing concerns.Where it really shines is once you're at, I would say probably about ten terabytes a month, give or take, in Managed NAT Gateway data processing is where it starts to consider this. The breakeven is around six or so but there is value to not having to think about things. Once you get to that level of spend, though it's worth devoting a little bit of infrastructure time to something like this.Ben: Yeah, that's effectively correct. The total cost of running the solution, like, all-in, there's eight Elastic IPs, four NAT Gateways, if you're—say you're four zones; could be less if you're in fewer zones—like, n NAT Gateways, n NAT instances, depending on how many zones you're in, and I think that's about it. And I said right in the documentation, if any of those baseline fees are a material number for your use case, then this is probably not the right solution. Because we're talking about saving thousands of dollars. Any of these small numbers for NAT Gateway hourly costs, NAT instance hourly costs, that shouldn't be a factor, basically.Corey: Yeah, it's like when I used to worry about costing my customers a few tens of dollars in Cost Explorer or CloudWatch or request fees against S3 for their Cost and Usage Reports. It's yeah, that does actually have a cost, there's no real way around it, but look at the savings they're realizing by going through that. Yeah, they're not going to come back and complaining about their five-figure consulting engagement costing an additional $25 in AWS charges and then lowering it by a third. So, there's definitely a difference as far as how those things tend to be perceived. But it's easy to miss the big stuff when chasing after the little stuff like that.This is part of the problem I have with an awful lot of cost tooling out there. They completely ignore cost components like this and focus only on the things that are easy to query via API, of, oh, we're going to cost-optimize your Kubernetes cluster when they think about compute and RAM. And, okay, that's great, but you're completely ignoring all the data transfer because there's still no great way to get at that programmatically. And it really is missing the forest for the trees.Ben: I think this is key to any cost reduction project or program that you're undertaking. When you look at a bill, look for the biggest spend items first and work your way down from there, just because of the impact you can have. And that's exactly what I did in this project. I saw that ‘EC2 Other' slash NAT Gateway was the big item and I started brainstorming ways that we could go about addressing that. And now I have my next targets in mind now that we've reduced this cost to effectively… nothing, extremely low compared to what it was, we have other new line items on our bill that we can start optimizing. But in any cost project, start with the big things.Corey: You have come a long way around to answer a question I get asked a lot, which is, “How do I become a cloud economist?” And my answer is, you don't. It's something that happens to you. And it appears to be happening to you, too. My favorite part about the solution that you built, incidentally, is that it is being released under the auspices of your employer, Chime Financial, which is immune to being acquired by Amazon just to kill this thing and shut it up.Because Amazon already has something shitty called Chime. They don't need to wind up launching something else or acquiring something else and ruining it because they have a Slack competitor of sorts called Amazon Chime. There's no way they could acquire you [unintelligible 00:27:45] going to get lost in the hallways.Ben: Well, I have confidence that Chime will be a good steward of the project. Chime's goal and mission as a company is to help everyone achieve financial peace of mind and we take that really seriously. We even apply it to ourselves and that was kind of the impetus behind developing this in the first place. You mentioned earlier we have Terraform support already and you're exactly right. I'd love to have CDK, CloudFormation, Pulumi supports, and other kinds of contributions are more than welcome from the community.So, if anybody feels like participating, if they see a feature that's missing, let's make this project the best that it can be. I suspect we can save many companies, hundreds of thousands or millions of dollars. And this really feels like the right direction to go in.Corey: This is easily a multi-billion dollar savings opportunity, globally.Ben: That's huge. I would be flabbergasted if that was the outcome of this.Corey: The hardest part is reaching these people and getting them on board with the idea of handling this. And again, I think there's a lot of opportunity for the project to evolve in the sense of different settings depending upon risk tolerance. I can easily see a scenario where in the event of a disruption to the NAT instance, it fails over to the Managed NAT Gateway, but fail back becomes manual so you don't have a flapping route table back and forth or a [hold 00:29:05] downtime or something like that. Because again, in that scenario, the failure mode is just well, you're paying four-and-a-half cents per gigabyte for a while until you wind up figuring out what's going on as opposed to the failure mode of you wind up disrupting connections on an ongoing basis, and for some workloads, that's not tenable. This is absolutely, for the common case, the right path forward.Ben: Absolutely. I think it's an enterprise-grade solution and the more knobs and dials that we add to tweak to make it more robust or adaptable to different kinds of use cases, the best outcome here would actually be that the entire solution becomes irrelevant because AWS fixes the NAT Gateway pricing. If that happens, I will consider the project a great success.Corey: I will be doing backflips like you wouldn't believe. I would sing their praises day in, day out. I'm not saying reduce it to nothing, even. I'm not saying it adds no value. I would change the way that it's priced because honestly, the fact that I can run an EC2 instance and be charged $0 on a per-gigabyte basis, yeah, I would pay a premium on an hourly charge based upon traffic volumes, but don't meter per gigabyte. That's where it breaks down.Ben: Absolutely. And why is it additive to data transfer, also? Like, I remember first starting to use VPC when it was launched and reading about the NAT instance requirement and thinking, “Wait a minute. I have to pay this extra management and hourly fee just so my private hosts could reach the internet? That seems kind of janky.”And Amazon established a norm here because Azure and GCP both have their own equivalent of this now. This is a business choice. This is not a technical choice. They could just run this under the hood and not charge anybody for it or build in the cost and it wouldn't be this thing we have to think about.Corey: I almost hate to say it, but Oracle Cloud does, for free.Ben: Do they?Corey: It can be done. This is a business decision. It is not a technical capability issue where well, it does incur cost to run these things. I understand that and I'm not asking for things for free. I very rarely say that this is overpriced when I'm talking about AWS billing issues. I'm talking about it being unpredictable, I'm talking about it being impossible to see in advance, but the fact that it costs too much money is rarely my complaint. In this case, it costs too much money. Make it cost less.Ben: If I'm not mistaken. GCPs equivalent solution is the exact same price. It's also four-and-a-half cents per gigabyte. So, that shows you that there's business games being played here. Like, Amazon could get ahead and do right by the customer by dropping this to a much more reasonable price.Corey: I really want to thank you both for taking the time to speak with me and building this glorious, glorious thing. Where can we find it? And where can we find you?Ben: alternat.cloud is going to be the place to visit. It's on Chime's GitHub, which will be released by the time this podcast comes out. As for me, if you want to connect, I'm on Twitter. @iamthewhaley is my handle. And of course, I'm on LinkedIn.Corey: Links to all of that will be in the podcast notes. Ben, thank you so much for your time and your hard work.Ben: This was fun. Thanks, Corey.Corey: Ben Whaley, staff software engineer at Chime Financial, and AWS Community Hero. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry rant of a comment that I will charge you not only four-and-a-half cents per word to read, but four-and-a-half cents to reply because I am experimenting myself with being a rent-seeking schmuck.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Yalla To The Cloud
Episode 119: Getting ready for the Re:Invent

Yalla To The Cloud

Play Episode Listen Later Nov 8, 2022 18:42


בפינה זו, נגיש לכם מידע על העבודה היומיומית בסביבת ענן מנקודת המבט שלנו.דוברי הפרק: בועז זינימן ואבי קינן. בפרק הקודם, דיברנו על VPC Reachability Analyzer. הסברנו מהו VPC, הצגנו דמו על השירות עצמו והבנו למה הוא מיועד ואילו בעיות הוא פותר. בנוסף, פירטנו על אפשרויות האוטומציה שניתן לבצע כדי לשדרג את הניטור וה- Compliance של התשתיות ענן שלנו. בפרק זה, נדבר על מה חשוב לדעת לפני שטסים ל-Re:Invent. איך לבנות תוכנית יעילה ועל מה לשים דגש למי שמבקר באירוע מחשוב הענן הגדול בעולם בפעם הראשונה. רוצים להתעדכן בתכנים נוספים בנושאי ענן וטכנולוגיות מתקדמות? הירשמו עכשיו לניוזלטר שלנו ותמיד תישארו בעניינים. להרשמה: https://www.israelclouds.com/newslettersignup

Yalla To The Cloud
Episode 118: VPC Reachability Analyzer

Yalla To The Cloud

Play Episode Listen Later Nov 3, 2022 12:48


בפינה זו, נגיש לכם מידע על העבודה היומיומית בסביבת ענן מנקודת המבט שלנו.דוברי הפרק: אריאל מונפו ודביר מזרחי. בפרק הקודם, הצגנו דמו בלייב על: * איך מתחילים לבנות DIY למוניטורינג לארגון שלכם. * איך להסתכל על תהליך הניטור הראשוני במלואו, שמורכב מהסתכלות על פעילות בודדת. * איזה ניטור יש על אותה פעילות (במקרה שלנו בדמו, שרת EC2). * בחירת המדדים שמעניינים אותנו מהמטריקות ניטור הקיימות (במקרה שלנו בדמו, אחוז שימוש המעבד). * בניית קוד פשוט שמוציא את המידע הזה עבור אותה הפעילות. בפרק זה, נדבר על VPC Reachability Analyzer. נסביר מהו VPC, נציג דמו על השירות עצמו ונבין למה הוא מיועד ואילו בעיות הוא פותר. בנוסף, נפרט על אפשרויות האוטומציה שניתן לבצע כדי לשדרג את הניטור וה- Compliance של התשתיות ענן שלנו. רוצים להתעדכן בתכנים נוספים בנושאי ענן וטכנולוגיות מתקדמות? הירשמו עכשיו לניוזלטר שלנו ותמיד תישארו בעניינים. להרשמה: https://www.israelclouds.com/newslettersignup

Armed American Radio
10-31-2022 HR 1 Stacey Abrams hates cops and plays race card, Glocks are now full auto?

Armed American Radio

Play Episode Listen Later Oct 31, 2022 53:53


Stacey Abrams hates cops and plays race card, Glocks are now full auto? Yup, according to VPC.

The Cloud Pod
183: The Cloud Pod competes for the Google Cloud Fly Cup

The Cloud Pod

Play Episode Listen Later Sep 30, 2022 45:05


On The Cloud Pod this week, AWS Enterprise Support adds incident detection and response, the announcement of Google Cloud Spanner, and Oracle expands to Spain. Thank you to our sponsor, Foghorn Consulting, which provides top notch cloud and DevOps engineers to the world's most innovative companies. Initiatives stalled because you're having trouble hiring? Foghorn can be burning down your DevOps and Cloud backlogs as soon as next week. Episode Highlights ⏰ AWS Enterprise Support adds incident detection and response ⏰ You can now get a 90-day free trial of Google Cloud Spanner ⏰ Oracle opens its newest cloud infrastructure region in Spain Top Quote

Vitalpoint Church
Vitality - Reach

Vitalpoint Church

Play Episode Listen Later Sep 11, 2022 27:12


In Luke 15, Jesus tells a three part parable that creates a narrative that portrays the focus of Jesus heart for those that have not yet been found. The lost sheep, the lost coin and the lost son. Vitalpoint has a mission to reach people and help them become fully devoted followers of Jesus. One of our core values is invest and invite in those around us that have not yet encountered Jesus. We will cast a vision for each VPC person to live in a way that helps reach more people, which means we will give heaven an excuse to party! Speaker: Ron Baker

How to Scale Commercial Real Estate
How to Position Yourself to Win in a Recession

How to Scale Commercial Real Estate

Play Episode Listen Later Sep 2, 2022 21:28


Despite the uncertainty, there are a lot of ways to create wealth during a recession.   Paul Neal, founder and Principal Funding Strategist at Vantage Point Commercial Capital, joins us to list action items you can follow to survive and thrive in a downturn. As a serial entrepreneur, Paul has owned six businesses in 30 years, and today, he brings his wealth of experience to discuss his thoughts on the market and how to find opportunities in real estate and in business.   [00:01 - 03:16] From Engineering to Network Marketing Paul on how he got into a network marketing business with his brother-in-law for 10 years Having a thick skin and being able to work together with his wife are things he learned from network marketing   [03:17 - 19:24] Strategies to Set Yourself up for Success in an Economic Slowdown Now back in real estate finance, Paul shares his view about the economy 80-20 your expenses and your business, shore up your balance sheet, and focus on cash  Protect yourself with liquidity and get in the habit of saving Align yourself with people who are ahead of you If you can come up with ideas, the capital will show up Paul sees a lot of demand in the single-family and new construction space The service industry will remain strong and there are many business acquisition and expansion opportunities What business acquisitions do Paul and his team avoid? Lessons from his career: think long term, you can't do everything by yourself, and it pays to have a mentor   [19:25 - 21:27] Closing Segment Reach out to Paul!  Links Below Final Words Tweetable Quotes   “The Pareto Principle, 20% of your expenses you probably need and 80% of them are probably fluff.” - Paul Neal “The key is the idea, not the capital.”  - Paul Neal “Whenever I've approached a business, I've always tried to go along with it and look beyond day one and have always been willing to put money in, even when I didn't have it.” - Paul Neal -----------------------------------------------------------------------------   Connect with Paul at the Vantage Point Commercial Capital's website and head over to vpc.capital/podcast-scaleCRE to get their free guide: Questions You MUST Ask Before Seeking Funding.   Connect with me:   I love helping others place money outside of traditional investments that both diversify a strategy and provide solid predictable returns.     Facebook   LinkedIn   Like, subscribe, and leave us a review on Apple Podcasts, Spotify, Google Podcasts, or whatever platform you listen on.  Thank you for tuning in!   Email me → sam@brickeninvestmentgroup.com Want to read the full show notes of the episode? Check it out below:   [00:00:00] Paul Neal: I would say, you know, seriously, look at, and you can do this on the business side, on the personal side, you know, take your credit card statements for the last 90 days, you know, your statements where you have auto-debits come out and, and look at all the charges coming out and try to ruthlessly, defend them, get with your accountant CPA or partner or spouse, and say, you know, you defend 'em and make sure you really need these. [00:00:33] Sam Wilson: Paul Neal is a serial entrepreneur. That's owned six businesses in 30 years. He uses that wealth of real-world experience to help entrepreneurs and real estate investors win by funding their growth and dreams. Paul, welcome to the show.  [00:00:45] Paul Neal: Hey, thanks, Sam. Happy to be here, man.  [00:00:47] Sam Wilson: Hey, the pleasure's mine. Paul, there are three questions. I ask every guest who comes on the show: in 90 seconds or less, can you tell me, where did you start? Where are you now? And how did you get there?  [00:00:56] Paul Neal: It all started when I was in college. I got in my first business, approached by my brother-in-law who's in marketing and thought, wow, I was studying engineering and didn't like it. He said, come work with me. We partnered up and had a really successful 10-year ride with that. And how I got here now was just a series. It's really, definitely not a straight line. Multiple businesses, I had some really successful ones. I had some, some massive failures and you know, I find myself kind of back where I've always loved to be, and that is in real estate and business finance.  [00:01:24] Sam Wilson: Wow. Okay. There's a lot, a lot to unpack there. A 10-year run. You guys launched a business that's, I mean, a 10-year, run's a pretty strong run for something that's like, Hey, will you come work with me? What was that business?  [00:01:34] Paul Neal: Yeah, actually, honestly, that was a network marketing business. And so that was really my first exposure to entrepreneurship. I had always been sort of conditioned to be an engineer. Just naturally pretty good at math and science. My father was an engineer. And I worked in the field a little bit when I was in school, through co-op and I just, I sat in an eight by eight, fluorescent lighted office eight hours a day. And I remember the arguing with people for about three hours over the little rubber booties on the product we were developing and thought at that point, I'm either going to slit my throat or do something else. And so he knew, we had a good relationship and he knew that I wanted something bigger and, he and my sister had been successful. And I thought, well, okay, I'll, do something different with them.  [00:02:18] Sam Wilson: Network marketing gets a really bad rap. And people see that, they hear it, they're like, oh my gosh. But I think there's some valuable things probably that people learn when they go into network marketing. Would you agree with that? And if so, what were those things?  [00:02:32] Paul Neal: Oh, undoubtedly. So yeah, I mean, I think the big takeaways are you have to have a vision and a dream of where you want to go. You have to face down your fears every day and just kind of put yourself out there because you know, rejection is the name of the game, right? You know, it's 99 to one. And so you get really a thick skin and I will say this, well, and another lesson, actually, my wife and I got to learn to work together, which was great. We've now just celebrated 31 years of marriage. So we survived that experience, you know, but it was all about failing forward and, you know, and the encouragement we got from the leadership over the years really inspired us to get, you know, to become better and stronger. And so that helped us in the, what I call the real world of business after the fact.  [00:03:16] Sam Wilson: Yeah, absolutely. That's cool. That's cool. Today you are back in real estate business finance. I want to spend probably the bulk of our time, if you don't mind, really talking about where you see us economically in a cycle, how you're positioning yourself, and kind of where you're seeing opportunity, and what people should be doing. I know it's a lot of things to unpack so maybe we'll just start, you know, where are we right now in your view, Paul? [00:03:41] Paul Neal: Yeah. So, great question, Sam. So, you know, we've had a hot run and obviously, in residential real estate, the economy's expanded quite a bit. There's been, you know, lots of evidence of inflation and talks of inflation. And, you know, there's a lot of reasons for that. A lot of its supply shock and you know, and how demand has shifted from products, goods, and services when people were locked in their homes and ordering everything from Amazon to now service shock, where people have gotten more than they need from Amazon and so now they want to go back out to restaurants and bars. They want to travel. As experience, I just got back from Florida, took me an extra day to get home from Florida to Virginia because they're short-staffed everywhere with the airlines because people are just traveling a mass among other reasons. And so, you know, the fed is aggressively moving to try to put the brakes on inflation. You know, they've bumped the prime rate 75 basis points twice. They'll probably do it again, come September. And you know, most people think that's a bad thing. The news talks about, the media talks about the fed hit and the prime rate rates are going up. The world's going to end. The reality is as they do that, they're attempting to put up the brakes on inflation. We're starting to see that ease. And I think we're going to see it ease quite a bit into the fourth quarter. And so the long end of the yield curve starts to come down so longer interest rates start to ease. We saw an increase on the 10-year note. It went from right at about 1% in December, peaked in June, right before the fed hit the gas pedal the first time. And it had peaked at like 3.5%. Well, it's since retraced all the way down to, it's battling right around 2.7 now. So, you know, people talk about the converted yield curve and so forth. So that's a signal that we're moving to recession. I will tell you this though. The other interesting thing is normally when we go into recession, you see a major upturn in unemployment. We have not seen that. And you know, there's a lot of, you know, people scratching their head, you know, from COVID taking a lot of the upper-end workers that could retire out of the workforce early that didn't come back to, I think, I heard the other day there maybe 3 million in that neighborhood, people who have perhaps have long COVID. And so they're not, you know, they're not coming back in the workforce either. So they were affected kind of adversely. So for whatever it is, we're not seeing unemployment really turn up sharply. So that's interesting, but. But I'll tell you this. I think really the key is to position yourself right now. That's what we're looking to do, you know, in the event of an economic slowdown because that always brings tremendous opportunity.  [00:06:12] Sam Wilson: When you say that, what comes to mind? Like, what are some action items? If you said to me, Hey, Sam, today here are three things or two things, whatever comes to mind, that you should be doing right now to position yourself.  [00:06:22] Paul Neal: Yeah, man. Absolutely. I'll keep it really simple. I think three things. The first thing would be, Hey, 80- 20 your expenses. You know, we talk about 80-20 principle, Pareto's principle. 20% of your expenses you probably need and 80% of them are probably fluff.  I would say, you know, seriously, look at, and you can do this on the business side, on the personal side, take your credit card statements for the last 90 days and your statements where you have auto-debits come out. And look at all the charges coming out and try to ruthlessly defend them. Get with your accountant CPA or partner or spouse and say, you know, you defend them and make sure you really need these. I don't know about you, but you know, many times I end up signing up for some service that, you know, four months later, I forgot I signed up for and don't use, but they're tapping my credit card, right? So how many of those do we have, right?  [00:07:07] Sam Wilson: I don't know what you're talking about, Paul.  [00:07:08] Paul Neal: Yeah, no, it never happens, right? So, 80- 20, you know, and including, sort of out of the box. You know, maybe you need to look at your staff and do you have anybody that's low producing or not effective. Maybe now's the time to trim some fat. Or how about this? Maybe buy out a partner. If you're an entrepreneur or a business owner, maybe, you know, they're getting close to retirement, maybe they don't want to go through a recession, maybe this is a good opportunity, right? Second thing, I would say shore up that balance sheet. Look at your debt. You know, is there anywhere you can, I would say pay off, but my next point's going to sort of preclude that. I would say, you know, can you extend your terms anywhere? Can you consolidate, can you get better rates in terms of what you currently have? Do you have some assets on there that aren't performing that maybe you could turn into cash, maybe a property you could sell that's not performing like you'd like it to do? Or just some, I don't know, maybe you've got a car laying around that's not, you just don't really need. Turn it to cash. It's a great time to do that. Which brings me to my third point in that is really focus on cash. Get some dry powder, you know, opportunities are going to come. I think it was Warren Buffett who said be fearful when others are greedy and greedy when others are fearful, right? So this is when most wealth is created, I believe, in a downturn, not in an upcycle. Things go on sale. Properties go on sale. Businesses go on sale. So from a cash standpoint, you know, get a line of credit if you need it. Look at your AR, accounts receivable. I see so many businesses that, you know, they take forever to get their money in the door. There's ways you can improve that, aggressive follow up, any deposits up front, maybe factoring. Different ways to do that, but really focus on cash for those opportunities that come up that will inevitably come to he who's ready for that.  [00:08:41] Sam Wilson: Yeah. I like that. I like that a lot. And I know people are going to laugh at me and say, well, gee, Sam, why didn't you do this a year and a half ago? But I've had no debt, even on my primary residence forever. And I'm like, I'm looking at it right now and I go, gosh, if I'm going to start stacking cash for an opportunity. So yeah, I'm looking at getting debt on my primary, which is one of those things it's kind of, you know, everybody's got their own view on what you should or shouldn't do in that respect, but it's one of those things just, in line with that, you know, stacking cash, on that comment though, if you're stacking cash, how do you protect that from the effects of inflation? I've got some business accounts and things myself that are, that have been, you know, it's idle and it's not a small sum of idle capital, but what do you recommend in that regard? So, Hey, we're stacking cash, but what do we do with it in the meantime?  [00:09:22] Paul Neal: Yeah. I mean, that's a good question. And I'll be the first to say, I'm not an expert there, but I would say that, you know, who is it that said, I forget, somebody famous said, I'm more concerned about the return of my investment and the return on my investment. And so I think in times of transition and turmoil, 'cause I think we're in that state right now, nobody really knows what's going to happen. Preservation of cash versus, you know, deployment of cash might be in the short term the more poignant question in my mind, I know it's a conservative approach. I tend there after having a business blow up in 2008, a business that was a multimillion-dollar business position to sell. And then liquidity went to zero. I had not expected that I knew we were going to have a slowdown. Didn't know. So you just don't know. And so again, I would say safe, liquid, and readily available and then reevaluate that in 90 days. You know, the world's going to look different in, I believe, October, November than it does today. And again, I think we'll have a little more sense of stability. We've got a midterm election coming up. There's a lot of things going on. So the plates, you know, it's going to look different at that point.  [00:10:26] Sam Wilson: What would you say to somebody maybe that isn't in that position where they can liquidate assets or they can shore up balance sheets or even go out, maybe even their expenses are pretty low? Let's say they're just getting started, right, and maybe maybe again, they don't have that capital to necessarily stack quickly. What do you say to that person right now? How can they position themselves to win in a recession?  [00:10:47] Paul Neal: I think obviously point number one of really seriously looking at your expenses. I think that you need to look at allocating some savings. I mean, you've got money coming in right now. I think that's critical to get in the habit of savings. And then I think you should really align yourself with people that are a few steps down the road ahead of you, that have been there. 'Cause we all started, well, I don't say we all started with nothing. I did. I didn't inherit any money, you know, I wasn't fortunate enough to have a famous last. So, if you're not in that camp, like, I wasn't then align yourself with people that are down the road ahead of you, or particularly in real estate investing, there are a lot of people you can follow. Obviously, Sam, is a great one, and learn from them. And then you can start looking at building and assembling a team. You know, really the key is the idea, not the capital. If you can come up with the ideas and you will, if you immerse yourself with the right people, then the capital will show up. You will have opportunities. It may not be to, you know, to buy a 30-unit apartment building day one. But you know, not everybody starts there.  [00:11:47] Sam Wilson: Absolutely. Let's talk about that for a minute. What are you, you said today you are involved in real estate, in business finance, where are you seeing opportunity right now in real estate? [00:11:56] Paul Neal: Yeah, we see a lot of new construction on the investor side. We have quite a few builders that we finance and their, you know, inventory is super low on the residential side. Depending on where you are in the country, I mean, if you're in a, you know, New York City or California, you're probably not going to have to high demand, but if you're in the Southeast or where people want to move to, there's a lot of demand and there's just not a lot of supply. And that's not going to slow down. I mean, you have 40% of the mortgages in residential real estate are below 3%. So people in those homes aren't going to sell, that inventory's not coming on the market, unless they're just forced to move. You have a large percentage of 'em are actually paid off. You have this big, the largest cadre of a demographic population coming into the I'm now ready to buy a home for the first time, the millennials that, you know, they want to move out of the apartment. You know, a lot of 'em are working hybrid work schedules, so they want some more room. They're moving out of the cities, they want homes and they just aren't out there. So I mean, I think single-family, if you can get that is great just from the demand. I have a lot of entrepreneurs because I find a lot of real estate investors are also business owners, entrepreneurs, and professionals, doctors, veterinarians, that sort of thing. Those that haven't are looking to buy the buildings they're in the buildings aren't cheap, but you know, what a great investment there from a commercial standpoint. They lease out part of the building, helps defer some of the costs. There's a lot of tax advantages there. I have a client, good friend of mine. She's an OB-GYN. She had a practice, bought her building about 13 years ago. Anyways, looking to retire in the next couple of years, has paid the building off. She was just recently acquired by one of these larger medical groups and they're not only paying her to buy the business, they're paying her a salary for a few more years, but they're going to rent the building back for her for $9,000 to $10,000 a month of cash flow for just, what a retirement cash flow, right?  [00:13:41] Sam Wilson: That's amazing. Absolutely love that. Yeah. It's stuff like that that makes a lot of sense. Let's talk about business finance. What are some things you're involved in right now, I guess, on the business finance, if you don't mind talking about that?  [00:13:53] Paul Neal: Yeah, sure. [00:13:53] Sam Wilson: Finance side of things that you say, Hey, these are unique areas where I see opportunity. And I guess specifically inside of that, what type of businesses do you see that are, I guess, inflation and recession-resistant?  [00:14:05] Paul Neal: Well, I think service businesses are inflation and recession-resistant. I mean, you know, you've got to have your, well, you look at lawn care for instance, you know, but any business where it demands a, you know, someone to come do something for you, those are in high demand. They're always in demand, right? You can't Amazon those, right? You can't shut that out of, you know, or offshore that. That's always a safe investment. We see a lot right now, we do a lot of business acquisition funding and we see a lot of partner buyouts. You have a lot of businesses that are well established, that the principles are aging out. They've done it. They've been there done that. They've got the t-shirt and they're, like, they want to sell. And a lot of times it's to maybe a junior, it's a family member, or someone in the company. But not always, there's a lot of turnovers. I mean, all these baby boomers are looking to retire, so I think that's a great opportunity. I have a young client. He's probably not 30 yet. And he's in the gourmet popcorn business. And so yeah, I know it's kind of funny, right, but he had, he has a very successful gourmet popcorn, retail establishment. And he's acquiring other established, that's his business plan. So as he adds additional popcorn businesses to his portfolio, he's growing his cash flow and his balance sheet. And he's been really successful at that. So there's a lot of opportunity out there.  [00:15:20] Sam Wilson: What are risks, I guess, that you are actively avoiding when you look at business acquisition, if somebody comes to you and they're looking for acquisition funding, what are some things that are just red flags that you go thanks for the call, but no? [00:15:34] Paul Neal: Yeah. So, startups. We don't do startup startups are usually risky. We want to see businesses that are cash flowing well. We want to see, you know, if you want to buy a business, we want to see that you have the ability to manage and grow the business. So some sort of experience, it doesn't have to be necessarily directly in that business, but you know, you have to have some relevant management business experience that can, you know, transfer over. Because the last thing we want for you or for us is for you to acquire the business and then, you know, to fail, you know, a year to two years later, 'cause you just didn't know what you were getting into. We try to stay out of the sort of risky industries, you know, your vice industries, things like that. Although they could be hugely profitable, just not our area. But you'd be amazed at the calls we get. So yeah, so I would say really it's basically cash flow is the key things, what's the history of the business, and if you're doing the acquiring, do you have experience? And a lot of times it's a business that's already in that field and they're just expanding their operations through little mergers and acquisitions, picking up a $2 million, you know, branch here, a million dollar branch over here, that sort of thing. And that again can be a super effective way to grow, particularly in a recession when, you know, you also may have a competitor that may not be as systematically effective as you, doesn't have the systems and the processes, and you've kind of nailed it and you can acquire them and turn that around. So their cash flow may not have to be as strong if you have the demonstrated ability to do that. [00:17:00] Sam Wilson: Right, right. No, that's absolutely true. I love that idea of expansion in a time of recession or when everybody else is contracting, I think that's a great time to go out and take market share. And I think that's really, really great there. Tell me this. If you rewind the tape, you look back at your last 20 or 30-year career, what is one thing that you feel like you've done very well that other people should emulate? And what is one thing maybe you'd say, Hey, here's a pitfall that somebody can avoid? [00:17:27] Paul Neal: That's a great question. I think, I don't know. I think doing well, I try to make long-term choices. Like, my wife is a long-term choice for instance and I chose well there. I don't know how that happened. God was good to me, you know? And whenever I've approached a business, I've always tried to go along with it and look beyond, you know, day one and have always been willing to put money in, even when I didn't have it, realizing that, you know, I couldn't do everything myself. The bad decisions are the times where I felt like I could do it all myself and, you know, to be the Jack of all trades and then the master of none. And so the guy who's building the business is also cutting the grass, so to speak. And that just, it didn't work. And it was because, you know, I thought, well, A, I can do it and B, you know, we can save a few bucks on the ramp up or whatever. And you know, inevitably it was just a bad decision. It would cost me time and ultimately a lot of money. [00:18:26] Sam Wilson: Right. Yeah. It's the time to market. It's the ramp-up speed. It's getting in your own way. It's the, you know, you're the bottleneck. And I think many of us, myself included him constantly reviewing things like that where it's like, why am I, this is, I am in my own way right here.  [00:18:41] Paul Neal: Totally. Yeah. Well, that brings me to another point, you know. It's like, so I've had a business coach on and off, but mostly on for the last 20 years. And I'm actually a certified business coach. I went through the process and had a few clients, really just to sharpen my own ax. I don't actively do it today, but we all need somebody who can look over our shoulder and say, you know, ask that question, like, Hey, you know, why are you doing that? You know, so defend that for me. You know, and I'm like, I really can't defend that for you, you know?  [00:19:12] Sam Wilson: Right, right. And that goes back to your credit card statement, defending those expenses where it's like, oh, whatever it is like, okay, yeah. I probably can't. So maybe you should cut it out.  [00:19:22] Paul Neal: Right, right. Probably don't need it. Yeah. [00:19:24] Sam Wilson: Probably don't need it, man. That's awesome. Paul, thank you for taking the time to come on the show today and really just kind of give us your history of business, how to position ourselves to win in a time of recession, kind of your thoughts here on the market. You've given us three very easy-to-follow steps on things that we should be doing right now to be ready for a downturn and how to, increase our market share in a time of recession. Certainly appreciate your insights on where you're seeing opportunity in the real estate space and also in the business side of things. It was great having you on the show today. If our listeners want to get in touch with you or learn more about you, what is the best way to do that? [00:19:56] Paul Neal: Yeah. Thanks, Sam. So I've set up a free resource. I've put together the key questions that you've got to ask and answer before you seek funding. I think this is a really awesome guidebook to kind of help you think through the process, to make sure that you really have to ask a lot of questions. It's a serious consideration when you look for funding of any type. We're spinning up a special page just for this, just for your listeners. It's my website, VPC, Victor- Paul-Charlie.capital. That's VPC, Victor- Paul-Charlie.capital/pocast, with a hyphen or a dash, scaleCRE.  [00:20:38] Sam Wilson: Fantastic. We'll make sure we put that in the show notes there, vpc.capital/podcast-scaleCRE. That link of course will be there in the show notes. Appreciate that. That's something, I'm actually, when we hang up here, Paul, I'm going to go download myself 'cause I'm really curious what's inside it. So thank you again for providing that for us and our listeners. And thank you again for coming on the show today. Certainly appreciate it.  [00:20:58] Paul Neal: Sam. It was a pleasure. Thank you for inviting me. I certainly enjoyed it. 

Screaming in the Cloud
Third Wave Security with Alex Marshall of Twingate

Screaming in the Cloud

Play Episode Listen Later Sep 1, 2022 31:46


About AlexAlex is the Chief Product Officer of Twingate, which he cofounded in 2019. Alex has held a range of product leadership roles in the enterprise software market over the last 16 years, including at Dropbox, where he was the first enterprise hire in the company's transformation from consumer to enterprise business. A focus of his product career has been using the power of design thinking to make technically complex products intuitive and easy to use. Alex graduated from Stanford University with a degree in Electrical Engineering.Links Referenced:twingate.com: https://twingate.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Sysdig. Sysdig secures your cloud from source to run. They believe, as do I, that DevOps and security are inextricably linked. If you wanna learn more about how they view this, check out their blog, it's definitely worth the read. To learn more about how they are absolutely getting it right from where I sit, visit Sysdig.com and tell them that I sent you. That's S Y S D I G.com. And my thanks to them for their continued support of this ridiculous nonsense.Corey: This episode is sponsored in part by Honeycomb. When production is running slow, it's hard to know where problems originate. Is it your application code, users, or the underlying systems? I've got five bucks on DNS, personally. Why scroll through endless dashboards while dealing with alert floods, going from tool to tool to tool that you employ, guessing at which puzzle pieces matter? Context switching and tool sprawl are slowly killing both your team and your business. You should care more about one of those than the other; which one is up to you. Drop the separate pillars and enter a world of getting one unified understanding of the one thing driving your business: production. With Honeycomb, you guess less and know more. Try it for free at honeycomb.io/screaminginthecloud. Observability: it's more than just hipster monitoring.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. This promoted episode is brought to us by our friends at Twingate, and in addition to bringing you this episode, they also brought me a guest. Alex Marshall is the Chief Product Officer at Twingate. Alex, thank you for joining me, and what is a Twingate?Alex: Yeah, well, thanks. Well, it's great to be here. What is Twingate? Well, the way to think about Twingate is we're really a network overlay layer. And so, the experience you have when you're running Twingate as a user is that network resources or network destinations that wouldn't otherwise be accessible to you or magically accessible to you and you're properly authenticated and authorized to access them.Corey: When you say it's a network overlay, what I tend to hear and the context I usually see that in, in the real world is, “Well, we're running some things in AWS and some things in Google Cloud, and I don't know because of a sudden sharp blow to the head, maybe Azure as well, and how do you get all of the various security network models of security groups on one side to talk to their equivalent on the other side?” And the correct answer is generally that you don't and you use something else that more or less makes the rest of that irrelevant. Is that the direction you're coming at this from, or do you view it differently?Alex: Yeah, so I think the way that we view this in terms of, like, why we decide to build a product in the first place is that if you look at, sort of like, the internet in 2022, like, there's one thing that's missing from the network routing table, which is authentication and authorization on each row [laugh]. And so, the way that we designed the product is we said, “Okay, we're not going to worry about everything, basically, above the network layer and we're going to focus on making sure that what we're controlling with the client is looking at outbound network connections and making sure that when someone accesses something and only when they access it, that we check to make sure that they're allowed access.” We're basically holding those network connections until someone's proven that they're allowed to access to, then we let it go. And so, from the standpoint of, like, figuring out, like, security groups and all that kind of stuff, we're basically saying, like, “Yeah, if you're allowed to access the database in AWS, or your home assistant on your home network, fine, we'll let you do that, but we'll only let you go there once you've proven you're allowed to. And then once you're there, then you know, we'll let you figure out how you want to authenticate into the destination system.” So, our view is, like, let's start at the network layer, and then that solves a lot of problems.Corey: When I call this a VPN, I know a couple of things are going to be true. One, you're almost certainly going to correct me on that because this is all about Zero Trust. This is the Year of our Lord 2022, after all. But also what I round to what basically becomes a VPN to my mind, there are usually two implementations or implementation patterns that I think about. One of them is the idea of client access, where I have a laptop; I'm in a Starbucks; I want to connect to a thing. And the other has historically been considered, site to site, or I have a data center that I want to have constantly connected to my cloud environment. Which side of that mental model do you tend to fall in? Or is that the wrong way to frame it?Alex: Mm-hm. The way we look at it and sort of the vision that we have for what the product should be, the problem that we should be solving for customers is what we want to solve for customers is that Twingate is a product that lets you be certain that your employees can work securely from anywhere. And so, you need a little bit of a different model to do that. And the two examples you gave are actually both entirely valid, especially given the fact that people just work from everywhere now. Like, resources everywhere, they use a lot of different devices, people work from lots of different networks, and so it's a really hard problem to solve.And so, the way that we look at it is that you really want to be running something or have a system in place that's always taking into account the context that user is in. So, in your example of someone's at a Starbucks, you know, in the public WiFi, last time I checked, Starbucks WiFi was unencrypted, so it's pretty bad for security. So, what we should do is you should take that context into account and then make sure that all that traffic is encrypted. But at the same time, like, you might be in the corporate office, network is perfectly safe, but you still want to make sure that you're authorizing people at the point in time they try to access something to make sure that they actually are entitled to access that database in the AWS network. And so, we're trying to get people away from thinking about this, like, point-to-point connection with a VPN, where you know, the usual experience we've all had as employees is, “Great. Now, I need to fire up the VPN. My internet traffic is going to be horrible. My battery's probably going to die. My—”Corey: Pull out the manual token that rotates with an RSA—Alex: Exactly.Corey: —token that spits out a different digital code every 30 seconds if the battery hasn't died or they haven't gotten their seeds leaked again, and then log in and the rest; in some horrible implementations type that code after your password for some Godforsaken reason. Yeah, we've all been down that path and it's like, “Yeah, just sign into the corporate VPN.” It's like, “Did you just tell me to go screw myself because that's what I heard.”Alex: [laugh]. Exactly. And that is exactly the situation that we're in. And the fact is, like, VPNs were invented a long time ago and they were designed to connect to networks, right? They were designed to connect a branch office to a corporate office, and they're just to join all the devices on the network.So, we're really, like—everybody has had this experience of VPN is suffering from the fact that it's the wrong tool for the job. Going back to, sort of like, this idea of, like, us being the network overlay, we don't want to touch any traffic that isn't intended to go to something that the company or the organization or the team wants to protect. And so, we're only going to gate traffic that goes to those network destinations that you actually want to protect. And we're going to make sure that when that happens, it's painless. So, for example, like, you know, I don't know, again, like, use your example again; you've been at Starbucks, you've been working your email, you don't really need to access anything that's private, and all of a sudden, like, you need to as part of your work that you're doing on the Starbucks WiFi is access something that's in AWS.Well, then the moment you do that, then maybe you're actually fine to access it because you've been authenticated, you know, and you're within the window, it's just going to work, right, so you don't have to go through this painful process of firing up the VPN like you're just talking about.Corey: There are a number of companies out there that, first, self-described as being, “Oh, we do Zero Trust.” And when I hear that, what I immediately hear in my own mind is, “I have something to sell you,” which, fair enough, we live in an industry. We're trying to have a society here. I get it. The next part that I wind up getting confused by then is, it seems like one of those deeply overloaded terms that exists to, more or less—in some cases to be very direct—well, we've been selling this thing for 15 years and that's the buzzword, so now we're going to describe it as the thing we do with a fresh coat of paint on it.Other times it seems to be something radically different. And, on some level, I feel like I could wind up building an entire security suite out of nothing other than things self-billing themselves as Zero Trust. What is it that makes Twingate different compared to a wide variety of other offerings, ranging from Seam to whatever the hell an XDR might be to, apparently according to RSA, a breakfast cereal?Alex: So, you're right. Like, Zero Trust is completely, like, overused word. And so, what's different about Twingate is that really, I think goes back to, like, why we started the company in the first place, which is that we started looking at the remote workspace. And this is, of course, before the pandemic, before everybody was actually working remotely and it became a really urgent problem.Corey: During the pandemic, of course, a lot of the traditional VPN companies are, “Huh. Why is the VPN concentrator glowing white in the rack and melting? And it sounds like screaming. What's going on?” Yeah, it turns out capacity provisioning and bottlenecking of an entire company tends to be a thing at scale.Alex: And so, you're right, like, that is exactly the conversation. We've had a bunch of customers over the last couple years, it's like their VPN gateway is, like, blowing up because it used to be that 10% of the workforce used it on average, and all of a sudden everybody had to use it. What's different about our approach in terms of what we observed when we started the company, is that what we noticed is that this term Zero Trust is kind of floating out there, but the only company that actually implemented Zero Trust was Google. So, if you think about the situations that you look at, Zero Trust is like, obvious. It's like, it's what you would want to do if you redesigned the internet, which is you'd want to say every network connection has to be authorized every single time it's made.But the internet isn't actually designed that way. It's designed default open instead of default closed. And so, we looked at the industry are, like, “Great. Like, Google's done it. Google has, like, tons and tons of resources. Why hasn't anyone else done it?”And the example that I like to talk about when we talk about inception of the business is we went to some products that are out there that were implementing the right technological approach, and one of these products is still in use today, believe it or not, but I went to the documentation page, and I hit print, and it was almost 50 pages of documentation to implement it. And so, when you look at that, you're, like, okay, like, maybe there's a usability problem here [laugh]. And so, what we really, really focus on is, how do we make this product as easy as possible to deploy? And that gets into, like, this area of change management. And so, if you're in IT or DevOps or engineering or security and you're listening to this, I'm sure you've been through this process where it's taken months to deploy something because it was just really technically difficult and because you had to change user behavior. So, the thing that we focus on is making sure that you didn't have to change user behavior.Corey: Every time you expect people to start doing things completely differently, congratulations, you've already lost before you've started.Alex: Yes, exactly. And so, the difference with our product is that you can switch off the VPN one day, have people install a Twingate client, and then tomorrow, they still access things with exactly the same addresses they used before. And this seems like such a minor point, but the fact that I don't have to rewrite scripts, I don't have to change my SSH proxy configuration, I don't have to do anything, all of those private DNS addresses or those private IP address, they'll still work because of the way that our client works on the device.Corey: So, what you're saying is fundamental; you could even do a slow rollout. It doesn't need to be a knife-switch cutover at two in the morning where you're scrambling around and, “Oh, my God, we forgot the entire accounting department.”Alex: Yep, that's exactly right. And that is, like, an attraction of deploying this is that you can actually deploy it department by department and not have to change all your infrastructure at the same time. So again, it's like pretty fundamental point here. It's like, if you're going to get adoption technology, it's not just about how cool the technology is under the hood and how advanced it is; it's actually thinking about from a customer and a business standpoint, like, how much is actually going to cost time-wise and effort-wise to move over to the new solution. So, we've really, really focused on that.Corey: Yeah. That is generally one of those things, that seems to be the hardest approach. I mean, let's back up a little bit here because I will challenge—likely—something that you said a few minutes ago, which is Google was the first and only company for a little while doing Zero Trust. Back in 2012, it turned out that we weren't calling it that then, but that is fundamentally what I built out of the ten-person startup that I was at, where I was the first ops hire, which generally comes in right around Series B when developers realize, okay, we can no longer lie to ourselves that we know what we're doing on an ops side. Everything's on fire and no one can sleep through the night. Help, help, help. Which is fine.I've never had tolerance or patience for ops people who insult people in those situations. It's, “Well, they got far enough along to hire you, didn't they? So, maybe show some respect.” But one of the things that I did was, being on the corporate network got you access to the printer in the corner and that was it. There was no special treatment of that network.And I didn't think much of it at the time, but I got some very strange looks and had some—uh, will call it interesting a decade later; most of the pain has faded—discussions with our auditor when we were going through some PCI work, and they showed up and said, “Great. Okay, where are the credentials for your directory?” And my response was, “Our what now?” And that's when I realized there's a certain point of scale. Back when I started as an independent consultant, everything I did for single-sign-on, for example, was my 1Password vault. Easy enough.Now, that we've scaled up beyond that, I'm starting to see the value of things like single-sign-on in a way that I never did before, and in hindsight, I'd like to go back and do things very differently as a result. Scale matters. What is the point of scale that you find is your sweet spot? Is it one person trying to connect to a whole bunch of nonsense? Is it small to midsize companies—and we should probably bound that because to me, a big company is still one that has 200 people there?Alex: To your original interesting point, which is that yeah, kudos to you for, like, implementing that, like, back then because we've had probably—Corey: I was just being lazy and it was what was there. It's like, “Why do I want to maintain a server in the closet? Honestly, I'm not sure that the office is that secure. And all it's going to do—what I'm I going to put on that? A SharePoint server? Please. We're using Macs.”Alex: Yeah, exactly. Yeah. So it's, we've had, like, I don't know at this point, thousands of customer conversations. The number of people have actually gone down that route implementing things themselves as a very small number. And I think that just shows how hard it is. So again, like, kudos.And I think the scale point is, I think, really critical. So, I think it's changed over time, but actually, the point at which a customer gets to a scale where I think a solution has, like, leveraged high value is when you get to maybe only 50, 75 people, which is a pretty small business. And the reason is that that's the point at which a bunch of tools start getting implemented a company, right? When you're five people, you're not going to install, like, an MDM or something on people's devices, right? When you get to 50, 75, 100, you start hiring your first IT team members. That's the point where them being able to, like, centralize management of things at the company becomes really critical.And so, one of the other aspects that makes this a little bit different terms of approach is that what we see is that there's a huge number of tools that have to be managed, and they have different configuration settings. You can't even get consistency on MDM is across different platforms, necessarily, right? Like, Linux, Windows, and Mac are all going to have slight differences, and so what we've been working with the platform towards is actually being the centralization point where we integrate with these different systems and then pull together, like, a consistent way to create those authentication authorization policies I was talking about before. And the last thing on SSO, just to sort of reiterate that, I think that you're talking about you're seeing the value of that, the other thing that we've, like, made a deliberate decision on is that we're not going to try to, like, re-solve, like, a bunch of these problems. Like, some of the things that we do on the user authentication point is that we rely on there being an SSO, like, user directory, that handles authentication, that handles, like, creating user groups. And we want to reuse that when people are using Twingate to control access to network destinations.So, for us, like, it's actually, you know, that point of scale comes fairly early. It only gets harder from there, and it's especially when that IT team is, like, a relatively small number of people compared to number of employees where it becomes really critical to be able to leverage all the technology they have to deploy.Corey: I guess this might be one of those areas where I'm not deep enough in your space to really see it the same way that you do, which is the whole reason I have people like you on the show: so I can ask these questions directly. What is the painful position that I find myself in that I should say, “Ah, I should bring Twingate in to solve this obnoxious, painful problem so I never have to think about it again.” What is it that you solve?Alex: Yeah, I mean, I think for what our customers tell us, it's providing a, like, consistent way to get access into, like, a wide variety of internal resources, and generally in multi-cloud environments. That's where it gets, like, really tricky. And the consistency is, like, really important because you're trying to provide access to your team—often like it's DevOps teams, but all kinds of people can access these things—trying to write access is a multiple different environments, again, there's a consistency problem where there are multiple different ways to provide that, and there isn't a single place to manage all that. And so, it gets really challenging to understand who has access to what, makes sure that credentials expire when they're supposed to expire, make sure that all the routing inside those remote destinations is set up correctly. And it just becomes, like, a real hassle to manage those things.So, that's the big one. And usually where people are coming from is that they've been using VPN to do that because they didn't know anything better exists, or they haven't found anything that's easy enough to deploy, right? So, that's really the problem that they're running into.Corey: There's also a lot of tribal knowledge that gets passed down. The oral tradition of, “I have this problem. What should I do? I know, I will consult the wise old sage.” “Well, where can you find the wise old sage?” “Under the rack of servers, swearing at them.” “Great, cool. Well, use a VPN. That's what we've used since time immemorial.” And then the sins are visited onto yet another generation.There's a sense that I have that companies that are started now are going to have a radically different security posture and a different way of thinking about these things than the quote-unquote, “Legacy companies.”—legacy, of course, being that condescending engineering term for ‘it makes money—who are migrating their way into a brave new world because they had the temerity to found themselves as companies before 2012.Alex: Absolutely. When we're working with customers, there is a sort of a sweet spot, both in terms of, like, the size and role that we were talking about before, but also just in terms of, like, where they are, in, sort of like, the sort of lifecycle of their company. And I think one of the most exciting things for us is that we get to work with companies that are kind of figuring this stuff out for the first time and they're taking a fresh look at, like, what the capabilities are out there in the landscape. And that's, I think, what makes this whole space, like, super, super interesting.There's some really, really fantastic things you can do. Just give you an example, again, that I think might resonate with your audience quite a bit is this whole topic of automation, right? Your time at the tribal knowledge of, like, “Oh, of course. You know, we set up a VPN and so on.” One of the things that I don't think is necessarily obvious in this space is that for the teams that—at companies that are deploying, configuring, managing internal network infrastructure, is that in the past, you've had to make compromises on infrastructure in order to accommodate access, right?Because it's kind of a pain to deploy a bunch of, like, VPN gateways, mostly for the end-user because they got to, like, choose which one they're connecting to. You potentially had to open up traffic routes to accommodate a VPN gateway that you wouldn't otherwise want to open up. And so, one of the things that's, like, really sort of fascinating about, like, a new way of looking at things is that what we allow with Twingate—and part of this is because we've really made sure that the product is, like, API-first in the very beginning, which allows us to very easily integrate in with things, like, Terraform and Pulumi for deployment automation, is that now you have a new way of looking at things, which is that you can build a network infrastructure that you want with the data flow rules that you want, and very easily provide access into, like, points of that infrastructure, whether that's an entire subnet or just a single host somewhere. I think these are the ways, like, the capabilities have been realized are possible until they, sort of like, understand some of these new technologies.Corey: This episode is sponsored in part by our friend EnterpriseDB. EnterpriseDB has been powering enterprise applications with PostgreSQL for 15 years. And now EnterpriseDB has you covered wherever you deploy PostgreSQL on-premises, private cloud, and they just announced a fully-managed service on AWS and Azure called BigAnimal, all one word. Don't leave managing your database to your cloud vendor because they're too busy launching another half-dozen managed databases to focus on any one of them that they didn't build themselves. Instead, work with the experts over at EnterpriseDB. They can save you time and money, they can even help you migrate legacy applications—including Oracle—to the cloud. To learn more, try BigAnimal for free. Go to biganimal.com/snark, and tell them Corey sent you.Corey: This feels like one of those technologies where the place that a customer starts from and where they wind up going are very far apart. Because I can see the metaphorical camel's nose under the tent flap being, “Ah, this is a VPN except it doesn't suck. Great.” But once you wind up with effectively an overlay network connecting all the things that you care about within an organization, it feels like that unlocks a whole universe of possibility.Alex: Mm-hm. Yeah, definitely. I mean, I think you hit the nail on the head there. Like, a lot of people approach us because they're having a lot of pain with VPN and all the operational difficulties they were talking about earlier, but I think what sort of starts to open up is there's some, sort of like, not obvious things that happen. And one of them is that all of a sudden, when you can limit access at a network connection level, you start to think about, like, credentials and access management a little differently, right?So, one of the problems that well-known is people set a bastion host. And they set bastion host so that there's, like, a limited way into the network and all the, you know, keys are stored in that bastion host and so on. So, you basically have a system where fine, we had bastion host set up because, A, we want limited ingress, and B, we want to make sure that we know exactly who has access to our internal resources. You could do away with that and with a simple, like, configuration change, you can basically say, “Even if this employee for whatever reason, we've forgotten to remove—revoke their SSH keys, even if they still have those keys, they can't access the destination because we're blocking network access at their actual device,” then you have a very different way to restrict access. So, it's still important to manage credentials, but you now have a way to actually block things out at a network level. And I think it's like when people start to realize that these capabilities are possible that they definitely start thinking about things a little bit differently. VPNs just don't allow this, like, level of granularity.Corey: I am a firm believer in the idea that any product with any kind of longevity gets an awful lot of its use case and product-market fit not from the people building it, but from the things that those folks learn from their customers. What did you learn from customers rolling out Twingate that reshaped how you thought about the space, or surprised you as far as use cases go?Alex: Yeah, so I think it's a really interesting question because one of the benefits of having a small business and being early on is that you have very close relationships with all your customers and they're really passionate about your product. And what that leads to is just a lot of, sort of like, knowledge sharing around, like, how they're using your product, which then helps inform the types of things that we build. So, one of the things that we've done internally to help us learn, but then also help us respond more quickly to customers, is we have this group called Twingate Labs. And it's really just a group of folks that are outside the engineering org that are just allowed to build whatever they want to try to prove out, like, interesting concepts. And a lot of those—I say a lot; honestly, probably all of those concepts have come from our customers, and so we've been able to, like, push the boundaries on that.And so, it just gave you an example, I mean, AWS can be sometimes a challenging product to manage and interact with, and so that team has, for example, built capabilities, again, using that just the regular Twingate API to show that it's possible to automatically configure resources in AWS based on tags. Now, that's not something that's in our product, but it's us showing our customers that, you know, we can respond quickly to them and then they actually, like, try to accommodate some, like, these special use cases they have. And if that works out, then great, we'll pull it into the product, right? So, I think that's, like, the nice thing about serving a smaller businesses is that you get a lot of that back and forth to your customers and they help us generate ideas, too.Corey: One thing that stands out to me from the testimonials from customers you have on your website has been a recurring theme that crops up that speaks to I guess, once I spend more than ten seconds thinking about it, one of the most obvious reasons that I would say, “Oh, Twingate? That sounds great for somebody else. We're never rolling it out here.” And that is the ease of adoption into environments that are not greenfield because I don't believe that something like this product will ever get deployed to something greenfield because this is exactly the kind of problem that you don't realize exists and don't have to solve for until it's too late because you already have that painful problem. It's an early optimization until suddenly, it's something you should have done six months ago. What is the rolling it out process for a company that presumably already is built out, has hired a bunch of people, and they already have something that, quote-unquote, “Works,” for granting access to things?Alex: Mm-hm. Yeah, so the beauty is that you can really deploy this side-by-side with an existing solution, so—whatever it happens to be; I mean, whether it's a VPN or something else—is you can put the side-by-side and the deployment process, just to talk a little bit about the architecture; we've talked a lot about this client that runs on the user's device, but on the remote network side, just to be really clear on this, there's a component called a connector that gets deployed inside the remote network, and it does not have to be installed on every single destination host. You're sort of thinking about it, sort of like this routing point inside that network, and that connector controls what traffic is allowed to go to internal locations based on the rules. So, from a deployment standpoint, it's really just put a connector in place and put it in place in whatever subnet you want to provide access to.And so you're—unlikely, but if your entire company has one subnet, great. You're done with one connector. But it does mean you can sort of gradually roll it out as it goes. And the connector can be deployed in a bunch of different environments, so we're just talking with AWS. Maybe it's inside a VPC, but we have a lot of people that actually just want to control access to specific services inside a Kubernetes cluster, and so you can deploy it as a container, right inside Kubernetes. And so, you can be, like, really specific about how you do that and then gradually roll it out to teams as they need it and without having to necessarily on that day actually shut off the old solution.So, just to your comment, by the way, on the greenfield versus, sort of like, brownfield, I think the greenfield story, I think, is changing a little bit, I think, especially to your comment earlier around younger companies. I think younger companies are realizing that this type of capability is an option and that they want to get in earlier. But the reality is that, you know, 98% of people are really in the established network situation, and so that's where that rollout process is really important.Corey: As you take a look throughout what you're seeing customers doing, what you see the industry doing as a result of that—because customers are, in fact, the industry, let's be clear here—what do you think is, I guess, the next wave of security offerings? I guess what I'm trying to do here is read the tea leaves and predict what the buzzwords will be all over the place that next RSA. But on a slightly more serious note, what do you see this is building towards? What are the trends that you're identifying in the space?Alex: There's a couple of things that we see. So one, sort of, way to look at this is that we're sort of in this, like, Third Wave. And I think these things change more slowly than—with all due respect to marketers—than marketers would [laugh] have you believe. And so, thinking about where we are, there's, like, Wave One is, like, good old happy days, we're all in the office, like, your computer can't move, like, all the data is in the office, like, everything is in one place, right?Corey: What if someone steals your desktop? Well, they're probably going to give themselves a hernia because that thing's heavy. Yeah.Alex: Exactly. And is it really worth stealing, right? But the Wave One was really, like, network security was actually just physical security, to that point; that's all it was, just, like, physically secure the premises.Wave Two—and arguably you could say we're kind of still in this—is actually the transition to cloud. So, let's convert all CapEx to OpEx, but that also introduces a different problem, which is that everything is off-network. So, you have to, like, figure out, you know, what you do about that.But Wave Three is really I think—and again, just to be clear, I think Wave Two, there are, like, multi-decade things that happen—and I'd say we're in the middle of, like, Wave Three. And I think that everyone is still, like, gradually adapting to this, which is what we describe it as sort of people everywhere, applications are everywhere, people are using a whole bunch of different devices, right? There is no such thing as BYOD in the early-2000s, late-90s, and people are accessing things from all kinds of different networks. And this presents a really, really challenging problem. So, I would argue, to your question, I think we're still in the middle of that Wave Three and it's going to take a long time to see that play through the industry. Just, things change slowly. That tribal knowledge takes time to change.The other thing that I think we very strongly believe in is that—and again, this is, sort of like, coming from our customers, too—is that people basically with security industry have had a tough time trying things out and adopting them because a lot of vendors have put a lot of blockers in place of doing that. There's no public documentation; you can't just go use the product. You got to talk to a salesperson who then filters you through—Corey: We have our fifth call with the sales team. We're hoping this is the one where they'll tell us how much it costs.Alex: Exactly. Or like, you know, now you get to the sales engineer, so you gradually adopt this knowledge. But ultimately, people just want to try the darn thing [laugh], right? So, I think we're big believers that I think hopefully, what we'll see in the security industry is that—we're trying to set an example here—is really that there's an old way of doing things, but a new way of doing things is make the product available for people to use, document the heck out of it, explain all the different use cases that exist for how to be successful your product, and then have these users actually then reach out to you when they want to have more in-depth conversation about things. So, those are the two big things, I'd say. I don't know if those are translated buzzwords at RSA, but those are two big trends we see.Corey: I look forward to having you back in a year or two and seeing how close we get to the reality. “Well, I guess we didn't see that acronym coming, but don't worry. They've been doing it for the last 15 years under different names, so it works out.” I really want to thank you for being as generous with your time as you have been. If people want to learn more, where should they go?Alex: Well, as we're just talking about, you try the product at twingate.com. So, that should be your first stop.Corey: And we will of course put links to that in the show notes. Thank you so much for being as forthcoming as you have been about all this stuff. I really appreciate your time.Alex: Yeah, thank you, Corey. I really appreciate it. Thanks.Corey: Alex Marshall, Chief Product Officer at Twingate. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with a long angry ranty comment about what you hated about the episode, which will inevitably get lost when it fails to submit because your crappy VPN concentrator just dropped it on the floor.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Kappa Kappa Psi Presents
District Council: Jayar Brenner

Kappa Kappa Psi Presents

Play Episode Listen Later Aug 31, 2022 13:00


Welcome to this episode! Join editor & host, Ryan Smith, as he interviews Jayar Brenner, North Central District Vice President for Communication, about his journey in Kappa Kappa Psi and goals for his District. Transcription is located here. Questions, Comments, Suggestions: smity@kkpsi.org

Screaming in the Cloud
How to Leverage AWS for Web Developers with Adam Elmore

Screaming in the Cloud

Play Episode Listen Later Aug 23, 2022 34:24


About AdamAdam is an independent cloud consultant that helps startups build products on AWS. He's also the host of AWS FM, a podcast with guests from around the AWS community, and an AWS DevTools Hero.Adam is passionate about open source and has made a handful of contributions to the AWS CDK over the years. In 2020 he created Ness, an open source CLI tool for deploying web sites and apps to AWS.Previously, Adam co-founded StatMuse—a Disney backed startup building technology that answers sports questions—and served as CTO for five years. He lives in Nixa, Missouri, with his wife and two children.Links Referenced: 17 Ways to Run Containers On AWS: https://www.lastweekinaws.com/blog/the-17-ways-to-run-containers-on-aws/ Twitter: https://twitter.com/aeduhm Twitch: https://www.twitch.tv/adamelmore TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Every once in a while, I encounter someone in the wild that… well, I'll just be direct, makes me feel a little bit uneasy, almost like someone's walking over my grave. And I think I've finally figured out elements of what that is. It feels sometimes like I run into people—ideally not while driving—who are trying to occupy sort of the same space in the universe, and I never quite know how to react to that.Today's guest is just one such person. Adam Elmore is an independent AWS consultant, has been all over the Twitters for a while, recently started live streaming basically his every waking moment because he is just that interesting. Adam, thank you for suffering my slings and arrows—Adam: [laugh].Corey: —and agreeing to chat with me today.Adam: I would say first of all, you don't need to be worried about anyone walking over your grave. [laugh]. That was very flattering.Corey: No, honestly, I have big enterprise companies looking to put me in my grave, but that's a separate threat model. We're good on that, for now.Adam: [laugh]. I got to set myself up here to—I'm just going to laugh a lot, and your editor or somebody's going to have to deal with that. And maybe the audience will see—[laugh].Corey: Hey, I prefer that as opposed to talking to people who have absolutely no sense of humor of which they are aware. Awesome, I have a list of companies that they should apply for immediately. So, when I say that we're trying to occupy elements of the same space in the universe, let me talk a little bit about what I mean by that. You are independent as a consultant, which is how I started this whole nonsense, and then I started gathering a company around me almost accidentally. You are an AWS Dev Tools Hero, whereas I am an AWS community villain, which is kind of a polar opposite slash anti-hero approach, and it's self-granted in my case. How did you stumble into the universe of AWS? You just realized one day you were too happy and what can you do to make yourself miserable, and this was the answer, or what?Adam: Yeah, I guess. So. I mean, I've been a software developer for 15 years, like, my whole career, that's kind of what I've done. And at some point, I started a startup called StatMuse. And I was able, as sort of a co-founder there, with venture backing, like, I was able to just kind of play with the cloud.And we deployed everything on AWS, so that was—like, I was there five years; it was sort of five years of running this, I would call it like a Digital Media Studio. Like, we built technology, but we did lots of experiments, so it felt like playing on AWS. Because we built kind of weird one-offs, these digital experiences for various organizations. The Hall of Fame was one of them. We did, like, a, like, a 3-D Talking bust of John Madden, so it was like all kinds of weird technology involved.But that was sort of five years of, I guess, spending venture money [laugh] to play on AWS. And some of that was Google money; I guess I never thought about that, but Google was an investor in StatMuse. [laugh]. Yeah, so we sort of like—I ran that for five years and was able to learn just a lot of AWS stuff that really excited me. I guess, coming from normal web development stuff, it was exciting just how much leverage you have with AWS, so I sort of dove in pretty hard. And then yeah, when I left StatMuse in 2019 I've just been, I guess, going even harder into that direction. I just really enjoy it.Corey: My first real exposure to AWS was at a company where the CTO was a, I guess we'll call him an extraordinarily early cloud evangelist. I was there as a contractor, and he was super excited and would tweet nonsensical things like, “I'm never going to rack a server ever again.” And I was a grumpy sysadmin type; I came from the ops world where anything that is new shouldn't be treated with disdain and suspicion because once you've been a sysadmin for 20 minutes, you've been there long enough to see today's shiny new shit become tomorrow's legacy garbage that you're stuck supporting. So, “Oh, great. What now?”I was very down on Cloud in those days and I encountered it with increasing frequency as I stumbled my way through my career. And at the end of 2016, I wound up deciding to go out independent and fix… well, what problems am I good at fixing that I can articulate in a sentence, and well, I'd gotten surprised by AWS bills from time to time—fortunately with someone else's money; the best kind of mistake to make—and well I know a few things. Let's get really into it. In time, I came to learn that cost and architecture the same thing in cloud, and now I don't know how the hell to describe myself. Other people love to describe me, usually with varying forms of profanity, but here we are. It really turns into the idea of forging something of your own path. And you've absolutely been doing that for at least the last three years as you become someone who's increasingly well known and simultaneously harder to describe.Adam: Yeah, I would say if you figure it out, if you know how to describe me, I would love to know because just coming up with the title—for this episode you needed, like, my title, I don't know what my title is. I'm also—like, we talked about independent, so nobody sort of gives me a title. I would love to just receive one if you think of one, [laugh] if anyone listening thinks of one… it's increasingly hard to, sort of like, even decide what I care most about. I know I need to, like, probably niche down, I feel like you've kind of niched into the billing stuff. I can't just be like, “I'm an AWS guy,” because AWS is so big. But yeah, I have no idea.Corey: Anyone who claims, “Oh, I'm an expert in AWS,” is lying or trying to sell something.Adam: [laugh]. Exactly.Corey: I love that. It's, “Really? I have some questions to establish that for you.” As far as naming what it is, you do, first piece of advice, never ever, ever, ever listen to someone who works at AWS; those people are awful at naming things, as evidenced by basically every service they've ever launched. But you are actually fairly close to being an AWS expert. You did a six-week speed-run through every certification that they offer and that is nothing short of astonishing. How'd it come about?Adam: It's a unique intersection of skills that I think I have. And I'm not very self-aware, I don't know all my strengths and weaknesses and I struggle to sort of nail those down, but I think one of my strengths is just ability to, like, consume information, I guess at a high volume. So, I'm like an auditory learner; I can listen to content really fast and sort of retain enough. And then I think the other skill I have is just I'm good at tests. I've always said that, like, going back to school, like, high school, I always felt like I was really good at multiple-choice tests. I don't know if that's a skill or some kind of innate talent.But I think those two things combined, and then, like, eight years of building on AWS, and that sort of frames how I was able to take all that on. And I don't know that I really set out thinking I will do it in six weeks. I took the first few and then did them pretty fast and thought, “I wonder how quickly I could do all of them.” And I just kind of at that point, it became this sort of goal. I have to take on certain challenges occasionally that just sound fun for no reason other than they sound fun and that was kind of the thing for those six weeks. [laugh].Corey: I have two certifications: Cloud Practitioner and the SysOps Administrator Associate. Those were interesting.Adam: You took the new one, right? The new SysOps with the labs and stuff I'd love to hear about that.Corey: I did, back when it was in beta. That was a really interesting experience and I'll definitely get to that, but I wound up, for example, getting a question wrong in the Cloud Practitioner exam four years ago or so, when it was, “How long does it take to restore an RDS instance from backup?” And I gave the honest answer instead of the by-the-book, correct answer. That's part of the problem is that I've been doing this stuff too long and I know how these things break and what the real world looks like. Certifications are also very much a snapshot at a point in time.Because I write the Last Week in AWS newsletter, I'm generally up-to-the-minute on what has changed, and things that were not possible yesterday, suddenly are possible today, so I need to know when was this certification launched. Oh, it was in early 2021. Yeah, I needed to be a lot more specific; which week? And then people look at me very strangely and here we are.The Systems Administrator Certification was interesting because this is the first one, to my knowledge, where they started doing a live lab as a—Adam: Yeah.Corey: Component of this. And I don't think it's a breach of the NDA to point out that one of the exams was, “Great. Configure CloudWatch out of the box to do this thing that it's supposed to do out of the box.” And I've got to say that making the service do what it's supposed to do with no caveats is probably the sickest shade I've ever seen anyone throw at AWS, like, configuring the service is so bad that it is going to be our test to prove you know what you're doing. That is amazing.Adam: [laugh]. Yeah, I don't have any shade through I'm not as good with the, like, ability to come off, like, witty and kind while still criticizing things. So, I generally just try not to because I'm bad at it. [laugh].Corey: It's why I generally advise people don't try, in seriousness. It's not that people can't be clever; it's that the failure mode of clever is ‘asshole' and I'm not a big fan of making people feel worse based upon the things that I say and do. It's occasionally I wind up getting yelled at by Amazonians saying that the people who built a service didn't feel great about something I said, and my instinctive immediate reaction is, “Oh, shit, that wasn't my intention. How did I screw this up?” Given a bit of time, I realized that well hang on a minute because I'm not—they're not my target audience. I'm trying to explain this to other customers.And, on some level, if you're going to charge tens of millions of dollars a month for a service or more, maybe make a better one, not for nothing. So, I see both sides of it. I'm not intentionally trying to cause pain, but I'm also not out here insulting people individually. Like, sometimes people make bad decisions, sometimes individually, sometimes in a group. And then we have a service name we have to live with, and all right, I guess I'm going to make fun of that forever. It's fun that keeps it engaging for me because otherwise, it's boring.Adam: No, I hear you. No, and somebody's got to do it. I'm glad you do it and do it so well because, I mean, you got to keep them honest. Like, that's the thing. Keep AWS in check.Corey: Something that I went through somewhat recently was a bit of an awakening. I have no problem revisiting old opinions and discovering that huh, I no longer agree with it; it's time to evolve that opinion. The CDK specifically was one of those where I looked at it and thought this thing looks a little hokey. So, I started using it in Python and sure enough, the experience was garbage. So cool, the CDK is a piece of crap. There we go. My job is easy.I was convinced to take a second look at it via TypeScript, a language I do not know and did not have any previous real experience with. So, I spent a few days just powering through it, and now I'm a convert. I think it's amazing. It is my default go-to for building AWS infrastructure. And all it took was a little bit of poking and prodding to get me to change my mind on that. You've taken it to another level and you started actively contributing to the AWS CDK. What was your journey with that, honestly, remarkable piece of software?Adam: Yeah, so I started contributing to CDK when I was actually doing a lot of Python development. So, I worked with a company that was doing—there was a Python shop. So actually, the first thing I contributed was a Python function construct, which is sort of the equivalent of the Node.js function construct, which like, you can just basically point at a TypeScript file and it transpiles it, bundles it, and does all that, right? So, it makes it easy to deploy TypeScript as a Lambda function.Well, I mean, it ends up being a JavaScript Lambda function, but anyway, that was the Python function construct. And then I sort of got really into it. So, I got pretty hooked on using the CDK in every place that I could. I'm a huge fan, and I do primarily write in TypeScript these days. I love being able to write TypeScript front-end and back, so built a lot of, like, Next.JS front-ends, and then I'm building back-ends with CDK TypeScript.Yeah, I've had, like, a lot of conversations about CDK. I think there's definitely a group that's sort of, against the CDK, if you're thinking in terms of, like, beginners. And I do see where, for people who aren't as familiar with AWS, or maybe this is their entry point into cloud development, it does a lot of things that maybe you're not aware of that, you know, you're now kind of responsible for. So, it's deploying—like, it makes it really easy to write, like, three lines of TypeScript that stand up an entire VPC with all this configuration and Managed NAT Gateways and [laugh] everything else. And you may not be aware of all the things you just stood up.So, CloudFormation maybe is a little more—sort of gives you that better visibility into what you're creating. So, I've definitely seen that pushback. But I think for people who really, like, have built a lot of applications on AWS, I think the CDK is just such a time-saver. I mean, I spend so much less time building the same things in the CDK versus CloudFormation. I'm a big fan.Corey: For me, I've learned enough about JavaScript to be dangerous and it seems like TypeScript is more or less trying to automate a bunch of people's jobs away, which is basically, from I can tell, their job is to go on the internet and complain about someone's JavaScript. So great, that that's really all it does is it complains, “Oh, this ambiguous. You should be more specific about it.” And great. Awesome. I still haven't gotten into scenarios where I've been caught out by typing issues, and very often I find that it just feels like sheer bloodymindedness, but I smile, nod, bend the knee and life goes on.Adam: [laugh]. When you've got a project that's, like, I don't know, a few months old—or better, a few years old—and you need to do, like, major refactoring, that's when TypeScript really saves you just a ton of time. Like, when you can make a change in a type or in actual implementation stuff and then see the ripple effects and then sort of go around the codebase and fix those things, it's just a lot easier than doing it in JavaScript and discovering stuff at runtime. So, I'm a big TypeScript fan. I don't know where it's all headed. I know there's people that are not fans of, like, transpiling your Lambda functions, for instance. Like, why not just ship good JavaScript? And I get that case, too. Yeah, but I've definitely—I felt the productivity boost, I guess—if that's the thing—from TypeScript.Corey: For me, I'm still at a point where I'm learning the edges of where things start and where they stop. But one of the big changes I made was that I finally, after 15 years, gave up my beloved Vim as my editor for this and started using VS Code. Because the reasons that I originally went with Vi were understandable when you realize what I was. I'm always going to be remoting into network gear or random—on maintained Unix boxes. Vi is going to be everywhere on everything and that's fine.Yeah, I don't do that anymore, and increasingly, I find that everything I'm writing is local. It is not something that is tied to a remote thing that I need to login and edit by hand. At that point, we are in disaster area. And suddenly it's nice. I mean things like tab completion, where it just winds up completing the rest of the variable name or, once you enable Copilot and absolutely not CodeWhisperer yet, it winds up you tab complete your entire application. Why not? It's just outsourcing it to Stack Overflow without that pesky copy and paste step.Adam: Yeah, I don't know how in the weeds you want to get on your p—I don't know, in terms of technical stuff, but Copilot both blows me away—there are days where it autocompletes something that I just, I can't fathom how—it pulled in not just, like, the patterns that it found, obviously, in training, but, like, the context in the file I'm working and sort of figured out what I was trying to do. Sometimes it blows me away. A lot of times, though, it frustrates me because of TypeScript. Like, I'm used to Typescript and types saving me from typing a lot. Like, I can tab-complete stuff because I have good types defined, right, or it's just inferred from the libraries I'm using.It's tough though when GitHub is fighting with TypeScript and VS Code. But it's funny that you came from Vim and you now live in VS Code. I really am trying to move from VS Code to, like, the Vim world, mostly because of Twitch streamers that blow my mind with what they can do in Vim [laugh] and how fast they can move. I do—every time I move my hand, like, over to the arrow keys, I feel a little sad and I wish I just did Vim.Corey: This episode is sponsored in part by our friends at Lambda Cloud. They offer GPU instances with pricing that's not only scads better than other cloud providers, but is also accessible and transparent. Also, check this out, they get a lot more granular in terms of what's available. AWS offers NVIDIA A100 GPUs on instances that only come in one size and cost $32/hour. Lambda offers instances that offer those GPUs as single card instances for $1.10/hour. That's 73% less per GPU. That doesn't require any long term commitments or predicting what your usage is gonna look like years down the road. So if you need GPUs, check out Lambda. In beta, they're offering 10TB of free storage and, this is key, data ingress and egress are both free. Check them out at lambdalabs.com/cloud. That's l-a-m-b-d-a-l-a-b-s.com/cloud.Corey: There are people who have just made it into an entire lifestyle, on some level. And I'm fair to middling; I've known people who are dark wizards at it. In practice, I found that my productivity was never constrained by how quickly I can type. It's one of those things where it's, I actually want to stop and have my brain catch up sometimes, believe it or not, for those who follow me on Twitter. It's the idea of wanting to make sure that I am able to intelligently and rationally wrap my head around what it is I'm doing.And okay, just type out a whole bunch of boilerplate is, like, the least valuable use of anything and that is where I find things like Copilot working super well, where I, if I'm doing CloudFormation, for example, the fact that it tab-completes all the necessary attributes and can go back and change them or whatnot, that's an enormous time saver. Same story with the CDK, although with some constructs, it doesn't quite understand which ones get certain values to it. And I really liked the idea behind it. I think this is in some ways, the future of IDEs, to a point.Adam: Oh, for sure. I think, like, the case, you call that with CloudFormation, you don't have really typeahead in VS Code, at least I'm not using anything. Maybe there are extensions that give you that in VS Code. But to have Copilot fill in required prompts on a CloudFormation template, that's a lifesaver. Because I just, every time I write CloudFormation, I've just got the docs up and I'm copying stuff I've done before or whatever; like, to save that time it's huge. But CodeWhisperer, not so much? Is it not, I guess, up to snuff? I haven't seen it or played with it at all.Corey: It's still very early days and it hasn't had exposure outside of Amazonian codebases to my understanding, so it's, like, “Learn to code like an Amazonian.” And you can fill in your own joke here on that one. I imagine it's like—isn't that—aren't they primarily a Java shop, for one? And all right. It turns out most of my code doesn't need to operate the way that there's does.Adam: I didn't know that they were training it just internally. Like, I'm assuming Copilot is trained on, like, Stack Overflow or something, right? Or just all of GitHub, I guess.Corey: And GitHub and a bunch of other things, and people are yelling at them for it, and I haven't been tracking that. But honestly, the CodeWhisperer announcement taught me things about Copilot, which is weird, which tells me that none of these companies are great at explaining this. Like I can just write a comment in this of, “Add an S3 bucket,” and then Copilot will tab-complete the entirety of adding an S3 bucket, usually even secure, which is awesome. They also fix the early Copilot teething problems of tab-completing people's AWS API credentials. You know, the—yeah, they've fixed a lot of that, thankfully.Adam: Yeah.Corey: But it's still one of those neat things that you can just basically start—it gets a little bit closer to describe what you want the application to do and then it'll automatically write it for you on the back-end. Sure, sometimes it makes naive decisions that do not bear out, but again, it's still early days. I'm optimistic.Adam: Yeah, that reminds me of, like, the, I mean, the serverless cloud, so serverless framework folks, like, what they're doing where they're sort of inferring your infrastructure based on you just write an app and it sort of creates the infrastructure as code for you, or just sort of infers it all from your code. So, if you start using a bucket, it'll create a bucket for that. That definitely seems to be a movement as well, where just do less as a developer [laugh] seems to be the theme.Corey: Yeah, just move up the stack. We see this time and time again. I mean, look at the—I use this analogy from time to time from the sysadmin world, but in the late-90s, if you wanted to build a web server, you needed a spare week and an intimate knowledge of GCC compiler flags. In time, it became oh, great, now it's rpm install, then yum install, then ensure present with something like Puppet, and then Docker has it, and now it's just a checkbox on the S3 page, and you're running a static site. Things don't get harder with time, and I don't think that as a developer, your time is best spent writing by hand the proper syntax for a for loop or whatnot.It's not the differentiated value. Talk to me instead about what you want that thing to do. That was my big problem with Lambda when it first came out and I spent two weeks writing my first Lambda function—because I'm bad at programming—where I had to learn the exact format of expected for input and output, and now any Lambda function I write takes me a couple of minutes to write because I'm also bad at programming and don't know what tests are.Adam: [laugh]. Tests are overrated, I don't spend a lot of time writing t—I mean, I do a lot of stuff alone and I do a lot of stuff for myself, so in those contexts, I'm not writing tests if I'm being honest. I stream now and everyone on the stream is constantly asking, “Where are the tests?” Like, there are no tests. I'm sorry. [laugh]. Was someone else's stream.Corey: Oh yeah, it used to be though, that you had to be a little sneakier to have other people do work for you. Copilot makes it easier and presumably CodeWhisperer will, too. Used to be that if AWS launched new service and I didn't know how to configure it, all I would do is restrict a role down to only being able to work with that service, attach that to a user and then just drop the credentials on Twitter or GitHub. And I waited 20 minutes and I came back and sure enough, someone configured it and was already up and mining Bitcoin. So, turn that off, take what they built, and off the production with it. Problem solved. Oh, and rotate those credentials, unless you enjoy pain. Problem solved. The end. And I don't know if it's a best practice, but it sure was effective.Adam: Yeah, that would do it. Well, they're just like scanners now, right, like they're just scanning GitHub public repos for any credentials that are leaked like that, and they're available within seconds. You can literally, like, push a public repo with credentials and it is being [laugh] used within minutes. It's nuts.Corey: GitHub has some automatic back channel thing—I believe; I haven't done an experiment lately, but I believe that AWS will intentionally shoot down the credential as soon as it gets reported, which is kind of amazing. I really should do some more experiments with it just to see how disastrous this can get.Adam: Yeah. No, I'd be curious. Please let me know. I guess you'll tweet about it so I'll see it.Corey: Can I borrow your account for a few minutes?Adam: Yeah. [laugh].Corey: Yeah, it's fun. Now, the secret to my 17 Ways to Run Containers On AWS is in almost every case, those containers can be crypto miners, so it's not just about having too many services do the same thing; it's the attack surface continues to grow and expand in the fullness of time. I'm not saying this is right or wrong; it is what it is, but it's also something that I think people have an understated appreciation for.Let's change topic a little bit. Something you've been doing lately and talking about is the idea of building a course on AWS. You're clearly capable of doing the engineering work. That's not in question. You've been a successful consultant for years, which tells me you also know how to deliver software that meets customer requirements, as opposed to, “Well, the spec was shitty, but I wrote it anyway,” because you don't last long as a consultant if you enjoy being able to afford to eat if that's the direction you go in. Now, you're drifting toward becoming a teacher. Tell me about that. First, what makes you think that's something you're good at?Adam: So, I don't know. I don't know that I'm good at it and I guess I'll find out. I've been streaming, like, on Twitch just my work days, and that's been early signs that I think I'm okay at it, at least. I think it's very different, obviously, like, a self-paced course are going to be very different from streaming for hours, so there's a lot more editing and thoughtfulness involved, but I do think, like, I've always wanted to teach. So, even before I got into technology—I was pretty late into technology; it was after high school. Like back in high school, I always thought I wanted to be a professor.I just enjoyed, I guess the idea of presenting ideas in ways that people understood. And I live in an area—so I live in the Ozarks, it's not a very tech literate area. It became, like, this thing where I felt like I could really explain technology to people who are non-technical. And that's not necessarily what my course—what I'm aiming to do. I'm trying to teach web developers how to leverage AWS, and then sort of get out of the maybe front-end only or maybe traditional web frameworks—like, they've only worked with stuff that they deploy to Heroku or whatever—trying to teach that crowd, how to leverage AWS and all these wonderful primitives that we have.So, that's not exactly the same thing, but that's sort of like, I feel like I do have the ability to translate technology to non-technical folks. And then I guess, like, for me, at this stage of my career, you know, I've done a lot of work for a company, for startups, for individual clients, and it feels very, like—I just always feel like I'm going in a hole. Like, I feel like, I'm doing this little thing and I'm serving this one customer, but the idea of being able to, I guess, serve more people and sort of spread my reach, the idea of creating something that I can share with a lot of developers who would maybe benefit from it, it just feels better, I guess. [laugh]. I don't know exactly all the reasons why that feels better, but like, at the end of the day, my consulting kind of feels like this thing I do because I just need money.And now that I need money less and less, I just feel like I'd rather do stuff that I actually am excited about. I'm actually really excited about the outcomes for creating a course where, you know, I think I can maybe—my style of teaching or something could resonate with some group of people. Yeah, so that's it. It's AWS for web devs. The thought is that I'm going to create courses after this. Like, I hope to move into more education, less consulting. That's where I'm at.Corey: I would say you're probably selling yourself fairly short. I've seen a lot of the content you've put out over the years and I learned a lot from it every time. I think that there are some folks who put courses out where, one, they don't have the baseline knowledge around what it is that they're teaching, it just feels like a grift, and another failure mode is that people know how to do the thing, but they have no idea how to teach it to someone who isn't them. And there's nothing inherently wrong with not knowing how to teach; it is its own distinct skill. The problem is when you don't recognize that about yourself and in turn, wind up having some somewhat significant challenges.Adam: Yeah. No, I know that one of the struggles is, I work with pretty obscure technologies on AWS. Not obscure, but like, I have a very specific way I build APIs on AWS and I don't know that's generally, if you're taking a bunch of web developers and trying to move them into AWS is probably not the stack that I use. So, that is part of it, but that's also kind of to my benefit, I guess. It works for me a little bit in that I'm less familiar with maybe the more beginner-friendly way to enter into AWS.It's been years, so I think I can kind of come at it a little fresh and that'll help me produce a course that maybe meets them where they're at better. Yeah, the grifting thing, I'm definitely sensitive to just this idea of putting out a course. It was hard for me to really go out there and say I was making a course, even on Twitter, because I just feel like there's, like, some stereotype—I don't know, there's an association with that, for me at least, for my perception of course creation. But I know that there are people who've done it right and do it for the right reasons. And I think to the extent that I could hit that, you know, both those things, do it right and do it for the right reasons, then it's exciting to me. And if I can't, and it turned out not good at teaching, then I'll move on and do more consulting, I guess, [laugh] or streaming on Twitch.Corey: You are very clearly self-aware enough that if you put something out and it isn't effective, I have zero doubt that you won't just stop selling it, you'll take it down and reach out to people. Because you, more so than most, seem very cognizant of the fact that a poor experience learning something does not in most people's cases, translate to, “Oh, my teacher is shitty.” Instead, it's, “Oh, I'm bad at this and I'm not smart enough to figure it out.” That's still the problem I run into with bad developer experience on a bunch of things that get launched. If I have a bad time, I assume it's, “Oh, I'm stupid. I wish someone had told me.”And first, they did, secondly, it's the sense that no, it's just not being very clearly explained and the folks who wrote the documentation or talking about it are too close to what they've built to understand what it's like to look at this thing from fresh eyes. They're doing a poor job of setting the stage to explain the value it brings and in what scenario, you should be using this.Adam: It's a long process. I want to launch the course in the fall, but in the process of building out the course, I'm really going to be doing workshops and individual—like, I just have a lot of friends that are web developers and I'm going to be kind of getting on with them and teaching them this material and just trying to see what resonates. I'm going to a lot of trouble, I guess, to make sure I'm not just putting out a thing just to say I made a course. Like, I don't actually want to say I made a course, so if I'm going to do it, it's like most things I do I really kind of throw myself into. And I know if I spend enough energy and effort, I think I can make something that at least helps some people. I guess we'll see.Corey: I look forward to it. Any idea as far as rough timeline goes?Adam: Yeah, I hope to launch in the fall. But if it takes longer, I don't know. I've heard people say, to do a course right, you should spend a year on it. And maybe that's what I do.Corey: No, I love that answer. It's great. You're just saying I want to launch in the fall, which is sufficiently vague, and if that winds up not being vague enough, you could always qualify with, “Well, I didn't say what year.”Adam: [laugh].Corey: So, great you know, it's always going to be the fall somewhere.Adam: [laugh]. I just know, like, when someone says you should spend a year I just do things very hard. Like I really, like, throw a lot of time and obsess, like, I'm very obsessive. And when I do something, it's hard for me imagine doing any one thing for a year because I burn myself out. Like, I obsess very hard for usually, like, three months, it's usually, like, a quarter, and then I fall off the face of the earth for three months and I basically mope around the house and I'm just too tired to do anything else. So, I think right now I'm streaming and that's kind of been my obsession. I'm three weeks in so we got a few more months and then we'll see, [laugh] we'll see how I maintain it.Corey: Well, I look forward to seeing how it comes out. You'll have to come back and let us know when it's ready for launch.Adam: Yeah, that sounds great.Corey: I really want to thank you for being so generous with your time and taking me through what you're up to. If people want to learn more, what's the best place for them to find you?Adam: Yeah, I think Twitter. I mean, I mostly hang out on Twitter, and these days Twitch. So, Twitter my handle—I guess you'll put it, like, in the thing description or something. It's like the phonetic—Corey: Oh, we will absolutely toss it into the show notes, where useful content goes to linger.Adam: [laugh]. It's like A-E-D-U-H-M. It's like a—it's the phonetic way of saying Adam, I guess. And then on Twitch, I'm adamelmore. So, those are the two places I spend most my time.Corey: And off to the show notes it goes. Thank you so much for being so generous with your time. I really appreciate it, Adam.Adam: Thank you so much for having me, Corey. I really appreciate it.Corey: Adam Elmore, independent AWS consultant. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an insulting comment that attempts to teach us exactly what we got wrong, but fails utterly because you're terrible at teaching things.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Cloud Posse DevOps
Cloud Posse DevOps "Office Hours" (2022-08-10)

Cloud Posse DevOps "Office Hours" Podcast

Play Episode Listen Later Aug 10, 2022 57:24


Cloud Posse holds public "Office Hours" every Wednesday at 11:30am PST to answer questions on all things related to DevOps, Terraform, Kubernetes, CICD. Basically, it's like an interactive "Lunch & Learn" session where we get together for about an hour and talk shop. These are totally free and just an opportunity to ask us (or our community of experts) any questions you may have. You can register here: https://cloudposse.com/office-hoursJoin the conversation: https://slack.cloudposse.com/Find out how we can help your company:https://cloudposse.com/quizhttps://cloudposse.com/accelerate/Learn more about Cloud Posse:https://cloudposse.comhttps://github.com/cloudpossehttps://sweetops.com/https://newsletter.cloudposse.comhttps://podcast.cloudposse.com/[00:00:00] Intro[00:01:12] GitHub Projects is now generally availablehttps://github.blog/2022-07-27-planning-next-to-your-code-github-projects-is-now-generally-available/[00:03:45] Developers can now run GitHub Action Runners on their own Mac (M1)https://github.blog/changelog/2022-08-09-github-actions-self-hosted-runners-now-support-apple-m1-hardware[00:06:05] Checkov adds policies for GitHub Actions, GitLab Runners, CircleCI, and Argohttps://bridgecrew.io/blog/checkov-enables-ci-cd-security-with-new-supply-chain-security-policies/[00:10:44] Slack Static Site Generatorhttps://saveslack.com/[00:13:14] Stop Using CPU Limits on Kuberneteshttps://home.robusta.dev/blog/stop-using-cpu-limits/[00:14:38] CDK for Terraform Is Now Generally Available (Reddit)https://www.hashicorp.com/blog/cdk-for-terraform-now-generally-available[00:23:53] Single API to take crash-consistent snapshots of a subset of EBS volumes attached to an Amazon EC2 instancehttps://aws.amazon.com/about-aws/whats-new/2022/08/amazon-ebs-crash-consistent-snapshots-subset-ebs-volumes-attached-amazon-ec2-instance/[00:25:42]  Designing the VPC's for my org and i'm looking for insights. What would you do differently if you had this luxury? @Isaac[00:43:49] How big of Devops team and how many apps to support? @Andrew[00:56:25] Outro#officehours,#cloudposse,#sweetops,#devops,#sre,#terraform,#kubernetes,#awsSupport the show

Screaming in the Cloud
Cloud-Hosted Database Services with Benjamin Anderson

Screaming in the Cloud

Play Episode Listen Later Jul 21, 2022 35:39


About BenjaminBenjamin Anderson is CTO, Cloud at EDB, where he is responsible for developing and driving strategy for the company's Postgres-based cloud offerings. Ben brings over ten years' experience building and running distributed database systems in the cloud for multiple startups and large enterprises. Prior to EDB, he served as chief architect of IBM's Cloud Databases organization, built an SRE practice at database startup Cloudant, and founded a Y Combinator-funded hardware startup.Links Referenced: EDB: https://www.enterprisedb.com/ BigAnimal: biganimal.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.I come bearing ill tidings. Developers are responsible for more than ever these days. Not just the code that they write, but also the containers and the cloud infrastructure that their apps run on. Because serverless means it's still somebody's problem. And a big part of that responsibility is app security from code to cloud. And that's where our friend Snyk comes in. Snyk is a frictionless security platform that meets developers where they are - Finding and fixing vulnerabilities right from the CLI, IDEs, Repos, and Pipelines. Snyk integrates seamlessly with AWS offerings like code pipeline, EKS, ECR, and more! As well as things you're actually likely to be using. Deploy on AWS, secure with Snyk. Learn more at Snyk.co/scream That's S-N-Y-K.co/screamCorey: This episode is sponsored by our friends at Fortinet. Fortinet's partnership with AWS is a better-together combination that ensures your workloads on AWS are protected by best-in-class security solutions powered by comprehensive threat intelligence and more than 20 years of cybersecurity experience. Integrations with key AWS services simplify security management, ensure full visibility across environments, and provide broad protection across your workloads and applications. Visit them at AWS re:Inforce to see the latest trends in cybersecurity on July 25-26 at the Boston Convention Center. Just go over to the Fortinet booth and tell them Corey Quinn sent you and watch for the flinch. My thanks again to my friends at Fortinet.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. This promoted guest episode is brought to us by our friends at EDB. And not only do they bring us this promoted episode, they bring me their CTO for Cloud, Benjamin Anderson. Benjamin, thank you so much for agreeing to suffer the slings and arrows that I will no doubt throw at you in a professional context, because EDB is a database company, and I suck at those things.Benjamin: [laugh]. Thanks, Corey. Nice to be here.Corey: Of course. So, databases are an interesting and varied space. I think we can all agree—or agree to disagree—that the best database is, of course, Route 53, when you misuse TXT records as a database. Everything else is generally vying for number two. EDB was—back in the days that I was your customer—was EnterpriseDB, now rebranded as EDB, which is way faster to say, and I approve of that.But you were always the escalation point of last resort. When you're stuck with a really weird and interesting Postgres problem, EDB was where you went because if you folks couldn't solve the problem, it was likely not going to get solved. I always contextualized you folks as a consulting shop. That's not really what you do. You are the CTO for Cloud.And, ah, interesting. Do databases behave differently in cloud environments? Well, they do when you host them as a managed service, which is an area you folks have somewhat recently branched into. How'd you get there?Benjamin: Ah, that's interesting. So, there's a bunch of stuff to unpack there. I think EDB has been around for a long time. It's something like 13, 14, 15 years, something like that, and really it's just been kind of slowly growing, right? We did start very much as a product company. We built some technology to help customers get from Oracle database on to Postgres, way back in 2007, 2008.That business has just slowly been growing. It's been going quite well. Frankly, I only joined about 18 months ago, and it's really cool tech, right? We natively understand some things that Oracle is doing. Customers don't have to change their schemas to migrate from Oracle to Postgres. There's some cool technology in there.But as you point out, I think a lot of our position in the market has not been that product focused. There's been a lot of people seeing us as the Postgres experts, and as people who can solve Postgres problems, in general. We have, for a long time, employed a lot of really sharp Postgres people. We still employ a lot of really sharp Postgres people. That's very much, in a lot of ways, our bread and butter. That we're going to fix Postgres problems as they come up.Now, over the past few years, we've definitely tried to shift quite a bit into being more of a product company. We've brought on a bunch of people who've been doing more enterprise software product type development over the past few years, and really focusing ourselves more and more on building products and investing in ourselves as a product company. We're not a services company. We're not a consulting company. We do, I think, provide the best Postgres support in the market. But it's been a journey. The cloud has been a significant part of that as well, right? You can't get away.Corey: Oh, yeah. These days, when someone's spinning up a new workload, it's unlikely—in most cases—they're going to wind up spinning up a new data center, if they don't already have one. Yes, there's still a whole bunch of on-prem workloads. But increasingly, the default has become cloud. Instead of, “Why cloud?” The question's become, “Why not?”Benjamin: Right, exactly. Then, as people are more and more accepting of managed services, you have to be a product company. You have to be building products in order to support your database customers because what they want his managed services. I was working in managed databases and service, something like, ten years ago, and it was like pulling teeth. This is after RDS launched. This was still pulling teeth trying to get people to think about, oh, I'm going to let you run my database. Whereas, now obviously, it's just completely different. We have to build great products in order to succeed in the database business, in general.Corey: One thing that jumped out at me when you first announced this was the URL is enterprisedb.com. That doesn't exactly speak to, you know, non-large companies, and EDB is what you do. You have a very corporate logo, but your managed service is called BigAnimal, which I absolutely love. It actually expresses a sense of whimsy and personality that I can no doubt guess that a whole bunch of people argued against, but BigAnimal, it is. It won through. I love that. Was that as contentious as I'm painting it to be, or people actually have a sense of humor sometimes?Benjamin: [laugh]. Both, it was extremely contentious. I, frankly, was one of the people who was not in favor of it at first. I was in favor of something that was whimsical, but maybe not quite that whimsical.Corey: Well, I call it Postgres-squeal, so let's be very clear here that we're probably not going to see eye-to-eye on most anything in pronunciation things. But we can set those differences aside and have a conversation.Benjamin: Absolutely, no consider that. It was deliberate, though, to try to step away a little bit from the blue-suit-and-tie, enterprise, DB-type branding. Obviously, a lot of our customers are big enterprises. We're good at that. We're not trying to be the hip, young startup targeting business in a lot of ways. We have a wide range of customers, but we want to branch out a little bit.Corey: One of the challenges right now is if I spin up an environment inside of AWS, as one does, and I decide I certainly don't want to take the traditional approach of running a database on top of an EC2 instance—the way that we did in the olden days—because RDS was crappy. Now that it's slightly less crappy, that becomes a not ideal path. I start looking at their managed database offerings, and there are something like 15 distinct managed databases that they offer, and they never turn anything off. And they continue to launch things into the far future. And it really feels, on some level, like 20 years from now—what we call a DBA today—their primary role is going to look a lot more like helping a company figure out which of Amazon's 40 managed databases is the appropriate fit for this given workload. Yet, when I look around at what the industry has done, it seems that when we're talking about relational databases. Postgres has emerged back when I was, more or less, abusing servers in person in my data center days, it was always MySQL. These days, Postgres is the de facto standard, full stop. I admit that I was mostly keeping my aura away from any data that was irreplaceable at that time. What happened? What did I miss?Benjamin: It's a really good question. And I certainly am not a hundred percent on all the trends that went on there. I know there's a lot of folks that are not happy about the MySQL acquisition by Oracle. I think there's a lot of energy that was adopted by the NoSQL movement, as well. You have people who didn't really care about transactional semantics that were using MySQL because they needed a place to store their data. And then, things like MongoDB and that type of system comes along where it's significantly easier than MySQL, and that subset of the population just sort of drifts away from MySQL.Corey: And in turn, those NoSQL projects eventually turn into something where, okay, now we're trying to build a banking system on top of it, and it's, you know, I guess you can use a torque wrench as a hammer if you're really creative about it, but it seems like there's a better approach.Benjamin: Yeah, exactly. And those folks are coming back around to the relational databases, exactly. At the same time, the advancements in Postgres from the early eight series to today are significant, right? We shouldn't underestimate how much Postgres has really moved forward. It wasn't that long ago that replication was hardly a thing and Postgres, right? It's been a journey.Corey: One thing that your website talks about is that you accelerate your open-sourced database transformation. And this is a bit of a hobby horse I get on from time to time. I think that there are a lot of misunderstandings when people talk about this. You have the open-source purists—of which I shamefully admit I used to be one—saying that, “Oh, it's about the idea of purity and open and free as in software.” Great. Okay, awesome. But when I find that corporate customers are talking about when they say open-source database, they don't particularly care if they have access to the source code because they're not going to go in and patch a database engine, we hope. But what they do care about is regardless of where they are today—even if they're perfectly happy there—they don't want to wind up beholden to a commercial database provider, and/or they don't want to wind up beholden to the environment that is running within. There's a strategic Exodus that's available in theory, which on some level serves to make people feel better about not actually Exodus-ing, but it also means if they're doing a migration at some point, they don't also have to completely redo their entire data plan.Benjamin: Yeah, I think that's a really good point. I mean, I like to talk—there's a big rat's nest of questions and problems in here—but I generally like talk to about open APIs, talk about standards, talk about how much is going to have to change if you eliminate this vendor. We're definitely not open-source purists. Well, we employ a lot of open-source purists. I also used to be an open—Corey: Don't let them hear you say that, then. Fair enough. Fair enough.Benjamin: [laugh] we have proprietary software at EDB, as well. There's a kind of wide range of businesses that we participate in. Glad to hear you also mention this where-it's-hosted angle, as well. I think there's some degree to which people are—they figured out that having at least open APIs or an open-source-ish database is a good idea rather than being beholden to proprietary database. But then, immediately forget that when they're picking a cloud vendor, right? And realizing that putting their data in Cloud Vendor A versus Cloud Vendor B is also putting them in a similar difficult situation. They need to be really wary of when they're doing that. Now, obviously, I work at an independent software company, and I have some incentive to say this, but I do think it's true. And you know, there's meaningful data gravity risk.Corey: I assure you, I have no incentive. I don't care what cloud provider you're on. My guidance has been, for years, to—as a general rule—pick a provider, I care about which one, and go all in until there's a significant reason to switch. Trying to build an optionality, “Oh, everything we do should be fully portable at an instance notice.” Great. Unless you're actually doing it, you're more or less, giving up a whole bunch of shortcuts and feature velocity you could otherwise have, in the hopes of one day you'll do a thing, but all the assumptions you're surrounded by baked themselves in regardless. So, you're more or less just creating extra work for yourself for no defined benefit. This is not popular in some circles, where people try to sell something that requires someone to go multi-cloud, but here we are.Benjamin: No, I think you're right. I think people underestimate the degree to which the abstractions are just not very good, right, and the degree to which those cloud-specific details are going to leak in if you're going to try to get anything done, you end up in kind of a difficult place. What I see more frequently is situations where we have a big enterprise—not even big, even medium-sized companies where maybe they've done an acquisition or two, they've got business units that are trying to do things on their own. And they end up in two or three clouds, sort of by happenstance. It's not like they're trying to do replication live between two clouds, but they've got one business unit in AWS and one business unit and Azure, and somebody in the corporate—say enterprise architect or something like that—really would like to make things consistent between the two so they get a consistent security posture and things like that. So, there are situations where the multi-cloud is a reality at a certain level, but maybe not at a concrete technical level. But I think it's still really useful for a lot of customers.Corey: You position your cloud offering in two different ways. One of them is the idea of BigAnimal, and the other—well, it sort of harkens back to when I was in sixth grade going through the American public school system. They had a cop come in and talk to us and paint to this imaginary story of people trying to push drugs. “Hey, kid. You want to try some of this?” And I'm reading this and it says EDB, Postgres for Kubernetes. And I'm sent back there, where it's like, “Hey, kid. You want to run your stateful databases on top of Kubernetes?” And my default answer to that is good lord, no. What am I missing?Benjamin: That's a good question. Kubernetes has come a long way—I think is part of that.Corey: Oh, truly. I used to think of containers as a pure story for stateless things. And then, of course, I put state into them, and then, everything exploded everywhere because it turns out, I'm bad at computers. Great. And it has come a long way. I have been tracking a lot of that. But it still feels like the idea being that you'd want to have your database endpoints somewhere a lot less, I guess I'll call it fickle, if that makes sense.Benjamin: It's an interesting problem because we are seeing a lot of people who are interested in our Kubernetes-based products. It's actually based on—we recently open-sourced the core of it under a project called cloud-native PG. It's a cool piece of technology. If you think about sort of two by two. In one corner, you've got self-managed on-premise databases. So, you're very, very slow-moving, big-iron type, old-school database deployments. And on the opposite corner, you've got fully-managed, in the cloud, BigAnimal, Amazon RDS, that type of thing. There's a place on that map where you've got customers that want a self-service type experience. Whether that's for production, or maybe it's even for dev tests, something like that. But you don't want to be giving the management capability off to a third party.For folks that want that type of experience, trying to build that themselves by, like, wiring up EC2 instances, or doing something in their own data center with VMware, or something like that, can be extremely difficult. Whereas if you've go to a Kubernetes-based product, you can get that type of self-service experience really easily, right? And customers can get a lot more flexibility out of how they run their databases and operate their databases. And what sort of control they give to, say application developers who want to spin up a new database for a test or for some sort of small microservice, that type of thing. Those types of workloads tend to work really well with this first-party Kubernetes-based offering. I've been doing databases on Kubernetes in managed services for a long time as well. And I don't, frankly, have any concerns about doing it. There are definitely some sharp edges. And if you wanted to do to-scale, you need to really know what you're doing with Kubernetes because the naive thing will shoot you in the foot.Corey: Oh, yes. So, some it feels almost like people want to cosplay working for Google, but they don't want to pass the technical interview along the way. It's a bit of a weird moment for it.Benjamin: Yeah, I would agree.Corey: I have to go back to my own experiences with using RDS back at my last real job before I went down this path. We were migrating from EC2-Classic to VPC. So, you could imagine what dates me reasonably effectively. And the big problem was the database. And the joy that we had was, “Okay, we have to quiesce the application.” So, the database is now quiet, stop writes, take a snapshot, restore that snapshot into the environment. And whenever we talk to AWS folks, it's like, “So, how long is this going to take?” And the answer was, “Guess.” And that was not exactly reassuring. It went off without a hitch because every migration has one problem. We were sideswiped in an Uber on the way home. But that's neither here nor there. This was two o'clock in the morning, and we finished in half the maintenance time we had allotted. But it was the fact that, well, guess we're going to have to take the database down for many hours with no real visibility, and we hope it'll be up by morning. That wasn't great. But that was the big one going on, on an ongoing basis, there were maintenance windows with a database. We just stopped databasing for a period of time during a fairly broad maintenance window. And that led to a whole lot of unfortunate associations in my mind with using relational databases for an awful lot of stuff. How do you handle maintenance windows and upgrading and not tearing down someone's application? Because I have to assume, “Oh, we just never patch anything. It turns out that's way easier,” is in fact, the wrong answer.Benjamin: Yeah, definitely. As you point out, there's a bunch of fundamental limitations here, if we start to talk about how Postgres actually fits together, right? Pretty much everybody in RDS is a little bit weird. The older RDS offerings are a little bit weird in terms of how they do replication. But most folks are using Postgres streaming replication, to do high availability, Postgres in managed services. And honestly, of course—Corey: That winds up failing over, or the application's aware of both endpoints and switches to the other one?Benjamin: Yeah—Corey: Sort of a database pooling connection or some sort of proxy?Benjamin: Right. There's a bunch of subtleties that get into their way. You say, well, did the [vit 00:16:16] failover too early, did the application try to connect and start making requests before the secondaries available? That sort of thing.Corey: Or you misconfigure it and point to the secondary, suddenly, when there's a switchover of some database, suddenly, nothing can write, it can only read, then you cause a massive outage on the weekend?Benjamin: Yeah. Yeah.Corey: That may have been of an actual story I made up.Benjamin: [laugh] yeah, you should use a managed service.Corey: Yeah.Benjamin: So, it's complicated, but even with managed services, you end up in situations where you have downtime, you have maintenance windows. And with Postgres, especially—and other databases as well—especially with Postgres, one of the biggest concerns you have is major version upgrades, right? So, if I want to go from Postgres 12 to 13, 13 to 14, I can't do that live. I can't have a single cluster that is streaming one Postgres version to another Postgres version, right?So, every year, people want to put things off for two years, three years sometimes—which is obviously not to their benefit—you have this maintenance, you have some sort of downtime, where you perform a Postgres upgrade. At EDB, we've got—so this is a big problem, this is a problem for us. We're involved in the Postgres community. We know this is challenging. That's just a well-known thing. Some of the folks that are working EDB are folks who worked on the Postgres logical replication tech, which arrived in Postgres 10. Logical replication is really a nice tool for doing things like change data capture, you can do Walter JSON, all these types of things are based on logical replication tech.It's not really a thing, at least, the code that's in Postgres itself doesn't really support high availability, though. It's not really something that you can use to build a leader-follower type cluster on top of. We have some techs, some proprietary tech within EDB that used to be called bi-directional replication. There used to be an open-source project called bi-directional replication. This is a kind of a descendant of that. It's now called Postgres Distributed, or EDB Postgres Distributed is the product name. And that tech actually allows us—because it's based on logical replication—allows us to do multiple major versions at the same time, right? So, we can upgrade one node in a cluster to Postgres 14, while the other nodes in the clusters are at Postgres 13. We can then upgrade the next node. We can support these types of operations in a kind of wide range of maintenance operations without taking a cluster down from maintenance.So, there's a lot of interesting opportunities here when we start to say, well, let's step back from what your typical assumptions are for Postgres streaming replication. Give ourselves a little bit more freedom by using logical replication instead of physical streaming replication. And then, what type of services, and what type of patterns can we build on top of that, that ultimately help customers build, whether it's faster databases, more highly available databases, so on and so forth.Corey: Let's face it, on-call firefighting at 2am is stressful! So there's good news and there's bad news. The bad news is that you probably can't prevent incidents from happening, but the good news is that incident.io makes incidents less stressful and a lot more valuable. incident.io is a Slack-native incident management platform that allows you to automate incident processes, focus on fixing the issues and learn from incident insights to improve site reliability and fix your vulnerabilities. Try incident.io, recover faster and sleep more.Corey: One approach that I took for, I guess you could call it backup sort of, was intentionally staggering replication between the primary and the replica about 15 minutes or so. So, if I drop a production table or something like that, I have 15 short minutes to realize what has happened and sever the replication before it is now committed to the replica and now I'm living in hell. It felt like this was not, like, option A, B, or C, or the right way to do things. But given that meeting customers where they are as important, is that the sort of thing that you support with BigAnimal, or do you try to talk customers into not being ridiculous?Benjamin: That's not something we support now. It's not actually something that I hear that many asks for these days. It's kind of interesting, that's a pattern that I've run into a lot in the past.Corey: I was an ancient, grumpy sysadmin. Again, I'm dating myself here. These days, I just store everything at DNS text records, and it's way easier. But I digress.Benjamin: [laugh] yeah, it's something that we see a lot for and we had support for a point-in-time restore, like pretty much anybody else in the business at this point. And that's usually the, “I fat-fingered something,” type response. Honestly, I think there's room to be a bit more flexible and room to do some more interesting things. I think RDS is setting a bar and a lot of database services out there and kind of just meeting that bar. And we all kind of need to be pushing a little bit more into more interesting spaces and figuring out how to get customers more value, get customers to get more out of their money for the database, honestly.Corey: One of the problems we tend to see, in the database ecosystem at large, without naming names or companies or anything like that, is that it's a pretty thin and blurry line between database advocate, database evangelist, and database zealot. Where it feels like instead, we're arguing about religion more than actual technical constraints and concerns. So, here's a fun question that hopefully isn't too much of a gotcha. But what sort of workloads would you actively advise someone not to use BigAnimal for in the database world? But yes, again, if you try to run a DNS server, it's probably not fit for purpose without at least a shim in the way there. But what sort of workloads are you not targeting that a customer is likely to have a relatively unfortunate time with?Benjamin: Large-scale analytical workloads is the easy answer to that, right? If you've got a problem where you're choosing between Postgres and Snowflake, you're seriously considering—you actually have as much data that you seriously be considering Snowflake? You probably don't want to be using Postgres, right? You want to be using something that's column, or you want to be using a query planner that really understands a columnar layout that's going to get you the sorts of performance that you need for those analytical workloads. We don't try to touch that space.Corey: Yeah, we're doing some of that right now with just the sheer volume of client AWS bills we have. We don't really need a relational model for a lot of it. And Athena is basically fallen down on the job in some cases, and, “Oh, do you want to use Redshift, that's basically Postgres.” It's like, “Yeah, it's Postgres, if it decided to run on bars of gold.” No, thank you. It just becomes this ridiculously overwrought solution for what feels like it should be a lot similar. So, it's weird, six months ago or so I wouldn't have had much of an idea what you're talking about. I see it a lot better now. Generally, by virtue of trying to do something the precise wrong way that someone should.Benjamin: Right. Yeah, exactly. I think there's interesting room for Postgres to expand here. It's not something that we're actively working on. I'm not aware of a lot happening in the community that Postgres is, for better or worse, extremely extensible, right? And if you see the JSON-supported Postgres, it didn't exist, I don't know, five, six years ago. And now it's incredibly powerful. It's incredibly flexible. And you can do a lot of quote-unquote, schemaless stuff straight in Postgres. Or you look at PostGIS, right, for doing GIS geographical data, right? That's really a fantastic integration directly in the database.Corey: Yeah, before that people start doing ridiculous things almost looks similar to a graph database or a columnar store somehow, and yeah.Benjamin: Yeah, exactly. I think sometimes somebody will do a good column store that's an open-source deeply integrated into Postgres, rather than—Corey: I've seen someone build one on top of S3 bucket with that head, a quarter of a trillion objects in it. Professional advice, don't do that.Benjamin: [laugh]. Unless you're Snowflake. So, I mean, it's something that I'd like to see Postgres expand into. I think that's an interesting space, but not something that, at least especially for BigAnimal, and frankly, for a lot of EDB customers. It's not something we're trying to push people toward.Corey: One thing that I think we are seeing a schism around is the idea that some vendors are one side of it, some are on the other, where on the one side, you have, oh, every workload should have a bespoke, purpose-built database that is exactly for this type of workload. And the other school of thought is you should generally buy us for a general-purpose database until you have a workload that is scaled and significant to a point where running that on its own purpose-built database begins to make sense. I don't necessarily think that is a binary choice, where do you tend to fall on that spectrum?Benjamin: I think everybody should use Postgres. And I say not just because I work in a Postgres company.Corey: Well, let's be clear. Before this, you were at IBM for five years working on a whole bunch of database stuff over there, not just Postgres. And you, so far, have not struck me as the kind of person who's like, “Oh, so what's your favorite database?” “The one that pays me.” We've met people like that, let's be very clear. But you seem very even-handed in those conversations.Benjamin: Yeah, I got my start in databases, actually, with Apache CouchDB. I am a committer on CouchDB. I worked on a managed at CouchDB service ten years ago. At IBM, I worked on something in nine different open-source databases and managed services. But I love having conversations about, like, well, I've got this workload, should I use Postgres, rr should I use Mongo, should I use Cassandra, all of those types of discussions. Frankly, though, I think in a lot of cases people are—they don't understand how much power they're missing out on if they don't choose a relational database. If they don't understand the relational model well enough to understand that they really actually want that. In a lot of cases, people are also just over-optimizing too early, right? It's just going to be much faster for them to get off the ground, get product in customers hands, if they start with something that they don't have to think twice about. And they don't end up with this architecture with 45 different databases, and there's only one guy in the company that knows how to manage the whole thing.Corey: Oh, the same story of picking a cloud provider. It's, “Okay, you hire a team, you're going to build a thing. Which cloud provider do you pick?” Every cloud provider has a whole matrix and sales deck, and the rest. The right answer, of course, is the one your team's already familiar with because learning a new cloud provider while trying not to run out of money at your startup, can't really doesn't work super well.Benjamin: Exactly. Yeah.Corey: One thing that I think has been sort of interesting, and when I saw it, it was one of those, “Oh, I sort of like them.” Because I had that instinctive reaction and I don't think I'm alone in this. As of this recording a couple of weeks ago, you folks received a sizable investment from private equity. And default reaction to that is, “Oh, well, I guess I put a fork in the company, they're done.” Because the narrative is that once private equity takes an investment, well, that company's best days are probably not in front of it. Now, the counterpoint is that this is not the first time private equity has invested in EDB, and you folks from what I can tell are significantly better than you were when I was your customer a decade ago. So clearly, there is something wrong with that mental model. What am I missing?Benjamin: Yeah. Frankly, I don't know. I'm no expert in funding models and all of those sorts of things. I will say that my experience has been what I've seen at EDB, has definitely been that maybe there's private equity, and then there's private equity. We're in this to build better products and become a better product company. We were previously owned by a private equity firm for the past four years or so. And during the course of those four years, we brought on a bunch of folks who were very product-focused, new leadership. We made a significant acquisition of a company called 2ndQuadrant, which they employed a lot of the European best Postgres company. Now, they're part of EDB and most of them have stayed with us. And we built the managed cloud service, right? So, this is a pretty significant—private equity company buying us to invest in the company. I'm optimistic that that's what we're looking at going forward.Corey: I want to be clear as well, I'm not worried about what I normally would be in a private equity story about this, where they're there to save money and cut costs, and, “Do we really need all these database replicas floating around,” and, “These backups, seems like that's something we don't need.” You have, at last count, 32 Postgres contributors, 7 Postgres committers, and 3 core members. All of whom would run away screaming loudly and publicly, in the event that such a thing were taking place. Of all the challenges and concerns I might have about someone running a cloud service in the modern day. I do not have any fear that you folks are not doing what will very clearly be shown to be the right thing by your customers for the technology that you're building on top of. That is not a concern. There are companies I do not have that confidence in, to be clear.Benjamin: Yeah, I'm glad to hear that. I'm a hundred percent on board as well. I work here, but I think we're doing the right thing, and we're going to be doing great stuff going forward.Corey: One last topic I do want to get into a little bit is, on some level, launching in this decade, a cloud-hosted database offering at a time when Amazon—whose product strategy of yes is in full display—it seems like something ridiculous, that is not necessarily well thought out that why would you ever try to do this? Now, I will temper that by the fact that you are clearly succeeding in this direction. You have customers who say nice things about you, and the reviews have been almost universally positive anywhere I can see things. The negative ones are largely complaining about databases, which I admit might be coming from me.Benjamin: Right, it is a crowded space. There's a lot of things happening. Obviously, Amazon, Microsoft, Google are doing great things, both—Corey: Terrible things, but great, yes. Yes.Benjamin: [laugh] right, there's good products coming in. I think AlloyDB is not necessarily a great product. I haven't used it myself yet, but it's an interesting step in the direction. I'm excited to see development happening. But at the end of the day, we're a database company. Our focus is on building great databases and supporting great databases. We're not entering this business to try to take on Amazon from an infrastructure point of view. In fact, the way that we're structuring the product is really to try to get the strengths of both worlds. We want to give customers the ability to get the most out of the AWS or Azure infrastructure that they can, but come to us for their database.Frankly, we know Postgres better than anybody else. We have a greater ability to get bugs fixed in Postgres than anybody else. We've got folks working on the database in the open. We got folks working on the database proprietary for us. So, we give customers things like break/fix support on that database. If there is a bug in Postgres, there's a bug in the tech that sits around Postgres. Because obviously, Postgres is not a batteries-included system, really. We're going to fix that for you. That's part of the contract that we're giving to our customers. And I know a lot of smaller companies maybe haven't been burned by this sort of thing very much. We start to talk about enterprise customers and medium, larger-scale customers, this starts to get really valuable. The ability to have assurance on top of your open-source product. So, I think there's a lot of interesting things there, a lot of value that we can provide there.I think also that I talked a little bit about this earlier, but like the box, this sort of RDS-shaped box, I think is a bit too small. There's an opportunity for smaller players to come in and try to push the boundaries of that. For example, giving customers more support by default to do a good job using their database. We have folks on board that can help consult with customers to say, “No, you shouldn't be designing your schemas that way. You should be designing your schemas this way. You should be using indexes here,” that sort of stuff. That's been part of our business for a long time. Now, with a managed service, we can bake that right into the managed service. And that gives us the ability to kind of make that—you talk about shared responsibility between the service writer and the customer—we can change the boundaries of that shared responsibility a little bit, so that customers can get more value out of the managed database service than they might expect otherwise.Corey: There aren't these harsh separations and clearly defined lines across which nothing shall pass, when it makes sense to do that in a controlled responsible way.Benjamin: Right, exactly. Some of that is because we're a database company, and some of that is because, frankly, we're much smaller.Corey: I'll take it a step further beyond that, as well, that I have seen this pattern evolve a number of times where you have a customer running databases on EC2, and their AWS account managers suggests move to RDS. So, they do. Then, move to Aurora. So, they do. Then, I move this to DynamoDB. At which point, it's like, what do you think your job is here, exactly? Because it seems like every time we move databases, you show up in a nicer car. So, what exactly is the story here, and what are the incentives? Where it just feels like there is a, “Whatever you're doing is not the way that it should be done. So, it's time to do, yet, another migration.”There's something to be said for companies who are focused around a specific aspect of things. Then once that is up and working and running, great. Keep on going. This is fine. As opposed to trying to chase the latest shiny, on some level. I have a big sense of, I guess, affinity for companies that wind up knowing where they start, and most notably, where they stop.Benjamin: Yeah, I think that's a really good point. I don't think that we will be building an application platform anytime soon.Corey: “We're going to run Lambda functions on top of a database.” It's like, “Congratulations. That is the weirdest stored procedure I can imagine this week, but I'm sure we can come up with a worse one soon.”Benjamin: Exactly.Corey: I really want to thank you for taking the time to speak with me so much about how you're thinking about this, and what you've been building over there. If people want to learn more, where's the best place to go to find you?Benjamin: biganimal.com.Corey: Excellent. We will throw a link to that in the show notes and it only just occurred to me that the Postgres mascot is an elephant, and now I understand why it's called BigAnimal. Yeah, that's right. He who laughs last, thinks slowest, and today, that's me. I really want to thank you for being so generous with your time. I appreciate it.Benjamin: Thank you. I really appreciate it.Corey: Benjamin Anderson, CTO for Cloud at EDB. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment that you then wind up stuffing into a SQLite database, converting to Base64, and somehow stuffing into the comment field.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Screaming in the Cloud
Kubernetes and OpenGitOps with Chris Short

Screaming in the Cloud

Play Episode Listen Later Jul 14, 2022 39:01


About ChrisChris Short has been a proponent of open source solutions throughout his over two decades in various IT disciplines, including systems, security, networks, DevOps management, and cloud native advocacy across the public and private sectors. He currently works on the Kubernetes team at Amazon Web Services and is an active Kubernetes contributor and Co-chair of OpenGitOps. Chris is a disabled US Air Force veteran living with his wife and son in Greater Metro Detroit. Chris writes about Cloud Native, DevOps, and other topics at ChrisShort.net. He also runs the Cloud Native, DevOps, GitOps, Open Source, industry news, and culture focused newsletter DevOps'ish.Links Referenced: DevOps'ish: https://devopsish.com/ EKS News: https://eks.news/ Containers from the Couch: https://containersfromthecouch.com opengitops.dev: https://opengitops.dev ChrisShort.net: https://chrisshort.net Twitter: https://twitter.com/ChrisShort TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Coming back to us since episode two—it's always nice to go back and see the where are they now type of approach—I am joined by Senior Developer Advocate at AWS Chris Short. Chris, been a few years. How has it been?Chris: Ha. Corey, we have talked outside of the podcast. But it's been good. For those that have been listening, I think when we recorded I wasn't even—like, when was season two, what year was that? [laugh].Corey: Episode two was first pre-pandemic and the rest. I believe—Chris: Oh. So, yeah. I was at Red Hat, maybe, when I—yeah.Corey: Yeah. You were doing Red Hat stuff, back when you got to work on open-source stuff, as opposed to now, where you're not within 1000 miles of that stuff, right?Chris: Actually well, no. So, to be clear, I'm on the EKS team, the Kubernetes team here at AWS. So, when I joined AWS in October, they were like, “Hey, you do open-source stuff. We like that. Do more.” And I was like, “Oh, wait, do more?” And they were like, “Yes, do more.” “Okay.”So, since joining AWS, I've probably done more open-source work than the three years at Red Hat that I did. So, that's kind of—you know, like, it's an interesting point when I talk to people about it because the first couple months are, like—you know, my friends are like, “So, are you liking it? Are you enjoying it? What's going on?” And—Corey: Do they beat you with reeds? Like, all the questions people have about companies? Because—Chris: Right. Like, I get a lot of random questions about Amazon and AWS that I don't know the answer to.Corey: Oh, when I started telling people, I fixed Amazon bills, I had to quickly pivot that to AWS bills because people started asking me, “Well, can you save me money on underpants?” It's I—Chris: Yeah.Corey: How do you—fine. Get the prime credit card. It docks 5% off the bill, so there you go. But other than that, no, I can't.Chris: No.Corey: It's—Chris: Like, I had to call my bank this morning about a transaction that I didn't recognize, and it was from Amazon. And I was like, that's weird. Why would that—Corey: Money just flows one direction, and that's the wrong direction from my employer.Chris: Yeah. Like, what is going on here? It shouldn't have been on that card kind of thing. And I had to explain to the person on the phone that I do work at Amazon but under the Web Services team. And he was like, “Oh, so you're in IT?”And I'm like, “No.” [laugh]. “It's actually this big company. That—it's a cloud company.” And they're like, “Oh, okay, okay. Yeah. The cloud. Got it.” [laugh]. So, it's interesting talking to people about, “I work at Amazon.” “Oh, my son works at Amazon distribution center,” blah, blah, blah. It's like, cool. “I know about that, but very little. I do this.”Corey: Your son works in Amazon distribution center. Is he a robot? Is normally my next question on that? Yeah. That's neither here nor there.So, you and I started talking a while back. We both write newsletters that go to a somewhat similar audience. You write DevOps'ish. I write Last Week in AWS. And recently, you also have started EKS News because, yeah, the one thing I look at when I'm doing these newsletters every week is, you know what I want to do? That's right. Write more newsletters.Chris: [laugh].Corey: So, you are just a glutton for punishment? And, yeah, welcome to the addiction, I suppose. How's it been going for you?Chris: It's actually been pretty interesting, right? Like, we haven't pushed it very hard. We're now starting to include it in things. Like we did Container Day; we made sure that EKS news was on the landing page for Container Day at KubeCon EU. And you know, it's kind of just grown organically since then.But it was one of those things where it's like, internally—this happened at Red Hat, right—when I started live streaming at Red Hat, the ultimate goal was to do our product management—like, here's what's new in the next version thing—do those live so anybody can see that at any point in time anywhere on Earth, the second it's available. Similar situation to here. This newsletter actually is generated as part of a report my boss puts together to brief our other DAs—or developer advocates—you know, our solutions architects, the whole nine yards about new EKS features. So, I was like, why can't we just flip that into a weekly newsletter, you know? Like, I can pull from the same sources you can.And what's interesting is, he only does the meeting bi-weekly. So, there's some weeks where it's just all me doing it and he ends up just kind of copying and pasting the newsletter into his document, [laugh] and then adds on for the week. But that report meeting for that team is now getting disseminated to essentially anyone that subscribes to eks.news. Just go to the site, there's a subscribe thing right there. And we've gotten 20 issues in and it's gotten rave reviews, right?Corey: I have been a subscriber for a while. I will say that it has less Chris Short personality—Chris: Mm-hm.Corey: —to it than DevOps'ish does, which I have to assume is by design. A lot of The Duckbill Group's marketing these days is no longer in my voice, rather intentionally, because it turns out that being a sarcastic jackass and doing half-billion dollar AWS contracts can not to be the most congruent thing in the world. So okay, we're slowly ameliorating that. It's professional voice versus snarky voice.Chris: Well, and here's the thing, right? Like, I realized this year with DevOps'ish that, like, if I want to take a week off, I have to do, like, what you did when your child was born. You hired folks to like, do the newsletter for you, or I actually don't do the newsletter, right? It's binary: hire someone else to do it, or don't do it. So, the way I structured this newsletter was that any developer advocate on my team could jump in and take over the newsletter so that, you know, if I'm off that week, or whatever may be happening, I, Chris Short, am not the voice. It is now the entire developer advocate team.Corey: I will challenge you on that a bit. Because it's not Chris Short voice, that's for sure, but it's also not official AWS brand voice either.Chris: No.Corey: It is clearly written by a human being who is used to communicating with the audience for whom it is written. And that is no small thing. Normally, when oh, there's a corporate newsletter; that's just a lot of words to say it's bad. This one is good. I want to be very clear on that.Chris: Yeah, I mean, we have just, like, DevOps'ish, we have sections, just like your newsletter, there's certain sections, so any new, what's new announcements, those go in automatically. So, like, that can get delivered to your inbox every Friday. Same thing with new blog posts about anything containers related to EKS, those will be in there, then Containers from the Couch, our streaming platform, essentially, for all things Kubernetes. Those videos go in.And then there's some ecosystem news as well that I collect and put in the newsletter to give people a broader sense of what's going on out there in Kubernetes-land because let's face it, there's upstream and then there's downstream, and sometimes those aren't in sync, and that's normal. That's how Kubernetes kind of works sometimes. If you're running upstream Kubernetes, you are awesome. I appreciate you, but I feel like that would cause more problems and it's worse sometimes.Corey: Thank you for being the trailblazers. The rest of us can learn from your misfortune.Chris: [laugh]. Yeah, exactly. Right? Like, please file your bugs accordingly. [laugh].Corey: EKS is interesting to me because I don't see a lot of it, which is, probably, going to get a whole lot of, “Wait, what?” Moments because wait, don't you deal with very large AWS bills? And I do. But what I mean by that is that EKS, until you're using its Fargate expression, charges for the control plane, which rounds to no money, and the rest is running on EC2 instances running in a company's account. From the billing perspective, there is no difference between, “We're running massive fleets of EKS nodes.” And, “We're managing a whole bunch of EC2 instances by hand.”And that feels like an interesting allegory for how Kubernetes winds up expressing itself to cloud providers. Because from a billing perspective, it just looks like one big single-tenant application that has some really strange behaviors internally. It gets very chatty across AZs when there's no reason to, and whatnot. And it becomes a very interesting study in how to expose aspects of what's going on inside of those containers and inside of the Kubernetes environment to the cloud provider in a way that becomes actionable. There are no good answers for this yet, but it's something I've been seeing a lot of. Like, “Oh, I thought you'd be running Kubernetes. Oh, wait, you are and I just keep forgetting what I'm looking at sometimes.”Chris: So, that's an interesting point. The billing is kind of like, yeah, it's just compute, right? So—Corey: And my insight into AWS and the way I start thinking about it is always from a billing perspective. That's great. It's because that means the more expensive the services, the more I know about it. It's like, “IAM. What is that?” Like, “Oh, I have no idea. It's free. How important could it be?” Professional advice: do not take that philosophy, ever.Chris: [laugh]. No. Ever. No.Corey: Security: it matters. Oh, my God. It's like you're all stars. Your IAM policy should not be. I digress.Chris: Right. Yeah. Anyways, so two points I want to make real quick on that is, one, we've recently released an open-source project called Carpenter, which is really cool in my purview because it looks at your Kubernetes file and says, “Oh, you want this to run on ARM instance.” And you can even go so far as to say, right, here's my limits, and it'll find an instance that fits those limits and add that to your cluster automatically. Run your pod on that compute as long as it needs to run and then if it's done, it'll downsize—eventually, kind of thing—your cluster.So, you can basically just throw a bunch of workloads at it, and it'll auto-detect what kind of compute you will need and then provision it for you, run it, and then be done. So, that is one-way folks are probably starting to save money running EKS is to adopt Carpenter as your autoscaler as opposed to the inbuilt Kubernetes autoscaler. Because this is instance-aware, essentially, so it can say, like, “Oh, your massive ARM application can run here,” because you know, thank you, Graviton. We have those processors in-house. And you know, you can run your ARM64 instances, you can run all the Intel workloads you want, and it'll right size the compute for your workloads.And I'll look at one container or all your containers, however you want to configure it. Secondly, the good folks over at Kubecost have opencost, which is the open-source version of Kubecost, basically. So, they have a service that you can run in your clusters that will help you say, “Hey, maybe this one notes too heavy; maybe this one notes too light,” and you know, give you some insights into Kubernetes spend that are a little bit more granular as far as usage and things like that go. So, those two projects right there, I feel like, will give folks an optimal savings experience when it comes to Kubernetes. But to your point, it's just compute, right? And that's really how we treat it, kind of, here internally is that it's a way to run… compute, Kubernetes, or ECS, or any of those tools.Corey: A fairly expensive one because ignoring entirely for a second the actual raw cost of compute, you also have the other side of it, which is in every environment, unless you are doing something very strange or pre-funding as a one-person startup in your spare time, your payroll costs will it—should—exceed your AWS bill by a fairly healthy amount. And engineering time is always more expensive than services time. So, for example, looking at EKS, I would absolutely recommend people use that rather than rolling their own because—Chris: Rolling their own? Yeah.Corey: —get out of that engineering space where your time is free. I assure you from a business context, it is not. So, there's always that question of what you can do to make things easier for people and do more of the heavy lifting.Chris: Yeah, and to your rather cheeky point that there's 17 ways to run a container on AWS, it is answering that question, right? Like those 17 ways, like, how much of this do you want to run yourself, you could run EKS distro on EC2 instances if you want full control over your environment.Corey: And then run IoT Greengrass core on top within that cluster—Chris: Right.Corey: So, I can run my own Lambda function runtime, so I'm not locked in. Also, DynamoDB local so I'm not locked into AWS. At which point I have gone so far around the bend, no one can help me.Chris: Well—Corey: Pro tip, don't do that. Just don't do that.Chris: But to your point, we have all these options for compute, and specifically containers because there's a lot of people that want to granularly say, “This is where my engineering team gets involved. Everything else you handle.” If I want EKS on Spot Instances only, you can do that. If you want EKS to use Carpenter and say only run ARM workloads, you can do that. If you want to say Fargate and not have anything to manage other than the container file, you can do that.It's how much does your team want to manage? That's the customer obsession part of AWS coming through when it comes to containers is because there's so many different ways to run those workloads, but there's so many different ways to make sure that your team is right-sized, based off the services you're using.Corey: I do want to change gears a bit here because you are mostly known for a couple of things: the DevOps'ish newsletter because that is the oldest and longest thing you've been doing the time that I've known you; EKS, obviously. But when prepping for this show, I discovered you are now co-chair of the OpenGitOps project.Chris: Yes.Corey: So, I have heard of GitOps in the context of, “Oh, it's just basically your CI/CD stuff is triggered by Git events and whatnot.” And I'm sitting here going, “Okay, so from where you're sitting, the two best user interfaces in the world that you have discovered are YAML and Git.” And I just have to start with the question, “Who hurt you?”Chris: [laugh]. Yeah, I share your sentiment when it comes to Git. Not so much with YAML, but I think it's because I'm so used to it. Maybe it's Stockholm Syndrome, maybe the whole YAML thing. I don't know.Corey: Well, it's no XML. We'll put it that way.Chris: Thankfully, yes because if it was, I would have way more, like, just template files laying around to build things. But the—Corey: And rage. Don't forget rage.Chris: And rage, yeah. So, GitOps is a little bit more than just Git in IaC—infrastructure as Code. It's more like Justin Garrison, who's also on my team, he calls it infrastructure software because there's four main principles to GitOps, and if you go to opengitops.dev, you can see them. It's version one.So, we put them on the website, right there on the page. You have to have a declared state and that state has to live somewhere. Now, it's called GitOps because Git is probably the most full-featured thing to put your state in, but you could use an S3 bucket and just version it, for example. And make it private so no one else can get to it.Corey: Or you could use local files: copy-of-copy-of-this-thing-restored-parentheses-use-this-one-dot-final-dot-doc-dot-zip. You know, my preferred naming convention.Chris: Ah, yeah. Wow. Okay. [laugh]. Yeah.Corey: Everything I touch is terrifying.Chris: Yes. Geez, I'm sorry. So first, it's declarative. You declare your state. You store it somewhere. It's versioned and immutable, like I said. And then pulled automatically—don't focus so much on pull—but basically, software agents are applying the desired state from source. So, what does that mean? When it's—you know, the fourth principle is implemented, continuously reconciled. That means those software agents that are checking your desired state are actually putting it back into the desired state if it's out of whack, right? So—Corey: You're talking about agents running it persistently on instances, validating—Chris: Yes.Corey: —a checkpoint on a cron. How is this meaningfully different than a Puppet agent running in years past? Having spent I learned to speak publicly by being a traveling trainer for Puppet; same type of model, and in fact, when I was at Pinterest, we wound up having a fair bit—like, that was their entire model, where they would have—the Puppet's code would live in an S3 bucket that was then copied down, I believe, via Git, and then applied to the instance on a schedule. Like, that sounds like this was sort of a early days GitOps.Chris: Yeah, exactly. Right? Like so it's, I like to think of that as a component of GitOps, right? DevOps, when you talk about DevOps in general, there's a lot of stuff out there. There's a lot of things labeled DevOps that maybe are, or maybe aren't sticking to some of those DevOps core things that make you great.Like the stuff that Nicole Forsgren writes about in books, you know? Accelerate is on my desk for a reason because there's things that good, well-managed DevOps practices do. I see GitOps as an actual implementation of DevOps in an open-source manner because all the tooling for GitOps these days is open-source and it all started as open-source. Now, you can get, like, Flux or Argo—Argo, specifically—there's managed services out there for it, you can have Flux and not maintain it, through an add-on, on EKS for example, and it will reconcile that state for you automatically. And the other thing I like to say about GitOps, specifically, is that it moves at the speed of the Kubernetes Audit Log.If you've ever looked at a Kubernetes audit log, you know it's rather noisy with all these groups and versions and kinds getting thrown out there. So, GitOps will say, “Oh, there's an event for said thing that I'm supposed to be watching. Do I need to change anything? Yes or no? Yes? Okay, go.”And the change gets applied, or, “Hey, there's a new Git thing. Pull it in. A change has happened inGit I need to update it.” You can set it to reconcile on events on time. It's like a cron or it's like an event-driven architecture, but it's combined.Corey: How does it survive the stake through the heart of configuration management? Because before I was doing all this, I wasn't even a T-shaped engineer: you're broad across a bunch of things, but deep in one or two areas, and one of mine was configuration management. I wrote part of SaltStack, once upon a time—Chris: Oh.Corey: —due to a bunch of very strange coincidences all hitting it once, like, I taught people how to use Puppet. But containers ultimately arose and the idea of immutable infrastructure became a thing. And these days when we were doing full-on serverless, well, great, I just wind up deploying a new code bundle to the Lambdas function that I wind up caring about, and that is a immutable version replacement. There is no drift because there is no way to log in and change those things other than through a clear deployment of this as the new version that goes out there. Where does GitOps fit into that imagined pattern?Chris: So, configuration management becomes part of your approval process, right? So, you now are generating an audit log, essentially, of all changes to your system through the approval process that you set up as part of your, how you get things into source and then promote that out to production. That's kind of the beauty of it, right? Like, that's why we suggest using Git because it has functions, like, requests and issues and things like that you can say, “Hey, yes, I approve this,” or, “Hey, no, I don't approve that. We need changes.” So, that's kind of natively happening with Git and, you know, GitLab, GitHub, whatever implementation of Git. There's always, kind of—Corey: Uh, JIF-ub is, I believe, the pronunciation.Chris: JIF-ub? Oh.Corey: Yeah. That's what I'm—Chris: Today, I learned. Okay.Corey: Exactly. And that's one of the things that I do for my lasttweetinaws.com Twitter client that I build—because I needed it, and if other people want to use it, that's great—that is now deployed to 20 different AWS commercial regions, simultaneously. And that is done via—because it turns out that that's a very long to execute for loop if you start down that path—Chris: Well, yeah.Corey: I wound up building out a GitHub Actions matrix—sorry a JIF-ub—actions matrix job that winds up instantiating 20 parallel builds of the CDK deploy that goes out to each region as expected. And because that gets really expensive with native GitHub Actions runners for, like, 36 cents per deploy, and I don't know how to test my own code, so every time I have a typo, that's another quarter in the jar. Cool, but that was annoying for me so I built my own custom runner system that uses Lambda functions as runners running containers pulled from ECR that, oh, it just runs in parallel, less than three minutes. Every time I commit something between I press the push button and it is out and running in the wild across all regions. Which is awesome and also terrifying because, as previously mentioned, I don't know how to test my code.Chris: Yeah. So, you don't know what you're deploying to 20 regions sometime, right?Corey: But it also means I have a pristine, re-composable build environment because I can—Chris: Right.Corey: Just automatically have that go out and the fact that I am making a—either merging a pull request or doing a direct push because I consider main to be my feature branch as whenever something hits that, all the automation kicks off. That was something that I found to be transformative as far as a way of thinking about this because I was very tired of having to tweak my local laptop environment to, “Oh, you didn't assume the proper role and everything failed again and you broke it. Good job.” It wound up being something where I could start developing on more and more disparate platforms. And it finally is what got me away from my old development model of everything I build is on an EC2 instance, and that means that my editor of choice was Vim. I use the VS Code now for these things, and I'm pretty happy with it.Chris: Yeah. So, you know, I'm glad you brought up CDK. CDK gives you a lot of the capabilities to implement GitOps in a way that you could say, like, “Hey, use CDK to declare I need four Amazon EKS clusters with this size, shape, and configuration. Go.” Or even further, connect to these EKS clusters to RDS instances and load balancers and everything else.But you put that state into Git and then you have something that deploys that automatically upon changes. That is infrastructure as code. Now, when you say, “Okay, main is your feature branch,” you know, things happen on main, if this were running in Kubernetes across a fleet of clusters or the globe-wide in 20 regions, something like Flux or Argo would kick in and say, “There's been a change to source, main, and we need to roll this out.” And it'll start applying those changes. Now, what do you get with GitOps that you don't get with your configuration?I mean, can you rollback if you ever have, like, a bad commit that's just awful? I mean, that's really part of the process with GitOps is to make sure that you can, A, roll back to the previous good state, B, roll forward to a known good state, or C, promote that state up through various environments. And then having that all done declaratively, automatically, and immutably, and versioned with an audit log, that I think is the real power of GitOps in the sense that, like, oh, so-and-so approve this change to security policy XYZ on this date at this time. And that to an auditor, you just hand them a log file on, like, “Here's everything we've ever done to our system. Done.” Right?Like, you could get to that state, if you want to, which I think is kind of the idea of DevOps, which says, “Take all these disparate tools and processes and procedures and culture changes”—culture being the hardest part to adopt in DevOps; GitOps kind of forces a culture change where, like, you can't do a CAB with GitOps. Like, those two things don't fly. You don't have a configuration management database unless you absolutely—Corey: Oh, you CAB now but they're all the comments of the pull request.Chris: Right. Exactly. Like, don't push this change out until Thursday after this other thing has happened, kind of thing. Yeah, like, that all happens in GitHub. But it's very democratizing in the sense that people don't have to waste time in an hour-long meeting to get their five minutes in, right?Corey: DoorDash had a problem. As their cloud-native environment scaled and developers delivered new features, their monitoring system kept breaking down. In an organization where data is used to make better decisions about technology and about the business, losing observability means the entire company loses their competitive edge. With Chronosphere, DoorDash is no longer losing visibility into their applications suite. The key? Chronosphere is an open-source compatible, scalable, and reliable observability solution that gives the observability lead at DoorDash business, confidence, and peace of mind. Read the full success story at snark.cloud/chronosphere. That's snark.cloud slash C-H-R-O-N-O-S-P-H-E-R-E.Corey: So, would it be overwhelmingly cynical to suggest that GitOps is the means to implement what we've all been pretending to have implemented for the last decade when giving talks at conferences?Chris: Ehh, I wouldn't go that far. I would say that GitOps is an excellent way to implement the things you've been talking about at all these conferences for all these years. But keep in mind, the technology has changed a lot in the, what 11, 12 years of the existence of DevOps, now. I mean, we've gone from, let's try to manage whole servers immutably to, “Oh, now we just need to maintain an orchestration platform and run containers.” That whole compute interface, you go from SSH to a Docker file, that's a big leap, right?Like, you don't have bespoke sysadmins; you have, like, a platform team. You don't have DevOps engineers; they're part of that platform team, or DevOps teams, right? Like, which was kind of antithetical to the whole idea of DevOps to have a DevOps team. You know, everybody's kind of in the same boat now, where we see skill sets kind of changing. And GitOps and Kubernetes-land is, like, a platform team that manages the cluster, and its state, and health and, you know, production essentially.And then you have your developers deploying what they want to deploy in when whatever namespace they've been given access to and whatever rights they have. So, now you have the potential for one set of people—the platform team—to use one set of GitOps tooling, and your applications teams might not like that, and that's fine. They can have their own namespaces with their own tooling in it. Like, Argo, for example, is preferred by a lot of developers because it has a nice UI with green and red dots and they can show people and it looks nice, Flux, it's command line based. And there are some projects out there that kind of take the UI of Argo and try to run Flux underneath that, and those are cool kind of projects, I think, in my mind, but in general, right, I think GitOps gives you the choice that we missed somewhat in DevOps implementations of the past because it was, “Oh, we need to go get cloud.” “Well, you can only use this cloud.” “Oh, we need to go get this thing.” “Well, you can only use this thing in-house.”And you know, there's a lot of restrictions sometimes placed on what you can use in your environment. Well, if your environment is Kubernetes, how do you restrict what you can run, right? Like you can't have an easily configured say, no open-source policy if you're running Kubernetes. [laugh] so it becomes, you know—Corey: Well, that doesn't stop some companies from trying.Chris: Yeah, that's true. But the idea of, like, enabling your developers to deploy at will and then promote their changes as they see fit is really the dream of DevOps, right? Like, same with production and platform teams, right? I want to push my changes out to a larger system that is across the globe. How do I do that? How do I manage that? How do I make sure everything's consistent?GitOps gives you those ways, with Kubernetes native things like customizations, to make consistent environments that are robust and actually going to be reconciled automatically if someone breaks the glass and says, “Oh, I need to run this container immediately.” Well, that's going to create problems because it's deviated from state and it's just that one region, so we'll put it back into state.Corey: It'll be dueling banjos, at some point. You'll try and doing something manually, it gets reverted automatically. I love that pattern. You'll get bored before the computer does, always.Chris: Yeah. And GitOps is very new, right? When you think about the lifetime of GitOps, I think it was coined in, like, 2018. So, it's only four years old, right? When—Corey: I prefer it to ChatOps, at least, as far as—Chris: Well, I mean—Corey: —implementation and expression of the thing.Chris: —ChatOps was a way to do DevOps. I think GitOps—Corey: Well, ChatOps is also a way to wind up giving whoever gets access to your Slack workspace root in production.Chris: Mmm.Corey: But that's neither here nor there.Chris: Mm-hm.Corey: It's yeah, we all like to pretend that's not a giant security issue in our industry, but that's a topic for another time.Chris: Yeah. And that's why, like, GitOps also depends upon you having good security, you know, and good authorization and approval processes. It enforces that upon—Corey: Yeah, who doesn't have one of those?Chris: Yeah. If it's a sole operation kind of deal, like in your setup, your case, I think you kind of got it doing right, right? Like, as far as GitOps goes—Corey: Oh, to be clear, we are 11 people and we do have dueling pull requests and all the rest.Chris: Right, right, right.Corey: But most of the stuff I talk about publicly is not our production stuff, so it really is just me. Just as a point of clarity there. I've n—the 11 people here do not all—the rest of you don't just sit there and clap as I do all the work.Chris: Right.Corey: Most days.Chris: No, I'm sure they don't. I'm almost certain they don't clap… for you. I mean, they would—Corey: No. No, they try and talk me out of it in almost every case.Chris: Yeah, exactly. So, the setup that you, Corey Quinn, have implemented to deploy these 20 regions is kind of very GitOps-y, in the sense that when main changes, it gets updated. Where it's not GitOps-y is what if the endpoint changes? Does it get reconciled? That's the piece you're probably missing is that continuous reconciliation component, where it's constantly checking and saying, “This thing out there is deployed in the way I want it. You know, the way I declared it to be in my source of truth.”Corey: Yeah, when you start having other people getting involved, there can—yeah, that's where regressions enter. And it's like, “Well, I know where things are so why would I change the endpoint?” Yeah, it turns out, not everyone has the state of the entire application in their head. Ideally it should live in—Chris: Yeah. Right. And, you know—Corey: —you know, Git or S3.Chris: —when I—yeah, exactly. When I think about interactions of the past coming out as a new DevOps engineer to work with developers, it's always been, will developers have access to prod or they don't? And if you're in that environment with—you're trying to run a multi-billion dollar operation, and your devs have direct—or one Dev has direct access to prod because prod is in his brain, that's where it's like, well, now wait a minute. Prod doesn't have to be only in your brain. You can put that in the codebase and now we know what is in your brain, right?Like, you can almost do—if you document your code, well, you can have your full lifecycle right there in one place, including documentation, which I think is the best part, too. So, you know, it encourages approval processes and automation over this one person has an entire state of the system in their head; they have to go in and fix it. And what if they're not on call, or in Jamaica, or on a cruise ship somewhere kind of thing? Things get difficult. Like, for example, I just got back from vacation. We were so far off the grid, we had satellite internet. And let me tell you, it was hard to write an email newsletter where I usually open 50 to 100 tabs.Corey: There's a little bit of internet out Californ-ie way.Chris: [laugh].Corey: Yeah it's… it's always weird going from, like, especially after pandemic; I have gigabit symmetric here and going even to re:Invent where I'm trying to upload a bunch of video and whatnot.Chris: Yeah. Oh wow.Corey: And the conference WiFi was doing its thing, and well, Verizon 5G was there but spotty. And well, yeah. Usual stuff.Chris: Yeah. It's amazing to me how connectivity has become so ubiquitous.Corey: To the point where when it's not there anymore, it's what do I do with myself? Same story about people pushing back against remote development of, “Oh, I'm just going to do it all on my laptop because what happens if I'm on a plane?” It's, yeah, the year before the pandemic, I flew 140,000 miles domestically and I was almost never hamstrung by my ability to do work. And my only local computer is an iPad for those things. So, it turns out that is less of a real world concern for most folks.Chris: Yeah I actually ordered the components to upgrade an old Nook that I have here and turn it into my, like, this is my remote code server, that's going to be all attached to GitHub and everything else. That's where I want to be: have Tailscale and just VPN into this box.Corey: Tailscale is transformative.Chris: Yes. Tailscale will change your life. That's just my personal opinion.Corey: Yep.Chris: That's not an AWS opinion or anything. But yeah, when you start thinking about your network as it could be anywhere, that's where Tailscale, like, really shines. So—Corey: Tailscale makes the internet work like we all wanted to believe that it worked.Chris: Yeah. And Wireguard is an excellent open-source project. And Tailscale consumes that and puts an amazingly easy-to-use UI, and troubleshooting tools, and routing, and all kinds of forwarding capabilities, and makes it kind of easy, which is really, really, really kind of awesome. And Tailscale and Kubernetes—Corey: Yeah, ‘network' and ‘easy' don't belong in the same sentence, but in this case, they do.Chris: Yeah. And trust me, the Kubernetes story in Tailscale, there is a lot of there. I understand you might want to not open ports in your VPC, maybe, but if you use Tailscale, that node is just another thing on your network. You can connect to that and see what's going on. Your management cluster is just another thing on the network where you can watch the state.But it's all—you're connected to it continuously through Tailscale. Or, you know, it's a much lighter weight, kind of meshy VPN, I would say, if I had to sum it up in one sentence. That was not on our agenda to talk about at all. Anyways. [laugh]Corey: No, no. I love how many different topics we talk about on these things. We'll have to have you back soon to talk again. I really want to thank you for being so generous with your time. If people want to learn more about what you're up to and how you view these things, where can they find you?Chris: Go to ChrisShort.net. So, Chris Short—I'm six-four so remember, it's Short—dot net, and you will find all the places that I write, you can go to devopsish.com to subscribe to my newsletter, which goes out every week. This year. Next year, there'll be breaks. And then finally, if you want to follow me on Twitter, Chris Short: at @ChrisShort on Twitter. All one word so you see two s's. Like, it's okay, there's two s's there.Corey: Links to all of that will of course be in the show notes. It's easier for people to do the clicky-clicky thing as a general rule.Chris: Clicky things are easier than the wordy things, yes.Corey: Says the Kubernetes guy.Chris: Yeah. Says the Kubernetes guy. Yeah, you like that, huh? Like I said, Argo gives you a UI. [laugh].Corey: Thank you [laugh] so much for your time. I really do appreciate it.Chris: Thank you. This has been fun. If folks have questions, feel free to reach out. Like, I am not one of those people that hides behind a screen all day and doesn't respond. I will respond to you eventually.Corey: I'm right here, Chris. Come on, come on. You're calling me out in front of myself. My God.Chris: Egh. It might take a day or two, but I will respond. I promise.Corey: Thanks again for your time. This has been Chris Short, senior developer advocate at AWS. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice and if it's YouTube, click the thumbs-up button. Whereas if you've hated this podcast, same thing, smash the buttons five-star review and leave an insulting comment that is written in syntactically correct YAML because it's just so easy to do.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Valley Presbyterian Church
All the Light: Colossians 1

Valley Presbyterian Church

Play Episode Listen Later Jul 11, 2022 10:31


Naomi Kinsman offers this morning's message on Colossians 1:1-14. Naomi is our Curator for NextGen at VPC.

radio-immo.fr, l'information immobilière
Le DPE collectif - Les As de la Copro

radio-immo.fr, l'information immobilière

Play Episode Listen Later Jul 7, 2022 51:13


Les As de la copro », le retour. VPC, location touristique, DPE collectif, cession de cabinet, prêts bancaires collectifs, garantie financière du syndic, fiche récapitulative du contrat type. On fait le point et on débat. SITES INTERNET : https://angc-association.fr/ https://fr.foncia.com/ https://www.ca-immobilier.fr/ https://www.lagraulet-avocat.fr/ https://oralia.fr/

Screaming in the Cloud
Granted, Common Fate, and AWS Functionality with Chris Norman

Screaming in the Cloud

Play Episode Listen Later Jun 30, 2022 33:34


About ChrisChris is a robotics engineer turned cloud security practitioner. From building origami robots for NASA, to neuroscience wearables, to enterprise software consulting, he is a passionate builder at heart. Chris is a cofounder of Common Fate, a company with a mission to make cloud access simple and secure.Links: Common Fate: https://commonfate.io/ Granted: https://granted.dev Twitter: https://twitter.com/chr_norm TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Let's face it, on-call firefighting at 2am is stressful! So there's good news and there's bad news. The bad news is that you probably can't prevent incidents from happening, but the good news is that incident.io makes incidents less stressful and a lot more valuable. incident.io is a Slack-native incident management platform that allows you to automate incident processes, focus on fixing the issues and learn from incident insights to improve site reliability and fix your vulnerabilities. Try incident.io, recover faster and sleep more.Corey: This episode is sponsored in part by Honeycomb. When production is running slow, it's hard to know where problems originate. Is it your application code, users, or the underlying systems? I've got five bucks on DNS, personally. Why scroll through endless dashboards while dealing with alert floods, going from tool to tool to tool that you employ, guessing at which puzzle pieces matter? Context switching and tool sprawl are slowly killing both your team and your business. You should care more about one of those than the other; which one is up to you. Drop the separate pillars and enter a world of getting one unified understanding of the one thing driving your business: production. With Honeycomb, you guess less and know more. Try it for free at honeycomb.io/screaminginthecloud. Observability: it's more than just hipster monitoring.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. It doesn't matter where you are on your journey in cloud—you could never have heard of Amazon the bookstore—and you encounter AWS and you spin up an account. And within 20 minutes, you will come to the realization that everyone in this space does. “Wow, logging in to AWS absolutely blows goats.”Today, my guest, obviously had that reaction, but unlike most people I talked to, decided to get up and do something about it. Chris Norman is the co-founder of Common Fate and most notably to how I know him is one of the original authors of the tool, Granted. Chris, thank you so much for joining me.Chris: Hey, Corey, thank you for having me.Corey: I have done podcasts before; I have done a blog post on it; I evangelize it on Twitter constantly, and even now, it is challenging in a few ways to explain holistically what Granted is. Rather than trying to tell your story for you, when someone says, “Oh, Granted, that seems interesting and impossible to Google for in isolation, so therefore, we know it's going to be good because all the open-source projects with hard to find names are,” what is Granted and what does it do?Chris: Granted is a command-line tool which makes it really easy for you to get access and assume roles when you're working with AWS. For me, when I'm using Granted day-to-day, I wake up, go to my computer—I'm working from home right now—crack open the MacBook and I log in and do some development work. I'm going to go and start working in the cloud.Corey: Oh, when I start first thing in the morning doing development work and logging into the cloud, I know. All right, I'm going to log in to AWS and now I know that my day is going downhill from here.Chris: [laugh]. Exactly, exactly. I think maybe the best days are when you don't need to log in at all. But when you do, I go and I open my terminal and I run this command. Using Granted, I ran this assume command and it authenticates me with single-sign-on into AWS, and then it opens up a console window in a particular account.Now, you might ask, “Well, that's a fairly standard thing.” And in fact, that's probably the way that the console and all of the tools work by default with AWS. Why do you need a third-party tool for this?Corey: Right. I've used a bunch of things that do varying forms of this and unlike Granted, you don't see me gushing about them. I want to be very clear, we have no business relationship. You're not sponsoring anything that I do. I'm not entirely clear on what your day job entails, but I have absolutely fallen in love with the Granted tool, which is why I'm dragging you on to this show, kicking and screaming, mostly to give me an excuse to rave about it some more.Chris: [laugh]. Exactly. And thank you for the kind words. And I'd say really what makes it special or why I've been so excited to be working on it is that it makes this access, particularly when you're working with multiple accounts, really, really easy. So, when I run assume and I open up that console window, you know, that's all fine and that's very similar to how a lot of the other tools and projects that are out there work, but when I want to open that second account and that second console window, maybe because I'm looking at like a development and a staging account at the same time, then Granted allows me to view both of those simultaneously in my browser. And we do that using some platform sort of tricks and building into the way that the browser works.Corey: Honestly, one of the biggest differences in how you describe what Granted is and how I view it is when you describe it as a CLI application because yes, it is that, but one of the distinguishing characteristics is you also have a Firefox extension that winds up leveraging the multi-container functionality extension that Firefox has. So, whenever I wind up running a single command—assume with a-c' flag, then I give it the name of my AWS profile, it opens the web console so I can ClickOps my heart's content inside of a tab that is locked to a container, which means I can have one or two or twenty different AWS accounts and/or regions up running simultaneously side-by-side, which is basically impossible any other way that I've ever looked at it.Chris: Absolutely, yeah. And that's, like, the big differentiating factor right now between Granted and between this sort of default, the native experience, if you're just using the AWS command line by itself. With Granted, you can—with these Firefox containers, all of your cookies, your profile, everything is all localized into that one container. It's actually it's a privacy features that are built into Firefox, which keeps everything really separate between your different profiles. And what we're doing with Granted is that we make it really easy to open a specific profiles that correspond with different AWS profiles that you're using.So, you'd have one which could be your development account, one which could be production or staging. And you can jump between these and navigate between them just as separate tabs in your browser, which is a massive improvement over, you know, what I've previously had to use in the past.Corey: The thing that really just strikes me about this is first, of course, the functionality and the rest, so I saw this—I forget how I even came across it—and immediately I started using it. On my Mac, it was great. I started using it when I was on the road, and it was less great because you built this thing in Go. It can compile and install on almost anything, but there were some assumptions that you had built into this in its early days that did not necessarily encompass all of the use cases that I use. For example, it hadn't really occurred to you that some lunatic would try and only use an iPad when they're on the road, so they have to be able to run this to get federated login links via SSHing into an EC2 instance running somewhere and not have it open locally.You seemed almost taken aback when I brought it up. Like, “What lunatic would do that?” Like, “Hi, I'm such a lunatic. Let's talk about this.” And it does that now, and it's awesome. It does seem to me though, and please correct me if I'm wrong on this assumption slash assessment that this is first and foremost aimed at desktop users, specifically people running Mac on the desktop, is that the genesis of it?Chris: It is indeed. And I think part of the cause behind that is that we originally built a tool for ourselves. And as we were building things and as we were working using the cloud, we were running things—you know, we like to think that we're following best practices when we're using AWS, and so we'd set up multiple accounts, we'd have a special account for development, a separate one for staging, a separate one for production, even internal tools that we would build, we would go and spin up an individual account for those. And then you know, we had lots of accounts. and to go and access those really easily was quite difficult.So, we definitely, we built it for ourselves first and I think that that's part of when we released it, it actually a little bit of cause for some of the initial problems. And some of the feedback that we had was that it's great to build tools for yourself, but when you're working in open-source, there's a lot of different diversity with how people are using things.Corey: We take different approaches. You want to try to align with existing best practices, whereas I am a loudmouth white guy who works in tech. So, what I do definitionally becomes a best practice in the ecosystem. It's easier to just comport with the ones that are already existing that smart people put together rather than just trying to competence your way through it, so you took a better path than I did.But there's been a lot of evolution to Granted as I've been using it for a while. I did a whole write-up on it and that got a whole bunch of eyes onto the project, which I can now admit was a nefarious plan on my part because popping into your community Slack and yelling at you for features I want was all well and good, but let's try and get some people with eyes on this who are smarter than me—which is not that high of a bar when it comes to SSO, and IAM, and federated login, and the rest—and they can start finding other enhancements that I'll probably benefit from. And sure enough, that's exactly what happened. My sneaky plan has come to fruition. Thanks for being a sucker, I guess. I mean—[laugh] it worked. I'm super thrilled by the product.Chris: [laugh]. I guess it's a great thing I think that the feedback and particularly something that's always been really exciting is just seeing new issues come through on GitHub because it really shows the kinds of interesting use cases and the kinds of interesting teams and companies that are using Granted to make their lives a little bit easier.Corey: When I go to the website—which again is impossible to Google—the website for those wondering is granted.dev. It's short, it's concise, I can say it on a podcast and people automatically know how to spell it. But at the top of the website—which is very well done by the way—it mentions that oh, you can, “Govern access to breakglass roles with Common Fate Cloud,” and it also says in the drop shadow nonsense thing in the upper corner, “Brought to you by Common Fate,” which is apparently the name of your company.So, the question I'll get to in a second is what does your company do, but first and foremost, is this going to be one of those rug-pull open-source projects where one day it's, “Oh, you want to log into your AWS accounts? Insert quarter to continue.” I'm mostly being a little over the top with that description, but we've all seen things that we love turn into molten garbage. What is the plan around this? Are you about to ruin this for the rest of us once you wind up raising a round or something? What's the deal?Chris: Yeah, it's a great question, Corey. And I think that to a degree, releasing anything like this that sits in the access workflow and helps you assume roles and helps you day-to-day, you know, we have a responsibility to uphold stability and reliability here and to not change things. And I think part of, like, not changing things includes not [laugh] rug-pulling, as you've alluded to. And I think that for some companies, it ends up that open-source becomes, like, a kind of a lead-generation tool, or you end up with, you know, now finally, let's go on add another login so that you have to log into Common Fate to use Granted. And I think that, to be honest, a tool like this where it's all about improving the speed of access, the incentives for us, like, it doesn't even make sense to try and add another login for to try to get people to, like, to say, login to Common Fate because that would make your signing process for AWS take even longer than it already does.Corey: Yeah, you decided that you know, what's the biggest problem? Oh, you can sleep at night, so let's go ahead and make it even worse, by now I want you to be this custodian of all my credentials to log into all of my accounts. And now you're going to be critical path, so if you're down, I'm not able to log into anything. And oh, by the way, I have to trust you with full access to my bank stuff. I just can't imagine that is a direction that you would be super excited about diving head-first into.Chris: No, no. Yeah, certainly not. And I think that the, you know, building anything in this space, and with what we're doing with Common Fate, you know, we're building a cloud platform to try to make IAM a little bit easier to work with, but it's really sensitive around granting any kind of permission and I think that you really do need that trust. So, trying to build trust, I guess, with our open-source projects is really important for us with Granted and with this project, that it's going to continue to be reliable and continue to work as it currently does.Corey: The way I see it, one of the dangers of doing anything that is particularly open-source—or that leans in the direction of building in Amazon's ecosystem—it leads to the natural question of, well, isn't this just going to be some people say stolen—and I don't think those people understand how open-source works—by AWS themselves? Or aren't they going to build something themselves at AWS that's going to wind up stomping this thing that you've built? And my honest and remarkably cynical answer is that, “You have built a tool that is a joy to use, that makes logging into AWS accounts streamlined and efficient in a variety of different patterns. Does that really sound like something AWS would do?” And followed by, “I wish they would because everyone would benefit from that rising tide.”I have to be very direct and very clear. Your product should not exist. This should be something the provider themselves handles. But nope. Instead, it has to exist. And while I'm glad it does, I also can't shake the feeling that I am incredibly annoyed by the fact that it has to.Chris: Yeah. Certainly, certainly. And it's something that I think about a little bit. I like to wonder whether there's maybe like a single feature flag or some single sort of configuration setting in AWS where they're not allowing different tabs to access different accounts, they're not allowing this kind of concurrent access. And maybe if we make enough noise about Granted, maybe one of the engineers will go and flick that switch and they'll just enable it by default.And then Granted itself will be a lot less relevant, but for everybody who's using AWS, that'll be a massive win because the big draw of using Granted is mainly just around being able to access different accounts at the same time. If AWS let you do that out of the box, hey, that would be great and, you know, I'd have a lot less stuff to maintain.Corey: Originally, I had you here to talk about Granted, but I took a glance at what you're actually building over at Common Fate and I'm about to basically hijack slash derail what probably is going to amount the rest of this conversation because you have a quick example on your site for by developers, for developers. You show a quick Python script that tries to access a S3 bucket object and it's denied. You copy the error message, you paste it into what you're building over a Common Fate, and in return, it's like, “Oh. Yeah, this is the policy that fixes it. Do you want us to apply it for you?”And I just about fell out of my chair because I have been asking for this explicit thing for a very long time. And AWS doesn't do it. Their IAM access analyzer claims to. Like, “Oh, just go look at CloudTrail and see what permissions it uses and we'll build a policy to scope it down.” “Okay. So, it's S3 access. Fair enough. To what object or what bucket?” “Guess,” is what it tells you there.And it's, this is crap. Who thinks this is a good user experience? You have built the thing that I wish AWS had built in natively. Because let's be honest here, I do what an awful lot of people do and overscope permissions massively just because messing around with the bare minimum set of permissions in many cases takes more time than building the damn thing in the first place.Chris: Oh, absolutely. Absolutely. And in fact, this—was a few years ago when I was consulting—I had a really similar sort of story where one of the clients that we were working with, the CTO of this company, he was needing to grant us access to AWS and we were needing to build a particular service. And he said, “Okay, can you just let me know the permissions that you will need and I'll go and deploy the role for this.” And I came back and I said, “Wait. I don't even know the permissions that I'm going to need because the damn thing isn't even built yet.”So, we went sort of back and forth around this. And the compromise ended up just being you know, way too much access. And that was sort of part of the inspiration for, you know, really this whole project and what we're building with Common Fate, just trying to make that feedback loop around getting to the right level of permissions a lot faster.Corey: Yeah, I am just so overwhelmingly impressed by the fact that you have built—and please don't take this as a criticism—but a set of very simple tools. Not simple in the terms of, “Oh, that's, like, three lines of bash, and a fool could write that on a weekend.” No. Simple in the sense of it solves a problem elegantly and well and it's straightforward—well, straightforward as anything in the world of access control goes—to wrap your head around exactly what it does. You don't tend to build these things by sitting around a table brainstorming with someone you met at co-founder dating pool or something and wind up figuring out, “Oh, we should go and solve that. That sounds like a billion-dollar problem.”This feels very much like the outcome of when you're sitting around talking to someone and let's start by drinking six beers so we become extraordinarily honest, followed immediately by let's talk about what sucks. What pisses you off the most? It feels like this is sort of the low-hanging fruit of things that upset people when it comes to AWS. I mean, if things had gone slightly differently, instead of focusing on AWS bills, IAM was next on my list of things to tackle just because I was tired of smacking my head into it.This is very clearly a problem space that you folks have analyzed deeply, worked within, and have put a lot of thought into. I want to be clear, I've thrown a lot of feature suggestions that you for Granted from start to finish. But all of them have been around interface stuff and usability and expanding use cases. None of them have been, “Well, that seems screamingly insecure.” Because it hasn't been.Chris: [laugh].Corey: It has been effective, start to finish, I think that from a security posture, you make terrific choices, in many cases better than ones I would have made a starting from scratch myself. Everything that I'm looking at in what you have built is from a position of this is absolutely amazing and it is transformative to my own workflows. Now, how can we improve it?Chris: Mmm. Thank you, Corey. And I'll say as well, maybe around the security angle, that one of the goals with Granted was to try and do things a little bit better than the default way that AWS does them when it comes to security. And it's actually been a bit of a source for challenges with some of the users that we've been working with with Granted because one of the things we wanted to do was encrypt the SSO token. And this is the token that when you sign in to AWS, kind of like, it allows you to then get access to all of the rest of the accounts.So, it's like a pretty—it's a short-lived token, but it's a really sensitive one. And you know, by default, it's just stored in plain text on your disk. So, we dump to a file and, you know, anything that can go and read that, they can go and get it. It's also a little bit hard to revoke and to lock people out. There's not really great workflows around that on AWS's side.So, we thought, “Okay, great. One of the goals for Granted can be that we will go and store this in your keychain in your system and we'll work natively with that.” And that's actually been a cause for a little bit of a hassle for some users, though, because by doing that and by storing all of this information in the keychain, it's actually broken some of the integrations with the rest of the tooling, which kind of expects tokens and things to be in certain places. So, we've actually had to, as part of dealing with that with Granted, we've had to give users the ability to opt out for that.Corey: DoorDash had a problem. As their cloud-native environment scaled and developers delivered new features, their monitoring system kept breaking down. In an organization where data is used to make better decisions about technology and about the business, losing observability means the entire company loses their competitive edge. With Chronosphere, DoorDash is no longer losing visibility into their applications suite. The key? Chronosphere is an open-source compatible, scalable, and reliable observability solution that gives the observability lead at DoorDash business, confidence, and peace of mind. Read the full success story at snark.cloud/chronosphere. That's snark.cloud slash C-H-R-O-N-O-S-P-H-E-R-E.Corey: That's why I find this so, I think, just across the board, fantastic. It's you are very clearly engaged with your community. There's a community Slack that you have set up for this. And I know, I know, too many Slacks; everyone has this problem. This is one of those that is worth hanging in, at least from my perspective, just because one of the problems that you have, I suspect, is on my Mac it's great because I wind up automatically updating it to whatever the most recent one is every time I do a brew upgrade.But on the Linux side of the world, you've discovered what many of us have discovered, and that is that packaging things for Linux is a freaking disaster. The current installation is, “Great. Here's basically a curl bash.” Or, “Here, grab this tarball and install it.” And that's fine, but there's no real way of keeping that updated and synced.So, I was checking the other day, oh wow, I'm something like eight versions behind on this box. But it still just works. I upgraded. Oh, wow. There's new functionality here. This is stuff that's actually really handy. I like this quite a bit. Let's see what else we can do.I'm just so impressed, start to finish, by just how receptive you've been to various community feedbacks. And as well—I want to be very clear on this point, too—I've had folks who actually know what they're doing in an InfoSec sense look at what you're up to, and none of them had any issues of note. I'm sure that they have a pile of things like, with that curl bash, they should really be doing a GPG check. Yes, yes, fine. Whatever. If that's your target threat model, okay, great. Here in reality-land for what I do, this is awesome.And they don't seem to have any problems with, “Oh, yeah. By the way, sending analytics back up”—which, okay, fine, whatever. “And it's not disclosing them.” Okay, that's bad. “And it's including the contents of your AWS credentials.”Ahhhh. I did encounter something that was doing that on the back-end once. [cough]—Serverless Framework—sorry, something caught in my throat for a second.Chris: [laugh].Corey: No faster way I can think of to erode trust in that. But everything you're doing just makes sense.Chris: Oh, I do remember that. And that was a little bit of a fiasco, really, around all of that, right? And it's great to hear actually around that InfoSec folks and security people being, you know, not unhappy, I guess, with a tool like this. It's been interesting for me personally. We've really come from a practitioner's background.You know, I wouldn't call myself a security engineer at all. I would call myself as a sometimes a software developer, I guess. I have been hacking my way around Go and definitely learning a lot about how the cloud has worked over the past seven, eight years or so, but I wouldn't call myself a security engineer, so being very cautious around how all of these things work. And we've really tried to defer to things like the system keychain and defer to things that we know are pretty safe and work.Corey: The thing that I also want to call out as well is that your licensing is under the MIT license. This is not one of those, “Oh, you're required to wind up doing a bunch of branding stuff around it.” And, like some people say, “Oh, you have to own the trademark for all of these things.” I mean, I'm not an expert in international trademark law, let's be very clear, but I also feel that trademarking a term that is already used heavily in the space such as the word ‘Granted,' feels like kind of an uphill battle. And let's further be clear that it doesn't matter what you call this thing.In fact, I will call attention to an oddity that I've encountered a fair bit. After installing it, the first thing you do is you run the command ‘granted.' That sets it up, it lets you configure your browser, what browser you want to use, and it now supports standard out for that headless, EC2 use case. Great. Awesome. Love it. But then the other binary that ships with it is Assume. And that's what I use day-to-day. It actually takes me a minute sometimes when it's been long enough to remember that the tool is called Granted and not Assume what's up with that?Chris: So, part of the challenge that we ran into when we were building the Granted project is that we needed to export some environment variables. And these are really important when you're logging into AWS because you have your access key, your secret key, your session token. All of those, when you run the assume command, need to go into the terminal session that you called it. This doesn't matter so much when you're using the console mode, which is what we mentioned earlier where you can open 100 different accounts if you want to view all of those at the same time in your browser. But if you want to use it in your terminal, we wanted to make it look as really smooth and seamless as possible here.And we were really inspired by this approach from—and I have to shout them out and kind of give credit to them—a tool called AWSume—they're spelled A-W-S-U-M-E—Python-based tool that they don't do as much with single-sign-on, but we thought they had a really nice, like, general approach to the way that they did the scripting and aliasing. And we were inspired by that and part of that means that we needed to have a shell script that called this executable, which then will export things back out into the shell script. And we're doing all this wizardry under the hood to make the user experience really smooth and seamless. Part of that meant that we separated the commands into granted and assume and the other part of the naming for everything is that I felt Granted had a far better ring to it than calling the whole project Assume.Corey: True. And when you say assume, is it AWS or not? I've used the AWSume project before; I've used AWS Vault out of 99 Designs for a while. I've used—for three minutes—the native AWS SSO config, and that is just trash. Again, they're so good at the plumbing, so bad at the porcelain, I think is the criticism that I would levy toward a lot of this stuff.Chris: Mmm.Corey: And it's odd to think there's an entire company built around just smoothing over these sharp, obnoxious edges, but I'm saying this as someone who runs a consultancy and have five years that just fixes the bill for this one company. So, there's definitely a series of cottage industries that spring up around these things. I would be thrilled, on some level, if you wound up being completely subsumed by their product advancements, but it's been 15 years for a lot of this stuff and we're still waiting. My big failure mode that I'm worried about is that you never are.Chris: Yeah, exactly, exactly. And it's really interesting when you think about all of these user experience gaps in AWS being opportunities for, I guess, for companies like us, I think, trying to simplify a lot of the complexity for things. I'm interested in sort of waiting for a startup to try and, like, rebuild the actual AWS console itself to make it a little bit faster and easier to use.Corey: It's been done and attempted a bunch of different times. The problem is that the console is a lot of different things to a lot of different people, and as you step through that, you can solve for your use case super easily. “Yeah, what do I care? I use RDS, I use some VPC nonsense, and I use EC2. The end.” “Great. What about IAM?”Because I promise you're using that whether you know it or not. And okay, well, I'm talking to someone else who's DynamoDB, and someone else is full-on serverless, and someone else has more money than sense, so they mostly use SageMaker, and so on and so forth. And it turns out that you're effectively trying to rebuild everything. I don't know if that necessarily works.Chris: Yeah, and I think that's a good point around maybe while we haven't seen anything around that sort of space so far. You go to the console, and you click down, you see that list of 200 different services and all of those have had teams go and actually, like, build the UI and work with those individual APIs. Yeah.Corey: Any ideas as far as what's next for features on Granted?Chris: I think that, for us, it's continuing to work with everybody who's using it, and with a focus of stability and performance. We actually had somebody in the community raise an issue because they have an AWS config file that's over 7000 lines long. And I kind of pity that person, potentially, for their day-to-day. They must deal with so much complexity. Granted is currently quite slow when the config files get very big. And for us, I think, you know, we built it for ourselves; we don't have that many accounts just yet, so working to try to, like, make it really performant and really reliable is something that's really important.Corey: If you don't mind a feature request while we're at it—and I understand that this is more challenging than it looks like—I'm willing to fund this as a feature bounty that makes sense. And this also feels like it might be a good first project for a very particular type of person, I would love to get tab completion working in Zsh. You have it—Chris: Oh.Corey: For Fish because there's a great library that automatically populates that out, but for the Zsh side of it, it's, “Oh, I should just wind up getting Zsh completion working,” and I fell down a rabbit hole, let me tell you. And I come away from this with the perception of yeah, I'm not going to do it. I have not smart enough to check those boxes. But a lot of people are so that is the next thing I would love to see. Because I will change my browser to log into the AWS console for you, but be damned if I'm changing my shell.Chris: [laugh]. I think autocomplete probably should be higher on our roadmap for the tool, to be honest because it's really, like, a key metric and what we're focusing on is how easy is it to log in. And you know, if you're not too sure what commands to use or if we can save you a few keystrokes, I think that would be the, kind of like, reaching our goals.Corey: From where I'm sitting, you definitely have. I really want to thank you for taking the time to not only build this in the first place, but also speak with me about it. If people want to learn more, where's the best place to find you?Chris: So, you can find me on Twitter, I'm @chr_norm, or you can go and visit granted.dev and you'll have a link to join the Slack community. And I'm very active on the Slack.Corey: You certainly are, although I will admit that I fall into the challenge of being in just the perfectly opposed timezone from you and your co-founder, who are in different time zones to my understanding; one of you is on Australia and one of you was in London; you're the London guy as best I'm aware. And as a result, invariably, I wind up putting in feature requests right when no one's around. And, for better or worse, in the middle of the night is not when I'm usually awake trying to log into AWS. That is Azure time.Chris: [laugh]. Yeah, no, we don't have the US time zone properly covered yet for our community support and help. But we do have a fair bit of the world timezone covered. The rest of the team for Common Fate is all based in Australia and I'm out here over in London.Corey: Yeah. I just want to thank you again, for just being so accessible and, like, honestly receptive to feedback. I want to be clear, there's a way to give feedback and I do strive to do it constructively. I didn't come crashing into your Slack one day with a, “You know what your problem is?” I prefer to take the, “This is awesome. Here's what I think would be even better. Does that make sense?” As opposed to the imperious demands and GitHub issues and whatnot? It's, “I'd love it if it did this thing. Doesn't do this thing. Can you please make it do this thing?” Turns out that's the better way to drive change. Who knew?Chris: Yeah. [laugh]. Yeah, definitely. And I think that one of the things that's been the best around our journey with Granted so far has been listening to feedback and hearing from people how they would like to use the tool. And a big thank you to you, Corey, for actually suggesting changes that make it not only better for you, but better for everybody else who's using Granted.Corey: Well, at least as long as we're using my particular byzantine workload patterns in some way, or shape, or form, I'll hear that. But no, it's been an absolute pleasure and I really want to thank you for your time as well.Chris: Yeah, thank you for having me.Corey: Chris Norman, co-founder of Common Fate, as well as one of the two primary developers originally behind the Granted project that logs you into AWS without you having to lose your mind. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry, incensed, raging comment that talks about just how terrible all of this is once you spend four hours logging into your AWS account by hand first.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Jon Myer Podcast
Ep#72 UK Social Media Finalist Mat Batterbee

Jon Myer Podcast

Play Episode Listen Later Jun 28, 2022 49:30


UK Social Media Awards 2022 Finalist => Best Use of LinkedIN Growth & Awareness And here I thought he was a finalist for best looking beard. Maybe that's another award coming up. You might know him from all his Microsoft Dynamics videos, social posts on LI or you might know him from his beard. It doesn't matter because he's a really cool guy you should follow. Joining me today is Mat “This Guy” Batterbee, Global Head of Business Applications at Ingram Micro. A huge shoutout to our friends at Veeam for sponsoring this episode. Veeam Backup for AWS can Easily protect all your Amazon EC2, RDS, and VPC data. Wait, they protect the VPC data too? Simplify AWS backup and Recovery while ensuring security and compliance. Before I bring Mat onto the show, grab yourself a beer, and a beard, hit that like, sub and notification because we're going to have a good time.

Valley Presbyterian Church
VPC Worship 6.5.2022

Valley Presbyterian Church

Play Episode Listen Later Jun 6, 2022 28:39


Jenny gives a personal sermon about why and how she thinks church matters right now. At the end, the congregation is invited to engage in prayer stations that will invite them to learn more about the strategic goals of VPC, offer prayers for the church and invite ownership into what's possible for the future. Check out more about what Valley Presbyterian is up to at valleypreschurch.org

Kingdom Life
The Meaning of Pentecost

Kingdom Life

Play Episode Listen Later Jun 5, 2022 34:29


Welcome to the Kingdom Life Podcast, a preaching ministry of Venice Presbyterian Church. It is our desire that as you hear these messages preached by senior pastor Chris Romig and the preaching team of VPC you will grow in the understanding of our Triune God and as a disciple of Jesus Chris bringing the good news of the gospel to a world in darkness.

Kingdom Life
God's Company

Kingdom Life

Play Episode Listen Later May 29, 2022 31:03


Welcome to the Kingdom Life Podcast, a preaching ministry of Venice Presbyterian Church. It is our desire that as you hear these messages preached by senior pastor Chris Romig and the preaching team of VPC you will grow in the understanding of our Triune God and as a disciple of Jesus Chris bringing the good news of the gospel to a world in darkness.

Kingdom Life
Behold Your Mother

Kingdom Life

Play Episode Listen Later May 8, 2022 36:49


Welcome to the Kingdom Life Podcast, a preaching ministry of Venice Presbyterian Church. It is our desire that as you hear these messages preached by senior pastor Chris Romig and the preaching team of VPC you will grow in the understanding of our Triune God and as a disciple of Jesus Chris bringing the good news of the gospel to a world in darkness.

Kingdom Life
The Peace that Passes Understanding

Kingdom Life

Play Episode Listen Later May 1, 2022 25:34


Welcome to the Kingdom Life Podcast, a preaching ministry of Venice Presbyterian Church. It is our desire that as you hear these messages preached by senior pastor Chris Romig and the preaching team of VPC you will grow in the understanding of our Triune God and as a disciple of Jesus Chris bringing the good news of the gospel to a world in darkness.

XenTegra - Nutanix Weekly
Nutanix Weekly: Understanding Layer 2 stretch in Nutanix Disaster Recovery-as-a-Service

XenTegra - Nutanix Weekly

Play Episode Listen Later Apr 26, 2022 31:14 Transcription Available


PurposeThe infrastructure for Nutanix Disaster Recovery-as-a-Service (DRaaS) supports a tenant cluster and a production virtual private cloud (VPC) for each customer. Customers generally have production VMs running in their on-premises cluster, which is connected to the DRaaS VPC using an IPsec tunnel. This is used by the DRaaS workflow to replicate on-premises production data.During a disaster recovery situation or while running disaster recovery tests, VMs will failover from on-premises to the DRaaS cluster. When this occurs, all VMs in one subnet of an on-premises network (e.g., 192.168.10.0/24) usually failover to DRaaS. If the customer chooses to preserve the IP, VMs in DRaaS come up with the same IPs as on-premises (e.g., 192.168.10.0/24 network).In this type of disaster recovery situation, customers can choose which critical VMs are replicated to DRaaS. But in those cases, on-premises VMs cannot communicate with VMs in DRaaS. Host: Andy WhitesideCo-host: Harvey GreenCo-host: Jirah Cox