Podcasts about Terraform

UFO Undercover Hosted By Joe Montaldo Tonight's guest SEAN PATRICK HAZLETT Author Tonight Where Talking Aliens and UFOs The Authors Quill normal air time is 3 to 5 pm cst running late to day dto storms SEAN PATRICK HAZLETT Sean Patrick Hazlett is a technologist, finance professional, and science fiction, fantasy, horror, and non-fiction author and editor working in Silicon Valley. He is a winner of the Writers of the Future Contest, and over forty of his short stories have appeared in publications such as The Year's Best Military and Adventure SF, Year's Best Hardcore Horror, Terraform, Galaxy's Edge, Writers of the Future, Grimdark Magazine, Vastarien, and Abyss & Apex, among others. He is also the editor of the Weird World War III and Weird World War IV anthologies. He is an active member of the Horror Writers Association. In graduate school, Sean assisted future Secretary of Defense Ashton B. Carter at the Harvard-Stanford Preventive Defense Project where he developed strategic options for confronting Iran's nuclear program. For this analysis, he won the 2006 Policy Analysis Exercise Award at the Harvard Kennedy School of Government. Sean also worked as an intelligence analyst focusing on strategic war games and simulations for the Pentagon, where he drew on his experience training the US military as a cavalry officer in the US Army during the Iraq and Afghan wars. Sean holds a Master of Business Administration from Harvard Business School, a Master in Public Policy from the Harvard Kennedy School of Government, and bachelor's degrees in History and Electrical Engineering from Stanford University. Enter your email Join my email list By clicking submit, you agree to share your email address with the site owner and Mailchimp to receive marketing, updates, and other emails from the site owner. Use the unsubscribe link in those emails to opt out at any time. SHARE THIS: Click to share on Twitter (Opens in new window)Click to share on Facebook (Opens in new window) SEARCH Search for: Search … AUTHORS Sean Patrick Hazlett Weird World War III & IV: To-Date Promotional Summary: December 11th 2022 Update Weird World War III & IV: To-Date Promotional Summary: October 19th 2022 Update Win 5 Free Books in September 2022 and Other General Updates! Weird World War III & IV: To-Date Promotional Summary: August 5th 2022 Update Weird World War III & IV: To-Date Promotional Summary: June 9th 2022 Update WEIRD WORLD WAR: CHINA Launch Date January 2, 2024 31 days to go. December 2023 M T W T F S S 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 « Dec FOLLOW ME ON TWITTER BLOG STATS 11,311 hits What You Can Do to Save the UAP Amendment with Steve Bassett This One Thing Made Them Ban Red Night Vision Goggles in Vietnam (with Mark Anthony) Clearing Out Negative Energies with a Little Help from the Stars with Jeffrey Seelman The Afterlife Frequency with Mark Anthony MH370 and the Curious Case of Lieutenant Commander Edward C. Lin with Ashton Forbes CATEGORIES Analysis Delaware Editing Hiking Pleasant Hill Promotion Reviews Sasquan Uncategorized Weird World War III Weird World War III Authors Weird World War IV Writing Writing Statistics and Revenue What You Can Do to Save the UAP Amendment with Steve Bassett This One Thing Made Them Ban Red Night Vision Goggles in Vietnam (with Mark Anthony) Clearing Out Negative Energies with a Little Help from the Stars with Jeffrey Seelman The Afterlife Frequency with Mark Anthony MH370 and the Curious Case of Lieutenant Commander Edward C. Lin with Ashton Forbes

Ned Bellavance worked in the world of tech for more than a decade before joining the family profession as an educator. He joins Corey on Screaming in the Cloud to discuss his shift from engineer to educator and content creator, the intricacies of Terraform, and how changes in licensing affect the ecosystem.About NedNed is an IT professional with more than 20 years of experience in the field. He has been a helpdesk operator, systems administrator, cloud architect, and product manager. In 2019, Ned founded Ned in the Cloud LLC to work as an independent educator, creator, and consultant. In this new role, he develops courses for Pluralsight, runs multiple podcasts, writes books, and creates original content for technology vendors.Ned is a Microsoft MVP since 2017 and a HashiCorp Ambassador since 2020.Ned has three guiding principles: embrace discomfort, fail often, and be kind.Links Referenced: Ned in the Cloud: https://nedinthecloud.com/ LinkedIn: https://www.linkedin.com/in/ned-bellavance/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. My guest today is Ned Bellavance, who's the founder and curious human over at Ned in the Cloud. Ned, thank you for joining me.Ned: Yeah, it's a pleasure to be here, Corey.Corey: So, what is Ned in the Cloud? There are a bunch of easy answers that I feel don't give the complete story like, “Oh, it's a YouTube channel,” or, “Oh no, it's the name that you wound up using because of, I don't know, easier to spell the URL or something.” Where do you start? Where do you stop? What are you exactly?Ned: What am I? Wow, I didn't know we were going to get this deep into philosophical territory this early. I mean, you got to ease me in with something. But so, Ned in the Cloud is the name of my blog from back in the days when we all started up a blog and hosted on WordPress and had fun. And then I was also at the same time working for a value-added reseller as a consultant, so a lot of what went on my blog was stuff that happened to me in the world of consulting.And you're always dealing with different levels of brokenness when you go to clients, so you see some interesting things, and I blogged about them. At a certain point, I decided I want to go out and do my own thing, mostly focused on training and education and content creation and I was looking for a company name. And I went through—I had a list of about 40 different names. And I showed them to my wife, and she's like, “Why don't you go Ned in the Cloud? Why are you making this more complicated than it needs to be?”And I said, “Well, I'm an engineer. That is my job, by definition, but you're probably right. I should just go with Ned in the Cloud.” So, Ned in the Cloud now is a company, just me, focused on creating educational content for technical learners on a variety of different platforms. And if I'm delivering educational content, I am a happy human, and if I'm not doing that, I'm probably out running somewhere.Corey: I like that, and I'd like to focus on education first. There are a number of reasons that people will go in that particular direction, but what was it for you?Ned: I think it's kind of in the heritage of my family. It's in my blood to a certain degree because my dad is a teacher, my mom is a teacher-turned-librarian, my sister is a teacher, my wife is a teacher, her mother is a teacher. So, there was definitely something in the air, and I think at a certain point, I was the black sheep in the sense that I was the engineer. Look, this guy over here. And then I ended up deciding that I really liked training people and learning and teaching, and became a teacher of sorts, and then they all went, “Welcome to the fold.”Corey: It's fun when you get to talk to people about the things that they're learning because when someone's learning something I find that it's the time when their mind is the most open. I don't think that that's something that you don't get to see nearly as much once someone already, quote-unquote, “Knows a thing,” because once that happens, why would you go back and learn something new? I have always learned the most—even about things that I've built myself—by putting it in the hands of users and seeing how they honestly sometimes hold it wrong and make mistakes that don't make sense to me, but absolutely make sense to them. Learning something—or rather, teaching something—versus building that thing is very much an orthogonal skill set, and I don't think that there's enough respect given to that understanding.Ned: It's an interesting sphere of people who can both build the thing and then teach somebody else to build the thing because you're right, it's very different skill sets. Being able to teach means that you have to empathize with the human being that you're teaching and understand that their perspective is not yours necessarily. And one of the skills that you build up as an instructor is realizing when you're making a whole bunch of assumptions because you know something really well, and that the person that you're teaching is not going to have that context, they're not going to have all those assumptions baked in, so you have to actually explain that stuff out. Some of my instruction has been purely online video courses through, like, Pluralsight; less of a feedback loop there. I have to publish the entire course, and then I started getting feedback, so I really enjoy doing live trainings as well because then I get the questions right away.And I always insist, like, if I'm delivering a lecture, and you have a question, please don't wait for the end. Please interrupt me immediately because you're going to forget what that question is, you're going to lose your train of thought, and then you're not going to ask it. And the whole class benefits when someone asks a question, and I benefit too. I learn how to explain that concept better. So, I really enjoy the live setting, but making the video courses is kind of nice, too.Corey: I learned to speak publicly and give conference talks as a traveling contract trainer for Puppet years ago, and that was an eye-opening experience, just because you don't really understand something until you're teaching other people how it works. It's how I learned Git. I gave a conference talk that explained Git to people, and that was called a forcing function because I had four months to go to learn this thing I did not fully understand and welp, they're not going to move the conference for me, so I guess I'd better hustle. I wouldn't necessarily recommend that approach. These days, it seems like you have a, let's say, disproportionate level of focus on the area of Infrastructure as Code, specifically you seem to be aiming at Terraform. Is that an accurate way of describing it?Ned: That is a very accurate way of describing it. I discovered Terraform while I was doing my consulting back in 2016 era, so this was pretty early on in the product's lifecycle. But I had been using CloudFormation, and at that time, CloudFormation only supported JSON, which meant it was extra punishing. And being able to describe something more succinctly and also have access to all these functions and loops and variables, I was like, “This is amazing. Where were you a year ago?” And so, I really just jumped in with both feet into Terraform.And at a certain point, I was at a conference, and I went past the Pluralsight booth, and they mentioned that they were looking for instructors. And I thought to myself, well, I like talking about things, and I'm pretty excited about this Terraform thing. Why don't I see if they're looking for someone to do a Terraform course? And so, I went through their audition process and sure enough, that is exactly what they were looking for. They had no getting started course for Terraform at the time. I published the course in 2017, and it has been in the top 50 courses ever since on Pluralsight. So, that told me that there's definitely an appetite and maybe this is an area I should focus on a little bit more.Corey: It's a difficult area to learn. About two months ago, I started using Terraform for the first time in anger in ages. I mean, I first discovered it when I was on my way back from one of those Puppet trainings, and the person next to me was really excited about this thing that we're about to launch. Turns out that was Mitchell Hashimoto and Armon was sitting next to him on the other side. Why he had a middle seat, I'll never know.But it was a really fun conversation, just talking about how he saw the world and what he was planning on doing. And a lot of that vision was realized. What I figured out a couple months ago is both that first, I'm sort of sad that Terraform is as bad as it is, but it's the best option we've got because everything else is so much worse. It is omnipresent, though. Effectively, every client I've ever dealt with on AWS billing who has a substantial estate is managing it via Terraform.It is the lingua franca of cloud across the board. I just wish it didn't require as much care and feeding, especially for the getting-started-with-a-boilerplate type of scenario. So, much of what you type feels like it's useless stuff that should be implicit. I understand why it's not, but it feels that way. It's hard to learn.Ned: It certainly can be. And you're right, there's a certain amount of boilerplate and [sigh] code that you have to write that seems pointless. Like, do I have to actually spell this all out? And sometimes the answer is yes, and sometimes the answer is you should use a module for that. Why are you writing this entire VPC configuration out yourself? And that's the sort of thing that you learn over time is that there are shortcuts, there are ways to make the code simpler and require less care and feeding.But I think ultimately, your infrastructure, just like your software, evolves, changes has new requirements, and you need to manage it in the same way that you want to manage your software. And I wouldn't tell a software developer, “Oh, you know, you could just write it once and never go back to it. I'm sure it's fine.” And by the same token, I wouldn't tell an infrastructure developer the same thing. Now, of course, people do that and never go back and touch it, and then somebody else inherits that infrastructure and goes, “Oh, God. Where's the state data?” And no one knows, and then you're starting from scratch. But hopefully, if you have someone who's doing it responsibly, they'll be setting up Terraform in such a way that it is maintainable by somebody else.Corey: I'd sure like to hope so. I have encountered so many horrible examples of code and wondering what malicious person wrote this. And of course, it was me, 6 or 12 months ago.Ned: Always [laugh].Corey: I get to play architect around a lot of these things. In fact, that's one of the problems that I've had historically with an awful lot of different things that I've basically built, called it feature complete, let it sit for a while using the CDK or whatnot, and then oh, I want to make a small change to it. Well, first, I got to spend half a day during the entire line dependency updates and seeing what's broken and how all of that works. It feels like for better or worse, Terraform is a lot more stable than that, as in, old versions of Terraform code from blog posts from 2016 will still effectively work. Is that accurate? I haven't done enough exploring in that direction to be certain.Ned: The good thing about Terraform is you can pin the version of various things that you're using. So, if you're using a particular version of the AWS provider, you can pin it to that specific version, and it won't automatically upgrade you to the latest and greatest. If you didn't do that, then you'll get bit by the update bug, which certainly happens to some folks when they changed the provider from version 3 to version 4 and completely changed how the S3 bucket object was created. A lot of people's scripts broke that day, so I think that was the time for everyone to learn what the version argument is and how it works. But yeah, as long as you follow that general convention of pinning versions of your modules and of your resource provider, you should be in a pretty stable place when you want to update it.Corey: Well, here's the $64,000 question for you, then. Does Dependabot on your GitHub repo begin screaming at you as soon as you've done that because in one of its dependencies in some particular weird edge cases when they're dealing with unsanitized, internet-based input could wind up taking up too many system resources, for example? Which is, I guess, in an ideal world, it wouldn't be an issue, but in practice, my infrastructure team is probably not trying to attack the company from the inside. They have better paths to get there, to be very blunt.Ned: [laugh].Corey: Turns out giving someone access to a thing just directly is way easier than making them find it. But that's been one of the frustrating parts where, especially when it encounters things like, I don't know, corporate security policies of, “Oh, you must clear all of these warnings,” which well-intentioned, poorly executed seems to be the takeaway there.Ned: Yeah, I've certainly seen some implementations of tools that do static scanning of Terraform code and will come up with vulnerabilities or violations of best practice, then you have to put exceptions in there. And sometimes it'll be something like, “You shouldn't have your S3 bucket public,” which in most cases, you shouldn't, but then there's the one team that's actually publishing a front-facing static website in the S3 bucket, and then they have to get, you know, special permission from on high to ignore that warning. So, a lot of those best practices that are in the scanning tools are there for very good reasons and when you onboard them, you should be ready to see a sea of red in your scan the first time and then look through that and kind of pick through what's actually real, and we should improve in our code, and what's something that we can safely ignore because we are intentionally doing it that way.Corey: I feel like there's an awful lot of… how to put this politely… implicit dependencies that are built into things. I'll wind up figuring out how to do something by implementing it and that means I will stitch together an awful lot of blog posts, things I found on Stack Overflow, et cetera, just like a senior engineer and also Chat-Gippity will go ahead and do those things. And then the reason—like, someone asks me four years later, “Why is that thing there?” And… “Well, I don't know, but if I remove it, it might stop working, so…” there was almost a cargo-culting style of, well, it's always been there. So, is that necessary? Is it not?I'm ashamed by how often I learned something very fundamental in a system that I've been using for 20 years—namely, the command line—just by reading the man page for a command that I already, quote-unquote, “Already know how to use perfectly well.” Yeah, there's a lot of hidden gems buried in those things.Ned: Oh, my goodness, I learned something about the Terraform CLI last week that I wish I'd known two years ago. And it's been there for a long time. It's like, when you want to validate your code with the terraform validate, you can initialize without initializing the back-end, and for those who are steeped in Terraform, that means something and for everybody else, I'm sorry [laugh]. But I discovered that was an option, and I was like, “Ahhh, this is amazing.” But to get back to the sort of dependency problems and understanding your infrastructure better—because I think that's ultimately what's happening when you have to describe something using Infrastructure as Code—is you discover how the infrastructure actually works versus how you thought it worked.If you look at how—and I'm going to go into Azure-land here, so try to follow along with me—if you go into Azure-land and you look at how they construct a load balancer, the load balancer is not a single resource. It's about eight different resources that are all tied together. And AWS has something similar with how you have target groups, and you have the load balancer component and the listener and the health check and all that. Azure has the same thing. There's no actual load balancer object, per se.There's a bunch of different components that get slammed together to form that load balancer. When you look in the portal, you don't see any of that. You just see a load balancer, and you might think this is a very simple resource to configure. When it actually comes time to break it out into code, you realize, oh, this is eight different components, each of which has its own options and arguments that I need to understand. So, one of the great things that I have seen a lot of tooling up here around is doing the import of existing infrastructure into Terraform by pointing the tool at a collection of resources—whatever they are—and saying, “Go create the Terraform code that matches that thing.” And it's not going to be the most elegant code out there, but it will give you a baseline for what all the settings actually are, and other resource types are, and then you can tweak it as needed to add in input variables or remove some arguments that you're not using.Corey: Yeah, I remember when they first announced the importing of existing state. It's wow, there's an awful lot of stuff that it can be aware of that I will absolutely need to control unless I want it to start blowing stuff away every time I run the—[unintelligible 00:15:51] supposedly [unintelligible 00:15:52] thing against it. And that wasn't a lot of fun. But yeah, this is the common experience of it. I only recently was reminded of the fact that I once knew, and I'd forgotten that a public versus private subnet in AWS is a human-based abstraction, not something that is implicit to the API or the way they envision subnets existing. Kind of nice, but also weird when you have to unlearn things that you've thought you'd learned.Ned: That's a really interesting example of we think of them as very different things, and when we draw nice architecture diagrams there—these are the private subnets and these are the public ones. And when you actually go to create one using Terraform—or really another tool—there's no box that says ‘private' or ‘make this public.' It's just what does your route table look like? Are you sending that traffic out the internet gateway or are you sending it to some sort of NAT device? And how does traffic come back into that subnet? That's it. That's what makes it private versus public versus a database subnet versus any other subnet type you want to logically assign within AWS.Corey: Yeah. It's kind of fun when that stuff hits.Ned: [laugh].Corey: I am curious, as you look across the ecosystem, do you still see that learning Terraform is a primary pain point for, I guess, the modern era of cloud engineer, or has that sunk below the surface level of awareness in some ways?Ned: I think it's taken as a given to a certain degree that if you're a cloud engineer or an aspiring cloud engineer today, one of the things you're going to learn is Infrastructure as Code, and that Infrastructure as Code is probably going to be Terraform. You can still learn—there's a bunch of other tools out there; I'm not going to pretend like Terraform is the end-all be-all, right? We've got—if you want to use a general purpose programming language, you have something like Pulumi out there that will allow you to do that. If you want to use one of the cloud-native tools, you've got something like CloudFormation or Azure has Bicep. Please don't use ARM templates because they hurt. They're still JSON only, so at least CloudFormation added YAML support in there. And while I don't really like YAML, at least it's not 10,000 lines of code to spin up, like, two domain controllers in a subnet.Corey: I personally wind up resolving the dichotomy between oh, should we go with JSON or should we go with YAML by picking the third option everyone hates more. That's why I'm a staunch advocate for XML.Ned: [laugh]. I was going to say XML. Yeah oh, as someone who dealt with SOAP stuff for a while, yeah, XML was particularly painful, so I'm not sad that went away. JSON for me, I work with it better, but YAML is more readable. So, it's like it's, pick your poison on that. But yeah, there's a ton of infrastructure tools out there.They all have basically the same concepts behind them, the same core concepts because they're all deploying the same thing at the end of the day and there's only so many ways you can express that concept. So, once you learn one—say you learned CloudFormation first—then Terraform is not as big of a leap. You're still declaring stuff within a file and then having it go and make those things exist. It's just nuances between the implementation of Terraform versus CloudFormation versus Bicep.Corey: I wish that there were more straightforward abstractions, but I think that as soon as you get those, that inherently limits what you're able to do, so I don't know how you square that circle.Ned: That's been a real difficult thing is, people want some sort of universal cloud or infrastructure language and abstraction. I just want a virtual machine. I don't care what kind of platform I'm on. Just give me a VM. But then you end up very much caring [laugh] what kind of VM, what operating system, what the underlying hardware is when you get to a certain level.So, there are some workloads where you're like, I just needed to run somewhere in a container and I really don't care about any of the underlying stuff. And that's great. That's what Platform as a Service is for. If that's your end goal, go use that. But if you're actually standing up infrastructure for any sort of enterprise company, then you need an abstraction that gives you access to all the underlying bits when you want them.So, if I want to specify different placement groups about my VM, I need access to that setting to create a placement group. And if I have this high-level of abstraction of a virtual machine, it doesn't know what a placement group is, and now I'm stuck at that level of abstraction instead of getting down to the guts, or I'm going into the portal or the CLI and modifying it outside of the tool that I'm supposed to be using.Corey: I want to change gears slightly here. One thing that has really been roiling some very particular people with very specific perspectives has been the BSL license change that Terraform has wound up rolling out. So far, the people that I've heard who have the strongest opinions on it tend to fall into one of three categories: either they work at HashiCorp—fair enough, they work at one of HashiCorp's direct competitors—which yeah, okay, sure, or they tend to be—how to put this delicately—open-source evangelists, of which I freely admit I used to be one and then had other challenges I needed to chase down in other ways. So, I'm curious as to where you, who are not really on the vendor side of this at all, how do you see it shaking out?Ned: Well, I mean, just for some context, essentially what HashiCorp decided to do was to change the licensing from Mozilla Public licensing to BSL for, I think eight of their products and Terraform was amongst those. And really, this sort of tells you where people are. The only one that anybody really made any noise about was Terraform. There's plenty of people that use Vault, but I didn't see a big brouhaha over the fact that Vault changed its licensing. It's really just about Terraform. Which tells you how important it is to the ecosystem.And if I look at the folks that are making the most noise about it, it's like you said, they basically fall into one of two camps: it's the open-source code purists who believe everything should be licensed in completely open-source ways, or at least if you start out with an open-source license, you can't convert to something else later. And then there is a smaller subset of folks who work for HashiCorp competitors, and they really don't like the idea of having to pay HashiCorp a regular fee for what used to be ostensibly free to them to use. And so, what they ended up doing was creating a fork of Terraform, just before the licensing change happened and that fork of Terraform was originally called OpenTF, and they had an OpenTF manifesto. And I don't know about you, when I see the word ‘manifesto,' I back away slowly and try not to make any sudden moves.Corey: You really get the sense there's going to be a body count tied to this. And people are like, “What about the Agile Manifesto?” “Yeah, what about it?”Ned: [laugh]. Yeah, I'm just—when I see ‘manifesto,' I get a little bit nervous because either someone is so incredibly passionate about something that they've kind of gone off the deep end a little bit, or they're being somewhat duplicitous, and they have ulterior motives, let's say. Now, I'm not trying to cast aspersions on anybody. I can't read anybody's mind and tell you exactly what their intention was behind it. I just know that the manifesto reads a little bit like an open-source purist and a little bit like someone having a temper tantrum, and vacillating between the two.But cooler heads prevailed a little bit, and now they have changed the name to OpenTofu, and it has been accepted by the Linux Foundation as a project. So, it's now a member of the Linux Foundation, with all the gravitas that that comes with. And some people at HashiCorp aren't necessarily happy about the Linux Foundation choosing to pull that in.Corey: Yeah, I saw a whole screed, effectively, that their CEO wound up brain-dumping on that frankly, from a messaging perspective, he would have been better served as not to say anything at all, to be very honest with you.Ned: Yeah, that was a bit of a yikes moment for me.Corey: It's very rare that you will listen yourself into trouble as opposed to opening your mouth and getting yourself into trouble.Ned: Exactly.Corey: You wouldn't think I would be one of those—of all people who would have made that observation, you wouldn't think I would be on that list, yet here I am.Ned: Yeah. And I don't think either side is entirely blameless. I understand the motivations behind HashiCorp wanting to make the change. I mean, they're a publicly traded company now and ostensibly that means that they should be making some amount of money for their investors, so they do have to bear that in mind. I don't necessarily think that changing the licensing of Terraform is the way to make that money.I think in the long-term, it's not going—it may not hurt them a lot, but I don't think it's going to help them out a lot, and it's tainted the goodwill of the community to a certain degree. On the other hand, I don't entirely trust what the other businesses are saying as well in their stead. So, there's nobody in this that comes out a hundred percent clean [laugh] on the whole process.Corey: Yeah, I feel like, to be direct, the direct competitors to HashiCorp along its various axes are not the best actors necessarily to complain about what is their largest competitor no longer giving them access to continue to compete against them with their own product. I understand the nuances there, but it also doesn't feel like they are the best ambassadors for that. I also definitely understand where HashiCorp is coming from where, why are we investing all this time, energy, and effort for people to basically take revenue away from us? But there's also the bigger problem, which is, by and large, compared to how many sites are running Terraform and the revenues that HashiCorp puts up for it, they're clearly failing to capture the value they have delivered in a massive way. But counterpoint, if they hadn't been open-source for their life until this point, would they have ever captured that market share? Probably not.Ned: Yeah, I think ultimately, the biggest competitor to their paid offering of Terraform is their free version of Terraform. It literally has enough bells and whistles already included and plenty of options for automating those things and solving the problems that their enterprise product solves that their biggest problem is not other competitors in the Terraform landscape; it's the, “Well, we already have something, and it's good enough.” And I'm not sure how you sell to that person, that's why I'm not in marketing, but I think that is their biggest competitor is the people who already have a solution and are like, “Why do I need to pay for your thing when my thing works well enough?”Corey: That's part of the strange thing that I'm seeing as I look across this entire landscape is it feels like this is not something that is directly going to impact almost anyone out there who's just using this stuff, either the open-source version as a paying customer of any of these things, but it is going to kick up a bunch of dust. And speaking of poor messaging, HashiCorp is not really killing it this quarter, where the initial announcement led to so many questions that were unclear, such as—like, they fixed this later in the frequently asked questions list, but okay, “I'm using Terraform right now and that's fine. I'm building something else completely different. Am I going to lose my access to Terraform if you decide to launch a feature that does what my company does?” And after a couple of days, they put up an indemnity against that. Okay, fine.Like, when Mongo did this, there was a similar type of dynamic that was emerging, but a lot fewer people are writing their own database engine to then sell onward to customers that are provisioning infrastructure on behalf of their customers. And where the boundaries lay for who was considered a direct Terraform competitor was unclear. I'm still not convinced that it is clear enough to bet the business on for a lot of these folks. It comes down to say what you mean, not—instead of hedging, you're not helping your cause any.Ned: Yeah, I think out of the different products that they have, some are very clear-cut. Like, Vault is a server that runs as a service, and so that's very clear what that product is and where the lines of delineation are around Vault. If I go stand up a bunch of Vault servers and offer them as a service, then that is clearly a competitor. But if I have an automation pipeline service and people can technically automate Terraform deployments with my service, even if that's not the core thing that I'm looking to do, am I now a competitor? Like, it's such a fuzzy line because Terraform isn't an application, it's not a server that runs somewhere, it's a CLI tool and a programming language. So yeah, those lines are very, very fuzzy. And I… like I said, it would be better if they say what they meant, as opposed to sort of the mealy-mouthed language that they ended up using and the need to publish multiple revisions of that FAQ to clarify their position on very specific niche use cases.Corey: Yeah, I'm not trying to be difficult or insulting or anything like that. These are hard problems that everyone involved is wrestling with. It just felt a little off, and I think the messaging did them no favors when that wound up hitting. And now, everyone is sort of trying to read the tea leaves and figure out what does this mean because in isolation, it doesn't mean anything. It is a forward-looking thing.Whatever it is you're doing today, no changes are needed for you, until the next version comes out, in which case, okay, now do we incorporate the new thing or don't we? Today, to my understanding, whether I'm running Terraform or OpenTofu entirely comes down to which binary am I invoking to do the apply? There is no difference of which I am aware. That will, of course, change, but today, I don't have to think about that.Ned: Right. OpenTofu is a literal fork of Terraform, and they haven't really added much in the way of features, so it should be completely compatible with Terraform. The two will diverge in the future as feature as new features get added to each one. But yeah, for folks who are using it today, they might just decide to stay on the version pre-fork and stay on that for years. I think HashiCorp has pledged 18 months of support for any minor version of Terraform, so you've got at least a year-and-a-half to decide. And we were kind of talking before the recording, 99% of people using Terraform do not care about this. It does not impact their daily workflow.Corey: No. I don't see customers caring at all. And also, “Oh, we're only going to use the pre-fork version of Terraform,” they're like, “Thanks for the air cover because we haven't updated any of that stuff in five years, so tha”—Ned: [laugh].Corey: “Oh yeah, we're doing it out of license concern. That's it. That's the reason we haven't done anything recent with it.” Because once it's working, changes are scary.Ned: Yeah.Corey: Terraform is one of those scary things, right next to databases, that if I make a change that I don't fully understand—and no one understands everything, as we've covered—then this could really ruin my week. So, I'm going to be very cautious around that.Ned: Yeah, if metrics are to be believed across the automation platforms, once an infrastructure rollout happens with a particular version of Terraform, that version does not get updated. For years. So, I have it on good authority that there's still Terraform version 0.10 and 0.11 running on these automation platforms for really old builds where people are too scared to upgrade to, like, post 0.12 where everything changed in the language.I believe that. People don't want to change it, especially if it's working. And so, for most people, this licensing chain doesn't matter. And all the constant back and forth and bickering just makes people feel a little nervous, and it might end up pushing people away from Terraform as a platform entirely, as opposed to picking a side.Corey: Yeah, and I think that that is probably the fair way to view it at this point where right now—please, friends at HashiCorp and HashiCorp competitors don't yell at me for this—it's basically a nerd slap-fight at the moment.Ned: [laugh].Corey: And of one of the big reasons that I also stay out of these debates almost entirely is that I married a corporate attorney who used to be a litigator and I get frustrated whenever it comes down to license arguments because you see suddenly a bunch of engineers who get to cosplay as lawyers, and reading the comments is infuriating once you realize how a little bit of this stuff works, which I've had 15 years of osmotic learning on this stuff. Whenever I want to upset my wife, I just read some of these comments aloud and then our dinner conversation becomes screaming. It's wonderful.Ned: Bad legal takes? Yeah, before—Corey: Exactly.Ned: Before my father became a social studies teacher, he was a lawyer for 20 years, and so I got to absorb some of the thought process of the lawyer. And yeah, I read some of these takes, and I'm like, “That doesn't sound right. I don't think that would hold up in any court of law.” Though a lot of the open-source licensing I don't think has been tested in any sort of court of law. It's just kind of like, “Well, we hope this stands up,” but nobody really has the money to check.Corey: Yeah. This is the problem with these open-source licenses as well. Very few have never been tested in any meaningful way because I don't know about you, but I don't have a few million dollars in legal fees lying around to prove the point.Ned: Yeah.Corey: So, it's one of those we think this is sustainable, and Lord knows the number of companies that have taken reliances on these licenses, they're probably right. I'm certainly not going to disprove the fact—please don't sue me—but yeah, this is one of those things that we're sort of assuming is the case, even if it's potentially not. I really want to thank you for taking the time to discuss how it is you view these things and talk about what it is you're up to. If people want to learn more, where's the best place for them to find you?Ned: Honestly, just go to my website. It's nedinthecloud.com. And you can also find me on LinkedIn. I don't really go for Twitter anymore.Corey: I envy you. I wish I could wean myself off of it. But we will, of course, include a link to that in the show notes. Thank you so much for being so generous with your time. It's appreciated.Ned: It's been a pleasure. Thanks, Corey.Corey: Net Bellavance, founder and curious human at Ned in the Cloud. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment that I will then fork under a different license and claim as my own.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business, and we get to the point. Visit duckbillgroup.com to get started.

The Authors Quill normal air time is 3 to 5 pm cst running late to day dto storms SEAN PATRICK HAZLETT Sean Patrick Hazlett is a technologist, finance professional, and science fiction, fantasy, horror, and non-fiction author and editor working in Silicon Valley. He is a winner of the Writers of the Future Contest, and over forty of his short stories have appeared in publications such as The Year's Best Military and Adventure SF, Year's Best Hardcore Horror, Terraform, Galaxy's Edge, Writers of the Future, Grimdark Magazine, Vastarien, and Abyss & Apex, among others. He is also the editor of the Weird World War III and Weird World War IV anthologies. He is an active member of the Horror Writers Association. In graduate school, Sean assisted future Secretary of Defense Ashton B. Carter at the Harvard-Stanford Preventive Defense Project where he developed strategic options for confronting Iran's nuclear program. For this analysis, he won the 2006 Policy Analysis Exercise Award at the Harvard Kennedy School of Government. Sean also worked as an intelligence analyst focusing on strategic war games and simulations for the Pentagon, where he drew on his experience training the US military as a cavalry officer in the US Army during the Iraq and Afghan wars. Sean holds a Master of Business Administration from Harvard Business School, a Master in Public Policy from the Harvard Kennedy School of Government, and bachelor's degrees in History and Electrical Engineering from Stanford University. Enter your email Join my email list By clicking submit, you agree to share your email address with the site owner and Mailchimp to receive marketing, updates, and other emails from the site owner. Use the unsubscribe link in those emails to opt out at any time. SHARE THIS: Click to share on Twitter (Opens in new window)Click to share on Facebook (Opens in new window) SEARCH Search for: Search … AUTHORS Sean Patrick Hazlett Weird World War III & IV: To-Date Promotional Summary: December 11th 2022 Update Weird World War III & IV: To-Date Promotional Summary: October 19th 2022 Update Win 5 Free Books in September 2022 and Other General Updates! Weird World War III & IV: To-Date Promotional Summary: August 5th 2022 Update Weird World War III & IV: To-Date Promotional Summary: June 9th 2022 Update WEIRD WORLD WAR: CHINA Launch Date January 2, 2024 31 days to go. December 2023 M T W T F S S 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 « Dec FOLLOW ME ON TWITTER BLOG STATS 11,311 hits What You Can Do to Save the UAP Amendment with Steve Bassett This One Thing Made Them Ban Red Night Vision Goggles in Vietnam (with Mark Anthony) Clearing Out Negative Energies with a Little Help from the Stars with Jeffrey Seelman The Afterlife Frequency with Mark Anthony MH370 and the Curious Case of Lieutenant Commander Edward C. Lin with Ashton Forbes CATEGORIES Analysis Delaware Editing Hiking Pleasant Hill Promotion Reviews Sasquan Uncategorized Weird World War III Weird World War III Authors Weird World War IV Writing Writing Statistics and Revenue What You Can Do to Save the UAP Amendment with Steve Bassett This One Thing Made Them Ban Red Night Vision Goggles in Vietnam (with Mark Anthony) Clearing Out Negative Energies with a Little Help from the Stars with Jeffrey Seelman The Afterlife Frequency with Mark Anthony MH370 and the Curious Case of Lieutenant Commander Edward C. Lin with Ashton Forbes

Today on Heavy Networking, sponsored by Juniper, we'll talk about how Juniper's Apstra software can help you operate your on-prem data center more like a public cloud; meaning service provisioning that's repeatable, standardized, and straightforward to consume. We'll also talk about how Apstra now works with Terraform to help streamline network self-service. The post HN711: Get Cloud-Like Operation Of Your Data Center With Juniper Apstra And Terraform (Sponsored) appeared first on Packet Pushers.

Today on Heavy Networking, sponsored by Juniper, we'll talk about how Juniper's Apstra software can help you operate your on-prem data center more like a public cloud; meaning service provisioning that's repeatable, standardized, and straightforward to consume. We'll also talk about how Apstra now works with Terraform to help streamline network self-service. The post HN711: Get Cloud-Like Operation Of Your Data Center With Juniper Apstra And Terraform (Sponsored) appeared first on Packet Pushers.

Today on Heavy Networking, sponsored by Juniper, we'll talk about how Juniper's Apstra software can help you operate your on-prem data center more like a public cloud; meaning service provisioning that's repeatable, standardized, and straightforward to consume. We'll also talk about how Apstra now works with Terraform to help streamline network self-service. The post HN711: Get Cloud-Like Operation Of Your Data Center With Juniper Apstra And Terraform (Sponsored) appeared first on Packet Pushers.

In the past, we had entire civilizations devoted to agriculture, but in the future, we might have entire planets devoted to it.Compare news coverage. Spot media bias. Avoid algorithms. Try Ground News today and get 30% off your subscription by going to https://ground.news/isaacarthurVisit our Website: http://www.isaacarthur.netJoin Nebula: https://go.nebula.tv/isaacarthurSupport us on Patreon: https://www.patreon.com/IsaacArthurSupport us on Subscribestar: https://www.subscribestar.com/isaac-arthurFacebook Group: https://www.facebook.com/groups/1583992725237264/Reddit: https://www.reddit.com/r/IsaacArthur/Twitter: https://twitter.com/Isaac_A_Arthur on Twitter and RT our future content.SFIA Discord Server: https://discord.gg/53GAShECredits: Agriworlds Episode 423; November 30, 2023Written, Produced & Narrated by:Isaac ArthurMusic Courtesy of: Epidemic Sound http://epidemicsound.com/creatorSergey Cheremisinov, "Labyrinth"Taras Harkavyi, "Alpha and..."See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

Mike Perham is the creator of Sidekiq, a background job processor for Ruby. He's also the creator of Faktory a similar product for multiple language environments. We talk about the RubyConf keynote and Ruby's limitations, supporting products as a solo developer, and some ideas for funding open source like a public utility. Recorded at RubyConf 2023 in San Diego. -- A few topics covered: Sidekiq (Ruby) vs Faktory (Polyglot) Why background job solutions are so common in Ruby Global Interpreter Lock (GIL) Ractors (Actor concurrency) Downsides of Multiprocess applications When to use other languages Getting people to pay for Sidekiq Keeping a solo business Being selective about customers Ways to keep support needs low Open source as a public utility Mike Mike's blog mastodon Sidekiq faktory From Employment to Independence Ruby Ractor The Practical Effects of the GVL on Scaling in Ruby Transcript You can help correct transcripts on GitHub. Introduction [00:00:00] Jeremy: I'm here at RubyConf San Diego with Mike Perham. He's the creator of Sidekiq and Faktory. [00:00:07] Mike: Thank you, Jeremy, for having me here. It's a pleasure. Sidekiq [00:00:11] Jeremy: So for people who aren't familiar with, I guess we'll start with Sidekiq because I think that's what you're most known for. If people don't know what it is, maybe you can give like a small little explanation. [00:00:22] Mike: Ruby apps generally have two major pieces of infrastructure powering them. You've got your app server, which serves your webpages and the browser. And then you generally have something off on the side that... It processes, you know, data for a million different reasons, and that's generally called a background job framework, and that's what Sidekiq is. [00:00:41] It, Rails is usually the thing that, that handles your web stuff, and then Sidekiq is the Sidekiq to Rails, so to speak. [00:00:50] Jeremy: And so this would fit the same role as, I think in Python, there's celery. and then in the Ruby world, I guess there is, uh, Resque is another kind of job. [00:01:02] Mike: Yeah, background job frameworks are quite prolific in Ruby. the Ruby community's kind of settled on that as the, the standard pattern for application development. So yeah, we've got, a half a dozen to a dozen different, different examples throughout history, but the major ones today are, Sidekiq, Resque, DelayedJob, GoodJob, and, and, and others down the line, yeah. Why background jobs are so common in Ruby [00:01:25] Jeremy: I think working in other languages, you mentioned how in Ruby, there's this very clear, preference to use these job scheduling systems, these job queuing systems, and I'm not. I'm not sure if that's as true in, say, if somebody's working in Java, or C sharp, or whatnot. And I wonder if there's something specific about Ruby that makes people kind of gravitate towards this as the default thing they would use. [00:01:52] Mike: That's a good question. What makes Ruby... The one that so needs a background job system. I think Ruby, has historically been very single threaded. And so, every Ruby process can only do so much work. And so Ruby oftentimes does, uh, spin up a lot of different processes, and so having processes that are more focused on one thing is, is, is more standard. [00:02:24] So you'll have your application server processes, which focus on just serving HTTP responses. And then you have some other sort of focused process and that just became background job processes. but yeah, I haven't really thought of it all that much. But, uh, you know, something like Java, for instance, heavily multi threaded. [00:02:45] And so, and extremely heavyweight in terms of memory and startup time. So it's much more frequent in Java that you just start up one process and that's it. Right, you just do everything in that one process. And so you may have dozens and dozens of threads, both serving HTTP and doing work on the side too. Um, whereas in Ruby that just kind of naturally, there was a natural split there. Global Interpreter Lock [00:03:10] Jeremy: So that's actually a really good insight, because... in the keynote at RubyConf, Mats, the creator of Ruby, you know, he mentioned the, how the fact that there is this global, interpreter lock, [00:03:23] or, or global VM lock in Ruby, and so you can't, really do multiple things in parallel and make use of all the different cores. And so it makes a lot of sense why you would say like, okay, I need to spin up separate processes so that I can actually take advantage of, of my, system. [00:03:43] Mike: Right. Yeah. And the, um, the GVL. is the acronym we use in the Ruby community, or GIL. Uh, that global lock really kind of is a forcing function for much of the application architecture in Ruby. Ruby, uh, applications because it does limit how much processing a single Ruby process can do. So, uh, even though Sidekiq is heavily multi threaded, you can only have so many threads executing. [00:04:14] Because they all have to share one core because of that global lock. So unfortunately, that's, that's been, um, one of the limiter, limiting factors to Sidekiq scalability is that, that lock and boy, I would pay a lot of money to just have that lock go away, but. You know, Python is going through a very long term experiment about trying to remove that lock and I'm very curious to see how well that goes because I would love to see Ruby do the same and we'll see what happens in the future, but, it's always frustrating when I come to another RubyConf and I hear another Matt's keynote where he's asked about the GIL and he continues to say, well, the GIL is going to be around, as long as I can tell. [00:04:57] so it's a little bit frustrating, but. It's, it's just what you have to deal with. Ractors [00:05:02] Jeremy: I'm not too familiar with them, but they, they did mention during the keynote I think there Ractors or something like that. There, there, there's some way of being able to get around the GIL but there are these constraints on them. And in the context of Sidekiq and, and maybe Ruby in general, how do you feel about those options or those solutions? [00:05:22] Mike: Yeah, so, I think it was Ruby 3. 2 that introduced this concept of what they call a Ractor, which is like a thread, except it does not have the global lock. It can run independent to the global lock. The problem is, is because it doesn't use the global lock, it has pretty severe constraints on what it can do. [00:05:47] And the, and more specifically, the data it can access. So, Ruby apps and Rails apps throughout history have traditionally accessed a lot of global data, a lot of class level data, and accessed all this data in a, in a read only fashion. so there's no race conditions because no one's changing any of it, but it's still, lots of threads all accessing the same variables. [00:06:19] Well, Ractors can't do that at all. The only data Ractors can access is data that they own. And so that is completely foreign to Ruby application, traditional Ruby applications. So essentially, Ractors aren't compatible with the vast majority of existing Ruby code. So I, I, I toyed with the idea of prototyping Sidekiq and Ractors, and within about a minute or two, I just ran into these, these, uh... [00:06:51] These very severe constraints, and so that's why you don't see a lot of people using Ractors, even still, even though they've been out for a year or two now, you just don't see a lot of people using them, because they're, they're really limited, limited in what they can do. But, on the other hand, they're unlimited in how well they can scale. [00:07:12] So, we'll see, we'll see. Hopefully in the future, they'll make a lot of improvements and, uh, maybe they'll become more usable over time. Downsides of multiprocess (Memory usage) [00:07:19] Jeremy: And with the existence of a job queue or job scheduler like Sidekiq, you're able to create additional processes to get around that global lock, I suppose. What are the... downsides of doing so versus another language like we mentioned Java earlier, which is capable of having true parallelism in the same process. [00:07:47] Mike: Yeah, so you can start up multiple Ruby processes to process things truly in parallel. The issue is that you do get some duplication in terms of memory. So your Ruby app maybe take a gigabyte per process. And, you can do copy on write forking. You can fork and get some memory sharing with copy on write semantics on Unix operating systems. [00:08:21] But you may only get, let's say, 30 percent memory savings. So, there's still a significant memory overhead to forking, you know, let's say, eight processes versus having eight threads. You know, you, you, you may have, uh, eight threads can operate in a gigabyte process, but if you want to have eight processes, that may take, let's say, four gigabytes of RAM. [00:08:48] So you, you still, it's not going to cost you eight gigabytes of RAM, you know, it's not like just one times eight, but, there's still a overhead of having those separate processes. [00:08:58] Jeremy: would you say it's more of a cost restriction, like it costs you more to run these applications, or are there actual problems that you can't solve because of this restriction. [00:09:13] Mike: Help me understand, what do you mean by restriction? Do you mean just the GVL in general, or the fact that forking processes still costs memory? [00:09:22] Jeremy: I think, well, it would be both, right? So you're, you have two restrictions right now. You have the, the GVL, which means you can't have parallelism within the same process. And then your other option is to spin up a bunch of processes, which you have said is the downside there is that you're using a lot more RAM. [00:09:43] I suppose my question is that Does that actually stop you from doing anything? Like, if you throw more money at the problem, you go like, we're going to have more instances, I'll pay for the RAM, it's fine, can that basically get you out of these situations or are these limitations actually stopping you from, from doing things you could do in other languages? [00:10:04] Mike: Well, you certainly have to manage the multiple processes, right? So you've gotta, you know, if one child process crashes, you've gotta have a parent or supervisor process watching all that and monitoring and restarting the process. I don't think it restricts you. Necessarily, it just, it adds complexity to your deployment. [00:10:24] and, and it's just a question of efficiency, right? Instead of being able to deploy on a, on a one gigabyte droplet, I've got to deploy to a four gigabyte droplet, right? Because I just, I need the RAM to run the eight processes. So it, it, it's more of just a purely a function of how much money am I going to have to throw at this problem. [00:10:45] And what's it going to cost me in operational costs to operate this application in production? When to use other languages? [00:10:53] Jeremy: So during the. Keynote, uh, Matz had mentioned that Rails, is really suitable as this one person framework, like you can have a very small team or maybe even yourself and, and build this product. And so I guess from... Your perspective, once you cross a certain threshold, is like, what Ruby and what Sidekiq provides not enough, and that's why you need to start looking into other languages? [00:11:24] Or like, where's the, turning point, or the, if you [00:11:29] Mike: Right, right. The, it's all about the problem you're trying to solve, right? At the end of the day, uh, the, the question is just what are we trying to solve and how are we trying to solve it? So at a higher level, you got to think about the architecture. if the problem you're trying to solve, if the service you're trying to build, if the app you're trying to operate. [00:11:51] If that doesn't really fall into the traditional Ruby application architecture, then you, you might look at it in another language or another ecosystem. something like Go, for instance, can compile down to a single binary, which makes deployment really easy. It makes shipping up a product. on to a user's machine, much simpler than deploying a Ruby application onto a user's desktop machine, for instance, right? [00:12:22] Um, Ruby does have this, this problem of how do you package everything together and deploy it somewhere? Whereas Go, when you can just compile to a single binary, now you've just got a single thing. And it's just... Drop it on the file system and execute it. It's easy. So, um, different, different ecosystems have different application architectures, which empower different ways of solving the same problems. [00:12:48] But, you know, Rails as a, as a one man framework, or sorry, one person framework, It, it, I don't, I don't necessarily, that's a, that's sort of a catchy marketing slogan, but I just think of Rails as the most productive framework you can use. So you, as a single person, you can maximize what you ship and the, the, the value that you can create because Rails is so productive. [00:13:13] Jeremy: So it, seems like it's maybe the, the domain or the type of application you're making. Like you mentioned the command line application, because you want to be able to deliver it to your user easily. Just give them a binary, something like Go or perhaps Rust makes a lot more sense. and then I could see people saying that if you're doing something with machine learning, like the community behind Python, it's, they're just, they're all there. [00:13:41] So Room for more domains in Ruby [00:13:41] Mike: That was exactly the example I was going to use also. Yeah, if you're doing something with data or AI, Python is going to be a more, a more traditional, natural choice. that doesn't mean Ruby can't do it. That doesn't mean, you wouldn't be able to solve the problem with Ruby. And, and there's, that just also means that there's more space for someone who wants to come in and make an impact in the Ruby community. [00:14:03] Find a problem that Ruby's not really well suited to solving right now and build the tooling out there to, to try and solve it. You know, I, I saw a talk, from the fellow who makes the Glimmer gem, which is a native UI toolkit. Uh, a gem for building native UIs in Ruby, which Ruby traditionally can't do, but he's, he's done an amazing job at sort of surfacing APIs to build these, um, these native, uh, native applications, which I think is great. [00:14:32] It's awesome. It's, it's so invigorating to see Ruby in a new space like that. Um, I talked to someone else who's doing the Polars gem, which is focused on data processing. So it kind of takes, um, Python and Pandas and brings that to Ruby, which is, is awesome because if you're a Ruby developer, now you've got all these additional tools which can allow you to solve new sets of problems out there. [00:14:57] So that's, that's kind of what's exciting in the Ruby community right now is just bring it into new spaces. Faktory [00:15:03] Jeremy: In addition to Sidekiq, you have, uh, another product called Faktory, I believe. And so does that serve a, a similar purpose? Is that another job scheduling, job queueing system? [00:15:16] Mike: It is, yes. And it's, it's, it's similar in a way to Sidekiq. It looks similar. It's got similar concepts at the core of it. At the end of the day, Sidekiq is limited to Ruby. Because Sidekiq executes in a Ruby VM, it executes the jobs, and the jobs are, have to be written in Ruby because you're running in the Ruby VM. [00:15:38] Faktory was my attempt to bring, Sidekiq functionality to every other language. I wanted, I wanted Sidekiq for JavaScript. I wanted Sidekiq for Go. I wanted Sidekiq for Python because A, a lot of these other languages also could use a system, a background job system. And the problem though is that. [00:16:04] As a single man, I can't port Sidekiq to every other language. I don't know all the languages, right? So, Faktory kind of changes the architecture and, um, allows you to execute jobs in any language. it, it replaces Redis and provides a server where you just fetch jobs, and you can use it from it. [00:16:26] You can use that protocol from any language to, to build your own worker processes that execute jobs in whatever language you want. [00:16:35] Jeremy: When you say it replaces Redis, so it doesn't use Redis, um, internally, it has its own. [00:16:41] Mike: It does use Redis under the covers. Yeah, it starts Redis as a child process and, connects to it over a Unix socket. And so it's really stable. It's really fast. from the outside, the, the worker processes, they just talk to Faktory. They don't know anything about Redis at all. [00:16:59] Jeremy: I see. And for someone who, like we mentioned earlier in the Python community, for example, there is, um, Celery. For someone who is using a task scheduler like that, what's the incentive to switch or use something different? [00:17:17] Mike: Well, I, I always say if you're using something right now, I'm not going to try and convince you to switch necessarily. It's when you have pain that you want to switch and move away. Maybe you have Maybe there's capabilities in the newer system that you really need that the old system doesn't provide, but Celery is such a widely known system that I'm not necessarily going to try and convince people to move away from it, but if people are looking for a new system, one of the things that Celery does that Faktory does not do is Celery provides like data adapters for using store, lots of different storage systems, right? [00:17:55] Faktory doesn't do that. Faktory is more, has more of the Rails mantra of, you know, Omakase where we choose, I choose to use Redis and that's it. You don't, you don't have a choice for what to use because who cares, you know, at the end of the day, let Faktory deal with it. it's, it's not something that, You should even necessarily be concerned about it. [00:18:17] Just, just try Faktory out and see how it works for you. Um, so I, I try to take those operational concerns off the table and just have the user focus on, you know, usability, performance, and that sort of thing. but it is, it's, it's another background job system out there for people to try out and see if they like that. [00:18:36] And, and if they want to, um, if they know Celery and they want to use Celery, more power to Faktory them. Sidekiq (Ruby) or Faktory (Polyglot) [00:18:43] Jeremy: And Sidekiq and Faktory, they serve a very similar purpose. For someone who they have a new project, they haven't chosen a job. scheduling system, if they were using Ruby, would it ever make sense for them to use Faktory versus use Sidekiq? [00:19:05] Mike: Uh Faktory is excellent in a polyglot situation. So if you're using multiple languages, if you're creating jobs in Ruby, but you're executing them in Python, for instance, um, you know, if you've, I have people who are, Creating jobs in PHP and executing them in Python, for instance. That kind of polyglot scenario, Sidekiq can't do that at all. [00:19:31] So, Faktory is useful there. In terms of Ruby, Ruby is just another language to Faktory. So, there is a Ruby API for using Faktory, and you can create and execute Ruby jobs with Faktory. But, you'll find that in the Ruby community, Sidekiq is much widely... much more widely used and understood and known. So if you're just using Ruby, I think, I think Sidekiq is the right choice. [00:19:59] I wouldn't look at Faktory. But if you do need, find yourself needing that polyglot tool, then Faktory is there. Temporal [00:20:07] Jeremy: And this is maybe one, maybe one layer of abstraction higher, but there's a product called Temporal that has some of this job scheduling, but also this workflow component. I wonder if you've tried that out and how you think about that product? [00:20:25] Mike: I've heard of them. I don't know a lot about the product. I do have a workflow API, the Sidekiq batches, which allow you to fan out jobs and then, and then execute callbacks when all the jobs in that, in that batch are done. But I don't, provide sort of a, a high level. Graphical Workflow Editor or anything like that. [00:20:50] Those to me are more marketing tools that you use to sell the tool for six figures. And I don't think they're usable. And I don't think they're actually used day to day. I provide an API for developers to use. And developers don't like moving blocks of code around in a GUI. They want to write code. And, um, so yeah, temporal, I, like I said, I don't know much about them. [00:21:19] I also, are they a venture capital backed startup? [00:21:22] Jeremy: They are, is my understanding, [00:21:24] Mike: Yeah, that, uh, any, any sort of venture capital backed startup, um, who's building technical infrastructure. I, I would look long and hard at, I'm, I think open source is the right core to build on. Of course I sell commercial software, but. I'm bootstrapped. I'm profitable. [00:21:46] I'm going to be around forever. A VC backed startup, they tend to go bankrupt, because they either get big or they go out of business. So that would be my only comment is, is, be a little bit leery about relying on commercial venture capital based infrastructure for, for companies, uh, long term. Getting people to pay for Sidekiq [00:22:05] Jeremy: So I think that's a really interesting part about your business is that I think a lot of open source maintainers have a really big challenge figuring out how to make it as a living. The, there are so many projects that they all have a very permissive license and you can use them freely one example I can think of is, I, I talked with, uh, David Kramer, who's the CTO at Sentry, and he, I don't think they use it anymore, but they, they were using Nginx, right? [00:22:39] And he's like, well, Nginx, they have a paid product, like Nginx. Plus that or something. I don't know what the name is, but he was like, but I'm not going to pay for it. Right. I'm just going to use the free one. Why would I, you know, pay for the, um, the paid thing? So I, I, I'm kind of curious from your perspective when you were coming up with Sidekiq both as an open source product, but also as a commercial one, how did you make that determination of like to make a product where it's going to be useful in its open source form? [00:23:15] I can still convince people to pay money for it. [00:23:19] Mike: Yeah, the, I was terrified, to be blunt, when I first started out. when I started the Sidekiq project, I knew it was going to take a lot of time. I knew if it was successful, I was going to be doing it for the next decade. Right? So I started in 2012, and here I am in 2023, over a decade, and I'm still doing it. [00:23:38] So my expectation was met in that regard. And I knew I was not going to be able to last that long. If I was making zero dollars, right? You just, you burn out. Nobody can last that long. Well, I guess there are a few exceptions to that rule, but yeah, money, I tend to think makes things a little more sustainable for sure. [00:23:58] Especially if you can turn it into a full time job solving and supporting a project that you, you love and, and is, is, you know, your, your, your baby, your child, so to speak, your software, uh, uh, creation that you've given to the world. but I was terrified. but one thing I did was at the time I was blogging a lot. [00:24:22] And so I was telling people about Sidekiq. I was telling people what was to come. I was talking about ideas and. The one thing that I blogged about was financial experiments. I said bluntly to the, to, to the Ruby community, I'm going to be experimenting with financial stability and sustainability with this project. [00:24:42] So not only did I create this open source project, but I was also publicly saying I I need to figure out how to make this work for the next decade. And so eventually that led to Sidekiq Pro. And I had to figure out how to build a closed source Ruby gem, which, uh, There's not a lot of, so I was kind of in the wild there. [00:25:11] But, you know, thankfully all the pieces came together and it was actually possible. I couldn't have done it if it wasn't possible. Like, we would not be talking if I couldn't make a private gem. So, um, but it happened to work out. Uh, and it allowed me to, to gate features behind a paywall effectively. And, and yeah, you're right. [00:25:33] It can be tough to make people pay for software. but I'm a developer who's selling to other developers, not, not just developers, open source developers, and they know that they have this financial problem, right? They know that there's this sustainability problem. And I was blunt in saying, this is my solution to my sustainability. [00:25:56] So, I charge what I think is a very fair price. It's only a thousand dollars a year to a hobbyist. That may seem like a lot of money to a business. It's a drop in the bucket. So it was easy for developers to say, Hey, listen, we want to buy this tool for a thousand bucks. It'll ensure our infrastructure is maintained for the next decade. [00:26:18] And it's, and it's. And it's relatively cheap. It's way less than, uh, you know, a salary or even a laptop. So, so that's, that's what I did. And, um, it's, it worked out great. People, people really understood. Even today, I talk to people and they say, we, we signed up for Sidekiq Pro to support you. So it's, it's, it's really, um, invigorating to hear people, uh, thank me and, and they're, they're actively happy that they're paying me and our customers. [00:26:49] Jeremy: it's sort of, uh, maybe a not super common story, right, in terms of what you went through. Because when I think of open core businesses, I think of companies like, uh, GitLab, which are venture funded, uh, very different scenario there. I wonder, like, in your case, so you started in 2012, and there were probably no venture backed competitors, right? [00:27:19] People saying that we're going to make this job scheduling system and some VC is going to give me five million dollars and build a team to work on this. It was probably at the time, maybe it was Rescue, which was... [00:27:35] Mike: There was a venture backed system called IronMQ, [00:27:40] Jeremy: Hmm. [00:27:41] Mike: And I'm not sure if they're still around or not, but they... They took, uh, one or more funding rounds. I'm not sure exactly, but they were VC backed. They were doing, background jobs, scheduled jobs, uh, you know, running container, running container jobs. They, they eventually, I think, wound up sort of settling on Docker containers. [00:28:06] They'll basically spin up a Docker container. And that container can do whatever it wants. It can execute for a second and then shut down, or it can run for, for however long, but they would, um, yeah, I, yeah, I'll, I'll stop there because I don't know the actual details of exactly their system, but I'm not sure if they're still around, but that's the only one that I remember offhand that was around, you know, years ago. [00:28:32] Yeah, it's, it's mostly, you know, low level open source infrastructure. And so, anytime you have funded startups, they're generally using that open source infrastructure to build their own SaaS. And so SaaS's are the vast majority of where you see sort of, uh, commercial software. [00:28:51] Jeremy: so I guess in that way it, it, it gave you this, this window or this area where you could come in and there wasn't, other than that iron, product, there wasn't this big money that you were fighting against. It was sort of, it was you telling people openly, I'm, I'm working on this thing. [00:29:11] I need to make money so that I can sustain it. And, if you, yeah. like the work I do, then, you know, basically support me. Right. And, and so I think that, I'm wondering how we can reproduce that more often because when you see new products, a lot of times it is VC backed, right? [00:29:35] Because people say, I need to work on this. I need to be paid. and I can't ask a team to do this. For nothing, right? So [00:29:44] Mike: Yeah. It's. It's a wicked problem. Uh, it's a really, really hard problem to solve if you take vc you there, that that really kind of means that you need to be making tens if not hundreds of millions of dollars in sales. If you are building a small or relatively small. You know, put small in quotes there because I don't really know what that means, but if you have a small open source project, you can't charge huge amounts for it, right? [00:30:18] I mean, Sidekiq is a, I would call a medium sized open source project, and I'm charging a thousand bucks for it. So if you're building, you know, I don't know, I don't even want to necessarily give example, but if you're building some open source project, and It's one of 300 libraries that people's applications will depend on. [00:30:40] You can't necessarily charge a thousand dollars for that library. depending on the size and the capabilities, maybe you can, maybe you can't. But there's going to be a long tail of open source projects that just, they can't, they can't charge much, if anything, for them. So, unfortunately, we have, you know, these You kind of have two pathways. [00:31:07] Venture capital, where you've got to sell a ton, or free. And I've kind of walked that fine line where I'm a small business, I can charge a small amount because I'm bootstrapped. And, and I don't need huge amounts of money, and I, and I have a project that is of the right size to where I can charge a decent amount of money. [00:31:32] That means that I can survive with 500 or a thousand customers. I don't need to have a hundred million dollars worth of customers. Because I, you know, when I started the business, one of the constraints I said is I don't want to hire anybody. I'm just going to be solo. And part of the, part of my ability to keep a low price and, and keep running sustainably, even with just You know, only a few hundred customers is because I'm solo. [00:32:03] I don't have the overhead of investors. I don't have the overhead of other employees. I don't have an office space. You know, my overhead is very small. So that is, um, you know, I just kind of have a unique business in that way, I guess you might say. Keeping the business solo [00:32:21] Jeremy: I think that's that's interesting about your business as well But the fact that you've kept it you've kept it solo which I would imagine in most businesses, they need support people. they need, developers outside of maybe just one. Um, there's all sorts of other, I don't think overhead is the right word, but you just need more people, right? [00:32:45] And, and what do you think it is about Sidekiq that's made it possible for it to just be a one person operation? [00:32:52] Mike: There's so much administrative overhead in a business. I explicitly create business policies so that I can run solo. you know, my support policy is officially you get one email ticket or issue per quarter. And, and anything more than that, I can bounce back and say, well, you're, you're requiring too much support. [00:33:23] In reality, I don't enforce that at all. And people email me all the time, but, but things like. Things like dealing with accounting and bookkeeping and taxes and legal stuff, licensing, all that is, yeah, a little bit of overhead, but I've kept it as minimal as I can. And part of that is I don't want to hire another employee because then that increases the administrative overhead that I have. [00:33:53] And Sidekiq is so tied to me and my knowledge that if I hire somebody, they're probably not going to know Ruby and threading and all the intricate technical detail necessary to build and maintain and support the system. And so really you'll kind of regress a little bit. We won't be able to give as good support because I'm busy helping that other employee. Being selective about customers [00:34:23] Mike: So, yeah, it's, it's a tightrope act where you've got to really figure out how can I scale myself as far as possible without overwhelming myself. The, the overwhelming thing that I have that I've never been able to solve. It's just dealing with billing inquiries, customers, companies, emailing me saying, how do we buy this thing? [00:34:46] Can I get an invoice? Every company out there, it seems wants an invoice. And the problem with invoicing is it takes a lot more. manual labor and administrative overhead to issue that invoice to collect payment on the invoice. So that's one of the reasons why I have a very strict policy about credit card only for, for the vast majority of my customers. [00:35:11] And I demand that companies pay a lot more. You have to have a pretty big enterprise license if you want an invoice. And if the company, if the company comes back and complains and says, well, you know, that's ridiculous. We don't, we don't want to pay that much. We don't need it that much. Uh, you know, I, I say, okay, well then you have two, two things, two, uh, two things. [00:35:36] You can either pay with a credit card or you can not use Sidekiq. Like, that's, that's it. I'm, I don't need your money. I don't want the administrative overhead of dealing with your accounting department. I just want to support my, my customers and build my software. And, and so, yeah, I don't want to turn into a billing clerk. [00:35:55] So sometimes, sometimes the, the, the best thing in business that you can do is just say no. [00:36:01] Jeremy: That's very interesting because I think being a solo... Person is what probably makes that possible, right? Because if you had the additional staff, then you might say like, Well, I need to pay my staff, so we should be getting, you know, as much business as [00:36:19] Mike: Yeah. Chasing every customer you can, right. But yeah. [00:36:22] Every customer is different. I mean, I have some customers that just, they never contact me. They pay their bill really fast or right on time. And they're paying me, you know, five figures, 20, a year. And they just, it's a, God bless them because those are, are the. [00:36:40] Best customers to have and the worst customers are the ones who are paying 99 bucks a month and everything that they don't understand or whatever is a complaint. So sometimes, sometimes you, you want to, vet your customers from that perspective and say, which one of these customers are going to be good? [00:36:58] Which ones are going to be problematic? [00:37:01] Jeremy: And you're only only person... And I'm not sure how many customers you have, but [00:37:08] Mike: I have 2000 [00:37:09] Jeremy: 2000 customers. [00:37:10] Okay. [00:37:11] Mike: Yeah. [00:37:11] Jeremy: And has that been relatively stable or has there been growth [00:37:16] Mike: It's been relatively stable the last couple of years. Ruby has, has sort of plateaued. Um, it's, you don't see a lot of growth. I'm getting probably, um, 15, 20 percent growth maybe. Uh, so I'm not growing like a weed, like, you know, venture capital would want to see, but steady incremental growth is, is, uh, wonderful, especially since I do very little. [00:37:42] Sales and marketing. you know, I come to RubyConf I, I I tweet out, you know, or I, I toot out funny Mastodon Toots occasionally and, and, um, and, and put out new releases of the software. And, and that's, that's essentially my, my marketing. My marketing is just staying in front of developers and, and, and being a presence in the Ruby community. [00:38:06] But yeah, it, it's, uh. I, I, I see not a, not a huge amount of churn, but I see enough sales to, to, to stay up and keep my head above water and to keep growing, um, slowly but surely. Support needs haven't grown [00:38:20] Jeremy: And as you've had that steady growth, has the support burden not grown with it? [00:38:27] Mike: Not as much because once customers are on Sidekiq and they've got it working, then by and large, you don't hear from them all that much. There's always GitHub issues, you know, customers open GitHub issues. I love that. but yeah, by and large, the community finds bugs. and opens up issues. And so things remain relatively stable. [00:38:51] I don't get a lot of the complete newbie who has no idea what they're doing and wants me to, to tell them how to use Sidekiq that I just don't see much of that at all. Um, I have seen it before, but in that case, generally, I, I, I politely tell that person that, listen, I'm not here to educate you on the product. [00:39:14] It's there's documentation in the wiki. Uh, and there's tons of, of more Ruby, generic Ruby, uh, educational material out there. That's just not, not what I do. So, so yeah, by and large, the support burden is, is not too bad because once people are, are up and running, it's stable and, and they don't, they don't need to contact me. [00:39:36] Jeremy: I wonder too, if that's perhaps a function of the price, because if you're a. new developer or someone who's not too familiar with how to do job processing or what they want to do when you, there is the open source product, of course. but then the next step up, I believe is about a hundred dollars a month. [00:39:58] And if you're somebody who is kind of just getting started and learning how things work, you're probably not going to pay that, is my guess. And so you'll never hear from them. [00:40:11] Mike: Right, yeah, that's a good point too, is the open source version, which is what people inevitably are going to use and integrate into their app at first. Because it's open source, you're not going to email me directly, um, and when people do email me directly, Sidekiq support questions, I do, I reply literally, I'm sorry I don't respond to private email, unless you're a customer. [00:40:35] Please open a GitHub issue and, um, that I try to educate both my open source users and my commercial customers to try and stay in GitHub issues because private email is a silo, right? Private email doesn't help anybody else but them. If I can get people to go into GitHub issues, then that's a public record. [00:40:58] that people can search. Because if one person has that problem, there's probably a dozen other people that have that same problem. And then that other, those other 11 people can search and find the solution to their problem at four in the morning when I'm asleep. Right? So that's, that's what I'm trying to do is, is keep, uh, keep everything out in the open so that people can self service as much as possible. Sidekiq open source [00:41:24] Jeremy: And on the open source side, are you still primarily the main contributor? Or do you have other people that are [00:41:35] Mike: I mean, I'd say I do 90 percent of the work, which is why I don't feel guilty about keeping 100 percent of the money. A lot of open source projects, when they look for financial sustainability, they also look for how can we split this money amongst the team. And that's, that's a completely different topic that I've. [00:41:55] is another reason why I've stayed solo is if I hire an employee and I pay them 200, 000 a year as a developer, I'm meanwhile keeping all the rest of the profits of the company. And so that almost seems a little bit unfair. because we're both still working 40 hours a week, right? Why am I the one making the vast majority of the, of the profit and the money? [00:42:19] Um, so, uh, I've always, uh, that's another reason why I've stayed solo, but, but yeah, having a team of people working on something, I do get, regular commits, regular pull requests from people, fixing a bug that they found or just making a tweak that. that they saw, that they thought they could improve. [00:42:42] A little more rarely I get a significant improvement or feature, as a pull request. but Sidekiq is so stable these days that it really doesn't need a team of people maintaining it. The volume of changes necessary, I can easily keep up with that. So, I'm still doing 90 95 percent of the work. Are there other Sidekiq-like opportunities out there? [00:43:07] Jeremy: Yeah, so I think Sidekiq has sort of a unique positioning where it's the code base itself is small enough where you can maintain it yourself and you have some help, but primarily you're the main maintainer. And then you have enough customers who are willing to, to pay for the benefit it gives them on top of what the open source product provides. [00:43:36] cause it's, it's, you were talking about how. Every project people work on, they have, they could have hundreds of dependencies, right? And to ask somebody to, to pay for each of them is, is probably not ever going to happen. And so it's interesting to think about how you have things like, say, you know, OpenSSL, you know, it's a library that a whole bunch of people rely on, but nobody is going to pay a monthly fee to use it. [00:44:06] You have things like, uh, recently there was HashiCorp with Terraform, right? They, they decided to change their license because they, they wanted to get, you know, some of that value back, some of the money back, and the community basically revolted. Right? And did a fork. And so I'm kind of curious, like, yeah, where people can find these sweet spots like, like Sidekiq, where they can find this space where it's just small enough where you can work on it on your own and still get people to pay for it. [00:44:43] It's, I'm trying to picture, like, where are the spaces? Open source as a public utility [00:44:48] Mike: We need to look at other forms of financing beyond pure capitalism. If this is truly public infrastructure that needs to be maintained for the long term, then why are we, why is it that we depend on capitalism to do that? Our roads, our water, our sewer, those are not Capitalist, right? Those are utilities, that's public infrastructure that we maintain, that the government helps us maintain. [00:45:27] And in a sense, tech infrastructure is similar or could be thought of in a similar fashion. So things like Open Collective, things like, uh, there's a, there's a organization in Europe called NLNet, I think, out of the Netherlands. And they do a lot of grants to various open source projects to help them improve the state of digital infrastructure. [00:45:57] They support, for instance, Mastodon as a open source project that doesn't have any sort of corporate backing. They see that as necessary social media infrastructure, uh, for the long term. And, and I, and I think that's wonderful. I like to see those new directions being explored where you don't have to turn everything into a product, right? [00:46:27] And, and try and market and sale, um, and, and run ads and, and do all this stuff. If you can just make the case that, hey, this is, this is useful public infrastructure that so many different, um, Technical, uh, you know, applications and businesses could rely on, much like FedEx and DHL use our roads to the benefit of their own, their own corporate profits. [00:46:53] Um, why, why, why shouldn't we think of tech infrastructure sort of in a similar way? So, yeah, I would like to see us explore more. in that direction. I understand that in America that may not happen for quite a while because we are very, capitalist focused, but it's encouraging to see, um, places like Europe, uh, a little more open to, to trialing things like, cooperatives and, and grants and large long term grants to, to projects to see if they can, uh, provide sustainability in, in, you know, in a new way. [00:47:29] Jeremy: Yeah, that's a good point because I think right now, a lot of the open source infrastructure that we all rely on, either it's being paid for by large companies and at the whim of those large companies, if Google decides we don't want to pay for you to work on this project anymore, where does the money come from? [00:47:53] Right? And on the other hand, there's the thousands, tens of thousands of people who are doing it. just for free out of the, you know, the goodness of their, their heart. And that's where a lot of the burnout comes from. Right. So I think what you're saying is that perhaps a lot of these pieces that we all rely on, that our, our governments, you know, here in the United States, but also around the world should perhaps recognize as this is, like you said, this is infrastructure, and we should be. [00:48:29] Paying these people to keep the equivalent of the roads and, and, uh, all that working. [00:48:37] Mike: Yeah, I mean, I'm not, I'm not claiming that it's a perfect analogy. There's, there's, there's lots of questions that are unanswered in that, right? How do you, how do you ensure that a project is well maintained? What does that even look like? What does that mean? you know, you can look at a road and say, is it full of potholes or is it smooth as glass, right? [00:48:59] It's just perfectly obvious, but to a, to a digital project, it's, it's not as clear. So, yeah, but, but, but exploring those new ways because turning everybody into a businessman so that they can, they can keep their project going, it, it, it itself is not sustainable, right? so yeah, and that's why everything turns into a SaaS because a SaaS is easy to control. [00:49:24] It's easy to gatekeep behind a paywall and it's easy to charge for, whereas a library on GitHub. Yeah. You know, what do you do there? You know, obviously GitHub has sponsors, the sponsors feature. You've got Patreon, you've got Open Collective, you've got Tidelift. There's, there's other, you know, experiments that have been run, but nothing has risen to the top yet. [00:49:47] and it's still, it's still a bit of a grind. but yeah, we'll see, we'll see what happens, but hopefully people will keep experimenting and, and maybe, maybe governments will start. Thinking in the direction of, you know, what does it mean to have a budget for digital infrastructure maintenance? [00:50:04] Jeremy: Yeah, it's interesting because we, we started thinking about like, okay, where can we find spaces for other Sidekiqs? But it sounds like maybe, maybe that's just not realistic, right? Like maybe we need more of a... Yeah, a rethinking of, I guess the, the structure of how people get funded. Yeah. [00:50:23] Mike: Yeah, sometimes the best way to solve a problem is to think at a higher level. You know, we, the, the sustainability problem in American Silicon Valley based open source developers is naturally going to tend toward venture capital and, and capitalism. And I, you know, I think, I think that's, uh, extremely problematic on a, on a lot of different, in a lot of different ways. [00:50:47] And, and so sometimes you need to step back and say, well, maybe we're, maybe we just don't have the right tool set to solve this problem. But, you know, I, I. More than that, I'm not going to speculate on because it is a wicked problem to solve. [00:51:04] Jeremy: Is there anything else you wanted to, to mention or thought we should have talked about? [00:51:08] Mike: No, I, I, I loved the talk, of sustainability and, and open source. And I, it's, it's a, it's a topic really dear to my heart, obviously. So I, I am happy to talk about it at length with anybody, anytime. So thank you for having me. [00:51:25] Jeremy: All right. Thank you very much, Mike.

Brian Merchant is the technology columnist at the Los Angeles Times and author of the new book, "Blood in the Machine: the Origins of the Rebellion against Big Tech." He is also the author of the 2017 book, "The One Device: the Secret History of the iPhone." Brian is a founder of VICE's speculative fiction outlet Terraform, and published a best-of anthology, "Terraform: Watch/Worlds/Burn, with MCD Books, in 2022." He was our guest on Episode 134 of the Agents of Innovation podcast, filmed and recorded at the 40th Annual Miami Book Fair, on November 19, 2023. This can also be viewed on our YouTube channel here: https://youtu.be/7m6mZUb6JUs You can learn more about Brian Merchant at: https://brianmerchant.org Find his new book at: https://www.hachettebookgroup.com/titles/brian-merchant/blood-in-the-machine/9780316487740/ Learn more about the Miami Book Fair at: https://www.miamibookfair.com Follow the Agents of Innovation podcast on: Facebook: https://www.facebook.com/AgentsOfInnovationPodcast Instagram: https://www.instagram.com/innovationradio/ Twitter: https://twitter.com/agentinnovation You can support this podcast and our Fearless Journeys community on our Patreon account: www.patreon.com/fearlessjourneys You can also join our network through the Fearless Journeys community at: https://www.fearlessjourneys.org/

¡Bienvenidxs a Navegando el Espacio Cripto! En esta edición exploramos el registro oficial del spot ETF de ETH de BlackRock, la nueva L2 que el exchange OKX lanzará, una nueva jugada estratégica de Terraform Labs y muchas otras noticias más. ¡Acompáñanos! 00:00 Intro. 01:38 ¿De qué hablaremos esta semana? 06:51 Precios 15:18 BlackRock Presenta Solicitud de ETF de Ethereum Spot https://decrypt.co/es/206355/blackrock-presenta-solicitud-de-etf-de-ethereum-spot  20:12 (NO) tendremos ETF de XRP https://cointelegraph.com/news/delaware-authorities-refer-fake-blackrock-xrp-trust-filing-to-state-s-justice-department 26:15 Ark Invest lanza 5 nuevos fondos para estrategias de inversión con ETH y BTC https://blockworks.co/news/spot-crypto-etf-ark-invest-21shares 29:55 OpenAI despide a su director ejecutivo Sam Altman; destacan ‘pérdida de confianza' https://www.theguardian.com/technology/2023/nov/17/openai-ceo-sam-altman-fired 40:27 OKX lanzará su propia Layer 2 X1 https://www.coindesk.com/tech/2023/11/14/cryptocurrency-exchange-okx-coming-out-with-layer-2-x1-built-on-polygon-technology/ 44:26 También se rumora de la L2 de Kraken https://www.coindesk.com/tech/2023/11/07/kraken-said-to-seek-partner-to-help-build-it-a-layer-2-blockchain-network/ 45:04 Infura crece con grandes jugadores como Microsoft y Tencent https://cointelegraph.com/news/microsoft-tencent-infura-network-partnership 47:53 Disney se asocia con Dapper Labs para lanzar una plataforma de Pines digitales https://www.theblock.co/post/263005/disney-and-dapper-labs-launch-star-wars-pixar-digital-pin-platform 52:39 Polkadot transformará el mercado de commodities de agricultura https://alexablockchain.com/agrotoken-launching-on-polkadot/ PancakeSwap lanza un marketplace para gaming https://www.theblock.co/post/263010/pancakeswap-launches-gaming-marketplace 55:12 Terraform Labs compra un proveedor de información cross-chain https://www.theblock.co/post/262739/terraform-labs-acquires-cross-chain-data-provider-as-it-pursues-post-kwon-plans 59:26 The Block valuada en 70 millones https://www.axios.com/pro/fintech-deals/2023/11/13/the-block-70-million-foresight-ventures Learn more about your ad choices. Visit megaphone.fm/adchoices

Cloud Posse holds LIVE "Office Hours" every Wednesday to answer questions on all things related to AWS, DevOps, Terraform, Kubernetes, CI/CD. Register at https://cloudposse.com/office-hoursSupport the show

We dive into the dynamics of open source projects and monetization today, specifically starting around the TerraForm and open tofu split. That topic is one that we love to chew over and potentially over analyze, but today's discussion is different. We go into how ecosystems are built both in open and proprietary and cloud systems, and look at sort of a historical perspective on what makes a project successful from an ecosystem perspective. We also dive into why some projects work like that, and why some projects don't. Today's episode gives a new take on some of the dynamics going on in the open source communities through the lens of what happened with Open Tofu and TerraForm. Transcript: https://otter.ai/u/ONDvgS9yGMrSN-bXMTy9KMC-XPs?utm_source=copy_url Photo by James Wheeler: https://www.pexels.com/photo/lake-pebbles-under-body-of-water-1574181/

Cloud Posse holds LIVE "Office Hours" every Wednesday to answer questions on all things related to AWS, DevOps, Terraform, Kubernetes, CI/CD. Register at https://cloudposse.com/office-hoursSupport the show

Amir Szekely, Owner at CloudSnorkel, joins Corey on Screaming in the Cloud to discuss how he got his start in the early days of cloud and his solo project, CloudSnorkel. Throughout this conversation, Corey and Amir discuss the importance of being pragmatic when moving to the cloud, and the different approaches they see in developers from the early days of cloud to now. Amir shares what motivates him to develop open-source projects, and why he finds fulfillment in fixing bugs and operating CloudSnorkel as a one-man show. About AmirAmir Szekely is a cloud consultant specializing in deployment automation, AWS CDK, CloudFormation, and CI/CD. His background includes security, virtualization, and Windows development. Amir enjoys creating open-source projects like cdk-github-runners, cdk-turbo-layers, and NSIS.Links Referenced: CloudSnorkel: https://cloudsnorkel.com/ lasttootinaws.com: https://lasttootinaws.com camelcamelcamel.com: https://camelcamelcamel.com github.com/cloudsnorkel: https://github.com/cloudsnorkel Personal website: https://kichik.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn, and this is an episode that I have been angling for for longer than you might imagine. My guest today is Amir Szekely, who's the owner at CloudSnorkel. Amir, thank you for joining me.Amir: Thanks for having me, Corey. I love being here.Corey: So, I've been using one of your open-source projects for an embarrassingly long amount of time, and for the longest time, I make the critical mistake of referring to the project itself as CloudSnorkel because that's the word that shows up in the GitHub project that I can actually see that jumps out at me. The actual name of the project within your org is cdk-github-runners if I'm not mistaken.Amir: That's real original, right?Corey: Exactly. It's like, “Oh, good, I'll just mention that, and suddenly everyone will know what I'm talking about.” But ignoring the problems of naming things well, which is a pain that everyone at AWS or who uses it knows far too well, the product is basically magic. Before I wind up basically embarrassing myself by doing a poor job of explaining what it is, how do you think about it?Amir: Well, I mean, it's a pretty simple project, which I think what makes it great as well. It creates GitHub runners with CDK. That's about it. It's in the name, and it just does that. And I really tried to make it as simple as possible and kind of learn from other projects that I've seen that are similar, and basically learn from my pain points in them.I think the reason I started is because I actually deployed CDK runners—sorry, GitHub runners—for one company, and I ended up using the Kubernetes one, right? So, GitHub in themselves, they have two projects they recommend—and not to nudge GitHub, please recommend my project one day as well—they have the Kubernetes controller and they have the Terraform deployer. And the specific client that I worked for, they wanted to use Kubernetes. And I tried to deploy it, and, Corey, I swear, I worked three days; three days to deploy the thing, which was crazy to me. And every single step of the way, I had to go and read some documentation, figure out what I did wrong, and apparently the order the documentation was was incorrect.And I had to—I even opened tickets, and they—you know, they were rightfully like, “It's open-source project. Please contribute and fix the documentation for us.” At that point, I said, “Nah.” [laugh]. Let me create something better with CDK and I decided just to have the simplest setup possible.So usually, right, what you end up doing in these projects, you have to set up either secrets or SSM parameters, and you have to prepare the ground and you have to get your GitHub token and all those things. And that's just annoying. So, I decided to create a—Corey: So much busy work.Amir: Yes, yeah, so much busy work and so much boilerplate and so much figuring out the right way and the right order, and just annoying. So, I decided to create a setup page. I thought, “What if you can actually install it just like you install any app on GitHub,” which is the way it's supposed to be right? So, when you install cdk-github-runners—CloudSnorkel—you get an HTML page and you just click a few buttons and you tell it where to install it and it just installs it for you. And it sets the secrets and everything. And if you want to change the secret, you don't have to redeploy. You can just change the secret, right? You have to roll the token over or whatever. So, it's much, much easier to install.Corey: And I feel like I discovered this project through one of the more surreal approaches—and I had cause to revisit it a few weeks ago when I was redoing my talk for the CDK Community Day, which has since happened and people liked the talk—and I mentioned what CloudSnorkel had been doing and how I was using the runners accordingly. So, that was what I accidentally caused me to pop back up with, “Hey, I've got some issues here.” But we'll get to that. Because once upon a time, I built a Twitter client for creating threads because shitposting is my love language, I would sit and create Twitter threads in the middle of live keynote talks. Threading in the native client was always terrible, and I wanted to build something that would help me do that. So, I did.And it was up for a while. It's not anymore because I'm not paying $42,000 a month in API costs to some jackass, but it still exists in the form of lasttootinaws.com if you want to create threads on Mastodon. But after I put this out, some people complained that it was slow.To which my response was, “What do you mean? It's super fast for me in San Francisco talking to it hosted in Oregon.” But on every round trip from halfway around the world, it became a problem. So, I got it into my head that since this thing was fully stateless, other than a Lambda function being fronted via an API Gateway, that I should deploy it to every region. It didn't quite fit into a Cloudflare Worker or into one of the Edge Lambda functions that AWS has given up on, but okay, how do I deploy something to every region?And the answer is, with great difficulty because it's clear that no one was ever imagining with all those regions that anyone would use all of them. It's imagined that most customers use two or three, but customers are different, so which two or three is going to be widely varied. So, anything halfway sensible about doing deployments like this didn't work out. Again, because this thing was also a Lambda function and an API Gateway, it was dirt cheap, so I didn't really want to start spending stupid amounts of money doing deployment infrastructure and the rest.So okay, how do I do this? Well, GitHub Actions is awesome. It is basically what all of AWS's code offerings wish that they were. CodeBuild is sad and this was kind of great. The problem is, once you're out of the free tier, and if you're a bad developer where you do a deploy on every iteration, suddenly it starts costing for what I was doing in every region, something like a quarter of per deploy, which adds up when you're really, really bad at programming.Amir: [laugh].Corey: So, their matrix jobs are awesome, but I wanted to do some self-hosted runners. How do I do that? And I want to keep it cheap, so how do I do a self-hosted runner inside of a Lambda function? Which led me directly to you. And it was nothing short of astonishing. This was a few years ago. I seem to recall that it used to be a bit less well-architected in terms of its elegance. Did it always use step functions, for example, to wind up orchestrating these things?Amir: Yeah, so I do remember that day. We met pretty much… basically as a joke because the Lambda Runner was a joke that I did, and I posted on Twitter, and I was half-proud of my joke that starts in ten seconds, right? But yeah, no, the—I think it always used functions. I've been kind of in love with the functions for the past two years. They just—they're nice.Corey: Oh, they're magic, and AWS is so bad at telling their story. Both of those things are true.Amir: Yeah. And the API is not amazing. But like, when you get it working—and you know, you have to spend some time to get it working—it's really nice because then you have nothing to manage, ever. And they can call APIs directly now, so you don't have to even create Lambdas. It's pretty cool.Corey: And what I loved is you wind up deploying this thing to whatever account you want it to live within. What is it, the OIDC? I always get those letters in the wrong direction. OIDC, I think, is correct.Amir: I think it's OIDC, yeah.Corey: Yeah, and it winds up doing this through a secure method as opposed to just okay, now anyone with access to the project can deploy into your account, which is not ideal. And it just works. It spins up a whole bunch of these Lambda functions that are using a Docker image as the deployment environment. And yeah, all right, if effectively my CDK deploy—which is what it's doing inside of this thing—doesn't complete within 15 minutes, then it's not going to and the thing is going to break out. We've solved the halting problem. After 15 minutes, the loop will terminate. The end.But that's never been a problem, even with getting ACM certificates spun up. It completes well within that time limit. And its cost to me is effectively nothing. With one key exception: that you made the choice to use Secrets Manager to wind up storing a lot of the things it cares about instead of Parameter Store, so I think you wind up costing me—I think there's two of those different secrets, so that's 80 cents a month. Which I will be demanding in blood one of these days if I ever catch you at re:Invent.Amir: I'll buy you beer [laugh].Corey: There we go. That'll count. That'll buy, like, several months of that. That works—at re:Invent, no. The beers there are, like, $18, so that'll cover me for years. We're set.Amir: We'll split it [laugh].Corey: Exactly. Problem solved. But I like the elegance of it, I like how clever it is, and I want to be very clear, though, it's not just for shitposting. Because it's very configurable where, yes, you can use Lambda functions, you can use Spot Instances, you can use CodeBuild containers, you can use Fargate containers, you can use EC2 instances, and it just automatically orchestrates and adds these self-hosted runners to your account, and every build gets a pristine environment as a result. That is no small thing.Amir: Oh, and I love making things configurable. People really appreciate it I feel, you know, and gives people kind of a sense of power. But as long as you make that configuration simple enough, right, or at least the defaults good defaults, right, then, even with that power, people still don't shoot themselves in the foot and it still works really well. By the way, we just added ECS recently, which people really were asking for because it gives you the, kind of, easy option to have the runner—well, not the runner but at least the runner infrastructure staying up, right? So, you can have auto-scaling group backing ECS and then the runner can start up a lot faster. It was actually very important to other people because Lambda, as fast that it is, it's limited, and Fargate, for whatever reason, still to this day, takes a minute to start up.Corey: Yeah. What's wild to me about this is, start to finish, I hit a deploy to the main branch and it sparks the thing up, runs the deploy. Deploy itself takes a little over two minutes. And every time I do this, within three minutes of me pushing to commit, the deploy is done globally. It is lightning fast.And I know it's easy to lose yourself in the idea of this being a giant shitpost, where, oh, who's going to do deployment jobs in Lambda functions? Well, kind of a lot of us for a variety of reasons, some of which might be better than others. In my case, it was just because I was cheap, but the massive parallelization ability to do 20 simultaneous deploys in a matrix configuration that doesn't wind up smacking into rate limits everywhere, that was kind of great.Amir: Yeah, we have seen people use Lambda a lot. It's mostly for, yeah, like you said, small jobs. And the environment that they give you, it's kind of limited, so you can't actually install packages, right? There is no sudo, and you can't actually install anything unless it's in your temp directory. But still, like, just being able to run a lot of little jobs, it's really great. Yeah.Corey: And you can also make sure that there's a Docker image ready to go with the stuff that you need, just by configuring how the build works in the CDK. I will admit, I did have a couple of bug reports for you. One was kind of useful, where it was not at all clear how to do this on top of a Graviton-based Lambda function—because yeah, that was back when not everything really supported ARM architectures super well—and a couple of other times when the documentation was fairly ambiguous from my perspective, where it wasn't at all clear, what was I doing? I spent four hours trying to beat my way through it, I give up, filed an issue, went to get a cup of coffee, came back, and the answer was sitting there waiting for me because I'm not convinced you sleep.Amir: Well, I am a vampire. My last name is from the Transylvania area [laugh]. So—Corey: Excellent. Excellent.Amir: By the way, not the first time people tell me that. But anyway [laugh].Corey: There's something to be said for getting immediate responsiveness because one of the reasons I'm always so loath to go and do a support ticket anywhere is this is going to take weeks. And then someone's going to come back with a, “I don't get it.” And try and, like, read the support portfolio to you. No, you went right into yeah, it's this. Fix it and your problem goes away. And sure enough, it did.Amir: The escalation process that some companies put you through is very frustrating. I mean, lucky for you, CloudSnorkel is a one-man show and this man loves solving bugs. So [laugh].Corey: Yeah. Do you know of anyone using it for anything that isn't ridiculous and trivial like what I'm using it for?Amir: Yeah, I have to think whether or not I can… I mean, so—okay. We have a bunch of dedicated users, right, the GitHub repo, that keep posting bugs and keep posting even patches, right, so you can tell that they're using it. I even have one sponsor, one recurring sponsor on GitHub that uses it.Corey: It's always nice when people thank you via money.Amir: Yeah. Yeah, it is very validating. I think [BLEEP] is using it, but I also don't think I can actually say it because I got it from the GitHub.Corey: It's always fun. That's the beautiful part about open-source. You don't know who's using this. You see what other things people are working on, and you never know, is one of their—is this someone's side project, is it a skunkworks thing, or God forbid, is this inside of every car going forward and no one bothered to tell me about that. That is the magic and mystery of open-source. And you've been doing open-source for longer than I have and I thought I was old. You were originally named in some of the WinAMP credits, for God's sake, that media player that really whipped the llama's ass.Amir: Oh, yeah, I started real early. I started about when I was 15, I think. I started off with Pascal or something or even Perl, and then I decided I have to learn C and I have to learn Windows API. I don't know what possessed me to do that. Win32 API is… unique [laugh].But once I created those applications for myself, right, I think there was—oh my God, do you know the—what is it called, Sherlock in macOS, right? And these days, for PowerToys, there is the equivalent of it called, I don't know, whatever that—PowerBar? That's exactly—that was that. That's a project I created as a kid. I wanted something where I can go to the Run menu of Windows when you hit Winkey R, and you can just type something and it will start it up, right?I didn't want to go to the Start menu and browse and click things. I wanted to do everything with the keyboard. So, I created something called Blazerun [laugh], which [laugh] helped you really easily create shortcuts that went into your path, right, the Windows path, so you can really easily start them from Winkey R. I don't think that anyone besides me used it, but anyway, that thing needed an installer, right? Because Windows, you got to install things. So, I ended up—Corey: Yeah, these days on Mac OS, I use Alfred for that which is kind of long in the tooth, but there's a launch bar and a bunch of other stuff for it. What I love is that if I—I can double-tap the command key and that just pops up whatever I need it to and tell the computer what to do. It feels like there's an AI play in there somewhere if people can figure out how to spend ten minutes on building AI that does something other than lets them fire their customer service staff.Amir: Oh, my God. Please don't fire customer service staff. AI is so bad.Corey: Yeah, when I reach out to talk to a human, I really needed a human.Amir: Yes. Like, I'm not calling you because I want to talk to a robot. I know there's a website. Leave me alone, just give me a person.Corey: Yeah. Like, you already failed to solve my problem on your website. It's person time.Amir: Exactly. Oh, my God. Anyway [laugh]. So, I had to create an installer, right, and I found it was called NSIS. So, it was a Nullsoft “SuperPiMP” installation system. Or in the future, when Justin, the guy who created Winamp and NSIS, tried to tone down a little bit, Nullsoft Scriptable Installation System. And SuperPiMP is—this is such useless history for you, right, but SuperPiMP is the next generation of PiMP which is Plug-in Mini Packager [laugh].Corey: I remember so many of the—like, these days, no one would ever name any project like that, just because it's so off-putting to people with sensibilities, but back then that was half the stuff that came out. “Oh, you don't like how this thing I built for free in the wee hours when I wasn't working at my fast food job wound up—you know, like, how I chose to name it, well, that's okay. Don't use it. Go build your own. Oh, what you're using it anyway. That's what I thought.”Amir: Yeah. The source code was filled with profanity, too. And like, I didn't care, I really did not care, but some people would complain and open bug reports and patches. And my policy was kind of like, okay if you're complaining, I'm just going to ignore you. If you're opening a patch, fine, I'm going to accept that you're—you guys want to create something that's sensible for everybody, sure.I mean, it's just source code, you know? Whatever. So yeah, I started working on that NSIS. I used it for myself and I joined the forums—and this kind of answers to your question of why I respond to things so fast, just because of the fun—I did the same when I was 15, right? I started going on the forums, you remember forums? You remember that [laugh]?Corey: Oh, yeah, back before they all became terrible and monetized.Amir: Oh, yeah. So, you know, people were using NSIS, too, and they had requests, right? They wanted. Back in the day—what was it—there was only support for 16-bit colors for the icon, so they want 32-bit colors and big colors—32—big icon, sorry, 32 pixels by 32 pixels. Remember, 32 pixels?Corey: Oh, yes. Not well, and not happily, but I remember it.Amir: Yeah. So, I started just, you know, giving people—working on that open-source and creating up a fork. It wasn't even called ‘fork' back then, but yeah, I created, like, a little fork of myself and I started adding all these features. And people were really happy, and kind of created, like, this happy cycle for myself: when people were happy, I was happy coding. And then people were happy by what I was coding. And then they were asking for more and they were getting happier, the more I responded.So, it was kind of like a serotonin cycle that made me happy and made everybody happy. So, it's like a win, win, win, win, win. And that's how I started with open-source. And eventually… NSIS—again, that installation system—got so big, like, my fork got so big, and Justin, the guy who works on WinAMP and NSIS, he had other things to deal with. You know, there's a whole history there with AOL. I'm sure you've heard all the funny stories.Corey: Oh, yes. In fact, one thing that—you want to talk about weird collisions of things crossing, one of the things I picked up from your bio when you finally got tired of telling me no and agreed to be on the show was that you're also one of the team who works on camelcamelcamel.com. And I keep forgetting that's one of those things that most people have no idea exists. But it's very simple: all it does is it tracks Amazon products that you tell it to and alerts you when there's a price drop on the thing that you're looking at.It's something that is useful. I try and use it for things of substance or hobbies because I feel really pathetic when I'm like, get excited emails about a price drop in toilet paper. But you know, it's very handy just to keep an idea for price history, where okay, am I actually being ripped off? Oh, they claim it's their big Amazon Deals day and this is 40% off. Let's see what camelcamelcamel has to say.Oh, surprise. They just jacked the price right beforehand and now knocked 40% off. Genius. I love that. It always felt like something that was going to be blown off the radar by Amazon being displeased, but I discovered you folks in 2010 and here you are now, 13 years later, still here. I will say the website looks a lot better now.Amir: [laugh]. That's a recent change. I actually joined camel, maybe two or three years ago. I wasn't there from the beginning. But I knew the guy who created it—again, as you were saying—from the Winamp days, right? So, we were both working in the free—well, it wasn't freenode. It was not freenode. It was a separate IRC server that, again, Justin created for himself. It was called landoleet.Corey: Mmm. I never encountered that one.Amir: Yeah, no, it was pretty private. The only people that cared about WinAMP and NSIS ended up joining there. But it was a lot of fun. I met a lot of friends there. And yeah, I met Daniel Green there as well, and he's the guy that created, along with some other people in there that I think want to remain anonymous so I'm not going to mention, but they also were on the camel project.And yeah, I was kind of doing my poor version of shitposting on Twitter about AWS, kind of starting to get some traction and maybe some clients and talk about AWS so people can approach me, and Daniel approached me out of the blue and he was like, “Do you just post about AWS on Twitter or do you also do some AWS work?” I was like, “I do some AWS work.”Corey: Yes, as do all of us. It's one of those, well crap, we're getting called out now. “Do you actually know how any of this stuff works?” Like, “Much to my everlasting shame, yes. Why are you asking?”Amir: Oh, my God, no, I cannot fix your printer. Leave me alone.Corey: Mm-hm.Amir: I don't want to fix your Lambdas. No, but I do actually want to fix your Lambdas. And so, [laugh] he approached me and he asked if I can help them move camelcamelcamel from their data center to AWS. So, that was a nice big project. So, we moved, actually, all of camelcamelcamel into AWS. And this is how I found myself not only in the Winamp credits, but also in the camelcamelcamel credits page, which has a great picture of me riding a camel.Corey: Excellent. But one of the things I've always found has been that when you take an application that has been pre-existing for a while in a data center and then move it into the cloud, you suddenly have to care about things that no one sensible pays any attention to in the land of the data center. Because it's like, “What do I care about how much data passes between my application server and the database? Wait, what do you mean that in this configuration, that's a chargeable data transfer? Oh, dear Lord.” And things that you've never had to think about optimizing are suddenly things are very much optimizing.Because let's face it, when it comes to putting things in racks and then running servers, you aren't auto-scaling those things, so everything tends to be running over-provisioned, for very good reasons. It's an interesting education. Anything you picked out from that process that you think it'd be useful for folks to bear in mind if they're staring down the barrel of the same thing?Amir: Yeah, for sure. I think… in general, right, not just here. But in general, you always want to be pragmatic, right? You don't want to take steps are huge, right? So, the thing we did was not necessarily rewrite everything and change everything to AWS and move everything to Lambda and move everything to Docker.Basically, we did a mini lift-and-shift, but not exactly lift-and-shift, right? We didn't take it as is. We moved to RDS, we moved to ElastiCache, right, we obviously made use of security groups and session connect and we dropped SSH Sage and we improved the security a lot and we locked everything down, all the permissions and all that kind of stuff, right? But like you said, there's stuff that you start having to pay attention to. In our case, it was less the data transfer because we have a pretty good CDN. There was more of IOPS. So—and IOPS, specifically for a database.We had a huge database with about one terabyte of data and a lot of it is that price history that you see, right? So, all those nice little graphs that we create in—what do you call them, charts—that we create in camelcamelcamel off the price history. There's a lot of data behind that. And what we always want to do is actually remove that from MySQL, which has been kind of struggling with it even before the move to AWS, but after the move to AWS, where everything was no longer over-provisioned and we couldn't just buy a few more NVMes on Amazon for 100 bucks when they were on sale—back when we had to pay Amazon—Corey: And you know, when they're on sale. That's the best part.Amir: And we know [laugh]. We get good prices on NVMe. But yeah, on Amazon—on AWS, sorry—you have to pay for io1 or something, and that adds up real quick, as you were saying. So, part of that move was also to move to something that was a little better for that data structure. And we actually removed just that data, the price history, the price points from MySQL to DynamoDB, which was a pretty nice little project.Actually, I wrote about it in my blog. There is, kind of, lessons learned from moving one terabyte from MySQL to DynamoDB, and I think the biggest lesson was about hidden price of storage in DynamoDB. But before that, I want to talk about what you asked, which was the way that other people should make that move, right? So again, be pragmatic, right? If you Google, “How do I move stuff from DynamoDB to MySQL,” everybody's always talking about their cool project using Lambda and how you throttle Lambda and how you get throttled from DynamoDB and how you set it up with an SQS, and this and that. You don't need all that.Just fire up an EC2 instance, write some quick code to do it. I used, I think it was Go with some limiter code from Uber, and that was it. And you don't need all those Lambdas and SQS and the complication. That thing was a one-time thing anyway, so it doesn't need to be super… super-duper serverless, you know?Corey: That is almost always the way that it tends to play out. You encounter these weird little things along the way. And you see so many things that are tied to this is how architecture absolutely must be done. And oh you're not a real serverless person if you don't have everything running in Lambda and the rest. There are times where yeah, spin up an EC2 box, write some relatively inefficient code in ten minutes and just do the thing, and then turn it off when you're done. Problem solved. But there's such an aversion to that. It's nice to encounter people who are pragmatists more than they are zealots.Amir: I mostly learned that lesson. And both Daniel Green and me learned that lesson from the Winamp days. Because we both have written plugins for Winamp and we've been around that area and you can… if you took one of those non-pragmatist people, right, and you had them review the Winamp code right now—or even before—they would have a million things to say. That code was—and NSIS, too, by the way—and it was so optimized. It was so not necessarily readable, right? But it worked and it worked amazing. And Justin would—if you think I respond quickly, right, Justin Frankel, the guy who wrote Winamp, he would release versions of NSIS and of Winamp, like, four versions a day, right? That was before [laugh] you had CI/CD systems and GitHub and stuff. That was just CVS. You remember CVS [laugh]?Corey: Oh, I've done multiple CVS migrations. One to Git and a couple to Subversion.Amir: Oh yeah, Subversion. Yep. Done ‘em all. CVS to Subversion to Git. Yep. Yep. That was fun.Corey: And these days, everyone's using Git because it—we're beginning to have a monoculture.Amir: Yeah, yeah. I mean, but Git is nicer than Subversion, for me, at least. I've had more fun with it.Corey: Talk about damning with faint praise.Amir: Faint?Corey: Yeah, anything's better than Subversion, let's be honest here.Amir: Oh [laugh].Corey: I mean, realistically, copying a bunch of files and directories to a.bak folder is better than Subversion.Amir: Well—Corey: At least these days. But back then it was great.Amir: Yeah, I mean, the only thing you had, right [laugh]?Corey: [laugh].Amir: Anyway, achieving great things with not necessarily the right tools, but just sheer power of will, that's what I took from the Winamp days. Just the entire world used Winamp. And by the way, the NSIS project that I was working on, right, I always used to joke that every computer in the world ran my code, every Windows computer in the world when my code, just because—Corey: Yes.Amir: So, many different companies use NSIS. And none of them cared that the code was not very readable, to put it mildly.Corey: So, many companies founder on those shores where they lose sight of the fact that I can point to basically no companies that died because their code was terrible, yeah, had an awful lot that died with great-looking code, but they didn't nail the business problem.Amir: Yeah. I would be lying if I said that I nailed exactly the business problem at NSIS because the most of the time I would spend there and actually shrinking the stub, right, there was appended to your installer data, right? So, there's a little stub that came—the executable, basically, that came before your data that was extracted. I spent, I want to say, years of my life [laugh] just shrinking it down by bytes—by literal bytes—just so it stays under 34, 35 kilobytes. It was kind of a—it was a challenge and something that people appreciated, but not necessarily the thing that people appreciate the most. I think the features—Corey: Well, no I have to do the same thing to make sure something fits into a Lambda deployment package. The scale changes, the problem changes, but somehow everything sort of rhymes with history.Amir: Oh, yeah. I hope you don't have to disassemble code to do that, though because that's uh… I mean, it was fun. It was just a lot.Corey: I have to ask, how much work went into building your cdk-github-runners as far as getting it to a point of just working out the door? Because I look at that and it feels like there's—like, the early versions, yeah, there wasn't a whole bunch of code tied to it, but geez, the iterative, “How exactly does this ridiculous step functions API work or whatnot,” feels like I'm looking at weeks of frustration. At least it would have been for me.Amir: Yeah, yeah. I mean, it wasn't, like, a day or two. It was definitely not—but it was not years, either. I've been working on it I think about a year now. Don't quote me on that. But I've put a lot of time into it. So, you know, like you said, the skeleton code is pretty simple: it's a step function, which as we said, takes a long time to get right. The functions, they are really nice, but their definition language is not very straightforward. But beyond that, right, once that part worked, it worked. Then came all the bug reports and all the little corner cases, right? We—Corey: Hell is other people's use cases. Always is. But that's honestly better than a lot of folks wind up experiencing where they'll put an open-source project up and no one ever knows. So, getting users is often one of the biggest barriers to a lot of this stuff. I've found countless hidden gems lurking around on GitHub with a very particular search for something that no one had ever looked at before, as best I can tell.Amir: Yeah.Corey: Open-source is a tricky thing. There needs to be marketing brought into it, there needs to be storytelling around it, and has to actually—dare I say—solve a problem someone has.Amir: I mean, I have many open-source projects like that, that I find super useful, I created for myself, but no one knows. I think cdk-github-runners, I'm pretty sure people know about it only because you talked about it on Screaming in the Cloud or your newsletter. And by the way, thank you for telling me that you talked about it last week in the conference because now we know why there was a spike [laugh] all of a sudden. People Googled it.Corey: Yeah. I put links to it as well, but it's the, yeah, I use this a lot and it's great. I gave a crappy explanation on how it works, but that's the trick I've found between conference talks and, dare I say, podcast episodes, you gives people a glimpse and a hook and tell them where to go to learn more. Otherwise, you're trying to explain every nuance and every intricacy in 45 minutes. And you can't do that effectively in almost every case. All you're going to do is drive people away. Make it sound exciting, get them to see the value in it, and then let them go.Amir: You have to explain the market for it, right? That's it.Corey: Precisely.Amir: And I got to say, I somewhat disagree with your—or I have a different view when you say that, you know, open-source projects needs marketing and all those things. It depends on what open-source is for you, right? I don't create open-source projects so they are successful, right? It's obviously always nicer when they're successful, but—and I do get that cycle of happiness that, like I was saying, people create bugs and I have to fix them and stuff, right? But not every open-source project needs to be a success. Sometimes it's just fun.Corey: No. When I talk about marketing, I'm talking about exactly what we're doing here. I'm not talking take out an AdWords campaign or something horrifying like that. It's you build something that solved the problem for someone. The big problem that worries me about these things is how do you not lose sleep at night about the fact that solve someone's problem and they don't know that it exists?Because that drives me nuts. I've lost count of the number of times I've been beating my head against a wall and asked someone like, “How would you handle this?” Like, “Oh, well, what's wrong with this project?” “What do you mean?” “Well, this project seems to do exactly what you want it to do.” And no one has it all stuffed in their head. But yeah, then it seems like open-source becomes a little more corporatized and it becomes a lead gen tool for people to wind up selling their SaaS services or managed offerings or the rest.Amir: Yeah.Corey: And that feels like the increasing corporatization of open-source that I'm not a huge fan of.Amir: Yeah. I mean, I'm not going to lie, right? Like, part of why I created this—or I don't know if it was part of it, but like, I had a dream that, you know, I'm going to get, oh, tons of GitHub sponsors, and everybody's going to use it and I can retire on an island and just make money out of this, right? Like, that's always a dream, right? But it's a dream, you know?And I think bottom line open-source is… just a tool, and some people use it for, like you were saying, driving sales into their SaaS, some people, like, may use it just for fun, and some people use it for other things. Or some people use it for politics, even, right? There's a lot of politics around open-source.I got to tell you a story. Back in the NSIS days, right—talking about politics—so this is not even about politics of open-source. People made NSIS a battleground for their politics. We would have translations, right? People could upload their translations. And I, you know, or other people that worked on NSIS, right, we don't speak every language of the world, so there's only so much we can do about figuring out if it's a real translation, if it's good or not.Back in the day, Google Translate didn't exist. Like, these days, we check Google Translate, we kind of ask a few questions to make sure they make sense. But back in the day, we did the best that we could. At some point, we got a patch for Catalan language, I'm probably mispronouncing it—but the separatist people in Spain, I think, and I didn't know anything about that. I was a young kid and… I just didn't know.And I just included it, you know? Someone submitted a patch, they worked hard, they wanted to be part of the open-source project. Why not? Sure I included it. And then a few weeks later, someone from Spain wanted to change Catalan into Spanish to make sure that doesn't exist for whatever reason.And then they just started fighting with each other and started making demands of me. Like, you have to do this, you have to do that, you have to delete that, you have to change the name. And I was just so baffled by why would someone fight so much over a translation of an open-source project. Like, these days, I kind of get what they were getting at, right?Corey: But they were so bad at telling that story that it was just like, so basically, screw, “You for helping,” is how it comes across.Amir: Yeah, screw you for helping. You're a pawn now. Just—you're a pawn unwittingly. Just do what I say and help me in my political cause. I ended up just telling both of them if you guys can agree on anything, I'm just going to remove both translations. And that's what I ended up doing. I just removed both translations. And then a few months later—because we had a release every month basically, I just added both of them back and I've never heard from them again. So sort of problem solved. Peace the Middle East? I don't know.Corey: It's kind of wild just to see how often that sort of thing tends to happen. It's a, I don't necessarily understand why folks are so opposed to other people trying to help. I think they feel like there's this loss of control as things are slipping through their fingers, but it's a really unwelcoming approach. One of the things that got me deep into the open-source ecosystem surprisingly late in my development was when I started pitching in on the SaltStack project right after it was founded, where suddenly everything I threw their way was merged, and then Tom Hatch, the guy who founded the project, would immediately fix all the bugs and stuff I put in and then push something else immediately thereafter. But it was such a welcoming thing.Instead of nitpicking me to death in the pull request, it just got merged in and then silently fixed. And I thought that was a classy way to do it. Of course, it doesn't scale and of course, it causes other problems, but I envy the simplicity of those days and just the ethos behind that.Amir: That's something I've learned the last few years, I would say. Back in the NSIS day, I was not like that. I nitpicked. I nitpicked a lot. And I can guess why, but it just—you create a patch—in my mind, right, like you create a patch, you fix it, right?But these days I get, I've been on the other side as well, right? Like I created patches for open-source projects and I've seen them just wither away and die, and then five years later, someone's like, “Oh, can you fix this line to have one instead of two, and then I'll merge it.” I'm like, “I don't care anymore. It was five years ago. I don't work there anymore. I don't need it. If you want it, do it.”So, I get it these days. And these days, if someone creates a patch—just yesterday, someone created a patch to format cdk-github-runners in VS Code. And they did it just, like, a little bit wrong. So, I just fixed it for them and I approved it and pushed it. You know, it's much better. You don't need to bug people for most of it.Corey: You didn't yell at them for having the temerity to contribute?Amir: My voice is so raw because I've been yelling for five days at them, yeah.Corey: Exactly, exactly. I really want to thank you for taking the time to chat with me about how all this stuff came to be and your own path. If people want to learn more, where's the best place for them to find you?Amir: So, I really appreciate you having me and driving all this traffic to my projects. If people want to learn more, they can always go to cloudsnorkel.com; it has all the projects. github.com/cloudsnorkel has a few more. And then my private blog is kichik.com. So, K-I-C-H-I-K dot com. I don't post there as much as I should, but it has some interesting AWS projects from the past few years that I've done.Corey: And we will, of course, put links to all of that in the show notes. Thank you so much for taking the time. I really appreciate it.Amir: Thank you, Corey. It was really nice meeting you.Corey: Amir Szekely, owner of CloudSnorkel. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an insulting comment. Heck, put it on all of the podcast platforms with a step function state machine that you somehow can't quite figure out how the API works.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

Merriam-Webster's Word of the Day for November 6, 2023 is: terraform • TEH-ruh-form • verb To terraform something (such as a planet or moon) is to transform it so that it is suitable for supporting human life. // With Earth being the only Goldilocks planet within our reach, some argue that the survival of humanity depends on our ability to eventually terraform Mars. See the entry > Examples: “Simulation games now routinely engage with climate change, but usually from a place of wish fulfillment. Surviving Mars lets players use magical future technology to terraform the Red Planet into a new Eden, creating a backup home in case Earth is ravaged beyond redemption.” — Mark Hill, Wired, 6 Jan. 2022 Did you know? In the world of science fiction, life (uh) finds a way. Such is the goal of terraforming, a concept that has long served as a sci-fi staple. In fact, the word terraform can be traced to the genre's Golden Age: the first known use of terraform was by the science fiction writer Jack Williamson who, writing under the pseudonym Will Stewart, included the word in his 1942 story “Collision Orbit.” Terraform applies the Latin noun terra (“land, earth”) as a prefix to the English verb form (“to shape or develop”). (Terra is evidenced in many English words, including terrain, terrace, and terra-cotta.) You may ask, “what is the future of terraforming?” Suffice it to say, we have no earthly idea.

In this episode, I spoke with Anton Babenko, who is an AWS Community Hero and creator of the Serverless.TF framework. We spoke at length about serverless development with Terraform and the problems that Serverless.TF aims to solve. We also discussed the recent seismic split in the Terraform community, with HashiCorp's license change and the initiation of the OpenTofu movement. Anton didn't hold back on his feelings about the oversized reaction to HashiCorp's announcement, and I agree with him!Links from the episode:* Serverless.TF framework* Serverless.TF example for Lambda* Anton's various other projects* Hire Anton for consulting engagements* HashiCorp's license change announcement* OpenTofuYou can find Anton on X as @antonbabenko-----For more stories about real-world use of serverless technologies, please subscribe to the channel and follow me on X as @theburningmonk.And if you're hungry for more insights, best practices, and invaluable tips on building serverless apps, make sure to subscribe to our free newsletter and elevate your serverless game! https://theburningmonk.com/subscribeOpening theme song:Cheery Monday by Kevin MacLeodLink: https://incompetech.filmmusic.io/song/3495-cheery-mondayLicense: http://creativecommons.org/licenses/by/4.0

Cloud Posse holds LIVE "Office Hours" every Wednesday to answer questions on all things related to AWS, DevOps, Terraform, Kubernetes, CI/CD. Register at https://cloudposse.com/office-hoursSupport the show

In episode 165 of our SAP on Azure video podcast we talk about SAP HANA on Azure Large Instances will be retired by 30 June 2025 and the transition to Virtual Machines, Configuring Azure NetApp Files (ANF) Application Volume Group (AVG) for zonal SAP HANA deployment, Changes to the Azure reservation exchange policy, a look back at the ABAP Story, A Cool use of Open AI in Eclipse, Microsoft Business Applications Launch Event introduces wave of new AI-powered capabilities for Dynamics 365 and Power Platform and Azure OpenAI powered SAP Self-Services. Then we take a closer look at SDAF, or the SAP Deployment Automation Framework, which started as nice tool to simplify the deployment of SAP systems in a consistent and repeatable way. In the meantime SDAF has grown into a huge project, is leveraged by the Azure Center for SAP Solution and actually by lots of partners and customers. Kimmo Forss and Hemanth Damecharla show us the latest features, like the Configuration Editor and the integration in Azure DevOps Find all the links mentioned here: https://www.saponazurepodcast.de/episode165 Reach out to us for any feedback / questions: * Robert Boban: https://www.linkedin.com/in/rboban/ * Goran Condric: https://www.linkedin.com/in/gorancondric/ * Holger Bruchelt: https://www.linkedin.com/in/holger-bruchelt/ #Microsoft #SAP #Azure #SAPonAzure #SDAF #Infrastructure ## Summary created by AI Key Topics: * News from the weekend: The team shared some news about Azure VMs, reserved instances, exchange policy, and SAP on Azure video podcast. * Introduction of Kimmo and Hemanth: Kimmo and Hemanth are part of the SAP on Azure development team based in Helsinki, Finland, working on the SAP Deployment Automation Framework. * Overview of SAP Deployment Automation Framework: The framework is an open source tool that helps customers deploy and configure SAP systems on Azure using Terraform and Ansible, with a modular and extensible design. * New features and demos of the framework: The team showed some new features and demos of the framework, such as the configuration editor, the application service, the private DNS, the application manifest, and the software download. * Questions and feedback: The team answered some questions and feedback from the audience, such as the support for different SAP versions and platforms, the open source contribution model, the validation and testing process, and the deployment time and complexity.

Today we embark on a fascinating journey into the world of AWS Lambda functions and how to make them accessible to the public. In a recent use case, involving the creation of a public Lambda function for AWS users, we asked ourselves some interesting questions. How can you securely, cost-effectively, and conveniently publish AWS resources, especially Lambda functions, for others to use? And... can we possibly make some money out of this? Join us as we explore various options and share our findings for making your AWS resources available to the world. We dive into the Serverless Application Repository (SAR), an AWS treasure trove for publishing resources. And SAR isn't the only way! We also discuss alternatives like CloudFormation templates, GitHub publishing, Terraform modules, and container images. We explore the pros and cons of these methods and debate the implications in terms of cost, security, and ease of use. Finally, we touch on the AWS Marketplace as a platform to monetize your AWS resources.

Cloud Posse holds LIVE "Office Hours" every Wednesday to answer questions on all things related to AWS, DevOps, Terraform, Kubernetes, CI/CD. Register at https://cloudposse.com/office-hoursSupport the show

Welcome to an adrenaline-pumping episode of The Mob Mentality Show! In this captivating installment, titled "Mastering the High Stakes Arena: A Story of Mobbing Under Pressure with Giovanni Asproni," we dive into the thrilling world of Mob Programming in high-pressure environments. Join us as we embark on an exploration of high stakes work, dissecting the very definition of what "high stakes" truly means. Giovanni Asproni takes us on a roller-coaster ride through his enthralling story of Mob Programming under the intense pressure of a high-stakes scenario.  Discover the challenges and triumphs faced during those critical first moments with a new team amidst a high-stakes project. Giovanni provides insights into the diverse skill sets and experience levels within and outside the mob, shedding light on whether the decision to mob was driven by the high stakes or not. Uncover the significance of bugs in high-stakes products, and the accessibility and security constraints in this specific situation. In the midst of high-stakes work, Giovanni emphasizes the unwavering commitment to quality, exemplified by full Test-Driven Development (TDD) and the art of story slicing. Delve into the world of unorthodox design for observability and the dynamics of decision-making in high-pressure mob settings. Explore the rapid shift to radical candor while the pressure mounts and learn strategies for managing stress within the ensemble. Giovanni shares his experiences of mobbing under pressure with unfamiliar team members, touching upon the role of a team/mob firewall/gatekeeper and the value of pairing during meetings. Gain insights into overcoming major challenges when mobbing versus working solo and the critical importance of the bus factor and resilience in high-stakes endeavors. Unearth the secrets behind maintaining high automation levels and Giovanni's epic 16-hour mobbing marathon learning Terraform. Discover the nuanced approach to TDD within the mob while having manual testing support outside of it. And yes, we even explore the surreal scenario of having a prime minister as a major stakeholder - you won't want to miss that story! Join us as we discuss their race against time to deliver and authentically demonstrate lean principles within the mob. Uncover the concept of "Productive Paranoia," and understand the impact of real vs. artificial crisis modes on work dynamics. Is it possible to channel the benefits of crisis-mode work into non-crisis projects without causing destruction? Giovanni shares his thoughts not only on this but also on coaching in a crisis and whether non-XP ways of working naturally lead to crises. Don't miss this high-energy, high-insight episode as we unravel the secrets of how a team mastered a high-stakes arena through Mob Programming. Tune in and immerse yourself in the world of ensemble teaming like never before!  Video and show notes: https://youtu.be/-tgjrd8sL2U 

Does ChatGPT stink for Test automation? How can AI help you with regression testing? And what is a real-world example of using AI with RPA, DevOps, and more? Find out in this episode of the Test Guild New Shows for the week of Oct 22. So, grab your favorite cup of coffee or tea, and let's do this. Time News Title Link 0:25  Applitoools FREE Account Offer https://applitools.info/joe  0:38 AI-POWERED REGRESSION TESTING https://testguild.me/katalon 2:12 Mobile Testing Experience Summit Register https://testguild.me/ckdtml 3:12 Language Models (LLMs) have enhanced my prototyping https://testguild.me/ct5yqg 4:30 ChatGPT Sucks for Test Automation https://testguild.me/h01r8l 6:07 UiPath launches Autopilot https://testguild.me/ufr8o0 7:14 Kubiya ChatGPT-like assistant for DevOps processes https://testguild.me/qpn2g3 8:17 Terraform testing https://testguild.me/2rr1ws  

Hosts Dan Ivovich, Owen Bickford, and Sundi Myint kick off the 11th season of the Elixir Wizards podcast. This season's theme is “Branching Out from Elixir,” which expands the conversation to compare notes with experts from other communities; they discuss their experiences with other languages like JavaScript, PHP, Python, Ruby, C#, Go, and Dart before and after learning Elixir. This season's conversations will illuminate how problems are solved in different languages vs. Elixir; upcoming episode topics teased include education, data processing, deployment strategies, and garbage collection; the hosts express excitement for conversations analyzing similarities and differences between communities. Topics Discussed in this Episode Season 11 branches out from Elixir to compare notes with other programming communities Sundi, Owen, and Dan introduce the season theme and their interest in exploring these conversations The hosts compare their experiences with PHP, JavaScript, Python, Ruby, C#, Go, Dart and Elixir The Wizards compare and contrast differences in their personal experience building similar things with different languages Dan dreams in Ruby and uses it for quick prototypes Comparing problem-solving approaches across languages will reframe perspectives Upcoming episodes explore data processing workflows, machine learning, and game development Pop Quiz: Who's that Pokémon... or language, or framework? Links Mentioned https://smartlogic.io/ https://codepen.io/ https://i.redd.it/0lg7979qtr511.jpg

Welcome to The Cloud Pod episode 231! This week Justin and Matthew are discussing updates to Terraform testing for code validation, some new tools from Docker, look into the now generally available AWS DataZone, and dig into the evolution of passkeys over at Google. Slide into the passenger seat and let's check out this week's cloud news.  Titles we almost went with this week:

Cloud Posse holds LIVE "Office Hours" every Wednesday to answer questions on all things related to AWS, DevOps, Terraform, Kubernetes, CI/CD. Register at https://cloudposse.com/office-hoursSupport the show

Kuba Martin (@cube2222_2) is Software Engineering Team Lead at Spacelift and Interim Tech Lead of OpenTofu, the open-source fork of Terraform. Terraform is a declarative infrastructure-as-code (IaC) tool that recently switched to a source-available license. Spacelift and other companies that heavily relied on Terraform came together to fork it into a community-driven project originally called OpenTF, which has now become OpenTofu and is governed by the Linux Foundation.  Contributor is looking for a community manager! If you want to know more, shoot us an email at eric@scalevp.com. Subscribe to Contributor on Substack for email notifications! In this episode we discuss: Two kinds of forks How OpenTofu handled the opportunity to rethink their licensing and copyright Finding hundreds of pledges to the OpenTF Manifesto The benefits of a technical steering committee Recreating the community registry Links: OpenTofu Spacelift Terraform Gruntwork Harness env0 Scalr

Adam Jacob is CEO of DevOps platform System Initiative and Co-Founder of infrastructure automation platform Chef. This is Adam's second time on the Open Source Startup Podcast, and this episode is packed with learnings. We discuss the distribution benefits of open source and why some products should be open source and others should not, challenges with the Open Core business model, HashiCorp's license change and the community's response to fork Terraform to create OpenTofu, and much more!

Should you upgrade to Terraform 1.6 already? How to avoid blind spots when monitoring Lambda functions? An unusual way to utilize the CDK.

Soren Martius is Co-Founder & CEO of Terramate, the infrastructure-as-code management platform that sits on top of Terraform. Their open source project, also called Terramate, has 3K GitHub stars and adds capabilities such as code generation, stacks, orchestration, change detection, and data sharing to Terraform. In this episode, we discuss building the Terramate community alongside the Terraform community, focusing 70% of the team's effort on Terramate cloud, how HashiCorp's license change impacts the open source community and Terraform builders, and much more!

Jeff Geerling, Owner of Midwestern Mac, joins Corey on Screaming in the Cloud to discuss the importance of storytelling, problem-solving, and community in the world of cloud. Jeff shares how and why he creates content that can appeal to anybody, rather than focusing solely on the technical qualifications of his audience, and how that strategy has paid off for him. Corey and Jeff also discuss the impact of leading with storytelling as opposed to features in product launches, and what's been going on in the Raspberry Pi space recently. Jeff also expresses the impact that community has on open-source companies, and reveals his take on the latest moves from Red Hat and Hashicorp. About JeffJeff is a father, author, developer, and maker. He is sometimes called "an inflammatory enigma".Links Referenced:Personal webpage: https://jeffgeerling.com/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. A bit off the beaten path of the usual cloud-focused content on this show, today I'm speaking with Jeff Geerling, YouTuber, author, content creator, enigma, and oh, so much more. Jeff, thanks for joining me.Jeff: Thanks for having me, Corey.Corey: So, it's hard to figure out where you start versus where you stop, but I do know that as I've been exploring a lot of building up my own home lab stuff, suddenly you are right at the top of every Google search that I wind up conducting. I was building my own Kubernete on top of a Turing Pi 2, and sure enough, your teardown was the first thing that I found that, to be direct, was well-documented, and made it understandable. And that's not the first time this year that that's happened to me. What do you do exactly?Jeff: I mean, I do everything. And I started off doing web design and then I figured that design is very, I don't know, once it started transitioning to everything being JavaScript, that was not my cup of tea. So, I got into back-end work, databases, and then I realized to make that stuff work well, you got to know the infrastructure. So, I got into that stuff. And then I realized, like, my home lab is a great place to experiment on this, so I got into Raspberry Pis, low-power computing efficiency, building your own home lab, all that kind of stuff.So, all along the way, with everything I do, I always, like, document everything like crazy. That's something my dad taught me. He's an engineer in radio. And he actually hired me for my first job, he had me write an IT operations manual for the Radio Group in St. Louis. And from that point forward, that's—I always start with documentation. So, I think that was probably what really triggered that whole series. It happens to me too; I search for something, I find my old articles or my own old projects on GitHub or blog posts because I just put everything out there.Corey: I was about to ask, years ago, I was advised by Scott Hanselman to—the third time I find myself explaining something, write a blog post about it because it's easier to refer people back to that thing than it is for me to try and reconstruct it on the fly, and I'll drop things here and there. And the trick is, of course, making sure it doesn't sound dismissive and like, “Oh, I wrote a thing. Go read.” Instead of having a conversation with people. But as a result, I'll be Googling how to do things from time to time and come up with my own content as a result.It's at least a half-step up from looking at forums and the rest, where I realized halfway through that I was the one asking the question. Like, “Oh, well, at least this is useful for someone.” And I, for better or worse, at least have a pattern of going back and answering how I solved a thing after I get there, just because otherwise, it's someone asked the question ten years ago and never returns, like, how did you solve it? What did you do? It's good to close that loop.Jeff: Yeah, and I think over 50% of what I do, I've done before. When you're setting up a Kubernetes cluster, there's certain parts of it that you're going to do every time. So, whatever's not automated or the tricky bits, I always document those things. Anything that is not in the readme, is not in the first few steps, because that will help me and will help others. I think that sometimes that's the best success I've found on YouTube is also just sharing an experience.And I think that's what separates some of the content that really drives growth on a YouTube channel or whatever, or for an organization doing it because you bring the experience, like, I'm a new person to this Home Assistant, for instance, which I use to automate things at my house. I had problems with it and I just shared those problems in my video, and that video has, you know, hundreds of thousands of views. Whereas these other people who know way more than I could ever know about Home Assistant, they're pulling in fewer views because they just get into a tutorial and don't have that perspective of a beginner or somebody that runs into an issue and how do you solve that issue.So, like I said, I mean, I just always share that stuff. Every time that I have an issue with anything technological, I put it on GitHub somewhere. And then eventually, if it's something that I can really formulate into an outline of what I did, I put a blog post up on my blog. I still, even though I write I don't know how many words per week that goes into my YouTube videos or into my books or anything, I still write two or three blog posts a week that are often pretty heavy into technical detail.Corey: One of the challenges I've always had is figuring out who exactly I'm storytelling for when I'm putting something out there. Because there's a plethora, at least in cloud, of beginner content of, here's how to think about cloud, here's what the service does, here's why you should use it et cetera, et cetera. And that's all well and good, but often the things that I'm focusing on presuppose a certain baseline level of knowledge that you should have going into this. If you're trying to figure out the best way to get some service configured, I probably shouldn't have to spend the first half of the article talking about what AWS is, as a for instance. And I think that inherently limits the size of the potential audience that would be interested in the content, but it's also the kind of stuff that I wish was out there.Jeff: Yeah. There's two sides to that, too. One is, you can make content that appeals to anybody, even if they have no clue what you're talking about, or you can make content that appeals to the narrow audience that knows the base level of understanding you need. So, a lot of times with—especially on my YouTube channel, I'll put things in that is just irrelevant to 99% of the population, but I get so many comments, like, “I have no clue what you said or what you're doing, but this looks really cool.” Like, “This is fun or interesting.” Just because, again, it's bringing that story into it.Because really, I think on a base level, a lot of programmers especially don't understand—and infrastructure engineers are off the deep end on this—they don't understand the interpersonal nature of what makes something good or not, what makes something relatable. And trying to bring that into technical documentation a lot of times is what differentiates a project. So, one of the products I love and use and recommend everywhere and have a book on—a best-selling book—is Ansible. And one of the things that brought me into it and has brought so many people is the documentation started—it's gotten a little bit more complex over the years—but it started out as, “Here's some problems. Here's how you solve them.”Here's, you know, things that we all run into, like how do you connect to 12 servers at the same time? How do you have groups of servers? Like, it showed you all these little examples. And then if you wanted to go deeper, there was more documentation linked out of that. But it was giving you real-world scenarios and doing it in a simple way. And it used some little easter eggs and fun things that made it more interesting, but I think that that's missing from a lot of technical discussion and a lot of technical documentation out there is that playfulness, that human side, the get from Point A to Point B and here's why and here's how, but here's a little interesting way to do it instead of just here's how it's done.Corey: In that same era, I was one of the very early developers behind SaltStack, and I think one of the reasons that Ansible won in the market was that when you started looking into SaltStack, it got wrapped around its own axle talking about how it uses ZeroMQ for a full mesh between all of the systems there, as long—sorry [unintelligible 00:07:39] mesh network that all routes—not really a mesh network at all—it talks through a single controller that then talks to all of its subordinate nodes. Great. That's awesome. How do I use this to install a web server, is the question that people had. And it was so in love with its own cleverness in some ways. Ansible was always much more approachable in that respect and I can't understate just how valuable that was for someone who just wants to get the problem solved.Jeff: Yeah. I also looked at something like NixOS. It's kind of like the arch of distributions of—Corey: You must be at least this smart to use it in some respects—Jeff: Yeah, it's—Corey: —has been the every documentation I've had with that.Jeff: [laugh]. There's, like, this level of pride in what it does, that doesn't get to ‘and it solves this problem.' You can get there, but you have to work through the barrier of, like, we're so much better, or—I don't know what—it's not that. Like, it's just it doesn't feel like, “You're new to this and here's how you can solve a problem today, right now.” It's more like, “We have this golden architecture and we want you to come up to it.” And it's like, well, but I'm not ready for that. I'm just this random developer trying to solve the problem.Corey: Right. Like, they should have someone hanging out in their IRC channel and just watch for a week of who comes in and what questions do they have when they're just getting started and address those. Oh, you want to wind up just building a Nix box EC2 for development? Great, here's how you do that, and here's how to think about your workflow as you go. Instead, I found that I had to piece it together from a bunch of different blog posts and the rest and each one supposed that I had different knowledge coming into it than the others. And I felt like I was getting tangled up very easily.Jeff: Yeah, and I think it's telling that a lot of people pick up new technology through blog posts and Substack and Medium and whatever [Tedium 00:09:19], all these different platforms because it's somebody that's solving a problem and relating that problem, and then you have the same problem. A lot of times in the documentation, they don't take that approach. They're more like, here's all our features and here's how to use each feature, but they don't take a problem-based approach. And again, I'm harping on Ansible here with how good the documentation was, but it took that approach is you have a bunch of servers, you want to manage them, you want to install stuff on them, and all the examples flowed from that. And then you could get deeper into the direct documentation of how things worked.As a polar opposite of that, in a community that I'm very much involved in still—well, not as much as I used to be—is Drupal. Their documentation was great for developers but not so great for beginners and that was always—it still is a difficulty in that community. And I think it's a difficulty in many, especially open-source communities where you're trying to build the community, get more people interested because that's where the great stuff comes from. It doesn't come from one corporation that controls it, it comes from the community of users who are passionate about it. And it's also tough because for something like Drupal, it gets more complex over time and the complexity kind of kills off the initial ability to think, like, wow, this is a great little thing and I can get into it and start using it.And a similar thing is happening with Ansible, I think. We were at when I got started, there were a couple hundred modules. Now there's, like, 4000 modules, or I don't know how many modules, and there's all these collections, and there's namespaces now, all these things that feel like Java overhead type things leaking into it. And that diminishes that ability for me to see, like, oh, this is my simple tool that solving these problems.Corey: I think that that is a lost art in the storytelling side of even cloud marketing, where they're so wrapped around how they do what they do that they forget, customers don't care. Customers care very much about their problem that they're trying to solve. If you have an answer for solving that problem, they're very interested. Otherwise, they do not care. That seems to be a missing gap.Jeff: I think, like, especially for AWS, Google, Azure cloud platforms, when they build their new services, sometimes you're, like, “And that's for who?” For some things, it's so specialized, like, Snowmobile from Amazon, like, there's only a couple customers on the planet in a given year that needs something like that. But it's a cool story, so it's great to put that into your presentation. But some other things, like, especially nowadays with AI, seems like everybody's throwing tons of AI stuff—spaghetti—at the wall, seeing what will stick and then that's how they're doing it. But that really muddies up everything.If you have a clear vision, like with Apple, they just had their presentation on the new iPhone and the new neural engine and stuff, they talk about, “We see your heart patterns and we tell you when your heart is having problems.” They don't talk about their AI features or anything. I think that leading with that story and saying, like, here's how we use this, here's how customers can build off of it, those stories are the ones that are impactful and make people remember, like, oh Apple is the company that saves people's lives by making watches that track their heart. People don't think that about Google, even though they might have the same feature. Google says we have all these 75 sensors in our thing and we have this great platform and Android and all that. But they don't lead with the story.And that's something where I think corporate Apple is better than some of the other organizations, no matter what the technology is. But I get that feeling a lot when I'm watching launches from Amazon and Google and all their big presentations. It seems like they're tech-heavy and they're driven by, like, “What could we do with this? What could you do with this new platform that we're building,” but not, “And this is what we did with this other platform,” kind of building up through that route.Corey: Something I've been meaning to ask someone who knows for a while, and you are very clearly one of those people, I spend a lot of time focusing on controlling cloud costs and I used to think that Managed NAT Gateways were very expensive. And then I saw the current going rates for Raspberries Pi. And that has been a whole new level of wild. I mean, you mentioned a few minutes ago that you use Home Assistant. I do too.But I was contrasting the price between a late model, Raspberry Pi 4—late model; it's three years old if this point of memory serves, maybe four—versus a used small form factor PC from HP, and the second was less expensive and far more capable. Yeah it drags a bit more power and it's a little bit larger on the shelf, but it was basically no contest. What has been going on in that space?Jeff: I think one of the big things is we're at a generational improvement with those small form-factor little, like, tiny-size almost [nook-sized 00:13:59] PCs that were used all over the place in corporate environments. I still—like every doctor's office you go to, every hospital, they have, like, a thousand of these things. So, every two or three or four years, however long it is on their contract, they just pop all those out the door and then you get an E-waste company that picks up a thousand of these boxes and they got to offload them. So, the nice thing is that it seems like a year or two ago, that really started accelerating to the point where the price was driven down below 100 bucks for a fully built-out little x86 Mini PC. Sure, it's, you know, like you said, a few generations old and it pulls a little bit more power, usually six to eight watts at least, versus a Raspberry Pi at two to three watts, but especially for those of us in the US, electricity is not that expensive so adding two or three watts to your budget for a home lab computer is not that bad.The other part of that is, for the past two-and-a-half years because of the global chip shortages and because of the decisions that Raspberry Pi made, there were so few Raspberry Pis available that their prices shot up through the roof if you wanted to get one in any timely fashion. So, that finally is clearing up, although I went to the Micro Center near me yesterday, and they said that they have not had stock of Raspberry Pi 4s for, like, two months now. So, they're coming, but they're not distributed evenly everywhere. And still, the best answer, especially if you're going to run a lot of things on it, is probably to buy one of those little mini PCs if you're starting out a home lab.Or there's some other content creators who build little Kubernetes clusters with multiple mini PCs. Three of those stack up pretty nicely and they're still super quiet. I think they're great for home labs. I have two of them over on my shelf that I'm using for testing and one of them is actually in my rack. And I have another one on my desk here that I'm trying to set up for a five gigabit home router since I finally got fiber internet after years with cable and I'm still stuck on my old gigabit router.Corey: Yeah, I wound up switching to a Protectli, I think is what it's called for—it's one of those things I've installed pfSense on. Which, I'm an old FreeBSD hand and I haven't kept up with it, but that's okay. It feels like going back in time ten years, in some respects—Jeff: [laugh].Corey: —so all right. And I have a few others here and there for various things that I want locally. But invariably, I've had the WiFi controller; I've migrated that off. That lives on an EC2 box in Ohio now. And I do wind up embracing cloud services when I don't want it to go down and be consistently available, but for small stuff locally, I mean, I have an antenna on the roof doing an ADS-B receiver dance that's plugged into a Pi Zero.I have some backlogged stuff on this, but they've gotten expensive as alternatives have dropped in price significantly. But what I'm finding as I'm getting more into 3D printing and a lot of hobbyist maker tools out there, everything is built with the Raspberry Pi in mind; it has the mindshare. And yeah, I can get something with similar specs that are equivalent, but then I've got to do a whole bunch of other stuff as soon as it gets into controlling hardware via GPIO pins or whatnot. And I have to think about it very differently.Jeff: Yeah, and that's the tough thing. And that's the reason why Raspberry Pis, even though they're three years old, even though they're hard to get, they still are fetching—on the used market—way more than the original MSRP. It's just crazy. But the reason for that is the Raspberry Pi organization. And there's two: there's the Raspberry Pi Foundation that's goals are to increase educational computing and accessibility for computers for kids and learning and all that, then there's the Raspberry Pi trading company that makes the Raspberry Pis.The Trading Company has engineers who sit there 24/7 working on the software, working on the kernel drivers, working on hardware bugs, listening to people on the forums and in GitHub and everywhere, and they're all English-speaking people there—they're over in the UK—and they manufacture their own boards. So, there's a lot of things on top of that, even though they're using some silicons of Broadcom chips that are a little bit locked down and not completely open-source like some other chips might be, they're a phone number you could call if you need the support or there's a forum that has activity that you can get help in and their software that's supported. And there's a newer Linux kernel and the kernel is updated all the time. So, all those advantages mean you get a little package that will work, it'll sip two watts of power, sitting 24/7. It's reliable hardware.There's so many people that use it that it's so well tested that almost any problem you could ever run into, someone else has and there's a blog post or a forum post talking about it. And even though the hardware is not super powerful—it's three years old—you can add on a Coral TPU and do face recognition and object recognition. And throw in Frigate for Home Assistant to get notifications on your phone when your mom walks up to the door. There's so many things you can do with them and they're so flexible that they're still so valuable. I think that they really knocked it out of the park with that model, the Raspberry Pi 4, and the compute module 4, which is still impossible to get. I have not been able to buy one for two years now. Luckily, I bought 12 two-and-a-half years ago [laugh] otherwise I would be running out for all my projects that I do.Corey: Yeah. I got two at the moment and two empty slots in the Turing Pi 2, which I'll care more about if I can actually get the thing up and booted. But it presupposes you have a Windows computer or otherwise, ehh, watch this space; more coming. Great. Like, do I build a virtual machine on top of something else? It leads down the path super quickly of places I thought I'd escaped from.Jeff: Yeah, you know, outside of the Pi realm, that's the state of the communities. It's a lot of, like, figuring out your own things. I did a project—I don't know if you've heard of Mr. Beast—but we did a project for him that involves a hundred single-board computers. We couldn't find Raspberry Pi's so we had to use a different single-board computer that was available.And so, I bought an older one thinking, oh, this is, like, three or four years old—it's older than the Pi 4—and there must be enough support now. But still, there's, like, little rough edges everywhere I went and we ended up making them work, but it took us probably an extra 30 to 40 hours of development work to get those things running the same way as a Raspberry Pi. And that's just the way of things. There's so much opportunity.If one of these Chinese manufacturers that makes most of these things, if one of them decided, you know what? We're going to throw tons of money into building support for these things, get some English-speaking members of these forums to build up the community, all that stuff, I think that they could have a shot at Raspberry Pi's giant portion of the market. But so far, I haven't really seen that happen. So far, they're spamming hardware. And it's like, the hardware is awesome. These chips are great if you know how to deal with them and how to get the software running and how to deal with Linux issues, but if you don't, then they're not great because you might not even get the thing to boot.Corey: I want to harken back to something you said a minute ago, where there's value in having a community around something, where you can see everyone else has already encountered a problem like this. I think that folks who weren't around for the rise of cloud have no real insight into how difficult it used to be just getting servers into racks and everything up, and okay, they're identical, and seven of them are working, but that eighth one isn't for some strange reason. And you spend four hours troubleshooting what turns out to be a bad cable or something not seated properly and it's awful. Cloud got away from a lot of that nonsense. But it's important—at least to me—to not be Captain Edgecase, where if you pick some new cloud provider and Google for how to set up a load balancer and no one's done it before you, that's not great. Whereas if I'm googling now in the AWS realm and no one has done, the thing I'm trying to do, that should be something of a cautionary flag of maybe this isn't how most people go about approaching production. Really think twice about this.Jeff: Yep. Yeah, we ran into that on a project I was working on was using Magento—which I don't know if anybody listening uses Magento, but it's not fun—and we ran into some things where it's like, “We're doing this, and it says that they do this on their official supported platform, but I don't know how they are because the code just doesn't exist here.” So, we ran into some weird edge cases on AWS with some massive infrastructure for the databases, and I ran into scaling issues. But even there, there were forum posts in AWS here and there that had little nuggets that helped us to figure out a way to get around it. And like you say, that is a massive advantage for AWS.And we ran into an issue with, we were one of the first customers trying out the new Lambda functions for RDS—or I don't remember exactly what it was called initially—but we ended up not using that. But we ran into some of these issues and figured out we were the first customer running into this weird scaling thing when we had a certain size of database trying to use it with these Lambda calls. And eventually, they got those things solved, but with AWS, they've seen so many things and some other cloud providers haven't seen these things. So, when you have certain types of applications that need to scale in certain ways, that is so valuable and the community of users, the ability to pull from that community when you need to hire somebody in an emergency, like, we need somebody to help us get this project done and we're having this issue, you can find somebody that is, like, okay, I know how to get you from Point A to Point B and get this project out the door. You can't do that on certain platforms.And open-source projects, too. We've always had that problem in Drupal. The amount of developers who are deep into Drupal to help with the hard problems is not vast, so the ones who can do that stuff, they're all hired off and paid a handsome sum. And if you have those kinds of problems you realize, I either going to need to pay a ton of money or we're just going to have to not do that thing that we wanted to do. And that's tough.Corey: What I've found, sort of across the board, has been that there's a lot of, I guess, open-source community ethos that has bled into a lot of this space and I wanted to make sure that we have time to talk about this because I was incensed a while back when Red Hat decided, “Oh, you know that whole ten-year commitment on CentOS? That project that we acquired and are now basically stabbing in the face?”—disclosure. I used to be part of the CentOS project years ago when I was on network staff for the Freenode IRC network—then it was, “Oh yeah, we're just going to basically undermine our commitments to you and now you can pay us if you want to get that support there.” And that really set me off. Was nice to see you were right there as well in almost lockstep with me, pointing out that this is terrible, just as far as breaking promises you've made to customers. Has your anger cooled any? Because mine hasn't.Jeff: It has not. My temper has cooled. My anger has not. I don't think that they get it. After all the backlash that they got after that, I don't think that the VP-level folks at Red Hat understand that this is already impacting them and will impact them much more in the future because people like me and you, people who help other people build infrastructure and people who recommend operating systems and people who recommend patterns and things, we're just going to drop off using CentOS because it doesn't exist. It does exist and some other people are saying, “Oh, it's actually better to use this new CentOS, you know, Stream. Stream is amazing.” It's not. It's not the same thing. It's different. And—Corey: I used to work at a bank. That was not an option. I mean, granted at the bank for the production systems it was always [REL 00:25:18], but being able to spin up a pre-production environment without having to pay license fees on every VM. Yeah.Jeff: Yeah. And not only that, they did this announcement and framed it a certain way, and the community immediately saw. You know, I think that they're just angry about something, and whether it was a NASA contract with Rocky Linux, or whether it was something Oracle did, who knows, but it seems petty in retrospect, especially in comparison to the amount of backlash that came out of it. And I really don't think that they understand the thing that they had with that Red Hat Enterprise Linux is not a massive growth opportunity for Red Hat. It's, in some ways, a dying product in terms of compared to using cloud stuff, it doesn't matter.You could use CoreOS, you could use NixOS, and you could use anything, it doesn't really matter. For people like you and me, we just want to deploy our software. And if it's containers, it really doesn't matter. It's just the people in government or in certain organizations that have these roles that you have to use whatever FIPS and all that kind of stuff. So, it's not like it's a hyper-growth opportunity for them.CentOS was, like, the only reason why all the software, especially on the open-source side, was compatible with Red Hat because we could use CentOS and it was easy and simple. They took that—well, they tried to take that away and everybody's like, “That's—what are you doing?” Like, I posted my blog post and I think that sparked off quite a bit of consternation, to the point where there was a lot of personal stuff going on. I basically said, “I'm not supporting Red Hat Enterprise Linux for any of my work anymore.” Like, “From this point forward, it's not supported.”I'll support OpenELA, I'll support Rocky Linux or Oracle Linux or whatever because I can get free versions that I don't have to sign into a portal and get a license and download the license and integrate it with my CI work. I'm an open-source developer. I'm not going to pay for stuff or use 16 free licenses. Or I was reached out to and they said, “We'll give you more licenses. We'll give you extra.” And it's like, that's not how this works. Like, I don't have to call Debian and Ubuntu and [laugh] I don't even have to call Oracle to get licenses. I can just download their software and run it.So, you know, I don't think they understood the fact that they had that. And the bigger problem for me was the two-layer approach to destroying all the trust that the community had. First was in, I think it was 2019 when they said—we're in the middle of CentOS 8's release cycle—they said, “We're dropping CentOS 8. It's going to be Stream now.” And everybody was up in arms.And then Rocky Linux and [unintelligible 00:27:52] climbed in and gave us what we wanted: basically, CentOS. So, we're all happy and we had a status quo, and Rocky Linux 9 and [unintelligible 00:28:00] Linux nine came out after Red Hat 9, and the world was a happy place. And then they just dumped this thing on us and it's like, two major release cycles in a row, they did it again. Like, I don't know what this guy's thinking, but in one of the interviews, one of the Red Hat representatives said, “Well, we wanted to do this early in Red Hat 9's release cycle because people haven't started migrating.” It's like, well, I already did all my automation upgrades for CI to get all my stuff working in Rocky Linux 9 which was compatible with Red Hat Enterprise Linux 9. Am I not one of the people that's important to you?Like, who's important to you? Is it only the people who pay you money or is it also the people that empower your operating system to be a premier Enterprise Linux operating system? So, I don't know. You can tell. My anger has not died down. The amount of temper that I have about it has definitely diminished because I realize I'm talking at a wall a lot of times, when I'm having conversations on Twitter, private conversations and email, things like that.Corey: People come to argue; they don't come to actually have a discussion.Jeff: Yeah. I think that they just, they don't see the community aspect of it. They just see the business aspect. And the business aspect, if they want to figure out ways that they can get more people to pay them for their software, then maybe they should provide more value and not just cut off value streams. It doesn't make sense to me from a long-term business perspective.From a short term, maybe there were some clients who said, “Oh, shoot. We need this thing stable. We're going to pay for some more licenses.” But the engineers that those places are going to start making plans of, like, how do we make this not happen again. And the way to not make that happen, again is to use, maybe Ubuntu or maybe [unintelligible 00:29:38] or something. Who knows? But it's not going to be increasing our spend with Red Hat.Corey: That's what I think a lot of companies are missing when it comes to community as well, where it's not just a place to go to get support for whatever it is you're doing and it's not a place [where 00:29:57] these companies view prospective customers. There's more to it than that. There has to be a social undercurrent on this. I look at the communities I spend time in and in some of them dating back long enough, I've made lifelong significant friendships out of those places, just through talking about our lives, in addition to whatever the community is built around. You have to make space for that, and companies don't seem to fully understand that.Jeff: Yeah, I think that there's this thing that a community has to provide value and monetizable value, but I don't think that you get open-source if you think that that's what it is. I think some people in corporate open-source think that corporate open-source is a value stream opportunity. It's a funnel, it's something that is going to bring you more customers—like you say—but they don't realize that it's a community. It's like a group of people. It's friends, it's people who want to make the world a better place, it's people who want to support your company by wearing your t-shirt to conferences, people want to put on your red fedora because it's cool. Like, it's all of that. And when you lose some of that, you lose what makes your product differentiated from all the other ones on the market.Corey: That's what gets missed. I think that there's a goodwill aspect of it. People who have used the technology and understand its pitfalls are likelier to adopt it. I mean, if you tell me to get a website up and running, I am going to build an architecture that resembles what I've run before on providers that I've run on before because I know what the failure modes look like; I know how to get things up and running. If I'm in a hurry, trying to get something out the door, I'm going to choose the devil that I know, on some level.Don't piss me off as a community member and incentivize me to change that estimation the next time I've got something to build. Well, that doesn't show up on this quarter's numbers. Well, we have so little visibility into how decisions get made many companies that you'll never know that you have a detractor who's still salty about something you did five years ago and that's the reason the bank decided not to because that person called in their political favors to torpedo that deal and have a sweetheart offer from your competitor, et cetera and so on and so forth. It's hard to calculate the actual cost of alienating goodwill. But—Jeff: Yeah.Corey: I wish companies had a longer memory for these things.Jeff: Yeah. I mean, and thinking about that, like, there was also the HashiCorp incident where they kind of torpedoed all developer goodwill with their Terraform and other—Terraform especially, but also other products. Like, I probably, through my book and through my blog posts and my GitHub examples have brought in a lot of people into the HashiCorp ecosystem through Vagrant use, and through Packer and things like that. At this point, because of the way that they treated the open-source community with the license change, a guy like me is not going to be enthusiastic about it anymore and I'm going to—I already had started looking at alternatives for Vagrant because it doesn't mesh with modern infrastructure practices for local development as much, but now it's like that enthusiasm is completely gone. Like I had that goodwill, like you said earlier, and now I don't have that goodwill and I'm not going to spread that, I'm not going to advocate for them, I'm not going to wear their t-shirt [laugh], you know when I go out and about because it just doesn't feel as clean and cool and awesome to me as it did a month ago.And I don't know what the deal is. It's partly the economy, money's drying up, things like that, but I don't understand how the people at the top can't see these things. Maybe it's just their organization isn't set up to show the benefits from the engineers underneath, who I know some of these engineers are, like, “Yeah, I'm sorry. This was dumb. I still work here because I get a paycheck, but you know, I can't say anything on social media, but thank you for saying what you did on Twitter.” Or X.Corey: Yeah. It's nice being independent where you don't really have to fear the, well if I say this thing online, people might get mad at me and stop doing business with me or fire me. It's well, yeah, I mean, I would have to say something pretty controversial to drive away every client and every sponsor I've got at this point. And I don't generally have that type of failure mode when I get it wrong. I really want to thank you for taking the time to talk with me. If people want to learn more, where's the best place for them to find you?Jeff: Old school, my personal website, jeffgeerling.com. I link to everything from there, I have an About page with a link to every profile I've ever had, so check that out. It links to my books, my YouTube, all that kind of stuff.Corey: There's something to be said for picking a place to contact you that will last the rest of your career as opposed to, back in the olden days, my first email address was the one that my ISP gave me 25 years ago. I don't use that one anymore.Jeff: Yep.Corey: And having to tell everyone I corresponded with that it was changing was a pain in the butt. We'll definitely put a link to that one in the [show notes 00:34:44]. Thank you so much for taking the time to speak with me. I appreciate it.Jeff: Yeah, thanks. Thanks so much for having me.Corey: Jeff Geerling, YouTuber, author, content creator, and oh so very much more. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment that we will, of course, read [in action 00:35:13], just as soon as your payment of compute modules for Raspberries Pi show up in a small unmarked bag.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

Last week in security news: When It Comes to Email Security, The Cloud You Pick Matters, Enable external pipeline deployments to AWS Cloud by using IAM Roles Anywhere, How AWS threat intelligence deters threat actors, and more!Links: When It Comes to Email Security, The Cloud You Pick Matters Enable external pipeline deployments to AWS Cloud by using IAM Roles Anywhere Get the full benefits of IMDSv2 and disable IMDSv1 across your AWS infrastructure  How AWS threat intelligence deters threat actors Overhauling AWS Account Access with Terraform

Kalen Arndt is a triple VCDX & Staff Solutions Engineer at HashiCorp! In this episode we went through best practices for Terraform module development and a LIVE DEMO! Resources: https://twitter.com/kalenarndt https://www.linkedin.com/in/kalenarndt/ https://registry.terraform.io/

This week, we discuss Cisco's acquisition of Splunk, AWS's investment in Anthropic, and VC Market Overview Presentations. Plus, we share some thoughts on Dungeons and Dragons, as well as standardized testing. Watch the YouTube Live Recording of Episode (https://www.youtube.com/watch?v=rNyPzGCpfT0) 434 (https://www.youtube.com/watch?v=rNyPzGCpfT0) Runner-up Titles So much about sheep Maybe Ten Middle mega-cap The data stays the same, only the story changes Ribbon Wall Rundown Splunk Cisco acquires cybersecurity company Splunk in cash deal worth $28 billion (https://www.cnbc.com/2023/09/21/cisco-acquiring-splunk-for-157-a-share-in-cash.html) Splunk Is Good For Cisco, But Cisco Needs To Convince Splunk Customers That Cisco Is Good For Them (https://www.forrester.com/blogs/splunk-is-good-for-cisco-but-cisco-needs-to-convince-splunk-customers-that-cisco-is-good-for-them/) AWS to invest up to $4B in Anthropic Google invested $300 million in AI firm founded by former OpenAI researchers (https://www.theverge.com/2023/2/3/23584540/google-anthropic-investment-300-million-openai-chatgpt-rival-claude) Amazon agreed to invest up to $4 billion into Anthropic (https://www.axios.com/newsletters/axios-pro-rata-e50e38f2-cb3f-4ec6-ab85-a758a8daf33e.html?chunk=1&utm_term=emshare#story1) I's $240B Question (https://www.sequoiacap.com/article/follow-the-gpus-perspective) Anti Portfolio (https://www.bvp.com/anti-portfolio) Relevant to your Interests Upbound Contributes Control Plane Provider Technology to Crossplane (https://blog.upbound.io/donate-upjet-provider-project-to-cncf) Your iPhone can now restore your Apple TV if the streaming box has problems (https://www.theverge.com/2023/9/19/23880671/apple-tv-4k-hd-iphone-restore-recovery) Elon Musk's Neuralink is recruiting patients for its first human trial (https://www.cnbc.com/2023/09/20/elon-musks-neuralink-is-recruiting-patients-for-its-first-human-trial.html) Roblox acquires voice moderation startup Speechly | TechCrunch (https://techcrunch.com/2023/09/20/roblox-acquires-voice-moderation-startup-speechly/?guccounter=1) Harness launches Gitness, an open source GitHub competitor | TechCrunch (https://techcrunch.com/2023/09/21/oh-gitness-harness-launches-gitness-an-open-source-github-competitor/?guccounter=1) Broadcom-VMware Deal Inches Closer In China: Report | CRN (https://www.crn.com/news/channel-news/broadcom-vmware-deal-inches-closer-in-china-report) 1Password rolls out public passkey support to its mobile apps and web extensions (https://www.theverge.com/2023/9/20/23880714/1password-mobile-passkey-support-web-browser-extension-release-date) Intel Unveils Industry-Leading Glass Substrates (https://www.intel.com/content/www/us/en/newsroom/news/intel-unveils-industry-leading-glass-substrates.html) Amazon's Prime Video will show ads unless you pay $3 more per month (https://www.engadget.com/amazons-prime-video-will-show-ads-unless-you-pay-3-more-per-month-111709384.html) Salesforce to acquire Airkit.ai, a low-code platform for building AI customer service agents | TechCrunch (https://techcrunch.com/2023/09/21/salesforce-airkit/) Spreadsheets are the long tail of datasets that don't have their own SaaS tool yet (https://x.com/davidsacks/status/1078755080478715904?s=46&t=zgzybiDdIcGuQ_7WuoOX0A) Microsoft Cloud hiring to "implement global small modular reactor and microreactor" strategy to power data centers (https://www.datacenterdynamics.com/en/news/microsoft-cloud-hiring-to-implement-global-small-modular-reactor-and-microreactor-strategy-to-power-data-centers/) Demand for Software Developers will STILL outweigh the supply. (https://x.com/DThompsonDev/status/1706015535861768404?s=20) No sacred masterpieces (https://basta.substack.com/p/no-sacred-masterpieces) Vista Equity Partners has quietly topped $100 billion in assets under management (https://www.axios.com/newsletters/axios-pro-rata-e50e38f2-cb3f-4ec6-ab85-a758a8daf33e.html?chunk=0&utm_term=emshare#story0) ll iPhone 15 Models Can Be Connected To An Ethernet Cable Through The USB-C Port Via Dongle To Enable Incredibly Fast Wired Speeds (https://wccftech.com/all-iphone-15-models-can-connect-to-ethernet-cable-with-usb-c-cable/) U.S. Accuses Amazon of Illegally Protecting Monopoly in Online Retail (https://www.nytimes.com/2023/09/26/technology/ftc-amazon.html) Google Podcasts to shut down in 2024 with listeners migrated to YouTube Music (https://techcrunch.com/2023/09/26/google-podcasts-to-shut-down-in-2024-with-listeners-migrated-to-youtube-music/) Tech layoffs are all but a thing of the past (https://techcrunch.com/2023/09/26/tech-layoffs-are-all-but-a-thing-of-the-past/) Terraform fork OpenTF gets renamed to OpenTofu (https://www.theregister.com/2023/09/20/terraform_fork_opentf_opentofu/) What's Up With Open Terraform? — Arrested DevOps (https://overcast.fm/+BvUXjLzWQ) Open source is at a crossroads with Steve O'Grady from RedMonk (Changelog Interviews #558) (https://changelog.com/podcast/558) Ads are coming to Amazon Prime Video, unless you pay more (https://www.theverge.com/2023/9/22/23885242/amazon-prime-tv-movies-streaming-ads-subscription-date) Airlines Are Just Banks Now (https://www.theatlantic.com/ideas/archive/2023/09/airlines-banks-mileage-programs/675374/) (https://www.datacenterdynamics.com/en/news/microsoft-cloud-hiring-to-implement-global-small-modular-reactor-and-microreactor-strategy-to-power-data-centers/)## Nonsense F-35 crash: Pilot called 911 after parachuting into backyard (https://www.bbc.com/news/world-us-canada-66890941) You can find love on Tinder for $500 a month, if you qualify for its elite tier (https://www.engadget.com/you-can-find-love-on-tinder-for-500-a-month-if-you-qualify-for-its-elite-tier-213159522.html) Apple Podcasts adds original programming from Apple Music, Apple News+ and other apps | TechCrunch (https://techcrunch.com/2023/09/26/apple-podcasts-adds-original-programming-from-apple-music-apple-news-and-other-apps/) Listener Feedback Slack's revamped UI feels like a step in the wrong direction (https://www.androidpolice.com/slack-revamped-ui-wrong-direction/) Brett's Slack Tip: Cmd/Ctrl+Shift+S when the new Slack design lands on you. It'll give you your Slack community sidebar back. Conferences Oct 3rd Enterprise DevOps Techcon (https://enterprisedevopstechcon.nl/?utm_source=cote&utm_campaign=devrel&utm_medium=newsletter&utm_content=newsletterUpcoming), Utrecht. October 6, 2023, KCD Texas 2023 (https://community.cncf.io/events/details/cncf-kcd-texas-presents-kcd-texas-2023/), CFP Closes: August 30, 2023 October 5 - 6, 2023, Devopsdays Indianapolis 2023 (https://devopsdays.org/events/2023-indianapolis/welcome/) Oct 9th Spring Tour Amsterdam (https://connect.tanzu.vmware.com/EMEA_P7_DG_FE_Q324_Event_S1TourAmsterdam_TanzuLP-AltS1TBanner.html?utm_source=cote&utm_campaign=devrel&utm_medium=newsletter&utm_content=newsletterUpcoming) Oct 10th, 17th, 24th talk series: Building a Path to Production: A Guide for Managers and Leaders in Platform Engineering (https://series.brighttalk.com/series/6011/?utm_source=cote&utm_campaign=devrel&utm_medium=newsletter&utm_content=newsletterUpcoming) November 6-9, 2023, KubeCon NA (https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/), SDT's a sponsor, Matt's there November 6-9, 2023 VMware Explore Barcelona (https://www.vmware.com/explore/eu.html), Coté's attending Jan 29, 2024 to Feb 1, 2024 That Conference Texas (https://that.us/events/tx/2024/schedule/) If you want your conference mentioned, let's talk media sponsorships. SDT news & hype Join us in Slack (http://www.softwaredefinedtalk.com/slack). Get a SDT Sticker! Send your postal address to stickers@softwaredefinedtalk.com (mailto:stickers@softwaredefinedtalk.com) and we will send you free laptop stickers! Follow us: Twitch (https://www.twitch.tv/sdtpodcast), Twitter (https://twitter.com/softwaredeftalk), Instagram (https://www.instagram.com/softwaredefinedtalk/), Mastodon (https://hachyderm.io/@softwaredefinedtalk), BlueSky (https://bsky.app/profile/softwaredefinedtalk.com), LinkedIn (https://www.linkedin.com/company/software-defined-talk/), TikTok (https://www.tiktok.com/@softwaredefinedtalk), Threads (https://www.threads.net/@softwaredefinedtalk) and YouTube (https://www.youtube.com/channel/UCi3OJPV6h9tp-hbsGBLGsDQ/featured). Use the code SDT to get $20 off Coté's book, Digital WTF (https://leanpub.com/digitalwtf/c/sdt), so $5 total. Become a sponsor of Software Defined Talk (https://www.softwaredefinedtalk.com/ads)! Recommendations Brandon: iPhone Messages Stickers (https://9to5mac.com/2023/09/23/how-to-make-use-iphone-messages-stickers-ios-17/) Coté: Notes.app: iOS 17 Notes and Reminders Features (https://www.macrumors.com/guide/ios-17-notes-reminders/). Photo Credits Header (https://unsplash.com/photos/eubgK-4bzKA) Artwork (https://unsplash.com/photos/colorful-software-or-web-code-on-a-computer-monitor-Skf7HxARcoc)