Podcast appearances and mentions of aurora serverless

Are relational databases cool again? In this episode of the AWS Podcast, host Simon Elisha sits down with Josh Hart, Principal Solutions Architect at AWS, to explore how traditional databases are getting a modern makeover. They dive into three game-changing innovations from AWS that are solving age-old database scaling headaches: Aurora Serverless v2 for seamless vertical scaling, Aurora Limitless Database for hassle-free sharding, and the RDS Data API for simplified connection management. Whether you're wrestling with dev/test environments or running production workloads at scale, this episode unpacks practical solutions that could save you time, money, and operational headaches. * Check out more examples of building modern data architectures on AWS in the AWS Data for SaaS patterns repository on GitHub (https://github.com/aws-samples/data-for-saas-patterns ) * Dive deeper into scaling relational databases with this 2 part blog series on the AWS database blog (https://aws.amazon.com/blogs/database/scale-your-relational-database-for-saas-part-1-common-scaling-patterns/ ) * Head over to YouTube to find more on-demand content for scaling databases (https://www.youtube.com/playlist?list=PLoqD0z_296PbKATwUcaGowmOJwlE_2ysP )

Today we take a deep dive into the world of PostgreSQL extensions, featuring our conversation with Heikki Linnakangas, co-founder of Neon, an expert Postgres hacker, and contributor to pgvector. Heikki shares the fascinating story of how Neon was born in 2021, driven by the vision of creating an open-source platform akin to Aurora Serverless. He delves into the early challenges and triumphs of building a fully remote startup during pandemic times, including the excitement of growing from a handful of employees to a bustling company with 600,000 databases under management.In this episode, we explore:From a startup idea to managing over 600K databasesBuilding a remote-first culture and the power of annual off-site meetingsTime travel… how Neon's unique storage system enables quick point-in-time recoveryUpcoming Postgres features like asynchronous IO and multi-threadingHow understanding new algorithms can help database managementLinks mentioned:NeonHeikki Linnakangas on GitHubHeikki Linnakangas on LinkedIn

AWS Morning Brief for the week of January 16, 2024 with Corey Quinn. Links:AWS Accounts discontinues the use of security challenge questionsAWS CodeBuild now supports a X-Large Linux compute typeIntroducing Open Job Description, an open specification for portable render jobs Reducing Inference Times by 87% for Darwinbox's Talent Search Engine Using AWS InferentiaAmazon ECS supports a native integration with Amazon EBS volumes for data-intensive workloadsThis is sad: AWS to Shut down Aurora Serverless v1. 

Thu, 11 Jan 2024 19:30:00 +0000 https://podcast.cloudonaut.io/84-aurora-serverless-is-dead-long-live-aurora-serverless 4fde3015f18cb6f7b6fc446b320a39ba AWS announced the end of life for Aurora Serverless v1, Andreas and Michael discuss the consequences for their workloads. Andreas and Michael Wittig are building on AWS since 2009. Follow their journey of developing products like bucketAV, marbot, and HyperEnv and learn from practice. Topics AWS product launches in 2023 AWS CloudShell supports Docker AWS Marketplace reduces fees Auto-scaling hooks and ELB connection draining Aurora Serverless v1 EOL Keep Terraform providers up to date! OpenTofu generally available NAT instance AMI out of maintenance EC2 Instance Connect Endpoints not HA? Links AWS Product Launch Count By Year by Sumiya AWS CloudShell now supports Docker in 13 Regions AWS announced reduced marketplace fees during the Partner Keynote Connect to your instances without requiring a public IPv4 address using EC2 Instance Connect Endpoint OpenTofu is going GA Subscribe Make sure you are not missing upcoming shows … Podcast feed YouTube channel Newsletter Projects bucketAV — Antivirus protection for Amazon S3 marbot — AWS Monitoring made simple! HyperEnv for GitHub Actions — Deploy self-hosted GitHub runners on AWS with ease! attachmentAV — Antivirus for Atlassian Jira and Confluence Contact and Feedback hello@cloudonaut.io Mastodon (Andreas) Mastodon (Michael) LinkedIn (Andreas) LinkedIn (Michael) 84 full AWS announced the end of life for Aurora Serverless v1, Andreas and Michael discuss the consequences for their workloads. no Andreas Wittig and Michael Wittig focusing on AWS Cloud

Adam's wondering why people hate Aurora Serverless? What's happening with AWS rumors and Preinvent / Reinvent? Dax thinks Pulumi is the best infrastructure as code tool. And Adam wonders if he can stop being a jerk on Twitter.Want to carry on the conversation? Join us in Discord.Links:Features • GitHub ActionsUsing Aurora Serverless v2 - Amazon AuroraLet's EncryptLow-Latency Content Delivery Network (CDN) - Amazon CloudFront - Amazon Web ServicesRedwoodJS: The App Framework for Startups | RedwoodJS.comAdonisJS - A fully featured web framework for Node.jsPulumi - Universal Infrastructure as Code | PulumiTerraform by HashiCorpOpen Source Development Framework - AWS Cloud Development Kit - AWSTopics discussed: (00:00) - What's in a name? (00:30) - Dax makes his own LUTS and Adam's still cold (03:26) - Why do people hate Aurora Serverless V2 (10:31) - Preinvent, Reinvent, and AWS Rumors (12:26) - Whats up with Open Search Serverless? (29:18) - Lowering the bar for developer experience (39:43) - SST update tease (41:07) - Issues with Amazon CDK (42:39) - Why is Pulumi is the best infastructure as code tool? (51:00) - Do you lose the ability to roll back safely? (52:37) - Being a jerk on Twitter (59:02) - Hitting a milestone and stopping (01:00:51) - Is AI really the next big thing?

Andreas Wittig, Co-Author of Amazon Web Services in Action and Co-Founder of marbot, joins Corey on Screaming in the Cloud to discuss ways to keep a book up to date in an ever-changing world, the advantages of working with a publisher, and how he began the journey of writing a book in the first place. Andreas also recalls how much he learned working on the third edition of Amazon Web Services in Action and how teaching can be an excellent tool for learning. Since writing the first edition, Adreas's business has shifted from a consulting business to a B2B product business, so he and Corey also discuss how that change came about and the pros and cons of each business model. About AndreasAndreas is the Co-Author of Amazon Web Services in Action and Co-Founder of marbot - AWS Monitoring made simple! He is also known on the internet as cloudonaut through the popular blog, podcast, and youtube channel he created with his brother Michael. Links Referenced: Amazon Web Services in Action: https://www.amazon.com/Amazon-Services-Action-Andreas-Wittig/dp/1617295116 Rapid Docker on AWS: https://cloudonaut.io/rapid-docker-on-aws/ bucket/av: https://bucketav.com/ marbot: https://marbot.io/ cloudonaut.io: https://cloudonaut.io TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. It's been a few years since I caught up with Andreas Wittig, who is also known in the internet as cloudonaut, and much has happened since then. Andreas, how are you?Andreas: Hey, absolutely. Thank you very much. I'm happy to be here in the show. I'm doing fine.Corey: So, one thing that I have always held you in some high regard for is that you have done what I have never had the attention span to do: you wrote a book. And you published it a while back through Manning, it was called Amazon Web Services in Action. That is ‘in action' two words, not Amazon Web Services Inaction of doing absolutely nothing about it, which is what a lot of companies in the space seem to do instead.Andreas: [laugh]. Yeah, absolutely. So. And it was not only me. I've written the book together with my brother because back in 2015, Manning, for some reason, wrote in and asked us if we would be interested in writing the book.And we had just founded our own consulting company back then and we had—we didn't have too many clients at the very beginning, so we had a little extra time free. And then we decided, okay, let's do the book. And let's write a book about Amazon Web Services, basically, a deep introduction into all things AWS. So, this was 2015, and it was indeed a lot of work, much more [laugh] than we expected. So, first of all, the hard part is, what do you want to have in the book? So, what's the TOC? What is important and must be in?And then you start writing and have examples and everything. So, it was really an interesting journey. And doing it together with a publisher like Manning was also really interesting because we learned a lot about writing. You have kind of a coach, an editor that helps you through that process. So, this was really a hard and fun experience.Corey: There's a lot of people that have said very good things about writing the book through a traditional publisher. And they also say that one of the challenges is it's a blessing and a curse, where you basically have someone standing over your shoulder saying, “Is it done yet? Is it done yet? Is it done yet?” The consensus that seems to have emerged from people who have written books is, “That was great, please don't ever ask me to do it again.”And my operating theory is that no one wants to write a book. They want to have written a book. Which feels like two very different things most of the time. But the reason you're back on now is that you have gone the way of the terrible college professor, where you're going to update the book, and therefore you get to do a whole new run of textbooks and make everyone buy it and kill the used market, et cetera. And you've done that twice now because you have just recently released the third edition. So, I have to ask, how different is version one from version two and from version three? Although my apologies; we call them ‘editions' in the publishing world.Andreas: [laugh]. Yeah, yeah. So, of course, as you can imagine, things change a lot in AWS world. So, of course, you have to constantly update things. So, I remember from first to second edition, we switched from CloudFormation in JSON to YAML. And now to the third edition, we added two new chapters. This was also important to us, so to keep also the scope of the book in shape.So, we have in the third edition, two new chapters. One is about automating deployments, recovering code deploy, [unintelligible 00:03:59], CloudFormation rolling updates in there. And then there was one important topic missing at all in the book, which was containers. And we finally decided to add that in, and we have now container chapter, starting with App Runner, which I find quite an interesting service to observe right now, and then our bread and butter service: ECS and Fargate. So, that's basically the two new chapters. And of course, then reworking all the other chapters is also a lot of work. And so, many things change over time. Cannot imagine [laugh].Corey: When was the first edition released? Because I believe the second one was released in 2018, which means you've been at this for a while.Andreas: Yeah. So, the first was 2015, the second 2018, three years later, and then we had five years, so now this third edition was released at the beginning of this year, 2023.Corey: Eh, I think you're right on schedule. Just March of 2020 lasted three years. That's fine.Andreas: Yeah [laugh].Corey: So, I have to ask, one thing that I've always appreciated about AWS is, it feels like with remarkably few exceptions, I can take a blog post written on how to do something with AWS from 2008 and now in 2023, I can go through every step along with that blog post. And yeah, I might have trouble getting some of the versions and services and APIs up and running, but the same steps will absolutely work. There are very few times where a previously working API gets deprecated and stops working. Is this the best way to proceed? Absolutely not.But you can still spin up the m1.medium instance sizes, or whatever it was, or [unintelligible 00:05:39] on small or whatever the original only size that you could get was. It's just there are orders of magnitude and efficiency gains you can do by—you can go through by using more modern approaches. So, I have to ask, was there anything in the book as you revised it—two times now—that needed to come out because it was now no longer working?Andreas: So, related to the APIs that's—they are really very stable, you're right about that. So, the problem is, our first few chapters where we have screenshots of how you go through the management—Corey: Oh no.Andreas: —console [laugh]. And you can probably, you can redo them every three months, probably, because the button moves or a step is included or something like that. So, the later chapters in the book, where we focus a lot on the CLI or CloudFormation and stuff like—or SDKs, they are pretty stable. But the first few [ones 00:06:29] are a nightmare to update all those screenshots. And then sometimes, so I was going through the book, and then I noticed, oh, there's a part of this chapter that I can completely remove nowadays.So, I can give you an example. So, I was going through the chapter about Simple Storage Service S3, and I—there was a whole section in the chapter about read-after-write consistency. Because back then, it was important that you knew that after updating an object or reading an object before it was created the first time, you could get outdated versions for a little while, so this was eventually consistent. But nowadays, AWS has changed that and basically now, S3 has this strong read-after-write consistency. So, I basically could remove that whole part in the chapter which was quite complicated to explain to the reader, right, so I [laugh] put a lot of effort into that.Corey: You think that was confusing? I look at the sea of systems I had to oversee at one company, specifically to get around that problem. It's like, well, we can now take this entire application and yeet it into the ocean because it was effectively a borderline service to that just want to ens—making consistency guarantees. It's not a common use case, but it is one that occurs often enough to be a problem. And of course, when you need it, you really need it. That was a nice under-the-hood change that was just one day, surprise, it works that way. But I'm sure it was years of people are working behind the scenes, solving for impossible problems to get there, and cetera, et cetera.Andreas: Yeah, yeah. But that's really cool is to remove parts of the book that are now less complicated. This is really cool. So, a few other examples. So, things change a lot. So, for example, EFS, so we have EFS, Elastic File System, in the book as well. So, now we have new throughput modes, different limits. So, there's really a lot going on and you have to carefully go through all the—Corey: Oh, when EFS launched, it was terrible. Now, it's great just because it's gotten so much more effective and efficient as a service. It's… AWS releases things before they're kind of ready, it feels like sometimes, and then they improve with time. I know there have been feature deprecations. For example, for some reason, they are no longer allowing us to share out a bucket via BitTorrent, which, you know, in 2006 when it came out, seemed like a decent idea to save on bandwidth. But here in 2023, no one cares about it.But I'm also keeping a running list of full-on AWS services that have been deprecated or have the deprecations announced. Are any of those in the book in any of its editions? And if and when there's a fourth edition, will some of those services have to come out?Andreas: [laugh]. Let's see. So, right after the book was published—because the problem with books is they get printed, right; that's the issue—but the target of the book, AWS, changes. So, a few weeks after the printed book was out, we found out that we have an issue in our one of our examples because now S3 buckets, when you create them, they have locked public access enabled by default. And this was not the case before. And one of our example relies on that it can create object access control lists, and this is not working now anymore. [laugh].So yeah, there are things changing. And we have, the cool thing about Manning is they have that what they call a live book, so you can read it online and you can have notes from other readers and us as the authors along the text, and there we can basically point you in the right direction and explain what happened here. So, this is how we try to keep the book updated. Of course, the printed one stays the same, but the ebook can change over time a little bit.Corey: Yes, ebooks are… at least keeping them updated is a lot easier, I would imagine. It feels like that—speaking of continuous builds and automatic CI/CD approaches—yeah, well, we could build a book just by updating some text in a Git repo or its equivalent, and pressing go, but it turns out that doing a whole new print run takes a little bit more work.Andreas: Yeah. Because you mentioned the experience of writing a book with a publisher and doing it on your own with self-publishing, so we did both in the past. We have Amazon Web Services in Action with Manning and we did another book, Rapid Docker on AWS in self-publishing. And what we found out is, there's really a lot of effort that goes into typesetting and layouting a book, making sure it looks consistent.And of course, you can just transform some markdown into a epub and PDF versions, but if a publisher is doing that, the results are definitely different. So, that was, besides the other help that we got from the publisher, very helpful. So, we enjoyed that as well.Corey: What is the current state of the art—since I don't know the answer to this one—around updating ebook versions? If I wind up buying an ebook on Kindle, for example, will they automatically push errata down automatically through their system, or do they reserve that for just, you know, unpublishing books that they realized shouldn't be on the Marketplace after people have purchased them?Andreas: [laugh]. So—Corey: To be fair, that only happened once, but I'm still giving them grief for it a decade and change later. But it was 1984. Of all the books to do that, too. I digress.Andreas: So, I'm not a hundred percent sure how it works with the Kindle. I know that Manning pushes out new versions by just emailing all the customers who bought the book and sending them a new version. Yeah.Corey: Yeah. It does feel, on some level, like there needs to be at least a certain degree of substantive change before they're going to start doing that. It's like well, good news. There was a typo on page 47 that we're going to go ahead and fix now. Two letters were transposed in a word. Now, that might theoretically be incredibly important if it's part of a code example, which yes, send that out, but generally, A, their editing is on point, so I didn't imagine that would sneak through, and 2, no one cares about a typo release and wants to get spammed over it?Andreas: Definitely, yeah. Every time there's a reprint of the book, you have the chance to make small modifications, to add something or remove something. That's also a way to keep it in shape a little bit.Corey: I have to ask, since most people talk about AWS services to a certain point of view, what is your take on databases? Are you sticking to the actual database services or are you engaged in my personal hobby of misusing everything as a database by holding it wrong?Andreas: [laugh]. So, my favorite database for starting out is DynamoDB. So, I really like working with DynamoDB and I like the limitations and the thing that you have to put some thoughts into how to structure your data set in before. But we also use a lot of Aurora, which really find an interesting technology. Unfortunately, Aurora Serverless, it's not becoming a product that I want to use. So, version one is now outdated, version two is much too expensive and restricted. So—Corey: I don't even know that it's outdated because I'm seeing version one still get feature updates to it. It feels like a divergent service. That is not what I would expect a version one versus version two to be. I'm with you on Dynamo, by the way. I started off using that and it is cheap is free for most workloads I throw at it. It's just a great service start to finish. The only downside is that if I need to move it somewhere else, then I have a problem.Andreas: That's true. Yeah, absolutely.Corey: I am curious, as far as you look across the sea of change—because you've been doing this for a while and when you write a book, there's nothing that I can imagine that would be better at teaching you the intricacies of something like AWS than writing a book on it. I got a small taste of this years ago when I shot my mouth off and committed to give a talk about Git. Well, time to learn Git. And teaching it to other people really solidifies a lot of the concepts yourself. Do you think that going through the process of writing this book has shaped how you perform as an engineer?Andreas: Absolutely. So, it's really interesting. So,I added the third edition and I worked on it mostly last year. And I didn't expect to learn a lot during that process actually, because I just—okay, I have to update all the examples, make sure everything work, go through the text, make sure everything is up to date. But I learned things, not only new things, but I relearned a lot of things that I wasn't aware of anymore. Or maybe I've never been; I don't know exactly [laugh].But it's always, if you go into the details and try to explain something to others, you learn a lot about that. So, teaching is a very good way to, first of all gather structure and a deep understanding of a topic and also dive into the details. Because when you write a book, every time you write a sentence, ask the question, is that really correct? Do I really know that or do I just assume that? So, I check the documentation, try to find out, is that really the case or is that something that came up myself?So, you'll learn a lot by doing that. And always come to the limits of the AWS documentation because sometimes stuff is just not documented and you need to figure out, what is really happening here? What's the real deal? And then this is basically the research part. So, I always find that interesting. And I learned a lot in during the third edition, while was only adding two new chapters and rewriting a lot of them. So, I didn't expect that.Corey: Do you find that there has been an interesting downstream effect from having written the book, that for better or worse, I've always no—I always notice myself responding to people who have written a book with more deference, more acknowledgment for the time and effort that it takes. And some books, let's be clear, are terrible, but I still find myself having that instinctive reaction because they saw something through to be published. Have you noticed it changing other aspects of your career over the past, oh, dear Lord, it would have been almost ten years now.Andreas: So, I think it helped us a lot with our consulting business, definitely. Because at the very beginning, so back in 2015, at least here in Europe and Germany, AWS was really new in the game. And being the one that has written a book about AWS was really helping… stuff. So, it really helped us a lot for our consulting work. I think now we are into that game of having to update the book [laugh] every few years, to make sure it stays up to date, but I think it really helped us for starting our consulting business.Corey: And you've had a consulting business for a while. And now you have effectively progressed to the next stage of consulting business lifecycle development, which is, it feels like you're becoming much more of a product company than you were in years past. Is that an accurate perception from the outside or am I misunderstanding something fundamental?Andreas: You know, absolutely, that's the case. So, from the very beginning, basically, when we founded our company, so eight years ago now, so we always had to go to do consulting work, but also do product work. And we had a rule of thumb that 20% of our time goes into product development. And we tried a lot of different things. So, we had just a few examples that failed completely.So, we had a Time [Series 00:17:49] as a Service offering at the very beginning of our journey, which failed completely. And now we have Amazon Timestream, which makes that totally—so now the market is maybe there for that. We tried a lot of things, tried content products, but also as we are coming from the software development world, we always try to build products. And over the years, we took what we learned from consulting, so we learned a lot about, of course, AWS, but also about the market, about the ecosystem. And we always try to bring that into the market and build products out of that.So nowadays, we really transitioned completely from consulting to a product company, as you said. So, we do not do any consulting anymore with one few exception with one of our [laugh] best or most important clients. But we are now a product company. And we only a two-person company. So, the idea was always how to scale a company without growing the team or hiring a lot of people, and a consulting business is definitely not a good way to do that, so yeah, this was why always invested into products.And now we have two products in the AWS Marketplace which works very well for us because it allows us to sell worldwide and really easily get a relationship up and running with our customers, and that pay through their AWS bill. So, that's really helping us a lot. Yeah.Corey: A few questions on that. At first it always seems to me that writing software or building a product is a lot like real estate in that you're doing a real estate development—to my understanding since I live in San Francisco and this is a [two exit 00:19:28] town; I still rent here—I found though, that you have to spend a lot of money and effort upfront and you don't get to start seeing revenue on that for years, which is why the VC model is so popular where you'll take $20 million, but then in return they want to see massive, outsized returns on that, which—it feels—push an awful lot of perfectly sustainable products into things that are just monstrous.Andreas: Hmm, yeah. Definitely.Corey: And to my understanding, you bootstrapped. You didn't take a bunch of outside money to do this, right?Andreas: No, no, we have completely bootstrapping and basically paying the bills with our consulting work. So yeah, I can give you one example. So, bucketAV is our solution to scan S3 buckets for malware, and basically, this started as an open-source project. So, this was just a side project we are working on. And we saw that there is some demand for that.So, people need ways to scan their objects—for example, user uploads—for malware, and we just tried to publish that in the AWS Marketplace to sell it through the Marketplace. And we don't really expect that this is a huge deal, and so we just did, I don't know, Michael spent a few days to make sure it's possible to publish that and get in shape. And over time, this really grew into an important, really substantial part of our business. And this doesn't happen overnight. So, this adds up, month by month. And you get feedback from customers, you improve the product based on that. And now this is one of the two main products that we sell in the Marketplace.Corey: I wanted to ask you about the Marketplace as well. Are you finding that that has been useful for you—obviously, as a procurement vehicle, it means no matter what country a customer is in, they can purchase it, it shows up on the AWS bill, and life goes on—but are you finding that it has been an effective way to find new customers?Andreas: Yes. So, I definitely would think so. It's always funny. So, we have completely inbound sales funnel. So, all customers find us through was searching the Marketplace or Google, probably. And so, what I didn't expect that it's possible to sell a B2B product that way. So, we don't know most of our customers. So, we know their name, we know the company name, but we don't know anyone there. We don't know the person who buys the product.This is, on the one side, a very interesting thing as a two-person company. You cannot build a huge sales process and I cannot invest too much time into the sales process or procurement process, so this really helps us a lot. The downside of it is a little bit that we don't have a close relationship with our customers and sometimes it's a little tricky for us to find important person to talk to, to get feedback and stuff. But on the other hand, yeah, it really helps us to sell to businesses all over the world. And we sell to very small business of course, but also to large enterprise customers. And they are fine with that process as well. And I think, even the large enterprises, they enjoy that it's so easy [laugh] to get a solution up and running and don't have to talk to any salespersons. So, enjoy it and I think our customers do as well.Corey: This is honestly the first time I've ever heard a verifiable account a vendor saying, “Yeah, we put this thing on the Marketplace, and people we've never talked to find us on the Marketplace and go ahead and buy.” That is not the common experience, let's put it that way. Now true, an awful lot of folks are selling enterprise software on this and someone—I forget who—many years ago had a great blog post on why no enterprise software costs $5,000. It either is going to cost $500 or it's going to cost 100 grand and up because the difference is, is at some point, you'd have a full-court press enterprise sales motion to go and sell the thing. And below a certain point, great, people are just going to be able to put it on their credit card and that's fine. But that's why you have this giant valley of there is very little stuff priced in that sweet spot.Andreas: Yeah. So, I think maybe it's important to mention that our products are relatively simple. So, they are just for a very small niche, a solution for a small problem. So, I think that helps a lot. So, we're not selling a full-blown cloud security solution; we only focus on that very small part: scanning S3 objects for malware.For example, on marbot,f the other product that we sell, which is monitoring of AWS accounts. Again, we focus on a very simple way to monitor AWS workloads. And so, I think that is probably why this is a successful way for us to find new customers because it's not a very complicated product where you have to explain a lot. So, that's probably the differentiator here.Corey: Having spent a fair bit of time doing battle with compliance goblins—which is, to be clear, I'm not describing people; I'm describing processes—in many cases, we had to do bucket scanning for antivirus, just to check a compliance box. From our position, there was remarkably little risk of a user-generated picture of a receipt that is input sanitized to make sure it is in fact a picture, landing in an S3 bucket and then somehow infecting one of the Linux servers through which it passed. So, we needed something that just checked the compliance box or we would not be getting the gold seal on our website today. And it was, more or less, a box-check as opposed to something that solved a legitimate problem. This was also a decade and change ago. Has that changed to a point now where there are legitimate threats and concerns around this, or is it still primarily just around make the auditor stop yelling at me, please?Andreas: Mmm. I think it's definitely to tick the checkbox, to be compliant with this, some regulation. On the other side, I think there are definitely use cases where it makes a lot of sense, especially when it comes to user-generated content of all kinds, especially if you're not only consuming it internally, but maybe also others can immediately start downloading that. So, that is where we see many of our customers are coming with that scenario that they want to make sure that the files that people upload and others can download are not infected. So, that is probably the most important use case.Corey: There's also, on some level, an increasing threat of ransomware. And for a long time, I was very down on the ideas of all these products that hit the market to defend S3 buckets against ransomware. Until one day, there was an AWS security blog post talking about how they found it. And yeah, we've we have seen this in the wild; it is causing problems for companies; here's what to do about it. Because it's one of those areas where I can't trust a vendor who's trying to sell me something to tell me that this problem exists.I mean, not to cast aspersions, but they're very interested, they're very incentivized to tell that story, whereas AWS is not necessarily incentivized to tell a story like that. So, that really brought it home for me that no, this is a real thing. So, I just want to be clear that my opinion on these things does in fact, evolve. It's not, “Well, I thought it was dumb back in 2012, so clearly it's still dumb now.” That is not my position, I want to be very clear on that.I do want to revisit for a moment, the idea of going from a consultancy that is a services business over to a product business because we've toyed with aspects of that here at The Duckbill Group a fair bit. We've not really found long-term retainer services engagements that add value that we are comfortable selling. And that means as a result that when you sell fixed duration engagements, it's always a sell, sell, sell, where's the next project coming from? Whereas with product businesses, it's oh, the grass is always greener on the other side. It's recurring revenue. Someone clicks, the revenue sticks around and never really goes away. That's the dream from where I sit on the services side of the fence, wistfully looking across and wondering what if. Now that you've made that transition, what sucks about product businesses that you might not have seen going into it?Andreas: [laugh]. Yeah, that a good question. So, on the one side, it was really also our dream to have a product business because it really changes the way we work. We can block large parts of our calendar to do deep-focus work, focus on things, find new solutions, and really try to make a solution that really fits to problem and uses all the AWS capabilities to do so. And on the other side, a product business involves, of course, selling the product, which is hard.And we are two software engineers, [laugh] and really making sure that we optimize our sales and there's search engine optimization, all that stuff, this is really hard for us because we don't know anything about that and we always have to find an expert, or we need to build a knowledge ourself, try things out, and so on. So, that whole part of selling the product, this is really a challenge for us. And then of course, product business evolves a lot of support work. So, we get support emails multiple times per hour, and we have to answer them and be as fast as possible with that. So, that is, of course, something that you do not have to do with consulting work.And not always that, the questions are many times really simple questions that pointed people in the right direction, find part of the documentation that answers the question. So, that is a constant stream of questions coming in that you have to answer. So, the inbox is always full [laugh]. So, that is maybe a small downside of a product business. But other than that, yeah, compared to a consulting business, it really gives us many flexibilities with planning our work day around the rest of our lives. That's really what we enjoy about a product company.Corey: I was very careful to pick an expensive problem that was only a business-hours problem. So, I don't wind up with a surprise, middle-of-the-night panic phone call. It's yeah, it turns out that AWS billing operate during business hours in the US Pacific Time. The end. And there are no emergencies here; there are simply curiosities that will, in the fullness of time take weeks to get resolved.Andreas: Mmm. Yeah.Corey: I spent too many years on call, in that sense. Everyone who's built a product company the first time always says the second time, the engineering? Meh, there are ways to solve that. Solving the distribution problem. That's the thing I want to focus on next.And I feel like I sort of went into this backwards in that I don't really have a product to sell people but I somehow built an audience. And to be honest, it's partly why. It's because I didn't know what I was going to be doing after 18 months and I knew that whatever it was going to be, I needed an audience to tell about it, so may as well start the work of building the audience now. So, I have to imagine if nothing else, your book has been a tremendous source of building a community. When I mentioned the word cloudonaut to people who have been learning AWS, more often than not, they know who you are.Andreas: Yeah.Corey: Although I admit they sometimes get you confused with your brother.Andreas: [laugh]. Yes, that's not too hard. Yeah, yeah, cloudonaut is definitely—this was always our, also a side project of we was just writing about things that we learned about AWS. Whenever we, I don't know, for example, looked into a new series, we wrote a blog post about that. Later, we did start a podcast and YouTube videos during the pandemic, of course, as everyone did. And so, I think this was always fun stuff to do. And we like sharing what we learn and getting into discussion with the community, so this is what we still do and enjoy as well, of course. Yeah.Corey: I really want to thank you for taking the time to catch up and see what you've been up to these last few years with a labor of love and the pivot to a product company. If people want to learn more, where's the best place for them to find you?Andreas: So definitely, the best place to find me is cloudonaut.io. So, this basically points you to all [laugh] what I do. Yeah, that's basically the one domain and URL that you need to know.Corey: Excellent. And we will put that in the show notes, of course. Thank you so much for taking the time to speak with me today. I really appreciate it.Andreas: Yeah, it was a pleasure to be back here. I'm big fan of podcasts and also of Screaming in the Cloud, of course, so it was a pleasure to be here again.Corey: [laugh]. You are always welcome. Andreas Wittig, co-author of Amazon Web Services in Action, now up to its third edition. And of course, the voice behind cloudonaut. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry, insulting comment that I will at one point be able to get to just as soon as I find something to deal with your sarcasm on the AWS Marketplace.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

目前云上服务的无服务化是一个大趋势，在应用程序方面有Lambda，在一些数据分析工具和数据库、数据仓库服务上，AWS也都推出了无服务版本，例如ETL服务Glue，数仓服务Redshift，Hadoop服务EMR，本节目针对无服务化的云数据库Aurora Serverless做了一些整理，分享给听众朋友们。希望加入群聊的可以加微信：52155266，注明「播客听众」

On this episode of Remote Ruby, we have an awesome guest joining us. Today, we have Ken Collins, who's a Principal Engineer and Cloud Architect at Custom Ink, an active member in the Ruby community for over fifteen years, a Microsoft open source contributor, PC Gamer, and an AWS Serverless Hero. We have so much to discuss today, as Ken fills us in on Lamby, Custom Ink, how Lambda evolved, a gem called Lambdakiq, and if you're looking for cost optimization, why Lambda is the best compute service out there. We'll also learn how CloudFormation can help developers, how CloudWatch Events is used, and we'll hear about the different database options Amazon has such as Aurora Serverless, DynamoDB, and RDS. If you've never used Lambda, it's a good time to try it out. Andrew realized he's in the perfect place to try it since he recently built a proxy one. Download this episode to learn much more! [00:01:52] Ken tells us about himself and his background[00:04:47] Custom Ink makes some great products, and we'll learn how Lamby came to be, the stuff they build, the cool tech behind it, and the services, such as AWS Lambda.[00:08:16] How did Lambda evolve?[00:09:17] Ken details what the OCI format is, and how Lambda works compared to deploying to a traditional server. We hear about Lambda releasing Function URLs, a free API gateway, and what it does.[00:12:16] We hear the whole process from end-to-end, starting from a web request, what happens, how it gets to Rails, Dynos are running, the database gets affected, and how those containers can be used for other things like an event driven architectures.[00:16:03] Chris asks Ken how Kubernetes and Lambda compare. Also, we hear how background jobs and cron jobs fit in, and a gem that Ken wrote called, Lambdakiq.[00:20:30] How does Ken manage connections being made and the events being sent to the right place? Also, Chris wonders if CloudFormation is something you should learn as one of the starting points or you should later for it to be more useful, and Ken tells us about the AWS Cloud Development Kit and what it does.[00:24:10] Amazon has many different database options and Ken explains that you can use any database you want, wherever you want.[00:25:39] Ken explains the differences between Aurora Serverless, DynamoDB, and RDS.   [00:30:23] We're going back to talking about Lambda now and Ken tells us about their website, a documentation website where they cover things, and a Quick Start Guide on how you can deploy a new Rails APP on Rails 3.2 to Lambda in 5 minutes.[00:33:02] Chris mentions how Taylor Otwell modified Laravel to run on Lambda, and Vapor is their tool for deploying to Lambda.[00:36:25] Are there any gotchas? Chris heard people were talking about Rails being slow to boot and issues with connecting to your Lambda to a VPC was slow. Ken tells us the VPC has been solved very well.[00:39:31] Ken and Chris chat about the hardest things are learning and change management, like setting up CI for the first time can be challenging, Heroku is amazing but has its limits, and using CloudWatch Logs which is a change for people. Also, Ken shares a hotspot with Lambda, and he tells us about Lambda Punch and New Relic. [00:42:47] Ken tells us to use CloudWatch Events for setting up Cronjobs that run on a schedule.[00:44:51] Chris wonders if there are concerns or ways you have to change things for assets, and Ken explains what they do with turning on the magic environment variable, but if you need something else, it goes into the CI/CD Pipeline creation.[00:48:30] Andrew is going to try Lambda now, and we hear Ken's thoughts on how different development is from production when you use Lambda. Find out why he loves Microsoft's Development Containers Specification, and Chris mentions DHH's MRSK project and what it's going to do.[00:56:06] Find out where to follow Ken, if you're interested in Custom Ink, check them out, and please try out Lambda because he could use some contributors to help write the guides.Panelists:Jason CharnesChris OliverAndrew MasonGuest:Ken CollinsSponsor:HoneybadgerLinks:Jason Charnes TwitterChris Oliver TwitterAndrew Mason TwitterKen Collins TwitterKen Collins GitHubKen Collins (Dev.to)Lamby-GitHubCustom InkCustom Ink ProductsLambdakiqAmazon Aurora ServerlessAmazon DynamoDBAmazon RDSLambyFull Stack Radio Podcast-Episode 120-Taylor Otwell-Serverless Laravel with VaporLambda PunchNew Relic-GitHubAmazon CloudWatch EventsDevelopment ContainersRemote Ruby Podcast-Episode 165: GitHub Codespaces & Docker with Benjamin WoodMRSK: Deploy Web apps anywhereRuby Radar TwitterRuby for All Podcast

As soluções serverless estão por toda parte, mas elas valem a pena? Vamos calcular os custos do novo RDS Aurora Serverless e analisar quais tipos de projetos se beneficiam mais dessa solução revolucionária de banco de dados relacional em nuvem da AWS.O curso AWS 2.0 está sendo preparado com muito cuidado e dedicação para atender às principais demandas de mercado para profissionais e empreendedores de tecnologia.Inscreva-se agora para aproveitar todas as vantagens do pré-lançamento:https://www.uminventorqualquer.com.br/curso-aws/Inscreva-se no Canal Wesley Milan para acompanhar os Reviews de serviços AWS:https://bit.ly/3LqiYwgInscreva-se no Canal Wesley Milan em Inglês e recomende a seus amigos gringos:https://bit.ly/3LqFjtAMe siga no Instagram: https://bit.ly/3tfzAj0LinkedIn: https://www.linkedin.com/in/wesleymilan/Podcast: https://bit.ly/3qa5JH1

O RDS Aurora Serverless V1 revolucionou o mercado, o Aurora Serverless V2 veio para fazer isso novamente. Vamos ver juntos os maiores benefícios deste Banco de Dados Relacional em Nuvem e como ele pode transformar seus conceitos de escalabilidade, redundância e resiliência. Vamos ver como criar aplicações e aplicativos de alta performance ficou muito mais fácil e mais barato.Video do Review do Aurora Serverless V2:https://youtu.be/zVBuKSJ9-M8O curso AWS 2.0 está sendo preparado com muito cuidado e dedicação para atender às principais demandas de mercado para profissionais e empreendedores de tecnologia.Inscreva-se agora para aproveitar todas as vantagens do pré-lançamento:https://www.uminventorqualquer.com.br/curso-aws/Inscreva-se no Canal Wesley Milan para acompanhar os Reviews de serviços AWS:https://bit.ly/3LqiYwgInscreva-se no Canal Wesley Milan em Inglês e recomende a seus amigos gringos:https://bit.ly/3LqFjtAMe siga no Instagram: https://bit.ly/3tfzAj0LinkedIn: https://www.linkedin.com/in/wesleymilan/Podcast: https://bit.ly/3qa5JH1

Cette semaine est assez calme, j'ai tout de même noté des changements dans le programme AWS Activate pour les startups, Aurora Serverless qui est supporté par CloudFormation, un nouveau type d'instance EC2 pour l'entrainement de gros (très gros) modèles AIML, et surtout, les S3 Object Lambda qui s'enrichissent des requêtes HEAD et LIST. Je vous explique à quoi ca peut servir à la fin de cet épisode.

Cette semaine est assez calme, j'ai tout de même noté des changements dans le programme AWS Activate pour les startups, Aurora Serverless qui est supporté par CloudFormation, un nouveau type d'instance EC2 pour l'entrainement de gros (très gros) modèles AIML, et surtout, les S3 Object Lambda qui s'enrichissent des requêtes HEAD et LIST. Je vous explique à quoi ca peut servir à la fin de cet épisode.

Two brothers discussing all things AWS every week. Hosted by Andreas and Michael Wittig presented by cloudonaut.

Renato joins Adam to discuss the differences between Aurora Serverless v1 and v2, how he's used AWS certifications to learn topics he might not dive into otherwise, and the benefits of speaking at conferences when you're introverted.

Want to give your ears a break and read this as an article? You're looking for this link.https://www.lastweekinaws.com/blog/the-aurora-serverless-road-not-taken/Never miss an episode Join the Last Week in AWS newsletter Subscribe wherever you get your podcasts Help the show Leave a review Share your feedback Subscribe wherever you get your podcasts What's Corey up to? Follow Corey on Twitter (@quinnypig) See our recent work at the Duckbill Group Apply to work with Corey and the Duckbill Group to help lower your AWS bill

I was excited when AWS announced Aurora Serverless at re:Invent 2017. Disappointment followed shortly after. Even after Aurora Serverless became a generally available service in August 2018, it was missing important features like multi-AZ deployments and read replication. Unfortunately, the innovative service never achieved a breakthrough. Therefore, I used Aurora Serverless in exceptional cases only. Four years later, AWS is making a fresh start with Aurora Serverless v2. Reason enough to take a closer look at the new service.

Cloud Posse holds public "Office Hours" every Wednesday at 11:30am PST to answer questions on all things related to DevOps, Terraform, Kubernetes, CICD. Basically, it's like an interactive "Lunch & Learn" session where we get together for about an hour and talk shop. These are totally free and just an opportunity to ask us (or our community of experts) any questions you may have. You can register here: https://cloudposse.com/office-hoursJoin the conversation: https://slack.cloudposse.com/Find out how we can help your company:https://cloudposse.com/quizhttps://cloudposse.com/accelerate/Learn more about Cloud Posse:https://cloudposse.comhttps://github.com/cloudpossehttps://sweetops.com/https://newsletter.cloudposse.comhttps://podcast.cloudposse.com/[00:00:00] Intro[00:01:29] Git.io shutting down 2022-04-29 (GitHub provides 4 days notice!!!)https://github.blog/changelog/2022-04-25-git-io-deprecation/[00:02:53] Cloud Posse build-harness: update links to cloudposse.tools/build-harnesshttps://github.com/cloudposse/build-harness/issues/314[00:04:34] Google donates the Istio service mesh to the CNCFhttps://techcrunch.com/2022/04/25/google-donates-the-istio-service-mesh-to-the-cloud-native-computing-foundation/[00:05:05] AWS's Log4j patches blew holes in its own securityhttps://www.theregister.com/AMP/2022/04/20/aws_log4j_patches/[00:05:42] Fairwinds Helmfile Alternative: declaratively manage multiple Helm chart releaseshttps://github.com/FairwindsOps/reckoner[00:06:48] [2018] Kubernetes Edge Computing at Chick-fil-Ahttps://medium.com/@cfatechblog/edge-computing-at-chick-fil-a-7d67242675e2[00:08:17] Finally, a terraform-registry-proxy for “airgapped” environmentshttps://github.com/jasonwbarnett/terraform-registry-proxy[00:22:00] Aurora Serverless v1 is GA[00:23:26] Use IAM to control access to a resource based on the account, OU or organization that contains the resourcehttps://aws.amazon.com/about-aws/whats-new/2022/04/iam-access-resource-organization/[00:24:36] Karpenter workload consolidation/defragmentationhttps://github.com/aws/karpenter/issues/1091[00:29:37] How have folks automated AWS IAM Access Key + Secret Key rotation policies [00:34:23] Opinions and thoughts on K8s ingress controllers for high volume deployments. [00:42:25] What advice do you have for how to communicate expectations when people decide to use something brand new that is still super beta/rough, are having problems, and are annoyed that things aren't working?[00:52:30] Are you doomed without a tool like Spacelift? [01:00:23] Outro #officehours,#cloudposse,#sweetops,#devops,#sre,#terraform,#kubernetes,#awsSupport the show (https://cloudposse.com/office-hours/)

Cos'è un database serverless e per quali casi d'uso ha senso considerarlo? Come funziona Amazon Aurora Serverless? E cosa cambia nella nuova versione dell'engine in developer preview? In questo episodio ospito Renato Losio, Principal Cloud Architect in Funambol e AWS Data Hero, per parlare della sua esperienza decennale con Amazon RDS, ma soprattutto dei vantaggi e delle principali differenze dei database serverless (e gestiti) disponibili su AWS. Link: Amazon Aurora Serverless. Video (ENG): A First Look at Aurora Serverless v2.

About Rebecca MarshburnRebecca's interested in the things that interest people—What's important to them? Why? And when did they first discover it to be so? She's also interested in sharing stories, elevating others' experiences, exploring the intersection of physical environments and human behavior, and crafting the perfect pun for every situation. Today, Rebecca is the Head of Content & Community at Common Room. Prior to Common Room, she led the AWS Serverless Heroes program, where she met the singular Jeremy Daly, and guided content and product experiences for fashion magazines, online blogs, AR/VR companies, education companies, and a little travel outfit called Airbnb.Twitter: @beccaodelayLinkedIn: Rebecca MarshburnCompany: www.commonroom.ioPersonal work (all proceeds go to the charity of the buyer's choice): www.letterstomyexlovers.comWatch this episode on YouTube: https://youtu.be/VVEtxgh6GKI This episode sponsored by CBT Nuggets and Lumigo.Transcript:Rebecca: What a day today is! It's not every day you turn 100 times old, and on this day we celebrate Serverless Chats 100th episode with the most special of guests. The gentleman whose voice you usually hear on this end of the microphone, doing the asking, but today he's going to be doing the telling, the one and only, Jeremy Daly, and me. I'm Rebecca Marshburn, and your guest host for Serverless Chats 100th episode, because it's quite difficult to interview yourself. Hey Jeremy!Jeremy: Hey Rebecca, thank you very much for doing this.Rebecca: Oh my gosh. I am super excited to be here, couldn't be more honored. I'll give your listeners, our listeners, today, the special day, a little bit of background about us. Jeremy and I met through the AWS Serverless Heroes program, where I used to be a coordinator for quite some time. We support each other in content, conferences, product requests, road mapping, community-building, and most importantly, I think we've supported each other in spirit, and now I'm the head of content and community at Common Room, and Jeremy's leading Serverless Cloud at Serverless, Inc., so it's even sweeter that we're back together to celebrate this Serverless Chats milestone with you all, the most important, important, important, important part of the podcast equation, the serverless community. So without further ado, let's begin.Jeremy: All right, hit me up with whatever questions you have. I'm here to answer anything.Rebecca: Jeremy, I'm going to ask you a few heavy hitters, so I hope you're ready.Jeremy: I'm ready to go.Rebecca: And the first one's going to ask you to step way, way, way, way, way back into your time machine, so if you've got the proper attire on, let's do it. If we're going to step into that time machine, let's peel the layers, before serverless, before containers, before cloud even, what is the origin story of Jeremy Daly, the man who usually asks the questions.Jeremy: That's tough. I don't think time machines go back that far, but it's funny, when I was in high school, I was involved with music, and plays, and all kinds of things like that. I was a very creative person. I loved creating things, that was one of the biggest sort of things, and whether it was music or whatever and I did a lot of work with video actually, back in the day. I was always volunteering at the local public access station. And when I graduated from high school, I had no idea what I wanted to do. I had used computers at the computer lab at the high school. I mean, this is going back a ways, so it wasn't everyone had their own computer in their house, but I went to college and then, my first, my freshman year in college, I ended up, there's a suite-mate that I had who showed me a website that he built on the university servers.And I saw that and I was immediately like, "Whoa, how do you do that"? Right, just this idea of creating something new and being able to build that out was super exciting to me, so I spent the next couple of weeks figuring out how to do HTML, and this was before, this was like when JavaScript was super, super early and we're talking like 1997, and everything was super early. I was using this, I eventually moved away from using FrontPage and started using this thing called HotDog. It was a software for HTML coding, but I started doing that, and I started building websites, and then after a while, I started figuring out what things like CGI-bins were, and how you could write Perl scripts, and how you could make interactions happen, and how you could capture FormData and serve up different things, and it was a lot of copying and pasting.My major at the time, I think was psychology, because it was like a default thing that I could do. But then I moved into computer science. I did computer science for about a year, and I felt that that was a little bit too narrow for what I was hoping to sort of do. I was starting to become more entrepreneurial. I had started selling websites to people. I had gone to a couple of local businesses and started building websites, so I actually expanded that and ended up doing sort of a major that straddled computer science and management, like business administration. So I ended up graduating with a degree in e-commerce and internet marketing, which is sort of very early, like before any of this stuff seemed to even exist. And then from there, I started a web development company, worked on that for 12 years, and then I ended up selling that off. Did a startup, failed the startup. Then from that startup, went to another startup, worked there for a couple of years, went to another startup, did a lot of consulting in between there, somewhere along the way I found serverless and AWS Cloud, and then now it's sort of led me to advocacy for building things with serverless and now I'm building sort of the, I think what I've been dreaming about building for the last several years in what I'm doing now at Serverless, Inc.Rebecca: Wow. All right. So this love story started in the 90s.Jeremy: The 90s, right.Rebecca: That's an incredible, era and welcome to 2021.Jeremy: Right. It's been a journey.Rebecca: Yeah, truly, that's literally a new millennium. So in a broad way of saying it, you've seen it all. You've started from the very HotDog of the world, to today, which is an incredible name, I'm going to have to look them up later. So then you said serverless came along somewhere in there, but let's go to the middle of your story here, so before Serverless Chats, before its predecessor, which is your weekly Off-by-none newsletter, and before, this is my favorite one, debates around, what the suffix "less" means when appended to server. When did you first hear about Serverless in that moment, or perhaps you don't remember the exact minute, but I do really want to know what struck you about it? What stood out about serverless rather than any of the other types of technologies that you could have been struck by and been having a podcast around?Jeremy: Right. And I think I gave you maybe too much of a surface level of what I've seen, because I talked mostly about software, but if we go back, I mean, hardware was one of those things where hardware, and installing software, and running servers, and doing networking, and all those sort of things, those were part of my early career as well. When I was running my web development company, we started by hosting on some hosting service somewhere, and then we ended up getting a dedicated server, and then we outgrew that, and then we ended up saying, "Well maybe we'll bring stuff in-house". So we did on-prem for quite some time, where we had our own servers in the T1 line, and then we moved to another building that had a T3 line, and if anybody doesn't know what that is, you probably don't need to anymore.But those are the things that we were doing, and then eventually we moved into a co-location facility where we rented space, and we rented electricity, and we rented all the utilities, the bandwidth, and so forth, but we had Blade servers and I was running VMware, and we were doing all this kind of stuff to manage the infrastructure, and then writing software on top of that, so it was a lot of work. I know I posted something on Twitter a few weeks ago, about how, when I was, when we were young, we used to have to carry a server on our back, uphill, both ways, to the data center, in the snow, with no shoes, and that's kind of how it felt, that you were doing a lot of these things.And then 2008, 2009, as I was kind of wrapping up my web development company, we were just in the process of actually saying it's too expensive at the colo. I think we were paying probably between like $5,000 and $7,000 a month between the ... we had leases on some of the servers, you're paying for electricity, you're paying for all these other things, and we were running a fair amount of services in there, so it seemed justifiable. We were making money on it, that wasn't the problem, but it just was a very expensive fixed cost for us, and when the cloud started coming along and I started actually building out the startup that I was working on, we were building all of that in the cloud, and as I was learning more about the cloud and how that works, I'm like, I should just move all this stuff that's in the co-location facility, move that over to the cloud and see what happens.And it took a couple of weeks to get that set up, and now, again, this is early, this is before ELB, this is before RDS, this is before, I mean, this was very, very early cloud. I mean, I think there was S3 and EC2. I think those were the two services that were available, with a few other things. I don't even think there were VPCs yet. But anyways, I moved everything over, took a couple of weeks to get that over, and essentially our bill to host all of our clients' sites and projects went from $5,000 to $7,000 a month, to $750 a month or something like that, and it's funny because had I done that earlier, I may not have sold off my web development company because it could have been much more profitable, so it was just an interesting move there.So we got into the cloud fairly early and started sort of leveraging that, and it was great to see all these things get added and all these specialty services, like RDS, and just taking the responsibility because I literally was installing Microsoft SQL server on an EC2 instance, which is not something that you want to do, you want to use RDS. It's just a much better way to do it, but anyways, so I was working for another startup, this was like startup number 17 or whatever it was I was working for, and we had this incident where we were using ... we had a pretty good setup. I mean, everything was on EC2 instances, but we were using DynamoDB to do some caching layers for certain things. We were using a sharded database, MySQL database, for product information, and so forth.So the system was pretty resilient, it was pretty, it handled all of the load testing we did and things like that, but then we actually got featured on Good Morning America, and they mentioned our app, it was the Power to Mobile app, and so we get mentioned on Good Morning America. I think it was Good Morning America. The Today Show? Good Morning America, I think it was. One of those morning shows, anyways, we got about 10,000 sign-ups in less than a minute, which was amazing, or it was just this huge spike in traffic, which was great. The problem was, is we had this really weak point in our system where we had to basically get a lock on the database in order to get an incremental-ID, and so essentially what happened is the database choked, and then as soon as the database choked, just to create user accounts, other users couldn't sign in and there was all kinds of problems, so we basically lost out on all of this capability.So I spent some time doing a lot of research and trying to figure out how do you scale that? How do you scale something that fast? How do you have that resilience in there? And there's all kinds of ways that we could have done it with traditional hardware, it's not like it wasn't possible to do with a slightly better strategy, but as I was digging around in AWS, I'm looking around at some different things, and we were, I was always in the console cause we were using Dynamo and some of those things, and I came across this thing that said "Lambda," with a little new thing next to it. I'm like, what the heck is this?So I click on that and I start reading about it, and I'm like, this is amazing. We don't have to spin up a server, we don't have to use Chef, or Puppet, or anything like that to spin up these machines. We can basically just say, when X happens, do Y, and it enlightened me, and this was early 2015, so this would have been right after Lambda went GA. Had never heard of Lambda as part of the preview, I mean, I wasn't sort of in that the re:Invent, I don't know, what would you call that? Vortex, maybe, is a good way to describe the event.Rebecca: Vortex sounds about right. That's about how it feels by the end.Jeremy: Right, exactly. So I wasn't really in that, I wasn't in that group yet, I wasn't part of that community, so I hadn't heard about it, and so as I started playing around with it, I immediately saw the value there, because, for me, as someone who again had managed servers, and it had built out really complex networking too. I think some of the things you don't think about when you move to an on-prem where you're managing your stuff, even what the cloud manages for you. I mean, we had firewalls, and we had to do all the firewall rules ourselves, right. I mean, I know you still have to do security groups and things like that in AWS, but just the level of complexity is a lot lower when you're in the cloud, and of course there's so many great services and systems that help you do that now.But just the idea of saying, "wait a minute, so if I have something happen, like a user signup, for example, and I don't have to worry about provisioning all the servers that I need in order to handle that," and again, it wasn't so much the server aspect of it as it was the database aspect of it, but one of the things that was sort of interesting about the idea of Serverless 2 was this asynchronous nature of it, this idea of being more event-driven, and that things don't have to happen immediately necessarily. So that just struck me as something where it seemed like it would reduce a lot, and again, this term has been overused, but the undifferentiated heavy-lifting, we use that term over and over again, but there is not a better term for that, right?Because there were just so many things that you have to do as a developer, as an ops person, somebody who is trying to straddle teams, or just a PM, or whatever you are, so many things that you have to do in order to get an application running, first of all, and then even more you have to do in order to keep it up and running, and then even more, if you start thinking about distributing it, or scaling it, or getting any of those things, disaster recovery. I mean, there's a million things you have to think about, and I saw serverless immediately as this opportunity to say, "Wait a minute, this could reduce a lot of that complexity and manage all of that for you," and then again, literally let you focus on the things that actually matter for your business.Rebecca: Okay. As someone who worked, how should I say this, in metatech, or the technology of technology in the serverless space, when you say that you were starting to build that without ELB even, or RDS, my level of anxiety is like, I really feel like I'm watching a slow horror film. I'm like, "No, no, no, no, no, you didn't, you didn't, you didn't have to do that, did you"?Jeremy: We did.Rebecca: So I applaud you for making it to the end of the film and still being with us.Jeremy: Well, the other thing ...Rebecca: Only one protagonist does that.Jeremy: Well, the other thing that's interesting too, about Serverless, and where it was in 2015, Lambda goes GA, this will give you some anxiety, there was no API gateway. So there was no way to actually trigger a Lambda function from a web request, right. There was no VPC access in Lambda functions, which meant you couldn't connect to a database. The only thing you do is connect via HDP, so you could connect to DynamoDB or things like that, but you could not connect directly to RDS, for example. So if you go back and you look at the timeline of when these things were released, I mean, if just from 2015, I mean, you literally feel like a caveman thinking about what you could do back then again, it's banging two sticks together versus where we are now, and the capabilities that are available to us.Rebecca: Yeah, you're sort of in Plato's cave, right, and you're looking up and you're like, "It's quite dark in here," and Lambda's up there, outside, sowing seeds, being like, "Come on out, it's dark in there". All right, so I imagine you discovering Lambda through the console is not a sentence you hear every day or general console discovery of a new product that will then sort of change the way that you build, and so I'm guessing maybe one of the reasons why you started your Off-by-none newsletter or Serverless Chats, right, is to be like, "How do I help tell others about this without them needing to discover it through the console"? But I'm curious what your why is. Why first the Off-by-none newsletter, which is one of my favorite things to receive every week, thank you for continuing to write such great content, and then why Serverless Chats? Why are we here today? Why are we at number 100? Which I'm so excited about every time I say it.Jeremy: And it's kind of crazy to think about all the people I've gotten a chance to talk to, but so, I think if you go back, I started writing blog posts maybe in 2015, so I haven't been doing it that long, and I certainly wasn't prolific. I wasn't consistent writing a blog post every week or every, two a week, like some people do now, which is kind of crazy. I don't know how that, I mean, it's hard enough writing the newsletter every week, never mind writing original content, but I started writing about Serverless. I think it wasn't until the beginning of 2018, maybe the end of 2017, and there was already a lot of great content out there. I mean, Ben Kehoe was very early into this and a lot of his stuff I read very early.I mean, there's just so many people that were very early in the space, I mean, Paul Johnson, I mean, just so many people, right, and I started reading what they were writing and I was like, "Oh, I've got some ideas too, I've been experimenting with some things, I feel like I've gotten to a point where what I could share could be potentially useful". So I started writing blog posts, and I think one of the earlier blog posts I wrote was, I want to say 2017, maybe it was 2018, early 2018, but was a post about serverless security, and what was great about that post was that actually got me connected with Ory Segal, who had started PureSec, and he and I became friends and that was the other great thing too, is just becoming part of this community was amazing.So many awesome people that I've met, but so I saw all this stuff people were writing and these things people were doing, and I got to maybe August of 2018, and I said to myself, I'm like, "Okay, I don't know if people are interested in what I'm writing". I wasn't writing a lot, but I was writing a little bit, but I wasn't sure people were overly interested in what I was writing, and again, that idea of the imposter syndrome, certainly everything was very early, so I felt a little bit more comfortable. I always felt like, well, maybe nobody knows what they're talking about here, so if I throw something into the fold it won't be too, too bad, but certainly, I was reading other things by other people that I was interested in, and I thought to myself, I'm like, "Okay, if I'm interested in this stuff, other people have to be interested in this stuff," but it wasn't easy to find, right.I mean, there was sort of a serverless Twitter, if you want to use that terminology, where a lot of people tweet about it and so forth, obviously it's gotten very noisy now because of people slapped that term on way too many things, but I don't want to have that discussion, but so I'm reading all this great stuff and I'm like, "I really want to share it," and I'm like, "Well, I guess the best way to do that would just be a newsletter."I had an email list for my own personal site that I had had a couple of hundred people on, and I'm like, "Well, let me just turn it into this thing, and I'll share these stories, and maybe people will find them interesting," and I know this is going to sound a little bit corny, but I have two teenage daughters, so I'm allowed to be sort of this dad-jokey type. I remember when I started writing the first version of this newsletter and I said to myself, I'm like, "I don't want this to be a newsletter." I was toying around with this idea of calling it an un-newsletter. I didn't want it to just be another list of links that you click on, and I know that's interesting to some people, but I felt like there was an opportunity to opine on it, to look at the individual links, and maybe even tell a story as part of all of the links that were shared that week, and I thought that that would be more interesting than just getting a list of links.And I'm sure you've seen over the last 140 issues, or however many we're at now, that there's been changes in the way that we formatted it, and we've tried new things, and things like that, but ultimately, and this goes back to the corny thing, I mean, one of the first things that I wanted to do was, I wanted to basically thank people for writing this stuff. I wanted to basically say, "Look, this is not just about you writing some content". This is big, this is important, and I appreciate it. I appreciate you for writing that content, and I wanted to make it more of a celebration really of the community and the people that were early contributors to that space, and that's one of the reasons why I did the Serverless Star thing.I thought, if somebody writes a really good article some week, and it's just, it really hits me, or somebody else says, "Hey, this person wrote a great article," or whatever. I wanted to sort of celebrate that person and call them out because that's one of the things too is writing blog posts or posting things on social media without a good following, or without the dopamine hit of people liking it, or re-tweeting it, and things like that, it can be a pretty lonely place. I mean, I know I feel that way sometimes when you put something out there, and you think it's important, or you think people might want to see it, and just not enough people see it.It's even worse, I mean, 240 characters, or whatever it is to write a tweet is one thing, or 280 characters, but if you're spending time putting together a tutorial or you put together a really good thought piece, or story, or use case, or something where you feel like this is worth sharing, because it could inspire somebody else, or it could help somebody else, could get them past a bump, it could make them think about something a different way, or get them over a hump, or whatever. I mean, that's just the kind of thing where I think people need that encouragement, and I think people deserve that encouragement for the work that they're doing, and that's what I wanted to do with Off-by-none, is make sure that I got that out there, and to just try to amplify those voices the best that I could. The other thing where it's sort of progressed, and I guess maybe I'm getting ahead of myself, but the other place where it's progressed and I thought was really interesting, was, finding people ...There's the heavy hitters in the serverless space, right? The ones we all know, and you can name them all, and they are great, and they produce amazing content, and they do amazing things, but they have pretty good engines to get their content out, right? I mean, some people who write for the AWS blog, they're on the AWS blog, right, so they're doing pretty well in terms of getting their things out there, right, and they've got pretty good engines.There's some good dev advocates too, that just have good Twitter followings and things like that. Then there's that guy who writes the story. I don't know, he's in India or he's in Poland or something like that. He writes this really good tutorial on how to do this odd edge-case for serverless. And you go and you look at their Medium and they've got two followers on Medium, five followers on Twitter or something like that. And that to me, just seems unfair, right? I mean, they've written a really good piece and it's worth sharing right? And it needs to get out there. I don't have a huge audience. I know that. I mean I've got a good following on Twitter. I feel like a lot of my Twitter followers, we can have good conversations, which is what you want on Twitter.The newsletter has continued to grow. We've got a good listener base for this show here. So, I don't have a huge audience, but if I can share that audience with other people and get other people to the forefront, then that's important to me. And I love finding those people and those ideas that other people might not see because they're not looking for them. So, if I can be part of that and help share that, that to me, it's not only a responsibility, it's just it's incredibly rewarding. So ...Rebecca: Yeah, I have to ... I mean, it is your 100th episode, so hopefully I can give you some kudos, but if celebrating others' work is one of your main tenets, you nail it every time. So ...Jeremy: I appreciate that.Rebecca: Just wanted you to know that. So, that's sort of the Genesis of course, of both of these, right?Jeremy: Right.Rebecca: That underpins the foundational how to share both works or how to share others' work through different channels. I'm wondering how it transformed, there's this newsletter and then of course it also has this other component, which is Serverless Chats. And that moment when you were like, "All right, this newsletter, this narrative that I'm telling behind serverless, highlighting all of these different authors from all these different global spaces, I'm going to start ... You know what else I want to do? I don't have enough to do, I'm going to start a podcast." How did we get here?Jeremy: Well, so the funny thing is now that I think about it, I think it just goes back to this tenet of fairness, this idea where I was fortunate, and I was able to go down to New York City and go to Serverless Days New York in late 2018. I was able to ... Tom McLaughlin actually got me connected with a bunch of great people in Boston. I live just outside of Boston. We got connected with a bunch of great people. And we started the Serverless Days Boston for 2019. And we were on that committee. I started traveling and I was going to conferences and I was meeting people. I went to re:Invent in 2018, which I know a lot of people just don't have the opportunity to do. And the interesting thing was, is that I was pulling aside brilliant people either in the hallway at a conference or more likely for a very long, deep discussion that we would have about something at a pub in Northern Ireland or something like that, right?I mean, these were opportunities that I was getting that I was privileged enough to get. And I'm like, these are amazing conversations. Just things that, for me, I know changed the way I think. And one of the biggest things that I try to do is evolve my thinking. What I thought a year ago is probably not what I think now. Maybe call it flip-flopping, whatever you want to call it. But I think that evolving your thinking is the most progressive thing that you can do and starting to understand as you gain new perspectives. And I was talking to people that I never would have talked to if I was just sitting here in my home office or at the time, I mean, I was at another office, but still, I wasn't getting that context. I wasn't getting that experience. And I wasn't getting those stories that literally changed my mind and made me think about things differently.And so, here I was in this privileged position, being able to talk to these amazing people and in some cases funny, because they're celebrities in their own right, right? I mean, these are the people where other people think of them and it's almost like they're a celebrity. And these people, I think they deserve fame. Don't get me wrong. But like as someone who has been on that side of it as well, it's ... I don't know, it's weird. It's weird to have fans in a sense. I love, again, you can be my friend, you don't have to be my fan. But that's how I felt about ...Rebecca: I'm a fan of my friends.Jeremy: So, a fan and my friend. So, having talked to these other people and having these really deep conversations on serverless and go beyond serverless to me. Actually I had quite a few conversations with some people that have nothing to do with serverless. Actually, Peter Sbarski and I, every time we get together, we only talk about the value of going to college for some reason. I don't know why. It has usually nothing to do with serverless. So, I'm having these great conversations with these people and I'm like, "Wow, I wish I could share these. I wish other people could have this experience," because I can tell you right now, there's people who can't travel, especially a lot of people outside of the United States. They ... it's hard to travel to the United States sometimes.So, these conversations are going on and I thought to myself, I'm like, "Wouldn't it be great if we could just have these conversations and let other people hear them, hopefully without bar glasses clinking in the background. And so I said, "You know what? Let's just try it. Let's see what happens. I'll do a couple of episodes. If it works, it works. If it doesn't, it doesn't. If people are interested, they're interested." But that was the genesis of that, I mean, it just goes back to this idea where I felt a little selfish having conversations and not being able to share them with other people.Rebecca: It's the very Jeremy Daly tenet slogan, right? You got to share it. You got to share it ...Jeremy: Got to share it, right?Rebecca: The more he shares it, it celebrates it. I love that. I think you do ... Yeah, you do a great job giving a megaphone so that more people can hear. So, in case you need a reminder, actually, I'll ask you, I know what the answer is to this, but do you know the answer? What was your very first episode of Serverless Chats? What was the name, and how long did it last?Jeremy: What was the name?Rebecca: Oh yeah. Oh yeah.Jeremy: Oh, well I know ... Oh, I remember now. Well, I know it was Alex DeBrie. I absolutely know that it was Alex DeBrie because ...Rebecca: Correct on that.Jeremy: If nobody, if you do not know Alex DeBrie, not only is he an AWS data hero, as well as the author of The DynamoDB Book, but he's also like the most likable person on the planet too. It is really hard if you've ever met Alex, that you wouldn't remember him. Alex and I started communicating, again, we met through the serverless space. I think actually he was working at Serverless Inc. at the time when we first met. And I think I met him in person, finally met him in person at re:Invent 2018. But he and I have collaborated on a number of things and so forth. So, let me think what the name of it was. "Serverless Purity Versus Practicality" or something like that. Is that close?Rebecca: That's exactly what it was.Jeremy: Oh, all right. I nailed it. Nailed it. Yes!Rebecca: Wow. Well, it's a great title. And I think ...Jeremy: Don't ask me what episode number 27 was though, because no way I could tell you that.Rebecca: And just for fun, it was 34 minutes long and you released it on June 17th, 2019. So, you've come a long way in a year and a half. That's some kind of wildness. So it makes sense, like, "THE," capital, all caps, bold, italic, author for databases, Alex DeBrie. Makes sense why you selected him as your guest. I'm wondering if you remember any of the ... What do you remember most about that episode? What was it like planning it? What was the reception of it? Anything funny happened recording it or releasing it?Jeremy: Yeah, well, I mean, so the funny thing is that I was incredibly nervous. I still am, actually a lot of guests that I have, I'm still incredibly nervous when I'm about to do the actual interview. And I think it's partially because I want to do justice to the content that they're presenting and to their expertise. And I feel like there's a responsibility to them, but I also feel like the guests that I've had on, some of them are just so smart, and the things they say, just I'm in awe of some of the things that come out of these people's mouths. And I'm like, "This is amazing and people need to hear this." And so, I feel like we've had really good episodes and we've had some okay episodes, but I feel like I want to try to keep that level up so that they owe that to my listener to make sure that there is high quality episode that, high quality information that they're going to get out of that.But going back to the planning of the initial episodes, so I actually had six episodes recorded before I even released the first one. And the reason why I did that was because I said, "All right, there's no way that I can record an episode and then wait a week and then record another episode and wait a week." And I thought batching them would be a good idea. And so, very early on, I had Alex and I had Nitzan Shapira and I had Ran Ribenzaft and I had Marcia Villalba and I had Erik Peterson from Cloud Zero. And so, I had a whole bunch of these episodes and I reached out to I think, eight or nine people. And I said, "I'm doing this thing, would you be interested in it?" Whatever, and we did planning sessions, still a thing that I do today, it's still part of the process.So, whenever I have a guest on, if you are listening to an episode and you're like, "Wow, how did they just like keep the thing going ..." It's not scripted. I don't want people to think it's scripted, but it is, we do review the outline and we go through some talking points to make sure that again, the high-quality episode and that the guest says all the things that the guest wants to say. A lot of it is spontaneous, right? I mean, the language is spontaneous, but we do, we do try to plan these episodes ahead of time so that we make sure that again, we get the content out and we talk about all the things we want to talk about. But with Alex, it was funny.He was actually the first of the six episodes that I recorded, though. And I wasn't sure who I was going to do first, but I hadn't quite picked it yet, but I recorded with Alex first. And it was an easy, easy conversation. And the reason why it was an easy conversation was because we had talked a number of times, right? It was that in a pub, talking or whatever, and having that friendly chat. So, that was a pretty easy conversation. And I remember the first several conversations I had, I knew Nitzan very well. I knew Ran very well. I knew Erik very well. Erik helped plan Serverless Days Boston with me. And I had known Marcia very well. Marcia actually had interviewed me when we were in Vegas for re:Invent 2018.So, those were very comfortable conversations. And so, it actually was a lot easier to do, which probably gave me a false sense of security. I was like, "Wow, this was ... These came out pretty well." The conversations worked pretty well. And also it was super easy because I was just doing audio. And once you add the video component into it, it gets a little bit more complex. But yeah, I mean, I don't know if there's anything funny that happened during it, other than the fact that I mean, I was incredibly nervous when we recorded those, because I just didn't know what to expect. If anybody wants to know, "Hey, how do you just jump right into podcasting?" I didn't. I actually was planning on how can I record my voice? How can I get comfortable behind a microphone? And so, one of the things that I did was I started creating audio versions of my blog posts and posting them on SoundCloud.So, I did that for a couple of ... I'm sorry, a couple of blog posts that I did. And that just helped make me feel a bit more comfortable about being able to record and getting a little bit more comfortable, even though I still can't stand the sound of my own voice, but hopefully that doesn't bother other people.Rebecca: That is an amazing ... I think we so often talk about ideas around you know where you want to go and you have this vision and that's your goal. And it's a constant reminder to be like, "How do I make incremental steps to actually get to that goal?" And I love that as a life hack, like, "Hey, start with something you already know that you wrote and feel comfortable in and say it out loud and say it out loud again and say it out loud again." And you may never love your voice, but you will at least feel comfortable saying things out loud on a podcast.Jeremy: Right, right, right. I'm still working on the, "Ums" and, "Ahs." I still do that. And I don't edit those out. That's another thing too, actually, that one of the things I do want people to know about this podcast is these are authentic conversations, right? I am probably like ... I feel like I'm, I mean, the most authentic person that I know. I just want authenticity. I want that out of the guests. The idea of putting together an outline is just so that we can put together a high quality episode, but everything is authentic. And that's what I want out of people. I just want that authenticity, and one of the things that I felt kept that, was leaving in, "Ums" and, "Ahs," you know what I mean? It's just, it's one of those things where I know a lot of podcasts will edit those out and it sounds really polished and finished.Again, I mean, I figured if we can get the clinking glasses out from the background of a bar and just at least have the conversation that that's what I'm trying to achieve. And we do very little editing. We do cut things out here and there, especially if somebody makes a mistake or they want to start something over again, we will cut that out because we want, again, high quality episodes. But yeah, but authenticity is deeply important to me.Rebecca: Yeah, I think it probably certainly helps that neither of us are robots because robots wouldn't say, "Um" so many times. As I say, "Uh." So, let's talk about, Alex DeBrie was your first guest, but there's been a hundred episodes, right? So, from, I might say the best guest, as a hundredth episode guests, which is our very own Jeremy Daly, but let's go back to ...Jeremy: I appreciate that.Rebecca: Your guests, one to 99. And I mean, you've chatted with some of the most thoughtful, talented, Serverless builders and architects in the industry, and across coincident spaces like ML and Voice Technology, Chaos Engineering, databases. So, you started with Alex DeBrie and databases, and then I'm going to list off some names here, but there's so many more, right? But there's the Gunnar Grosches, and the Alexandria Abbasses, and Ajay Nair, and Angela Timofte, James Beswick, Chris Munns, Forrest Brazeal, Aleksandar Simovic, and Slobodan Stojanovic. Like there are just so many more. And I'm wondering if across those hundred conversations, or 99 plus your own today, if you had to distill those into two or three lessons, what have you learned that sticks with you? If there are emerging patterns or themes across these very divergent and convergent thinkers in the serverless space?Jeremy: Oh, that's a tough question.Rebecca: You're welcome.Jeremy: So, yeah, put me on the spot here. So, yeah, I mean, I think one of the things that I've, I've seen, no matter what it's been, whether it's ML or it's Chaos Engineering, or it's any of those other observability and things like that. I think the common thing that threads all of it is trying to solve problems and make people's lives easier. That every one of those solutions is like, and we always talk about abstractions and, and higher-level abstractions, and we no longer have to write ones and zeros on punch cards or whatever. We can write languages that either compile or interpret it or whatever. And then the cloud comes along and there's things we don't have to do anymore, that just get taken care of for us.And you keep building these higher level of abstractions. And I think that's a lot of what ... You've got this underlying concept of letting somebody else handle things for you. And then you've got this whole group of people that are coming at it from a number of different angles and saying, "Well, how will that apply to my use case?" And I think a lot of those, a lot of those things are very, very specific. I think things like the voice technology where it's like the fact that serverless powers voice technology is only interesting in the fact as to say that, the voice technology is probably the more interesting part, the fact that serverless powers it is just the fact that it's a really simple vehicle to do that. And basically removes this whole idea of saying I'm building voice technology, or I'm building a voice app, why do I need to worry about setting up servers and all this kind of stuff?It just takes that away. It takes that out of the equation. And I think that's the perfect idea of saying, "How can you take your use case, fit serverless in there and apply it in a way that gets rid of all that extra overhead that you shouldn't have to worry about." And the same thing is true of machine learning. And I mean, and SageMaker, and things like that. Yeah, you're still running instances of it, or you still have to do some of these things, but now there's like SageMaker endpoints and some other things that are happening. So, it's moving in that direction as well. But then you have those really high level services like NLU API from IBM, which is the Watson Natural Language Processing.You've got AP recognition, you've got the vision API, you've got sentiment analysis through all these different things. So, you've got a lot of different services that are very specific to machine learning and solving a discrete problem there. But then basically relying on serverless or at least presenting it in a way that's serverless, where you don't have to worry about it, right? You don't have to run all of these Jupiter notebooks and things like that, to do machine learning for a lot of cases. This is one of the things I talk about with Alexandra Abbas, was that these higher level APIs are just taking a lot of that responsibility or a lot of that heavy lifting off of your plate and allowing you to really come down and focus on the things that you're doing.So, going back to that, I do think that serverless, that the common theme that I see is that this idea of worrying about servers and worrying about patching things and worrying about networking, all that stuff. For so many people now, that's just not even a concern. They didn't even think about it. And that's amazing to think of, compute ... Or data, or networking as a utility that is now just available to us, right? And I mean, again, going back to my roots, taking it for granted is something that I think a lot of people do, but I think that's also maybe a good thing, right? Just don't think about it. I mean, there are people who, they're still going to be engineers and people who are sitting in the data center somewhere and racking servers and doing it, that's going to be forever, right?But for the things that you're trying to build, that's unimportant to you. That is the furthest from your concern. You want to focus on the problem that you're trying to solve. And so I think that, that's a lot of what I've seen from talking to people is that they are literally trying to figure out, "Okay, how do I take what I'm doing, my use case, my problem, how do I take that to the next level, by being able to spend my cycles thinking about that as opposed to how I'm going to serve it up to people?"Rebecca: Yeah, I think it's the mantra, right, of simplify, simplify, simplify, or maybe even to credit Bruce Lee, be like water. You're like, "How do I be like water in this instance?" Well, it's not to be setting up servers, it's to be doing what I like to be doing. So, you've interviewed these incredible folks. Is there anyone left on your list? I'm sure there ... I mean, I know that you have a large list. Is there a few key folks where you're like, "If this is the moment I'm going to ask them, I'm going to say on the hundredth episode, 'Dear so-and-so, I would love to interview you for Serverless Chats.'" Who are you asking?Jeremy: So, this is something that, again, we have a stretch list of guests that we attempt to reach out to every once in a while just to say, "Hey, if we get them, we get them." But so, I have a long list of people that I would absolutely love to talk to. I think number one on my list is certainly Werner Vogels. I mean, I would love to talk to Dr. Vogels about a number of things, and maybe even beyond serverless, I'm just really interested. More so from a curiosity standpoint of like, "Just how do you keep that in your head?" That vision of where it's going. And I'd love to drill down more into the vision because I do feel like there's a marketing aspect of it, that's pushing on him of like, "Here's what we have to focus on because of market adoption and so forth. And even though the technology, you want to move into a certain way," I'd be really interesting to talk to him about that.And I'd love to talk to him more too about developer experience and so forth, because one of the things that I love about AWS is that it gives you so many primitives, but at the same time, the thing I hate about AWS is it gives you so many primitives. So, you have to think about 800 services, I know it's not that many, but like, what is it? 200 services, something like that, that all need to kind of connect together. And I love that there's that diversity in those capabilities, it's just from a developer standpoint, it's really hard to choose which ones you're supposed to use, especially when several services overlap. So, I'm just curious. I mean, I'd love to talk to him about that and see what the vision is in terms of, is that the idea, just to be a salad bar, to be the Golden Corral of cloud services, I guess, right?Where you can choose whatever you want and probably take too much and then not use a lot of it. But I don't know if that's part of the strategy, but I think there's some interesting questions, could dig in there. Another person from AWS that I actually want to talk to, and I haven't reached out to her yet just because, I don't know, I just haven't reached out to her yet, but is Brigid Johnson. She is like an IAM expert. And I saw her speak at re:Inforce 2019, it must have been 2019 in Boston. And it was like she was speaking a different language, but she knew IAM so well, and I am not a fan of IAM. I mean, I'm a fan of it in the sense that it's necessary and it's great, but I can't wrap my head around so many different things about it. It's such a ...It's an ongoing learning process and when it comes to things like being able to use tags to elevate permissions. Just crazy things like that. Anyways, I would love to have a conversation with her because I'd really like to dig down into sort of, what is the essence of IAM? What are the things that you really have to think about with least permission? Especially applying it to serverless services and so forth. And maybe have her help me figure out how to do some of the cross role IAM things that I'm trying to do. Certainly would love to speak to Jeff Barr. I did meet Jeff briefly. We talked for a minute, but I would love to chat with him.I think he sets a shining example of what a developer advocate is. Just the way that ... First of all, he's probably the only person alive who knows every service at AWS and has actually tried it because he writes all those blog posts about it. So that would just be great to pick his brain on that stuff. Also, Adrian Cockcroft would be another great person to talk to. Just this idea of what he's done with microservices and thinking about the role, his role with Netflix and some of those other things and how all that kind of came together, I think would be a really interesting conversation. I know I've seen this in so many of his presentations where he's talked about the objections, what were the objections of Lambda and how have you solved those objections? And here's the things that we've done.And again, the methodology of that would be really interesting to know. There's a couple of other people too. Oh, Sam Newman who wrote Building Microservices, that was my Bible for quite some time. I had it on my iPad and had a whole bunch of bookmarks and things like that. And if anybody wants to know, one of my most popular posts that I've ever written was the ... I think it was ... What is it? 16, 17 architectural patterns for serverless or serverless microservice patterns on AWS. Can't even remember the name of my own posts. But that post was very, very popular. And that even was ... I know Matt Coulter who did the CDK. He's done the whole CDK ... What the heck was that? The CDKpatterns.com. That was one of the things where he said that that was instrumental for him in seeing those patterns and being able to use those patterns and so forth.If anybody wants to know, a lot of those patterns and those ideas and those ... The sort of the confidence that I had with presenting those patterns, a lot of that came from Sam Newman's work in his Building Microservices book. So again, credit where credit is due. And I think that that would be a really fascinating conversation. And then Simon Wardley, I would love to talk to. I'd actually love to ... I actually talked to ... I met Lin Clark in Vegas as well. She was instrumental with the WebAssembly stuff, and I'd love to talk to her. Merritt Baer. There's just so many people. I'm probably just naming too many people now. But there are a lot of people that I would love to have a chat with and just pick their brain.And also, one of the things that I've been thinking about a lot on the show as well, is the term "serverless." Good or bad for some people. Some of the conversations we have go outside of serverless a little bit, right? There's sort of peripheral to it. I think that a lot of things are peripheral to serverless now. And there are a lot of conversations to be had. People who were building with serverless. Actually real-world examples.One of the things I love hearing was Yan Cui's "Real World Serverless" podcast where he actually talks to people who are building serverless things and building them in their organizations. That is super interesting to me. And I would actually love to have some of those conversations here as well. So if anyone's listening and you have a really interesting story to tell about serverless or something peripheral to serverless please reach out and send me a message and I'd be happy to talk to you.Rebecca: Well, good news is, it sounds like A, we have at least ... You've got at least another a hundred episodes planned out already.Jeremy: Most likely. Yeah.Rebecca: And B, what a testament to Sam Newman. That's pretty great when your work is referred to as the Bible by someone. As far as in terms of a tome, a treasure trove of perhaps learnings or parables or teachings. I ... And wow, what a list of other folks, especially AWS power ... Actually, not AWS powerhouses. Powerhouses who happened to work at AWS. And I think have paved the way for a ton of ways of thinking and even communicating. Right? So I think Jeff Barr, as far as setting the bar, raising the bar if you will. For how to teach others and not be so high-level, or high-level enough where you can follow along with him, right? Not so high-level where it feels like you can't achieve what he's showing other people how to do.Jeremy: Right. And I just want to comment on the Jeff Barr thing. Yeah.Rebecca: Of course.Jeremy: Because again, I actually ... That's my point. That's one of the reasons why I love what he does and he's so perfect for that position because he's relatable and he presents things in a way that isn't like, "Oh, well, yeah, of course, this is how you do this." I mean, it's not that way. It's always presented in a way to make it accessible. And even for services that I'm not interested in, that I know that I probably will never use, I generally will read Jeff's post because I feel it gives me a good overview, right?Rebecca: Right.Jeremy: It just gives me a good overview to understand whether or not that service is even worth looking at. And that's certainly something I don't get from reading the documentation.Rebecca: Right. He's inviting you to come with him and understanding this, which is so neat. So I think ... I bet we should ... I know that we can find all these twitter handles for these folks and put them in the show notes. And I'm especially ... I'm just going to say here that Werner Vogels's twitter handle is @Werner. So maybe for your hundredth, all the listeners, everyone listening to this, we can say, "Hey, @Werner, I heard that you're the number one guest that Jeremy Daly would like to interview." And I think if we get enough folks saying that to @Werner ... Did I say that @Werner, just @Werner?Jeremy: I think you did.Rebecca: Anyone if you can hear it.Jeremy: Now listen, he did retweet my serverless musical that I did. So ...Rebecca: That's right.Jeremy: I'm sort of on his radar maybe.Rebecca: Yeah. And honestly, he loves serverless, especially with the number of customers and the types of customers and ... that are doing incredible things with it. So I think we've got a chance, Jeremy. I really do. That's what I'm trying to say.Jeremy: That's good to know. You're welcome anytime. He's welcome anytime.Rebecca: Do we say that @Werner, you are welcome anytime. Right. So let's go back to the genesis, not necessarily the genesis of the concept, right? But the genesis of the technology that spurred all of these other technologies, which is AWS Lambda. And so what ... I don't think we'd be having these conversations, right, if AWS Lambda was not released in late 2014, and then when GA I believe in 2015.Jeremy: Right.Rebecca: And so subsequently the serverless paradigm was thrust into the spotlight. And that seems like eons ago, but also three minutes ago.Jeremy: Right.Rebecca: And so I'm wondering ... Let's talk about its evolution a bit and a bit of how if you've been following it for this long and building it for this long, you've covered topics from serverless CI/CD pipelines, observability. We already talked about how it's impacted voice technologies or how it's made it easy. You can build voice technology without having to care about what that technology is running on.Jeremy: Right.Rebecca: You've even talked about things like the future and climate change and how it relates to serverless. So some of those sort of related conversations that you were just talking about wanting to have or having had with previous guests. So as a host who thinks about these topics every day, I'm wondering if there's a topic that serverless hasn't touched yet or one that you hope it will soon. Those types of themes, those threads that you want to pull in the next 100 episodes.Jeremy: That's another tough question. Wow. You got good questions.Rebecca: That's what I said. Heavy hitters. I told you I'd be bringing it.Jeremy: All right. Well, I appreciate that. So that's actually a really good question. I think the evolution of serverless has seen its ups and downs. I think one of the nice things is you look at something like serverless that was so constrained when it first started. And it still has constraints, which are good. But it ... Those constraints get lifted. We just talked about Adrian's talks about how it's like, "Well, I can't do this, or I can't do that." And then like, "Okay, we'll add some feature that you can do that and you can do that." And I think that for the most part, and I won't call it anything specific, but I think for the most part that the evolution of serverless and the evolution of Lambda and what it can do has been thoughtful. And by that I mean that it was sort of like, how do we evolve this into a way that doesn't create too much complexity and still sort of holds true to the serverless ethos of sort of being fairly easy or just writing code.And then, but still evolve it to open up these other use cases and edge cases. And I think that for the most part, that it has held true to that, that it has been mostly, I guess, a smooth ride. There are several examples though, where it didn't. And I said I wasn't going to call anything out, but I'm going to call this out. I think RDS proxy wasn't great. I think it works really well, but I don't think that's the solution to the problem. And it's a band-aid. And it works really well, and congrats to the engineers who did it. I think there's a story about how two different teams were trying to build it at the same time actually. But either way, I look at that and I say, "That's a good solution to the problem, but it's not the solution to the problem."And so I think serverless has stumbled in a number of ways to do that. I also feel EFS integration is super helpful, but I'm not sure that's the ultimate goal to share ... The best way to share state. But regardless, there are a whole bunch of things that we still need to do with serverless. And a whole bunch of things that we still need to add and we need to build, and we need to figure out better ways to do maybe. But I think in terms of something that doesn't get talked about a lot, is the developer experience of serverless. And that is, again I'm not trying to pitch anything here. But that's literally what I'm trying to work on right now in my current role, is just that that developer experience of serverless, even though there was this thoughtful approach to adding things, to try to check those things off the list, to say that it can't do this, so we're going to make it be able to do that by adding X, Y, and Z.As amazing as that has been, that has added layers and layers of complexity. And I'll go back way, way back to 1997 in my dorm room. CGI-bins, if people are not familiar with those, essentially just running on a Linux server, it was a way that it would essentially run a Perl script or other types of scripts. And it was essentially like you're running PHP or you're running Node, or you're running Ruby or whatever it was. So it would run a programming language for you, run a script and then serve that information back. And of course, you had to actually know ins and outs, inputs and outputs. It was more complex than it is now.But anyways, the point is that back then though, once you had the script written. All you had to do is ... There's a thing called FTP, which I'm sure some people don't even know what that is anymore. File transfer protocol, where you would basically say, take this file from my local machine and put it on this server, which is a remote machine. And you would do that. And the second you did that, magically it was updated and you had this thing happening. And I remember there were a lot of jokes way back in the early, probably 2017, 2018, that serverless was like the new CGI-bin or something like that. But more as a criticism of it, right? Or it's just CGI-bins reborn, whatever. And I actually liked that comparison. I felt, you know what? I remember the days where I just wrote code and I just put it to some other server where somebody was dealing with it, and I didn't even have to think about that stuff.We're a long way from that now. But that's how serverless felt to me, one of the first times that I started interacting with it. And I felt there was something there, that was something special about it. And I also felt the constraints of serverless, especially the idea of not having state. People rely on things because they're there. But when you don't have something and you're forced to think differently and to make a change or find a way to work around it. Sometimes workarounds, turn into best practices. And that's one of the things that I saw with serverless. Where people were figuring out pretty quickly, how to build applications without state. And then I think the problem is that you had a lot of people who came along, who were maybe big customers of AWS. I don't know.I'm not going to say that you might be influenced by large customers. I know lots of places are. That said, "We need this." And maybe your ... The will gets bent, right. Because you just... you can only fight gravity for so long. And so those are the kinds of things where I feel some of the stuff has been patchwork and those patchwork things haven't ruined serverless. It's still amazing. It's still awesome what you can do within the course. We're still really just focusing on fast here, with everything else that's built. With all the APIs and so forth and everything else that's serverless in the full-service ecosystem. There's still a lot of amazing things there. But I do feel we've become so complex with building serverless applications, that you can't ... the Hello World is super easy, but if you're trying to build an actual application, it's a whole new mindset.You've got to learn a whole bunch of new things. And not only that, but you have to learn the cloud. You have to learn all the details of the cloud, right? You need to know all these different things. You need to know cloud formation or serverless framework or SAM or something like that, in order to get the stuff into the cloud. You need to understand the infrastructure that you're working with. You may not need to manage it, but you still have to understand it. You need to know what its limitations are. You need to know how it connects. You need to know what the failover states are like.There's so many things that you need to know. And to me, that's a burden. And that's adding new types of undifferentiated heavy-lifting that shouldn't be there. And that's the conversation that I would like to have continuing to move forward is, how do you go back to a developer experience where you're saying you're taking away all this stuff. And again, to call out Werner again, he constantly says serverless is about writing code, but ask anybody who builds serverless applications. You're doing a lot more than writing code right now. And I would love to see us bring the conversation back to how do we get back there?Rebecca: Yeah. I think it kind of goes back to ... You and I have talked about this notion of an ode to simplicity. And it's sort of what you want to write into your ode, right? If we're going to have an ode to simplicity, how do we make sure that we keep the simplicity inside of the ode?Jeremy: Right.Rebecca:So I've got ... I don't know if you've seen these.Jeremy: I don't know.Rebecca: But before I get to some wrap-up questions more from the brainwaves of Jeremy Daly, I don't want to forget to call out some long-time listener questions. And they wrote in a via Twitter and they wanted to perhaps pick your brain on a few things.Jeremy: Okay.Rebecca: So I don't know if you're ready for this.Jeremy: A-M-A. A-M-A.Rebecca: I don't know if you've seen these. Yeah, these are going to put you in the ...Jeremy: A-M-A-M. Wait, A-M-A-A? Asked me almost anything? No, go ahead. Ask me anything.Rebecca: A-M-A-A. A-M-J. No. Anyway, we got it. Ask Jeremy almost anything.Jeremy: There you go.Rebecca: So there's just three to tackle for today's episode that I'm going to lob at you. One is from Ken Collins. "What will it take to get you back to a relational database of Lambda?"Jeremy: Ooh, I'm going to tell you right now. And without a doubt, Aurora Serverless v2. I played around with that right after re:Invent 2000. What was it? 20. Yeah. Just came out, right? I'm trying to remember what year it is at this point.Rebecca: Yes. Indeed.Jeremy: When that just ... Right when that came out. And I had spent a lot of time with Aurora Serverless v1, I guess if you want to call it that. I spent a lot of time with it. I used it on a couple of different projects. I had a lot of really good success with it. I had the same pains as everybody else did when it came to scaling and just the slowness of the scaling and then ... And some of the step-downs and some of those things. There were certainly problems with it. But v2 just the early, early preview version of v2 was ... It was just a marvel of engineering. And the way that it worked was just ... It was absolutely fascinating.And I know it's getting ready or it's getting close, I think, to being GA. And when that becomes GA, I think I will have a new outlook on whether or not I can fit RDS into my applications. I will say though. Okay. I will say, I don't think that transactional applications should be using relational databases though. One of the things that was sort of a nice thing about moving to serverless, speak

In this episode of AWS TechChat we continue with part two of our four part re:Invent 2020 series with this episode covering all Application Development, Containers, and Database announcements. For our developer community, we talked about: * Using CodeGuru’s new Security detectors to help you find and remediate security issues in your code * Python support for CodeGuru’s in preview * We shared another new service, DevOps Guru in preview, for measuring and improving an application’s operational performance * Lambda now supports up to 10 GB of memory and 6 vCPU cores and a billing granularity reduction down to 1ms * Amazon API Gateway now supports integration with Step Functions StartSyncExecution for HTTP APIs * Appflow simplifies cloud app integrations for connect customers with Customer Profiles * Similarly, Appflow can provide similar app integrations with those 3rd party apps to HoneyCode. * For those Amplify users, deploy Fargate containers through the Amplify CLI and you get a new AdminUI to boot that deploys all the underlying bits for you. * AWS Proton to bridge the gap between platform and development teams In containers we kicked it off with EKS. * First, cluster add-ons managed through the EKS console, CLI, or API. * Run EKS on premises with EKS Distribution * EKS on Fargate now has built in logging with Fluent Bit under the hood * You can now see all your Kubernetes resources in the EKS console without needing extra tools * Public registries for your container images with ECR public and the ECR public gallery * Use your existing containers as a lambda package format * ECS Deployment Circuit Breaker is in preview to stop deployments from getting worse and auto-rollback In database land we covered * Bablefish, not the mythological creature, but a translation layer between Aurora PostgresSQL and Microsoft SQL. * v2 of Aurora Serverless has arrived, considerably faster and scales in a fraction of second, with scaling so fast it is perfect for those event driven applications. * Data Exchange adds revision access rules for governing access * RDS Service Delivery Partners for when you want someone to build, deploy, and manage your RDS deployments * RDS Cross-Region backups comes to RDS for Oracle * Share data across Redshift clusters with data sharing in preview and pull data from partners directly via the RedShift Console. * RedShift Federated query comes to RDS for MySQL and Aurora MySQL * Redshift Automatic Table Optimization to keep your data warehouse running in tip top shape automatically. * Move RedShift clusters easily across Availability Zones. * JSON supports in preview for RedShift * Finally, AQUA comes to RedShift in Preview as a caching layer to speed up queries. Stay tuned as we cover all aspects of re:invent 2020 in our coming multi-part re:Invent update

最新情報を "ながら" でキャッチアップ！ ラジオ感覚放送 「毎日AWS」 おはようございます、サーバーワークスの加藤です。 今日は 12/1, 2 に出たアップデートをピックアップしてご紹介。 感想は Twitter にて「#サバワ」をつけて投稿してください！ ■ UPDATE PICKUP Amazon Aurora Serverless v2 がプレビューで発表 AQUA for Amazon Redshift をプレビューで発表 Babelfish for Amazon Aurora PostgreSQL がプレビューで発表 ■ re:Invent 開催中 公式ページから登録して今すぐ参加しましょう！ re:Invent に関するサーバーワークスの情報発信はこちら ■ サーバーワークスSNS Twitter / Facebook ■ サーバーワークスブログ サーバーワークスエンジニアブログ

Andreas Wittig and Michael Wittig from cloudonaut are discussing Andy Jassy's keynote from re:Invent 2020. The focus is on the newly announced services and features: ECS Anywhere, EBS volumes (gp3), Aurora Serverless v3, Lambda Container Support, and many more.

Amazon Aurora Serverless is an on-demand, automatic scaling configuration for Aurora where the database automatically starts up, shuts down, and scales capacity up or down based on your application's needs. It enables you to run your database in the cloud without managing any database instances. Aurora Serverless is a simple, cost-effective option for infrequent, intermittent, or unpredictable workloads. In this session, we explore these use cases, take a look under the hood, and delve into the future of serverless databases.

It was never easier to scale your compute layer. EC2 Auto Scaling, Fargate, and Lambda enable horizontal scaling. But how do you scale your database? Use a NoSQL database like DynamoDB, one could say. But what if you don't want to miss all the advantages of an SQL database? You should check out Amazon Aurora Serverless, a cloud-native SQL database.

In Episode 64 of Mobycast, Jon shares his thoughts on Gluecon 2019 and then dives into one of his favorite sessions which focused on AWS' Aurora Serverless. Welcome to Mobycast, a weekly conversation about cloud-native development, AWS, and building distributed systems.

In Episode 64 of Mobycast, Jon shares his thoughts on Gluecon 2019 and then dives into one of his favorite sessions which focused on AWS’ Aurora Serverless. Welcome to Mobycast, a weekly conversation about cloud-native development, AWS, and building distributed systems.

Amazon Aurora Serverless is an on-demand, autoscaling configuration for Aurora (MySQL-compatible edition) where the database automatically starts up, shuts down, and scales up or down capacity based on your application's needs. It enables you to run your database in the cloud without managing any database instances. Aurora Serverless is a simple, cost-effective option for infrequent, intermittent, or unpredictable workloads. In this session, we explore these use cases, take a look under the hood, and delve into the future of serverless databases. We also hear a case study from a customer building new functionality on top of Aurora Serverless.

Build faster, more scalable database applications with Amazon Aurora, a MySQL- and PostgreSQL-compatible relational database built for the cloud. We cover Aurora Serverless, which automatically scales your database up and down to meet demand; Fast Database Cloning, which makes data instantly available for application development; Backtrack, which rolls back your database between test runs; and Performance Insights, which helps assess the load on your database and optimize your SQL queries.

The emergence of serverless infrastructure and services represents a fundamental shift in how developers approach architecting applications. This is especially relevant in the world of SaaS where systems must efficiently and cost-effectively respond to continually shifting multi-tenant loads and profiles. We'll conduct an end-to-end review of all the elements of a serverless SaaS architecture that leverages a combination of AWS Lambda, Fargate, and Aurora Serverless. We'll look at how serverless influence the core elements of your architecture, including tenant isolation, service decomposition, management and monitoring, deployment, and identity. Complete Title: AWS re:Invent 2018: [REPEAT] Architecting Next Generation Serverless SaaS Solutions on AWS (ARC324-R)

Aurora, from Amazon Web Services (AWS), is a MySQL-compatible service for complex database structures. It offers capabilities and opportunities. But with Aurora, you’re putting a lot of trust in AWS to “just work” in ways not traditional to relational database services (RDS). David Torgerson, Principal DevOps Engineer at Lucidchart, is a mystery wrapped in an enigma and virtually impossible to Google. He shares Lucidchart’s experience with migrating away from a traditional RDS to Aurora to free up developer time. Some of the highlights of the show include: Trade off of making someone else partially responsible for keeping your site up Lucidchart’s overall database costs decreased 25% after switching to Aurora Aurora unknowns: What is an I/Op in Aurora? When you write one piece of data, does it count as six I/Ops? Multi-master Aurora is coming for failover time and disaster recovery purposes Aurora drawbacks: No dedicated DevOps, increased failover time, and misleading performance speed Providers offer ways to simplify your business processes, but not ways to get out of using their products due to vendor and platform lock-in Lucidchart is skeptical about Aurora Serverless; will use or not depending on performance Links: Corey's architecture diagram on AWS Lucidchart Lucidchart’s Data Migration to Amazon Aurora Preview of Amazon Aurora Multi-master Sign Up This is My Architecture re:Invent Digital Ocean

Andy Jassy, CEO of Amazon Web Services, delivers his AWS re:Invent 2017 keynote, featuring the latest news and announcements, including the launches of Amazon Elastic Containers for Kubernetes (EKS), AWS Fargate, Aurora Multi-Master, Aurora Serverless, DynamoDB Global Tables, Amazon Neptune, S3 Select, Amazon Sagemaker, AWS DeepLens, Amazon Rekognition Video, Amazon Kinesis Video Streams, Amazon Transcribe, Amazon Translate, Amazon Comprehend, AWS IoT 1-Click, AWS IoT Device Management, AWS IoT Device Defender, AWS IoT Analytics, Amazon FreeRTOS, and Greengrass ML Inference. Guest speakers include Dr. Matt Wood, of AWS; Roy Joseph, of Goldman Sachs; Mark Okerstrom, of Expedia; and Michelle McKenna-Doyle, of the NFL.

There’s no clever title this week, just straight to the point of covering the highlights of AWS re:Invent this week. They got the kubernetes now! There’s a passel of releases as well. We also discuss some other news like Meg Whitman leaving HPE (on good standing), net neutrality, WeWork buying Meetup, and Arby’s. For reals! Pre-Roll SDT News SDT got a new logo! SDT got 1,000 logo stickers to give away! You can get a sticker but completing this survey (https://www.surveymonkey.com/r/SSCKN86) or sending us your address in Slack. US Addresses only until Matt can come and get some stickers. We’ll be doing a live show - probably - on Jan 16 at the CloudAustin Meetup (https://www.meetup.com/CloudAustin/events/244102686/). Check out the Software Defined Talk Members Only White-Paper Ex (https://www.patreon.com/sdt)e (https://www.patreon.com/sdt)g (https://www.patreon.com/sdt)esi (https://www.patreon.com/sdt)s (https://www.patreon.com/sdt) podcast Join us all in the SDT Slack (http://www.softwaredefinedtalk.com/slack). Upcoming SDT newsletter (http://eepurl.com/dbM2_X). Misc. news before re:Invent coverage Changing of the guard at HPE (https://www.theregister.co.uk/2017/11/21/hpe_meg_whitman/). WeWork buys MeetUp (https://www.wired.com/story/why-wework-is-buying-meetup/). Net Neutrality (https://www.wired.com/story/heres-how-the-end-of-net-neutrality-will-change-the-internet/) - I realize this is naive, but I feel like things already operate this way. EFF write-up (https://www.eff.org/deeplinks/2017/11/lump-coal-internets-stocking-fcc-poised-gut-net-neutrality-rules) Stratechery (https://stratechery.com/2017/light-touch-cable-and-dsl-the-broadband-tradeoff-the-importance-of-antitrust/) & follow-up (https://stratechery.com/2017/light-touch-cable-and-dsl-the-broadband-tradeoff-the-importance-of-antitrust/) This week in PE: OOOHH-OOOO! BARRACUDA (https://www.theregister.co.uk/2017/11/27/barracuda_private_equity/)! Also, Arby’s: eat all you want you’ll die anyway (https://www.cnbc.com/2017/11/28/roark-capital-to-buy-buffalo-wild-wings-for-2-point-9-billion.html). Work in tech? Time to ask for a raise. (https://www.wsj.com/articles/tech-boom-creates-new-order-for-world-markets-1511260200) Good overview (http://www.computerweekly.com/feature/Redefining-OpenStack-Addressing-the-identity-and-integration-for-enterprise-readiness) of the end of OpenStack’s big tent theory. AWS re:Invent AWS Business Update Amazon Web Services has an $18 billion revenue (https://www.channele2e.com/news/live-blog-amazon-web-services-ceo-andy-jassy/) run rate and the business is growing 42 percent year over year New AWS Services (100+ new total) Loosely break into themes of Containers, Databases, AI/ML, and IOT Amazon MQ (https://aws.amazon.com/blogs/aws/amazon-mq-managed-message-broker-service-for-activemq/) - Apache ActiveMQ as a Service (lunches eaten?) AppSync (https://aws.amazon.com/blogs/aws/introducing-amazon-appsync/) - GraphQL as a Service (lunches eaten?) Aurora Serverless (https://aws.amazon.com/blogs/aws/in-the-works-amazon-aurora-serverless/) - burst database consumption Comprehend (https://aws.amazon.com/blogs/aws/amazon-comprehend-continuously-trained-natural-language-processing/) - Natural Language Processing across 98 languages DeepLens (https://aws.amazon.com/blogs/aws/deeplens/) - video camera with AI embedded DynamoDB Global (https://aws.amazon.com/blogs/aws/new-for-amazon-dynamodb-global-tables-and-on-demand-backup/) - similar to Azure/Google initiatives EC2 Bare Metal Instances (https://aws.amazon.com/blogs/aws/new-amazon-ec2-bare-metal-instances-with-direct-access-to-hardware/) - lots of competitors try to differentiate on this (lunches eaten?) came out of the VMware work i3.metal instance types c5 AMIs can work too (new KVM-based instance type) EC2 Instance types, up to 25Gbps networking H1 (https://aws.amazon.com/blogs/aws/new-h1-instances-fast-dense-storage-for-big-data-applications/) - higher throughput to storage, replacing D2 instances M5 (https://aws.amazon.com/blogs/aws/m5-the-next-generation-of-general-purpose-ec2-instances/) - 1.15Gbps write to storage, encrypted at rest, multipurpose instances, new Nitro hypervisor Deep dive on EC2 virtualization/bare metal (http://www.brendangregg.com/blog/2017-11-29/aws-ec2-virtualization-2017.html) T2 Unlimited (https://aws.amazon.com/blogs/aws/new-t2-unlimited-going-beyond-the-burst-with-high-performance/) - good for microservices, bursty workloads with a credit system Elastic Container Service for Kubernetes (https://aws.amazon.com/blogs/aws/amazon-elastic-container-service-for-kubernetes/) (EKS) - called it! upstream K8s automatically runs K8s with three masters across three AZs monitoring/healthchecks built in, managed service Fargate (https://aws.amazon.com/blogs/aws/aws-fargate/) - Containers on demand, no host/orchestrator needed similar to Azure Container Instances apparently Google has App Engine Flexible which is similar (thanks JP!) So, Matt: why would I use EKS instead of Fargate, etc.? Another write-up (https://www.enterprisetech.com/2017/11/30/kubernetes-momentum-builds-new-aws-tools/). FreeRTOS (https://aws.amazon.com/freertos/) - AWS bought(?) existing open source (https://www.freertos.org) IoT operating system vendor Glacier/S3 Select (https://aws.amazon.com/blogs/aws/s3-glacier-select/) - run SQL-like queries against your buckets and storage (CSV & JSON) GuardDuty (https://aws.amazon.com/blogs/aws/amazon-guardduty-continuous-security-monitoring-threat-detection/) - continuous security monitoring & threat detection (lunches eaten?) IoT Analytics (https://aws.amazon.com/blogs/aws/launch-presenting-aws-iot-analytics/) - MQTT processing, reporting & storage IoT Device Defender (https://aws.amazon.com/blogs/aws/in-the-works-aws-sepio-secure-your-iot-fleet/) - reporting, alerting & mitigation of existing IoT fleets IoT Device Management (https://aws.amazon.com/blogs/aws/aws-iot-device-management/) - lifecycle, management & monitoring of IoT devices Kinesis Video Streams (https://aws.amazon.com/blogs/aws/amazon-kinesis-video-streams-serverless-video-ingestion-and-storage-for-vision-enabled-apps/) - video ingestion/processing service Media Services (https://aws.amazon.com/blogs/aws/aws-media-services-process-store-and-monetize-cloud-based-video/) - YouTube as a Service, including monetization. Seems there should be an embeddable player somewhere. Neptune (https://aws.amazon.com/blogs/aws/amazon-neptune-a-fully-managed-graph-database-service/) - managed graph database service (lunches eaten?) Rekognition Video (https://aws.amazon.com/blogs/aws/launch-welcoming-amazon-rekognition-video-service/) - Rekognition now does video SageMaker (https://aws.amazon.com/blogs/aws/sagemaker/) - framework for building AI services Sumerian (https://aws.amazon.com/blogs/aws/launch-presenting-amazon-sumerian/) - VR/AR/3D IDE and platform? Systems Manager (https://aws.amazon.com/blogs/aws/aws-systems-manager/) - custom dashboards based off of tags, ties into AWS system management tools Time Sync Service (https://aws.amazon.com/blogs/aws/keeping-time-with-amazon-time-sync-service/) - AWS NTP Translate (https://aws.amazon.com/blogs/aws/introducing-amazon-translate-real-time-text-language-translation/) - Google & MS already have this Transcribe (https://aws.amazon.com/blogs/aws/amazon-transcribe-scalable-and-accurate-automatic-speech-recognition/) - speech recognition, we should use this! More: The New Stack (https://thenewstack.io/aws-takes-kubernetes-offers-serverless-database-service/), The Register (https://www.theregister.co.uk/2017/11/29/amazon_aws_kubernetes/). This kind of over-the-top analysis (https://blog.codeship.com/aws-reinvent-a-musical-review-of-the-2017-keynote/) is kinda our thing (https://www.patreon.com/sdt). BACK OFF, MAN! AWS Strategy Update On Hybrid Cloud: “In the fullness of time — I don’t know if it’s five, 10 or 15 years out — relatively few companies will own their own data centers. Those that do will have a much smaller footprint. It will be a transition and it won’t happen overnight.” Link (https://www.channele2e.com/news/live-blog-amazon-web-services-ceo-andy-jassy/) More: ‘Is Multi-Cloud Real?: “We certainly get asked about it a lot. Most enterprises, when they think about a plan for moving to the cloud, they think they will distribute workloads across a couple of cloud providers. But few actually make that decision because you have to standardize on lowest common denominator when you go multi-cloud. AWS is so far ahead and you don’t want to handicap developer teams. Asking developers to be fluent in multiple cloud platforms is a lot. And all the cloud providers have volume discounts. If you split workloads across multi-cloud, you’re diminishing those discounts. In practice, companies pick a predominate cloud provider for their workloads. And they may have a secondary cloud provider just in case they want to switch providers.’ AWS re:Invent Preview Review ✔SaaS lunches will be eaten? ✔Amazon Kubernetes Service? This Week in Kubernetes All about AWS this week! Well, GKS did get rid of billing for cluster managers Coté finished up this pile of crap (get a preview!) (https://docs.google.com/document/d/13JaEeN3Vww_Lu5FTgFgArl16HUQ2Oo4lIR1ua_7zZU4/edit?usp=sharing) and right after emailing it in was reminded that Ben wrote this up already, (https://stratechery.com/2016/how-google-cloud-platform-is-challenging-aws/) plus an update based on re:Invent this week (https://stratechery.com/2017/aws-fargate-and-kubernetes-support-embrace-and-extend-awss-execution-advantage/). End-roll Conferences Coté’s junk: NEXT WEEK, FOOLS! SpringOne Platform registration open (https://2017.springoneplatform.io/ehome/s1p/registration), Dec 4th to 5th. Use the code S1P200_Cote for $200 off registration (https://2017.springoneplatform.io/ehome/s1p/registration). Coté and many others speaking. Coté will be doing a tiny talk at CloudAustin on December 19th (https://www.meetup.com/CloudAustin/events/244459662/). Matt’s (not) on the Road! Taking it off for the Holidays. Recommendations Matt Ray: Art of War, backlaid by Wu Tang Clan (https://www.youtube.com/watch?v=qCk7ozsr428) Brandon: Hindenburg audio editor (https://hindenburg.com/). Coté: Programmed Inequality (http://amzn.to/2Aj4StV); drink after the kids go to bed; Mindhunter (https://www.netflix.com/title/80114855); Jim and Andy (https://www.netflix.com/title/80209608).