Podcasts about yan cui

Have you ever wished you could quit your job and go create content all day long while working for yourself? You can! In this episode, Yan Cui joins Allen Helton to talk about full-time content creation and consultancy within the serverless world. The two discuss Yan's journey to greatness, weigh in on the use of generative AI when it comes to content, and cover tips and tricks to get better engagement. About Yan Yan is an AWS Serverless Hero offering training and consulting on AWS and serverless applications. With experience running production workloads at scale in AWS since 2010, Yan has a proven track record of leveraging technology to enhance business outcomes. He has served as an architect and principal engineer across various industries, including banking, e-commerce, social networks, live streaming, and mobile gaming. Yan has worked on large-scale systems handling millions of concurrent users and processing billions of events daily. He runs the site "The Burning Monk", where he shares serverless tips, insights, and best practices, and hosts the podcast Real World Serverless, featuring discussions with practitioners from around the globe. Links Twitter - https://x.com/theburningmonk LinkedIn - https://www.linkedin.com/in/theburningmonk/ The Burning Monk - https://theburningmonk.com Production Ready Serverless Course - http://productionreadyserverless.com/ LLRT Deep Dive - https://www.youtube.com/watch?v=_K4ABY60_oo --- Send in a voice message: https://podcasters.spotify.com/pod/show/readysetcloud/message Support this podcast: https://podcasters.spotify.com/pod/show/readysetcloud/support

Chatting with Founder of THEBURNINGMONK Limited, Developer Advocate at Lumigo, Host at Real-World Serverless podcast, Writer at Master Serverless Newsletter, AWS Serverless Hero, Public Speaker- Yan Cui from Amsterdam, North Holland, Netherlands- Yan Cui said about his work & answered some of my questions. more info at https://smartcherrysthoughts.com

In this episode, we discuss best practices for working with AWS Lambda. We cover how Lambda functions work under the hood, including cold starts and warm starts. We then explore different invocation types - synchronous, asynchronous, and event-based. For each, we share tips on performance, cost optimization, and monitoring. Other topics include function structure, logging, instrumentation, and security. Throughout the episode, we aim to provide a solid mental model for serverless development and share our experiences to help you build efficient and robust Lambda applications.

years after following him for his serverless content, i was glad to be on Yan Cui's pod to talk Rise of AI Engineer.https://realworldserverless.com/episode/92i

Welcome to the epic tale of AWS Bites! In this chapter, we embark on a perilous journey through the challenges of developing distributed applications on AWS. We encounter fierce foes in the form of deployment times and limited access to real AWS services during local development. But fear not, for we have powerful tools at our disposal, including the legendary LocalStack and Serverless offline. And if that's not enough, we have tips and tricks for optimizing our development flow without local simulations, using well-structured code and unit tests. We even share CloudFormation tricks to speed up deployment times and reveal the secret of speeding up the development of IAM policies with Session Policies. So grab your swords and join us on this epic adventure to overcome the challenges of local development on AWS!

In this episode I talk to Yan Cui, who is an AWS Serverless Hero, all about Serverless technologies. Chapters: 00:00 Serverless Architecture with Yan Cui 01:58 What do we mean by Serverless Architecture? 05:42 What is the core problem Serverless solves? 11:06 Do we need to think differently to be able to use Serverless? 15:27 What is the difference between serverless and managed services? 19:17 Is Vendor Lock-in really a problem? 27:42 Multicloud - Is it really worth it? 33:46 Is ColdStart a real problem? What kind of apps get impacted? 43:25 Monitoring serverless applications 48:22 Usecases when serverless may not be the best solution 54:27 Future of serverless 57:31 How should a developer learn about serverless? I hope you enjoy the discussion and learn from it. Please hit the like button, share it with your network and also subscribe to the channel. References: Yan Cui - https://theburningmonk.com Courses - https://productionreadyserverless.com/ Corey Quinn on MultiCloud -    • Corey Quinn: The ...   Linkedin Yan - https://www.linkedin.com/in/theburnin... Twitter Yan - https://twitter.com/theburningmonk Other playlists to watch: Distributed Systems and Databases -    • Distributed Syste...   Software Engineering -    • Software Engineering   Distributed systems practices -    • Distributed Systems   Cheers, The GeekNarrator

I left in a bit more of the pre-show than usual, let me know if you enjoyed that part.  AWS Serverless hero (man that's a cool title

In this episode of the AWS Bites podcast, we explore the best practices for creating and configuring S3 Buckets, Amazon Web Services' popular object storage service. We will learn how to set up buckets correctly from the start, avoiding common pitfalls and ensuring efficient management. We provide a quick recap of Amazon S3, covering buckets, objects, and various use cases. Discover the importance of globally unique bucket names, versioning, and observability through logging and metrics. We will tell you how to ensure the security of your buckets with encryption options and proper access controls. Finally, we discuss S3 integrations and additional settings you might consider for your workload. Don't miss this insightful episode packed with practical tips and expert advice. Tune in now and optimize your S3 Bucket setup for success!

Alex Debrie and Bob Haffner recap their favorite announcements from 2022 re:Invent #data #dataengineering #aws Connect with Alex Twitter: @alexbdebrie Blog: alexdebrie.com Book: dynamodbbook.com Connect with Bob Twitter - @bobhaffner LinkedIn - linkedin.com/in/bobhaffner Top Announcements https://aws.amazon.com/blogs/aws/top-announcements-of-aws-reinvent-2022/ Yan Cui's Choreography vs Orchestration blog post https://theburningmonk.com/2020/08/choreography-vs-orchestration-in-the-land-of-serverless/

There are lots of options for programming languages on AWS these days but one of the most popular ones remains JavaScript. In this episode of AWS Bites we discuss what it's like to develop with JavaScript, Node.js and TypeScript on AWS and what's new in this field. We explore why you would choose JavaScript and what are the trade-offs that come with this choice. We present some of the main features of the all-new AWS SDK v3 for JavaScript. We discuss runtime support and tooling for AWS Lambda and finally some interesting developments in the JavaScript ecosystem for the cloud and AWS. - Our previous episode on What language to use for lambda: https://www.youtube.com/watch?v=S0tpReRa6m4 - AI as a Service by Eoin Shanaghy and Peter Elger (book): https://www.manning.com/books/ai-as-a-service - Node.js Design Patterns by Mario Casciaro and Luciano Mammino (book): https://www.nodejsdesignpatterns.com/ - AWS SDK for JavaScript v3 high level concepts (including command based model): https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/index.html#high-level-concepts - AWS SDK for JavaScript v3 paginators using Async Iterators: https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/index.html#paginators - Mocking support for the AWS SDK for JavaScript v3: https://aws.amazon.com/blogs/developer/mocking-modular-aws-sdk-for-javascript-v3-in-unit-tests/ - Various interesting benchmarks on different Lambda runtimes: https://github.com/theam/aws-lambda-benchmark - https://filia-aleks.medium.com/benchmarking-all-aws-lambda-runtimes-in-2021-cold-start-part-1-e4146fe89385 - https://www.simform.com/blog/aws-lambda-performance/ - Support for ESM modules in AWS Lambda (Node.js 14): https://aws.amazon.com/about-aws/whats-new/2022/01/aws-lambda-es-modules-top-level-await-node-js-14/ - The Middy Framework (middleware pattern for AWS Lambda): https://middy.js.org/ - Lambda Power Tools library for TypeScript: https://awslabs.github.io/aws-lambda-powertools-typescript/ - Yan Cui's article on performance improvements with bundling: https://lumigo.io/blog/3-major-ways-to-improve-aws-lambda-performance/ - ZX project (scripting with JavaScript) by Google: https://github.com/google/zx Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on Twitter: - https://twitter.com/eoins - https://twitter.com/loige

AWS Lambda just got a big upgrade in ephemeral storage: you can now have up to 10 GB of storage for your /tmp folder! Before this was limited to “only” 512 Mb… But is this really useful? What can we do now that we couldn't do before? Also, is this going to have an impact on price? And how does it compare with other storage capabilities that are available in Lambda? Eoin and Luciano are on the case to try to find some answers to these compelling questions, for the greater serverless good! In this episode, we mentioned the following resources: - Official AWS announcement blog post: https://aws.amazon.com/about-aws/whats-new/2022/03/aws-lambda-configure-ephemeral-storage/ - Will Dady on Twitter about 10GB of ephemeral storage now enabling interesting CI/CD use cases: https://twitter.com/WillDady/status/1507110176209322018 - Yan Cui's post on Lumigo's blog “Welcome to 10GB of tmp storage with Lambda”: https://lumigo.io/blog/welcome-to-10gb-of-tmp-storage-with-lambda/ - Lambda cost comparison with ephemeral storage spreadsheet: https://docs.google.com/spreadsheets/d/1_oGjLN0BeRR8CWfgdjeYiIknRTugdmJOhGkAjLCTStw/edit?usp=sharing This episode is also available on YouTube: https://www.youtube.com/AWSBites You can listen to AWS Bites wherever you get your podcasts: - Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017 - Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q - Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw== - Breaker: https://www.breaker.audio/aws-bites - RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on Twitter: - https://twitter.com/eoins - https://twitter.com/loige

In this episode of AWS Bites podcast, Luciano and Eoin talk about some ideas to stay update with the evergreen world of AWS. This is a much less structured episode than usual and we informally discuss some of our favourite resources including blogs, articles, YouTube channels, other podcasts and Twitter profiles. In this episode we mentioned the following resources: - AWS what's new (official): https://aws.amazon.com/new/ - AWS Podcast ​​https://aws.amazon.com/podcasts/aws-podcast/ - Werner Vogel's blog: https://www.allthingsdistributed.com/ - Jeremy Daly's Off by None Newsletter: https://offbynone.io/ - Cloudonaut's blog: https://cloudonaut.io/ - Cloudonaut's YouTube channel: https://www.youtube.com/cloudonaut - Yan Cui's blog: https://theburningmonk.com/ - Yan Cui's Real World Serverless: https://realworldserverless.com/ - AWS Builder Library: https://aws.amazon.com/builders-library/ - Borislav Hadzhiev's blog: https://bobbyhadz.com/ - Marcia Villalba's Foobar on YouTube: https://www.youtube.com/c/FooBar_codes - Adam Elmore's AWS.fm podcast https://aws.fm/ - Emily Shea on Twitter: https://twitter.com/em__shea - Emily Shea's re:Invent 2021 “Getting started building your first serverless web application”: https://www.youtube.com/watch?v=DdyhdnWVukc - Danilo Poccia on Twitter: https://twitter.com/danilop - Heitor Lessa on Twitter: https://twitter.com/heitor_lessa This episode is also available on YouTube: https://www.youtube.com/AWSBites You can listen to AWS Bites wherever you get your podcasts: - Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017 - Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q - Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw== - Breaker: https://www.breaker.audio/aws-bites - RSS: ​​https://anchor.fm/s/6a3312a0/podcast/rss Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on Twitter: - https://twitter.com/eoins - https://twitter.com/loige

Video: https://youtu.be/Cxaf8E00GMMSlides: https://docs.google.com/presentation/d/1sJSqNy-t-kVxzrWlqMTp_03nI7Zo8Znr7k0f0C6L9ig/edit?usp=sharingTimestamps:[00:00:00] Intro[00:02:17] Part 1 - Components: Code Organization for Real Apps [00:04:26] What we learned from React [00:07:46] Part 2 - Architecture: Choreography vs Orchestration [00:13:05] Retries and Timeouts [00:14:37] Part 3 - Time: React vs Temporal[00:16:34] Elevator Pitch [00:17:13] Programming Model [00:18:44] Comparing React and Temporal Principles [00:19:11] Live Demo: Amazon One Click Button [00:23:49] Talk Recap [00:24:16] React and Temporal Full Comparison [00:24:42] Conclusion: EnablementTranscript [00:00:00] Once again, I want to thank you all for tuning in and joining, React New York 2021 without further ado, I'll pass it on to Shawn. All right, so hi everyone. Hello, React new York. It is my home town in the U S and I miss everyone back in New York. I am currently based in Seattle, but I'm here to talk about React for the Backend. In 2020 I actually thought that I had given my last React talk because I was all tapped out. I had said everything I wanted to say, and then React New York came by and said, do you want to speak? And I was like, oh, I really wanted to speak for React New York. So here's my presentation about what I've been working on and what I think the parallels have been for React. And I think there's some generalizable lessons, even if you don't end up using Temporal. So, the inspiration for this talk came from Guillermo Rauch, the creator of Next.js. And he was the first person to point out that Temporal.io, does to backend and infra what React did to frontend. Temporal engine is quite complex, much like React, but the surface exposed to developers a beautiful render function and I'm a bit upset because he realized there's before me and I have been working on Temporal for a few months now. So important caveats before I start this talk. What I'm presenting to you is alpha for TypeScript. Temporal is typically a goal or Java based application, but we're developing TypeScript and hopefully launching it soon. And then finally "React for the backend" is an analogy, not a design goal.  The way I treat this is like, it's a, it's basically like crabs. And one of the most entertaining facts that I've ever found is that nature has apparently tried to evolve crabs five independent times. And in fact, there's a word in evolutionary biology for it called Carcinization. And of course, this is really good for a lot of memes. So tired convergent evolution is not uncommon, especially when species have similar selecting pressures in their environments, wired. Everything is Crab. And perhaps everything is React, because we have similar design space problems. So I'll tell a little bit of the story through three parts there's Components, and we'll tell it through the story of Uber, talk about architecture, we'll talk through the story of YouTube, and Time will tell you through the story of Amazon. So a lot to cover, I'm going to try to go really fast. Don't worry. I'll share the slides on my Twitter later on. Okay.  [00:02:17] Part 1 - Components: Code Organization for Real Apps  So part one is about components. You see this a lot on YouTube. Probably you're watching now on YouTube or live streaming. And yeah, you know, like three hour live stream and that's it. Very cool. I think we, we know how to break things down and React has really helped us be more productive by being able to break things down into the components and knowing how to compose them together in a predictable way. But there's a lot of things unanswered in things like this in, in full stack, clones of major well-known apps, which is the hard parts. What like a typical Uber trip, we'll have all these steps like search pricing match. Pick-up drop-off rating tipping, payment, email, uh, and so on and so forth. And typically the naive way of organizing all this is basically one after the other, right? Like this is search goes to pricing, goes to matching, goes to pick upgoes to dropoff goes to rating goes to tipping goes to payment, goes to email, imagine that these are all managed by separate teams and scaled independently. Then you realize, like, this is only the happy path. Then you have to throw in a whole bunch of things that can happen along the way. An Uber trip is basically a long running process with humans in the loop and humans are very, very messy by nature. So how would you write an Uber clone? good luck with a lot of data technologies that you would typically reach for just naively, because you will have to discover all these systems and all these use cases and edge cases along the way. So when people say full stack, they often really mean like this half drawn horse meme. I think this is particularly funny so I take every opportunity I can get to show it, but to be honest, a lot of us front end developers are probably the other way the half-drawn Dragon where we're frontend a very good and in the backend, we'll just like, you know, stick some stuff on Firebase and something. And in reality, if you look at the backend systems, most companies, especially at scale, go towards some form of very complex micro service, system. I don't have the chart for Uber, but Hail-0 is probably a good comparison. Netflix, Twitter, and It's not really avoidable. If you want to scale a company to any significant size, you probably have to break them up into independent services because you're going to ship your org chart anyway. [00:04:26] What we learned from React  The thing I realized as a React developer, as a front end developer, is that actually we had a pretty good run in the past seven, eight years of React in terms of the fact that front end developers know how to organize code at least in terms of the component level. So we moved from the jQuery era where everything was just kind of spaghetti all over the place to at least something more organized where event handlers are strongly tied, locally tied with renders, but essentially managed by React's runtime.  So a few key lessons from React that I personally draw [00:05:00] is that you want to have a component and a renderer model. Like, so essentially the user or the developer writes components. And then the react core team writes the render and that handles a lot of the boilerplate that you might typically forget. And this is everything to do with on mounting or having local states. And it gives you a very nice, non-leaky abstraction that you can write. Second, you can also guarantee work and correctness, which is originally what drew Jordan walk to make something like React because he was working on Facebook messenger and there was a lot of inconsistent state within Facebook manager because of the spaghetti code. So correctness, meaning that we embrace functional programming to produce a virtual DOM view is a pure function of state. If you look at the old enough React talks, you will see a lot of v = f(d), so view as a pure function of data. And finally the programming model. We like to say that it's just JavaScript. There's no custom syntax with templating syntax to learn. I think all these three lessons , there are actually a lot more, but all of these three lessons are where I'm going to focus on for this talk. And I think whenever you tackle any programming paradigm, any framework, any design question, you might want to run it through some of these ideas. So whenever I talk about React principles, I always like to bring up the fact that there's this often overlooked repo called react-basic. And it's actually in the official React organization on GitHub. And this is Sebastian Markbage, who is the tech lead of React. And he wrote down six years ago, his principles on what he thinks makes up React on a fundamental basis? No, JSX just like, what are the principles that we're designing for? We are designing for a simple, pure transformation, abstraction, composition, state, memoization. The words that he uses are very theoretical sometimes, but you feel it every single day when you write React. So there's a lot of things else apart from that, that reacts has done for front end programming. Apart from deterministic renders, we have useState with a reduction of boiler plate with unmounting, child components in con in the very careful design composition, um, side effects where, you know, we have used effect or use memo. And actually a lot of people don't know. I don't, I forget my source. I think it's Sophie Alpert, but, one-third of the React code base is actually just normalization of events across browsers. So you don't even have to worry about it. And creating synthetic events for that. They also produce a dev tool and manage a central scheduler and obviously the success of React over the past five, six years has really shown Testament to how great all these decisions have been.  If you want to learn more about the talks that I've done and my perspectives on some of these React principles, I've done three talks. One is at React Rally. The second at JSConf and the third, ,at React Summit. So you can check out my YouTube for more conversations on that. I don't have time here. Okay.  [00:07:46] Part 2 - Architecture: Choreography vs Orchestration  So that was part one where we talked about Components and the React revolution. So part two, we're going to talk about architecture. So a bit, one level higher than just components. And I'm going to motivate this question with a question of how would you write YouTube. And again, if you look on YouTube for how to write YouTube tutorials, you can get full-stack clones of YouTube, which is pretty impressive, you know, write YouTube in three hours using Firebase. That's very impressive. Unfortunately the hard parts of YouTube also come in. And there are a bunch of Googlers actually who actually went and interviewed YouTube engineers on how YouTube works on the backend. There's a bunch of work that goes on in the background. So you need to upload your file. You need to analyze for metadata. You need to split it up into chunks. You need to process these chunks in parallel, and then you need to stitch it back. And by the way, processing, you have to produce an array of formats, right? From like, 240 P to like 1400 P or something like that. And then you have to stitch all these chunks back into the continuous videos that you actually see in stream. You need to notify subscribers, you need to produce automatic captions and you need to produce thumbnails. And that is again, just the happy path. Right. So, what about all these other features? It's. For example, YouTube premiere is a scheduled release of a YouTube video or feeding into the recommendation algorithm. That must be the most craziest batch job in the world. And you need to scale this process, whatever, whatever you design for 30,000 hours of video uploaded every hour. That's the sheer amount of volume that's going on YouTube today, which is just insane. Like, like any design that you make at scale is going to break in some respect. So I think, I think that's, that's really interesting to consider. And I learned about this actually, and I thought more about this because I interviewed one of our users who is Descript (hi! I'm editing this transcript in Descript rn lol). Descript is a audio transcription platform and their entire business is transcribing audio and then making it easy for you to edit audio. I do it for my podcast every single day and millions of people use it. I think it's really cool.  So their problem was that when a user hits transcribe, it kicks off asynchronous multi-stage and parallelized process that involves reading, encoding audio, chunk splitting. external API calls, merging results that may potentially arrive out of order and then verifying their alignment. So there's a lot of [00:10:00] nuance here that can get really tricky. And if any part of the process fails, you need to try it again. So, this is typically the kind of architectures that people build up incrementally over time, as they discover all these use cases and then find holes and patch them because it's too late to rewrite something. There's a lot of decisions that goes into here. And this is normal. This is natural. I think you run into basically the eight fallacies of distributed computing, which has actually discussed or discovered back in 1994 by people at Sun Microsystems. I love these cartoons but it can be a little bit hard to read. So here's a more organized version of them. At the bare minimum, don't forget distributed computing fallacy number one, which is that the network may or may not be reliable or compute may or may not be reliable. So, what that means in practice is that when you're calling system a, B, C, D E F G you may actually need to introduce hardening layers because at every point and you cross system boundaries, you have a chance of failure and that multiplies exponentially, as you have more and more services tied up in your systems like we saw for the Uber example, like we saw for the YouTube example. You need to add in timeouts and retries. And what that means is that you need to persist the number of times you timed out, when you timed out, what jobs you timed up. So you need a database every single time, and then you need a scheduler or a timer to say when the next time is going out, I'm going to try this again. And you need to write this for every service. If the ma the maintainer for every service needs to maintain both the code and the infrastructure for this. This is a lot of how I was talking about things when I was exploring the serverless world. So here's a real life example from the AWS blog where they said that you were using dead letter queues to replay messages when such things as failures occur. This is a fine looking example until you try to scale it. And again it looks like a complete mess, complete track, and it's very hard to keep in your head, and pretty soon when you're explaining this to your CTO you look like the Pepe Silvia meme.  So the solution that I found is really to have a central orchestrator, right? Instead of every single system maintainer writing their own API hardening layer, which is a production requirements, as you find more and more of these bugs, you should centralize it with a centralized team that takes care of the orchestration of all these different services. And that's in the business, what we call choreography, which is A to B to C versus orchestration, which is a central orchestrator coordinating the dance between AB and C, and then storing both the infrastructure and the code for the scheduler and the database. So there's a really good article on this by Yan Cui in the burningmonk.com so I highly recommend checking it out where he talks about choreography versus orchestration, with real life examples that people use in AWS, but also it's not specific to any cloud. It's a architecture design pattern, which I think fundamentally, if you start off with this, it's really hard to rearchitect to this. I mean, it's, it's possible because people are doing it, but also it's a conscious, architectural choice that you might not know that you're making if you don't know about it. So, I guess a lot of my message here is to tell you that orchestration is a thing.  [00:13:05] Retries and Timeouts  Also, so you want to declaratively put into your framework retries and timeouts, so for example, this is actually our API. You want to be able to say, all right, here's the default retry policy. Whenever I fire off an activity, an activity is just like an external API call, for example. So when I fire up an activity, I want it to be retried every second. If it fails, I need a backoff coefficient, like exponential backoff. This is very similar to the TCP protocol so that if the endpoint is failing or getting rate limited, I don't keep retrying, and then building up a DDoS attack on myself, I actually back off and put more and more intervals in between until some maximum interval, let's say a hundred seconds. And then I give myself a maximum attempt, so I can say like, all right. I don't want any retries. I can just say have a maximum attempt of one. Or let's say, I want a linear back off and not an exponential for whatever reason. And I want to try to a maximum of five times - you want to have this all declarative so that you can tweak this as you understand your system and you scale your system. Right? So I think this is a really interesting programming model that just puts retries into the code that you write. And that's only possible when you have your centralized orchestrator, no matter what system, not just Temporal.  Okay. So the case that I'm making is really for choreography versus orchestration. And I, the analogy that I make for front end versus the back end is that it's kind of like vanilla or jQuery versus react. React has a react as the central orchestrator, orchestrating all the components. And I think that's a really interesting architectural analogy that you can make and learn from React. All right.  [00:14:37] Part 3 - Time  Part three - Time. I'm doing very good on time. I think better than I thought, which means that we'll have time for a live demo, which is really awesome. So let's talk a little bit about Temporal.  [00:14:45] What is Temporal?  What is Temporal? Temporal is the open source platform for orchestrating highly reliable mission-critical applications at scale. I love talking a little bit about the history, the reason because our CEO started at Amazon as the tech lead for what became Amazon SQS. Our [00:15:00] CTO was at Microsoft and it was the principal architect of the Durable Task Framework, which became a Microsoft's version of Durable Functions, and then finally they joined Uber and worked on Cadence, which is the open-source version of their workflow orchestration platform and Cadence became so popular that they spun out and became Temporal. And since then it's been adopted by a lot of well-known household name companies, especially in the developer world. There are a lot of people hiring for Temporal developers, which I really like to see because it's not just being used, but also it's creating jobs for people and it's becoming a desirable skillset. And most recently last week we had Netflix presenting about how they used Temporal for their CI/CD.  Temporal has three components or produces three products that are used in sync. The main star is Temporal server, which is comparable to the React runtime that you might see, then there's Devtools, which is the UI that you might want to inspect the state of things. And then the SDK is, which is what you use to code. So I think all those are really comparable to what we have in React and having been in the React world for while, like, it's really amazing to see the analogies that we have. We have exactly the same thing. For me, the really sort of the seal of approval comes from Mitchell Hashimoto who, created Hashi Corp, saying that without Temporal, we would have spent a significant amount of time rebuilding Temporal, which actually to me is the best form of validation because Mitchell is one of the best developers in distributed systems and he says it's hard and he says it does it well.   All right. Enough social proof you want actual facts? I would just give it straight to you.  [00:16:34] Elevator Pitch  So because your workloads like the YouTube encoding, or like the Uber journey and this technology was developed at Uber i s long running and it ties together multiple services. You want to standardize timeouts and retries and you want to make it easy for every team to have production grade retries and timeouts.  Because this work is so important. You must never drop any work. You must log all progress. In other words, you must use event sourcing. And then finally, because this work is so complex, you want to use generic programming languages, instead of Domain specific languages. So you want to model a dynamic, asynchronous logic, and then you want to reuse, test, version and migrated it.  So that's the pitch in one screen. But I'll just break it down for what it means, and then we'll go into a demo.  [00:17:13] Programming Model  So to me, The, the closest analogy to React is the programming model, because React spends a lot of time on API design and in the workflow orchestration world there are a lot of JSON or DAG based domain specific languages. So you, you write a bunch of JSON or you do boxes and arrows boxes and arrows boxes and arrows, sometimes you've even write XML, which is very interesting as well. What I find with all these is that they're actually really good for manipulating visually. But they get very tricky when you need to do programming language constructs, like variables, functions, loops, branching statements and all the things that we've invented in programming languages over the past few years. So if you use "just JavaScript" or "just programming languages", you have all the tooling available. You can use all the libraries that are available. You can use all the testing and code version, quality controls available. If you write your own, you have to rebuild all this dev tooling from scratch for yourself. So that's essentially what this is. Here's an example from one of the big clouds where this is their workflow orchestrator model, where you write Jason and it's really hard. It actually goes off the screen and I couldn't really fit everything on one screen. And with Temporal literal just JavaScript you call an endpoint you use that the result of that end point to call other end points, for example. It's a very simple example, but in built here is default retry policies that have been worked out. So both of these handle reliability on rails, it's just, we differ in the programming model and the engineering that it takes to maintain one of these SDKs is I'm learning. It's very, very immense. So it's really interesting. [00:18:44] Comparing React and Temporal Principles  So, again, back to the core principles that we talked about early on from React. React d ecided on using a framework, decided on correctness and decided on a programming model, and Temporal, in a very similar way. The developer writes workflows and the Temporal core team writes the orchestrator, which is Temporal server. In terms of correctness, React insists on functional programming, Temporal insists on event sourcing in deterministic workflows and then programming model, you want "just JavaScript" or just programming languages, not any custom DSL syntax.  [00:19:11] Live Demo: Amazon One Click Button  So the final example that I'm gonna motivate is which is like, I'm, I've been trying to re progressively reduce the complexity of my examples. So we met from Uber, which is like a super long running, a lot of humans in the loop to YouTube, which is not so much humans in the loop. You upload it once and then everything else takes over from there.  Now I just want to build one feature, which is a one-click buy button in React or in front end. It's actually super easy. It's a button. That's the literal simplest thing you can possibly do. You put an onclick handler. You're done. If you want to do a one-click buy, you do a setTimeout, and then say like, okay, if you want to cancel this within some window, with Amazon is 30 minutes, we can cancel it. But if you want to persist it, imagine if some person clicks, closes the browser and then changes their mind, opens the browser again, and it's gone. You're screwed. You don't have any other way to implement one-click [00:20:00] purchases. You need to implement timers on the backend to do this. I was watching this old talk from Joel Spolsky where he talks about the engineering for the one-click buy button. And I put it up on my YouTube because this is such an old talk. And I was afraid to link to the timestamp, but you can check it out as it's just a three minute video where he tells the story about how Amazon moved from shopping cart to one-click buy I mean they still have a shopping cart but it's that important because in online e-commerce actually even up to today the abandonment rate for shopping carts is 70%. So imagine if you implement this one feature, you improve your sales by I don't what's the inverse of 70%, three times. That's really amazing. So I think it's just fascinating and it's not just about Amazon. It's not about one click buy. It's about user experience. It's about making things easy and intuitive and that often involves turning synchronous things into asynchronous things and in persisting them so that they persist in the background. So I have a little demo here. I'm going to go really, really fast, but you can check out the code in temporalio/samples-node.  There's the specific path this year but it's basically a Next.js demo where I have a Next.js folder here. This is going to be pretty standard for a lot of React developers. Hopefully you're familiar with Next.js, so you can learn it. It's got some pages and an API routes where I have serverless functions that call and send signals to my workflow functions. I have also a Temporal folder where I have written my workflows and activities. The activities are just a little logs obviously, cause they don't interact with any backends, but they could. And then the workflow coordinates the states in the background of all of these.  So I can show you the code, but essentially I kick off a one-click buy with a purchase and then I set a timer and promise.race with a five second wait. So if I receive a cancel signal during that timer then that cancels if not, it goes through and the purchase is confirmed. Obviously I can. And what's fascinating about Temporal is that every single step is persisted in automatically saved. So in other words, I can sleep for 30 days. I can sleep for a year. I can sleep for five years and it doesn't matter because it's all persisted and wakes up automatically. So the compute the, this serverless function can be. The worker or a Temporal server itself can go down. You can just bring you back up again and it carries on as though nothing happened because of event sourcing. So, I'm gonna, I'm going to go ahead and run this. I think it's uh, demos I'm always stressed out about live demos. Okay. I mean, I did test it before the talk. It's just that whenever I'm streaming, like it adds an extra latency thing and that goes haywire. So, Let's see if I have this demo available. All right. So I also want to pull this out, which is the UI layer. These are the my test runs. But I have here at one-click purchase UI, and literally, I, you know, I, I want to implement this without a shopping cart, but I want to be able to cancel within some certain amount of time. So if I click buy, uh, it clicks, it handles it's. It sends a workflow. And that workflow starts in starts in the background and it's running, right. It's waiting for the timer to proceed. So I'm going to hit the timer, uh, and you can see that a timer started and time of ended, uh, within that five second window that I specified, obviously I should make it longer if I, if I really wanted to show this along the way. Um, so, uh, this, this is, this is as is purchased, um, and we can, uh, and now we've confirmed it. Um, but if I ever want to click buy, and then I can click the. That also fires off a different workflow, uh, where it sees that it receives the cancel signal from me. Uh, so, so I signaled it to cancel. And that's a very useful model as well. So this actually shows off a lot of the core principles of Temporal, which is you kick off a workflow, you can set durable timers, you can send it human signals and you can get out data as well with queries. There's a lot of interesting elements behind that, but that's the core demo that I wanted to show off. So maybe I'll write a YouTube example and then I'll go on to an Uber example and be a billionaire.  [00:23:49] Talk Recap  So ultimately I just want it to recap again, what we covered. We covered components, we covered architecture we covered time, and these are all the three elements I wanted to compare reacts and Temporal, and explain a little bit of how we think about doing the hard parts of making clones of very popular projects. Why is it so interesting? It's a little bit like the crabs story, you know. Obviously the founders of Temporal are not front end developers. They didn't even know react at all.  [00:24:16] React and Temporal Full Comparison  But they independently evolved a lot of the same principles and this that's, I haven't even gone into like the full comparison. So we talked a little bit about deterministic functions and local state and composition, but we haven't talked about normalization and how that compares dev tools. Testing is also super interesting thing as well as the central runtime. So there's a lot here, which I think. And fascinated by, and I'm obsessed by applying the lessons from React to things that are not React. [00:24:42] Conclusion: Enablement And I think overall, when I asked my CEO, like, what is the core message that we want to deliver is actually about enablements. Like we enable people to do things that they're not formally trained to do because we wrapped it up rapid all in a central runtime or central framework. So, uh, I always loved the Alfred north Whitehead quote that [00:25:00] civilization advances by extending the numberof things that we can do without thinking about it.   So for me, my version of it is that B2B software advances by extending the number of jobs we can perform without formal training. And the message overall here is that Temporal lets backend developers or, just general full stack developers do distributed systems right? So that's it.  I blasted through that. I only took 26 minutes. Really great for me, cause I was worried that it would take 50 and I'm happy to answer any questions you can hit me up on Twitter at Swyx. You can read my long form blog posts about why Temporal and then you can join our mailing list, YouTube or Slack. Thank you. Alright, thank you very much things. So I think that was a really, really nice. And you did, uh, went through that quite quickly. Uh, when I see the comments, people love the, like the most right there, because I could fail because I could fail. It's always like that. So, uh, yeah. Um, the nice thank you for the presentation. With this talk, I think it's actually the last talk of the event and I want to thanks everyone for joining us and thanks to everyone, thanks to all the speakers, of course, for being part of this event, uh, React New York 2021 and the sponsors. Um, I think this would be a good afternoon, I guess, or good night, depending on where we are in the world. Right. Have a good one. Everyone. 

Yan joins Adam to discuss their shared love for AppSync & VTL and his experiences as a thought leader and content creator in the serverless space.

Nesse episódio case, falamos com o Ricardo Torres direto de Amsterdam para saber como a New10, uma Fintech europeia, está usando serverless para endereçar suas necessidades de soluções no mercado financeiro. Palestra do Flavio Losada sobre single table com DynamoDB: https://youtu.be/vH2N1wPO7MI Repositório Real World no GitHub: https://github.com/gothinkster/realworld Episódio com Yan Cui (em inglês): https://open.spotify.com/episode/795F0tcr8NEWWNcRf8hKbV?si=uUpFCwzKSGKfXfix9XoJeg

Yan Cui (aka Twitter's The Burning Monk) is goldmine of knowledge when it comes to Serverless Architectures. But what is Serverless? And where did the inspiration for his Twitter handle come from? Aled and Alasdair spoke to him to find out...To Yan going Serverless makes "all the sense in the world" and he has been an AWS Service Hero since 2018. With experience spanning e-commerce, streaming, gaming and social media platforms we had plenty to talk about!What kinds of applications are suited to serverless? How do you get started? And why should you consider adopting a serverless architecture? These questions, and more, answered in this week's podcast!Learn more about Serverless on AWS here.Connect with Yan Cui here, read his blog here and find out more about his Production-Ready Serverless workshops here.About the hostsAled Sage (VP Engineering at Cloudsoft) brings 20 years of experience developing distributed applications, mostly in the enterprise sector. Particular areas of interest include devops, automation, cloud, concurrency and distributed systems. Prior to joining Cloudsoft, Aled was Chief Architect at Enigmatec Corporation. He holds a PhD (Computer Science) from the University of St Andrews.Connect with Aled Alasdair Hodge (Principal Engineer and Solutions Architect at Cloudsoft) is a leadingauthority in cloud, software applications and automation across all major cloud platforms, he has been engaged in the design and optimisation of cloud services in banking and finance and other service-based sectors such as telecoms, electronic design and supply-chain automation for over 12 years.Connect with Alasdair About CloudsoftCloudsoft specialise in applications, automation and the cloud. Our top-down, application-centric approach is what sets us apart and empowers our customers to tame their IT estates and drive real business value.Our public cloud IT transformation expertise is embedded our Tempo cloud services, helping our clients to transform faster and exploit the benefits of cloud. We're an active contributor to the Apache Brooklyn open source project, an AWS Advanced Consulting Partner with Well-Architected, Microsoft Workloads, DevOps Competencies, a FinOps Foundation Member and a FinOps Certified Service Provider.Our flagship AMP software is a Digital Platform Conductor, enabling customers in Banking, Defence and other heavily regulated industries to cut through the complexity of their estates. This maximises value from their technology investments, drives down costs and reduces risks. Cloudsoft AMP is the only product which strategically manages applications throughout their lifecycles across hybrid infrastructures, regardless of environment or location.We're based in Edinburgh, UK, but work with clients around the world. 

About Patrick StrzelecPatrick Strzelec is a fullstack developer with a focus on building GraphQL gateways and serverless microservices. He is currently working as a technical lead at NorthOne making banking effortless for small businesses.LinkedIn: Patrick StrzelecNorthOne Careers: www.northone.com/about/careersWatch this episode on YouTube: https://youtu.be/8W6lRc03QNU  This episode sponsored by CBT Nuggets and Lumigo. TranscriptJeremy: Hi everyone. I'm Jeremy Daly, and this is Serverless Chats. Today, I'm joined by Patrick Strzelec. Hey, Patrick, thanks for joining me.Patrick: Hey, thanks for having me.Jeremy: You are a lead developer at NorthOne. I'd love it if you could tell the listeners a little bit about yourself, your background, and what NorthOne does.Patrick: Yeah, totally. I'm a lead developer here at NorthOne, I've been focusing on building out our GraphQL gateway here, as well as some of our serverless microservices. What NorthOne does, we are a banking experience for small businesses. Effectively, we are a deposit account, with many integrations that act almost like an operating system for small businesses. Basically, we choose the best partners we can to do things like check deposits, just your regular transactions you would do, as well as any insights, and the use cases will grow. I'd like to call us a very tailored banking experience for small businesses.Jeremy: Very nice. The thing that is fascinating, I think about this, is that you have just completely embraced serverless, right?Patrick: Yeah, totally. We started off early on with this vision of being fully event driven, and we started off with a monolith, like a Python Django big monolith, and we've been experimenting with serverless all the way through, and somewhere along the journey, we decided this is the tool for us, and it just totally made sense on the business side, on the tech side. It's been absolutely great.Jeremy: Let's talk about that because this is one of those things where I think you get a business and a business that's a banking platform. You're handling some serious transactions here. You've got a lot of transactions that are going through, and you've totally embraced this. I'd love to have you take the listeners through why you thought it was a good idea, what were the business cases for it? Then we can talk a little bit about the adoption process, and then I know there's a whole bunch of stuff that you did with event driven stuff, which is absolutely fascinating.Then we could probably follow up with maybe a couple of challenges, and some of the issues you face. Why don't we start there. Let's start, like who in your organization, because I am always fascinated to know if somebody in your organization says, “Hey we absolutely need to do serverless," and just starts beating that drum. What was that business and technical case that made your organization swallow that pill?Patrick: Yeah, totally. I think just at a high level we're a user experience company, we want to make sure we offer small businesses the best banking experience possible. We don't want to spend a lot of time on operations, and trying to, and also reliability is incredibly important. If we can offload that burden and move faster, that's what we need to do. When we're talking about who's beating that drum, I would say our VP, Blake, really early on, seemed to see serverless as this amazing fit. I joined about three years ago today, so I guess this is my anniversary at the company. We were just deciding what to build. At the time there was a lot of architecture diagrams, and Blake hypothesized that serverless was a great fit.We had a lot of versions of the world, some with Apache Kafka, and a bunch of microservices going through there. There's other versions with serverless in the mix, and some of the tooling around that, and this other hypothesis that maybe we want GraphQL gateway in the middle of there. It was one of those things that we wanted to test our hypothesis as we go. That ties into this innovation velocity that serverless allows for. It's very cheap to put a new piece of infrastructure up in serverless. Just the other day we wanted to test Kinesis for an event streaming use case, and that was just a half an hour to set up that config, and you could put it live in production and test it out, which is completely awesome.I think that innovation velocity was the hypothesis. We could just try things out really quickly. They don't cost much at all. You only pay for what you use for the most part. We were able to try that out, and as well as reliability. AWS really does a good job of making sure everything's available all the time. Something that maybe a young startup isn't ready to take on. When I joined the company, Blake proposed, “Okay, let's try out GraphQL as a gateway, as a concept. Build me a prototype." In that prototype, there was a really good opportunity to try serverless. They just ... Apollo server launched the serverless package, that was just super easy to deploy.It was a complete no-brainer. We tried it out, we built the case. We just started with this GraphQL gateway running on serverless. AWS Lambda. It's funny because at first, it's like, we're just trying to sell them development. Nobody's going to be hitting our services. It was still a year out from when we were going into production. Once we went into prod, this Lambda's hot all the time, which is interesting. I think the cost case breaks down there because if you're running this thing, think forever, but it was this GraphQL server in front of our Python Django monolift, with this vision of event driven microservices, which has fit well for banking. If you just think about the banking world, everything is pretty much eventually consistent.Just, that's the way the systems are designed. You send out a transaction, it doesn't settle for a while. We were always going to do event driven, but when you're starting out with a team of three developers, you're not going to build this whole microservices environment and everything. We started with that monolith with the GraphQL gateway in front, which scaled pretty nicely, because we were able to sort of, even today we have the same GraphQL gateway. We just changed the services backing it, which was really sweet. The adoption process was like, let's try it out. We tried it out with GraphQL first, and then as we were heading into launch, we had this monolith that we needed to manage. I mean, manually managing AWS resources, it's easier than back in the day when you're managing your own virtual machines and stuff, but it's still not great.We didn't have a lot of time, and there was a lot of last-minute changes we needed to make. A big refactor to our scheduling transactions functions happened right before launch. That was an amazing serverless use case. And there's our second one, where we're like, “Okay, we need to get this live really quickly." We created this work performance pattern really quickly as a test with serverless, and it worked beautifully. We also had another use case come up, which was just a simple phone scheduling service. We just wrapped an API, and just exposed some endpoints, but it was just a lot easier to do with serverless. Just threw it off to two developers, figure out how you do it, and it was ready to be live. And then ...Jeremy: I'm sorry to interrupt you, but I want to get to this point, because you're talking about standing up infrastructure, using infrastructure as code, or the tools you're using. How many developers were working on this thing?Patrick: How many, I think at the time, maybe four developers on backend functionality before launch, when we were just starting out.Jeremy: But you're building a banking platform here, so this is pretty sophisticated. I can imagine another business case for serverless is just the sense that we don't have to hire an operations team.Patrick: Yeah, exactly. We were well through launching it. I think it would have been a couple of months where we were live, or where we hired our first dev ops engineer. Which is incredible. Our VP took a lot of that too, I'm sure he had his hands a little more dirty than he did like early on. But it was just amazing. We were able to manage all that infrastructure, and scale was never a concern. In the early stages, maybe it shouldn't be just yet, but it was just really, really easy.Jeremy: Now you started with four, and I think, what are you now? Somewhere around 25 developers? Somewhere in that space now?Patrick: About 25 developers now, we're growing really fast. We doubled this year during COVID, which is just crazy to think about, and somehow have been scaling somewhat smoothly at least, in terms of just being able to output as a dev team promote. We'll probably double again this year. This is maybe where I shamelessly plug that we're hiring, and we always are, and you could visit northone.com and just check out the careers page, or just hit me up for a warm intro. It's been crazy, and that's one of the things that serverless has helped with us too. We haven't had this scaling bottleneck, which is an operations team. We don't need to hire X operations people for a certain number of developers.Onboarding has been easier. There was one example of during a major project, we hired a developer. He was new to serverless, but just very experienced developer, and he had a production-ready serverless service ready in a month, which was just an insane ramp-up time. I haven't seen that very often. He didn't have to talk to any of our operation staff, and we'd already used serverless long enough that we had all of our presets and boilerplates ready, and permissions locked down, so it was just super easy. It's super empowering just for him to be able to just play around with the different services. Because we hit that point where we've invested enough that every developer when they opened a branch, that branch deploys its own stage, which has all of the services, AWS infrastructure deployed.You might have a PR open that launches an instance of Kinesis, and five SQS queues, and 10 Lambdas, and a bunch of other things, and then tear down almost immediately, and the cost isn't something we really worry about. The innovation velocity there has been really, really good. Just being able to try things out. If you're thinking about something like Kinesis, where it's like a Kafka, that's my understanding, and if you think about the organizational buy-in you need for something like Kafka, because you need to support it, come up with opinions, and all this other stuff, you'll spend weeks trying it out, but for one of our developers, it's like this seems great.We're streaming events, we want this to be real-time. Let's just try it out. This was for our analytics use case, and it's live in production now. It seems to be doing the thing, and we're testing out that use case, and there isn't that roadblock. We could always switch off to a different design if you want. The experimentation piece there has been awesome. We've changed, during major projects we've changed the way we've thought about our resources a few times, and in the end it works out, and often it is about resiliency. It's just jamming queues into places we didn't think about in the first place, but that's been awesome.Jeremy: I'm curious with that, though, with 25 developers ... Kinesis for the most part works pretty well, but you do have to watch those iterator ages, and make sure that they're not backing up, or that you're losing events. If they get flooded or whatever, and also sticking queues everywhere, sounds like a really good idea, and I'm a big fan of that, but it also, that means there's a lot of queues you have to manage, and watch, and set alarms and all that kind of stuff. Then you also talked about a pretty, what sounds like a pretty great CI/CD process to spin up new branches and things like that. There's a lot of dev ops-y ops work that is still there. How are you handling that now? Do you have dedicated ops people, or do you just have your developers looking after that piece of it?Patrick: I would say we have a very spirited group of developers who are inspired. We do a lot of our code-sharing via internal packages. A few of our developers just figured out some of our patterns that we need, whether it's like CI, or how we structure our events stores, or how we do our Q subscriptions. We manage these internal packages. This won't scale well, by the way. This is just us being inspired and trying to reduce some of this burden. It is interesting, I've listened to this podcast and a few others, and this idea of infrastructure as code being part of every developer's toolbox, it's starting to really resonate with our team.In our migration, or our swift shift to full, I'd say doing serverless properly, we've learned to really think in it. Think in terms of infrastructure in our creating solutions. Not saying we're doing serverless the right way now, but we certainly did it the wrong way in the past, where we would spin up a bunch of API gateways that would talk to each other. A lot of REST calls going around the spider web of communication. Also, I'll call these monster Lambdas, that have a whole procedure list that they need to get through, and a lot of points of failure. When we were thinking about the way we're going to do Lambda now, we try to keep one Lambda doing one thing, and then there's pieces of infrastructure stitching that together. EventBridge between domain boundaries, SQS for commands where we can, instead of using API gateway. I think that transitions pretty well into our big break. I'm talking about this as our migration to serverless. I want to talk more about that.Jeremy: Before we jump into that, I just want to ask this question about, because again, I call those fat, some people call them fat Lambdas, I call them Lambda lifts. I think there's Lambda lifts, then fat Lambdas, then your single-purpose functions. It's interesting, again, moving towards that direction, and I think it's super important that just admitting that you're like, we were definitely doing this wrong. Because I think so many companies find that adopting serverless is very much so an evolution, and it's a learning thing where the teams have to figure out what works for them, and in some cases discovering best practices on your own. I think that you've gone through that process, I think is great, so definitely kudos to you for that.Before we get into that adoption and the migration or the evolution process that you went through to get to where you are now, one other business or technical case for serverless, especially with something as complex as banking, I think I still don't understand why I can't transfer personal money or money from my personal TD Bank account to my wife's local checking account, why that's so hard to do. But, it seems like there's a lot of steps. Steps that have to work. You can't get halfway through five steps in some transaction, and then be like, oops we can't go any further. You get to roll that back and things like that. I would imagine orchestration is a huge piece of this as well.Patrick: Yeah, 100%. The banking lends itself really well to these workflows, I'll call them. If you're thinking about even just the start of any banking process, there's this whole application process where you put in all your personal information, you send off a request to your bank, and then now there's this whole waterfall of things that needs to happen. All kinds of checks and making sure people aren't on any fraud lists, or money laundering lists, or even just getting a second dive from our compliance department. There's a lot of steps there, and even just keeping our own systems in sync, with our off-provider and other places. We definitely lean on using step functions a lot. I think they work really, really well for our use case. Just the visual, being able to see this is where a customer is in their onboarding journey, is very, very powerful.Being able to restart at any point of their, or even just giving our compliance team a view into that process, or even adding a pause portion. I think that's one of the biggest wins there, is that we could process somebody through any one of our pipelines, and we may need a human eye there at least for this point in time. That's one of the interesting things about the banking industry is. There are still manual processes behind the scenes, and there are, I find this term funny, but there are wire rooms in banks where there are people reviewing things and all that. There are a lot of workflows that just lend themselves well to step functions. That pausing capability and being able to return later with a response, so that allows you to build other internal applications for your compliance teams and other teams, or just behind the scenes calls back, and says, "Okay, resume this waterfall."I think that was the visualization, especially in an events world when you're talking about like sagas, I guess, we're talking about distributed transactions here in a way, where there's a lot of things happening, and a common pattern now is the saga pattern. You probably don't want to be doing two-phase commits and all this other stuff, but when we're looking at sagas, it's the orchestration you could do or the choreography. Choreography gets very messy because there's a lot of simplistic behavior. I'm a service and I know what I need to do when these events come through, and I know which compensating events I need to dump, and all this other stuff. But now there's a very limited view.If a developer is trying to gain context in a certain domain, and understand the chain of events, although you are decoupled, there's still this extra coupling now, having to understand what's going on in your system, and being able to share it with external stakeholders. Using step functions, that's the I guess the serverless way of doing orchestration. Just being able to share that view. We had this process where we needed to move a lot of accounts to, or a lot of user data to a different system. We were able to just use an orchestrator there as well, just to keep an eye on everything that's going on.We might be paused in migrating, but let's say we're moving over contacts, a transaction list, and one other thing, you could visualize which one of those are in the red, and which one we need to come in and fix, and also share that progress with external stakeholders. Also, it makes for fun launch parties I'd say. It's kind of funny because when developers do their job, you press a button, and everything launches, and there's not really anything to share or show.Jeremy: There's no balloons or anything like that.Patrick: Yeah. But it was kind of cool to look at these like, the customer is going through this branch of the logic. I know it's all green. Then I think one of the coolest things was just the retry ability as well. When somebody does fail, or when one of these workflows fails, you could see exactly which step, you can see the logs, and all that. I think one of the challenges we ran into there though, was because we are working in the banking space, we're dealing with sensitive data. Something I almost wish AWS solved out of the box, would be being able to obfuscate some of that data. Maybe you can't, I'm not sure, but we had to think of patterns for tokenization for instance.Stripe does this a lot where certain parts of their platform, you just get it, you put in personal information, you get back a token, and you use that reference everywhere. We do tokenization, as well as we limit the amount of details flowing through steps in our orchestrators. We'll use an event store with identifiers flowing through, and we'll be doing reads back to that event store in between steps, to do what we need to do. You lose some of that debug-ability, you can't see exactly what information is flowing through, but we need to keep user data safe.Jeremy: Because it's the use case for it. I think that you mentioned a good point about orchestration versus choreography, and I'm a big fan of choreography when it makes sense. But I think one of the hardest lessons you learn when you start building distributed systems is knowing when to use choreography, and knowing when to use orchestration. Certainly in banking, orchestration is super important. Again, with those saga patterns built-in, that's the kind of thing where you can get to a point in the process and you don't even need to do automated rollbacks. You can get to a failure state, and then from there, that can be a pause, and then you can essentially kick off the unwinding of those things and do some of that.I love that idea that the token pattern and using just rehydrating certain steps where you need to. I think that makes a ton of sense. All right. Let's move on to the adoption and the migration process, because I know this is something that really excites you and it should because it is cool. I always know, as you're building out applications and you start to add more capabilities and more functionality and start really embracing serverless as a methodology, then it can get really exciting. Let's take a step back. You had a champion in your organization that was beating the drum like, "Let's try this. This is going to make a lot of sense." You build an Apollo Lambda or a Lambda running Apollo server on it, and you are using that as a strangler pattern, routing all your stuff through now to your backend. What happens next?Patrick: I would say when we needed to build new features, developers just gravitated towards using serverless, it was just easier. We were using TypeScript instead of Python, which we just tend to like as an organization, so it's just easier to hop into TypeScript land, but I think it was just easier to get something live. Now we had all these Lambdas popping up, and doing their job, but I think the problem that happened was we weren't using them properly. Also, there was a lot of difference between each of our serverless setups. We would learn each time and we'd be like, okay, we'll use this parser function here to simplify some of it, because it is very bare-bones if you're just pulling the Serverless Framework, and it took a little ...Every service looked very different, I would say. Also, we never really took the time to sit back and say, “Okay, how do we think about this? How do we use what serverless gives us to enable us, instead of it just being an easy thing to spin up?" I think that's where it started. It was just easy to start. But we didn't embrace it fully. I remember having a conversation at some point with our VP being like, “Hey, how about we just put Express into one of our Lambdas, and we create this," now I know it's a Lambda lift. I was like, it was just easier. Everybody knows how to use Express, why don't we just do this? Why are we writing our own parsers for all these things? We have 10 versions of a make response helper function that was copy-pasted between repos, and we didn't really have a good pattern for sharing that code yet in private packages.We realized that we liked serverless, but we realized we needed to do it better. We started with having a serverless chapter reading between some of our team members, and we made some moves there. We created a shared boilerplate at some point, so it reduced some of the differences you'd see between some of the repositories, but we needed a step-change difference in our thinking, when I look back, and we got lucky that opportunity came up. At this point, we probably had another six Lambda services, maybe more actually. I want to say around, we'd probably have around 15 services at this point, without a governing body around patterns.At this time, we had this interesting opportunity where we found out we're going to be re-platforming. A big announcement we just made last month was that we moved on to a new bank partner called Bancorp. The bank partner that supports Chime, and they're like, I'll call them an engine boost. We put in a much larger, more efficient engine for our small businesses. If you just look at the capabilities they provide, they're just absolutely amazing. It's what we need to build forward. Their events API is amazing as well as just their base banking capabilities, the unit economics they can offer, the times on there, things were just better. We found out we're doing an engine swap. The people on the business side on our company trusted our technical team to do what we needed to do.Obviously, we need to put together a case, but they trusted us to choose our technology, which was awesome. I think we just had a really good track record of delivering, so we had free reign to decide what do we do. But the timeline was tight, so what we decided to do, and this was COVID times too, was a few of our developers got COVID tested, and we rented a house and we did a bubble situation. How in the NHL or MBA you have a bubble. We had a dev bubble.Jeremy: The all-star team.Patrick: The all-star team, yeah. We decided let's sit down, let's figure out what patterns are going to take us forward. How do we make the step-change at the same time as step-change in our technology stack, at the same time as we're swapping out this bank, this engine essentially for the business. In this house, we watched almost every YouTube video you can imagine on event driven and serverless, and I think leading up. I think just knowing that we were going to be doing this, I think all of us independently started prototyping, and watching videos, and reading a lot of your content, and Alex DeBrie and Yan Cui. We all had a lot of ideas already going in.When we all got to this house, we started off with this exercise, an event storming exercise, just popular in the domain-driven design community, where we just threw down our entire business on a wall with sticky notes, and it would have been better to have every business stakeholder there, but luckily we had two people from our product team there as representatives. That's how invested we were in building this outright, that we have products sitting in the room with us to figure it out.We slapped down our entire business on a wall, this took days, and then drew circles around it and iterated on that for a while. Then started looking at what the technology looks like. What are our domain boundaries, and what prototypes do we need to make? For a few weeks there, we were just prototyping. We built out what I'd called baby's first balance. That was the running joke where, how do we get an account opened with a balance, with the transactions minimally, with some new patterns. We really embraced some of this domain-driven-design thinking, as well as just event driven thinking. When we were rethinking architecture, three concepts became very important for us, not entirely new, but important. Item potency was a big one, dealing with distributed transactions was another one of those, as well as the eventual consistency. The eventual consistency portion is kind of funny because we were already doing it a lot.Our transactions wouldn't always settle very quickly. We didn't know about it, but now our whole system becomes eventually consistent typically if you now divide all of your architecture across domains, and decouple everything. We created some early prototypes, we created our own version of an event store, which is, I would just say an opinionated scheme around DynamoDB, where we keep track of revisions, payload, timestamp, all the things you'd want to be able to do event sourcing. That's another thing we decided on. Event sourcing seemed like the right approach for state, for a lot of our use cases. Banking, if you just think about a banking ledger, it is events or an accounting ledger. You're just adding up rows, add, subtract, add, subtract.We created a lot of prototypes for these things. Our events store pattern became basically just a DynamoDB with opinions around the schema, as well as a package of a shared code package with a simple dispatch function. One dispatch function that really looks at enforcing optimistic concurrency, and one that's a little bit more relaxed. Then we also had some reducer functions built into there. That was one of the packages that we created, as well as another prototype around that was how do we create the actual subscriptions to this event store? We landed on SNS to SQS fan-out, and it seems like fan-out first is the serverless way of doing a lot of things. We learned that along the way, and it makes sense. It was one of those things we read from a lot of these blogs and YouTube videos, and it really made sense in production, when all the data is streaming from one place, and then now you just add subscribers all over the place. Just new queues. Fan-out first, highly recommend. We just landed on there by following best practices.Jeremy: Great. You mentioned a bunch of different things in there, which is awesome, but so you get together in this house, you come up with all the events, you do this event storming session, which is always a great exercise. You get a pretty good visualization of how the business is going to run from an event standpoint. Then you start building out this event driven architecture, and you mentioned some packages that you built, we talked about step functions and the orchestration piece of this. Just give me a quick overview of the actual system itself. You said it's backed by DynamoDB, but then you have a bunch of packages that run in between there, and then there's a whole bunch of queues, and then you're using some custom packages. I think I already said that but you're using ... are you using EventBridge in there? What's some of the architecture behind all that?Patrick: Really, really good question. Once we created these domain boundaries, we needed to figure out how do we communicate between domains and within domains. We landed on really differentiating milestone events and domain events. I guess milestone events in other terms might be called integration events, but this idea that these are key business milestones. An account was open, an application was approved or rejected, things that every domain may need to know about. Then within our domains, or domain boundaries, we had these domain events, which might reduce to a milestone event, and we can maintain those contracts in the future and change those up. We needed to think about how do we message all these things across? How do we communicate? We landed on EventBridge for our milestone events. We have one event bus that we talked to all of our, between domain boundaries basically.EventBridge there, and then each of our services now subscribed to that EventBridge, and maintain their own events store. That's backed by DynamoDB. Each of our services have their own data store. It's usually an event stream or a projection database, but it's almost all Dynamo, which is interesting because our old platform used Postgres, and we did have relational data. It was interesting. I was really scared at first, how are we going to maintain relations and things? It became a non-issue. I don't even know why now that I think about it. Just like every service maintains its nice projection through events, and builds its own view of the world, which brings its own problems. We have DynamoDB in there, and then SNS to SQS fan-out. Then when we're talking about packages ...Jeremy: That's Office Streams?Patrick: Exactly, yeah. We're Dynamo streams to SNS, to SQS. Then we use shared code packages to make those subscriptions very easy. If you're looking at doing that SNS to SQS fan-out, or just creating SQS queues, there is a lot of cloud formation boilerplate that we were creating, and we needed to move really quick on this project. We got pretty opinionated quick, and we created our own subscription function that just generates all this cloud formation with naming conventions, which was nice. I think the opinions were good because early on we weren't opinionated enough, I would say. When you look in your AWS dashboard, the read for these aren't prefixed correctly, and there's all this garbage. You're able to have consistent naming throughout, make it really easy to subscribe to an event.We would publish packages to help with certain things. Our events store package was one of those. We also created a Lambda handlers package, which leverages, there's like a Lambda middlewares compose package out there, which is quite nice, and we basically, all the common functionality we're doing a lot of, like parsing a body from S3, or SQS or API gateway. That's just the middleware that we now publish. Validation in and out. We highly recommend the library Zod, we really embrace the TypeScript first object validation. Really, really cool package. We created all these middlewares now. Then subscription packages. We have a lot of shared code in this internal NPM repository that we install across.I think one challenge we had there was, eventually you extracted away too much from the cloud formation, and it's hard for new developers to ... It's easy for them to create events subscriptions, it's hard for them to evolve our serverless thinking because they're so far removed from it. I still think it was the right call in the end. I think this is the next step of the journey, is figuring out how do we share code effectively while not hiding away too much of serverless, especially because it's changing so fast.Jeremy: It's also interesting though that you take that approach to hide some of that complexity, and bake in some of that boilerplate that, someone's mostly didn't have to write themselves anyways. Like you said, they're copying and pasting between services, is not the best way to do it. I tried the whole shared packages thing one time, and it kind of worked. It's just like when you make a small change to that package and you have 14 services, that then you have to update to get the newest version. Sometimes that's a little frustrating. Lambda layers haven't been a huge help with some of that stuff either. But anyways, it's interesting, because again you've mentioned this a number of times about using queues.You did mention resiliency in there, but I want to touch on that point a little bit because that's one of those things too, where I would assume in a banking platform, you do not want to lose events. You don't want to lose things. and so if something breaks, or something gets throttled or whatever, having to go and retry those events, having the alerts in place to know that a queue is backed up or whatever. Then just, I'm thinking ordering issues and things like that. What kinds of issues did you face, and tell me a little bit more about what you've done for reliability?Patrick: Totally. Queues are definitely ... like SQS is a workhorse for our company right now. We use a lot of it. Dropping messages is one of the scariest things, so you're dead-on there. When we were moving to event driven, that was what scared me the most. What if we drop an event? A good example of that is if you're using EventBridge and you're subscribing Lambdas to it, I was under the impression early on that EventBridge retries forever. But I'm pretty sure it'll retry until it invokes twice. I think that's what we landed on.Jeremy: Interesting.Patrick: I think so, and don't quote me on this. That was an example of where drop message could be a problem. We put a queue in front of there, an SQS queue as the subscription there. That way, if there's any failure to deliver there, it's just going to retry all the time for a number of days. At that point we got to think about DLQs, and that's something we're still thinking about. But yeah, I think the reason we've been using queues everywhere is that now queues are in charge of all your retry abilities. Now that we've decomposed these Lambdas into one Lambda lift, into five Lambdas with queues in between, if anything fails in there, it just pops back into the queue, and it'll retry indefinitely. You can drop messages after a few days, and that's something we learned luckily in the prototyping stage, where there are a few places where we use dead letter queues. But one of the issues there as well was ordering. Ordering didn't play too well with ...Jeremy: Not with DLQs. No, it does not, no.Patrick: I think that's one lesson I'd want to share, is that only use ordering when you absolutely need it. We found ways to design some of our architecture where we didn't need ordering. There's places we were using FIFO SQS, which was something that just launched when we were building this thing. When we were thinking about messaging, we're like, "Oh, well we can't use SQS because they don't respect ordering, or it doesn't respect ordering." Then bam, the next day we see this blog article. We got really hyped on that and used FIFO everywhere, and then realized it's unnecessary in most use cases. So when we were going live, we actually changed those FIFO queues into just regular SQS queues in as many places as we can. Then so, in that use case, you could really easily attach a dead letter queue and you don't have to worry about anything, but with FIFO things get really, really gnarly.Ordering is an interesting one. Another place we got burned I think on dead-letter queues, or a tough thing to do with dead letter queues is when you're using our state machines, we needed to limit the concurrency of our state machines is another wishlist item in AWS. I wish there was just at the top of the file, a limit concurrent executions of your state machine. Maybe it exists. Maybe we just didn't learn to use it properly, but we needed to. There's a few patterns out there. I've seen the [INAUDIBLE] pattern where you can use the actual state machine flow to look back at how many concurrent executions you have, and pause. We landed on setting reserved concurrency in a number of Lambdas, and throwing errors. If we've hit the max concurrency and it'll pause that Lambda, but the problem with DLQs there was, these are all errors. They're coming back as errors.We're like, we're fine with them. This is a throttle error. That's fine. But it's hard to distinguish that from a poison message in your queue, so when do you dump those into DLQ? If it's just a throttling thing, I don't think it matters to us. That was another challenge we had. We're still figuring out dead letter queues and alerting. I think for now we just relied on CloudWatch alarms a lot for our alerting, and there's a lot you could do. Even just in the state machines, you can get pretty granular there. I know once certain things fail, and announced to your Slack channel. We use that Slack integration, it's pretty easy. You just go on a Slack channel, there's an email in there, you plop it into the console in AWS, and you have your very early alerting mechanism there.Jeremy: The thing with Elasticsearch ... not Elasticsearch, I'm sorry. I'm totally off-topic here. The thing with EventBridge and Lambda, these are one of those things that, again, they're nuances, but event bridge, as long as it can deliver to the Lambda service, then the Lambda service kicks off and queues it automatically. Then that will retry at a certain number of times. I think you can control that now. But then eventually if that retries multiple times and eventually fails, then that kicks it over to the DLQ or whatever. There's all different ways that it works like that, but that's why I always liked the idea of putting a queue in between there as well, because I felt you just had a little bit more control over exactly what happens.As long as it gets to the queue, then you know you haven't lost the message, or you hope you haven't lost a message. That's super interesting. Let's move on a little bit about the adoption issues. You mentioned a few of these things, obviously issues with concurrency and ordering, and some of that other stuff. What about some of the other challenges you had? You mentioned this idea of writing all these packages, and it pulls devs away from the CloudFormation a little bit. I do like that in that it, I think, accelerates a lot of things, but what are some of the other maybe challenges that you've been having just getting this thing up and running?Patrick: I would say IAM is an interesting one. Because we are in the banking space, we want to be very careful about what access do you give to what machines or developers, I think machines are important too. There've been cases where ... so we do have a separate developer set up with their own permissions, in development's really easy to spin up all your services within reason. But now when we're going into production, there's times where our CI doesn't have the permissions to delete a queue or create a queue, or certain things, and there's a lot of tweaking you have to do there, and you got to do a lot of thinking about your IAM policies as an organization, especially because now every developer's touching infrastructure.That becomes this shared operational overhead that serverless did introduce. We're still figuring that out. Right now we're functioning on least privilege, so it's better to just not be able to deploy than deploy something you shouldn't or read the logs that you shouldn't, and that's where we're starting. But that's something that, it will be a challenge for a little while I think. There's all kinds of interesting things out there. I think temporary IAM permissions is a really cool one. There are times we're in production and we need to view certain logs, or be able to access a certain queue, and there's tooling out there where you can, or at least so I've heard, you can give temporary permissions. You have this queue permission for 30 minutes, and it expires and it's audited, and I think there's some CloudTrail tie-in you could do there. I'm speaking about my wishlist for our next evolution here. I hope my team is listening ...Jeremy: Your team's listening to you.Patrick: ... will be inspired as well.Jeremy: What about ... because this is something too that I always found to be a challenge, especially when you start having multiple services, and you've talked about these domain events, but then milestone events. You've got different services that need to communicate across services, or across domains, and realize certain things like that. Service discovery in and of itself, and which queue are we mapping to, or which service am I talking to, and which version of the service am I talking to? Things like that. How have you been dealing with that stuff?Patrick: Not well, I would say. Very, very ad hoc. I think like right now, at least we have tight communication between the teams, so we roughly know which service we need to talk to, and we output our URLs in the cloud formation output, so at least you could reference the URLs across services, a little easier. Really, a GraphQL is one of the only service that really talks to a lot of our API gateways. At least there's less of that, knowing which endpoint to hit. Most of our services will read into EventBridge, and then within services, a lot of that's abstracted away, like the queue subscription's a little easier. Service discovery is a bit of a nightmare.Once our services grow, it'll be, I don't know. It'll be a huge challenge to understand. Even which services are using older versions of Node, for instance. I saw that AWS is now deprecating version 10 and we'll have to take a look internally, are we using version 10 anywhere, and how do we make sure that's fine, or even things like just knowing which services now have vulnerabilities in their NPM packages because we're using Node. That's another thing. I don't even know if that falls in service discovery, but it's an overhead of ...Jeremy: It's a service management too. It's a lot there. That actually made me, it brings me to this idea of observability too. You mentioned doing some CloudWatch alerts and some of that stuff, but what about using some observability tool or tracing like x-ray, and things like that? Have you been implementing any of that, and if you have, have you had any success and or problems with it?Patrick: I wish we had a better view of some of the observability tools. I think we were just building so quickly that we never really invested the time into trying them out. We did use X-Ray, so we rolled our own tooling internally to at least do what we know. X-Ray was one of those, but the problem with X-Ray is, we do subscribe all of our services, but X-Ray isn't implemented everywhere internally in AWS, so we lose our trail somewhere in that Dynamo stream to SNS, or SQS. It's not a full trace. Also, just digesting that huge graph of information is just very difficult. I don't use it often, I think it's a really cool graphic to show, “Hey, look, how many services are running, and it's going so fast."It's a really cool thing to look at, but it hasn't been very useful. I think our most useful tool for debugging and observability has been just our logging. We created a JSON logger package, so we get up JSON logs and we can actually filter off of different properties, and we ship those to Elasticsearch. Now you can have a view of all of the functions within a given domain at any point in time. You could really see the story. Because I think early on when we were opening up CloudWatch and you'd have like 10 tabs, and you're trying to understand this flow of information, it was very difficult.We also implemented our own trace ID pattern, and I think we just followed a Lumigo article where we introduced some properties, and in each of our Lambdas at a higher level, and one of our middlewares, and we were able to trace through. It's not ideal. Observability is something that we'll probably have to work on next. It's been tolerable for now, but I can't see the scaling that long.Jeremy: That's the other thing too, is even the shared package issue. It's like when you have an observability tool, they'll just install a layer or something, where you don't necessarily have to worry about updating your own tool. I always find if you are embracing serverless and you want to get rid of all that undifferentiated heavy lifting, observability tools, there's a lot of really good ones out there that are doing some great stuff, and they're specializing in it. It might be worth letting someone else handle that for you than trying to do it yourself internally.Patrick: Yeah, 100%. Do you have any that you've used that are particularly good? I know you work with serverless so-Jeremy: I played around with all of them, because I love this stuff, so it's always fun, but I mean, obviously Lumigo and Epsagon, and Thundra, and New Relic. They're all great. They all do things slightly differently, but they all follow a similar implementation pattern so that it's very easy to install them. We can talk more about some recommendations. I think it's just one of those things where in a modern application not having that insight is really hard. It can be really hard to debug stuff. If you look at some of the tools that AWS offers, I think they're there, it's just, they are maybe a little harder to implement, and not quite as refined and targeted as some of the observability tools. But still, you got to get there. Again, that's why I keep saying it's an evolution, it's a process. Maybe one time you get burned, and you're like, we really needed to have observability, then that's when it becomes more of a priority when you're moving fast like you are.Patrick: Yeah, 100%. I think there's got to be a priority earlier than later. I think I'll do some reading now that you've dropped some of these options. I have seen them floating around, but it's one of those things that when it's too late, it's too late.Jeremy: It's never too late to add observability though, so it should. Actually, a lot of them now, again, it makes it really, really easy. So I'm not trying to pitch any particular company, but take a look at some of them, because they are really great. Just one other challenge that I also find a lot of people run into, especially with serverless because there's all these artificial account limits in place. Even the number of queues you can create, and the number of concurrent Lambda functions in a particular region, and stuff like that. Have you run into any of those account limit issues?Patrick: Yeah. I could give you the easiest way to run into an account on that issue, and that is replay your entire EventBridge archive to every subscriber, and you will find a bottleneck somewhere. That's something ...Jeremy: Somewhere it'll fall over? Nice.Patrick: 100%. It's a good way to do some quick check and development to see where you might need to buffer something, but we have run into that. I think the solution there, and a lot of places was just really playing with concurrency where we needed to, and being thoughtful about where is their main concurrency in places that we absolutely needed to stay functioning. I think the challenge there is that eats into your total account concurrency, which was an interesting learning there. Definitely playing around there, and just being thoughtful about where you are replaying. A couple of things. We use replays a lot. Because we are using these milestone events between service boundaries, now when you launch a new service, you want to replay that whole history all the way through.We've done a lot of replaying, and that was one of the really cool things about EventBridge. It just was so easy. You just set up an archive, and it'll record everything coming through, and then you just press a button in the console, and it'll replay all of them. That was really awesome. But just being very mindful of where you're replaying to. If you replay to all of your subscriptions, you'll hit Lambda concurrency limits real quick. Even just like another case, early on we needed to replace ... we have our own domain events store. We want to replace some of those events, and those are coming off the Dynamo stream, so we were using dynamo to kick those to a stream, to SNS, and fan-out to all of our SQS queues. But there would only be one or two queues you actually needed to subtract to those events, so we created an internal utility just to dump those events directly into the SQS queue we needed. I think it's just about not being wasteful with your resources, because they are cheap. Sure.Jeremy: But if you use them, they start to cost money.Patrick: Yeah. They start to cost some money as well as they could lock down, they can lock you out of other functionality. If you hit your Lambda limits, now our API gateway is tapped.Jeremy: That's a good point.Patrick: You could take down your whole system if you just aren't mindful about those limits, and now you could call up AWS in a panic and be like, “Hey, can you update our limits?" Luckily we haven't had to do that yet, but it's definitely something in your back pocket if you need it, if you can make the case to AWS, that maybe you do need bigger limits than the default. I think just not being wasteful, being mindful of where you're replaying. I think another interesting thing there is dealing with partners too. It's really easy to scale in the Lambda world, but not every partner could handle that volume really quickly. If you're not buffering any event coming through EventBridge to your new service that hits a partner every time, you're going to hit their API rate limit really quickly, because they're just going to just go right through it.You might be doing thousands of API calls when you're instantiating a new service. That's one of those interesting things that we have to deal with, and particularly in our orchestrators, because they are talking to different partners, that's why we need to really make sure we could limit the concurrent executions of the state machines themselves. In a way, some of our architecture is too fast to scale.Jeremy: It's too good.Patrick: You still have to consider downstream. That, and even just, if you are using relational databases or anything else in your system, now you have to worry about connection limits and ...Jeremy: I have a whole talk I gave on that.Patrick: ... spikes in traffic.Jeremy: Yes, absolutely.Patrick: Really cool.Jeremy: I know all about it. Any final advice for companies like you that are trying to bite off a piece of the serverless apple, I guess, That's really bad. Anyways, any advice for people looking to get into this?Patrick: Yeah, totally. I would say start small. I think we were wise to just try it out. It might not land with your development team. If you don't really buy in, it's one of those things that could just end up unnecessarily messy, so start small, see if you like it in-shop, and then reevaluate, once you hit a certain point. That, and I would say shared boilerplate packages sooner than later. I know shared code is a problem, but it is nice to have an un-opinionated starter pack, that you're at least not doing anything really crazy. Even just things like having opinions around logging. In our industry, it's really important that you're not logging sensitive details.For us doing things like wrapping our HTTP clients to make sure we're not logging sensitive details, or having short Lambda packages that make sure out-of-the-box you're opinionated about not doing something terribly awful. I would say those two things. Start small and a boiler package, and maybe the third thing is just pay attention to the code smell of a growing Lambda. If you are doing three API calls in one Lambda, chances are you could probably break that up, and think about it in a more resilient way. If any one of those pieces fail, now you could have retry ability in each one of those. Those are the three things I would say. I could probably talk forever about the rest of our journey.Jeremy: I think that was great advice, and I love hearing about how companies are going through this process, what that process looks like, and I hope, I hope, I hope that companies listen to this and can skip a lot of these mistakes. I don't want to call them all mistakes, and I think it's just evolution. The stuff that you've done, we've all made them, we've all gone through that process, and the more we can solidify these practices and stuff like that, I think that more companies will benefit from hearing stories like these. Thank you very much for sharing that. Again, thank you so much for spending the time to do this and sharing all of this knowledge, and this journey that you've been on, and are continuing to be on. It would great to continue to get updates from you. If people want to contact you, I know you're not on Twitter, but what's the best way to reach out to you?Patrick: I almost wish I had a Twitter. It's the developer thing to have, so maybe in the future. Just on LinkedIn would be great. LinkedIn would be great, as well as if anybody's interested in working with our team, and just figuring out how to take serverless to the next level, just hit me up on LinkedIn or look at our careers page at northone.com, and I could give you a warm intro.Jeremy: That's great. Just your last name is spelled S-T-R-Z-E-L-E-C. How do you say that again? Say it in Polish, because I know I said it wrong in the beginning.Patrick: I guess for most people it would just be Strzelec, but if there are any Slavs in the audience, it's "Strzelec." Very intense four consonants last name.Jeremy: That is a lot of consonants. Anyways again, Patrick, thanks again. This was great.Patrick: Yeah, thank you so much, Jeremy. This has been awesome.

About Rebecca MarshburnRebecca's interested in the things that interest people—What's important to them? Why? And when did they first discover it to be so? She's also interested in sharing stories, elevating others' experiences, exploring the intersection of physical environments and human behavior, and crafting the perfect pun for every situation. Today, Rebecca is the Head of Content & Community at Common Room. Prior to Common Room, she led the AWS Serverless Heroes program, where she met the singular Jeremy Daly, and guided content and product experiences for fashion magazines, online blogs, AR/VR companies, education companies, and a little travel outfit called Airbnb.Twitter: @beccaodelayLinkedIn: Rebecca MarshburnCompany: www.commonroom.ioPersonal work (all proceeds go to the charity of the buyer's choice): www.letterstomyexlovers.comWatch this episode on YouTube: https://youtu.be/VVEtxgh6GKI This episode sponsored by CBT Nuggets and Lumigo.Transcript:Rebecca: What a day today is! It's not every day you turn 100 times old, and on this day we celebrate Serverless Chats 100th episode with the most special of guests. The gentleman whose voice you usually hear on this end of the microphone, doing the asking, but today he's going to be doing the telling, the one and only, Jeremy Daly, and me. I'm Rebecca Marshburn, and your guest host for Serverless Chats 100th episode, because it's quite difficult to interview yourself. Hey Jeremy!Jeremy: Hey Rebecca, thank you very much for doing this.Rebecca: Oh my gosh. I am super excited to be here, couldn't be more honored. I'll give your listeners, our listeners, today, the special day, a little bit of background about us. Jeremy and I met through the AWS Serverless Heroes program, where I used to be a coordinator for quite some time. We support each other in content, conferences, product requests, road mapping, community-building, and most importantly, I think we've supported each other in spirit, and now I'm the head of content and community at Common Room, and Jeremy's leading Serverless Cloud at Serverless, Inc., so it's even sweeter that we're back together to celebrate this Serverless Chats milestone with you all, the most important, important, important, important part of the podcast equation, the serverless community. So without further ado, let's begin.Jeremy: All right, hit me up with whatever questions you have. I'm here to answer anything.Rebecca: Jeremy, I'm going to ask you a few heavy hitters, so I hope you're ready.Jeremy: I'm ready to go.Rebecca: And the first one's going to ask you to step way, way, way, way, way back into your time machine, so if you've got the proper attire on, let's do it. If we're going to step into that time machine, let's peel the layers, before serverless, before containers, before cloud even, what is the origin story of Jeremy Daly, the man who usually asks the questions.Jeremy: That's tough. I don't think time machines go back that far, but it's funny, when I was in high school, I was involved with music, and plays, and all kinds of things like that. I was a very creative person. I loved creating things, that was one of the biggest sort of things, and whether it was music or whatever and I did a lot of work with video actually, back in the day. I was always volunteering at the local public access station. And when I graduated from high school, I had no idea what I wanted to do. I had used computers at the computer lab at the high school. I mean, this is going back a ways, so it wasn't everyone had their own computer in their house, but I went to college and then, my first, my freshman year in college, I ended up, there's a suite-mate that I had who showed me a website that he built on the university servers.And I saw that and I was immediately like, "Whoa, how do you do that"? Right, just this idea of creating something new and being able to build that out was super exciting to me, so I spent the next couple of weeks figuring out how to do HTML, and this was before, this was like when JavaScript was super, super early and we're talking like 1997, and everything was super early. I was using this, I eventually moved away from using FrontPage and started using this thing called HotDog. It was a software for HTML coding, but I started doing that, and I started building websites, and then after a while, I started figuring out what things like CGI-bins were, and how you could write Perl scripts, and how you could make interactions happen, and how you could capture FormData and serve up different things, and it was a lot of copying and pasting.My major at the time, I think was psychology, because it was like a default thing that I could do. But then I moved into computer science. I did computer science for about a year, and I felt that that was a little bit too narrow for what I was hoping to sort of do. I was starting to become more entrepreneurial. I had started selling websites to people. I had gone to a couple of local businesses and started building websites, so I actually expanded that and ended up doing sort of a major that straddled computer science and management, like business administration. So I ended up graduating with a degree in e-commerce and internet marketing, which is sort of very early, like before any of this stuff seemed to even exist. And then from there, I started a web development company, worked on that for 12 years, and then I ended up selling that off. Did a startup, failed the startup. Then from that startup, went to another startup, worked there for a couple of years, went to another startup, did a lot of consulting in between there, somewhere along the way I found serverless and AWS Cloud, and then now it's sort of led me to advocacy for building things with serverless and now I'm building sort of the, I think what I've been dreaming about building for the last several years in what I'm doing now at Serverless, Inc.Rebecca: Wow. All right. So this love story started in the 90s.Jeremy: The 90s, right.Rebecca: That's an incredible, era and welcome to 2021.Jeremy: Right. It's been a journey.Rebecca: Yeah, truly, that's literally a new millennium. So in a broad way of saying it, you've seen it all. You've started from the very HotDog of the world, to today, which is an incredible name, I'm going to have to look them up later. So then you said serverless came along somewhere in there, but let's go to the middle of your story here, so before Serverless Chats, before its predecessor, which is your weekly Off-by-none newsletter, and before, this is my favorite one, debates around, what the suffix "less" means when appended to server. When did you first hear about Serverless in that moment, or perhaps you don't remember the exact minute, but I do really want to know what struck you about it? What stood out about serverless rather than any of the other types of technologies that you could have been struck by and been having a podcast around?Jeremy: Right. And I think I gave you maybe too much of a surface level of what I've seen, because I talked mostly about software, but if we go back, I mean, hardware was one of those things where hardware, and installing software, and running servers, and doing networking, and all those sort of things, those were part of my early career as well. When I was running my web development company, we started by hosting on some hosting service somewhere, and then we ended up getting a dedicated server, and then we outgrew that, and then we ended up saying, "Well maybe we'll bring stuff in-house". So we did on-prem for quite some time, where we had our own servers in the T1 line, and then we moved to another building that had a T3 line, and if anybody doesn't know what that is, you probably don't need to anymore.But those are the things that we were doing, and then eventually we moved into a co-location facility where we rented space, and we rented electricity, and we rented all the utilities, the bandwidth, and so forth, but we had Blade servers and I was running VMware, and we were doing all this kind of stuff to manage the infrastructure, and then writing software on top of that, so it was a lot of work. I know I posted something on Twitter a few weeks ago, about how, when I was, when we were young, we used to have to carry a server on our back, uphill, both ways, to the data center, in the snow, with no shoes, and that's kind of how it felt, that you were doing a lot of these things.And then 2008, 2009, as I was kind of wrapping up my web development company, we were just in the process of actually saying it's too expensive at the colo. I think we were paying probably between like $5,000 and $7,000 a month between the ... we had leases on some of the servers, you're paying for electricity, you're paying for all these other things, and we were running a fair amount of services in there, so it seemed justifiable. We were making money on it, that wasn't the problem, but it just was a very expensive fixed cost for us, and when the cloud started coming along and I started actually building out the startup that I was working on, we were building all of that in the cloud, and as I was learning more about the cloud and how that works, I'm like, I should just move all this stuff that's in the co-location facility, move that over to the cloud and see what happens.And it took a couple of weeks to get that set up, and now, again, this is early, this is before ELB, this is before RDS, this is before, I mean, this was very, very early cloud. I mean, I think there was S3 and EC2. I think those were the two services that were available, with a few other things. I don't even think there were VPCs yet. But anyways, I moved everything over, took a couple of weeks to get that over, and essentially our bill to host all of our clients' sites and projects went from $5,000 to $7,000 a month, to $750 a month or something like that, and it's funny because had I done that earlier, I may not have sold off my web development company because it could have been much more profitable, so it was just an interesting move there.So we got into the cloud fairly early and started sort of leveraging that, and it was great to see all these things get added and all these specialty services, like RDS, and just taking the responsibility because I literally was installing Microsoft SQL server on an EC2 instance, which is not something that you want to do, you want to use RDS. It's just a much better way to do it, but anyways, so I was working for another startup, this was like startup number 17 or whatever it was I was working for, and we had this incident where we were using ... we had a pretty good setup. I mean, everything was on EC2 instances, but we were using DynamoDB to do some caching layers for certain things. We were using a sharded database, MySQL database, for product information, and so forth.So the system was pretty resilient, it was pretty, it handled all of the load testing we did and things like that, but then we actually got featured on Good Morning America, and they mentioned our app, it was the Power to Mobile app, and so we get mentioned on Good Morning America. I think it was Good Morning America. The Today Show? Good Morning America, I think it was. One of those morning shows, anyways, we got about 10,000 sign-ups in less than a minute, which was amazing, or it was just this huge spike in traffic, which was great. The problem was, is we had this really weak point in our system where we had to basically get a lock on the database in order to get an incremental-ID, and so essentially what happened is the database choked, and then as soon as the database choked, just to create user accounts, other users couldn't sign in and there was all kinds of problems, so we basically lost out on all of this capability.So I spent some time doing a lot of research and trying to figure out how do you scale that? How do you scale something that fast? How do you have that resilience in there? And there's all kinds of ways that we could have done it with traditional hardware, it's not like it wasn't possible to do with a slightly better strategy, but as I was digging around in AWS, I'm looking around at some different things, and we were, I was always in the console cause we were using Dynamo and some of those things, and I came across this thing that said "Lambda," with a little new thing next to it. I'm like, what the heck is this?So I click on that and I start reading about it, and I'm like, this is amazing. We don't have to spin up a server, we don't have to use Chef, or Puppet, or anything like that to spin up these machines. We can basically just say, when X happens, do Y, and it enlightened me, and this was early 2015, so this would have been right after Lambda went GA. Had never heard of Lambda as part of the preview, I mean, I wasn't sort of in that the re:Invent, I don't know, what would you call that? Vortex, maybe, is a good way to describe the event.Rebecca: Vortex sounds about right. That's about how it feels by the end.Jeremy: Right, exactly. So I wasn't really in that, I wasn't in that group yet, I wasn't part of that community, so I hadn't heard about it, and so as I started playing around with it, I immediately saw the value there, because, for me, as someone who again had managed servers, and it had built out really complex networking too. I think some of the things you don't think about when you move to an on-prem where you're managing your stuff, even what the cloud manages for you. I mean, we had firewalls, and we had to do all the firewall rules ourselves, right. I mean, I know you still have to do security groups and things like that in AWS, but just the level of complexity is a lot lower when you're in the cloud, and of course there's so many great services and systems that help you do that now.But just the idea of saying, "wait a minute, so if I have something happen, like a user signup, for example, and I don't have to worry about provisioning all the servers that I need in order to handle that," and again, it wasn't so much the server aspect of it as it was the database aspect of it, but one of the things that was sort of interesting about the idea of Serverless 2 was this asynchronous nature of it, this idea of being more event-driven, and that things don't have to happen immediately necessarily. So that just struck me as something where it seemed like it would reduce a lot, and again, this term has been overused, but the undifferentiated heavy-lifting, we use that term over and over again, but there is not a better term for that, right?Because there were just so many things that you have to do as a developer, as an ops person, somebody who is trying to straddle teams, or just a PM, or whatever you are, so many things that you have to do in order to get an application running, first of all, and then even more you have to do in order to keep it up and running, and then even more, if you start thinking about distributing it, or scaling it, or getting any of those things, disaster recovery. I mean, there's a million things you have to think about, and I saw serverless immediately as this opportunity to say, "Wait a minute, this could reduce a lot of that complexity and manage all of that for you," and then again, literally let you focus on the things that actually matter for your business.Rebecca: Okay. As someone who worked, how should I say this, in metatech, or the technology of technology in the serverless space, when you say that you were starting to build that without ELB even, or RDS, my level of anxiety is like, I really feel like I'm watching a slow horror film. I'm like, "No, no, no, no, no, you didn't, you didn't, you didn't have to do that, did you"?Jeremy: We did.Rebecca: So I applaud you for making it to the end of the film and still being with us.Jeremy: Well, the other thing ...Rebecca: Only one protagonist does that.Jeremy: Well, the other thing that's interesting too, about Serverless, and where it was in 2015, Lambda goes GA, this will give you some anxiety, there was no API gateway. So there was no way to actually trigger a Lambda function from a web request, right. There was no VPC access in Lambda functions, which meant you couldn't connect to a database. The only thing you do is connect via HDP, so you could connect to DynamoDB or things like that, but you could not connect directly to RDS, for example. So if you go back and you look at the timeline of when these things were released, I mean, if just from 2015, I mean, you literally feel like a caveman thinking about what you could do back then again, it's banging two sticks together versus where we are now, and the capabilities that are available to us.Rebecca: Yeah, you're sort of in Plato's cave, right, and you're looking up and you're like, "It's quite dark in here," and Lambda's up there, outside, sowing seeds, being like, "Come on out, it's dark in there". All right, so I imagine you discovering Lambda through the console is not a sentence you hear every day or general console discovery of a new product that will then sort of change the way that you build, and so I'm guessing maybe one of the reasons why you started your Off-by-none newsletter or Serverless Chats, right, is to be like, "How do I help tell others about this without them needing to discover it through the console"? But I'm curious what your why is. Why first the Off-by-none newsletter, which is one of my favorite things to receive every week, thank you for continuing to write such great content, and then why Serverless Chats? Why are we here today? Why are we at number 100? Which I'm so excited about every time I say it.Jeremy: And it's kind of crazy to think about all the people I've gotten a chance to talk to, but so, I think if you go back, I started writing blog posts maybe in 2015, so I haven't been doing it that long, and I certainly wasn't prolific. I wasn't consistent writing a blog post every week or every, two a week, like some people do now, which is kind of crazy. I don't know how that, I mean, it's hard enough writing the newsletter every week, never mind writing original content, but I started writing about Serverless. I think it wasn't until the beginning of 2018, maybe the end of 2017, and there was already a lot of great content out there. I mean, Ben Kehoe was very early into this and a lot of his stuff I read very early.I mean, there's just so many people that were very early in the space, I mean, Paul Johnson, I mean, just so many people, right, and I started reading what they were writing and I was like, "Oh, I've got some ideas too, I've been experimenting with some things, I feel like I've gotten to a point where what I could share could be potentially useful". So I started writing blog posts, and I think one of the earlier blog posts I wrote was, I want to say 2017, maybe it was 2018, early 2018, but was a post about serverless security, and what was great about that post was that actually got me connected with Ory Segal, who had started PureSec, and he and I became friends and that was the other great thing too, is just becoming part of this community was amazing.So many awesome people that I've met, but so I saw all this stuff people were writing and these things people were doing, and I got to maybe August of 2018, and I said to myself, I'm like, "Okay, I don't know if people are interested in what I'm writing". I wasn't writing a lot, but I was writing a little bit, but I wasn't sure people were overly interested in what I was writing, and again, that idea of the imposter syndrome, certainly everything was very early, so I felt a little bit more comfortable. I always felt like, well, maybe nobody knows what they're talking about here, so if I throw something into the fold it won't be too, too bad, but certainly, I was reading other things by other people that I was interested in, and I thought to myself, I'm like, "Okay, if I'm interested in this stuff, other people have to be interested in this stuff," but it wasn't easy to find, right.I mean, there was sort of a serverless Twitter, if you want to use that terminology, where a lot of people tweet about it and so forth, obviously it's gotten very noisy now because of people slapped that term on way too many things, but I don't want to have that discussion, but so I'm reading all this great stuff and I'm like, "I really want to share it," and I'm like, "Well, I guess the best way to do that would just be a newsletter."I had an email list for my own personal site that I had had a couple of hundred people on, and I'm like, "Well, let me just turn it into this thing, and I'll share these stories, and maybe people will find them interesting," and I know this is going to sound a little bit corny, but I have two teenage daughters, so I'm allowed to be sort of this dad-jokey type. I remember when I started writing the first version of this newsletter and I said to myself, I'm like, "I don't want this to be a newsletter." I was toying around with this idea of calling it an un-newsletter. I didn't want it to just be another list of links that you click on, and I know that's interesting to some people, but I felt like there was an opportunity to opine on it, to look at the individual links, and maybe even tell a story as part of all of the links that were shared that week, and I thought that that would be more interesting than just getting a list of links.And I'm sure you've seen over the last 140 issues, or however many we're at now, that there's been changes in the way that we formatted it, and we've tried new things, and things like that, but ultimately, and this goes back to the corny thing, I mean, one of the first things that I wanted to do was, I wanted to basically thank people for writing this stuff. I wanted to basically say, "Look, this is not just about you writing some content". This is big, this is important, and I appreciate it. I appreciate you for writing that content, and I wanted to make it more of a celebration really of the community and the people that were early contributors to that space, and that's one of the reasons why I did the Serverless Star thing.I thought, if somebody writes a really good article some week, and it's just, it really hits me, or somebody else says, "Hey, this person wrote a great article," or whatever. I wanted to sort of celebrate that person and call them out because that's one of the things too is writing blog posts or posting things on social media without a good following, or without the dopamine hit of people liking it, or re-tweeting it, and things like that, it can be a pretty lonely place. I mean, I know I feel that way sometimes when you put something out there, and you think it's important, or you think people might want to see it, and just not enough people see it.It's even worse, I mean, 240 characters, or whatever it is to write a tweet is one thing, or 280 characters, but if you're spending time putting together a tutorial or you put together a really good thought piece, or story, or use case, or something where you feel like this is worth sharing, because it could inspire somebody else, or it could help somebody else, could get them past a bump, it could make them think about something a different way, or get them over a hump, or whatever. I mean, that's just the kind of thing where I think people need that encouragement, and I think people deserve that encouragement for the work that they're doing, and that's what I wanted to do with Off-by-none, is make sure that I got that out there, and to just try to amplify those voices the best that I could. The other thing where it's sort of progressed, and I guess maybe I'm getting ahead of myself, but the other place where it's progressed and I thought was really interesting, was, finding people ...There's the heavy hitters in the serverless space, right? The ones we all know, and you can name them all, and they are great, and they produce amazing content, and they do amazing things, but they have pretty good engines to get their content out, right? I mean, some people who write for the AWS blog, they're on the AWS blog, right, so they're doing pretty well in terms of getting their things out there, right, and they've got pretty good engines.There's some good dev advocates too, that just have good Twitter followings and things like that. Then there's that guy who writes the story. I don't know, he's in India or he's in Poland or something like that. He writes this really good tutorial on how to do this odd edge-case for serverless. And you go and you look at their Medium and they've got two followers on Medium, five followers on Twitter or something like that. And that to me, just seems unfair, right? I mean, they've written a really good piece and it's worth sharing right? And it needs to get out there. I don't have a huge audience. I know that. I mean I've got a good following on Twitter. I feel like a lot of my Twitter followers, we can have good conversations, which is what you want on Twitter.The newsletter has continued to grow. We've got a good listener base for this show here. So, I don't have a huge audience, but if I can share that audience with other people and get other people to the forefront, then that's important to me. And I love finding those people and those ideas that other people might not see because they're not looking for them. So, if I can be part of that and help share that, that to me, it's not only a responsibility, it's just it's incredibly rewarding. So ...Rebecca: Yeah, I have to ... I mean, it is your 100th episode, so hopefully I can give you some kudos, but if celebrating others' work is one of your main tenets, you nail it every time. So ...Jeremy: I appreciate that.Rebecca: Just wanted you to know that. So, that's sort of the Genesis of course, of both of these, right?Jeremy: Right.Rebecca: That underpins the foundational how to share both works or how to share others' work through different channels. I'm wondering how it transformed, there's this newsletter and then of course it also has this other component, which is Serverless Chats. And that moment when you were like, "All right, this newsletter, this narrative that I'm telling behind serverless, highlighting all of these different authors from all these different global spaces, I'm going to start ... You know what else I want to do? I don't have enough to do, I'm going to start a podcast." How did we get here?Jeremy: Well, so the funny thing is now that I think about it, I think it just goes back to this tenet of fairness, this idea where I was fortunate, and I was able to go down to New York City and go to Serverless Days New York in late 2018. I was able to ... Tom McLaughlin actually got me connected with a bunch of great people in Boston. I live just outside of Boston. We got connected with a bunch of great people. And we started the Serverless Days Boston for 2019. And we were on that committee. I started traveling and I was going to conferences and I was meeting people. I went to re:Invent in 2018, which I know a lot of people just don't have the opportunity to do. And the interesting thing was, is that I was pulling aside brilliant people either in the hallway at a conference or more likely for a very long, deep discussion that we would have about something at a pub in Northern Ireland or something like that, right?I mean, these were opportunities that I was getting that I was privileged enough to get. And I'm like, these are amazing conversations. Just things that, for me, I know changed the way I think. And one of the biggest things that I try to do is evolve my thinking. What I thought a year ago is probably not what I think now. Maybe call it flip-flopping, whatever you want to call it. But I think that evolving your thinking is the most progressive thing that you can do and starting to understand as you gain new perspectives. And I was talking to people that I never would have talked to if I was just sitting here in my home office or at the time, I mean, I was at another office, but still, I wasn't getting that context. I wasn't getting that experience. And I wasn't getting those stories that literally changed my mind and made me think about things differently.And so, here I was in this privileged position, being able to talk to these amazing people and in some cases funny, because they're celebrities in their own right, right? I mean, these are the people where other people think of them and it's almost like they're a celebrity. And these people, I think they deserve fame. Don't get me wrong. But like as someone who has been on that side of it as well, it's ... I don't know, it's weird. It's weird to have fans in a sense. I love, again, you can be my friend, you don't have to be my fan. But that's how I felt about ...Rebecca: I'm a fan of my friends.Jeremy: So, a fan and my friend. So, having talked to these other people and having these really deep conversations on serverless and go beyond serverless to me. Actually I had quite a few conversations with some people that have nothing to do with serverless. Actually, Peter Sbarski and I, every time we get together, we only talk about the value of going to college for some reason. I don't know why. It has usually nothing to do with serverless. So, I'm having these great conversations with these people and I'm like, "Wow, I wish I could share these. I wish other people could have this experience," because I can tell you right now, there's people who can't travel, especially a lot of people outside of the United States. They ... it's hard to travel to the United States sometimes.So, these conversations are going on and I thought to myself, I'm like, "Wouldn't it be great if we could just have these conversations and let other people hear them, hopefully without bar glasses clinking in the background. And so I said, "You know what? Let's just try it. Let's see what happens. I'll do a couple of episodes. If it works, it works. If it doesn't, it doesn't. If people are interested, they're interested." But that was the genesis of that, I mean, it just goes back to this idea where I felt a little selfish having conversations and not being able to share them with other people.Rebecca: It's the very Jeremy Daly tenet slogan, right? You got to share it. You got to share it ...Jeremy: Got to share it, right?Rebecca: The more he shares it, it celebrates it. I love that. I think you do ... Yeah, you do a great job giving a megaphone so that more people can hear. So, in case you need a reminder, actually, I'll ask you, I know what the answer is to this, but do you know the answer? What was your very first episode of Serverless Chats? What was the name, and how long did it last?Jeremy: What was the name?Rebecca: Oh yeah. Oh yeah.Jeremy: Oh, well I know ... Oh, I remember now. Well, I know it was Alex DeBrie. I absolutely know that it was Alex DeBrie because ...Rebecca: Correct on that.Jeremy: If nobody, if you do not know Alex DeBrie, not only is he an AWS data hero, as well as the author of The DynamoDB Book, but he's also like the most likable person on the planet too. It is really hard if you've ever met Alex, that you wouldn't remember him. Alex and I started communicating, again, we met through the serverless space. I think actually he was working at Serverless Inc. at the time when we first met. And I think I met him in person, finally met him in person at re:Invent 2018. But he and I have collaborated on a number of things and so forth. So, let me think what the name of it was. "Serverless Purity Versus Practicality" or something like that. Is that close?Rebecca: That's exactly what it was.Jeremy: Oh, all right. I nailed it. Nailed it. Yes!Rebecca: Wow. Well, it's a great title. And I think ...Jeremy: Don't ask me what episode number 27 was though, because no way I could tell you that.Rebecca: And just for fun, it was 34 minutes long and you released it on June 17th, 2019. So, you've come a long way in a year and a half. That's some kind of wildness. So it makes sense, like, "THE," capital, all caps, bold, italic, author for databases, Alex DeBrie. Makes sense why you selected him as your guest. I'm wondering if you remember any of the ... What do you remember most about that episode? What was it like planning it? What was the reception of it? Anything funny happened recording it or releasing it?Jeremy: Yeah, well, I mean, so the funny thing is that I was incredibly nervous. I still am, actually a lot of guests that I have, I'm still incredibly nervous when I'm about to do the actual interview. And I think it's partially because I want to do justice to the content that they're presenting and to their expertise. And I feel like there's a responsibility to them, but I also feel like the guests that I've had on, some of them are just so smart, and the things they say, just I'm in awe of some of the things that come out of these people's mouths. And I'm like, "This is amazing and people need to hear this." And so, I feel like we've had really good episodes and we've had some okay episodes, but I feel like I want to try to keep that level up so that they owe that to my listener to make sure that there is high quality episode that, high quality information that they're going to get out of that.But going back to the planning of the initial episodes, so I actually had six episodes recorded before I even released the first one. And the reason why I did that was because I said, "All right, there's no way that I can record an episode and then wait a week and then record another episode and wait a week." And I thought batching them would be a good idea. And so, very early on, I had Alex and I had Nitzan Shapira and I had Ran Ribenzaft and I had Marcia Villalba and I had Erik Peterson from Cloud Zero. And so, I had a whole bunch of these episodes and I reached out to I think, eight or nine people. And I said, "I'm doing this thing, would you be interested in it?" Whatever, and we did planning sessions, still a thing that I do today, it's still part of the process.So, whenever I have a guest on, if you are listening to an episode and you're like, "Wow, how did they just like keep the thing going ..." It's not scripted. I don't want people to think it's scripted, but it is, we do review the outline and we go through some talking points to make sure that again, the high-quality episode and that the guest says all the things that the guest wants to say. A lot of it is spontaneous, right? I mean, the language is spontaneous, but we do, we do try to plan these episodes ahead of time so that we make sure that again, we get the content out and we talk about all the things we want to talk about. But with Alex, it was funny.He was actually the first of the six episodes that I recorded, though. And I wasn't sure who I was going to do first, but I hadn't quite picked it yet, but I recorded with Alex first. And it was an easy, easy conversation. And the reason why it was an easy conversation was because we had talked a number of times, right? It was that in a pub, talking or whatever, and having that friendly chat. So, that was a pretty easy conversation. And I remember the first several conversations I had, I knew Nitzan very well. I knew Ran very well. I knew Erik very well. Erik helped plan Serverless Days Boston with me. And I had known Marcia very well. Marcia actually had interviewed me when we were in Vegas for re:Invent 2018.So, those were very comfortable conversations. And so, it actually was a lot easier to do, which probably gave me a false sense of security. I was like, "Wow, this was ... These came out pretty well." The conversations worked pretty well. And also it was super easy because I was just doing audio. And once you add the video component into it, it gets a little bit more complex. But yeah, I mean, I don't know if there's anything funny that happened during it, other than the fact that I mean, I was incredibly nervous when we recorded those, because I just didn't know what to expect. If anybody wants to know, "Hey, how do you just jump right into podcasting?" I didn't. I actually was planning on how can I record my voice? How can I get comfortable behind a microphone? And so, one of the things that I did was I started creating audio versions of my blog posts and posting them on SoundCloud.So, I did that for a couple of ... I'm sorry, a couple of blog posts that I did. And that just helped make me feel a bit more comfortable about being able to record and getting a little bit more comfortable, even though I still can't stand the sound of my own voice, but hopefully that doesn't bother other people.Rebecca: That is an amazing ... I think we so often talk about ideas around you know where you want to go and you have this vision and that's your goal. And it's a constant reminder to be like, "How do I make incremental steps to actually get to that goal?" And I love that as a life hack, like, "Hey, start with something you already know that you wrote and feel comfortable in and say it out loud and say it out loud again and say it out loud again." And you may never love your voice, but you will at least feel comfortable saying things out loud on a podcast.Jeremy: Right, right, right. I'm still working on the, "Ums" and, "Ahs." I still do that. And I don't edit those out. That's another thing too, actually, that one of the things I do want people to know about this podcast is these are authentic conversations, right? I am probably like ... I feel like I'm, I mean, the most authentic person that I know. I just want authenticity. I want that out of the guests. The idea of putting together an outline is just so that we can put together a high quality episode, but everything is authentic. And that's what I want out of people. I just want that authenticity, and one of the things that I felt kept that, was leaving in, "Ums" and, "Ahs," you know what I mean? It's just, it's one of those things where I know a lot of podcasts will edit those out and it sounds really polished and finished.Again, I mean, I figured if we can get the clinking glasses out from the background of a bar and just at least have the conversation that that's what I'm trying to achieve. And we do very little editing. We do cut things out here and there, especially if somebody makes a mistake or they want to start something over again, we will cut that out because we want, again, high quality episodes. But yeah, but authenticity is deeply important to me.Rebecca: Yeah, I think it probably certainly helps that neither of us are robots because robots wouldn't say, "Um" so many times. As I say, "Uh." So, let's talk about, Alex DeBrie was your first guest, but there's been a hundred episodes, right? So, from, I might say the best guest, as a hundredth episode guests, which is our very own Jeremy Daly, but let's go back to ...Jeremy: I appreciate that.Rebecca: Your guests, one to 99. And I mean, you've chatted with some of the most thoughtful, talented, Serverless builders and architects in the industry, and across coincident spaces like ML and Voice Technology, Chaos Engineering, databases. So, you started with Alex DeBrie and databases, and then I'm going to list off some names here, but there's so many more, right? But there's the Gunnar Grosches, and the Alexandria Abbasses, and Ajay Nair, and Angela Timofte, James Beswick, Chris Munns, Forrest Brazeal, Aleksandar Simovic, and Slobodan Stojanovic. Like there are just so many more. And I'm wondering if across those hundred conversations, or 99 plus your own today, if you had to distill those into two or three lessons, what have you learned that sticks with you? If there are emerging patterns or themes across these very divergent and convergent thinkers in the serverless space?Jeremy: Oh, that's a tough question.Rebecca: You're welcome.Jeremy: So, yeah, put me on the spot here. So, yeah, I mean, I think one of the things that I've, I've seen, no matter what it's been, whether it's ML or it's Chaos Engineering, or it's any of those other observability and things like that. I think the common thing that threads all of it is trying to solve problems and make people's lives easier. That every one of those solutions is like, and we always talk about abstractions and, and higher-level abstractions, and we no longer have to write ones and zeros on punch cards or whatever. We can write languages that either compile or interpret it or whatever. And then the cloud comes along and there's things we don't have to do anymore, that just get taken care of for us.And you keep building these higher level of abstractions. And I think that's a lot of what ... You've got this underlying concept of letting somebody else handle things for you. And then you've got this whole group of people that are coming at it from a number of different angles and saying, "Well, how will that apply to my use case?" And I think a lot of those, a lot of those things are very, very specific. I think things like the voice technology where it's like the fact that serverless powers voice technology is only interesting in the fact as to say that, the voice technology is probably the more interesting part, the fact that serverless powers it is just the fact that it's a really simple vehicle to do that. And basically removes this whole idea of saying I'm building voice technology, or I'm building a voice app, why do I need to worry about setting up servers and all this kind of stuff?It just takes that away. It takes that out of the equation. And I think that's the perfect idea of saying, "How can you take your use case, fit serverless in there and apply it in a way that gets rid of all that extra overhead that you shouldn't have to worry about." And the same thing is true of machine learning. And I mean, and SageMaker, and things like that. Yeah, you're still running instances of it, or you still have to do some of these things, but now there's like SageMaker endpoints and some other things that are happening. So, it's moving in that direction as well. But then you have those really high level services like NLU API from IBM, which is the Watson Natural Language Processing.You've got AP recognition, you've got the vision API, you've got sentiment analysis through all these different things. So, you've got a lot of different services that are very specific to machine learning and solving a discrete problem there. But then basically relying on serverless or at least presenting it in a way that's serverless, where you don't have to worry about it, right? You don't have to run all of these Jupiter notebooks and things like that, to do machine learning for a lot of cases. This is one of the things I talk about with Alexandra Abbas, was that these higher level APIs are just taking a lot of that responsibility or a lot of that heavy lifting off of your plate and allowing you to really come down and focus on the things that you're doing.So, going back to that, I do think that serverless, that the common theme that I see is that this idea of worrying about servers and worrying about patching things and worrying about networking, all that stuff. For so many people now, that's just not even a concern. They didn't even think about it. And that's amazing to think of, compute ... Or data, or networking as a utility that is now just available to us, right? And I mean, again, going back to my roots, taking it for granted is something that I think a lot of people do, but I think that's also maybe a good thing, right? Just don't think about it. I mean, there are people who, they're still going to be engineers and people who are sitting in the data center somewhere and racking servers and doing it, that's going to be forever, right?But for the things that you're trying to build, that's unimportant to you. That is the furthest from your concern. You want to focus on the problem that you're trying to solve. And so I think that, that's a lot of what I've seen from talking to people is that they are literally trying to figure out, "Okay, how do I take what I'm doing, my use case, my problem, how do I take that to the next level, by being able to spend my cycles thinking about that as opposed to how I'm going to serve it up to people?"Rebecca: Yeah, I think it's the mantra, right, of simplify, simplify, simplify, or maybe even to credit Bruce Lee, be like water. You're like, "How do I be like water in this instance?" Well, it's not to be setting up servers, it's to be doing what I like to be doing. So, you've interviewed these incredible folks. Is there anyone left on your list? I'm sure there ... I mean, I know that you have a large list. Is there a few key folks where you're like, "If this is the moment I'm going to ask them, I'm going to say on the hundredth episode, 'Dear so-and-so, I would love to interview you for Serverless Chats.'" Who are you asking?Jeremy: So, this is something that, again, we have a stretch list of guests that we attempt to reach out to every once in a while just to say, "Hey, if we get them, we get them." But so, I have a long list of people that I would absolutely love to talk to. I think number one on my list is certainly Werner Vogels. I mean, I would love to talk to Dr. Vogels about a number of things, and maybe even beyond serverless, I'm just really interested. More so from a curiosity standpoint of like, "Just how do you keep that in your head?" That vision of where it's going. And I'd love to drill down more into the vision because I do feel like there's a marketing aspect of it, that's pushing on him of like, "Here's what we have to focus on because of market adoption and so forth. And even though the technology, you want to move into a certain way," I'd be really interesting to talk to him about that.And I'd love to talk to him more too about developer experience and so forth, because one of the things that I love about AWS is that it gives you so many primitives, but at the same time, the thing I hate about AWS is it gives you so many primitives. So, you have to think about 800 services, I know it's not that many, but like, what is it? 200 services, something like that, that all need to kind of connect together. And I love that there's that diversity in those capabilities, it's just from a developer standpoint, it's really hard to choose which ones you're supposed to use, especially when several services overlap. So, I'm just curious. I mean, I'd love to talk to him about that and see what the vision is in terms of, is that the idea, just to be a salad bar, to be the Golden Corral of cloud services, I guess, right?Where you can choose whatever you want and probably take too much and then not use a lot of it. But I don't know if that's part of the strategy, but I think there's some interesting questions, could dig in there. Another person from AWS that I actually want to talk to, and I haven't reached out to her yet just because, I don't know, I just haven't reached out to her yet, but is Brigid Johnson. She is like an IAM expert. And I saw her speak at re:Inforce 2019, it must have been 2019 in Boston. And it was like she was speaking a different language, but she knew IAM so well, and I am not a fan of IAM. I mean, I'm a fan of it in the sense that it's necessary and it's great, but I can't wrap my head around so many different things about it. It's such a ...It's an ongoing learning process and when it comes to things like being able to use tags to elevate permissions. Just crazy things like that. Anyways, I would love to have a conversation with her because I'd really like to dig down into sort of, what is the essence of IAM? What are the things that you really have to think about with least permission? Especially applying it to serverless services and so forth. And maybe have her help me figure out how to do some of the cross role IAM things that I'm trying to do. Certainly would love to speak to Jeff Barr. I did meet Jeff briefly. We talked for a minute, but I would love to chat with him.I think he sets a shining example of what a developer advocate is. Just the way that ... First of all, he's probably the only person alive who knows every service at AWS and has actually tried it because he writes all those blog posts about it. So that would just be great to pick his brain on that stuff. Also, Adrian Cockcroft would be another great person to talk to. Just this idea of what he's done with microservices and thinking about the role, his role with Netflix and some of those other things and how all that kind of came together, I think would be a really interesting conversation. I know I've seen this in so many of his presentations where he's talked about the objections, what were the objections of Lambda and how have you solved those objections? And here's the things that we've done.And again, the methodology of that would be really interesting to know. There's a couple of other people too. Oh, Sam Newman who wrote Building Microservices, that was my Bible for quite some time. I had it on my iPad and had a whole bunch of bookmarks and things like that. And if anybody wants to know, one of my most popular posts that I've ever written was the ... I think it was ... What is it? 16, 17 architectural patterns for serverless or serverless microservice patterns on AWS. Can't even remember the name of my own posts. But that post was very, very popular. And that even was ... I know Matt Coulter who did the CDK. He's done the whole CDK ... What the heck was that? The CDKpatterns.com. That was one of the things where he said that that was instrumental for him in seeing those patterns and being able to use those patterns and so forth.If anybody wants to know, a lot of those patterns and those ideas and those ... The sort of the confidence that I had with presenting those patterns, a lot of that came from Sam Newman's work in his Building Microservices book. So again, credit where credit is due. And I think that that would be a really fascinating conversation. And then Simon Wardley, I would love to talk to. I'd actually love to ... I actually talked to ... I met Lin Clark in Vegas as well. She was instrumental with the WebAssembly stuff, and I'd love to talk to her. Merritt Baer. There's just so many people. I'm probably just naming too many people now. But there are a lot of people that I would love to have a chat with and just pick their brain.And also, one of the things that I've been thinking about a lot on the show as well, is the term "serverless." Good or bad for some people. Some of the conversations we have go outside of serverless a little bit, right? There's sort of peripheral to it. I think that a lot of things are peripheral to serverless now. And there are a lot of conversations to be had. People who were building with serverless. Actually real-world examples.One of the things I love hearing was Yan Cui's "Real World Serverless" podcast where he actually talks to people who are building serverless things and building them in their organizations. That is super interesting to me. And I would actually love to have some of those conversations here as well. So if anyone's listening and you have a really interesting story to tell about serverless or something peripheral to serverless please reach out and send me a message and I'd be happy to talk to you.Rebecca: Well, good news is, it sounds like A, we have at least ... You've got at least another a hundred episodes planned out already.Jeremy: Most likely. Yeah.Rebecca: And B, what a testament to Sam Newman. That's pretty great when your work is referred to as the Bible by someone. As far as in terms of a tome, a treasure trove of perhaps learnings or parables or teachings. I ... And wow, what a list of other folks, especially AWS power ... Actually, not AWS powerhouses. Powerhouses who happened to work at AWS. And I think have paved the way for a ton of ways of thinking and even communicating. Right? So I think Jeff Barr, as far as setting the bar, raising the bar if you will. For how to teach others and not be so high-level, or high-level enough where you can follow along with him, right? Not so high-level where it feels like you can't achieve what he's showing other people how to do.Jeremy: Right. And I just want to comment on the Jeff Barr thing. Yeah.Rebecca: Of course.Jeremy: Because again, I actually ... That's my point. That's one of the reasons why I love what he does and he's so perfect for that position because he's relatable and he presents things in a way that isn't like, "Oh, well, yeah, of course, this is how you do this." I mean, it's not that way. It's always presented in a way to make it accessible. And even for services that I'm not interested in, that I know that I probably will never use, I generally will read Jeff's post because I feel it gives me a good overview, right?Rebecca: Right.Jeremy: It just gives me a good overview to understand whether or not that service is even worth looking at. And that's certainly something I don't get from reading the documentation.Rebecca: Right. He's inviting you to come with him and understanding this, which is so neat. So I think ... I bet we should ... I know that we can find all these twitter handles for these folks and put them in the show notes. And I'm especially ... I'm just going to say here that Werner Vogels's twitter handle is @Werner. So maybe for your hundredth, all the listeners, everyone listening to this, we can say, "Hey, @Werner, I heard that you're the number one guest that Jeremy Daly would like to interview." And I think if we get enough folks saying that to @Werner ... Did I say that @Werner, just @Werner?Jeremy: I think you did.Rebecca: Anyone if you can hear it.Jeremy: Now listen, he did retweet my serverless musical that I did. So ...Rebecca: That's right.Jeremy: I'm sort of on his radar maybe.Rebecca: Yeah. And honestly, he loves serverless, especially with the number of customers and the types of customers and ... that are doing incredible things with it. So I think we've got a chance, Jeremy. I really do. That's what I'm trying to say.Jeremy: That's good to know. You're welcome anytime. He's welcome anytime.Rebecca: Do we say that @Werner, you are welcome anytime. Right. So let's go back to the genesis, not necessarily the genesis of the concept, right? But the genesis of the technology that spurred all of these other technologies, which is AWS Lambda. And so what ... I don't think we'd be having these conversations, right, if AWS Lambda was not released in late 2014, and then when GA I believe in 2015.Jeremy: Right.Rebecca: And so subsequently the serverless paradigm was thrust into the spotlight. And that seems like eons ago, but also three minutes ago.Jeremy: Right.Rebecca: And so I'm wondering ... Let's talk about its evolution a bit and a bit of how if you've been following it for this long and building it for this long, you've covered topics from serverless CI/CD pipelines, observability. We already talked about how it's impacted voice technologies or how it's made it easy. You can build voice technology without having to care about what that technology is running on.Jeremy: Right.Rebecca: You've even talked about things like the future and climate change and how it relates to serverless. So some of those sort of related conversations that you were just talking about wanting to have or having had with previous guests. So as a host who thinks about these topics every day, I'm wondering if there's a topic that serverless hasn't touched yet or one that you hope it will soon. Those types of themes, those threads that you want to pull in the next 100 episodes.Jeremy: That's another tough question. Wow. You got good questions.Rebecca: That's what I said. Heavy hitters. I told you I'd be bringing it.Jeremy: All right. Well, I appreciate that. So that's actually a really good question. I think the evolution of serverless has seen its ups and downs. I think one of the nice things is you look at something like serverless that was so constrained when it first started. And it still has constraints, which are good. But it ... Those constraints get lifted. We just talked about Adrian's talks about how it's like, "Well, I can't do this, or I can't do that." And then like, "Okay, we'll add some feature that you can do that and you can do that." And I think that for the most part, and I won't call it anything specific, but I think for the most part that the evolution of serverless and the evolution of Lambda and what it can do has been thoughtful. And by that I mean that it was sort of like, how do we evolve this into a way that doesn't create too much complexity and still sort of holds true to the serverless ethos of sort of being fairly easy or just writing code.And then, but still evolve it to open up these other use cases and edge cases. And I think that for the most part, that it has held true to that, that it has been mostly, I guess, a smooth ride. There are several examples though, where it didn't. And I said I wasn't going to call anything out, but I'm going to call this out. I think RDS proxy wasn't great. I think it works really well, but I don't think that's the solution to the problem. And it's a band-aid. And it works really well, and congrats to the engineers who did it. I think there's a story about how two different teams were trying to build it at the same time actually. But either way, I look at that and I say, "That's a good solution to the problem, but it's not the solution to the problem."And so I think serverless has stumbled in a number of ways to do that. I also feel EFS integration is super helpful, but I'm not sure that's the ultimate goal to share ... The best way to share state. But regardless, there are a whole bunch of things that we still need to do with serverless. And a whole bunch of things that we still need to add and we need to build, and we need to figure out better ways to do maybe. But I think in terms of something that doesn't get talked about a lot, is the developer experience of serverless. And that is, again I'm not trying to pitch anything here. But that's literally what I'm trying to work on right now in my current role, is just that that developer experience of serverless, even though there was this thoughtful approach to adding things, to try to check those things off the list, to say that it can't do this, so we're going to make it be able to do that by adding X, Y, and Z.As amazing as that has been, that has added layers and layers of complexity. And I'll go back way, way back to 1997 in my dorm room. CGI-bins, if people are not familiar with those, essentially just running on a Linux server, it was a way that it would essentially run a Perl script or other types of scripts. And it was essentially like you're running PHP or you're running Node, or you're running Ruby or whatever it was. So it would run a programming language for you, run a script and then serve that information back. And of course, you had to actually know ins and outs, inputs and outputs. It was more complex than it is now.But anyways, the point is that back then though, once you had the script written. All you had to do is ... There's a thing called FTP, which I'm sure some people don't even know what that is anymore. File transfer protocol, where you would basically say, take this file from my local machine and put it on this server, which is a remote machine. And you would do that. And the second you did that, magically it was updated and you had this thing happening. And I remember there were a lot of jokes way back in the early, probably 2017, 2018, that serverless was like the new CGI-bin or something like that. But more as a criticism of it, right? Or it's just CGI-bins reborn, whatever. And I actually liked that comparison. I felt, you know what? I remember the days where I just wrote code and I just put it to some other server where somebody was dealing with it, and I didn't even have to think about that stuff.We're a long way from that now. But that's how serverless felt to me, one of the first times that I started interacting with it. And I felt there was something there, that was something special about it. And I also felt the constraints of serverless, especially the idea of not having state. People rely on things because they're there. But when you don't have something and you're forced to think differently and to make a change or find a way to work around it. Sometimes workarounds, turn into best practices. And that's one of the things that I saw with serverless. Where people were figuring out pretty quickly, how to build applications without state. And then I think the problem is that you had a lot of people who came along, who were maybe big customers of AWS. I don't know.I'm not going to say that you might be influenced by large customers. I know lots of places are. That said, "We need this." And maybe your ... The will gets bent, right. Because you just... you can only fight gravity for so long. And so those are the kinds of things where I feel some of the stuff has been patchwork and those patchwork things haven't ruined serverless. It's still amazing. It's still awesome what you can do within the course. We're still really just focusing on fast here, with everything else that's built. With all the APIs and so forth and everything else that's serverless in the full-service ecosystem. There's still a lot of amazing things there. But I do feel we've become so complex with building serverless applications, that you can't ... the Hello World is super easy, but if you're trying to build an actual application, it's a whole new mindset.You've got to learn a whole bunch of new things. And not only that, but you have to learn the cloud. You have to learn all the details of the cloud, right? You need to know all these different things. You need to know cloud formation or serverless framework or SAM or something like that, in order to get the stuff into the cloud. You need to understand the infrastructure that you're working with. You may not need to manage it, but you still have to understand it. You need to know what its limitations are. You need to know how it connects. You need to know what the failover states are like.There's so many things that you need to know. And to me, that's a burden. And that's adding new types of undifferentiated heavy-lifting that shouldn't be there. And that's the conversation that I would like to have continuing to move forward is, how do you go back to a developer experience where you're saying you're taking away all this stuff. And again, to call out Werner again, he constantly says serverless is about writing code, but ask anybody who builds serverless applications. You're doing a lot more than writing code right now. And I would love to see us bring the conversation back to how do we get back there?Rebecca: Yeah. I think it kind of goes back to ... You and I have talked about this notion of an ode to simplicity. And it's sort of what you want to write into your ode, right? If we're going to have an ode to simplicity, how do we make sure that we keep the simplicity inside of the ode?Jeremy: Right.Rebecca:So I've got ... I don't know if you've seen these.Jeremy: I don't know.Rebecca: But before I get to some wrap-up questions more from the brainwaves of Jeremy Daly, I don't want to forget to call out some long-time listener questions. And they wrote in a via Twitter and they wanted to perhaps pick your brain on a few things.Jeremy: Okay.Rebecca: So I don't know if you're ready for this.Jeremy: A-M-A. A-M-A.Rebecca: I don't know if you've seen these. Yeah, these are going to put you in the ...Jeremy: A-M-A-M. Wait, A-M-A-A? Asked me almost anything? No, go ahead. Ask me anything.Rebecca: A-M-A-A. A-M-J. No. Anyway, we got it. Ask Jeremy almost anything.Jeremy: There you go.Rebecca: So there's just three to tackle for today's episode that I'm going to lob at you. One is from Ken Collins. "What will it take to get you back to a relational database of Lambda?"Jeremy: Ooh, I'm going to tell you right now. And without a doubt, Aurora Serverless v2. I played around with that right after re:Invent 2000. What was it? 20. Yeah. Just came out, right? I'm trying to remember what year it is at this point.Rebecca: Yes. Indeed.Jeremy: When that just ... Right when that came out. And I had spent a lot of time with Aurora Serverless v1, I guess if you want to call it that. I spent a lot of time with it. I used it on a couple of different projects. I had a lot of really good success with it. I had the same pains as everybody else did when it came to scaling and just the slowness of the scaling and then ... And some of the step-downs and some of those things. There were certainly problems with it. But v2 just the early, early preview version of v2 was ... It was just a marvel of engineering. And the way that it worked was just ... It was absolutely fascinating.And I know it's getting ready or it's getting close, I think, to being GA. And when that becomes GA, I think I will have a new outlook on whether or not I can fit RDS into my applications. I will say though. Okay. I will say, I don't think that transactional applications should be using relational databases though. One of the things that was sort of a nice thing about moving to serverless, speak

In this episode, Yan Cui answers 10 questions about cloud.Yan is an AWS Serverless Hero and the author of Production-Ready Serverless. He helps organizations go faster and deliver more with less.Imagine if your feature velocity goes from months to days, and your systems become more scalable, more secure, more resilient, AND cheaper to run!MEET YAN CUI➡️  Twitter: https://twitter.com/theburningmonk➡️  LinkedIn: https://www.linkedin.com/in/theburningmonk/➡️  Youtube: https://www.youtube.com/channel/UCd2PaRjI5iAGgeld3lCFPNg➡️  Podcast: https://realworldserverless.com/➡️  Github: https://github.com/theburningmonk➡️  Website: https://theburningmonk.com/MEET PABLO PUIG➡️ LinkedIn: https://www.linkedin.com/in/pablo-puig-433295171/PODCAST "10 QUESTIONS ABOUT CLOUD"➡️ Web: https://roopu.cloud/podcast➡️ Spotify: https://open.spotify.com/show/4kH3z7x0Eydh1lBlvncLyQ➡️ Apple Podcasts: https://podcasts.apple.com/us/podcast/roopu-clouds-podcast/id1539635929MEET ROOPU CLOUD➡️ https://roopu.cloud#cloud #cloudcomputing #podcast #roopucloud

Sources (both podcasts are fully transcripted)- Serverless Properties with Johann Schleier-Smith: https://softwareengineeringdaily.com/2021/02/11/serverless-properties-with-johann-schleier-smith/- Azure Functions with Jeff Hollan: https://www.serverlesschats.com/88/- This is all you need to know about Lambda cold starts by Yan Cui: https://lumigo.io/blog/this-is-all-you-need-to-know-about-lambda-cold-starts/---Serverless Properties with Johann Schleier-Smith: https://softwareengineeringdaily.com/2021/02/11/serverless-properties-with-johann-schleier-smith/"The cold start problem got a lot of attention early on. And I'm happy to say that I think that for a lot of practical purposes is something that people can either consider resolved or worked around sufficiently that they don't need to worry about it so much. But let me go intoa little bit more detail on that. So what is the cold start problem? Well, in order to provide these secure execution environments, the cloud provider needs to create a VM for your workload, because that's really how you can guarantee that you're not going to be exposed to otherclients, other tenants.And so booting a VM traditionally means booting and operating system. Operating systems just simply aren't designed to boot up super-fast. It's not something that really matters. You're usually happy, or traditionally you'd be okay if a server booted up within a few minutes,because it's going to run for days. So what does it matter? And traditionally also the things that happen during boot up time involve things like probing for devices and figuring out whether you've upgraded the hardware and other things that have just no role in a serverlessenvironment. You know what the hardware is and you want to get going as quickly as possible because you want to be able to have that ability to expand elastically. And similarly in order to keep costs low you want to have that ability to just shut things off and effectively power down.And so what the cold start is really about is it's about that time that it takes. And to be clear, also, what's important about a cold start versus a warm start is that when you have a – Once you start it up, what you can do is you can just leave that function instance you. Can just leave it running so that it can do more than one request so you get to amortize your startup cost over many, many requests. So sort of two reasons why this startup time is becoming less of an issue, and they're actually both related to the technology that's in Firecracker.So Firecracker makes it much, much faster to boot up the VMs in part because it sort of strips down that kernel so that it has just simply has a much faster boot time. And so the boot times are, instead of seconds, they come down to something like 100 milliseconds or so. And there are a number of other techniques. Some of these are in Firecrackers. Some of these are in research papers that are about making these boot up processes much faster. For example, one thing that you can do is once you have booted an image of a virtual machine, what you can actually do is you could just save those pages essentially, save a state of the memory. And then when you need another one, you can simply clone that and you can use sort of copy and write semantics for that as well so that really you're just creating a new set of page tables to reference that underlying image."---This is all you need to know about Lambda cold starts by Yan Cui: https://lumigo.io/blog/this-is-all-you-need-to-know-about-lambda-cold-starts/---Azure Functions with Jeff Hollan: https://www.serverlesschats.com/88/In the last six months, we actually have been rolling out some machine learning, too. So we've got some folks in Microsoft Research who worked at looking at a bunch of historical data for functions. It's actually all open source. It's anonymized. But if you go to GitHub, you can actually see a bunch of Azure Functions anonymized data. And they trained a bunch of models. So that hopefully, Jeremy, if you were using Azure Functions and it's Monday at 8:00 AM, that our model, hopefully, would get smart enough over time to say, "Oh, there's a 70% chance that at Monday at 8:00 AM. Jeremy's about to hit this thing. We're actually just going to warm it up before he even executes it." So that's something that we've been rolling with for a while. But even then the ... And then just trying to make progress on the underlying technology, the underlying platform. There's a lot of components to building a multi-tenant secured service that all add a little bit of a national latency.So something we're aware of. And then I guess to the second part of that question is, we do have some options to fully mitigate it or partially mitigate it. The one is the fateful pinger. We have folks, I mentioned, you can create this Function app concept. You can have multiple functions in there. One thing that even I have done, and I would say don't quote me on this, but I'm on a podcast. Now, my name's right there. You can create another function in that same app that triggers on a timer. So a timer is a first-class concept in Functions. Just have that thing trigger once every 10 minutes, and your whole app is going to get poked every 10 minutes by us. You don't even have to poke it. We'll poke it ourselves on that interval and keep it warm.

Yan Cui is back for a second round! Ryan Jones sits down with the AWS Serverless Hero, author, and podcast host to discuss his AppSync Masterclass course, no-code vs. low-code, the abstraction of CloudFormation, and much more. Dive into the rabbit hole of serverless with us on this loaded episode that you won't want to miss. If you like this podcast and want more content, visit our website where it's all there and always free: talkingserverless.io You can find more from Yan Cui on his website and be sure to give him a follow on his Twitter: @theburningmonk --- Send in a voice message: https://anchor.fm/talking-serverless/message

What does it take to be consistent on a certain technology and build a career with valiant efforts? Yan Cui, AWS Servels Hero, Developer Advocate at lumigo.io and helping companies around the world adopt serverless as an independent consultant and the host of the Real-world serverless podcast shared his journey behind becoming a burning monk in the cloud ecosystem Book: Serverless Architectures on AWS, Second Edition | Manning Publications Guest: Blog: https://theburningmonk.com/ Podcast: https://realworldserverless.com Masterclass: http://appsyncmasterclass.com LinkedIn: https://www.linkedin.com/in/theburningmonk/ Sponsors: Sundog-Education: https://sundog-education.com Manning Publications: http://mng.bz/RM0v

In this episode, we talk about the past, present, and future of serverless and the cloud with Erica Windisch, principal software engineer at New Relic and founder of IOpipe, and Yan Cui, AWS Serverless Hero and principal consultant at The Burning Monk. Show Notes DevNews (sponsor) Triplebyte (sponsor) CodeNewbie (sponsor) RudderStack (sponsor) The Cloud Serverless New Relic IOpipe Amazon Web Services The Burning Monk Lumigo AWS Lambda OpenStack Eucalyptus Docker Kubernetes Cloudflare Edge Network Container Virtual machine Content Distribution Network (CDN) Edge computing Amazon Elastic Container Service AWS Fargate Managed services Arm NVIDIA NVIDIA to Acquire Arm for $40 Billion

This episode is sponsored by The Chief I/O. The Chief I/O serves Cloud-Native professionals with the knowledge and insights they need to build resilient and scalable systems and teams. Visit The Chief I/O, read our publication, and subscribe to our newsletter and RSS feed. You can also apply to become a writer. Visit www.thechief.io. In November 2017, The Register published an article, 'Lambda and serverless is one of the worst forms of proprietary lock-in we've ever seen in the history of humanity'. The article goes on and elaborates: "It's code that is tied not just to hardware – which we've seen before – but to a data center, you can't even get the hardware yourself. And that hardware is now custom fabbed for the cloud providers with dark fiber that runs all around the world, just for them. So literally, the application you write will never get the performance or responsiveness or the ability to be ported somewhere else without having the deployment footprint of Amazon." What happened next was nothing short of spectacular. Well known figures in the Cloud computing space such as John Arundel, Forrest Brazeal, Yan Cui started to have diverging opinions. Yan Cui is known for his serverless articles in medium and his blog. In an article published in lumigo.com titled “You are wrong about vendor lock-in” he wrote: The biggest misconception about serverless vendor lock-in arguments is that technology choices are never lock-ins. Being “locked in” implies that there is no escape, but that's not the case with technology choices. Not with serverless, not with databases, not with frameworks, not with programming languages. Instead, technology choices create coupling, and no matter the choices you make, your solution will always be coupled to something. Moving away from those technologies requires time and effort, but there is always a way out. I'm your host Kassandra Russel, and today we are going to discuss serverless architectures. We will examine arguments for and against this technology. Next, we will discuss architectures, triggers, and use cases for serverless. Most importantly, we will discuss how to get your serverless functions productionized. This episode is the first part of a series about Serverless; more topics will be discussed in the next episodes. If you are thinking about adopting serverless or if you are already using it, this episode will give you useful insights, so stay tuned. Computing started with bare metal servers, then with virtual machines and later with containers and distributed systems. However, in 2006 a product called Zimki offered the first functions as a Service. This allowed a “pay as you go” model for code execution. Zimki was not commercially successful, but it paved the way for a new business model for computing services. Because of Zimki Functions as a Service or FaaS became a new category in the cloud space. In 2008, Google offered Google App Engine, App Engine allowed “metered billing” for applications. This new offering from Google allowed developers to create functions using a custom Python Framework. The limitation of this is glaringly obvious, developers were not able to execute arbitrary code. In November of 2014, AWS officially announced AWS Lambda. A fully-fledged Functions as a Service Platform that allowed the execution of arbitrary code. In our DevOps weekly newsletter, Shipped, we curate must-read serverless tutorials, news, and stories. Each week, there are tons of articles published, we read them for you, choose the best ones and share them with you. You can subscribe to Shipped by visiting faun.dev/join. --- Send in a voice message: https://anchor.fm/thedevopsfauncast/message Support this podcast: https://anchor.fm/thedevopsfauncast/support

Today on the InfoQ Podcast, Yan Cui (a long time AWS Lambda user and consultant) and Wes Reisz discuss serverless architectures. The conversation starts by focusing on architectural patterns around choreography and orchestration. From there, the two move into updates on the current state of serverless cold start times, distributed tracing, and state. Today’s podcast, while not specific to AWS, does lean heavily on Yan’s expertise with AWS and AWS Lambda. Listen to the podcast for more. Curated transcript and more information on the https://bit.ly/2YPOLj5 Follow us on Facebook, Twitter, LinkedIn, Youtube: @InfoQ Follow us on Instagram: @infoqdotcom Stay informed on emerging trends, peer-validated early adoption of technologies, and architectural best practices. Subscribe to The Software Architects’ Newsletter: www.infoq.com/software-architects-newsletter/

In this episode, Jeremy chats with Yan Cui (aka theburningmonk) about how companies are adopting and implementing serverless, the current state of frameworks and developer tools, and what we should expect from serverless in the future.

In episode #4 of the Talking Serverless podcast Ryan Jones talks to Yan Cui an AWS Serverless Hero and the author of Production-Ready Serverless about ReInvent:2019 and other trends developing in the serverless ecosystem. You can find more from Yan Cui on his website. Twitter --- Send in a voice message: https://anchor.fm/talking-serverless/message

In this episode, we’re joined by AWS Serverless Hero, Yan Cui.Host: Elliot KiplingProduction: Kane HunterCheck us out:WebsiteYouTubeTwitterInstagramLinkedIn

In this episode of CloudSkills.fm I talk to Yan Cui, independent consultant and AWS Serverless Hero who has been running production workloads at scale in AWS since 2009.

רן, אלון, ודותן בפרק מספר 58 של באמפרס (369 קוסמי למניין רברס עם פלטפורמה) - סקירה של טכנלוגיות ודברים מעניינים מהזמן האחרון. רן - מקום נחמד להתחיל - repository בשם Hacker tools - מעניין הקדמה בסיסית אך כתובה היטב לכלים, שמיועדת כביכול להאקרים אבל רלוונטי גם מפתחים, אנשי אופרציה ועוד.מעיין checklist נחמד של כלים שימושיים, שכל מי שנמצא בעולם הזה כנראה כבר מכיר, ועדיין נוח וכתוב טוב - איך עושים backup? איך מתחברים ל-Remote Machine? ענייני אבטחה ועוד הרבה.בלוג-פוסט בשם Operable Software - איך כותבים תוכנה שקל לתפעל?מתחיל בסיסמאות מסוג “Simple is Complex” ואומר שכל מערכת פשוטה בסוף מסתבכת מתחת לפני השטח, כי המציאות יותר מורכבת ככל שמסתכלים מסביב ולעומק - ומה שצריך זה לא בהכרח פשטות אלא מודל מנטלי נכון של המערכת.לדוגמא - מפה של עיר עבור מערכת רכבת תחתית: ההיבט הגיאוגרפי פחות רלוונטי עבור הנוסעים, ויותר התחנות עצמן והקשרים בינהן. זה מודל פחות מדויק גיאוגרפית, אבל מאפשר מודל פשוט יותר עם מידע שקל יותר לתפוס ולהבין.ההשלכה על עולם התוכנה - איסוף לוגים או כל מערכת Monitoring או Visibility לדוגמא, מצריך חשיפה של אלמנטים שונים, אבל חשוב להבין מה חשוב באמת - לא כל שורות הלוג או כל ה - Metrics, על מנת שלא להציף את המשתמש ולהפוך את המערכת ללא שמישה.יכולים להיות כמה מודולים שונים של המערכת עבור משתמשים שונים שחושפים מידע שונה.חשוב לשים לב שההפשטה בשלב הראשון נובעת מהסתכלות על איזורים שונים כעל Black-box, וגם שמערכת משתפרת ומתכנסת עם הזמן (למשל - בתהחלה יש הרבה לוגים ועם הזמן מבינים מה רלוונטי), לפחות בתיאוריה.לא קל למצוא את המודל הנכון ויש הרבה ניסוי וטעייה בהתחלה, אבל מציאת המידע הרלוונטי (והמודל שמדגיש אותו) חשובה על מנת לבנות מערכת שעובדת כמו שצריך - ואת המודל הזה חשוב לבנות מראש.עוד שיקול הוא שבפרטים הקטנים השינויים יותר מהירים - ומודל מנטלי כללי אמור להיות יותר רובסטי ברוב המקרים - וכמובן שצריך לתקף את זה כל הזמן ולראות שעדיין רלוונטי.פרויקט קוד פתוח שרן עבד עליו לא מזמן - Kafka replication test suiteמתוך עבודה עם Multi-regions ורצון לבדוק את יכולות השכפול (Replication) של Kafka בתנאים אמיתיים - כמויות מידע אמיתיות (או לפחות נגזרת שלהם), Clusters אמיתיים, Latency אמיתי (אוקיאנוס בין ארה”ב לאירלנד…) וכו’.בדיקה של ביצועים, האם כל ההודעות באמת מגיעות ליעדן, ועוד.הפרויקט משתמש בשתי טכנולוגיות עיקריות - Kubernetes ו - Go יש Producer ו - Consumer מאוד ספציפיים שיודעים לעבוד אחד עם השני עם פרוטוקול מוגדר וכך אפשר לבדוק שכל ההודעות אכן הגיעו ובסדר הנכון (עם חלוקה פנימית לנושאים)כל זה רץ על שני Clusters שונים של Kafka, עם כלי רפלקיציה ביניהם - בעיקר uReplicator, (במקור מ- Uber). בנוסף, תוך כדי הניסוי מוזרקות תקלות, שיהיה מעניין.אין הנחות מיוחדות - Vanilla clusters של Kubernetes.בסה”כ שני Clusters עם כמה עשרות Nodes בכל אחד, עם תעבורה משמעותית ביניהם -שורה תחתונה:  הכל עובד, לפעמים קצת בעיות Performance.בונוס - הצוות שכתב את uReplicator ב- Uber אהב ואמר זה ישמש לבדיקת הפיצ’רים הבאים :-)כל הטוב הזה הוצג ע”י רן ב-Meetup של Go Israel (בפתח תקווה!), אבל זה כבר קרה בין הקלטת הפרק לפרסום - למתעניינים: יש מצגת.כלי שנקרא Git History - מאפשר לראות את כל ההיסטוריה של Git Repository בצורה ויזואלית נוחה.ה - Commits ושורות הקוד עפים לצדדים, ה - Diff נראה יפה, קצת כמו ויזואליזציה של מסע בזמן (או הכחדת חצי יקום). צעצוע נחמד.עמוד פייסבוק בשם My Tech Life - אנימטור ישראלי שמאייר את חייו כעובד היי-טק בישראל.קריקטורות יצירתיות על ביזנס, תרבות (מישהו אמר משקפיים נגד ספויילרים של משחקי הכס?) ועוד.סאגת AWS / Elasticsearch - כבר קרה לפני יותר מחודש ועדיין - AWS יצאו בהצהרה ש Elastic Search “לא מספיק פתוח”, ושהם יספקו מוצר “אמיתי” פתוח - בחינם.בגדול - לקחו הרבה פיצ’רים High-end והנגישו אותם בחינם. שזה נחמד, רק שזה בדיוק הענף עליו יושב המודל העסקי של Elastic (ורוב חברות הקוד הפתוח).ב-Elastic פחות אהבו את זה.למיטיבי שמע - יש פרק קרבורטור (365) שלם בנושא.תקציר - תעשיית הקוד הפתוח מבוססת על שכבת בסיס חינמית עם פיצ’רים מתקדמים (ותמיכה) בתשלום. ב-AWS אמרו תודה, לקחו את השכבה החינמית והנגישו את הפצ’רים המתקדמים כחלק מהמערכת שלהם, שהיא היום בחינם. המהלך נתפס כמעיין “הפרת אמון” - אותו אמון עליו מבוססת רוב התעשייה (היכולת להרוויח בהמשך ממוצר ששוחרר ללא תשלום בשכבה הבסיסית).אפשר להסתכל על זה מהרבה כיוונים - יש הרבה חברות שעשו את זה זה קודם (מקרים עם MongoDB למשל), ואולי יש כאן קצת(?) תמימות מצד היזמים, ועדיין ברגע שחברה בסדר גודל של AWS עושה את זה יש לכך השפעה מאוד משמעותית על התעשייה, ומוצרי קוד פתוח עשויים להיתקל ביותר קשיים בעתיד.הלקוח יכול לקבל בטווח הקצר יותר פיצ’רים מתקדמים בחינם, השאלה היא האם ימשיכו לפתח הרבה כאלו כשהמודל העסקי מאויים בפועל (ולא רק תיאורטית).עוד מסד נתונים - AresDB: מסד נתונים מבוסס GPU שנכתב ב-Uber ומשמש ל - Analytics.בלוג-פוסט ארוך (21 דקות קריאה לפי Pocket) ומאוד מעניין על אופן המימוש של analytics Database מעל GPU. ספציפית לעבודה עם ה-GPU משתמשים בספרייה קיימת, אבל יש הרבה ארכיטקטורה מעבר.מסבירים למה פתרונות קיימים עם Databases על GPU לא התאימו - מעניין הנדסית וגם ככיוון שנראה יותר ממנו בעתיד.גם כאן הפנייה למיטיבי שמע - פרק מאוד מעניין (ומלא רפרנסים על Skynet) עם Nvidia על GPU.עוד על פוליטיקה של עננים - בלוג מעניין וכתוב היטב שמשווה את המתח בין AWS ו - Kubernetes למתח ההיסטורי בין Windows ו - Linux.לא רעיון חדש, אבל מוסבר יפה - המאבק שבין ה-Vendor וה - Open Source. המציאות יותר מורכבת אבל עדיין מעניין ויפה.פרויקט של Google בשם Pi in the sky שיצא ביום הפאי הבינלאומי (3.14) - חישוב של פאי עד 31.4 טריליון ספרות אחרי הנקודה.מעניין האם מדובר בטריליון אמריקאי או בריטי (כן, מסתבר שזה לא אותו הדבר).ארכיטקטורה, חומרה, וכו’ - נחמד. זמן ריצה? 2,795 machine-days . . . חימום כדור הארץ על הדרך.תוכניות ל - Error handing לקראת Go 2.0 - ב- Go כמעט כל פונקציה מחזירה שני ערכים (או יותר), כשהאחרון הוא ערך השגיאה.ההצעה מדברת על שני Constructs חדשים - check ו -  handle (לא Try ו - Catch . . .), מה שיכול להפוך את הטיפול בשגיאות לקצת יותר זורם ואמין.תאריך ל - Go 2? בינתיים סופרים עדיין בגרסאות של 1.משהו . . .בהמשך לשיחה על Nuclio באחד הבאמפרס הקודמים (והיה את פרק 333 עם ירון חביב) - בלוג פוסט מעניין של Yan Cui, אחד המומחים בתחום של Serverless, שמשווה את Nuclio ו -  AWS Lambdaחזרה לעולם ה - Go ול - Goroutines - שמאפשר להאיץ חישוב לינארי ע”י חלוקה.הבלוג-פוסט הזה מראה שאם לא עושים את זה נכון - החישוב מאט . . . Spoiler Alert - זה נובע ממנגון Cache-lines לאופטימיזציה של ה - CPU, וכן - יש דרך “נכונה” לעשות את זה (באמצעות channels).דותן - התחלה קצת מלנכולית - לזכור את ג’ו. Joe Armstrong, האבא של Erlang (זו שמאחורי כל תשתיות הטלפוניה בעולם, ומשם ל - WhatsApp והרבה דברים שאתם מכירים). פוסט של אנשים שעבדו איתו וראו הכל, וזוכרים אותו כמישהו שכבר ראה את הכל מגיע ורק חיכה לראות המימוש של הרעיונות.הזדמנות להבין את Erlang ואת Elixir שבאה בעקבותיהבאותו הקשר - How Discord Scaled Elixir to 5,000,000 Concurrent Users - מתקשר יפה ומראה את היכולות של Elixir

Yan Cui joins Forrest to unpack the idea of FinDev. What should the finance department and the engineering org have in common, and why is serverless a catalyst for this transformation?

Yan Cui joins Forrest to unpack the idea of FinDev. What should the finance department and the engineering org have in common, and why is serverless a catalyst for this transformation?

Yan Cui joins Forrest to discuss serverless adoption challenges and goals in 2019.

Yan Cui joins Forrest to discuss serverless adoption challenges and goals in 2019.

Yan Cui, creator of the Production-Ready Serverless course and serverless consultant, joins us this episode to school Mike on serverless, talk about the real business value behind why an organization should be interested, and a whole lot of intricate details around this new paradigm.

Yan Cui joins Forrest to share best practices for running serverless apps at production scale.

Yan Cui joins Forrest to share best practices for running serverless apps at production scale.

Are you a blogger? Engineer? Web guru? What do you do? If you ask Yan Cui that question, be prepared for several different answers. Today, we’re talking to Yan, who is a principal engineer at DAZN. Also, he writes blog posts and is a course developer. His insightful, engaging, and understandable content resonates with various audiences. And, he’s an AWS serverless hero! Some of the highlights of the show include: Some people get tripped up because they don’t bring microservice practices they learned into the new world of serverless; face many challenges Educate others and share your knowledge; Yan does, as an AWS hero Chaos Engineering Meeting Serverless: Figuring out what types of failures to practice for depends on what services you are using Environment predicated on specific behaviors may mean enumerating bad things that could happen, instead of building a resilient system that works as planned API Gateway: Confusing for users because it can do so many different things; what is the right thing to do, given a particular context, is not always clear Now, serverless feels like a toy, but good enough to run production workflow; future of serverless - will continue to evolve and offer more flexibility Serverless is used to build applications; DevOps/IOT teams and enterprises are adopting serverless because it makes solutions more cost effective Links: Yan Cui on Twitter DAZN Production-Ready Serverless Theburningmonk.com Applying Principles of Chaos Engineering to Serverless AWS Heroes re:Invent Lambda Amazon S3 Service Disruption API Gateway Ben Kehoe Digital Ocean

Remember when the best game you could play on your phone was Snake? In 1998, Snake was preloaded on Nokia phones, and it was massively popular. That same year Half-Life won game of the year on PC. Metal Gear Solid came out for Playstation. The first version of Starcraft also came out in 1998. In The post Scalable Multiplayer Games with Yan Cui appeared first on Software Engineering Daily.

After raising $18 million, social networking startup Yubl made a series of costly mistakes. Yubl hired an army of expensive contractors to build out its iOS and Android apps. Drama at the executive level hurt morale for the full-time employees. Most problematic, the company was bleeding cash due to a massive over-investment in cloud services. The post Serverless Startup with Yan Cui appeared first on Software Engineering Daily.

After raising $18 million, social networking startup Yubl made a series of costly mistakes. Yubl hired an army of expensive contractors to build out its iOS and Android apps. Drama at the executive level hurt morale for the full-time employees. Most problematic, the company was bleeding cash due to a massive over-investment in cloud services. The post Serverless Startup with Yan Cui appeared first on Software Engineering Daily.

In this episode I talk with Yan Cui. We talk his introduction to functional programming via Erlang and F#, getting functional languages adopted in a company, type providers, we wrap up with a rundown of a number of different languages, and much more in between.

Podcast interview with Yan Cui of GameSys on their use of Neo4j for managing Game Economics

Carl and Richard talk to Yan Cui about his experience building games with F#. Yan talks about his experiences working with games through Facebook, the client side being predominantly Flash, but the back end systems being initially Java, then converted to C#, and now to F#. As Yan explains, F# is faster, smaller, easier to maintain and to read. The conversation spans over dealing with parallelism, massive scale, cloud implementation and data analytics as well - lots of cool stuff!Support this podcast at — https://redcircle.com/net-rocks/donations

Carl and Richard talk to Yan Cui about his experience building games with F#. Yan talks about his experiences working with games through Facebook, the client side being predominantly Flash, but the back end systems being initially Java, then converted to C#, and now to F#. As Yan explains, F# is faster, smaller, easier to maintain and to read. The conversation spans over dealing with parallelism, massive scale, cloud implementation and data analytics as well - lots of cool stuff!Support this podcast at — https://redcircle.com/net-rocks/donations