Podcasts about gregor hohpe

This interview was recorded for the GOTO Book Club.http://gotopia.tech/bookclubRead the full transcription of the interview here:https://gotopia.tech/episodes/331Maciej «MJ» Jedrzejewski - Tech Agnostic Architect & Author of "Master Software Architecture"Artur Skowroński - Head of Java & Kotlin Engineering at VirtusLab & Editor of "JVM Weekly"RESOURCESMJhttps://github.com/meaboutsoftwarehttps://www.linkedin.com/in/jedrzejewski-maciejhttps://www.fractionalarchitect.ioArturhttps://x.com/ArturSkowronskihttps://www.linkedin.com/in/arturskowronskihttps://www.jvm-weekly.comDESCRIPTIONThis conversation explores the evolution and complexities of software architecture, from early programming experiences to advanced design principles.It highlights practical gaps and the value of self-publishing, consulting, and addressing architectural pitfalls. Trends like microservices, serverless computing and AI are examined critically, emphasizing their limitations and supportive roles.Recommendations for further reading include Gregor Hohpe's "Software Architect Elevator", Martin Kleppmann's "Designing Data-Intensive Applications", "Software Architecture: The Hard Parts" and Nick Tune's "Architecture Modernization," offering deep insights into effective software practices.RECOMMENDED BOOKSMaciej «MJ» Jedrzejewski • Master Software Architecture • https://leanpub.com/master-software-architectureGregor Hohpe • Platform Strategy • https://amzn.to/4cxfYdbGregor Hohpe • The Software Architect Elevator • https://amzn.to/3F6d2axMartin Kleppmann • Designing Data-Intensive Applications • https://amzn.to/3mk2RojFord, Richards, Sadalage & Dehghani • Software Architecture: The Hard Parts • https://amzn.to/3QeMgjRNick Tune & Jean-Georges Perrin • Architecture Modernization • https://amzn.to/4b5ASiNSam Newman • Monolith to Microservices • https://amzn.to/2Nml96EVaughn Vernon & Tomasz Jaskula • Strategic Monoliths & Microservices • https://amzn.to/3AcUscjBlueskyTwitterInstagramLinkedInFacebookCHANNEL MEMBERSHIP BONUSJoin this channel to get early access to videos & other perks:https://www.youtube.com/channel/UCs_tLP3AiwYKwdUHpltJPuA/joinLooking for a unique learning experience?Attend the next GOTO conference near you! Get your ticket: gotopia.techSUBSCRIBE TO OUR YOUTUBE CHANNEL - new videos posted daily!

This interview was recorded for the GOTO Book Club.http://gotopia.tech/bookclubRead the full transcription of the interview here:https://gotopia.tech/episodes/323Gregor Hohpe - Author of "Platform Strategy", "The Software Architect Elevator", et al.James Lewis - Software Architect & Director at ThoughtworksRESOURCESGregorhttps://twitter.com/ghohpehttps://www.linkedin.com/in/ghohpehttps://architectelevator.comJameshttps://twitter.com/boicyhttps://linkedin.com/in/james-lewis-microserviceshttps://github.com/boicyhttps://www.bovon.orgDESCRIPTIONJames Lewis and Gregor Hohpe discuss the concept of dimensionality in decision-making, particularly in the context of innovation versus standardization. Hohpe emphasizes the importance of understanding and removing constraints to unlock new opportunities, citing historical shifts in technology and platform thinking as key examples.They explore how traditional one-dimensional views often limit progress and the challenges of adapting to new paradigms, especially in organizations. The discussion also touches on the role of architects in facilitating these shifts and the strategic focus needed for internal platforms to thrive in the face of evolving technologies.RECOMMENDED BOOKSGregor Hohpe • Platform Strategy • https://amzn.to/4cxfYdbGregor Hohpe • The Software Architect Elevator • https://amzn.to/3F6d2axGregor Hohpe • Cloud Strategy • https://amzn.to/3TOS3NvGregor Hohpe • Enterprise Integration Patterns, Vol 2 • https://amzn.to/3TNedQ3Gregor Hohpe & Bobby Woolf • Enterprise Integration Patterns • https://amzn.to/3DqII9lGregor Hohpe • 37 Things One Architect Knows About IT Transformation • https://amzn.to/3z8uhnwTwitterInstagramLinkedInFacebookLooking for a unique learning experience?Attend the next GOTO conference near you! Get your ticket: gotopia.techSUBSCRIBE TO OUR YOUTUBE CHANNEL - new videos posted daily!

Please Rate and Review us on your podcast app of choice!Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see hereEpisode list and links to all available episode transcripts here.Provided as a free resource by Data Mesh Understanding. Get in touch with Scott on LinkedIn.Transcript for this episode (link) provided by Starburst. You can download their Data Products for Dummies e-book (info-gated) here and their Data Mesh for Dummies e-book (info gated) here.Michael's LinkedIn: https://www.linkedin.com/in/mjtoland/Marta's LinkedIn: https://www.linkedin.com/in/diazmarta/Sadie's LinkedIn: https://www.linkedin.com/in/sadie-martin-06404125/Sean's LinkedIn: https://www.linkedin.com/in/seangustafson/The Magic of Platforms by Gregor Hohpe: https://platformengineering.org/talks-library/the-magic-of-platformsStart with why -- how great leaders inspire action | Simon Sinek: https://www.youtube.com/watch?v=u4ZoJKF_VuAIn this episode, guest host Michael Toland Senior Product Manager at Pathfinder Product Labs/Testdouble and host of the upcoming Data Product Management in Action Podcast facilitated a discussion with Sadie Martin, Product Manager at Fivetran (guest of episode #64), Sean Gustafson, Director of Engineering - Data Platform at Delivery Hero (guest of episode #274), and Marta Diaz, Product Manager Data Platform at Adevinta Spain. As per usual, all guests were only reflecting their own views.The topic for this panel was how to treat your data platform as a product. While many people in the data space are talking about data products, not nearly as many are treating the platform used for creating and managing those data products as a product itself. This is about moving beyond the IT services model for your data work. Platforms have life-cycles and need product management principles too! Also, in data mesh, it is crucial to understand that 'platform' can be plural, it doesn't have to be one monolithic platform, users don't care.Scott note: As per usual, I...

Gregor Hohpe, author of "Enterprise Integration Patterns", talks to Dave Farley about software architecture and how architects can transform businesses. They chat about: Gregor's current role and work with AWS (Amazon Web Services), the challenge of finding new architectural models in the cloud, "Gregor's Law" AND MORE! Thanks to Gregor for joining Dave on this episode of the Engineering Room. xxJOIN PATREON HERE ➡️ https://bit.ly/ContinuousDeliveryPatreon

Gregor Hohpe. is a world-class expert on software architecture and the role of the architect, he is a technologist and expert on the topics of large-scale systems and the public Cloud as well as lots of other stuff. Gregor is currently part of the Serverless team working as an Enterprise Strategist for Amazon at AWS, Previously he was Technical Director in the Office of the CTO at Google, and before that was Chief SW Architect at Allianz the German Insurance giant. Gregor is an international speaker, author of Several great books, as well as writing on his always thought-provoking blog, “The Architect Elevator”. and he's just published a new book called “Platform Strategy”.xxEqual Experts is a product software development consultancy with a network of over 1,000 experienced technology consultants globally. They increase the pace of innovation by using modern software engineering practices that embrace Continuous Delivery, Security, and Operability from the outset ➡️ https://bit.ly/3ASy8n0

Evelyn Osman, Principal Platform Engineer at AutoScout24, joins Corey on Screaming in the Cloud to discuss the dire need for developers to agree on a standardized tool set in order to scale their projects and innovate quickly. Corey and Evelyn pick apart the new products being launched in cloud computing and discover a large disconnect between what the industry needs and what is actually being created. Evelyn shares her thoughts on why viewing platforms as products themselves forces developers to get into the minds of their users and produces a better end result.About EvelynEvelyn is a recovering improviser currently role playing as a Lead Platform Engineer at Autoscout24 in Munich, Germany. While she says she specializes in AWS architecture and integration after spending 11 years with it, in truth she spends her days convincing engineers that a product mindset will make them hate their product managers less.Links Referenced:LinkedIn: https://www.linkedin.com/in/evelyn-osman/TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. My guest today is Evelyn Osman, engineering manager at AutoScout24. Evelyn, thank you for joining me.Evelyn: Thank you very much, Corey. It's actually really fun to be on here.Corey: I have to say one of the big reasons that I was enthused to talk to you is that you have been using AWS—to be direct—longer than I have, and that puts you in a somewhat rarefied position where AWS's customer base has absolutely exploded over the past 15 years that it's been around, but at the beginning, it was a very different type of thing. Nowadays, it seems like we've lost some of that magic from the beginning. Where do you land on that whole topic?Evelyn: That's actually a really good point because I always like to say, you know, when I come into a room, you know, I really started doing introductions like, “Oh, you know, hey,” I'm like, you know, “I'm this director, I've done this XYZ,” and I always say, like, “I'm Evelyn, engineering manager, or architect, or however,” and then I say, you know, “I've been working with AWS, you know, 11, 12 years,” or now I can't quite remember.Corey: Time becomes a flat circle. The pandemic didn't help.Evelyn: [laugh] Yeah, I just, like, a look at that the year, and I'm like, “Jesus. It's been that long.” Yeah. And usually, like you know, you get some odd looks like, “Oh, my God, you must be a sage.” And for me, I'm… you see how different services kind of, like, have just been reinventions of another one, or they just take a managed service and make another managed service around it. So, I feel that there's a lot of where it's just, you know, wrapping up a pretty bow, and calling it something different, it feels like.Corey: That's what I've been low-key asking people for a while now over the past year, namely, “What is the most foundational, interesting thing that AWS has done lately, that winds up solving for this problem of whatever it is you do as a company? What is it that has foundationally made things better that AWS has put out in the last service? What was it?” And the answers I get are all depressingly far in the past, I have to say. What's yours?Evelyn: Honestly, I think the biggest game-changer I remember experiencing was at an analyst summit in Stockholm when they announced Lambda.Corey: That was announced before I even got into this space, as an example of how far back things were. And you're right. That was transformative. That was awesome.Evelyn: Yeah, precisely. Because before, you know, we were always, like, trying to figure, okay, how do we, like, launch an instance, run some short code, and then clean it up. AWS is going to charge for an hour, so we need to figure out, you know, how to pack everything into one instance, run for one hour. And then they announced Lambda, and suddenly, like, holy shit, this is actually a game changer. We can actually write small functions that do specific things.And, you know, you go from, like, microservices, like, to like, tiny, serverless functions. So, that was huge. And then DynamoDB along with that, really kind of like, transformed the entire space for us in many ways. So, back when I was at TIBCO, there was a few innovations around that, even, like, one startup inside TIBCO that quite literally, their entire product was just Lambda functions. And one of their problems was, they wanted to sell in the Marketplace, and they couldn't figure out how to sell Lambda on the marketplace.Corey: It's kind of wild when we see just how far it's come, but also how much they've announced that doesn't change that much, to be direct. For me, one of the big changes that I remember that really made things better for customers—thought it took a couple of years—was EFS. And even that's a little bit embarrassing because all that is, “All right, we finally found a way to stuff a NetApp into us-east-1,” so now NFS, just like you used to use it in the 90s and the naughts, can be done responsibly in the cloud. And that, on some level, wasn't a feature launch so much as it was a concession to the ways that companies had built things and weren't likely to change.Evelyn: Honestly, I found the EFS launch to be a bit embarrassing because, like, you know, when you look closer at it, you realize, like, the performance isn't actually that great.Corey: Oh, it was horrible when it launched. It would just slam to a halt because you got the IOPS scaled with how much data you stored on it. The documentation explicitly said to use dd to start loading a bunch of data onto it to increase the performance. It's like, “Look, just sandbag the thing so it does what you'd want.” And all that stuff got fixed, but at the time it looked like it was clown shoes.Evelyn: Yeah, and that reminds me of, like, EBS's, like, gp2 when we're, like you know, we're talking, like, okay, provision IOPS with gp2. We just kept saying, like, just give yourself really big volume for performance. And it feel like they just kind of kept that with EFS. And it took years for them to really iterate off of that. Yeah, so, like, EFS was a huge thing, and I see us, we're still using it now today, and like, we're trying to integrate, especially for, like, data center migrations, but yeah, you always see that a lot of these were first more for, like, you know, data centers to the cloud, you know. So, first I had, like, EC2 classic. That's where I started. And I always like to tell a story that in my team, we're talking about using AWS, I was the only person fiercely against it because we did basically large data processing—sorry, I forget the right words—data analytics. There we go [laugh].Corey: I remember that, too. When it first came out, it was, “This sounds dangerous and scary, and it's going to be a flash in the pan because who would ever trust their core compute infrastructure to some random third-party company, especially a bookstore?” And yeah, I think I got that one very wrong.Evelyn: Yeah, exactly. I was just like, no way. You know, I see all these articles talking about, like, terrible disk performance, and here I am, where it's like, it's my bread and butter. I'm specialized in it, you know? I write code in my sleep and such.[Yeah, the interesting thing is, I was like, first, it was like, I can 00:06:03] launch services, you know, to kind of replicate when you get in a data center to make it feature comparable, and then it was taking all this complex services and wrapping it up in a pretty bow for—as a managed service. Like, EKS, I think, was the biggest one, if we're looking at managed services. Technically Elasticsearch, but I feel like that was the redheaded stepchild for quite some time.Corey: Yeah, there was—Elasticsearch was a weird one, and still is. It's not a pleasant service to run in any meaningful sense. Like, what people actually want as the next enhancement that would excite everyone is, I want a serverless version of this thing where I can just point it at a bunch of data, I hit an API that I don't have to manage, and get Elasticsearch results back from. They finally launched a serverless offering that's anything but. You have to still provision compute units for it, so apparently, the word serverless just means managed service over at AWS-land now. And it just, it ties into the increasing sense of disappointment I've had with almost all of their recent launches versus what I felt they could have been.Evelyn: Yeah, the interesting thing about Elasticsearch is, a couple of years ago, they came out with OpenSearch, a competing Elasticsearch after [unintelligible 00:07:08] kind of gave us the finger and change the licensing. I mean, OpenSearch actually become a really great offering if you run it yourself, but if you use their managed service, it can kind—you lose all the benefits, in a way.Corey: I'm curious, as well, to get your take on what I've been seeing that I think could only be described as an internal shift, where it's almost as if there's been a decree passed down that every service has to run its own P&L or whatnot, and as a result, everything that gets put out seems to be monetized in weird ways, even when I'd argue it shouldn't be. The classic example I like to use for this is AWS Config, where it charges you per evaluation, and that happens whenever a cloud resource changes. What that means is that by using the cloud dynamically—the way that they supposedly want us to do—we wind up paying a fee for that as a result. And it's not like anyone is using that service in isolation; it is definitionally being used as people are using other cloud resources, so why does it cost money? And the answer is because literally everything they put out costs money.Evelyn: Yep, pretty simple. Oftentimes, there's, like, R&D that goes into it, but the charges seem a bit… odd. Like from an S3 lens, was, I mean, that's, like, you know, if you're talking about services, that was actually a really nice one, very nice holistic overview, you know, like, I could drill into a data lake and, like, look into things. But if you actually want to get anything useful, you have to pay for it.Corey: Yeah. Everything seems to, for one reason or another, be stuck in this place where, “Well, if you want to use it, it's going to cost.” And what that means is that it gets harder and harder to do anything that even remotely resembles being able to wind up figuring out where's the spend going, or what's it going to cost me as time goes on? Because it's not just what are the resources I'm spinning up going to cost, what are the second, third, and fourth-order effects of that? And the honest answer is, well, nobody knows. You're going to have to basically run an experiment and find out.Evelyn: Yeah. No, true. So, what I… at AutoScout, we actually ended up doing is—because we're trying to figure out how to tackle these costs—is they—we built an in-house cost allocation solution so we could track all of that. Now, AWS has actually improved Cost Explorer quite a bit, and even, I think, Billing Conductor was one that came out [unintelligible 00:09:21], kind of like, do a custom tiered and account pricing model where you can kind of do the same thing. But even that also, there is a cost with it.I think that was trying to compete with other, you know, vendors doing similar solutions. But it still isn't something where we see that either there's, like, arbitrarily low pricing there, or the costs itself doesn't really quite make sense. Like, AWS [unintelligible 00:09:45], as you mentioned, it's a terrific service. You know, we try to use it for compliance enforcement and other things, catching bad behavior, but then as soon as people see the price tag, we just run away from it. So, a lot of the security services themselves, actually, the costs, kind of like, goes—skyrockets tremendously when you start trying to use it across a large organization. And oftentimes, the organization isn't actually that large.Corey: Yeah, it gets to this point where, especially in small environments, you have to spend more energy and money chasing down what the cost is than you're actually spending on the thing. There were blog posts early on that, “Oh, here's how you analyze your bill with Redshift,” and that was a minimum 750 bucks a month. It's, well, I'm guessing that that's not really for my $50 a month account.Evelyn: Yeah. No, precisely. I remember seeing that, like, entire ETL process is just, you know, analyze your invoice. Cost [unintelligible 00:10:33], you know, is fantastic, but at the end of the day, like, what you're actually looking at [laugh], is infinitesimally small compared to all the data in that report. Like, I think oftentimes, it's simply, you know, like, I just want to look at my resources and allocate them in a multidimensional way. Which actually isn't really that multidimensional, when you think about it [laugh].Corey: Increasingly, Cost Explorer has gotten better. It's not a new service, but every iteration seems to improve it to a point now where I'm talking to folks, and they're having a hard time justifying most of the tools in the cost optimization space, just because, okay, they want a percentage of my spend on AWS to basically be a slightly better version of a thing that's already improving and works for free. That doesn't necessarily make sense. And I feel like that's what you get trapped into when you start going down the VC path in the cost optimization space. You've got to wind up having a revenue model and an offering that scales through software… and I thought, originally, I was going to be doing something like that. At this point, I'm unconvinced that anything like that is really tenable.Evelyn: Yeah. When you're a small organization you're trying to optimize, you might not have the expertise and the knowledge to do so, so when one of these small consultancies comes along, saying, “Hey, we're going to charge you a really small percentage of your invoice,” like, okay, great. That's, like, you know, like, a few $100 a month to make sure I'm fully optimized, and I'm saving, you know, far more than that. But as soon as your invoice turns into, you know, it's like $100,000, or $300,000 or more, that percentage becomes rather significant. And I've had vendors come to me and, like, talk to me and is like, “Hey, we can, you know, for a small percentage, you know, we're going to do this machine learning, you know, AI optimization for you. You know, you don't have to do anything. We guaranteed buybacks your RIs.” And as soon as you look at the price tag with it, we just have to walk away. Or oftentimes we look at it, and there are truly very simple ways to do it on your own, if you just kind of put some thought into it.Corey: While we want to talking a bit before this show, you taught me something new about GameLift, which I think is a different problem that AWS has been dealing with lately. I've never paid much attention to it because it is the—as I assume from what it says on the tin, oh, it's a service for just running a whole bunch of games at scale, and I'm not generally doing that. My favorite computer game remains to be Twitter at this point, but that's okay. What is GameLift, though, because you want to shining a different light on it, which makes me annoyed that Amazon Marketing has not pointed this out.Evelyn: Yeah, so I'll preface this by saying, like, I'm not an expert on GameLift. I haven't even spun it up myself because there's quite a bit of price. I learned this fall while chatting with an SA who works in the gaming space, and it kind of like, I went, like, “Back up a second.” If you think about, like, I'm, you know, like, World of Warcraft, all you have are thousands of game clients all over the world, playing the same game, you know, on the same server, in the same instance, and you need to make sure, you know, that when I'm running, and you're running, that we know that we're going to reach the same point the same time, or if there's one object in that room, that only one of us can get it. So, all these servers are doing is tracking state across thousands of clients.And GameLift, when you think about your dedicated game service, it really is just multi-region distributed state management. Like, at the basic, that's really what it is. Now, there's, you know, quite a bit more happening within GameLift, but that's what I was going to explain is, like, it's just state management. And there are far more use cases for it than just for video games.Corey: That's maddening to me because having a global session state store, for lack of a better term, is something that so many customers have built themselves repeatedly. They can build it on top of primitives like DynamoDB global tables, or alternately, you have a dedicated region where that thing has to live and everything far away takes forever to round-trip. If they've solved some of those things, why on earth would they bury it under a gaming-branded service? Like, offer that primitive to the rest of us because that's useful.Evelyn: No, absolutely. And honestly, I wouldn't be surprised if you peeled back the curtain with GameLift, you'll find a lot of—like, several other you know, AWS services that it's just built on top of. I kind of mentioned earlier is, like, what I see now with innovation, it's like we just see other services packaged together and releases a new product.Corey: Yeah, IoT had the same problem going on for years where there was a lot of really good stuff buried in there, like IOT events. People were talking about using that for things like browser extensions and whatnot, but you need to be explicitly told that that's a thing that exists and is handy, but otherwise you'd never know it was there because, “Well, I'm not building anything that's IoT-related. Why would I bother?” It feels like that was one direction that they tended to go in.And now they take existing services that are, mmm, kind of milquetoast, if I'm being honest, and then saying, “Oh, like, we have Comprehend that does, effectively detection of themes, keywords, and whatnot, from text. We're going to wind up re-releasing that as Comprehend Medical.” Same type of thing, but now focused on a particular vertical. Seems to me that instead of being a specific service for that vertical, just improve the baseline the service and offer HIPAA compliance if it didn't exist already, and you're mostly there. But what do I know? I'm not a product manager trying to get promoted.Evelyn: Yeah, that's true. Well, I was going to mention that maybe it's the HIPAA compliance, but actually, a lot of their services already have HIPAA compliance. And I've stared far too long at that compliance section on AWS's site to know this, but you know, a lot of them actually are HIPAA-compliant, they're PCI-compliant, and ISO-compliant, and you know, and everything. So, I'm actually pretty intrigued to know why they [wouldn't 00:16:04] take that advantage.Corey: I just checked. Amazon Comprehend is itself HIPAA-compliant and is qualified and certified to hold Personal Health Information—PHI—Private Health Information, whatever the acronym stands for. Now, what's the difference, then, between that and Medical? In fact, the HIPAA section says for Comprehend Medical, “For guidance, see the previous section on Amazon Comprehend.” So, there's no difference from a regulatory point of view.Evelyn: That's fascinating. I am intrigued because I do know that, like, within AWS, you know, they have different segments, you know? There's, like, Digital Native Business, there's Enterprise, there's Startup. So, I am curious how things look over the engineering side. I'm going to talk to somebody about this now [laugh].Corey: Yeah, it's the—like, I almost wonder, on some level, it feels like, “Well, we wound to building this thing in the hopes that someone would use it for something. And well, if we just use different words, it checks a box in some analyst's chart somewhere.” I don't know. I mean, I hate to sound that negative about it, but it's… increasingly when I talk to customers who are active in these spaces around the industry vertical targeted stuff aimed at their industry, they're like, “Yeah, we took a look at it. It was adorable, but we're not using it that way. We're going to use either the baseline version or we're going to work with someone who actively gets our industry.” And I've heard that repeated about three or four different releases that they've put out across the board of what they've been doing. It feels like it is a misunderstanding between what the world needs and what they're able to or willing to build for us.Evelyn: Not sure. I wouldn't be surprised, if we go far enough, it could probably be that it's just a product manager saying, like, “We have to advertise directly to the industry.” And if you look at it, you know, in the backend, you know, it's an engineer, you know, kicking off a build and just changing the name from Comprehend to Comprehend Medical.Corey: And, on some level, too, they're moving a lot more slowly than they used to. There was a time where they were, in many cases, if not the first mover, the first one to do it well. Take Code Whisperer, their AI powered coding assistant. That would have been a transformative thing if GitHub Copilot hadn't beaten them every punch, come out with new features, and frankly, in head-to-head experiments that I've run, came out way better as a product than what Code Whisperer is. And while I'd like to say that this is great, but it's too little too late. And when I talk to engineers, they're very excited about what Copilot can do, and the only people I see who are even talking about Code Whisperer work at AWS.Evelyn: No, that's true. And so, I think what's happening—and this is my opinion—is that first you had AWS, like, launching a really innovative new services, you know, that kind of like, it's like, “Ah, it's a whole new way of running your workloads in the cloud.” Instead of you know, basically, hiring a whole team, I just click a button, you have your instance, you use it, sell software, blah, blah, blah, blah. And then they went towards serverless, and then IoT, and then it started targeting large data lakes, and then eventually that kind of run backwards towards security, after the umpteenth S3 data leak.Corey: Oh, yeah. And especially now, like, so they had a hit in some corners with SageMaker, so now there are 40 services all starting with the word SageMaker. That's always pleasant.Evelyn: Yeah, precisely. And what I kind of notice is… now they're actually having to run it even further back because they caught all the corporations that could pivot to the cloud, they caught all the startups who started in the cloud, and now they're going for the larger behemoths who have massive data centers, and they don't want to innovate. They just want to reduce this massive sysadmin team. And I always like to use the example of a Bare Metal. When that came out in 2019, everybody—we've all kind of scratched your head. I'm like, really [laugh]?Corey: Yeah, I could see where it makes some sense just for very specific workloads that involve things like specific capabilities of processors that don't work under emulation in some weird way, but it's also such a weird niche that I'm sure it's there for someone. My default assumption, just given the breadth of AWS's customer base, is that whenever I see something that they just announced, well, okay, it's clearly not for me; that doesn't mean it's not meeting the needs of someone who looks nothing like me. But increasingly as I start exploring the industry in these services have time to percolate in the popular imagination and I still don't see anything interesting coming out with it, it really makes you start to wonder.Evelyn: Yeah. But then, like, I think, like, roughly a year or something, right after Bare Metal came out, they announced Outposts. So, then it was like, another way to just stay within your data center and be in the cloud.Corey: Yeah. There's a bunch of different ways they have that, okay, here's ways you can run AWS services on-prem, but still pay us by the hour for the privilege of running things that you have living in your facility. And that doesn't seem like it's quite fair.Evelyn: That's exactly it. So, I feel like now it's sort of in diminishing returns and sort of doing more cloud-native work compared to, you know, these huge opportunities, which is everybody who still has a data center for various reasons, or they're cloud-native, and they grow so big, that they actually start running their own data centers.Corey: I want to call out as well before we wind up being accused of being oblivious, that we're recording this before re:Invent. So, it's entirely possible—I hope this happens—that they announce something or several some things that make this look ridiculous, and we're embarrassed to have had this conversation. And yeah, they're totally getting it now, and they have completely surprised us with stuff that's going to be transformative for almost every customer. I've been expecting and hoping for that for the last three or four re:Invents now, and I haven't gotten it.Evelyn: Yeah, that's right. And I think there's even a new service launches that actually are missing fairly obvious things in a way. Like, mine is the Managed Workflow for Amazon—it's Managed Airflow, sorry. So, we were using Data Pipeline for, you know, big ETL processing, so it was an in-house tool we kind of built at Autoscout, we do platform engineering.And it was deprecated, so we looked at a new—what to replace it with. And so, we looked at Airflow, and we decided this is the way to go, we want to use managed because we don't want to maintain our own infrastructure. And the problem we ran into is that it doesn't have support for shared VPCs. And we actually talked to our account team, and they were confused. Because they said, like, “Well, every new service should support it natively.” But it just didn't have it. And that's, kind of, what, I kind of found is, like, there's—it feels—sometimes it's—there's a—it's getting rushed out the door, and it'll actually have a new managed service or new service launched out, but they're also sort of cutting some corners just to actually make sure it's packaged up and ready to go.Corey: When I'm looking at this, and seeing how this stuff gets packaged, and how it's built out, I start to understand a pattern that I've been relatively down on across the board. I'm curious to get your take because you work at a fairly sizable company as an engineering manager, running teams of people who do this sort of thing. Where do you land on the idea of companies building internal platforms to wrap around the offerings that the cloud service providers that they use make available to them?Evelyn: So, my opinion is that you need to build out some form of standardized tool set in order to actually be able to innovate quickly. Now, this sounds counterintuitive because everyone is like, “Oh, you know, if I want to innovate, I should be able to do this experiment, and try out everything, and use what works, and just release it.” And that greatness [unintelligible 00:23:14] mentality, you know, it's like five talented engineers working to build something. But when you have, instead of five engineers, you have five teams of five engineers each, and every single team does something totally different. You know, one uses Scala, and other on TypeScript, another one, you know .NET, and then there could have been a [last 00:23:30] one, you know, comes in, you know, saying they're still using Ruby.And then next thing you know, you know, you have, like, incredibly diverse platforms for services. And if you want to do any sort of like hiring or cross-training, it becomes incredibly difficult. And actually, as the organization grows, you want to hire talent, and so you're going to have to hire, you know, a developer for this team, you going to have to hire, you know, Ruby developer for this one, a Scala guy here, a Node.js guy over there.And so, this is where we say, “Okay, let's agree. We're going to be a Scala shop. Great. All right, are we running serverless? Are we running containerized?” And you agree on those things. So, that's already, like, the formation of it. And oftentimes, you start with DevOps. You'll say, like, “I'm a DevOps team,” you know, or doing a DevOps culture, if you do it properly, but you always hit this scaling issue where you start growing, and then how do you maintain that common tool set? And that's where we start looking at, you know, having a platform… approach, but I'm going to say it's Platform-as-a-Product. That's the key.Corey: Yeah, that's a good way of framing it because originally, the entire world needed that. That's what RightScale was when EC2 first came out. It was a reimagining of the EC2 console that was actually usable. And in time, AWS improved that to the point where RightScale didn't really have a place anymore in a way that it had previously, and that became a business challenge for them. But you have, what is it now, 2, 300 services that AWS has put out, and out, and okay, great. Most companies are really only actively working with a handful of those. How do you make those available in a reasonable way to your teams, in ways that aren't distracting, dangerous, et cetera? I don't know the answer on that one.Evelyn: Yeah. No, that's true. So, full disclosure. At AutoScout, we do platform engineering. So, I'm part of, like, the platform engineering group, and we built a platform for our product teams. It's kind of like, you need to decide to [follow 00:25:24] those answers, you know? Like, are we going to be fully containerized? Okay, then, great, we're going to use Fargate. All right, how do we do it so that developers don't actually—don't need to think that they're running Fargate workloads?And that's, like, you know, where it's really important to have those standardized abstractions that developers actually enjoy using. And I'd even say that, before you start saying, “Ah, we're going to do platform,” you say, “We should probably think about developer experience.” Because you can do a developer experience without a platform. You can do that, you know, in a DevOps approach, you know? It's basically build tools that makes it easy for developers to write code. That's the first step for anything. It's just, like, you have people writing the code; make sure that they can do the things easily, and then look at how to operate it.Corey: That sure would be nice. There's a lack of focus on usability, especially when it comes to a number of developer tools that we see out there in the wild, in that, they're clearly built by people who understand the problem space super well, but they're designing these things to be used by people who just want to make the website work. They don't have the insight, the knowledge, the approach, any of it, nor should they necessarily be expected to.Evelyn: No, that's true. And what I see is, a lot of the times, it's a couple really talented engineers who are just getting shit done, and they get shit done however they can. So, it's basically like, if they're just trying to run the website, they're just going to write the code to get things out there and call it a day. And then somebody else comes along, has a heart attack when see what's been done, and they're kind of stuck with it because there is no guardrails or paved path or however you want to call it.Corey: I really hope—truly—that this is going to be something that we look back and laugh when this episode airs, that, “Oh, yeah, we just got it so wrong. Look at all the amazing stuff that came out of re:Invent.” Are you going to be there this year?Evelyn: I am going to be there this year.Corey: My condolences. I keep hoping people get to escape.Evelyn: This is actually my first one in, I think, five years. So, I mean, the last time I was there was when everybody's going crazy over pins. And I still have a bag of them [laugh].Corey: Yeah, that did seem like a hot-second collectable moment, didn't it?Evelyn: Yeah. And then at the—I think, what, the very last day, as everybody's heading to re:Play, you could just go into the registration area, and they just had, like, bags of them lying around to take. So, all the competing, you know, to get the requirements for a pin was kind of moot [laugh].Corey: Don't you hate it at some point where it's like, you feel like I'm going to finally get this crowning achievement, it's like or just show up at the buffet at the end and grab one of everything, and wow, that would have saved me a lot of pain and trouble.Evelyn: Yeah.Corey: Ugh, scavenger hunts are hard, as I'm about to learn to my own detriment.Evelyn: Yeah. No, true. Yeah. But I am really hoping that re:Invent proves me wrong. Embarrassingly wrong, and then all my colleagues can proceed to mock me for this ridiculous podcast that I made with you. But I am a fierce skeptic. Optimistic nihilist, but still a nihilist, so we'll see how re:Invent turns out.Corey: So, I am curious, given your experience at more large companies than I tend to be embedded with for any period of time, how have you found that these large organizations tend to pick up new technologies? What does the adoption process look like? And honestly, if you feel like throwing some shade, how do they tend to get it wrong?Evelyn: In most cases, I've seen it go… terrible. Like, it just blows up in their face. And I say that is because a lot of the time, an organization will say, “Hey, we're going to adopt this new way of organizing teams or developing products,” and they look at all the practices. They say, “Okay, great. Product management is going to bring it in, they're going to structure things, how we do the planning, here's some great charts and diagrams,” but they don't really look at the culture aspect.And that's always where I've seen things fall apart. I've been in a room where, you know, our VP was really excited about team topologies and say, “Hey, we're going to adopt it.” And then an engineering manager proceeded to say, “Okay, you're responsible for this team, you're responsible for that team, you're responsible for this team talking to, like, a team of, like, five engineers,” which doesn't really work at all. Or, like, I think the best example is DevOps, you know, where you say, “Ah, we're going to adopt DevOps, we're going to have a DevOps team, or have a DevOps engineer.”Corey: Step one: we're going to rebadge everyone with existing job titles to have the new fancy job titles that reflect it. It turns out that's not necessarily sufficient in and of itself.Evelyn: Not really. The Spotify model. People say, like, “Oh, we're going to do the Spotify model. We're going to do skills, tribes, you know, and everything. It's going to be awesome, it's going to be great, you know, and nice, cross-functional.”The reason I say it bails on us every single time is because somebody wants to be in control of the process, and if the process is meant to encourage collaboration and innovation, that person actually becomes a chokehold for it. And it could be somebody that says, like, “Ah, I need to be involved in every single team, and listen to know what's happening, just so I'm aware of it.” What ends up happening is that everybody differs to them. So, there is no collaboration, there is no innovation. DevOps, you say, like, “Hey, we're going to have a team to do everything, so your developers don't need to worry about it.” What ends up happening is you're still an ops team, you still have your silos.And that's always a challenge is you actually have to say, “Okay, what are the cultural values around this process?” You know, what is SRE? What is DevOps, you know? Is it seen as processes, is it a series of principles, platform, maybe, you know? We have to say, like—that's why I say, Platform-as-a-Product because you need to have that product mindset, that culture of product thinking, to really build a platform that works because it's all about the user journey.It's not about building a common set of tools. It's the user journey of how a person interacts with their code to get it into a production environment. And so, you need to understand how that person sits down at their desk, starts the laptop up, logs in, opens the IDE, what they're actually trying to get done. And once you understand that, then you know your requirements, and you build something to fill those things so that they are happy to use it, as opposed to saying, “This is our platform, and you're going to use it.” And they're probably going to say, “No.” And the next thing, you know, they're just doing their own thing on the side.Corey: Yeah, the rise of Shadow IT has never gone away. It's just, on some level, it's the natural expression, I think it's an immune reaction that companies tend to have when process gets in the way. Great, we have an outcome that we need to drive towards; we don't have a choice. Cloud empowered a lot of that and also has given tools to help rein it in, and as with everything, the arms race continues.Evelyn: Yeah. And so, what I'm going to continue now, kind of like, toot the platform horn. So, Gregor Hohpe, he's a [solutions architect 00:31:56]—I always f- up his name. I'm so sorry, Gregor. He has a great book, and even a talk, called The Magic of Platforms, that if somebody is actually curious about understanding of why platforms are nice, they should really watch that talk.If you see him at re:Invent, or a summit or somewhere giving a talk, go listen to that, and just pick his brain. Because that's—for me, I really kind of strongly agree with his approach because that's really how, like, you know, as he says, like, boost innovation is, you know, where you're actually building a platform that really works.Corey: Yeah, it's a hard problem, but it's also one of those things where you're trying to focus on—at least ideally—an outcome or a better situation than you currently find yourselves in. It's hard to turn down things that might very well get you there sooner, faster, but it's like trying to effectively cargo-cult the leadership principles from your last employer into your new one. It just doesn't work. I mean, you see more startups from Amazonians who try that, and it just goes horribly because without the cultural understanding and the supporting structures, it doesn't work.Evelyn: Exactly. So, I've worked with, like, organizations, like, 4000-plus people, I've worked for, like, small startups, consulted, and this is why I say, almost every single transformation, it fails the first time because somebody needs to be in control and track things and basically be really, really certain that people are doing it right. And as soon as it blows up in their face, that's when they realize they should actually take a step back. And so, even for building out a platform, you know, doing Platform-as-a-Product, I always reiterate that you have to really be willing to just invest upfront, and not get very much back. Because you have to figure out the whole user journey, and what you're actually building, before you actually build it.Corey: I really want to thank you for taking the time to speak with me today. If people want to learn more, where's the best place for them to find you?Evelyn: So, I used to be on Twitter, but I've actually got off there after it kind of turned a bit toxic and crazy.Corey: Feels like that was years ago, but that's beside the point.Evelyn: Yeah, precisely. So, I would even just say because this feels like a corporate show, but find me on LinkedIn of all places because I will be sharing whatever I find on there, you know? So, just look me up on my name, Evelyn Osman, and give me a follow, and I'll probably be screaming into the cloud like you are.Corey: And we will, of course, put links to that in the show notes. Thank you so much for taking the time to speak with me. I appreciate it.Evelyn: Thank you, Corey.Corey: Evelyn Osman, engineering manager at AutoScout24. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, and I will read it once I finish building an internal platform to normalize all of those platforms together into one.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business, and we get to the point. Visit duckbillgroup.com to get started.

“Platforms harmonize and standardize without restricting. By standardizing, they actually enable and allow people to do more things." Gregor Hohpe is back again for the second episode with his latest book “Platform Strategy”. In this episode, Gregor discussed in-depth about building platforms with a proper platform strategy. He began by describing what a platform is from a few different perspectives, the benefits it brings, and what strategy we should think about when building a platform. Gregor also emphasized the opposite difference between platforms and IT services, with the key difference of how a platform thrives with more scale. We then had a few fun discussions discussing building a platform on top of a cloud platform, the key skillset we need to build a good platform, and how we should build a proper platform abstraction. Towards the end, Gregor also covered the recent trend of building developer platforms and business capability platforms. Also, do not miss Gregor's fun analogy of fruit basket vs fruit salad when explaining a good platform strategy.   Listen out for: Strategy Book Series - [00:05:39] Definition of Platform - [00:07:58] Platform Benefits - [00:12:09] Platform Strategy - [00:17:25] Platform vs IT Service - [00:20:47] Platform Thrives With Scale - [00:25:39] Cloud Platform-Based Platform - [00:29:36] Skillset for Building Platform - [00:36:44] Abstraction, Not an Illusion - [00:44:19] Fruit Salad vs Fruit Basket - [00:47:32] Developer Platform - [00:51:34] Business Capability Platform - [00:55:48] 3 Tech Lead Wisdom - [00:59:31] _____ Gregor Hohpe's BioGregor Hohpe advises CTOs and senior IT executives on IT strategy, cloud architecture, and organizational transformation. He served as advisor to the Singapore government, chief architect at Allianz SE, and technical director at Google Cloud's CTO Office. He is widely known as co-author of the seminal book “Enterprise Integration Patterns” and as frequent speaker at conferences around the world. His accessible, but technically accurate essays were republished in “97 Things Every Software Architect Should Know” and “Best Software Writing”. He is an active member of the IEEE Software editorial advisory board. Follow Gregor: Website – https://architectelevator.com/ LinkedIn – linkedin.com/in/ghohpe/ X – @ghohpe Platform Strategy (with special coupon for TLJ listeners) – https://leanpub.com/platformstrategy/c/techlead _____ Our Sponsors Are you looking for a new cool swag? Tech Lead Journal now offers you some swags that you can purchase online. These swags are printed on-demand based on your preference, and will be delivered safely to you all over the world where shipping is available. Check out all the cool swags available by visiting techleadjournal.dev/shop. And don't forget to brag yourself once you receive any of those swags. Like this episode? Show notes & transcript: techleadjournal.dev/episodes/156 Follow @techleadjournal on LinkedIn, Twitter, and Instagram. Buy me a coffee or become a patron.

Multicloud -- or using more than one cloud service provider simultaneously across an organization -- is a topic surrounded by myths and confusion. In this episode, Enterprise Strategist Tom Godden covers eight practices for engaging a multicloud strategy, including identifying when it does and does not make sense, developing a clear strategy, and prioritizing security. Read the original blog post: https://aws.amazon.com/blogs/enterprise-strategy/proven-practices-for-developing-a-multicloud-strategy/ Gregor Hohpe's article Multicloud: From Buzzword to Decision Model: https://architectelevator.com/cloud/multi-cloud-decision-model/ AWS Solutions for Hybrid and Multicloud: https://aws.amazon.com/hybrid-multicloud/

Early in her career, Nora Schöner fell in love with infrastructure as code.In this episode, Nora Schöner, Senior Cloud Consultant at superluminar, explains why she's so passionate about tools like Terraform, AWS CDK, and Pulumi, how they lower the threshold for developers entering the industry, and how they simplify the way developers define and make changes to infrastructure.You'll learn:1. How infrastructure as code helped Nora enter the programming industry2. What impact will the HashiCorp licensing change have on Terraform users?3. Why infrastructure from code could be the new wave of cloud infrastructure management4. Why infrastructure as code isn't as wasteful as some people think5. How to improve diversity in technical teams__________About Nora:Nora Schöner is a German-based Cloud Engineer who has been working in the tech industry for over 10 years. She is an AWS Community Builder and focuses on Cloud Computing and DevOps, but also on empowering other women developers. Nora founded She 'n IT Nuremberg, a meetup to connect other women devs in the industry, and co-organizes the AWS UG Nuremberg. She blogs at wolkencode.de and likes drawing Manga, dancing, and enjoying delicious food.Nora's blog and Social Media:Blog: https://wolkencode.de LinkedIn: https://www.linkedin.com/in/nora-schoener/ Instagram: https://www.instagram.com/wolkencode/ Mastodon: @wolkencode@awscommunity.social X/Twitter: https://twitter.com/wolkencode Nora's article recommendation:Gregor Hohpe's blog post: https://architectelevator.com/cloud/iac-architecture-as-code/ __________About superluminar:superluminar is an AWS Advanced Consulting Partner based in Hamburg, Germany. Established in 2017, the founding team started with over eight years of cloud experience and has grown to 19 cloud enthusiasts in 2023. By being committed to a hands-on approach, knowledge transfer that has impact, as well as quality and a performant outcome, at superluminar, we work with our clients, we don't just work for them.Website: https://www.superluminar.io Industry: Information Technology & ServicesCompany size: 11-50 employeesHeadquarters: HamburgFounded: 2017__________About the host Elias:Elias is the VP for North America at Checkmk. He comes from a strategy consulting background but has been an entrepreneur for the better part of the last 10 years. In his spare time, he likes to do triathlons.Get in touch with Elias via LinkedIn or email podcast@checkmk.com __________Podcast Music:Music by Ströme, used by permission‚Panta Rhei‘ written by Mario Schoenhofer(c)+p 2022, Compost Medien GmbH & Co KGhttps://stroeme.com/ https://compost-rec.com/ The All Things Ops Podcast has recently been named as one of the top DevOps Podcasts. Check it out here: https://blog.feedspot.com/devops_podcasts/ This podcast is produced by our friends at SAWOO. 

Does your leadership team suffer from the "fear of missing out" on the next technological advancement, leading to impulsive decisions or strategy shifts? How can you evaluate an innovative new offering to determine if it will create actual value for your organization? In this episode, AWS leaders tackle long and short-term planning, talent development, digital transformation, and more tools to help Executives avoid "FOMO". Presenters: Jake Burns, Enterprise Strategist, AWS and Gregor Hohpe, Enterprise Strategist, AWS

Can enterprises ever truly "future-proof" their organization? Predictions have become less and less reliable, and five year plans less realistic. How can leaders keep their organizations nimble and responsive to whatever comes, while also being intentional about growth and resource allocation? Join two AWS leaders as they discuss how to think about "future-readiness" as an alternative framework. Presenters: Jake Burns, Enterprise Strategist, AWS and Gregor Hohpe, Enterprise Strategist, AWS

En este episodio hablamos con Jaime González, CTO de Pentasoft, que nos cuenta sobre arquitecturas orientadas a eventos y como lo ponen en práctica en producción en los productos SaaS que hacen en Pentasoft.Este es el episodio 20 de la tercera temporada del podcast de Charlas Técnicas de AWS.

We are talking about Event Driven Architecture examples today. There was an event in London a few weeks ago, called EDA Day. It was organised by GOTO with a lot of AWS contributors. It was neat because it was one day focused on event driven architectures. It showed the coming together of a 15 to 20 year old pattern of EDA, plus serverless. And all the bigger services on top of that, like Eventbridge and Step Functions. Gregor Hohpe's Keynote Gregor Hohpe did the keynote talk: 'I made everything loosely coupled. Does my app fall apart?' Gregor is an AWS enterprise strategist. And he talked about the event landscape and the complexities behind event driven and architecture. He had a diagram called: 'A calls B'. It looked pretty simple until you get to the million things you need to think about when A calls B! He said that there were three languages in a cloud native serverless domain. The business domain and how you talk about the business domain as a business person. The eventing architecture and how you talk about it as an architect. And the cloud native area, and how you talk about it as a cloud engineer. So DDD, event framework and CDK for automation. It's about having those three separate languages and how you talk. And bringing them together at the end. Serverless Espresso And one neat thing to mention is a developer advocate called Julian Wood. He's worth looking up on Twitter. He, Ben Smith, and a few others from Serverless Land, put together a demo called Serverless Espresso. You scan a barcode and through an event driven step function, event bridge sequence, you can order a coffee from your phone. It looks and sounds really simple. But you watch the whole thing happen. That's a great lab. So look up AWS labs, to see Serverless Espresso. It's well put together to show how you build an event driven architecture from the ground up. Ben Ellerby - Minimal Viable Migrations Another good speaker was Ben Ellerby. He worked in Theodo and is an AWS Serverless Hero. He has a thing about Minimal Viable Migrations. A lot of people think event driven is a greenfield or brand new thing. But he had a great talk about existing architecture and going event driven. He talked about doing a small part of your architecture and going bit by bit. By using an incremental model. David Boyne - Awesome EventBridge David Boyne joined AWS and does ‘Awesome EventBridge'. He has open source projects. And he does a great talk on 'Thinking Event First'. How to approach events and get your schemes right. And really think about your domain model and lock it in from day one. So he's got a bunch of tools as well. So it's worth looking up his resources on ‘awesome event bridge'. Marcia Villalba - FooBar Serverless Another great speaker was Marcia Villalba. She's one of the developer advocates at AWS. She's got great content on good practices and getting started. She has a really nice way of explaining these concepts. There is one thing I get nervous about around event driven and domain driven. People who are good at it tend to get very complicated very quickly and lose everyone. But Marcia's super at bringing these concepts across and helping normal teams, which is every team. Check out her FooBar Serverless YouTube channel. There is tons of developer friendly content from beginner to more advanced. It's one of my YouTube subscriptions that I watch quite regularly. Lego Talk - Sarah Hamilton and Sheen Brisals The last one to talk about is Lego. They sponsored the event. And they had two talks. Sarah Hamilton is one of the software engineers and she gave a really good talk about the advanced techniques they're using in their event driven architecture. My friend, Sheen Brisals was speaking as well. They have a fantastic story, which is well worth listening to. It's about how they moved to an event driven serverless architecture. There's a socio-technical element to this. How you organise your teams and the attitude is what I would call a core engineering competency and mindset. As opposed to an architectural pattern. Lego tells their story brilliantly. Product Leader panel The event ended with a panel of product leaders from Eventbridge, Step Functions and MongoDB. It was a really relaxed panel. Emily Shea, who we know well, was there. She works in go to market for serverless. It was a relaxed chat. No one was pushing any tools. They were shooting the breeze on good practice and what's coming down the track. The evolution of event driven architecture and the tie in with serverless. There's something in it! I don't want to say Serverless is becoming EDA or EDA is becoming serverless. But serverless enables EDA for sure. https://gotoldn.com/ https://theserverlessedge.com/ https://twitter.com/ServerlessEdge  

In this Breaking Changes episode, Postman Chief Evangelist Kin Lane welcomes Gregor Hohpe, Enterprise Strategist at Amazon Web Services. Gregor's work with enterprise infrastructure has been a foundation reading for the last 20 years. He shares ideas on removing constraints, handling change, and the power of simplicity in every design. He also offers insights into the software architect elevator and the trade-offs that are required for a successful design.

Materiały dodatkowe:Modular monolith: Primer, część 1 seriiModular Monolith: Architectural Drivers, część 2 seriiModular Monolith: Architecture Enforcement, część 3 seriiModular Monolith: Integration Styles, część 4 seriiModular Monolith: Domain-Centric Design, część 5 seriiModular Monolith with DDD, przykład modularnego monolitu w repozytorium Kamila na GithubieEnterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions, Gregor Hohpe

To help you become an awesome software architect, we have picked out our top four books to make 12 in total. We are looking at engineering books have influenced both ourselves and 'The Flywheel Effect' book. 1. 'Continuous Delivery' came out in 2011. And it has been massively influential in how high performing teams deliver their software today. It is still as fresh as it was when it was written. And a lot of teams would do well to actually read it again. 2. 'Domain Driven Design' is a good book on how to describe a domain, good domain models and the importance of collaboration, communication and shared understanding, including their chapter on ubiquitous languages. You can be in different types of stacks or scenarios, but the knowledge is abstract so it's broadly applicable. 3. The Simon Wardley Book I have got a print copy of it. And I find myself always coming back to it. I think it was out in 2011. It's chunky and quite academic. So it's not exactly an easy read. But it's as deep as well, as they say. So I'm a big fan and I always go back to it. I don't take every word of it literally. But it's definitely a good read and will challenge your thinking still to this day. 4. 'Accelerate' by Nicole Forsgren, Jez Humble, and Gene Kim. This is a game changer. I think everyone the industry understands that. It distills down and captures (with scientific backing) all of the things that we were trying to articulate or were trying to push or evolve in our ecosystem.The capabilities to drive improvement, the scientific backing and little snippets of good advice and guidance. It is one of the best. 5. 'Extreme Ownership' by Jocko Willink. There's some cracking guidance on how to own something and lead. One that sticks out is centralised command and leading up and down the line. It's a well thought out and structured book on how to think, modern leadership and how to motivate people to be successful. I enjoy reading about how to think through systems, particularly in a leadership position, in technical orbs and stuff like that. It helps you to think like a leader. 6. 'Team Topologies' by Matthew Skelton and Manuel Pais. It's such a powerful question to ask 'what type of team are you?' And the response is: 'what do you mean?'. The answer is that you're a platform team, an enablement team, a value stream team or you're not anything. And all the techniques are in it with different tools and team API's and stuff. I think it's really practical. You can pick it up and implement tomorrow. 7. 'Reaching Cloud Velocity' It covers how to succeed in the cloud. In other words what are the principles and tenets that you should apply. What are the cultural and organisational things you should think about as you're starting to move to the cloud. It looks at the architectural approaches and patterns you should adopt. And how to do security and governance. It also looks at what's your business strategy, now that you're in the cloud. 8. 'Designing Data Intensive Applications' is almost a bible for anything data related such as streaming, different types of databases and why you make decisions on certain types of databases. You get into the design and the nuance of it. And understanding the landscape. It's broken into 2 to 3 minute blocks. So you can get straight into it and get perspective or context. 9. 'Creativity Inc', by Ed Catmull. The book is about Pixar, who went up against Disney by direct selling films. The full title of the book is 'Overcoming the Unseen Forces That Stand in the Way of True Inspiration' . They talk about the inspiration of creating and then actually making it. And how they structured the company and all the challenges they had. 10. 'Working Backwards' by Colin Bryar and Bill Carr. We see Amazon from the outside eg. amazon.com, Amazon Prime, deliveries and Alexa. But how do they actually do it? How can they be so successful and set themselves up for success? What way are their leadership structured? 'Working Backwards' distils down and gives insight into how Amazon operates at that sort of scale. It looks at how they have remained successful despite their growth. 11. 'Ask Your Developer' by Jeff Lawson, looking at the developer centric approach at Twilio. There's a lot of good content on how to inspire great individuals and teams to be creative. There's a good chapter on developer experience, their golden path and off roading. And how they've organised around developer experience. 12. 'The Software Architect Elevator', by Gregor Hohpe. I love his concept of an architect riding the elevator to talk to the executive in the penthouse, going down to the basement to write code and then all the floors in between. He talks about the how an architect can behave and operate to be successful in a company. Gregor is the 'architect's architect'. Serverless Craic from The Serverless Edge theserverlessedge.com @ServerlessEdge

This week we introduce you to Modern Cloud.  Modern Cloud is a phrase that we use a lot. But a lot of people are asking: 'What does modern cloud, modern application or modern architecture actually mean?' It's hard to explain what it means without thinking about legacy cloud. Gregor Hohpe has a great description. He says; 'If you lift and shift to AWS (or any of the cloud providers), and don't modernise your architecture, all you've got is a fancy data centre!'. You've moved your old data centre to a cloud level data centre. So you've got a top of the range data centre, but you're not actually benefiting from Cloud. I spoke to a company a year or two ago, who had a five step maturity model for the cloud. And they asked me about serverless first. And when I responded they said: 'Oh, you've just described step seven!'. They hadn't moved beyond modern cloud practices. That's where Wardley Mapping comes to the fore by allowing you to understand the users, the user needs and the value chain of technology that you have at your disposal to meet those needs. As well as choosing the right tool for the job. So a well optimised EC2 instance, for this particular use case may be perfect. You can justify the additional runtime and operational burden, because you have good situational awareness about your tech stack and the choices you're making. Teams that are operating in the modern cloud have that situational awareness, and they understand the trade offs for the technology choices they make. They can justify that all the way up the chain to the actual needs of the users. We always describe inertia in Wardley maps and spot teams and tech that suffer from inertia. Inertia can be not looking after something within a Lambda function that means we go back and continuously look at it, fix it or patch it. We haven't thought about the longer term or what its role is other than for this solution. You can have an EC2 container with operational capability around it. We're patching all the time, all our dependencies are up to date, we're getting ready to move mobility, we're always thinking about the next thing and is this the right piece of kit for the problem we're trying to solve? There's a mindset element which comes from teams working in a modern cloud way. A modern cloud team can do everything themselves. They rarely need to go outside their team for help. They're ‘fast flow'. There's very few handoffs, You still need, not necessarily a DevOps team, but you need enabling teams. There are complicated sub-system teams to ensure that your streamline teams or your fringe stack teams have all the experience, expertise and capabilities to do what they need to do without having to talk to anybody. As Team Topologies say, your streamline teams are set up for success and they're able to move without friction, without handoffs or dependencies on anybody. They have the organisational structure to make sure it's done appropriately and the guardrails are the right ones for business. There's a good train analogy. The train driver and staff in the train can drive the train from A to B. But they still need someone to lay the track, work the signals and repair the train. You're not doing it on your own. There's other things providing other services. Well architected is a factor as well, which we've talked about. And there's 'time to market' and how fast the team can deliver. Observability is critical for the team's to know how they're delivering with good radiators and good dashboards with frequency, mean time to restore and your lead time for change. The DORA four keys metrics are part of that as well as the well architected pillar and your adherence to them. Teams should have a good understanding of where they are on that journey. It's critical for teams that are adopting modern cloud to know their current status and how well architected they are to know how to evolve and improve. It's worth going through the flywheel that we talk about in our upcoming book. We talk about how to be effective in the modern cloud. In the flywheel, we look at 'Purpose' which is the strategy of the business/company and 'Challenge' which is the ability to challenge, have an environment for success and the right culture in the organisation. 'Next Best Action' is from a developer's point of view and looks at design patterns, processes and DevOps. 'Long Term Value', is about architecture, data security and sustainability. It's interesting to look through those lenses at ‘modern cloud'. Whether you're a CTO, an architect, or a developer, it's good to understand the art of the possible. What does good actually look like? What can we actually do? And the direction and pathway towards that. You don't arrive overnight. Serverless Craic from The Serverless Edge theserverlessedge.com @ServerlessEdge

Luciano and Eoin take a deep dive into SQS as part of a series on AWS event services and event-driven architecture. We talk about the kind of problems SQS can solve, all of the SQS features and how to configure and use SQS to achieve reliability and scalability without all the complexity. We also take some time to detail how SQS works with Lambda in terms of scaling, batching and filtering. In this episode, we gave a special mention to three highly-recommended re:Invent 2021 talks on the topic of Enterprise Integration Patterns with AWS services: AWS re:Invent 2021 - Application integration patterns for microservices - Gregor Hohpe - https://www.youtube.com/watch?v=ttJAIQf7cTw AWS re:Invent 2021 - Building next-gen applications with event-driven architectures - Sam Dengler - https://www.youtube.com/watch?v=U5GZNt0iMZY AWS re:Invent 2021 - AWS re:Invent 2021 - Application integration patterns for microservices - Dirk Froehner - https://www.youtube.com/watch?v=QhfuzEkN3Ck - In addition, we mentioned the following resources. - SQS: https://aws.amazon.com/sqs/ - Using Lambda with SQS: https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html - Lambda SQS Scaling: https://aws.amazon.com/premiumsupport/knowledge-center/lambda-sqs-scaling/ - SLIC Watch (Serverless plugin for easy dashboards and alarms): https://github.com/fourTheorem/slic-watch This episode is also available on YouTube: https://www.youtube.com/AWSBites You can listen to AWS Bites wherever you get your podcasts: - Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017 - Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q - Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw== - Breaker: https://www.breaker.audio/aws-bites - RSS: https://anchor.fm/s/6a3312a0/podcast/rss Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on Twitter: - https://twitter.com/eoins - https://twitter.com/loige

Você já deve ter ouvido falar em monolitos, mas você sabe o que fazer com eles? No episódio de hoje do Hipsters Ponto Tech vamos conversar sobre monolitos, descobrir se são bons, se dá pra começar com isso, se dá pra manter e até onde se mantém. Participantes: Paulo Silveira, o host que ficou animado com o temaDayanne Cunha, Solutions Developer na bxblueFabricio Buzeto, CTO na bxblueMauricio Linhares, Tech Lead na DigitalOceanRoberta Arcoverde, Tech Lead na StackOverflow Links: Livro "Patterns of Enterprise Application Architecture" de Martin FowlerLivro "Enterprise Integration Patterns" de Gregor Hohpe e Bobby WoolfLivro "Practical Object-Oriented Design" de Sandi MetzBlog Shopify Conheça o Alura CasesInscreva-se no YouTube da AluraInscreva-se na newsletter Imersão, Aprendizagem e Tecnologia Produção e conteúdo: Alura Cursos de Tecnologia - https://www.alura.com.br === Caelum Escola de Tecnologia - https://www.caelum.com.br/ Edição e sonorização: Radiofobia Podcast e Multimídia

Você já deve ter ouvido falar em monolitos, mas você sabe o que fazer com eles? No episódio de hoje do Hipsters Ponto Tech vamos conversar sobre monolitos, descobrir se são bons, se dá pra começar com isso, se dá pra manter e até onde se mantém. Participantes: Paulo Silveira, o host que ficou animado com o temaDayanne Cunha, Solutions Developer na bxblueFabricio Buzeto, CTO na bxblueMauricio Linhares, Tech Lead na DigitalOceanRoberta Arcoverde, Tech Lead na StackOverflow Links: Livro "Patterns of Enterprise Application Architecture" de Martin FowlerLivro "Enterprise Integration Patterns" de Gregor Hohpe e Bobby WoolfLivro "Practical Object-Oriented Design" de Sandi MetzBlog Shopify Conheça o Alura CasesInscreva-se no YouTube da AluraInscreva-se na newsletter Imersão, Aprendizagem e Tecnologia Produção e conteúdo: Alura Cursos de Tecnologia - https://www.alura.com.br === Caelum Escola de Tecnologia - https://www.caelum.com.br/ Edição e sonorização: Radiofobia Podcast e Multimídia

“The cloud is a change in operating model. It isn't IT procurement. If you don't change the way your organization works, the cloud is going to look much more like another data center.“ Gregor Hohpe is the author of “Software Architect Elevator” and “Cloud Strategy”. In this episode, Gregor started our conversation by explaining the role of a software architect, the reason for the latest resurgence of the role, and his software architect elevator concept. He then described what a good architecture should look like and how to deal with trade-offs by using the analogy of financial options. We then discussed in-depth about the cloud and why adopting cloud requires a lifestyle change in order to benefit from it the most. Gregor also described why organizations need a good viable cloud strategy and debunked the concern of many organizations on cloud vendor lock-in. He also gave his tips on how organizations should approach building an in-house cloud platform and how to change the organization structure to embrace the cloud better. Towards the end, do not miss our insightful discussion on Gregor's law of excessive complexity! Listen out for: Career Journey - [00:06:48] Software Architect Role - [00:07:48] Software Architect Elevator - [00:12:07] An Architect Stands on 3 Legs - [00:14:37] Good Architecture - [00:18:08] Trade-offs - [00:21:09] Definition of Cloud - [00:25:55] Cloud is a Lifestyle Change - [00:28:56] Motivation for Moving to the Cloud - [00:32:18] Cloud Strategy - [00:36:43] Building up Cloud Strategy - [00:39:36] Patterns & Antipatterns - [00:43:57] Cloud is Not an Infrastructure Topic - [00:49:29] In-house Cloud Platform - [00:52:38] Gregor's Law of Excessive Complexity - [00:57:39] Organization Structure - [01:01:37] 3 Tech Lead Wisdom - [01:05:16] _____ Gregor Hohpe's Bio As an Enterprise Strategist at AWS, Gregor advises CTOs and tech leaders in their organizational and technology platform transformation. Prior to joining AWS, Gregor served as a Smart Nation Fellow to the Singapore government, as technical director in Google Cloud's Office of the CTO, and as Chief Architect at Allianz SE, where he oversaw the architecture of a global data center consolidation and deployed the first private cloud software delivery platform. He is an active member of the IEEE Software advisory board. Follow Gregor: LinkedIn – https://www.linkedin.com/in/ghohpe/ Twitter – https://twitter.com/ghohpe The Architect Elevator – https://architectelevator.com/ Cloud Strategy – https://cloudstrategybook.com Our Sponsor This episode is proudly sponsored by Emergence, the journal of business agility. This quarterly publication brings you inspiring stories from the most innovative companies and explores themes of new ways of working, reclaiming management, and humanizing business. Each issue is hand illustrated and 100% content. Use the promo code “techlead” to get a 10% discount on your annual subscription. Visit businessagility.institute/emergence to get your edition and support the publication supporting your podcast. Like this episode? Follow @techleadjournal on LinkedIn, Twitter, and Instagram. Pledge your support by becoming a patron. For more info about the episode (including quotes and transcript), visit techleadjournal.dev/episodes/50.

Patrick Kua was CTO and Chief Scientist at N26 in Berlin and is now an independent coach and mentor to CTOs and VPs Engineering. Patrick has written several books e.g. about how to talk to tech leads and he co-authored a book about evolutionary architecture. In this episode we will therefore discuss how software architecture can embrace change and support business goals in the long run. Patrick's home page Patrick's newsletter Command line tools for architecture decision records (ADRs) Architecture elevator (Gregor Hohpe)

Our podcast team catches up with Gregor Hohpe to hear about his book The Software Architect Elevator which looks at how to connect the boardroom to the IT engine room and how architects can drive digital transformations.

Pattern für Software-Entwicklung gibt es schon seit mehr als 25 Jahren. Aber schon davor gab es Patterns für Dinge z.B. in der Gebäude-Architektur. Und mittlerweile sind auch Patterns für andere Bereiche entstanden. So erlauben sie den Zugriff auf Erfahrungen über den Umgang mit Code und Menschen. Sogar Refactorings sind eigentlich Patterns für den Umgang mit Code. Links Patterns Christopher Alexander: “The Timeless Way of Building”, 1979, Oxford University Press, ISBN 978-0-19-502402-9 Peter Gabriel: “Patterns of Software” Kevlin Henney, Frank Buschmann et al: “Pattern-Oriented-Software-Architecture 1-5” POSA 1-5 , besonders POSA 5 Gregor Hohpe, Bobby Woolf: “Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions”, 2003, Addison Wesley, ISBN 978-0-32-120068-6 Gerard Mezaros: “xUnit Test Patterns: Refactoring Test Code”, 2007, Addison Wesley, ISBN 978-0-13-149505-0 Refactoring Michael Hungers Studienarbeit zu Refactoring Refactoring 2nd Ed Vortrag beim JUG Saxony Day Martin Fowler: “Refactoring: : Improving the Design of Existing Code”, 2nd Edition, 2018, Addison Wesley, ISBN 978-0-13-475759-9 Martin Fowler: Refactoring 2nd Edition Web Version Joshua Kerievsky: “Refactoring to Patterns”, 2004, Addison Wesley, ISBN 978-0-32-121335-8 Kent Beck: “Implementation Patterns”, 2007, Addison Wesley, ISBN 978-0-32-141309-3 Pramod Sadalage: “Refactoring Databases: Evolutionary Database Design”, 2011, Addison Wesley, ISBN 978-0-32-177451-4 Steve Freeman, Nat Pryce: “Growing Object-Oriented Software, Guided by Tests”, 2009, Addison Wesley, ISBN 978-0-32-150362-6 Adam Tornhill: “Your Code as a Crime Scene: Use Forensic Techniques to Arrest Defects, Bottlenecks, and Bad Design in Your Programs”, 2015, O'Reilly, ISBN 978-1-68-050038-7 Adam Tornhill: “Software Design X-Rays: Fix Technical Debt with Behavioral Code”, 2018, O'Reilly, ISBN 978-1-68-050272-5 , Software Design jQAssistant Michael Feathers: “Working Effectively with Legacy Code”, 2013, Addison Wesley, ISBN 978-0-13-117705-5 Leute Dave Hoover, Adewale Oshineye: “Apprenticeship Patterns: Guidance for the Aspiring Software”, 2009, O'Reilly, ISBN 978-0-59-651838-7 Philip Armour: “The Laws of Software Process”, 2003, Auerbach, ISBN 978-0-84-931489-6 Linda Rising: “Fearless Change: Patterns for Introducing New Ideas”, 2015, Addison Wesley, ISBN 978-0-13-439525-8

Gregor Hohpe, an Enterprise Strategist for AWS, sits down with fellow Enterprise Strategists, Jake Burns, to talk about his book, Cloud Strategy: A Decision-based Approach to Successful Cloud Migration. Listen in as he talks about the book, why he wrote it, and whats next.

In this episode of The Podlets Podcast, we welcome Michael Gasch from VMware to join our discussion on the necessity (or not) of formal education in working in the realm of distributed systems. There is a common belief that studying computer science is a must if you want to enter this field, but today we talk about the various ways in which individuals can teach themselves everything they need to know. What we establish, however, is that you need a good dose of curiosity and craziness to find your feet in this world, and we discuss the many different pathways you can take to fully equip yourself. Long gone are the days when you needed a degree from a prestigious school: we give you our hit-list of top resources that will go a long way in helping you succeed in this industry. Whether you are someone who prefers learning by reading, attending Meetups or listening to podcasts, this episode will provide you with lots of new perspectives on learning about distributed systems. Follow us: https://twitter.com/thepodlets Website: https://thepodlets.io Feeback: info@thepodlets.io https://github.com/vmware-tanzu/thepodlets/issues Hosts: Carlisia Campos Duffie Cooley Michael Gasch Key Points From This Episode: • Introducing our new host, Michael Gasch, and a brief overview of his role at VMware. • Duffie and Carlisia’s educational backgrounds and the value of hands-on work experience. • How they first got introduced to distributed systems and the confusion around what it involves. • Why distributed systems are about more than simply streamlining communication and making things work. • The importance and benefit of educating oneself on the fundamentals of this topic. • Our top recommended resources for learning about distributed systems and their concepts. • The practical downside of not having a formal education in software development. • The different ways in which people learn, index and approach problem-solving. • Ensuring that you balance reading with implementation and practical experience. • Why it’s important to expose yourself to discussions on the topic you want to learn about. • The value of getting different perspectives around ideas that you think you understand. • How systems thinking is applicable to things outside of computer science.• The various factors that influence how we build systems. Quotes: “When people are interacting with distributed systems today, or if I were to ask like 50 people what a distributed system is, I would probably get 50 different answers.” — @mauilion [0:14:43] “Try to expose yourself to the words, because our brains are amazing. Once you get exposure, it’s like your brain works in the background. All of a sudden, you go, ‘Oh, yeah! I know this word.’” — @carlisia [0:14:43] “If you’re just curious a little bit and maybe a little bit crazy, you can totally get down the rabbit hole in distributed systems and get totally excited about it. There’s no need for having formal education and the degree to enter this world.” — @embano1 [0:44:08] Learning resources suggested by the hosts: Book, Designing Data-Intensive Applications, M. Kleppmann Book, Distributed Systems, M. van Steen and A.S. Tanenbaum (free with registration) Book, Thesis on Raft, D. Ongaro. - Consensus - Bridging Theory and Practice (free PDF) Book, Enterprise Integration Patterns, B.Woolf, G. Hohpe Book, Designing Distributed Systems, B. Burns (free with registration) Video, Distributed Systems Video, Architecting Distributed Cloud Applications Video, Distributed Algorithms Video, Operating System - IIT Lectures Video, Intro to Database Systems (Fall 2018) Video, Advanced Database Systems (Spring 2018) Paper, Time, Clocks, and the Ordering of Events in a Distributed System Post, Notes on Distributed Systems for Young Bloods Post, Distributed Systems for Fun and Profit Post, On Time Post, Distributed Systems @The Morning Paper Post, Distributed Systems @Brave New Geek Post, Aphyr’s Class materials for a distributed systems lecture series Post, The Log - What every software engineer should know about real-time data’s unifying abstraction Post, Github - awesome-distributed-systems Post, Your Coffee Shop Doesn’t Use Two-Phase Commit Podcast, Distributed Systems Engineering with Apache Kafka ft. Jason Gustafson Podcast, The Systems Bible - The Beginner’s Guide to Systems Large and Small - John Gall Podcast, Systems Programming - Designing and Developing Distributed Applications - Richard Anthony Podcast, Distributed Systems - Design Concepts - Sunil Kumar Links Mentioned in Today’s Episode: The Podlets on Twitter — https://twitter.com/thepodlets Michael Gasch on LinkedIn — https://de.linkedin.com/in/michael-gasch-10603298 Michael Gasch on Twitter — https://twitter.com/embano1 Carlisia Campos on LinkedIn — https://www.linkedin.com/in/carlisia Duffie Cooley on LinkedIn — https://www.linkedin.com/in/mauilion VMware — https://www.vmware.com/ Kubernetes — https://kubernetes.io/ Linux — https://www.linux.org Brian Grant on LinkedIn — https://www.linkedin.com/in/bgrant0607 Kafka — https://kafka.apache.org/ Lamport Article — https://lamport.azurewebsites.net/pubs/time-clocks.pdf Designing Date-Intensive Applications — https://www.amazon.com/Designing-Data-Intensive-Applications-Reliable-Maintainable-ebook/dp/B06XPJML5D Designing Distributed Systems — https://www.amazon.com/Designing-Distributed-Systems-Patterns-Paradigms/dp/1491983647 Papers We Love Meetup — https://www.meetup.com/papers-we-love/ The Systems Bible — https://www.amazon.com/Systems-Bible-Beginners-Guide-Large/dp/0961825170 Enterprise Integration Patterns — https://www.amazon.com/Enterprise-Integration-Patterns-Designing-Deploying/dp/0321200683 Transcript: EPISODE 12 [INTRODUCTION] [0:00:08.7] ANNOUNCER: Welcome to The Podlets Podcast, a weekly show that explores Cloud Native one buzzword at a time. Each week, experts in the field will discuss and contrast distributed systems concepts, practices, tradeoffs and lessons learned to help you on your cloud native journey. This space moves fast and we shouldn’t reinvent the wheel. If you’re an engineer, operator or technically minded decision maker, this podcast is for you. [EPISODE] [00:00:41] CC: Hi, everybody. Welcome back. This is Episode 12, and we are going to talk about distributed systems without a degree or even with a degree, because who knows how much we learn in university. I am Carlisia Campos, one of your hosts. Today, I also have Duffie Cooley. Say hi, Duffie. [00:01:02] DC: Hey, everybody. [00:01:03] CC: And a new host for you, and this is such a treat. Michael Gasch, please tell us a little bit of your background. [00:01:11] MG: Hey! Hey, everyone! Thanks, Carlisia. Yes. So I’m new to the show. I just want to keep it brief because I think over the show we’ll discuss our backgrounds a little bit further. So right now, I’m with VMware. So I’ve been with VMware almost for five years. Currently, I'm in the office of the CTO. I’m a platform architect in the office of the CTO and I mainly use Kubernetes on a daily basis from an engineering perspective. So we build a lot of prototypes based on customer input or ideas that we have, and we work with different engineering teams. Kurbernetes has become kind of my bread and butter but lately more from a consumer perspective like developing with Kurbenetes or against Kubernetes, instead of the formal ware of mostly being around implementing and architecting Kubernetes. [00:01:55] CC: Nice. Very impressive. Duffie? [00:01:58] MG: Thank you. [00:01:59] DC: Yeah. [00:02:00] CC: Let’s give the audience a little bit of your backgrounds. We’ve done this before but just to frame the episodes, so people will know how we come in as distributed systems. [00:02:13] DC: Sure. In my experience, I spent – I don’t have a formal education history. I spent most of my time kind of like in a high school time. Then from there, basically worked into different systems administration, network administration, network architect, and up into virtualization and now containerization. So I’ve got a pretty hands-on kind of bootstrap experience around managing infrastructure, both at small-scale, inside of offices, and the way up to very large scale, working for some of the larger companies here in the Silicon Valley. [00:02:46] CC: All right. My turn I guess. So I do have a computer science degree but I don’t feel that I really went deep at all in distributed systems. My degree is also from a long time ago. So mainly, what I do know now is almost entirely from hands-on work experience. Even so, I think I'm very much lacking and I’m very interested in this episode, because we are going to go through some great resources that I am also going to check out later. So let’s get this party started. [00:03:22] DC: Awesome. So you want to just talk about kind of the general ideas behind distributed systems and like how you became introduced to them or like where you started in that journey? [00:03:32] CC: Yeah. Let’s do that. [00:03:35] DC: My first experience with the idea of distributed systems was in using them before I knew that they were distributed systems, right? One of the very first distributed systems as I look back on it that I ever actually spent any real time with was DNS, which I consider to be something of a distributed system. If you think about it, they have name servers, they have a bunch of caching servers. They solve many of the same sorts of problems. In a previous episode, we talked about how networking, just the general idea of networking and handling large-scale architecting networks. It’s also in a way very – has a lot of analogues into distributed systems. For me, I think working with and helping solve the problems that are associated with them over time gave me a good foundational understanding for when we were doing distributed systems as a thing later on in my career. [00:04:25] CC: You said something that caught my interest, and it’s very interesting, because obviously for people who have been writing algorithms, writing papers about distributed systems, they’re going to go yawning right now, because I’m going to say the obvious. As you start your journey programming, you read job requirements. You read or you must – should know distributed systems. Then I go, “What is distributed system? What do they really mean?” Because, yes, we understand apps stuck to apps and then there is API, but there’s always for me at least a question at the back of my head. Is that all there is to it? It sounds like it should be a lot more involved and complex and complicated than just having an app stuck on another app. In fact, it is because there are so many concepts and problems involved in distributed systems, right? From timing, clock, and sequence, and networking, and failures, how do you recover. There is a whole world in how do you log this properly, how do you monitor. There’s a whole world that revolves around this concept of systems residing in different places and [inaudible 00:05:34] each other. [00:05:37] DC: I think you made a very good point. I think this is sort of like there’s an analog to this in containers, oddly enough. When people say, “I want a container within and then the orchestration systems,” they think that that's just a thing that you can ask for. That you get a container and inside of that is going to be your file system and it’s going to do all those things. In a way, I feel like that same confusion is definitely related to distributed systems. When people are interacting with distributed systems today or if I were to ask like 50 people what a distributed system is, I would probably get 50 different answers. I think that you got a pretty concise definition there in that it is a set of systems that intercommunicate to perform some function. It’s like found at its base line. I feel like that's a pretty reasonable definition of what distributed systems are, and then we can figure out from there like what functions are they trying to achieve and what are some of the problems that we’re trying to solve with them. [00:06:29] CC: Yeah. That’s what it’s all about in my head is solving the problems because at the beginning, I was thinking, “Well, it must be just about communicating and making things work.” It’s the opposite of that. It’s like that’s a given. When a job says you need to understand about distributed systems, what they are really saying is you need to know how to deal with failures, not just to make it work. Make it work is sort of the easy part, but the whole world of where the failures can happen, how do you handle it, and that, to me is what needing to know distributed system comes in handy. In a couple different things, like at the top layer or 5% is knowing how to make things work, and 95% is knowing how to handle things when they don’t work, because it’s inevitable. [00:07:19] DC: Yeah, I agree. What do you think, Michael? How would you describe the context around distributed systems? What was the first one that you worked with? [00:07:27] MG: Exactly. It’s kind of similar to your background, Duffie, which is no formal degree or education on computer science right after high school and jumping into kind of my first job, working with computers, computer administration. I must say that from the age of I think seven or so, I was interested in computers and all that stuff but more from a hardware perspective, less from a software development perspective. So my take always was on disassembling the pieces and building my own computers than writing programs. In the early days, that just was me. So I completely almost missed the whole education and principles and fundamentals of how you would write a program for a single computer and then obviously also for how to write programs that run across a network of computers. So over time, as I progress on my career, especially kind of in the first job, which was like seven years of different Linux systems, Linux administrations, I kind of – Like you, Duffie, I dealt with distributed systems without necessarily knowing that I'm dealing with distributed systems. I knew that it was mostly storage systems, Linux file servers, but distributed file servers. Samba, if some of you recall that project. So I knew that things could fail. I know it could fail, for example, or I know it could not be writable, and so a client must be stuck but not necessarily I think directly related to fundamentals of how distributed systems work or don’t work. Over time, and this is really why I appreciate the Kubernetes project in community, I got more questions, especially when this whole container movement came up. I got so many questions around how does that thing work. How does scheduling work? Because scheduling kind of was close to my interest in the hardware design and low-level details. But I was looking at Kubernetes like, “Okay. There is the scheduler.” In the beginning, the documentation was pretty scarce around the implementation and all the control as for what’s going on. So I had to – I listen to a lot of podcasts and Brian Grant’s great talks and different shows that he gave from the Kubernetes space and other people there as well. In the end, I had more questions than answers. So I had to dig deeper. Eventually, that led me to a path of wanting to understand more formal theory behind distributed systems by reading the papers, reading books, taking some online classes just to get a basic understanding of those issues. So I got interested in results scheduling in distributed systems and consensus. So those were two areas that kind of caught my eyes like, “What is it? How do machines agree in a distributed system if so many things can go wrong?” Maybe we can explore this later on. So I’m going to park this for a bit. But back to your question, which was kind of a long-winded answer or a road to answering your question, Duffie. For me, a distributed system is like this kind of coherent network of computer machines that from the outside to an end-user or to another client looks like one gigantic big machine that is [inaudible 00:10:31] to run as fast. That is performing also efficient. It constitutes a lot of characteristics and properties that we want from our systems that a single machine usually can’t handle. But it looks like it's a big single machine to a client. [00:10:46] DC: I think that – I mean, it is interesting like, I don’t want to get into – I guess this is probably not just a distributed systems talk. But obviously, one of the questions that falls out for me when I hear that answer is then what is the difference between a micro service architecture and distributed systems, because I think it's – I mean, to your point, the way that a lot of people work with the app to learn to develop software, it’s like we’re going to develop a monolithic application just by nature. We’re going to solve a software problem using code. Then later on, when we decide to actually scale this thing or understand how to better operate it under a significant load, then we started thinking about, “Okay. Well, how do we have to architect this differently in such a way that it can support that load?” That’s where I feel like the beams cut across, right? We’re suddenly in a world where you’re not only just talking about microservices. You’re also talking about distributed systems because you’re going to start thinking about how to understand transactionality throughout that system, how to understand all of those consensus things that you're referring to. How do they affect it when I add mister network in there? That’s cool. [00:11:55] MG: Just one comment on this, Duffie, which took me a very long time to realize, which is coming – From my definition of what a distributed system is like this group of machines that they perform work in a certain sense or maybe even more abstracted like at a bunch of computers network together. What I kind of missed most of the time, and this goes back to the DNS example that you gave in the beginning, was the client or the clients are also part of this distributed system, because they might have caches, especially in DNS. So you always deal with this kind of state that is distributed everywhere. Maybe you don't even know where it kind of is distributed, and the client kind of works with a local stale data. So that is also part of a distributed system, and something I want to give credit to the Kafka community and some of the engineers on Kafka, because there was a great talk lately that I heard. It’s like, “Right. The client is also part of your distributed system, even though usually we think it's just the server. That those many server machines, all those microservices.” At least I missed that a long time. [00:12:58] DC: You should put a link to that talk in our [inaudible 00:13:00]. That would be awesome. It sounds great. So what do you think, Carlisia? [00:13:08] CC: Well, one thing that I wanted to mention is that Michael was saying how he’s been self-teaching distributed systems, and I think if we want to be competent in the area, we have to do that. I’m saying this to myself even. It’s very refreshing when you read a book or you read a paper and you really understand the fundamentals of an aspect of distributed system. A lot of things fall into place in your hands. I’m saying this because even prioritizing reading about and learning about the fundamentals is really hard for me, because you have your life. You have things to do. You have the minutiae in things to get done. But so many times, I struggle. In the rare occasions where I go, “Okay. Let me just learn this stuff trial and error,” it makes such a difference. Then once you learn, it stays with you forever. So it’s really good. It’s so refreshing to read a paper and understand things at a different level, and that is what this episode is. I don’t know if this is the time to jump in into, “So there are our recommendations.” I don't know how deep, Michael, you’re going to go. You have a ton of things listed. Everything we mention on the show is going to be on our website, on the show notes. So nobody needs to be necessarily taking notes. Anything thing I wanted to say is it would be lovely if people would get back to us once you listened to this. Let us know if you want to add anything to this list. It would be awesome. We can even add it to this list later and give a shout out to you. So it’d be great. [00:14:53] MG: Right. I don’t want to cover this whole list. I just wanted to be as complete as possible about a stuff that I kind of read or watched. So I just put it in and I just picked some highlights there if you want. [00:15:05] CC: Yeah. Go for it. [00:15:06] MG: Yeah. Okay. Perfect. Honestly, even though not the first in the list, but the first thing that I read, so maybe from kind of my history of how I approach things, was searching for how do computers work and what are some of the issues and how do computers and machines agree. Obviously, the classic paper that I read was the Lamport paper on “Time, Clocks, and the Ordering of Events in a Distributed System”. I want to be honest. First time I read it, I didn’t really get the full essence of the paper, because it doesn't prove in there. The mathematic proof for me didn't click immediately, and there were so many things and concepts and physics and time that were thrown at me where I was looking for answers and I had more questions than answers. But this is not to Leslie. This is more like by the time I just wasn't prepared for how deep the rabbit hole goes. So I thought, if someone asked me for – I only have time to read one book out of this huge list that I have there and all the other resources. Which one would it be? Which one would I recommend? I would recommend Designing Data-Intensive Apps by Martin Kleppmann, which I’ve been following his blog posts and some partial releases that he's done before fully releasing that book, which took him more than four years to release that book. It’s kind of almost the Bible, state-of-the-art Bible when it comes to all concepts in distributed systems. Obviously, consensus, network failures, and all that stuff but then also leading into modern data streaming, data platform architectures inspired by, for example, LinkedIn and other communities. So that would be the book that I would recommend to someone if – Who does have time to read one book. [00:16:52] DC: That’s a neat approach. I like the idea of like if you had one thing, if you have one way to help somebody ramp on distributed systems and stuff, what would it be? For me, it’s actually I don't think I would recommend a book, oddly enough. I feel like I would actually – I’d probably drive them toward the kind of project, like the kind [inaudible 00:17:09] project and say, “This is a distributed system all but itself.” Start tearing it apart to pieces and seeing how they work and breaking them and then exploring and kind of just playing with the parts. You can do a lot of really interesting things. This is actually another book in your list that was written by Brendan Burns about Designing Distributed Systems I think it’s called. That book, I think he actually uses Kubernetes as a model for how to go about achieving these things, which I think is incredibly valuable, because it really gets into some of the more stable distributed systems patterns that are around. I feel like that's a great entry point. So if I had one thing, if I had to pick one way to help somebody or to push somebody in the direction of trying to learn distributed systems, I would say identify those distributed systems that maybe you’re already aware of and really explore how they work and what the problems with them are and how they went about solving those problems. Really dig into the idea of it. It’s something you could put your hands on and play with. I mean, Kubernetes is a great example of this, and this is actually why I referred to it. [00:18:19] CC: The way that works for me when I’m learning something like that is to really think about where the boundaries are, where the limitations are, where the tradeoffs are. If you can take a smaller system, maybe something like The Kind Project and identify what those things are. If you can’t, then ask around. Ask someone. Google it. I don’t know. Maybe it will be a good episode topic for us to do that. This part is doing this to map things out. So maybe we can understand better and help people understand things better. So mainly like yeah. They try to do the distributed system thesis are. But for people who don’t even know what they could be, it’s harder to identify it. I don’t know what a good strategy for that would be, because you can read about distributed systems and then you can go and look at a project. How do you map the concept to learning to what you’re seeing in the code base? For me, that’s the hardest thing. [00:19:26] MG: Exactly. Something that kind of I had related experience was like when I went into software development, without having formal education on algorithms and data structures, sometimes in your head, you have the problem statement and you're like, “Okay. I would do it like that.” But you don't know the word that describes, for example, a heap structure or queue because you’ve never – Someone told you that is heap, that is a queue, and/or that is a stick. So, for me, reading the book was a bit easier. Even though I have done distributed systems, if you will, administration for many years, many years ago, I didn't realize that it was a distributed system because I never had this definition or I never had those failure scenarios in mind and it never had a word for consensus. So how would I search for something like how do machines agree? I mean, if you put that on Google, then likely they will come – Have a lot of stuff. But if you put it in consensus algorithm, likely you get a good hit on what the answer should be. [00:20:29] CC: It is really problematic when we don't know the names of things because – What you said is so right, because we are probably doing a lot of distributed systems without even knowing that that’s what it is. Then we go in the job interview, and people are, “Oh! Have you done a distributed system?” No. You have but you just don’t know how to name things. But that’s one – [00:20:51] DC: Yeah, exactly. [00:20:52] CC: Yeah. Right? That’s one issue. Another issue, which is a bigger issue though is at least that’s how it is for me. I don’t want to speak for anybody else but for me definitely. If I can’t name things and I face a problem and I solve it, every time I face that problem it’s a one-off thing because I can’t map to a higher concept. So every time I face that problem, it’s like, “Oh!” It’s not like, “Oh, yeah!” If this is this kind of problem, I have a pattern. I’m going to use that to this problem. So that’s what I’m saying. Once you learn the concept, you need to be able to name it. Then you can map that concept to problems you have. All of a sudden, if you have like three things [inaudible 00:21:35] use to solve this problem, because as you work with computers, coding, it’s like you see the same thing over and over again. But when you don’t understand the fundamentals, things are just like – It’s a bunch of different one-offs. It’s like when you have an argument with your spouse or girlfriend or boyfriend. Sometimes, it’s like you’re arguing 10 times in a month and you thought, “Oh! I had 10 arguments.” But if you’d stop and think about it, no. We had one argument 10 times. It’s very different than having 10 problems versus having 1 problem 10 times, if that makes sense. [00:22:12] MG: It does. [00:22:11] DC: I think it does, right? [00:22:12] MG: I just want to agree. [00:22:16] DC: I think it does make sense. I think it’s interesting. You’ve highlighted kind of an interesting pattern around the way that people learn, which I think is really interesting. That is like some people are able to read about patterns or software patterns or algorithms or architectures and have that suddenly be an index of their heads. They can actually then later on correlate what they've read with the experience that they’re having around the things they're working on. For some, it needs to be hands-on. They need to actually be able to explore that idea and understand and manipulate it and be able to describe how it works or functions in person, in reality. They need to have that hands-on like, “I need to touch it to understand it,” kind of experience. Those people also, as they go through those experiences, start building this index of patterns or algorithms in their head. They have this thing that they can correlate to, right, like, “Oh! This is a time problem,” or, “This is a consensus problem,” or what have you, right? [00:23:19] CC: Exactly. [00:23:19] DC: You may not know the word for that saying but you're still going to develop a pattern in your mind like the ability to correlate this particular problem with some pattern that you’ve seen before. What's interesting is I feel like people have taken different approaches to building that index, right? For me, it’s been troubleshooting. Somebody gives me a hard problem, and I dig into it and I figure out what the problem is, regardless of whether it's to do with distributed systems or cooking. It could be anything, but I always want to get right in there and figure out what that problem and start building a map in my mind of all of the players that are involved. For others, I feel like with an educational background, if you have an education background, I think that sometimes you end up coming to this with a set of patterns already instilled that you understand and you're just trying to apply those patterns to the experience you’re having instead. It’s just very – It’s like horse before the cart or cart before the horse. It’s very interesting when you think about it. [00:24:21] CC: Yes. [00:24:22] MG: The recommendation that I just want to give to people that are like me who like reading is that I went overboard a bit in the beginnings because I was so fascinated by all the stuff, and it went down the rabbit hole deeper, deeper, deeper, deeper. Reading and reading and reading. At some point, even coming to weird YouTube channels that talk about like, “Is time real and where does time emerge from?” It became philosophical even like the past where I went to. Now, the thing is, and this is why I like Duffie’s approach with like breaking things and then undergo like trying to break things and understanding how they work and how they can fail is that immediately you practice. You’re hands-on. So that would be my advice to people who are more like me who are fascinated by reading and all the theory that your brain and your mind is not really capable of kind of absorbing all the stuff and then remembering without practicing. Practicing can be breaking things or installing things or administrating things or even writing software. But for me, that was also a late realization that I should have maybe started doing things earlier than the time I spent reading. [00:25:32] CC: By doing, you mean, hands-on? [00:25:35] MG: Yeah. [00:25:35] CC: Anything specific that you would have started with? [00:25:38] MG: Yes. On Kubernetes – So going back those 15 years to my early days of Linux and Samba, which is a project. By the time, I think it was written in C or C++. But the problem was I wasn’t able to read the code. So the only thing that I had by then was some mailing lists and asking questions and not even knowing which questions to ask because of lack of words of understanding. Now, fast-forward into Kubernetes’ time, which got me deeper in distributed systems, I still couldn't read the code because I didn't know [inaudible 00:26:10]. But I forced myself to read the code, which helped a little bit for myself to understand what was going on because the documentation by then was lacking. These days, it’s easier, because you can just install [inaudible 00:26:20] way easier today. The hands-on piece, I mean. [00:26:23] CC: You said something interesting, Michael, and I have given this advice before because I use this practice all the time. It's so important to have a vocabulary. Like you just said, I didn't know what to ask because I didn’t know the words. I practice this all the time. To people who are in this position of distributed systems or whatever it is or something more specific that you are trying to learn, try to expose yourself to the words, because our brains are amazing. Once you get exposure, it’s like your brain works in the background. All of a sudden, you go, “Oh, yeah! I know this word.” So podcasts are great for me. If I don't know something, I will look for a podcast on the subject and I start listening to it. As the words get repeated, just contextually. I don’t have to go and get a degree or anything. Just by listening to the words being spoken in context, absorb the meaning of it. So podcasting is great or YouTube or anything that you can listen. Just in reading too, of course. The best thing is talking to people. But, again, it’s really – Sometimes, it’s not trivial to put yourself in positions where people are discussing these things. [00:27:38] DC: There are actually a number of Meetups here in the Bay Area, and there’s a number of Meetups – That whole Meetup thing is sort of nationwide across the entire US and around the world it seems like now lately. Those Meetups I feel like there are a number of Meetups in different subject areas. There’s one here in the Bay Area called Papers We Love, where they actually do explore interesting technical papers, which are obviously a great place to learn the words for things, right? This is actually where those words are being defined, right? When you get into the consensus stuff, they really get into – One even is Raft. There are many papers on Raft and many papers on multiple things that get into consensus. So definitely, whether you explore a meetup on a distributed system or in a particular application or in a particular theme like Kubernetes, those things are great places just to kind of get more exposure to what people are thinking about in these problems. [00:28:31] CC: That is such a great tip. [00:28:34] MG: Yeah. The podcast is twice as good as well, because for people, non-natives – English speaker, I mean. Oh, people. Not speakers. People. The thing is that the word you’re looking for might be totally different than the English word. For example, consensus in Germany has this totally different meaning. So if I would look that up in German, likely I would find nothing or not really related at all. So you have to go through translation and then finding the stuff. So what you said, Duffie, with PWL, Papers We Love, or podcasts, those words, often they are in English, those podcasts and they are natural consensus or charting or partitioning. Those are the words that you can at least look up like what does it mean. That’s what I did as well thus far. [00:29:16] CC: Yes. I also wanted to do a plus one for Papers We Love. It’s – They are everywhere and they also have an online. They have an online version of the Papers We Love Meetup, and a lot of the local ones film their meetups. So you can go through the history and see if they talked about any paper that you are interested in. Probably, I’m sure multiple locations talk about the same paper, so you can get different takes too. It’s really, really cool. Sometimes, it’s completely obscure like, “I didn’t get a word of what they were saying. Not one. What am I doing here?” But sometimes, they talk about things. You at least know what the thing is and you get like 10% of it. But some paper you don’t. People who deal with papers day in and day out, it’s very much – I don’t know. [00:30:07] DC: It’s super easy when going through a paper like that to have the imposter syndrome wash over you, right, because you’re like – [00:30:13] CC: Yes. Thank you. That’s what I wanted to say. [00:30:15] DC: I feel like I’ve been in this for 20 years. I probably know a few things, right. But in talking about reading this consensus paper going, “Can I buy a vowel? What is happening?” [00:30:24] CC: Yeah. Can I buy a vowel? That’s awesome, Duffie. [00:30:28] DC: But the other piece I want to call out to your point, which I think is important is that some people don't want to go out and be there in person. They don’t feel comfortable or safe exploring those things in person. So there are tons of resources like you have just pointed out like the online version of Papers We Love. You can also sign into Slack and just interact with people via text messaging, right? There’s a lot of really great resources out there for people of all types, including the amount of time that you have. [00:30:53] CC: For Papers We Love, it’s like going to language class. If you go and take a class in Italian, your first day, even though that is going to be super basic, you’re going to be like, “What?” You’ll go back in your third week. You start, “Oh! I’m getting this.” Then a month, three months, “Oh! I’m starting to be competent.” So you go once. You’re going to feel lost and experience imposter syndrome. But you keep going, because that is a format. First, you start absorbing what the format is, and that helps you understand the content. So once your mind absorbs the format, you’re like, “Okay. Now, I have – I know how to navigate this. I know what’s coming next.” So you don’t have to focus on that. You start focusing in the content. Then little but little, you become more proficient in understanding. Very soon, you’re going to be willing to write a paper. I’m not there yet. [00:31:51] DC: That’s awesome. [00:31:52] CC: At least that’s how I think it goes. I don’t know. [00:31:54] MG: I agree. [00:31:55] DC: It’s also changed over time. It’s fascinating. If you read papers from like 20 years ago and you read papers that are written more recently, it's interesting. The papers have changed their language when considering competition. When you're introducing a new idea with a paper, frequently that you are introducing it into a market full of competition. You're being very careful about the language, almost in a way to complicate the idea rather than to make it clear, which is challenging. There are definitely some papers that I’ve read where I was like, “Why are you using so many words to describe this simple idea?” It makes no sense, but yeah. [00:32:37] CC: I don’t want to make this episode all about Papers We Love. It was so good that you mentioned that, Duffie. It’s really good to be in a room where we’ll be watching something online where you see people asking questions and people go, “Oh! Why is this thing like this? Why is X like this,” or, “Why is Y doing like this?” Then you go, “Oh! I didn’t even think that X was important. I didn’t even know that Y was important.” So you stop picking up what the important things are, and that’s what makes it click is now you’ve – Hooking into the important concepts because people who know more than you are pointing out and asking questions. So you start paying attention to learning what the main things it should be paying attention to, which is different from reading the paper by yourself. It’s just a ton of content that you need to sort through. [00:33:34] DC: Yeah. I frequently self-describe it as a perspective junkie, because I feel like for any of us really to learn more about a subject that we feel we understand, we need the perspective of others to really engage, to expand our understanding of that thing. I feel like and I know how to make a peanut butter and jelly sandwich. I’ve done it a million times. It’s a solid thing. But then I watch my kid do it and I’m like, “I hadn’t thought of that problem.” [inaudible 00:33:59], right? This is a great example of that. Those communities like Papers We Love are great opportunity to understand the perspective of others around these hard ideas. When we’re trying to understand complex things like distributed systems, this is where it’s at. This is actually how we go about achieving this. There is a lot that you can do on your own but there is always going to be more that you can do together, right? You can always do more. You can always understand this idea faster. You can understand the complexity of a system and how to break it down into these things by exploiting it with other people. That's I feel like – [00:34:40] CC: That is so well said, so well said, and it’s the reason for this show to exist, right? We come on a show and we give our perspectives, and people get to learn from people with different backgrounds, what their takes are on distributed systems, cloud native. So this was such a major plug for the show. Keep coming back. You’re going to learn a ton. Also, it was funny that you – It was the second time you mentioned cooking, made a cooking reference, Duffie, which brings me to something I want to make sure I say on this episode. I added a few things for reference, three books. But the one that I definitely would recommend starting with is The Systems Bible by John Gall. This book is so cool, because it helps you see everything through systems. Everything is a system. A conversation can be a system. An interaction between two people can be a system. I’m not saying this book says that. It’s just like my translation and that you can look – Cooking is a system. There is a process. There is a sequence. It’s really, really cool and it really helps to have things framed in this way and then go out and read the other books on systems. I think it helps a lot. This is definitely what I am starting with and what I would recommend people start with, The Systems Bible. Did you two know this book? [00:36:15] MG: I did not. I don’t. [00:36:17] DC: I’m not aware of it either but I really appreciate the idea. I do think that that's true. If you develop a skill for understanding systems as they are, you’ll basically develop – Frequently, what you’re developing is the ability to recognize patterns, right? [00:36:32] CC: Exactly. [00:36:32] DC: You could recognize those patterns on anything. [00:36:37] MG: Yeah. That's a good segue for just something that came to my mind. Recently, I gave a talk on event-driven architectures. For someone who's not a software developer or architect, it can be really hard to grab all those concepts on asynchrony and eventual consistency and idempotency. There are so many words of like, “What is this all – It sounds weird, way too complex.” But I was reading a book some years ago by Gregor Hohpe. He’s the guy behind Enterprise Integration Patterns. That’s also a book that I have on my list here. He said, “Your barista doesn't use two-phase commit.” So he was basically making this analogy of he was in a coffee shop and he was just looking at the process of how the barista makes the coffee. You pay for it and all the things that can go wrong while your coffee is brewed and served to you. So he was making this relation between the real world and the life and human society to computer systems. There it clicked to me where I was like, “So many problems we solve every day, for example, agreeing on a time where we should meet for dinner or cooking, is a consensus problem, and we solve it.” We even solve it in the case of failure. I might not be able to call Duffie, because he is not available right now. So somehow, we figure out. I always thought that those problems just exist in computer science and distributed systems. But I realized actually that's just a subset of the real world as is. Looking at those problems through the lens of your daily life and you get up and all the stuff, there are so many things that are related to computer systems. [00:38:13] CC: Michael, I missed it. Was it an article you read? [00:38:16] MG: Yes. I need to put that in there as well. Yeah. It’s a plug. [00:38:19] CC: Please put that in there. Absolutely. So far from being any kind of expert in distributed systems, but I have noticed. I have caught myself using systems thinking for even complicated conversations. Even in my personal life, I started approaching things in the systems oriented and just the – just a high-level example. When I am working with systems, I can approach from the beginning, the end. It’s like a puzzle, putting the puzzle together, right? Sometimes, it starts from the middle. Sometimes, it starts from the edges. When I‘m having conversations that I need to be very strategic like I have one shot. Let’s say maybe I’m in a school meeting and I have to reach a consensus or have a solution or have a plan of action. I have to ask the right questions. My private self would do things linearly. Historically like, “Let’s go from the beginning and work out through the end.” Now, I don’t do that anymore. Not necessarily. Sometimes, I like, “Let me maybe ask the last question I would ask and see where it leads and just approach things from a different way.” I don’t know if this is making sense. [00:39:31] MG: It does. It does. [00:39:32] CC: But my thinking has changed. The way I see the possibilities is not a linear thing anymore. I see how you can truly switch things. I use this in programming a lot and also writing. Sometimes, when you’re a beginner writer, you start at the top and you go down to the conclusion. Sometimes, I start I the middle and go up, right? So you can start anywhere. It’s beautiful or it just gives you so many more options. Or maybe I’m just crazy. Don’t listen to me. [00:40:03] DC: I don’t think you’re crazy. I was going to say, one of the funny things about Michael’s point and your point both, it’s like in a way that they have kind of referred to Conway's law, the idea that people will build systems in the way that they communicate. So this is actually – It totally brings it back to that same point of thing, right? We by nature will build systems that we can understand, because that is the constraint in which we have to work, right? So it’s very interesting. [00:40:29] CC: Yeah. But it’s an interesting thing, because we are [inaudible 00:40:32] by the way we are forced to work. For example, I work with constraints and what I'm saying is that that has been influencing my way of thinking. So, yes, I built systems in the way I think but also because of the constraints that I’m dealing with that I have to be – the tradeoffs I need to make, that also turns around and influences the way I think, the way I see the world and the rest of the systems and all the rest of the world. Of course, as I change my thinking, possibly you can theorize that you go back and apply that. Apply things that you learn outside of your work back to your work. It’s a beautiful back-and-forth I think. [00:41:17] MG: I had the same experience with some – When I had to design kind of my first API and think of, “Okay. What would the consumer contract be and what would a consumer expect me to deliver in response and so on?” I was forcing myself and being explicit in communicating and not throwing everything at the client back to confusing but being very explicit and precise. Also on communication every day when you talk to people, being explicit and precise really helps to avoid a lot of problems and trouble. Be it partnership or amongst friends or at work. This is what I took from computer science actually back into my real world in order to taking all those perceptions, perceiving things from a different perspective, and being more precise and explicit in how I respond or communicated. [00:42:07] CC: My take on what you just said, Michael, is we design systems thinking how is this going to fail. We know this is going to fail. We’re going to design for that. We’re going to implement for that. In real life, for example, if I need to get an agreement from someone, I try to understand the person's thinking and just go, “I just had this huge thing this week. This is in my mind.” I’m not constantly thinking about this, I’m not crazy like that. Just a little bit crazy. It’s like, “How does this person think? What do they need to know? How far can I push?” Right? We need to make a decision quickly, so the approach is everything, and sometimes you only get one shot, so yeah. I mean, correct me if I’m wrong. That's how I heard or I interpreted what you just said. [00:42:52] MG: Yeah, absolutely. Spot on. Spot on. So I’m not crazy as well. [00:42:55] CC: Basically, I think we ended up turning this episode into a little bit of like, “Here are great references,” and also a huge endorsement for really going deep into distributed systems, because it’s going to be good for your jobs. It’s going to be good for your life. It’s going to be good for your health. We are crazy. [00:43:17] DC: I’m definitely crazy. You guys might be. I’m not. All right. So we started this episode with the idea of coming to learning distributed systems perhaps without a degree or without a formal education in it. We talked about a ride of different ideas on that subject. Like different approaches that each of us took, how each of us see the problem. Is there any important point that either of you want to throw back into the mix here or bring up in relation to that? [00:43:48] MG: Well, what I take from this episode, being my first episode and getting to know your background, Duffie and Carlisia, is that whoever is going to listen to this episode, whatever background you have, even though you might not be in computer systems or industry at all, I think we three all had approved that whatever background you have, if you’re just curious a little bit and maybe a little bit crazy, you can totally get down the rabbit hole in distributed systems and get totally excited about it. There’s no need for having formal education and the degree to enter this world. It might help but it’s kind of not a high bar that I was perceiving it to be 10 years ago, for example. [00:44:28] CC: Yeah. That’s a good point. My takeaway is it always puzzled me how some people are so good and experienced and such experts in distributed systems. I always look at myself. It’s like, “How am I lacking?” It’s like, “What memo did I miss? What class did I miss? What project did I not work on to get the experience?” What I’m seeing is you just need to put yourself in that place. You need to do the work. But the good news is achieving competency in distributed systems is doable. [00:45:02] DC: My takeaway is as we discussed before, I think that there is no one thing that comprises a distributed system. It is a number of things, right, and basically a number of behaviors or patterns that we see that comprise what a distributed system is. So when I hear people say, “I’m not an expert in distributed systems,” I think, “Well, perhaps you are and maybe you don’t know it already.” Maybe there's some particular set of patterns with which you are incredibly familiar. Like you understand DNS better than the other 20 people in the room. That exposes you to a set of patterns that certainly give you the capability of saying that you are an expert in that particular set of patterns. So I think that to both of your points, it’s like you can enter this stage where you want to learn about distributed systems from pretty much any direction. You can learn it from a CIS background. You can come it with no computer experience whatsoever, and it will obviously take a bit more work. But this is really just about developing and understanding around how these things communicate and the patterns with which they accomplish that communication. I think that’s the important part. [00:46:19] CC: All right, everybody. Thank you, Michael Gasch, for being with us now. I hope to – [00:46:25] MG: Thank you. [00:46:25] CC: To see you in more episodes [inaudible 00:46:27]. Thank you, Duffie. [00:46:30] DC: My pleasure. [00:46:31] CC: Again, I’m Carlisia Campos. With us was Duffie Cooley and Michael Gesh. This was episode 12, and I hope to see you next time. Bye. [00:46:41] DC: Bye. [00:46:41] MG: Goodbye. [END OF EPISODE] [00:46:43] ANNOUNCER: Thank you for listening to The Podlets Cloud Native Podcast. Find us on Twitter at https://twitter.com/ThePodlets and on the http://thepodlets.io/ website, where you'll find transcripts and show notes. We'll be back next week. Stay tuned by subscribing. [END]See omnystudio.com/listener for privacy information.

Bryan Reinero talks with Gregor Hohpe about IT Transformation, the process by which organizations adapt and reorganize themselves in response to evolution and how the Enterprise Architect leads that transformation.

Bryan Reinero talks with Gregor Hohpe  about IT Transformation, the process by which organizations adapt and reorganize themselves in response to the way our industry has evolved. Gregor discusses the role of the Enterprise Architect as a leader in the process of  IT Transformation by assessing pressures affecting engineering teams, available technologies, and  business goals […]

01:11 – What it means to be a Senior Engineer When You Don’t Want to Go Into Management 05:07 – Generativity: The difference between your team’s output with you on it and your team’s output without you. Gregor Hohpe: 37 Things One Architect Knows About IT Transformation (https://leanpub.com/37things) 13:46 – The Job of An Architect 22:09 – What are the managers doing? “It is too much to ask for your manager to be your career mentor?” Sam Gerstenzang: The Happy Demise of the 10X Engineer (https://a16z.com/2014/07/30/the-happy-demise-of-the-10x-engineer/) Everything's an Argument by Andrea A. Lunsford, et al. (https://www.amazon.com/gp/product/131908575X/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=131908575X&linkCode=as2&tag=therubyrep-20&linkId=d08b37c3206978c7992ae8c263c1f1dd) Reflections: Astrid: From Jessica’s perspective, what a software architect actually is supposed to be doing. Sam: Everything's an Argument by Andrea A. Lunsford, et al. (https://www.amazon.com/gp/product/131908575X/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=131908575X&linkCode=as2&tag=therubyrep-20&linkId=d08b37c3206978c7992ae8c263c1f1dd) Coraline: I’m going to read ^ book. Jessica: More surprise episodes! Darin: Think less about me and more about the team around me. And I’m looking for a mentor! Cheryl: Greater Than Code > Mission Taco This episode was brought to you by @therubyrep (https://twitter.com/therubyrep) of DevReps, LLC (http://www.devreps.com/). To pledge your support and to join our awesome Slack community, visit patreon.com/greaterthancode (https://www.patreon.com/greaterthancode). To make a one-time donation so that we can continue to bring you more content and transcripts like this, please do so at paypal.me/devreps (https://www.paypal.me/devreps). You will also get an invitation to our Slack community this way as well. Amazon links may be affiliate links, which means you’re supporting the show when you purchase our recommendations. Thanks!_T Special Guests: Cheryl Shaefer and Darin Wilson.

Check out RailsClips!   02:38 - Derick Bailey Introduction Twitter GitHub Blog Entreprogrammers RabbitMQ: Patterns for Applications by Derick Bailey 03:36 - RabbitMQ request-response Messaging Pattern 05:22 - Synchronous/Asynchronous; Chronological/Non-Chronological 10:33 - Why Do JS Devs Care About RabbitMQ? 12:10 - RabbitMQ and Complexity 14:04 - RabbitMQ’s Model Pub/Sub - Redis Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions by Gregor Hohpe Exchanges, Queues, and Bindings 22:15 - Event Emitters, Organizing Your Code Documentation 31:18 - Service Busses & Monitoring Systems NServiceBus 32:58 - How do you decide you need a messaging system? 36:40 - When Applications Crash… 39:24 - Event Sourcing Kafka 44:05 - Fault Tolerance/Failure Cases “Just let it fail” 50:21 - Putting RabbitMQ in Place Scheduling Long Wait vs Short Wait 58:28 - Formatting Your Messages RabbitMQ: Patterns for Applications by Derick Bailey 01:04:13 - “Saga” (Workflow) 01:05:10 - RabbitMQ For Developers Use code JSJABBER for 20% off the bundle! Picks W3Schools (AJ) 1984 by George Orwell (AJ) The edit button on the MDN page (AJ) [YouTube] W3Schools is just... Better (AJ) The Go Programming Language (AJ) [YouTube] Go Programming: Learn the Go Programming Language in One Video (AJ) hackthe.computer (AJ) Maze Algorithm (AJ) A* Algorithm (AJ) React Rally (Jamison) Web Design: The First 100 Years (Jamison) Evan Czaplicki: Let's be mainstream! User focused design in Elm @ Curry On Prague 2015 (Jamison) Paracord (Chuck) Soto Pocket Torch (Chuck) Exploring ES6: Upgrade to the next version of JavaScript by Dr. Axel Rauschmayer (Derick) Small World (Derick) Star Wars Darth Bane Trilogy (Derick) LEGO Star Wars The Empire Strikes Back Slave I Set #75060 (Derick)

Check out RailsClips!   02:38 - Derick Bailey Introduction Twitter GitHub Blog Entreprogrammers RabbitMQ: Patterns for Applications by Derick Bailey 03:36 - RabbitMQ request-response Messaging Pattern 05:22 - Synchronous/Asynchronous; Chronological/Non-Chronological 10:33 - Why Do JS Devs Care About RabbitMQ? 12:10 - RabbitMQ and Complexity 14:04 - RabbitMQ’s Model Pub/Sub - Redis Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions by Gregor Hohpe Exchanges, Queues, and Bindings 22:15 - Event Emitters, Organizing Your Code Documentation 31:18 - Service Busses & Monitoring Systems NServiceBus 32:58 - How do you decide you need a messaging system? 36:40 - When Applications Crash… 39:24 - Event Sourcing Kafka 44:05 - Fault Tolerance/Failure Cases “Just let it fail” 50:21 - Putting RabbitMQ in Place Scheduling Long Wait vs Short Wait 58:28 - Formatting Your Messages RabbitMQ: Patterns for Applications by Derick Bailey 01:04:13 - “Saga” (Workflow) 01:05:10 - RabbitMQ For Developers Use code JSJABBER for 20% off the bundle! Picks W3Schools (AJ) 1984 by George Orwell (AJ) The edit button on the MDN page (AJ) [YouTube] W3Schools is just... Better (AJ) The Go Programming Language (AJ) [YouTube] Go Programming: Learn the Go Programming Language in One Video (AJ) hackthe.computer (AJ) Maze Algorithm (AJ) A* Algorithm (AJ) React Rally (Jamison) Web Design: The First 100 Years (Jamison) Evan Czaplicki: Let's be mainstream! User focused design in Elm @ Curry On Prague 2015 (Jamison) Paracord (Chuck) Soto Pocket Torch (Chuck) Exploring ES6: Upgrade to the next version of JavaScript by Dr. Axel Rauschmayer (Derick) Small World (Derick) Star Wars Darth Bane Trilogy (Derick) LEGO Star Wars The Empire Strikes Back Slave I Set #75060 (Derick)

Check out RailsClips!   02:38 - Derick Bailey Introduction Twitter GitHub Blog Entreprogrammers RabbitMQ: Patterns for Applications by Derick Bailey 03:36 - RabbitMQ request-response Messaging Pattern 05:22 - Synchronous/Asynchronous; Chronological/Non-Chronological 10:33 - Why Do JS Devs Care About RabbitMQ? 12:10 - RabbitMQ and Complexity 14:04 - RabbitMQ’s Model Pub/Sub - Redis Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions by Gregor Hohpe Exchanges, Queues, and Bindings 22:15 - Event Emitters, Organizing Your Code Documentation 31:18 - Service Busses & Monitoring Systems NServiceBus 32:58 - How do you decide you need a messaging system? 36:40 - When Applications Crash… 39:24 - Event Sourcing Kafka 44:05 - Fault Tolerance/Failure Cases “Just let it fail” 50:21 - Putting RabbitMQ in Place Scheduling Long Wait vs Short Wait 58:28 - Formatting Your Messages RabbitMQ: Patterns for Applications by Derick Bailey 01:04:13 - “Saga” (Workflow) 01:05:10 - RabbitMQ For Developers Use code JSJABBER for 20% off the bundle! Picks W3Schools (AJ) 1984 by George Orwell (AJ) The edit button on the MDN page (AJ) [YouTube] W3Schools is just... Better (AJ) The Go Programming Language (AJ) [YouTube] Go Programming: Learn the Go Programming Language in One Video (AJ) hackthe.computer (AJ) Maze Algorithm (AJ) A* Algorithm (AJ) React Rally (Jamison) Web Design: The First 100 Years (Jamison) Evan Czaplicki: Let's be mainstream! User focused design in Elm @ Curry On Prague 2015 (Jamison) Paracord (Chuck) Soto Pocket Torch (Chuck) Exploring ES6: Upgrade to the next version of JavaScript by Dr. Axel Rauschmayer (Derick) Small World (Derick) Star Wars Darth Bane Trilogy (Derick) LEGO Star Wars The Empire Strikes Back Slave I Set #75060 (Derick)

In this Episode we talked about the new POSA 4 book which has recently been published. We talk to two of the authors, Kevlin Henney and Frank Buschmann (the third author, Doug Schmidt was not available - and he had also been on the podcast a couple of times :-)). The book contains a pattern language for distributed systems. It contains 114 patterns that had been published before by many different other authors. The patterns have been rewritten to form a consistent language. We basically talked through the different sections of the book, which gives a really good overview over the challenges and the solutions of building distributed systems. These sections include From Mud to Structure, Distribution Infrastructure, Event Demultiplexing and Dispatching, Interface Partitioning, Component Patitioning, Application Contrl, Concurrency, Synchronization, Object Interaction, Adaptazion and Extension, Modal Behaviour, Resource Management and finally, Database Access. The book references several other previous works (as listed below). Interestingly, many of these referenced works and authors have also been discussed previously on the podcast. Here are the back references: Domain Driven Design, Eric Evans Messaging Patterns, Gregor Hohpe POSA 2 Patterns, Doug Schmidt Concurrency: Part 1, Part 2, Part 3 and the interview with Goetz and Holmes Remoting Patterns Part 1 and Part 2 POSA3, Resource Management

In this Episode we talked about the new POSA 4 book which has recently been published. We talk to two of the authors, Kevlin Henney and Frank Buschmann (the third author, Doug Schmidt was not available - and he had also been on the podcast a couple of times :-)). The book contains a pattern language for distributed systems. It contains 114 patterns that had been published before by many different other authors. The patterns have been rewritten to form a consistent language. We basically talked through the different sections of the book, which gives a really good overview over the challenges and the solutions of building distributed systems. These sections include From Mud to Structure, Distribution Infrastructure, Event Demultiplexing and Dispatching, Interface Partitioning, Component Patitioning, Application Contrl, Concurrency, Synchronization, Object Interaction, Adaptazion and Extension, Modal Behaviour, Resource Management and finally, Database Access. The book references several other previous works (as listed below). Interestingly, many of these referenced works and authors have also been discussed previously on the podcast. Here are the back references: Domain Driven Design, Eric Evans Messaging Patterns, Gregor Hohpe POSA 2 Patterns, Doug Schmidt Concurrency: Part 1, Part 2, Part 3 and the interview with Goetz and Holmes Remoting Patterns Part 1 and Part 2 POSA3, Resource Management

In this Episode we talked about the new POSA 4 book which has recently been published. We talk to two of the authors, Kevlin Henney and Frank Buschmann (the third author, Doug Schmidt was not available - and he had also been on the podcast a couple of times :-)). The book contains a pattern language for distributed systems. It contains 114 patterns that had been published before by many different other authors. The patterns have been rewritten to form a consistent language. We basically talked through the different sections of the book, which gives a really good overview over the challenges and the solutions of building distributed systems. These sections include From Mud to Structure, Distribution Infrastructure, Event Demultiplexing and Dispatching, Interface Partitioning, Component Patitioning, Application Contrl, Concurrency, Synchronization, Object Interaction, Adaptazion and Extension, Modal Behaviour, Resource Management and finally, Database Access. The book references several other previous works (as listed below). Interestingly, many of these referenced works and authors have also been discussed previously on the podcast. Here are the back references: Domain Driven Design, Eric Evans Messaging Patterns, Gregor Hohpe POSA 2 Patterns, Doug Schmidt Concurrency: Part 1, Part 2, Part 3 and the interview with Goetz and Holmes Remoting Patterns Part 1 and Part 2 POSA3, Resource Management

In this episode, Gregor Hohpe gives us a great introduction to enterprise messaging based on his EAI Patterns book. Before we started discusssing the patterns in his book, we characterized messaging and talked about the various interaction styles. We also contrasted the messaging architectural style with an RPC based approach. We then took a look at the relationship to SOA, the role of contracts and the orchestration-vs-choreography discussion. We briefly discussed the nature of pattern languages before we then went through the different section in the book. There are six main sections: channel, message, routing, transfomation, endpoint as well as management and monitoring. We discussed the core patterns for each of these sections. This should give listeners a good high-level view of message-based systems. We concluded the discussion by looking at the critical importance of systems management and monitoring.

In this episode, Gregor Hohpe gives us a great introduction to enterprise messaging based on his EAI Patterns book. Before we started discusssing the patterns in his book, we characterized messaging and talked about the various interaction styles. We also contrasted the messaging architectural style with an RPC based approach. We then took a look at the relationship to SOA, the role of contracts and the orchestration-vs-choreography discussion. We briefly discussed the nature of pattern languages before we then went through the different section in the book. There are six main sections: channel, message, routing, transfomation, endpoint as well as management and monitoring. We discussed the core patterns for each of these sections. This should give listeners a good high-level view of message-based systems. We concluded the discussion by looking at the critical importance of systems management and monitoring.

In this episode, Gregor Hohpe gives us a great introduction to enterprise messaging based on his EAI Patterns book. Before we started discusssing the patterns in his book, we characterized messaging and talked about the various interaction styles. We also contrasted the messaging architectural style with an RPC based approach. We then took a look at the relationship to SOA, the role of contracts and the orchestration-vs-choreography discussion. We briefly discussed the nature of pattern languages before we then went through the different section in the book. There are six main sections: channel, message, routing, transfomation, endpoint as well as management and monitoring. We discussed the core patterns for each of these sections. This should give listeners a good high-level view of message-based systems. We concluded the discussion by looking at the critical importance of systems management and monitoring.