Podcast appearances and mentions of nicole forsgren

How do you eliminate the friction of development? Carl and Richard talk to Nicole Forsgren about her upcoming book on eliminating the friction from software development. Building on her earlier book, Accelerate, Nicole discusses the role of AI technologies in software development, along with more traditional DevOps elements, such as automating testing, deployment, telemetry, and more. There's never been a better time to pay attention to your tools and methods when it comes to software development - when you improve your workflow, your productivity soars!

In this wide-ranging episode, Casey and Christine dive deep into leadership transitions, complex systems, and the art of managing large-scale projects. Christine discusses her new role as Head of Engineering at a Series D startup, managing 60+ people and Casey discusses the counterintuitive challenges of scaling teams. Christine shares her synthetic biology progress, building an orbital shaker from scratch and exploring the intersection of hardware and biotech. The conversation spans from ancient Roman infrastructure projects to modern space exploration, touching on hiring challenges, the value of going to primary sources, and why some historical civilizations came tantalizingly close to industrial revolution but never quite made the leap.Arabian Sands by Wilfred ThesigerNow It Can Be Told about the Manhattan ProjectWhat Got You Here Won't Get You There by Marshall GoldsmithAccelerate by Nicole Forsgren, Jez Humble, and Gene KimRadical Candor by Kim ScottPlayground - Contemporary science fiction novel about seasteading and AIClimbing Gold - Alex Honnold's podcast (featuring Vitaliy Musiyenko's Sierra traverse story)Conversations with Tyler - Tyler Cowen's interview showLex Fridman Podcast - Episodes with Jeff Wasserstrom (China scholar) and Tim Sweeney (Epic Games)The Complete History & Strategy of Standard Oil (Part I) (Acquired)Founders Podcast - Episodes on Jeff Bezos shareholder letters, Jim Simons, John D. Rockefeller, and Estée LauderHow Andreessen Horowitz Disrupted VC & What's Coming Next (Ben and Marc Discussions)The DER Task Force - Distributed energy resources podcast (Jesse Peltan episode)Main Engine Cut Off - Space industry podcast T+303: The Trump 2024 Transition (with Mark Albrecht)The Almost-Industrial Revolutions of Rome and China Cost of Glory - Ancient history podcast (Julius Caesar series)Circuit Playground Express - Adafruit electronics learning boardAda Fruit Orbital Shaker Tutorial - DIY lab equipment projectThinking by My Wits - Website featuring Scholar Alex Jones's literary works

In episode 33 of Generationship, Rachel Chalmers is joined by Nicole Forsgren—developer productivity researcher and co-founder of DORA—to discuss how AI is reshaping the software development landscape. From coding with LLMs to evaluating trust in AI-generated outputs, they explore the future of developer experience and the enduring value of human intuition.

In episode 33 of Generationship, Rachel Chalmers is joined by Nicole Forsgren—developer productivity researcher and co-founder of DORA—to discuss how AI is reshaping the software development landscape. From coding with LLMs to evaluating trust in AI-generated outputs, they explore the future of developer experience and the enduring value of human intuition.

CTO Series: Engineering Leadership, Automation, and Trust with Dan Hollinger   In this CTO Series episode, we sit down with Dan Hollinger, an accomplished engineering leader passionate about fostering empathy, transparency, and trust in tech teams. Dan shares pivotal lessons from his career, from building scalable automation systems to navigating complex leadership challenges. We cover key strategies for aligning tech initiatives with business goals, fostering collaboration, and ensuring long-term technical health.   Defining Leadership Through Automation and Empowerment   “Enable your humans to focus on the interesting work—automation should take care of the rest.”   Dan recounts his transformative experience at CCP Games, makers of EVE Online, where a robust test automation system changed his perspective on scaling technical processes. This role introduced him to the power of automation in freeing up engineers to focus on more exploratory and impactful tasks. He emphasizes how empowering self-directed teams with high-level vision statements enables creativity and innovation.   Building Self-Correcting Processes   “Always retro your processes—don't let them run on autopilot.”   Dan explains the importance of self-correcting processes, using the SEV (Side Event) system as an example. He highlights how retrospectives can improve response times and prevent future crises. For Dan, consistent reviews are the key to maintaining agile, resilient systems that adapt to evolving needs.   Bridging the Gap Between Business and Tech   “There are no enemies—treat your colleagues like allies working toward a common goal.”   In cross-functional environments, Dan's mantra is to focus on the project and maintain open communication. Drawing from his experience in gaming, where multiple departments collaborate on creative projects, he underscores the importance of empathy and curiosity. Asking questions and breaking down solutions into smaller, reviewable pieces can diffuse conflict and build trust.   Future-Proofing Through Strategic Roadmapping   “The lifespan of the solution dictates the scope of the work.”   Dan shares his approach to strategic roadmapping by considering the expected longevity of technical solutions. He gives an example of building a feature flag system for a game studio that needed to support a long-term vision while adapting to a new game engine. His advice: break large goals into smaller, adaptable increments that align with future changes.   Navigating Leadership Challenges During Organizational Change   “Trust is your greatest currency during periods of uncertainty.”   Dan reflects on a particularly challenging period when a leadership change caused a significant exodus of engineers at his company, leaving him with only one engineer. Despite the setback, Dan leaned into transparency and empathy, earning the trust of departing team members, which helped him transfer knowledge and rebuild the team.   Expanding the Scope of Leadership   “My role expanded from leading an engineering team to caring about the morale of the entire company.”   A surprising revelation for Dan was realizing the broader impact of his leadership on non-engineering teams. He discusses how this shift required him to listen to and support colleagues across all departments, emphasizing the value of empathy-driven leadership.   The Book That Shaped Dan's Leadership Approach   “The DORA metrics help us measure what really matters for technical health.”   Dan highlights the book Accelerate by Nicole Forsgren et al., which introduced him to the DORA (DevOps Research and Assessment) metrics. These metrics help organizations measure software delivery performance and technical health, offering a data-driven approach to evaluate progress and identify improvement areas.   About Dan Hollinger   Dan Hollinger is a proven engineering leader who champions empowerment through support, empathy, and transparency. He fosters a culture of trust, prioritizing alignment over dictation. Technically adept, Dan advocates for automatable solutions and a blameless environment, ensuring his team thrives both personally and professionally in a collaborative space. You can link with Dan Hollinger on LinkedIn.

We discuss the four modes of coaching and navigate career growth in expanding / contracting companies with James Birchler. James shares highlights from the recent coaching / mentoring workshop he facilitated, and breaks down how each mode of coaching differs tactically. We also cover the dilemma of linear career/leadership growth vs. exponential company growth, different common communication challenges eng leaders face, why people / organizational challenges are harder than technical issues, and how to prepare for & execute uncomfortable conversations. James also shares his unique journey to technical leadership & how past management roles – even in non-tech spaces – have helped shape his thoughts on coaching & eng leadership today.ABOUT JAMES BIRCHLERJames Birchler is an engineering and product development leader, technical advisor, and an accredited Executive Coach from the UC Berkeley Haas School of Business Executive Coaching Institute.In his coaching practice, James focuses on self-awareness, integrity, accountability, and fostering a growth mindset that supports continuous learning and high performance.He focuses his technical advisory practice on common mechanisms and playbooks required at different phases and inflection points of startup growth and scaling: Hiring and interviewing, product development methodologies including Lean Startup and Agile, operational meeting cadence and communication flow, people management, technical leadership, vision/mission development, alignment, and execution.James implemented the Lean Startup methodologies with Eric Ries at IMVU (literally the first Lean Startup), where his team helped start the DevOps movement by building the infrastructure to ship code to production 50 times a day (which was a lot at the time!) and coining the term “continuous deployment.”He has more than 20 years of experience leading high-performance teams in growth environments, including startups and scaled organizations, including Amazon. He has delivered great consumer software products and implemented product development and innovation processes based on continuous learning and improvement.Presently James advises and coaches Series A+ startups in the US and Europe, and leads innovation practices in hyper-growth areas of last mile delivery technology for Amazon. Previously my roles included VP of Engineering & Operations, VP of Engineering, and Founder at several technology startups including IMVU, Caffeine.tv, SmugMug, iCracked, The Arts Coop, and Letters & Science.You can find James at jamesbirchler.com, LinkedIn, and Substack.SHOW NOTES:Highlights from James' recent coaching & mentoring workshop (2:41)Shared challenges around building trust in eng teams (5:25)The differences between coaching vs. mentoring (7:01)Building trust in order to best support your team members as a manager (9:38)Defining the advising mode of coaching (11:54)How supporting differs from advising (14:29)The story behind James' technical leadership journey (16:55)Transitioning from a PhD program & environmental planning career into tech (20:19)The dilemma of career growth: linear leadership growth vs. exponential company growth (23:53)Why organizational challenges are more complicated than technical puzzles (26:49)Navigating career growth during company contraction from the employee perspective (28:02)Preparing for uncomfortable conversations as a coach / manager (31:50)Strategies for actually having those tough conversations (35:36)Frameworks for helping others identify what they want (37:58)Rapid fire questions (42:44)LINKS AND RESOURCESStop 'Coaching' Your Tech Team (And What To Do Instead) - James' substack post on the four modes of development breaking down the core differences of coaching, advising, mentoring, and supporting roles and explaining how trust is the secret ingredient to all four.jamesbirchler.com - James' website where you can find info about his executive coaching and resources for engineering leaders and founders.How to lead with radical candor | Kim Scott - NYT bestselling author, Kim Scott, has cracked the code on giving valuable feedback in a way that builds genuine relationships, drives results, and creates positive workplaces.What Are People For? - In the twenty-two essays collected here, Wendell Berry conveys a deep concern for the American economic system and the gluttonous American consumer. Berry talks to the reader as one would talk to a next-door neighbor: never preachy, he comes across as someone offering sound advice. In the end, these essays offer rays of hope in an otherwise bleak forecast of America's future. Berry's program presents convincing steps for America's agricultural and cultural survival.New Happy: Getting Happiness Right in a World That's Got It Wrong - Happiness expert Stephanie Harrison draws upon hundreds of studies to offer a life-changing guide to finding the happiness you have been looking for, all based on a decade of research and brought to life with beautiful artwork.Accelerate: Building and Scaling High-Performing Technology Organizations - Through four years of groundbreaking research, Dr. Nicole Forsgren, Jez Humble, and Gene Kim set out to find a way to measure software delivery performance—and what drives it—using rigorous statistical methods. This book presents both the findings and the science behind that research. Readers will discover how to measure the performance of their teams, and what capabilities they should invest in to drive higher performance.Conversations on Science, Culture, and Time: Michel Serres with Bruno Latour - Although elected to the prestigious French Academy in 1990, Michel Serres has long been considered a maverick--a provocative thinker whose prolific writings on culture, science and philosophy have often baffled more than they have enlightened. In these five lively interviews with sociologist Bruno Latour, this increasingly important cultural figure sheds light on the ideas that inspire his highly original, challenging, and transdisciplinary essays.This episode wouldn't have been possible without the help of our incredible production team:Patrick Gallagher - Producer & Co-HostJerry Li - Co-HostNoah Olberding - Associate Producer, Audio & Video Editor https://www.linkedin.com/in/noah-olberding/Dan Overheim - Audio Engineer, Dan's also an avid 3D printer - https://www.bnd3d.com/Ellie Coggins Angus - Copywriter, Check out her other work at https://elliecoggins.com/about/

This interview was recorded at GOTO Amsterdam for GOTO Unscripted.http://gotopia.techRead the full transcription of this interview here:https://gotopia.tech/articles/329Flavia Naezer - Product Thinker, Public Speaker, ArtistJulian Wood - Serverless Developer Advocate at AWSRESOURCESFlaviahttps://github.com/flavianaezerhttps://www.linkedin.com/in/flavia-naezer-449b285https://x.com/flaviasomethingJulianhttps://twitter.com/julian_woodhttp://www.wooditwork.comhttps://www.linkedin.com/in/julianrwoodLinkshttps://dl.acm.org/doi/pdf/10.1145/3595878https://www.designcouncil.org.uk/our-resources/the-double-diamondDESCRIPTIONExplore the evolving relationship between technology and product management with Julian Wood and Flavia Naezer. Discover how Flavia's tech and product expertise highlights the need for user-centric design thinking and thorough research in developing internal tools and platforms.Discover how Julian and Flavia explore the intersection of tech and product management, highlighting the importance of user-centric design and thorough research in developing internal tools and platforms.RECOMMENDED BOOKSMarty Cagan • Inspired • https://amzn.to/4e5l2r2Anthony W. Ulwick • Jobs to Be Done • https://amzn.to/4elaVhuGregor Hohpe • Enterprise Integration Patterns, Vol 2 • https://amzn.to/3TNedQ3Gregor Hohpe • Platform Strategy • https://amzn.to/4fPLW7pStephanie Stimac • Design for Developers • https://amzn.to/3EhuN4TGene Kim, Jez Humble, Patrick Debois, John Willis & Nicole Forsgren • The DevOps Handbook • https://amzn.to/3WBjzCMMatthew Skelton & Manuel Pais • Team Topologies • http://amzn.to/3sVLyLQGene Kim, Nicole Forsgren & Jez Humble • Accelerate • https://amzn.to/3WCG5uTMarty Cagan • Empowered • https://amzn.to/42kuKAjTwitterInstagramLinkedInFacebookLooking for a unique learning experience?Attend the next GOTO conference near you! Get your ticket: gotopia.techSUBSCRIBE TO OUR YOUTUBE CHANNEL - new videos posted daily!

Applying 'Accelerate' Principles to Embedded Systems | Agile Embedded PodcastWelcome to the latest episode of the Agile Embedded Podcast with Jeff Gable and Luca Ingianni! In this episode, we address a listener's question about the book 'Accelerate' by Nicole Forsgren, Jez Humble, and Gene Kim. Jeff and Luca delve into how the principles from this book, which focuses on Lean Software and DevOps, can be applied to embedded systems development. They discuss the nuances of embedded systems, the relevance of DORA metrics, and share insights on how capabilities and processes from the book translate to the unique challenges of embedded systems. Tune in to understand how you can adapt and implement these best practices in your projects.00:00 Introduction to the Agile Embedded Podcast00:06 Overview of the Book 'Accelerate'00:50 Research Methodology and Key Findings02:56 DORA Metrics Explained05:30 Key Capabilities for Effective Organizations18:41 Applying 'Accelerate' Principles to Embedded Systems20:19 Challenges and Considerations in Embedded Systems34:10 The Importance of Logging and Feedback Loops37:43 Empowering Teams and Encouraging Experimentation41:58 Final Thoughts and Recommendations You can find Jeff at https://jeffgable.com.You can find Luca at https://luca.engineer.Want to join the agile Embedded Slack? Click here

This interview was recorded for GOTO Unscripted.https://gotopia.techRead the full transcription of this interview hereDavid Anderson - Software Architect at G-P/Globalization Partners & Author of "The Value Flywheel Effect"Charles Humble - Freelance Techie, Podcaster, Editor, Author & ConsultantRESOURCESDavidhttps://x.com/davidand393https://www.linkedin.com/in/david-anderson-belfasthttps://theserverlessedge.comCharleshttps://twitter.com/charleshumblehttps://linkedin.com/in/charleshumblehttps://mastodon.social/@charleshumblehttps://conissaunce.comLinkshttps://blog.container-solutions.com/adrian-cockcroft-on-serverless-continuous-resiliencehttps://www.wardleymaps.comDESCRIPTIONDavid Anderson, co-author of "The Value Flywheel Effect", shares his experiences and insights from his time at Liberty Mutual, where he drove significant technological transformation through a serverless first approach in a conversation with Charles Humble.Anderson discusses the importance of aligning business and IT strategy, fostering an environment of psychological safety, and enabling teams with the right tools and frameworks to achieve rapid software development. He emphasizes the value of principles-driven development, collaborative processes, and the impact of using the AWS Cloud Development Kit (CDK) to create reusable patterns. Anderson also highlights the continuous nature of software evolution and the importance of timing and momentum in driving successful change in large organizations. [...]RECOMMENDED BOOKSDavid Anderson, Marck McCann & Michael O'Reilly • The Value Flywheel EffectGregor Hohpe • The Software Architect ElevatorGene Kim, Nicole Forsgren & Jez Humble • AccelerateJim Collins • Turning the FlywheelGene Kim & Steve Spear • Wiring the Winning OrganizationGene Kim, Jez Humble, Patrick Debois, John Willis & Nicole Forsgren • The DevOps HandbookElisabeth Hendrickson • Explore It!Gerald M. Weinberg • Becoming a Technical LeaderTwitterInstagramLinkedInFacebookLooking for a unique learning experience?Attend the next GOTO conference near you! Get your ticket: gotopia.techSUBSCRIBE TO OUR YOUTUBE CHANNEL - new videos posted daily!

Johann Botha: When Agile Teams Become The Reason Agile Fails in Organizations Read the full Show Notes and search through the world's largest audio library on Scrum directly on the Scrum Master Toolbox Podcast website: http://bit.ly/SMTP_ShowNotes. When Agile teams push too hard for transformation, they risk becoming the enemy. Johann explains how corporate "immune systems" react against new ideas, even when they're beneficial. What strategies can Agile teams use to navigate organizational resistance and avoid self-sabotage? Johann emphasizes the importance of listening, finding safe spaces to experiment, and avoiding the trap of making Agile seem like an invasive force. Featured Book of the Week: No Rules Rules by Reed Hastings and Erin Meyer Johann shares his journey through influential books that shaped his approach to management, from Tom Peters' Liberation Management to Netflix's story in No Rules Rules. How do these books provide a roadmap for progressive management practices in today's fast-paced world? Johann also highlights key texts like Accelerate by Nicole Forsgren et al., and his own work, Competing in a Digital Future, offering listeners a rich library to explore. [IMAGE HERE] Do you wish you had decades of experience? Learn from the Best Scrum Masters In The World, Today! The Tips from the Trenches - Scrum Master edition audiobook includes hours of audio interviews with SM's that have decades of experience: from Mike Cohn to Linda Rising, Christopher Avery, and many more. Super-experienced Scrum Masters share their hard-earned lessons with you. Learn those today, make your teams awesome!   About Johann Botha Johann joins us from South Africa, helping build digital-age capabilities by developing practical skills to solve problems, grow people, and facilitate difficult change. A long-time proponent of Lean and Agile, Johann consults, coaches, speaks, and writes on the topic. He is also the chief examiner for the EXIN Agile Scrum product. You can link with Johann on LinkedIn and connect with Johann on Twitter.

Trying to measure developer effectiveness or productivity isn't a new problem. However, with the rise of fields like platform engineering and a new wave of potential opportunities from generative AI, the issue has come into greater focus in recent years.  In this episode of the Technology Podcast, hosts Scott Shaw and Prem Chandrasekaran speak to Abi Noda, CEO of software engineering intelligence platform DX, about measuring developer experience using the DevEx Framework — which Abi developed alongside Nicole Forsgren, Margaret-Anne Storey and Michaela Greiler. Taking in everything from the origins of the DevEx framework in SPACE metrics, to how technologists can better 'sell' the importance of developer experience to business stakeholders, listen for a fresh perspective on a topic that's likely to remain at the top of the industry's agenda for the forseeable future.   Read the DevEx Framework paper: https://queue.acm.org/detail.cfm?id=3595878 Read Abi's article (co-authored with Tim Cochran) on martinfowler.com: https://martinfowler.com/articles/measuring-developer-productivity-humans.html Listen to Abi's Engineering Enablement podcast: https://getdx.com/podcast/

This interview was recorded for GOTO Unscripted.https://gotopia.techRead the full transcription of this interview hereGene Kim - Author, Researcher, DevOps Enthusiast & Founder of IT RevolutionCharles Humble - Freelance Techie, Podcaster, Editor, Author & ConsultantRESOURCESGenehttps://twitter.com/RealGeneKimhttps://www.linkedin.com/in/realgenekimhttp://www.realgenekim.meCharleshttps://twitter.com/charleshumblehttps://linkedin.com/in/charleshumblehttps://mastodon.social/@charleshumblehttps://conissaunce.comLinkshttps://youtu.be/vLHFuQjJR8Yhttps://youtu.be/5_rrQND3lpQhttps://youtu.be/dMwGfRINpz0https://youtu.be/KDHyxnLdOqchttps://youtu.be/AxqX9ovGViwhttps://youtu.be/JAl3QFae_dEhttps://youtu.be/l3XwpSKqNZwhttps://youtu.be/wtmW89I941Ihttps://youtu.be/5OjqD-ow8GEhttps://youtu.be/hIwVqt6qtc4DESCRIPTIONJoin Gene Kim and Charles Humble as they demystify the complexities of organizational dynamics, offering a comprehensive guide to navigating challenges and fostering success through his five ideals, backed by real-world stories and expert discussions.Discover the keys to organizational success with Gene Kim and Charles Humble in an insightful conversation, backed by real-world stories and expert discussions. [...]RECOMMENDED BOOKSGene Kim & Steve Spear • Wiring the Winning OrganizationGene Kim • The Unicorn ProjectGene Kim, Kevin Behr & George Spafford • The Phoenix ProjectGene Kim, Nicole Forsgren & Jez Humble • AccelerateGene Kim, Jez Humble, Patrick Debois, John Willis & Nicole Forsgren • The DevOps HandbookGene Kim & John Willis • Beyond The Phoenix ProjectDaniel Kahneman • Thinking, Fast and SlowElisabeth Hendrickson • Explore It!Gerald M. Weinberg • Becoming a Technical LeaderTwitterInstagramLinkedInFacebookLooking for a unique learning experience?Attend the next GOTO conference near you! Get your ticket: gotopia.techSUBSCRIBE TO OUR YOUTUBE CHANNEL - new videos posted daily!

In this episode of the Engineering Room, we are pleased to welcome Dr. Nicole Forsgren. Dr. Forsgren is is an American technology executive, IT impact expert, and author. She joins Dave to talk about software developer productivity metrics, DORA, her part in one of the most impactful industry leading book's “accelerate”, her predictions for the future of software engineering under the influence of science and data and MUCH MORE.xxNicole on LinkedIn: https://www.linkedin.com/in/nicolefv/

Transcript: Agile FM radio for the agile community. [00:00:04] Joe Krebs: In today's episode of Agile FM, I have Lisa Crispin with me. She can be reached at very easy to remember lisacrispin. com. Lisa is an author of a total of five books. There's three I want to highlight here or four actually is Obviously, a lot of people have talked in 2009, when the book Agile Testing came out, a practical guide for testers and Agile teams.Then following that more Agile Testing, right? Then I thought it would be most Agile Testing, but it turned into Agile Testing Condensed in 2019 and just very recently a downloadable book, Holistic Testing, a mini book. Welcome to the podcast, Lisa. [00:00:47] Lisa Crispin: Thank you for inviting me. I'm honored to be part of the podcast.You've had so many amazing people on so many episodes. So it's great. [00:00:54] Joe Krebs: Thank you. And now it's one more with you. So thank you for joining. And we will be talking a little bit about a totally different topic than maybe the last 20 episodes I had maybe way back. I did some testing topics, but I cannot recall maybe the last 20 episodes.So we're not about testing a super important topic. I would not consider myself an expert in that. And I don't know of the audience who has been listening to maybe the last 20 episodes are very familiar with agile testing. Maybe everybody has a feeling about, when they hear the word testing, but there is a huge difference between agile testing.And let's say traditional testing methods. If you just want to summarize like very brief, I know a lot of people are familiar with some of those things, but what it is, if somebody says what is agile testing, why was this different to traditional testing methods? [00:01:47] Lisa Crispin: Yeah. I think that there are a couple of big differences.One is that testing this is just a truth and not necessarily something to do with agile, but testing is really just part of software development. So many people think of it as a phase that happens after you write code, but in modern software development we're testing all the time, all the way around that whole DevOps loop, really.And and so the whole team's getting engaged in it through the whole lifecycle and the focus. Is on bug prevention rather than bug detection. Of course, we want to detect the bugs that make it out to production so we can fix them quickly. But really what we want to do is prevent those bugs from happening in the first place.So there are all these great practices that were popularized by that extreme programming and agile, things like test driven development, continuous integration, test automation all the things that go into, the planning. Workshops and things where we talk about our new features and break them into stories and what's going to be valuable to customers, having those early conversations, getting that shared understanding, things like behavior driven development, where we think about what we're going to code before we code it.That's all really different from, I guess I would say a more traditional software development approach where, Oh, we focus on these requirements. The requirements and a lot of people think about testing is just make sure it met the requirements. But there's so much more to that. We've got all these quality attributes, like security and performance and all the things that we also need to test.So it's a huge area, but it's woven into software development, just like coding, just like design, just like architecture, just like monitoring and observability. It's all part of the process. [00:03:31] Joe Krebs: Yeah. It's like a QA baked in, if you want to see it this way. And then also the automation of all that, right?So automating everything you just said is probably also a concern. Not that's necessarily new to agile, but that's a focus as well now I don't know if I don't have necessarily data points around that but I have worked with a lot of Scrum teams and Agile teams in my career.And it seems, if somebody would say what are the challenges within these teams? And one of them is, you can almost always highlight that, and I say almost purposely because there are good exceptions, is to build an increment of work once per sprint. A lot of teams do not accomplish that, and it's often related to testing activities.Why is that, in your opinion, like when we're seeing these teams struggle to put an increment of work out or a piece of the product or whatever you want to call it if you don't use Scrum necessarily, but to build something that could potentially go out. It's the quality standards of going out. What are the struggles out there for teams, especially on the testing side?I see that as you just said, like it's always happening or often happens at the end, rather than in the front. [00:04:46] Lisa Crispin: Yes. Unfortunately, I see, still see a whole lot of scrum teams and other agile teams doing a mini waterfall where they have testers on the cross functional team. But. The testers are not being involved in the whole process, and the developers aren't taking up practices like tester development, because those things are hard to learn and a lot of places don't enable.The non testers to learn testing skills because they don't put those skills into the skills matrix that those people need to advance their careers. And the places I've worked where we succeeded with this sort of whole team holistic approach, everybody had testing skills in their skills matrix.And we all had to learn from each other and, testers had other skills in their taste, matrix, like database skills and at least the ability to read code and be able to pair or ensemble with somebody. So that's part of it. And I just think. It's people don't focus enough on that, on the early process of the business stakeholder has brought us a new feature.We need to test that idea before we do anything. Is this really giving, what value, what's the purpose of the feature? What value is it going to give to the customer and to the business? And a lot of times we don't ask those questions up front. And the stakeholders don't ask themselves and then they get, you deliver the feature and it's something the customers didn't even want.[00:06:11] Joe Krebs: Lisa, we need to code. We need to get software. Why would we talk about that? Why would we not just code? I'm kidding. [00:06:18] Lisa Crispin: Yeah. Yeah. And that's also required, that's why the whole concept of self organizing team works really well. When you really let the teams be autonomous, because then they can think how best, how can we best accomplish this than they can do?Let's do some risk storing before we try to slice this into stories and let's do good practices to slice that feature into small, consistently sized stories that give us a reliable cadence predictable cadence of the business can plan and. Take those risks that we identified, get concrete examples for the business stakeholders of how this should behave and turn those into tests that guide development.Then we can automate those tests. And now we have regression tests to provide a safety net. So that all fits together. And of course, these days, we also need to put the effort into kind of the right side of the DevOps loop. We're not going to prevent all the bugs. We're not going to know about all the unknown unknowns, no matter how hard we try.And. These cloud architectures are very complex. Our test environments never look like production, so there's always something unexpected that happens. And so we have to really do a good job of the telemetry for our code, gathering all the data, all the events, all the logging data for monitoring. For alerting and also for observability, if something happens that we didn't anticipate, so it wasn't on our dashboard.We didn't have an alert for it. We need to be able to quickly diagnose that problem and know what to do. And if we didn't have. Enough telemetry for diagnosing that problem without having to, Oh, we've got to go back and add more logging and redeploy to production so we could figure it out. Oh, how many times has my team done that?That's all part of it. And then learning from production using those. And we've got fantastic analytics tools these days. Learning from those and what are the customers do? What was most valuable to them? What did they do when they, especially I mostly have worked on web applications.What did they do again? We released this new feature in the UI. How did they use it? And it's, we can learn, we know that stuff now. So that feeds back into what changes should we make next? [00:08:29] Joe Krebs: All right. So it's, it comes full circle, right? What's interesting is there's this company, it's all over the news.It's Boeing, right? We're recording this in 2024 quality related issues. Now, that is an extreme example, obviously, but. We do have these kind of aha and wake up moments in software development too, right? So that we're shipping products and I remember times where testing, I purposely call it testing and not QA, testing personnel was outsourced.That was like many years ago. We actually felt oh, this activity can be outsourced somewhere else. And you just made a point of if we have self organizing teams, And we're starting with it and we're feeding in at the end of a loop back into the development efforts, how important that is and how we treated these activities in the past and how, what we thought of it is, it's shocking now looking back in 24, isn't it?[00:09:23] Lisa Crispin: Yeah, it's just, it just became so much part of our lives to run into that. And the inevitable happened, it generally didn't work very well. I've actually known somebody who led an outsourcing test team in India and was working with companies in the UK and Europe.They actually were able to take an agile approach and keep the testers involved through the whole loop. They had to work really hard to do that. And there were a lot of good practices they embraced to make that work. But you have to be very conscious. And and both sides have to be willing to do that extra work.[00:09:56] Joe Krebs: You just mentioned that there were some really cool analytics tools. I don't know if you want to share any of those because you seem very excited about this, [00:10:05] Lisa Crispin: the most, the one that I found the most useful and I, a couple of different places I worked at used it.It's called full story. And it actually. It captures all the events that are happening in the user interface and plays it back for you as a screencast. Now, it does block anything they type in. It keeps it anonymized. But you can see the cursor. And I can remember one time a team I was on, it's we put a whole new page in our UI, a new feature.We thought people would really love it. And we worked really hard on it and we, we tried to do a minimum viable version of it, but we still put some effort in it and we put it out there. And then we looked at the analytics and full story and we could see that people got to the page. Their cursor moved around and then they navigated off the page.So either it wasn't clear what that page was for, or they just couldn't figure it out. So that was really valuable. I was like, okay, can we come up with a new design for this page? If we think that's what the problem is, or should we just, okay, that was a good. Good learning opportunity. But as a tester, especially there, because we can't reproduce problems, we know there's a problem in production, can't reproduce it.But if we go and watch a session where somebody had the problem, and there are other things, mixed panel, won't play it back for you, but you can see every step that the person did. And even observability tools Honeycomb and LightStep can show you like the whole, they can trace the whole path of what did the user do.And that really helps us not only understand the production problem, but, Oh, there's a whole scenario. We didn't even think about testing that. And so there's so much we can learn because we're so bound by our cognitive bias, our unconscious biases that we know how we wanted it to work.[00:11:54] Joe Krebs: Yeah. [00:11:55] Lisa Crispin: And it's really hard to think outside the box and get away from your biases and really approach it like a customer who never saw it before would do. [00:12:03] Joe Krebs: Yeah. It's this is the typical thing, right? If a software engineer demonstrates their own software they produce and was like eight books on my machine, I'm sure you have heard that.And it's it's obvious that you would do this, right? And it's just not necessarily obvious for somebody else. But if you're like sitting in front of a screen developing something for a long time, it just becomes natural that you would be working like this. I myself have engineered software and and fell into that trap, right?It's oh my God, eye opening event. If somebody else looks at you or. Yeah, [00:12:33] Lisa Crispin: Even when you sometimes have different people, like I can remember an occasion that Timo was on with a, again, a web application and I was just changed in the UI, just adding something in the UI and I tested it. My, my manager tested it.One of the product owner tested it. And we all thought it looked great and it did look great. We didn't notice the other thing we had broken on the screen until we put it in the production and customers were like, Hey, I really do think things like pair programming, pair testing, ensemble, working in ensembles for both programming and testing, doing all the work together.Getting those diversity points does help hugely with that. My theory is we all have different unconscious biases. So maybe if we're all together, somebody will notice a problem. I don't have any science to back that up, but But that's why those kind of practices are especially important. [00:13:28] Joe Krebs: Yeah. [00:13:28] Lisa Crispin: To catch as many things as we can.[00:13:30] Joe Krebs: Yeah. So we both didn't have any kind of science to back this up, but let's talk a little bit about science. Okay. Because metrics, data points, evidence. What are some of the KPIs if somebody listens to this and says Oh, that sounds interesting. And we definitely have shortcomings on testing activities within Agile teams.Obviously there's the traditional way of testing. They're using very different data points. I have used some in the past, and I just want to verify those with you too. It's that's even useful and still up to date. What would be some good KPIs when somebody approaches you and says that's got to have that on your dashboard?[00:14:08] Lisa Crispin: I think you, I actually think one of my favorite metrics to use is cycle time, although that encompasses so many things, but just watching trends and cycle time. And if you're, if you've got, for example, if you've got good test coverage with your automated regression tests, you're going to be able to make changes really quickly and confidently.And if you have, a good deployment pipeline, you're going to Again, there's a lot of testing that goes into making sure your infrastructure is good and your pipeline is performing as it should, because it's all code to that reflects a whole lot of things. It's hard to isolate one thing in your cycle time but what counts is, how consistent are we at being able to frequently deliver small changes?So I think that's an important one. And in terms of. Did we catch all the problems? I think it gets really dangerous to do things like, Oh, let's count how many bugs got in production because all measures can be gained, but that's a really easy one to gain. But things like how many times did we have to roll back or revert a change in product in production?Because there was something we didn't catch and hopefully we detected that ourselves with an alert or with monitoring before the customers saw it. And now that we have so many release strategies where we can do, Canary releases or blue green deploy so that we can do testing and production safely.But still how many times did we have to roll back? How many times did we get to that point and realize we didn't catch everything? That can be a good, that can be a good thing to track. And depending on what caused it. If we had to, if we had a production failure because, somebody pulled the plug of the server out of the wall.That's not, that's just something that happened, but if it is something that the team's process failed in some way, we want to know about that. We want to improve it. And, just how frequently can we deploy I think, the thing with continuous delivery, so many teams are practicing that are trying to practice that you're not going to succeed at that if you're.If you're not preventing defects , and if you don't have good test automation, good automation the whole way through. [00:16:08] Joe Krebs: Yeah. [00:16:08] Lisa Crispin: And I think, deployment frequency, that's another one of the Dora key metrics. Yeah. That's a real that we know that correlates with high performing teams.And of course we shouldn't ignore, how do people feel are people burned out or do they feel happy about their jobs? That's a little harder metric to get. I was on a team, my last full time job, we really focused on cycle time as a metric and we didn't really have that many problems in production.So we didn't really bother to track how many times we had to revert because we were doing a good job, but. But but, how frequently were we going? What was our cycle time? But also we did a little developer joy survey. So once a week, we sent out a little 5 question survey based on Amy Edmondson's work.And now I would base it on I would also use Nicole Forsgren's space survey. Model, but that was just a little before that came out, but just asking just a few questions and multiple, from one to five, how did you feel about this? And it was really interesting because over time, if cycle time was longer, developer joy was down.So there's something happening here that people are not happy. And. Something's going wrong. That's affecting our cycle time. And then the reverse is true. When our cycle time was shorter, joy went up. So I think it's I think it's important and, you don't have to get real fancy with your measurements, just start just, I think you should first focus on what are we trying to improve and then find a metric to guide, to measure that.[00:17:41] Joe Krebs: I'm glad you said or mentioned code coverage. That's one of one of those I mentioned earlier. I've been working with it quite a bit and cycle time. Um, very powerful stuff. Now, with you, such, somebody who has written published about agile testing extensively we are in 2024. There was the years ahead.There are agile conferences. There is a lot going on. What are the trends you currently see in the testing world? What is what's happening right now? What do you think is influencing maybe tomorrow? The days coming, I know you have holistic testing yourself. So maybe that is one but I just want to see, what do you see happening in the agile testing? [00:18:24] Lisa Crispin: Oh, just all of software development, definitely evolving. I think one of the things is that we're starting to be more realistic and realize that executives don't care about testing. They care about how well does their product sell?How much money is the company making? We know that. Product quality and process product quality obviously affects that. And that's from a customer point of view. It's the customer who defines quality. And, back in the nineties, we testers thought we were defining quality. So that's a thing, a change that's occurred over time and really thinking about that and also knowing that our process quality has a huge impact on product quality and what are our, are the, What are practices?What are the most important practices we can be doing? Janet Gregory and who is my coauthor on four of those books and Selena Delesie they've been consultants for years and helped so many huge, even big companies through an agile transformation. And they've distilled their magic into their, what they call a quality practices assessment model.And they identified 10 quality practices that they feel are the most important and things like feedback loops. Things like communication, right? And the model helps you ask questions and see where is the team in these different different aspects of practices that would make them have a quality process, which would help them have a quality product.And it gives teams a kind of a roadmap. It's here's where we are now. What do we need to improve? Oh, we really need to get the continuous delivery and these things are on our way, things like that. So I think that's one realization that it ties back to the idea that testing is just part of software development and we've had for years.So like, how can I make the, president of this company understand that we testers are so important. We're not, but it's important that the team build that quality. [00:20:29] Joe Krebs: But you could also argue that maybe a CEO of a company or the leadership team would say, we also don't care if this is being expressed in one line of code or two lines of code.So it's not necessarily to testing. I think they're just saying we have, here's our product. But I think what has changed is that your competition is just one mouse click away. Yeah. Quality is a determining factor. Now, let's take this hypothetical CEO out there right now listening to our conversation and saying I do want to start to embrace agile testing and agile in general, but more of those things you just mentioned, what would be a good starting point for them?Obviously there's a lot of information right now keywords and buzzwords we just shared today. What would be a good starting point for them to start that journey, because that is obviously not something that's coming overnight. [00:21:20] Lisa Crispin: I think one of the most important things that leadership can do is to make, to enable the whole team to learn testing skills that will help them build quality.And that means making it part of their job description, making it part of their skills matrix for career advancement, because that gives them time. If developers are paid to write lines of code, that's what they're going to do. But if they're, it's okay, you're an autonomous team.You decide what practice you think will work for you. We're going to support you. It's going to, it's going to slow things down at first. Okay, like I was on a team in 2003 that was given this mission. Okay. Do what you think you need to do first. We decided what level of quality we wanted, of course.We wanted to write code that we would take home and show our moms and put it on our refrigerators, and and we all commit to that level of quality. How can we achieve that? We're seeing that test driven development has worked really well for a lot of teams. So let's do test driven development, which is really.Not that easy to learn, but when you have a leadership that lets you have that time to learn and support you, it pays off in the long run because eventually you're a lot more confident. You've got all these regression tests. You can go faster and things like continuous integration, refactoring, all these practices that we know are good.And we were empowered to adopt those. It was all part of all of our job descriptions. And that's, so we became a high performing team, not overnight. Yeah, within a few years and a part of our, part of what we did was spend a lot of time learning the business domain. It's a very complicated business domain.And so when the stakeholders came and said we want this feature and we asked them what, why do they want it? What was it supposed to do? What was the value? We could usually cut out half of what they thought they wanted. We can say, okay, if we did all of this, we think it's going to be this much effort, but we could do 80 percent of it for half the cost.How's that? Oh yeah. Oh yeah. Nobody ever turned us down on that one. So that's another way where you go fast, we eliminate things that customers don't want or need. And so yeah, it's the unicorn magic of a self true self organizing team. [00:23:30] Joe Krebs: Yeah. But I do think what you said is, , this one thing that just stood out to me It is an investment, it is an investment into the future.It's a really good feeling to have later on the capability of releasing software whenever you want. If that is not becoming a massive burden and the whole company needs to come together for all nighters to get a piece of software out of the door. Now you're not only an author here you're also a practitioner.You also work with teams and I just want to come back to that business case of agile testing. One more time. Do you have an example from a client recent or further back where you would say that stands out or that's an easy one? You remember where agile testing made a huge difference for an organization.I'm sure there are tons you have where you would say there was a significant impact for them based on introducing agile testing practices. [00:24:29] Lisa Crispin: I certainly, especially early on in the extreme programming and Agile adoption there was a few occasions where I joined a team that never had testers.They were doing the extreme programming practices and you may recall that the original extreme programming publications did not mention testers. They were all about testing and quality, but they didn't mention testers. And. So these teams were doing test driven development, continuous integration. They were writing really good code and then they were doing, they were doing their two week sprints and maybe, maybe it took them three sprints to develop what the customer wanted and then they give it to the customer and the customer is but that's not what I wanted.So they like, maybe we need a tester. So then they hired me. And I was like, oh okay, let's let's have some of the, some, okay, we're gonna do a new, some new features. Let's have some brainstorming sessions. How are we gonna, what is this new feature for? How are we gonna implement it?What are the risks? And start doing risk assessments. And how are we gonna mitigate those risks? Are we gonna do it through testing? Are we gonna do it through monitoring? And just asking those what if questions? What's the worst thing that could happen? That's my favorite question when we release this.And could we deploy this feature to production and have it not solve the customer problem? And just add, anyone could ask those questions. It doesn't have to be a tester, but I find the teams that don't have professional testers, specialists, they, nobody else thinks of those questions. They could. But they just, testing is a big area.It is a big set of skills. And anybody on that team, I know lots of developers who have those skills, but not every team has a developer like that, other specialists, like business analysts could also help, but there were even fewer business analysts back in the day than there were testers.And as soon, so as soon as the tester, and when I, one team I joined early on, okay, they're like, okay, Lisa you can be our tester. But you can't come to the planning meetings and you can't come to the standups. That's a little weird. I did as best I could without being involved in any of the planning.And so that's the end of the two weeks. They weren't finished. Nothing was really working. And I said, Oh, Hey, can we try it my way? Let me be involved in those early planning discussions. Let me be part of the standup and Oh, amazing. Next time we met our target. And and I was I couldn't support all the, there were 30 developers and one tester, but we agreed that one other person or two other people would wear the testing hat along with me every sprint or at least on a daily basis.And so they all started to get those testing skills. Yeah, they just test, like I say, testing is a big area and you don't know what you don't know. I see teams even today. That they don't have any testers because years ago they were told they didn't need them if they did these extreme programming practices and they're doing testers involvement, they're doing continuous integration.They're maybe even doing a little exploratory testing. They're doing pair programming, even some ensemble or mob programming. They're doing great stuff, but they're missing out all that stuff at the beginning to get the shared understanding with the stakeholders of what to build. [00:27:43] Joe Krebs: All those lines of code that were needed. Wouldn't need to be tested. [00:27:48] Lisa Crispin: And so they release the feature and bugs come in. And they're really, they're missing features. It's not what the customer needed. Too many gaps. . And of course, I want to say those aren't really bugs. But they're bad. Yeah. And if you'd had a risk storming session, if you had planning sessions where you got.Example mapping sessions, for example, where you got the business rules for the story, concrete examples for his business role, and then you turn those into tests to guide your development with behavior driven development. This would have solved your problem, but they didn't know to do that. Anybody could have learned those things, but we can't all know everything.[00:28:25] Joe Krebs: Yeah. We're almost out of time.But there's one question I wanted to ask you and it might be a short answer. I hope you condense it a little bit. But when somebody gets on your LinkedIn page Lisa Crispin there is you in a picture plus a donkey. And you have donkeys yourself.And how does this relate to, does it relate to your work? What do you find inspirational about donkeys? And what, why is, why did you even make your LinkedIn profile? It has to be, it has to be a story around it.[00:28:55] Lisa Crispin: It's interesting. A few years ago at European testing conference, we had an open space and somebody said, Oh, let's have an open space session on Lisa's donkeys.And then we got to talking about this and I actually have learned a lot. About Agile for my donkeys. And I think the biggest thing is trust. So donkeys are work on trust. So with horses, I've ridden horses all my life and had horses all my life as well. You can bribe or bully a horse into doing something, they're just, they're different.If you reward them enough, okay they'll go along with you. If you kick them hard enough, maybe they'll go. Donkeys are not that way. They're looking out for number one. They're looking out for their own safety. And if they think you might be getting them into a situation that's bad for them, they just flat won't do it.So that's how they get the reputation of being stubborn. You could beat a bloody, you could offer them any bribe you want. They're not doing it. And so I learned I had to earn my donkey's trust. That's so true of teams. We all have to trust each other. And when we don't trust each other. We can't make progress and the teams I've been on that have been high performing teams We had that trust so we could have discussions where we had different opinions We could express our opinions without anyone taking it personally Because we knew that we were all in it together and it was okay Anybody could feel safe To ask a question, anybody can feel safe to fail, but you have that trust that there's nothing bad is going to happen.And so I could bring my donkey right in that door in the house. I've taken them in schools. I've taken them to senior centers because they trust me. And if I did anything, if they came to harm while in my care, if I, let's say I was driving the cart and the collar rubbed a big sore on them, that would destroy the trust.And it would be really hard to build it back. And so we always need to be conscious of how we're treating each other in our software teams. [00:30:55] Joe Krebs: Yeah, wonderful. I did hear about the rumor of being stubborn. But I also always knew that donkeys are hardworking animals. [00:31:02] Lisa Crispin: They love to work hard.Yeah. [00:31:05] Joe Krebs: Awesome. Lisa, what a great ending. I'm glad we had time to even touch on that. That was a great insight. Thank you so much for all your insights around testing, but also at the end about donkeys. Thank you so much, Lisa. [00:31:17] Lisa Crispin: Oh, it's my pleasure.

In the Microsoft Research Podcast series What's Your Story, Johannes Gehrke explores the who behind the technical and scientific advancements helping to reshape the world. A systems expert whose 10 years with Microsoft spans research and product, Gehrke talks to members of the company's research community about what motivates their work and how they got where they are today.Partner Research Manager and leading developer experience expert Nicole Forsgren oversees Microsoft Research efforts to enhance software engineering effectiveness through the study of developer productivity, community, and well-being. In this episode, she discusses AI's potential impact on software engineering, what she loves about tech, and how thoughtful decision making—combined with listening to her gut—has led to opportunities as a developer, accounting professor, and founder and CEO of a startup that was eventually acquired by Google.Learn more:Nicole Forsgren at Microsoft ResearchNicole Forsgren websiteQuantifying the impact of developer experience | Microsoft Azure Blog, January 2024Yes, good DevEx increases productivity. Here is the data. | GitHub blog, January 2024Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations | Book, 2018

Ever wondered what it's like to be a VP or Director of Engineering? Kendra chats with Alex Robson about leadership in technology, what you can get out of coaching or an MBA program (should you be interested), and what makes a high performing team. We'll also chat about recommended content to hone your tech leadership skills. Alex Robson's site and blog: https://robsonconsulting.services Alex's content recommendations for folks who want to think more about technical leadership: "I believe Camille Fournier and Will Larson are wonderful writers with invaluable insights and advice. For product thinking, I recommend folks read The Lean Startup by Eric Ries, Principles of Lean Product Development Flow by Don Reinertsen, Safer Sooner Happier by Jonathan Smart, and Accelerate by Dr. Nicole Forsgren. Be sure to read books on leadership that are outside of engineering. Dan Pink's Drive and Eliyahu Goldratt's The Goal are two of my usual recommendations. Last but not least - read books that are about human behavior. Both economists and psychologists ask important questions that may help you unlock better ways to relate to and understand others. I love Daniel Kahneman's Thinking Fast and Slow, and highly recommend Mistakes Were Made (but not by me) by Carol Tavins and Elliot Aronson."

No Episódio #04 do Papo na Arena no formato de podcast, nossos hosts Arthur Castro (⁠@arthurklose⁠) e Aíquis Rodrigues (⁠@aiquis⁠) comentam sobre as seguintes notícias: Velocidade é tudo! Como a Ramp se tornou o SaaS de maior crescimento de todos os tempos Uma vídeoaula do CPO e CTO do Spotify sobre Inteligência Artificial e Generative AI Entrevista do comissário da MLS e do Eddy Cue da Apple sobre a parceria da MLS + Apple e o efeito Messi Uma história sobre sobrevivência a um layoff Próximos Eventos da Product Arena (10% de desconto aplicados nos links): 24/08 [SP] Papo na Arena: Como funciona uma área de Product Ops? - ⁠INSCRIÇÕES AQUI⁠ 26 e 27/08 [BH] Curso de Product Management - ED 57 - Belo Horizonte - INSCRIÇÕES AQUI 02/09 [SP] Curso de Product Ops, com Dudu Magalhães, Head of Product Ops na RD Station - ⁠INSCRIÇÕES AQUI ⁠ 16 e 17/08 [POA] Ed. 58 do Curso de Product Management - Porto Alegre - INSCRIÇÕES AQUI 23/09 [SP] Arena Sessions - Um dia de Workshops e Palestras - ⁠INSCRIÇÕES AQUI⁠ 30/09 [Online] Curso de Product Ops, com Dudu Magalhães, Head of Product Ops na RD Station - ⁠INSCRIÇÕES AQUI⁠ Leituras Complementares: Bloco 1: Ciclos de produto e o poder de execução Podcast do Lenny com Nicole Forsgren, pesquisadora da MS Research Bloco 2: Podcast do Lenny com Gustav Söderström, CPO e CTO do Spotify Bring the donuts - Newsletter do ken Norton Outras notícias: Apple fará evento dia 12 de setembro para lançar iPhone 15 A universidade de stanford liberou o "the sims” de AI deles como um projeto open source Braincast #512 - Silence Brand: O que o meme pode ensinar para as marcas Leu, assistiu ou escutou algum conteúdo bacana pra gente comentar? Manda lá no ⁠@productarena⁠ Siga nossas redes para ficar por dentro dos novos episódios: ⁠Instagram⁠ | ⁠LinkedIn⁠ Quer patrocinar? Manda um email para arthur@productarena.io

Brought to you by Ezra—The leading full-body cancer screening company | Coda—Meet the evolution of docs | Attio—The powerful, flexible CRM for fast-growing startups—Geoff Charles is VP of Product at Ramp—the fastest-growing SaaS startup of all time, Fast Company's #1 Most Innovative Company in North America, and a company I believe we should all study for how they operate, execute, and hire. At Ramp, Geoff has led the product team from the early days, including the development and release of 60+ products and features in the past year alone. He has been building financial services for over a decade, and his interview in Lenny's Newsletter quickly became one of the most widely read newsletter issues of all time. In today's podcast, we will discuss:• How velocity is at the heart of Ramp's culture and success• How writing can unlock clarity, creativity, and rapid problem-solving• How to empower your product team through context sharing• How to practically approach problems from first principles• How Ramp approaches hiring in a unique way• Suggestions for breaking into the world of product managementListen now on Apple, Spotify, Google, Overcast, and YouTube.Find the transcript for this episode and all past episodes at: https://www.lennyspodcast.com/episodes/. Today's transcript will be live by 8 a.m. PT.Where to find Geoff Charles:• Twitter: https://twitter.com/geoffintech• LinkedIn: https://www.linkedin.com/in/geoffrey-charles/Where to find Lenny:• Newsletter: https://www.lennysnewsletter.com• Twitter: https://twitter.com/lennysan• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/In this episode, we cover:(00:00) Geoff's background(04:49) An overview of Ramp(06:20) The importance of velocity at Ramp(08:50) Single-threaded goals and how to keep teams away from distractions(13:20) Setting lofty goals(15:17) How Ramp empowers teams(17:37) How Geoff's management style has evolved at Ramp(19:55) The product design process at Ramp(21:19) Ramp's system for sharing feedback(23:07) How Ramp handles bug fixes (24:15) Advice for PMs who want to move faster(29:29) Why velocity and impact can help protect against burnout(32:33) Planning vs. doing(37:54) Ramp's strategy documents (40:55) Finding your unique positioning(42:46) OKRs(44:53) The importance of first-principle thinking(48:53) How to use writing to think through problems (51:46) How Geoff carves out time for deep work(54:05) How Geoff manages tasks and stays organized(57:15) Why other roles share the PM load at Ramp(1:00:30) PM responsibilities at Ramp(1:01:46) Identifying A+ talent(1:06:02) The skills Ramp looks for when hiring(1:07:33) Advice for people wanting to break into product management (1:10:37) Lightning roundReferenced:• How Ramp builds product: https://www.lennysnewsletter.com/p/how-ramp-builds-product• Bill: https://www.bill.com/• Expensify: https://www.expensify.com/• Concur: https://www.concur.com/• Coupa: https://www.coupa.com/• Nicole Forsgren on Lenny's Podcast: https://www.lennyspodcast.com/how-to-measure-and-improve-developer-productivity-nicole-forsgren-microsoft-research-github-goo/• Sheryl Sandberg on LinkedIn: https://www.linkedin.com/in/sheryl-sandberg-5126652/• Getting Things Done: https://www.amazon.com/Getting-Things-Done-Stress-Free-Productivity/dp/0143126563• When Breath Becomes Air: https://www.amazon.com/When-Breath-Becomes-Paul-Kalanithi/dp/081298840X• The Bear on Hulu: https://www.hulu.com/series/the-bear-05eb6a8e-90ed-4947-8c0b-e6536cbddd5f• Whoop: https://www.whoop.com/Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.Lenny may be an investor in the companies discussed. Get full access to Lenny's Newsletter at www.lennysnewsletter.com/subscribe

This episode is brought to you by DX—a platform for measuring and improving developer productivity.—Dr. Nicole Forsgren is a developer productivity and DevOps expert who works with engineering organizations to make work better. Best known as co-author of the Shingo Publication Award-winning book Accelerate and the DevOps Handbook, 2nd edition and author of the State of DevOps Reports, she has helped some of the biggest companies in the world transform their culture, processes, tech, and architecture. Nicole is currently a Partner at Microsoft Research, leading developer productivity research and strategy, and a technical founder/CEO with a successful exit to Google. In a previous life, she was a software engineer, sysadmin, hardware performance engineer, and professor. She has published several peer-reviewed journal papers, has been awarded public and private research grants (funders include NASA and the NSF), and has been featured in the Wall Street Journal, Forbes, Computerworld, and InformationWeek. In today's podcast, we discuss:• Two frameworks for measuring developer productivity: DORA and SPACE• Benchmarks for what good and great look like• Common mistakes to avoid when measuring developer productivity• Resources and tools for improving your metrics• Signs your developer experience needs attention• How to improve your developer experience• Nicole's Four-Box framework for thinking about data and relationships—Find the full transcript at: https://www.lennyspodcast.com/how-to-measure-and-improve-developer-productivity-nicole-forsgren-microsoft-research-github-goo/#transcript—Where to find Nicole Forsgren:• Twitter: https://twitter.com/nicolefv• LinkedIn: https://www.linkedin.com/in/nicolefv/• Website: https://nicolefv.com/—Where to find Lenny:• Newsletter: https://www.lennysnewsletter.com• Twitter: https://twitter.com/lennysan• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/—In this episode, we cover:(00:00) Nicole's background(07:55) Unpacking the terms “developer productivity,” “developer experience,” and “DevOps”(10:06) How to move faster and improve practices across the board(13:43) The DORA framework(18:54) Benchmarks for success(22:33) Why company size doesn't matter (24:54) How to improve DevOps capabilities by working backward(29:23) The SPACE framework and choosing metrics(32:51) How SPACE and DORA work together(35:39) Measuring satisfaction(37:52) Resources and tools for optimizing metrics(41:29) Nicole's current book project(45:43) Common pitfalls companies run into when rolling out developer productivity/optimizations(47:42) How the DevOps space has progressed(50:07) The impact of AI on the developer experience and productivity(54:04) First steps to take if you're trying to improve the developer experience(55:15) Why Google is an example of a company implementing DevOps solutions well(56:11) The importance of clear communication(57:32) Nicole's Four-Box framework(1:05:15) Advice on making decisions (1:08:56) Lightning round—Referenced:• Chef: https://www.chef.io/• DORA: https://dora.dev/• GitHub: https://github.com/• Microsoft Research: https://www.microsoft.com/en-us/research/• What is DORA?: https://devops.com/what-is-dora-and-why-you-should-care/• Dustin Smith on LinkedIn: https://www.linkedin.com/in/dustin-smith-b0525458/• Nathen Harvey on LinkedIn: https://www.linkedin.com/in/nathen/• What is CI/CD?: https://about.gitlab.com/topics/ci-cd/• Trunk-based development: https://cloud.google.com/architecture/devops/devops-tech-trunk-based-development• DORA DevOps Quick Check: https://dora.dev/quickcheck/• Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations: https://www.amazon.com/Accelerate-Software-Performing-Technology-Organizations/dp/1942788339• The SPACE of Developer Productivity: https://queue.acm.org/detail.cfm?id=3454124• DevOps Metrics: Nicole Forsgren and Mik Kersten: https://queue.acm.org/detail.cfm?id=3182626• How to Measure Anything: Finding the Value of Intangibles in Business: https://www.amazon.com/How-Measure-Anything-Intangibles-Business/dp/1118539273/• GitHub Copilot: https://github.com/features/copilot• Tabnine: https://www.tabnine.com/the-leading-ai-assistant-for-software-development• Nicole's Decision-Making Spreadsheet: https://docs.google.com/spreadsheets/d/1wItAODkhZ-zKnnFbyDERCd8Hq2NQ03WPvCfigBQ5vpc/edit?usp=sharing• How to do linear regression and correlation analysis: https://www.lennysnewsletter.com/p/linear-regression-and-correlation-analysis• Good Strategy/Bad Strategy: The difference and why it matters: https://www.amazon.com/Good-Strategy-Bad-difference-matters/dp/1781256179/• Designing Your Life: How to Build a Well-Lived, Joyful Life: https://www.amazon.com/Designing-Your-Life-Well-Lived-Joyful/dp/1101875321• Ender's Game: https://www.amazon.com/Enders-Game-Ender-Quintet-1/dp/1250773024/ref=tmm_pap_swatch_0• Suits on Netflix: https://www.netflix.com/title/70195800• Ted Lasso on AppleTV+: https://tv.apple.com/us/show/ted-lasso• Never Have I Ever on Netflix: https://www.netflix.com/title/80179190• Eight Sleep: https://www.eightsleep.com/• COSRX face masks: https://www.amazon.com/COSRX-Advanced-Secretion-Hydrating-Moisturizing/dp/B08JSL9W6K/—Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.—Lenny may be an investor in the companies discussed. Get full access to Lenny's Newsletter at www.lennysnewsletter.com/subscribe

This interview was recorded for the GOTO Book Club.gotopia.tech/bookclubRead the full transcription of the interview hereMauricio Salatino - Author of "Platform Engineering on Kubernetes"  Thomas Vitale - Software Architect & Author of "Cloud Native Spring in Action"RESOURCESMauricio@salaboylinkedin.com/in/salaboysalaboy.comThomas@vitalethomasgithub.com/ThomasVitalelinkedin.com/in/vitalethomasthomasvitale.comDESCRIPTIONPlatform Engineering on Kubernetes accelerates development of cloud-based systems with vibrant open source tools of the Kubernetes ecosystem. You'll use powerful open source projects like Helm, Tekton, Knative, and Crossplane to automate your projects from testing through delivery. Learn how to package services, build and deploy services to a Kubernetes cluster, and combine different tools to solve the complex challenges of CD in a cloud native environment.* Book description: © manning.comThe interview is based on the book "Platform Engineering on Kubernetes".RECOMMENDED BOOKSMauricio Salatino • Platform Engineering on KubernetesMauricio Salatino,  Mariano De Maio & Esteban Aliverti • Mastering JBoss Drools 6Thomas Vitale • Cloud Native Spring in ActionDavid Farley • Modern Software EngineeringDave Farley & Jez Humble • Continuous DeliveryGene Kim, Jez Humble, Nicole Forsgren, Patrick Debois & John Willis • The DevOps HandbookForsgren, Humble & Kim • Accelerate: The Science of Lean Software and DevOpsJohn Arundel & Justin Domingus • Cloud Native DevOps with KubernetesTwitterLinkedInFacebookLooking for a unique learning experience?Attend the next GOTO conference near you! Get your ticket: gotopia.techSUBSCRIBE TO OUR YOUTUBE CHANNEL - new videos posted almost daily

Abi Noda, Co-founder & CEO @ DX returns to the show to discuss his latest research on measuring & improving developer productivity, and provides a practical, developer-focused framework to give you clear, actionable insights into what to measure and where to focus in order to improve developer productivity. Abi reveals the inspiration behind his whitepaper / research, elements of their new DevEx framework, and how eng leaders can implement it into their org's practice in order to increase developer productivity. We also cover the evolution of measuring developer experience (including output metrics, DORA & SPACE frameworks) and the benefits / shortcomings of each approach. In addition, learn not only the importance of having a dedicated DevEx team, but also how to implement these insights if your org isn't ready to have a dedicated team yet.ABOUT ABI NODAAbi (@abinoda) is the CEO and co-founder of DX, the world's first developer experience management platform. He was previously the CEO and founder of Pull Panda, which was acquired by GitHub in 2019. At GitHub he led research collaborations with Dr. Nicole Forsgren, McKinsey, and Microsoft Research, which was the impetus for founding DX."Oftentimes, organizations that are larger that get started with these types of measurements in their framework, they're really surprised. They realize that, 'Oh man, there's all these opportunities we didn't even realize and developers are telling us these are the most important things. These aren't the things we're working on and we need to shift our focus.' So, I think there's a huge opportunity to refocus by getting a holistic picture of the developer experience.”- Abi Noda   Join us at ELC Annual 2023!ELC Annual is our flagship conference for engineering leaders. You'll learn from experts in engineering and leadership, gain mentorship and support from like-minded professionals, expand your perspectives, build relationships across the tech industry, and leave with practical prove strategies.Join us this August 30-31 at the Fort Mason Center in San FranciscoFor tickets, head to https://sfelc.com/annual2023SHOW NOTES:The background behind Abi's developer productivity research & why it matters (2:50)The evolution of measuring developer productivity (5:50)Moving beyond output metrics to DORA (and how that fell short of solving engineering measurement problems) (7:43)Challenges, drawbacks, and limitations to current measurement approaches (like DORA & SPACE) (11:51)What is the SPACE framework & how it manifests in eng orgs (15:14)Distinction between measuring the notion of productivity vs. focusing on measurements that improve productivity (17:07)Overview of Abi's new DevEx framework & examples of it in use (19:52)Recommendations for frontline managers, ICs, engineers, etc. to apply the DevEx framework (22:26)How DevEx uncovers blind spots (like requirements quality) (24:21)When engineering orgs should consider separating out productivity (27:44)Strategies for broad-scope leaders to apply the DevEx framework (29:21)Using local teams to address specific DevEx issues (31:30)Why the VP of Eng / org leadership's values drive developer experience (33:00)Tips for implementing the DevEx framework as a startup vs. mature company (35:06)How Abi is incorporating DevEx strategies into his own company @ DX (37:47)What positive developer experience looks like within an eng team (39:35)The most important step a team w/o a DevEx team can take (41:29)Rapid fire questions (43:17)LINKS AND RESOURCESAbi's new DevEx whitepaper - “DevEx: What Actually Drives Productivity” by Abi Noda, DX, Margaret-Anne Storey, University of Victoria, Nicole Forsgren, Microsoft Research, Michaela Greiler, DXObviously Awesome: How to Nail Product Positioning so Customers Get It, Buy It, Love It - Obviously Awesome goes beyond teaching you what positioning is and why you should care. It gives you a step-by-step process that any startup can follow to position their product, service or company. This book will teach you how to find your product's “secret sauce” and then sell that sauce to those who crave it.Positioning: The Battle for Your Mind - The first book to deal with the problems of communicating to a skeptical, media-blitzed public, Positioning describes a revolutionary approach to creating a "position" in a prospective customer's mind-one that reflects a company's own strengths and weaknesses as well as those of its competitors.This episode wouldn't have been possible without the help of our incredible production team:Patrick Gallagher - Producer & Co-HostJerry Li - Co-HostNoah Olberding - Associate Producer, Audio & Video Editor https://www.linkedin.com/in/noah-olberding/Dan Overheim - Audio Engineer, Dan's also an avid 3D printer - https://www.bnd3d.com/Ellie Coggins Angus - Copywriter, Check out her other work at https://elliecoggins.com/about/

I consider Robin Yeman a friend, though our story starts out with me being awestruck after listening to Robin present at the DevOps Enterprise Summit in 2019. She was among an impressive list of speakers that year including Dr. Nicole Forsgren, Rosalind Radcliffe, and Jonathan Smart. In her role as a Senior Technical Fellow at Lockheed Martin, Robin partnered with Northrop Grumman Fellow, Suzette Johnson to present on a topic they called industrial DevOps. This is a hardcore mixing of software and systems engineering. What else would we expect from Lockheed Martin and Northrop Grummond? They are two of the DoD's most prolific defense contractors. During the talk, they discussed ways to apply DevOps and continuous delivery to significant cyber-physical systems. Cyber-physical systems are things like robotics, warfighting, transportation, and complex medical devices. My mind was blown. After they finished, I scurried to the stage to introduce myself. I had just begun working with the US Air Force and with the F-35 joint program office. Their experience and their materials would prove invaluable.Robin and Suzette were approachable and gracious; after introductions, we compared notes and realized we were all, in one way or another, supporting a controversial figure named Nick Chaillan. He was the Air Force's First Chief software officer. Over the course of the next few months, I reached out to Robin for insights and to bounce ideas about challenges to accelerating work done by and for the DoD. It's been a few years since that first introduction. That is how I went from a fan, to part of a network, to being a friend. The more I've gotten to know Robin, the more incredible and inspiring her journey has been. Like me, this Real Technologist grew up in a small town, the type with small graduating classes, and where you seem to know everybody. Oneida is in upstate New York, about 30 minutes from Syracuse.

In this episode, Abi is interviewed by Laura Tacho about the new paper he co-authored with Dr. Nicole Forsgren, Dr. Margaret-Anne Storey, and Dr. Michaela Greiler. Abi and Laura discuss the pitfalls of some of the common metrics organizations use, and how the new paper builds on prior frameworks such as DORA and SPACE to offer a new approach to measuring and improving developer productivity. Discussion topics: (2:20) Laura's background (3:59) Laura's view on git metrics (11:05) What developer experience (DevEx) is  (14:37) How the authors came together for this paper  (18:55) How DORA and SPACE are different (22:38) Limitations of DORA metrics  (24:43) Employing the DORA metrics at GitHub (27:47) What the SPACE framework is (30:44) Whether to use DORA or SPACE or both (33:54) Limitations of the SPACE framework (37:29) The need for a new approach  (38:46) What the new DevEx paper solves  (40:13) The three dimensions of developer experience  (40:54) Flow state  (43:10) Feedback loops (43:52) Cognitive load  (44:51) Why developer sentiment matters (47:58) Using both perceptual and workflow measures (50:59) Examples of perceptual and workflow measures  (54:05) How to collect metrics  (59:47) How other companies are measuring and improving developer experience (01:02:56) Advice for earlier-stage or growing organizations Resources for learning more about the DevEx framework:Read the new paper on ACM QueueRead Abi's announcement about the new paper Read how top companies measure developer productivity Connect with Abi and Laura Sign up for Laura's course, Measuring Development Team PerformanceConnect with Laura on LinkedIn or TwitterConnect with Abi on LinkedIn or Twitter

I denne Masterclass serien møter vi Torbjørn Larsen som er daglig leder i eget firma, Agile Enterprises. Han har blant annet bakgrunn fra NAV der han spilte en sentral rolle som IT-leder i deres digitaliseringsprosess. I del en av denne firedelte serien snakker Silvija og Torbjørn blant annet om IT-ledelse i offentlig sektor, digitalisering og nødvendige endringer for at dette skal fungere i praksis.-“Det er ikke flaks at Norge har et av verdens mest avanserte ligningssystemer”Dette LØRNER du: Smidighet i organisasjonerLedelse i offentlig sektor DigitaliseringsprosesserKreativitet på arbeidsplassenOmstillingstrykk i en mer og mer kompleks og utforutsigbar verdenAnbefalt litteraturEDGE - Values-Driven Digital Transformation, Jim Highsmith et alTeam Topologies: Organizing Business and Technology Teams for Fast Flow, Manuel Pais and Matthew SkeltonAccelerate, Nicole Forsgren, Jez Humble, Gene KimReinventing Organizations, Frederic Laloux Hosted on Acast. See acast.com/privacy for more information.

I del to av denne Masterclass serien vil vi høre om noen av Torbjørn sine favoritt-eksempler. Det første og mest åpenbare eksempelet for Torbjørn er NAV, men han forteller også om eksempler fra privat sektor. Du vil blant annet lære om Mikke-Mus IT, hvordan Norge har gjort NAV og Altinn til suksesser og hvordan fremtiden kan se ut.-“Det er god grunn til å begynne å automatisere rutineoppgaver”Dette LØRNER du: Eksempler fra offentlig og privat sektorStruktur i organisasjoner Store omstillinger i organisasjoner og bransjer ForretningsutviklingTverrfaglig utvikling Kundebaserte forretninger Nye forretningsmodellerAnbefalt litteratur:EDGE - Values-Driven Digital Transformation, Jim Highsmith et alTeam Topologies: Organizing Business and Technology Teams for Fast Flow, Manuel Pais and Matthew SkeltonAccelerate, Nicole Forsgren, Jez Humble, Gene KimReinventing Organizations, Frederic Laloux Hosted on Acast. See acast.com/privacy for more information.

I del tre av denne Masterclass serien snakker Torbjørn om sine favorittverktøy han har benyttet seg av i sitt arbeid. Her blir det blant annet snakket om hvordan koder i moderne tid blir skrevet for videreutvikling, samarbeidsverktøy og andre verktøy for smidige organisasjoner.-“Ikke undervurder endringene som har skjedd under panseret”Dette LØRNER du: SoftwareHvordan moderne koder blir skrevetVerktøy for smidige organisasjonerDevOpsAnbefalt litteratur:EDGE - Values-Driven Digital Transformation, Jim Highsmith et alTeam Topologies: Organizing Business and Technology Teams for Fast Flow, Manuel Pais and Matthew SkeltonAccelerate, Nicole Forsgren, Jez Humble, Gene KimReinventing Organizations, Frederic Laloux Hosted on Acast. See acast.com/privacy for more information.

I den siste delen av denne Masterclass serien går Silvija og Torbjørn gjennom Lørn som case der de foregående eksemplene og verktøyene blir brukt som grunnlag. I all hovedsak blir forretningsstrategi og implementering drøftet.-“Vi trenger å spenne ut lerretet, for så å løse de mindre problemene”Dette LØRNER du:Caseoppgave ForretningsstrategiAnbefalt litteratur: EDGE - Values-Driven Digital Transformation, Jim Highsmith et alTeam Topologies: Organizing Business and Technology Teams for Fast Flow, Manuel Pais and Matthew SkeltonAccelerate, Nicole Forsgren, Jez Humble, Gene KimReinventing Organizations, Frederic Laloux Hosted on Acast. See acast.com/privacy for more information.

This week's guest is Dr. Margaret-Anne Storey, who goes by the name Peggy. Peggy is a professor of Computer Science at the University of Victoria, the Chief Scientist at DX, and co-author of the SPACE Framework, which is the topic of focus in this episode. Today's conversation discusses what the SPACE framework is and what went into developing the metrics and categories. Peggy also shares where she sees this line of research heading next.  —Discussion points: (1:29) Peggy's background (4:01) What the SPACE framework is (5:55) Why the researchers came together for this paper(7:27) The process of writing this paper(9:52) How the SPACE categories and acronym emerged (11:50) The authors' intention for how this framework would be received(13:26) Finding a definition for what developer productivity is(17:08) The metrics included in the SPACE framework (24:48) How SPACE is different from DORA(26:17) Why lines of code and number of pull requests were included as example metrics(27:14) What Peggy is thinking about next—Mentions and links: Where to find Peggy: Twitter, WebsiteThe SPACE of Developer Productivity: There's more to it than you think by Nicole Forsgren, Margaret-Anne Storey, Chandra Madilla, Thomas Zimmerman, Brian Houck, and Jenna ButlerAbi's summary of the SPACE paper Peggy's talk, What Does Productivity Actually Mean for Developers? 

About GeneGene Kim is a multiple award-winning CTO, researcher and author, and has been studying high-performing technology organizations since 1999. He was founder and CTO of Tripwire for 13 years. He has written six books, including The Unicorn Project (2019), The Phoenix Project (2013), The DevOps Handbook (2016), the Shingo Publication Award winning Accelerate (2018), and The Visible Ops Handbook (2004-2006) series. Since 2014, he has been the founder and organizer of DevOps Enterprise Summit, studying the technology transformations of large, complex organizations.Links: The Phoenix Project: https://www.amazon.com/Phoenix-Project-DevOps-Helping-Business/dp/1942788290/ The Unicorn Project: https://www.amazon.com/Unicorn-Project-Developers-Disruption-Thriving/dp/B0812C82T9 The DevOps Enterprise Summit: https://events.itrevolution.com/ @RealGeneKim TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Cloud Economist Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: If you asked me to rank which cloud provider has the best developer experience, I'd be hard-pressed to choose a platform that isn't Google Cloud. Their developer experience is unparalleled and, in the early stages of building something great, that translates directly into velocity. Try it yourself with the Google for Startups Cloud Program over at cloud.google.com/startup. It'll give you up to $100k a year for each of the first two years in Google Cloud credits for companies that range from bootstrapped all the way on up to Series A. Go build something, and then tell me about it. My thanks to Google Cloud for sponsoring this ridiculous podcast.Corey: This episode is brought to us by our friends at Pinecone. They believe that all anyone really wants is to be understood, and that includes your users. AI models combined with the Pinecone vector database let your applications understand and act on what your users want… without making them spell it out. Make your search application find results by meaning instead of just keywords, your personalization system make picks based on relevance instead of just tags, and your security applications match threats by resemblance instead of just regular expressions. Pinecone provides the cloud infrastructure that makes this easy, fast, and scalable. Thanks to my friends at Pinecone for sponsoring this episode. Visit Pinecone.io to understand more.Corey Quinn: Welcome to Screaming in the Cloud. I'm Corey Quinn. I'm joined this week by a man who needs no introduction but gets one anyway. Gene Kim, most famously known for writing The Phoenix Project, but now the Wall Street Journal best-selling author of The Unicorn Project, six years later. Gene, welcome to the show.Gene Kim: Corey so great to be on. I was just mentioning before how delightful it is to be on the other side of the podcast. And it's so much smaller in here than I had thought it would be.Corey Quinn: Excellent. It's always nice to wind up finally meeting people whose work was seminal and foundational. Once upon a time, when I was a young, angry Unix systems administrator—because it's not like there's a second type of Unix administrator—[laughing] The Phoenix Project was one of those texts that was transformational, as far as changing the way I tended to view a lot of what I was working on and gave a glimpse into what could have been a realistic outcome for the world, or the company I was at, but somehow was simultaneously uplifting and incredibly depressing all at the same time. Now, The Unicorn Project does that exact same thing only aimed at developers instead of traditional crusty ops folks.Gene Kim: [laughing] Yeah, yeah. Very much so. Yeah, The Phoenix Project was very much aimed at ops leadership. So, Bill Palmer, the protagonist of that book was the VP of Operations at Parts Unlimited, and the protagonist in The Unicorn Project is Maxine Chambers, Senior Architect, and Developer, and I love the fact that it's told in the same timeline as The Phoenix Project, and in the first scene, she is unfairly blamed for causing the payroll outage and is exiled to The Phoenix Project, where she recoils in existential horror and then finds that she can't do anything herself. She can't do a build, she can't run her own tests. She can't, God forbid, do her own deploys. And I just love the opening third of the book where it really does paint that tundra that many developers find themselves in where they're just caught in decades of built-up technical debt, unable to do even the simplest things independently, let alone be able to independently develop tests or create value for customers. So, it was fun, very much fun, to revisit the Parts Unlimited universe.Corey Quinn: What I found that was fun about—there are few things in there I want to unpack. The first is that it really was the, shall we say, retelling of the same story in, quote/unquote, “the same timeframe”, but these books were written six years apart.Gene Kim: Yeah, and by the way, I want to first acknowledge all the help that you gave me during the editing process. Some of your comments are just so spot on with exactly the feedback I needed at the time and led to the most significant lift to jam a whole bunch of changes in it right before it got turned over to production. Yeah, so The Phoenix Project is told, quote, “in the present day,” and in the same way, The Unicorn Project is also told—takes place in the present day. In fact, they even start, plus or minus, on the same day. And there is a little bit of suspension of disbelief needed, just because there are certain things that are in the common vernacular, very much in zeitgeist now, that weren't six years ago, like “digital disruption”, even things like Uber and Lyft that feature prominently in the book that were just never mentioned in The Phoenix Project, but yeah, I think it was the story very much told in the same vein as like Ender's Shadow, where it takes place in the same timeline, but from a different perspective.Corey Quinn: So, something else that—again, I understand it's an allegory, and trying to tell an allegorical story while also working it into the form of a fictional work is incredibly complicated. That's something that I don't think people can really appreciate until they've tried to do something like it. But I still found myself, at various times, reading through the book and wondering, asking myself questions that, I guess, say more about me than they do about anyone else. But it's, “Wow, she's at a company that is pretty much scapegoating her and blaming her for all of us. Why isn't she quitting? Why isn't she screaming at people? Why isn't she punching the boss right in their stupid, condescending face and storming out of the office?” And I'm wondering how much of that is my own challenges as far as how life goes, as well as how much of it is just there for, I guess, narrative devices. It needed to wind up being someone who would not storm out when push came to shove.Gene Kim: But yeah, I think she actually does the last of the third thing that you mentioned where she does slam the sheet of paper down and say, “Man, you said the outage is caused by a technical failure and a human error, and now you're telling me I'm the human error?” And just cannot believe that she's been put in that position. Yeah, so thanks to your feedback and the others, she actually does shop her resume around. And starts putting out feelers, because this is no longer feeling like the great place to work that attracted her, eight years prior. The reality is for most people, is that it's sometimes difficult to get a new job overnight, even if you want to. But I think that Maxine stays because she believes in the mission. She takes a great deal of pride of what she's created over the years, and I think like most great brands, they do create a sense of mission and there's a deep sense of the customers they serve. And, there's something very satisfying about the work to her. And yeah, I think she is very much, for a couple of weeks, very much always thinking about, she won't be here for long, one way or another, but by the time she stumbles into the rebellion, the crazy group of misfits, the ragtag bunch of misfits, who are trying to find better ways of working and willing to break whatever rules it takes to take over the very ancient powerful order, she falls in love with a group. She found a group of kindred spirits who very much, like her, believe that developer productivity is one of the most important things that we can do as an organization. So, by the time that she looks up with that group, I mean, I think she's all thoughts of leaving are gone.Corey Quinn: Right. And the idea of, if you stick around, you can theoretically change things for the better is extraordinarily compelling. The challenge I've seen is that as I navigate the world, I've met a number of very gifted employees who, frankly wind up demonstrating that same level of loyalty and same kind of loyalty to companies that are absolutely not worthy of them. So my question has always been, when do I stick around versus when do I leave? I'm very far on the bailout as early as humanly possible side of that spectrum. It's why I'm a great consultant but an absolutely terrible employee.Gene Kim: [laughing] Well, so we were honored to have you at the DevOps Enterprise Summit. And you've probably seen that The Unicorn Project book is really dedicated to the achievements of the DevOps Enterprise community. It's certainly inspired by and dedicated to their efforts. And I think what was so inspirational to me were all these courageous leaders who are—they know what the mission is. I mean, they viscerally understand what the mission is and understand that the ways of working aren't working so well and are doing whatever they can to create better ways of working that are safer, faster, and happier. And I think what is so magnificent about so many of their journeys is that their organization in response says, “Thank you. That's amazing. Can we put you in a position of even more authority that will allow you to even make a more material, more impactful contribution to the organization?” And so it's been my observation, having run the conference for, now, six years, going on seven years is that this is a population that is being out promoted—has been promoted at a rate far higher than the population at large. And so for me, that's just an incredible story of grit and determination. And so yeah, where does grit and determination becomes sort of blind loyalty? That's ultimately self-punishing? That's a deep question that I've never really studied. But I certainly do understand that there is a time when no amount of perseverance and grit will get from here to there, and that's a fact.Corey Quinn: I think that it's a really interesting narrative, just to see it, how it tends to evolve, but also, I guess, for lack of a better term, and please don't hold this against me, it seems in many ways to speak to a very academic perspective, and I don't mean that as an insult. Now, the real interesting question is why I would think, well—why would accusing someone of being academic ever be considered as an insult, but my academic career was fascinating. It feels like it aligns very well with The Five Ideals, which is something that you have been talking about significantly for a long time. And in an academic setting that seems to make sense, but I don't see it thought of or spoken of in the same way on the ground. So first, can you start off by giving us an intro to what The Five Ideals are, and I guess maybe disambiguate the theory from the practice?Gene Kim: Oh for sure, yeah. So The Five Ideals are— oh, let's go back one step. So The Phoenix Project had The Three Ways, which were the principles for which you can derive all the observed DevOps practices from and The Four Types of Work. And so in The Five Ideals I used the concept of The Five Ideals and they are—the first—Corey Quinn: And the next version of The Nine whatever you call them at that point, I'm sure. It's a geometric progression.Gene Kim: Right or actually, isn't it the pri—oh, no. four isn't, four isn't prime. Yeah, yeah, I don't know. So, The Five Ideals is a nice small number and it was just really meant to verbalize things that I thought were very important, things I just gravitate towards. One is Locality and Simplicity. And briefly, that's just, to what degree can teams do what they need to do independently without having to coordinate, communicate, prioritize, sequence, marshal, deconflict, with scores of other teams. The Second Ideal is what I think the outcomes are when you have that, which is Focus, Flow and Joy. And so, Dr. Mihaly Csikszentmihalyi, he describes flow as a state when we are so engrossed in the work we love that we lose track of time and even sense of self. And that's been very much my experience, coding ever since I learned Clojure, this functional programming language. Third Ideal is Improvement of Daily Work, which shows up in The Phoenix Project to say that improvement daily work is even more important than daily work itself. Fourth Ideal is Psychological Safety, which shows up in the State of DevOps Report, but showed up prominently in Google's Project Oxygen, and even in the Toyota production process where clearly it has to be—in order for someone to pull the andon cord that potentially stops the assembly line, you have to have an environment where it's psychologically safe to do so. And then Fifth Ideal is Customer Focus, really focus on core competencies that create enduring, durable business value that customers are willing to pay for, versus context, which is everything else. And yeah, to answer your question, Where did it come from? Why do I think it is important? Why do I focus on that? For me, it's really coming from the State of DevOps Report, that I did with Dr. Nicole Forsgren and Jez Humble. And so, beyond all the numbers and the metrics and the technical practices and the architectural practices and the cultural norms, for me, what that really tells the story of is of The Five Ideals, as to what one of them is very much a need for architecture that allows teams to work independently, having a higher predictor of even, continuous delivery. I love that. And that from the individual perspective, the ideal being, that allows us to focus on the work we want to do to help achieve the mission with a sense of flow and joy. And then really elevating the notion that greatness isn't free, we need to improve daily work, we have to make it psychologically safe to talk about problems. And then the last one really being, can we really unflinchingly look at the work we do on an everyday basis and ask, what the customers care about it? And if customers don't care about it, can we question whether that work really should be done or not. So that's where for me, it's really meant to speak to some more visceral emotions that were concretized and validated through the State of DevOps Report. But these notions I am just very attracted to.Corey Quinn: I like the idea of it. The question, of course, is always how to put these into daily practice. How do you take these from an idealized—well, let's not call it a textbook, but something very similar to that—and apply it to the I guess, uncontrolled chaos that is the day-to-day life of an awful lot of people in their daily jobs.Gene Kim: Yeah. Right. So, the protagonist is Maxine and her role in the story, in the beginning, is just to recognize what not great looks like. She's lived and created greatness for all of her career. And then she gets exiled to this terrible Phoenix project that chews up developers and spits them out and they leave these husks of people they used to be. And so, she's not doing a lot of problem-solving. Instead, it's this recoiling from the inability for people to do builds or do their own tests or be able to do work without having to open up 20 different tickets or not being able to do their own deploys. She just recoil from this spending five days watching people do code merges, and for me, I'm hoping that what this will do, and after people read the book, will see this all around them, hopefully, will have a similar kind of recoiling reaction where they say, “Oh my gosh, this is terrible. I should feel as bad about this as Maxine does, and then maybe even find my fellow rebels and see if we can create a pocket of greatness that can become like the sublimation event in Dr. Thomas Kuhn's book, The Structure of Scientific Revolutions.” Create that kernel of greatness, of which then greatness then finds itself surrounded by even more greatness.Corey Quinn: What I always found to be fascinating about your work is how you wind up tying so many different concepts together in ways you wouldn't necessarily expect. For example, when I was reviewing one of your manuscripts before this went to print, you did reject one of my suggestions, which was just, retitle the entire thing. Instead of calling it The Unicorn Project. Instead, call it Gene Kim's Love Letter to Functional Programming. So what is up with that?Gene Kim: Yeah, to put that into context, for 25 years or more, I've self-identified as an ops person. The Phoenix Project was really an ops book. And that was despite getting my graduate degree in compiler design and high-speed networking in 1995. And the reason why I gravitated towards ops, because that was my observation, that that's where the saves were made. It was ops who saved the customer from horrendous, terrible developers who just kept on putting things into production that would then blow up and take everyone with it. It was ops protecting us from the bad adversaries who were trying to steal data because security people were so ineffective. But four years ago, I learned a functional programming language called Clojure and, without a doubt, it reintroduced the joy of coding back into my life and now, in a good month, I spend half the time—in the ideal—writing, half the time hanging out with the best in the game, of which I would consider this to be a part of, and then 20% of time coding. And I find for the first time in my career, in over 30 years of coding, I can write something for years on end, without it collapsing in on itself, like a house of cards. And that is an amazing feeling, to say that maybe it wasn't my inability, or my lack of experience, or my lack of sensibilities, but maybe it was just that I was sort of using the wrong tool to think with. That comes from the French philosopher Claude Lévi-Strauss. He said of certain things, “Is it a good tool to think with?” And I just find functional programming is such a better tool to think with, that notions like composability, like immutability, what I find so exciting is that these things aren't just for programming languages. And some other programming languages that follow the same vein are, OCaml, Lisp, ML, Elixir, Haskell. These all languages that are sort of popularizing functional programming, but what I find so exciting is that we see it in infrastructure and operations, too. So Docker is fundamentally immutable. So if you want to change a container, we have to make a new one. Kubernetes composes these containers together at the level of system of systems. Kafka is amazing because it usually reveals the desire to have this immutable data model where you can't change the past. Version control is immutable. So, I think it's no surprise that as our systems get more and more complex and distributed, we're relying on things like immutability, just to make it so that we can reason about them. So, it is something I love addressing in the book, and it's something I decided to double down on after you mentioned it. I'm just saying, all kidding aside is this a book for—Corey Quinn: Oh good, I got to make it worse. Always excited when that happens.Gene Kim: Yeah, I mean, your suggestion really brought to the forefront a very critical decision, which was, is this a book for technology leaders, or even business leaders, or is this a book developers? And, after a lot of soul searching, I decided no, this is a book for developers, because I think the sensibilities that we need to instill and the awareness we need to create these things around are the developers and then you just hope and pray that the book will be good enough that if enough engineers like it, then engineering leaders will like it. And if enough engineering leaders like it, then maybe some business leaders will read it as well. So that's something I'm eagerly seeing what will happen as the weeks, months, and years go by. Corey Quinn: This episode is sponsored in part by DataStax. The NoSQL event of the year is DataStax Accelerate in San Diego this May from the 11th through the 13th. I've given a talk previously called the myth of multi-cloud, and it's time for me to revisit that with... A sequel! Which is funny given that it's a NoSQL conference, but there you have it. To learn more, visit datastax.com that's D-A-T-A-S-T-A-X.com and I hope to see you in San Diego. This May.Corey Quinn: One thing that I always admired about your writing is that you can start off trying to make a point about one particular aspect of things. And along the way you tie in so many different things, and the functional programming is just one aspect of this. At some point, by the end of it, I half expected you to just pick a fight over vi versus Emacs, just for the sheer joy you get in effectively drawing interesting and, I guess, shall we say, the right level of conflict into it, where it seems very clear that what you're talking about is something thing that has the potential to be transformative and by throwing things like that in you're, on some level, roping people in who otherwise wouldn't weigh in at all. But it's really neat to watch once you have people's attention, just almost in spite of what they want, you teach them something. I don't know if that's a fair accusation or not, but it's very much I'm left with the sense that what you're doing has definite impact and reverberations throughout larger industries.Gene Kim: Yeah, I hope so. In fact, just to reveal this kind of insecurity is, there's an author I've read a lot of and she actually read this blog post that she wrote about the worst novel to write, and she called it The Yeomans Tour of the Starship Enterprise. And she says, “The book begins like this: it's a Yeoman on the Starship Enterprise, and all he does is admire the dilithium crystals, and the phaser, and talk about the specifications of the engine room.” And I sometimes worry that that's what I've done in The Unicorn Project, but hopefully—I did want to have that technical detail there and share some things that I love about technology and the things I hate about technology, like YAML files, and integrate that into the narrative because I think it is important. And I would like to think that people reading it appreciate things like our mutual distaste of YAML files, that we've all struggled trying to escape spaces and file names inside of make files. I mean, these are the things that are puzzles we have to solve, but they're so far removed from the business problem we're trying to solve that really, the purpose of that was trying to show the mistake of solving puzzles in our daily work instead of solving real problems.Corey Quinn: One thing that I found was really a one-two punch, for me at least, was first I read and give feedback on the book and then relatively quickly thereafter, I found myself at my first DevOps Enterprise Summit, and I feel like on some level, I may have been misinterpreted when I was doing my live-tweeting/shitposting-with-style during a lot of the opening keynotes, and the rest, where I was focusing on how different of a conference it was. Unlike a typical DevOps Days or big cloud event, it wasn't a whole bunch of relatively recent software startups. There were serious institutions coming out to have conversations. We're talking USAA, we're talking to US Air Force, we're talking large banks, we're talking companies that have a 200-year history, where you don't get to just throw everything away and start over. These are companies that by and large, have, in many ways, felt excluded to some extent, from the modern discussions of, well, we're going to write some stuff late at night, and by the following morning, it's in production. You don't get to do that when you're a 200-year-old insurance company. And I feel like that was on some level interpreted as me making fun of startups for quote/unquote, “not being serious,” which was never my intention. It's just this was a different conversation series for a different audience who has vastly different constraints. And I found it incredibly compelling and I intend to go back.Gene Kim: Well, that's wonderful. And, in fact, we have plans for you, Mr. Quinn.Corey Quinn: Uh-oh.Gene Kim: Yeah. I think when I say I admire the DevOps Enterprise community. I mean that I'm just so many different dimensions. The fact that these, leaders and—it's not leaders just in terms of seniority on the organization chart—these are people who are leading technology efforts to survive and win in the marketplace. In organizations that have been around sometimes for centuries, Barclays Bank was founded in the year 1634. That predates the invention of paper cash. HMRC, the UK version of the IRS was founded in the year 1200. And, so there's probably no code that goes that far back, but there's certainly values and—Corey Quinn: Well, you'd like to hope not. Gene Kim: Yeah, right. You never know. But there are certainly values and traditions and maybe even processes that go back centuries. And so that's what's helped these organizations be successful. And here are a next generation of leaders, trying to make sure that these organizations see another century of greatness. So I think that's, in my mind, deeply admirable.Corey Quinn: Very much so. And my only concern was, I was just hoping that people didn't misinterpret my snark and sarcasm as aimed at, “Oh, look at these crappy—these companies are real companies and all those crappy SAS companies are just flashes in the pan.” No, I don't believe that members of the Fortune 500 are flash in the pan companies, with a couple notable exceptions who I will not name now, because I might want some of them on this podcast someday. The concern that I have is that everyone's work is valuable. Everyone's work is important. And what I'm seeing historically, and something that you've nailed, is a certain lack of stories that apply to some of those organizations that are, for lack of a better term, ossified into their current process model, where they there's no clear path for them to break into, quote/unquote, “doing the DevOps.”Gene Kim: Yeah. And the business frame and the imperative for it is incredible. Tesla is now offering auto insurance bundled into the car. Banks are now having to compete with Apple. I mean, it is just breathtaking to see how competitive the marketplaces and the need to understand the customer and deliver value to them quickly and to be able to experiment and innovate and out-innovate the competition. I don't think there's any business leader on the planet who doesn't understand that software is eating the world and they have to that any level of investment they do involves software at some level. And so the question is, for them, is how do they get educated enough to invest and manage and lead competently? So, to me it really is like the sleeping giant awakening. And it's my genuine belief is that the next 50 years, as much value as the tech giants have created: Facebook, Amazon, Netflix, Google, Microsoft, they've generated trillions of dollars of economic value. When we can get eighteen million developers, as productive as an engineer at a tech giant is, that will generate tens of trillions of dollars of economic value per year. And so, when you generate that much economic activity, all problems become solvable, you look at climate change, you take a look at the disparity between rich and poor. All things can be fixed when you significantly change the economic economy in this way. So, I'm extremely hopeful and I know that the need for things like DevOps are urgent and important.Corey Quinn: I guess that that's probably the best way of framing this. So you wrote one version that was aimed at operators back in 2013, this one was aimed at developers, and effectively retails and clarifies an awful lot of the same points. As a historical ops person, I didn't feel left behind by The Unicorn Project, despite not being its target market. So I guess the question on everyone's mind, are you planning on doing a third iteration, and if so, for what demographic?Gene Kim: Yeah, nothing at this point, but there is one thing that I'm interested in which is the role of business leaders. And Sarah is an interesting villain. One of my favorite pieces of feedback during the review process was, “I didn't think I could ever hate Sarah more. And yet, I did find her even to be more loathsome than before.” She's actually based on a real person, someone that I worked with.Corey Quinn: That's the best part, is these characters are relatable enough that everyone can map people they know onto various aspects of them, but can't ever disclose the entire list in public because that apparently has career consequences.Gene Kim: That's right. Yes, I will not say who the character is based on but there's, in the last scene of the book that went to print, Sarah has an interesting interaction with Maxine, where they meet for lunch. And, I think the line was, “And it wasn't what Maxine had thought, and she's actually looking forward to the next meeting.” I think that leaves room for it. So one of the things I want to do with some friends and colleagues is just understand, why does Sarah act the way she does? I think we've all worked with someone like her. And there are some that are genuinely bad actors, but I think a lot of them are doing something, based on genuine, real motives. And it would be fun, I thought, to do something with Elizabeth Henderson, who we decided to start having a conversation like, what does she read? What is her background? What is she good at? What does her resume look like? And what caused her to—who in technology treated her so badly that she treats technology so badly? And why does she behave the way she does? And so I think she reads a lot of strategy books. I think she is not a great people manager, I think she maybe has come from the mergers and acquisition route that viewed people as fungible. And yeah, I think she is definitely a creature of economics, was lured by an external investor, about how good it can be if you can extract value out of the company, squeeze every bit of—sweat every asset and sell the company for parts. So I would just love to have a better understanding of, when people say they work with someone like a Sarah, is there a commonality to that? And can we better understand Sarah so that we can both work with her and also, compete better against her, in our own organizations?Corey Quinn: I think that's probably a question best left for people to figure out on their own, in a circumstance where I can't possibly be blamed for it.Gene Kim: [laughing].That can be arranged, Mr. Quinn.Corey Quinn: All right. Well, if people want to learn more about your thoughts, ideas, feelings around these things, or of course to buy the book, where can they find you?Gene Kim: If you're interested in the ideas that are in The Unicorn Project, I would point you to all of the freely available videos on YouTube. Just Google DevOps Enterprise Summit and anything that's on the plenary stage are specifically chosen stories that very much informed The Unicorn Project. And the best way to reach me is probably on Twitter. I'm @RealGeneKim on Twitter, and feel free to just @ mention me, or DM me. Happy to be reached out in whatever way you can find me. Corey Quinn: You know where the hate mail goes then. Gene, thank you so much for taking the time to speak with me, I appreciate it.Gene Kim: And Corey, likewise, and again, thank you so much for your unflinching feedback on the book and I hope you see your fingerprints all over it and I'm just so delighted with the way it came out. So thanks to you, Corey. Corey Quinn: As soon as my signed copy shows up, you'll be the first to know.Gene Kim: Consider it done. Corey Quinn: Excellent, excellent. That's the trick, is to ask people for something in a scenario in which they cannot possibly say no. Gene Kim, multiple award-winning CTO, researcher, and author. Pick up his new book, The Wall Street Journal best-selling The Unicorn Project. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on Apple Podcasts. If you hated this podcast, please leave a five-star review on Apple Podcasts and leave a compelling comment.Announcer: This has been this week's episode of Screaming in the Cloud. You can also find more Corey at ScreamingintheCloud.com or wherever fine snark is sold.This has been a HumblePod production. Stay humble.

This is a special episode from our new show “Engineering Founders” - Should you build B2C or B2B? What about implementing a top-down or a bottoms up sales strategy? How do you think about pricing? These are many of the dilemmas early founders face in the early stages. We sit down with Abi Noda to explore his experiences co-founding DX and Pull Panda and examine the differences, trade-offs and considerations behind building for consumer vs. B2B, pricing, early sales and product adoption strategies! For more episodes of Engineering Founders, subscribe here: https://engineering-founders.simplecast.com/P.S. The Engineering Leadership Podcast will return after the winter holidays on January 3rd!ABOUT ABI NODAAbi Noda is the CEO and co-founder of DX, the world's first developer experience management platform. He was previously the CEO and founder of Pull Panda, which was acquired by GitHub in 2019. At GitHub he led research collaborations with Dr. Nicole Forsgren, McKinsey, and Microsoft Research, which was the impetus for founding DX."It's really good to try to sell starting on day one. That's probably, in my opinion, the best way to validate an idea, a B2B idea, is to try and go sell it and by sell it I mean literally go get money for like pre-committed customers. So it really de-risks a huge component of, I think, why these types of businesses fail, which is they just aren't able even identify, reach and successfully convert buyers.”- Abi Noda   ABOUT DXDX is the world's first developer experience management platform, helping organizations measure and improve top drivers of developer productivity and engagement.DX is designed by leading software engineering researchers, providing science-backed metrics, workflows, and education that empower teams to improve.Interested in joining an ELC Peer Group?ELCs Peer Groups provide a virtual, curated, and ongoing peer learning opportunity to help you navigate the unknown, uncover solutions and accelerate your learning with a small group of trusted peers.Apply to join a peer group HERE: sfelc.com/peerGroupsSHOW NOTES:Abi's journey founding DX and Pull Panda (3:07)Building your business as a side-project for consumers vs. enterprise software (6:07)If you just got laid off and want to start a business, you need to hear this (10:02)The best way to validate a B2B idea (12:44)Differences with how you talk about your product in a competitive vs. uncompetitive market (15:58)How to think about pricing for bottoms-up or top-down sales motion (17:17)Choosing the right persona to pursue as customers (20:58)How experience at large companies can help you understand how to approach enterprise product adoption (24:44)Investor expectations with bottoms-up/top-down sales and identifying ICPs (32:06)Incentivizing users to adopt new features (34:12)Closing deals and getting to the implementation stage (37:51)How Abi maximized advisor relationships (40:04)Rapid fire questions (45:27)LINKS AND RESOURCESLenny Rachitsky's Newsletter - a weekly advice column about product, growth, and your career.7 Powers: The Foundations of Business Strategy - Hamilton Helmer's comprehensive business strategy guide centered around power and the conditions that create the potential for persistent differential returns.Nail It Then Scale It - Nathan Furr and Paul Ahlstrom's guide to increasing success and reducing risk when launching a high-growth company.

Sejal Amin is the newly-named Chief Technology Officer at Shutterstock where she has a broad mandate to drive Shutterstock's digital transformation into a full-service creative platform for hundreds of thousands of creative professionals around the world. She talks with 3Pillar's Scott Varho and Elisabeth Beller about moving from software project to product on this episode of The Innovation Engine. Tune in to the full conversation to hear how Shutterstock is making the leap from project to product, what a value stream is and how to manage it within a company, and why using a body of metrics is vital to drive decision-making during digital transformation.   Resources: Connect with Sejal on LinkedIn Read: Accelerate: Building and Scaling High Performing Technology Organizations by Nicole Forsgren, PhD, Jez Humble, and Gene Kim Read: Project to Product: How to Survive and Thrive in the Age of Digital Disruption with the Flow Framework by Mik Kersten Read: Sooner Safer Happier: Antipatterns and Patterns for Business Agility by Jonathan Smart If you have any feedback about the podcast or guests you'd like to hear interviewed, send your suggestions to info@3pillarglobal.com.   Learn more and get the full show notes at: 3PillarGlobal.com

Whether you're in engineering, working alongside engineering teams, or you're just looking to uplevel your career in tech - I have a treat for you this week, my love!  Today, I invite three extraordinary and powerful leaders in the tech industry to talk about their unique experiences building great engineering teams. Join us as Jossie, Kathryn & Sushma share their approaches to getting ahead as engineering leaders, and how to build better relationships with these important teams in your own organization.  Ready to learn more about how engineering interacts with the rest of the organization - and gain some powerful leadership tools along the way? Let's go to the show!  We dive into: How these incredible women became VPs of engineering - and why they do what they do! 4 key metrics you can use to highlight how engineering is moving towards company goals How to make sure the engineering team supports business strategy (even when the rest of the organization may not fully understand what they do!) One VITAL piece to building good communication between engineering and the rest of the organization Their BEST advice to help you on your path to VP! And so much more!   **Useful links** Book recommendations: Accelerate: Building and Scaling High Performing Technology Organizations, by Nicole Forsgren, Jez Humble & Gene Kim Team Topologies: Organizing Business and Technology Teams for Fast Flow by Matthew Skelton & Manuel Pais If you are ready to uplevel your career, and get a boost (and a salary bump) by shortcutting your way to success, find out more about Toni's Coaching at: https://tonicollis.com/workwithtoni  Alternatively, go straight ahead and book a free Discovery Call, to find out more and discuss the type of support you would most benefit from: https://bit.ly/DiscoverToni  Catch the show notes, and more details about today's episode here: https://tonicollis.com/episode121  Join the Leading Women in Tech community in Slack where we discuss all-the-things for women's tech leadership, covering everything from early-career leadership to C-level executives.

What is the Second Cloud Transformation? In this episode we are talking about the second cloud transformation. A lot of people are talking about what happens when you get to the cloud? You have already moved to the cloud. But you are not achieving the cloud successes like being event driven or serverless. It can be a surprise for organisations that work in traditional ways. Their frame of reference is all on premise like big data centres or new mainframe stuff. So this new paradigm causes uncertainty around what can be done. What does a good cloud solution look like? Or what's a good implementation for your context? There's a learning curve that people need to climb and to become more comfortable. When companies move to the cloud, it's usually part of a transformation. But the reason why we call it 'The Second Cloud Transformation' is because there's another step change. The best person I heard describe this is Shaun Braun from 3M. He did a keynote at AWS re:Invent in 2021. He talked about people moving to the cloud and measuring data. And then they transform by shutting things down and redoing other things. Enterprises and people from mature companies are struggling. They are in the cloud but having problems with account creation, observability, security or how to deploy or provision things. There are infrastructure things that they haven't done. Because they haven't modernised. Sometimes the move to the cloud happens quickly. People have not been left behind. But they've been focusing on other things and they haven't gone back to first principles. To rethink how the software capabilities work. You need to move away from big upfront design. And shift design, decision making and architectural decision making onto the teams. Because they're the ones who know the business problem and business aims best. And safe space is a big thing. Situational awareness and creating an environment that is safe for the organisation. Because teams will make mistakes. But you have got to encourage experimentation in a safe way. A good enabler for that is thinking about and setting up policies for the services you want to start leveraging out of the box. In the second phase of The Value Flywheel Effect you map out your capability. So you do need to have an honest assessment of the capabilities you have and the teams in the organisation. Because maybe the engineers don't understand security. At least then you know that you've got to level people up. 'Shift left' is a great mechanism, because your teams have to do more and there's going to be gaps. So in some ways the second cloud transformation is filling those gaps. And you end up with well rounded teams. We talk about asking the question: 'what does good look like?' One of the best books to answer that question is 'Accelerate' by Nicole Forsgren.In the book there are four key metrics for building and scaling high performance technology organisations. Another go to book for this particular topic is 'Reaching Cloud Velocity', by Jonathan Allen. How to spin up new teams and the mitosis approach? It's a phenomenally good reference, similar to 'Accelerate'. And we reference both those books in our book 'The Value Flywheel Effect' which is available for preorder! When you get into the cloud you need to enable real transformation! This is what we are calling the second cloud transformation. TheServerlessEdge.com and @ServerlessEdge  Shaun Braun from 3M at AWS re:Invent Accelerate Reaching Cloud Velocity The Value Flywheel Effect Book with IT Revolution

HOW TO MAKE YOUR DAYS BETTER WITH DEVOPS - KEYNOTE BY NICOLE FORGSGREN How can we support our work and improve our teamwork? DevOps was originally started to take care of people doing the work while they built great software. In this talk, Nicole Forsgren discusses how DevOps practices can not only help us ship software with speed and stability, they can also reduce burnout, improve our culture, and communicate better. - The DEVOPS Conference has been organized annually as a virtual event. On November 1, 2022, the event is going to Copenhagen, Denmark, and will be live-streamed too. Join us for a full day to discuss today's DevOps: cloud native, platform engineering, GreenOps, psychological safety and more. - -Book your early bird ticket by September 12: https://hubs.li/Q01l8PZ70 -Sign up for the free live streaming: https://hubs.li/Q01l8Qsy0 -Watch all talks from The DEVOPS Conference 2022: https://hubs.li/Q01l9hFt0 - Follow Nicole on Twitter: @nicolefv

In this special episode, Dr. Nicole Forsgren, author of award-winning book Accelerate and co-author of "The SPACE of Developer Productivity", talks about her work with DORA, the inspiration behind the SPACE framework, and how she's thinking about developer experience.Watch the on-demand fireside chat or read the announcement of Nicole joining DX as a strategic advisor. 

This episode originally aired on Software Engineering Radio.Randy Shoup is the VP of Engineering and Chief Architect at eBay. He was previously the VP of Engineering at WeWork and Stitch Fix, a Director of Engineering at Google Cloud where he worked on App Engine, and a Chief Engineer and Distinguished Architect at eBay in 2004. Topics covered: eBay's origins as a single C++ class The five-year migration to Java services Sharing a database between the old and new systems Building a distributed tracing system Working with bare metal Why most companies should stick to cloud Why individual services should own their own data storage How scale has caused solutions to change Rejoining a former company The Accelerate Book Improving delivery time.  Related Links:@randyshoupOpenTelemetryLightStepHoneycombAccelerate BookThe MemoValue Stream MappingThe Epic Story of Dropbox's Exodus from the Amazon Cloud EmpireTranscript:[00:00:00] Jeremy: Today, I'm talking to Randy Shoup, he's the VP of engineering and chief architect at eBay.[00:00:05] Jeremy: He was previously the VP of engineering at WeWork and stitch fix, and he was also a chief engineer and distinguished architect at eBay back in 2004. Randy, welcome back to software engineering radio. This will be your fifth appearance on the show. I'm pretty sure that's a record.[00:00:22] Randy: Thanks, Jeremy, I'm really excited to come back. I always enjoy listening to, and then also contributing to software engineering radio.Back at, Qcon 2007, you spoke with Markus Volter he's he was the founder of SE radio. And you were talking about developing eBay's new search engine at the time.[00:00:42] Jeremy: And kind of looking back, I wonder if you could talk a little bit about how eBay was structured back then, maybe organizationally, and then we can talk a little bit about the, the tech stack and that sort of thing.[00:00:53] Randy: Oh, sure. Okay. Yeah. Um, so eBay started in 1995. I just want to like, you know, orient everybody. Same, same as the web. Same as Amazon, same as a bunch of stuff. So E-bay was actually almost 10 years old when I joined. That seemingly very old first time. Um, so yeah. What was ebay's tech stack like then? So E-bay current has gone through five generations of its infrastructure.It was transitioning between the second and the third when I joined in 2004. Um, so the. Iteration was Pierre Omidyar, the founder three-day weekend three-day labor day weekend in 1995, playing around with this new cool thing called the web. He wasn't intending to build a business. He just was playing around with auctions and wanted to put up a webpage.So he had a Perl backend and every item was a file and it lived on this little 486 tower or whatever you had at the time. Um, so that wasn't scalable and wasn't meant to be. The second generation of eBay's architecture was what we called V2 very, you know, creatively, uh, that was a C++ monolith. Um, an ISAPI DLL with essentially well at its worst, which grew to 3.4 million lines of code in that single DLL and basically in a single class, not just in a single, like repo or a single file, but in a single class.So that was very unpleasant to work in. As you can imagine, um, eBay had about a thousand engineers at the time and they were, you know, as you can imagine, like really stepping on each other's toes and not being able to make much forward progress. So starting in, I want to call it 2002. So two years before I joined, um, they were migrating to the creatively named V3 and V3 architecture was Java, and.you know, not microservices, but like we didn't even have that term, but it wasn't even that it was mini applications. So I'm actually going to take a step back. V2 was a monolith. So like all of eBay's code in that single DLL and like that was buying and selling and search and everything. And then we had two monster databases, a primary and a backup big Oracle machines on some hardware that was bigger, you know, bigger than refrigerators and that ran eBay for a bunch of years, before we changed the upper part of the stack, we, um, chopped up the, that single monolithic database into a bunch of, um, domain specific databases or entity specific databases, right?So a set of databases around users, you know, sharded by the user ID could talk about all that. If you want, you know, items again, sharded by item ID transactions, sharded by transaction ID... I think when I joined, it was the several hundred instances of, uh, Oracle databases, um, you know, spread around, but still that monolithic front end.And then in 2002, I wanna say we started migrating into that V3 that I was saying, okay. So that's, uh, that was a rewrite in Java, again, many applications. So you take the front end and instead of having it be in one big unit, it was this, uh, ER file, EAR, file, if run and people remember back to, you know, those stays in Java, um, you know, 220 different of those.So like here is the, you know, one of them for the search pages, you know, so the, you know, one application be the search application and it would, you know, do all the search related stuff, the handful of pages around search, uh, ditto for, you know, the buying area, ditto for the, you know, checkout area, ditto for the selling area...220 of those. Um, and that was again, domain, um, vertically sliced domains. And then the relationship between those V3, uh, applications and the databases was a many to many things. So like many applicants, many of those applications interact with items. So they would interact with those item databases. Many of them would interact with users.And so they would interact with a user databases, et cetera, uh, happy to go into as much gory detail as you want about all that. But like, that's what, uh, but we were in the transition period. You know, when I, uh, between the V2 monolith to the V3 mini applications in, uh, 2004, I'm just going to pause there and like, let me know where you want to take it.[00:05:01] Jeremy: Yeah. So you were saying that it was, um, it started as Perl, then it became a C++, and that's kind of interesting that you said it was all in one class, right? So it's wow. That's gotta be a gigantic [00:05:16] Randy: I mean, completely brutal. Yeah. 3.4 million lines of code. Yeah. We were hitting compiler limits on the number of methods per class.[00:05:22] Jeremy: Oh my gosh.[00:05:23] Randy: I'm, uh, uh, scared that I have that. I happen to know that at least at the time, uh, Microsoft allowed you 16 K uh, methods per class, and we were hitting that limit.So, uh, not great.[00:05:36] Jeremy: So it's just kind of interesting to think about how do you walk through that code, right? You have, I guess you just have this giant file.[00:05:45] Randy: Yeah. I mean, there were, you know, different methods. Um, but yeah, it was a big man. I mean, it was a monolith, it was, uh, you know, it was a spaghetti mess. Um, and you know, as you can imagine, Amazon went through a really similar thing by the way. So this wasn't soup. I mean, it was bad, but like we weren't the only people that were making that, making that a mistake.Um, and just like Amazon, where they were, uh, they did like one update a quarter (laughs) , you know, at that period, like 2000, uh, we were doing something really similar, like very, very slow. Um, you know, updates and, uh, when we moved to V3, you know, the idea was to get to do changes much faster. And we were very proud of ourselves starting in 2004 that we, uh, upgraded the whole site every two weeks.And we didn't have to do the whole site, but like each of those individual applications that I was mentioning, right. Those 220 applications, each of those would roll out on this biweekly cadence. Um, and they had interdependencies. And so we rolled them out in this dependency order in any way, lots of, lots of complexity associated with that.Um, yeah, there you go.[00:06:51] Jeremy: the V3 that, that was written in Java, I'm assuming this was a, as a complete rewrite. You, you didn't use the C++ code at all.[00:07:00] Randy: Yeah. And, uh, it was, um, we migrated, uh, page by page. So, uh, you know, in the transition period, which lasted probably five years, um, there were pages, you know, in the beginning, all pages were served by V2. In the end, all pages are served by V3 and, you know, over time you iterate and you like rewrite in parallel, you know, rewrite and maintain in parallel the V3 version of XYZ page and the V2 version of XYZ page.Um, and then when you're ready, you start to test out at low percentages of traffic, you know, what would, what does V3 look like? Is it correct? And when it isn't do you go and fix it, but then ultimately you migrate the traffic over, um, to fully take, get fully be in the V3 world, and then you, you know, remove or comment out or whatever.The, the code that supported that in the V2 monolith.[00:07:54] Jeremy: And then you had mentioned using Oracle databases. Did you have a set for V2 and a separate V3 and you were kind of trying to keep them in sync?[00:08:02] Randy: Oh, great question. Thank you for asking that question. No, uh, no. We had the databases. Um, so again, as I mentioned, we had pre-demonolith that's my that's a technical term, uh, pre broken up the databases starting in, let's call it 2000. Uh, actually I'm almost certain that's 2000. Cause we had a major site outage in 1999, which everybody still remembers who was there at the time.Uh wasn't me or I wasn't there at the time. Uh, but you know, you can look it up. Uh, anyway, so yeah, starting in 2000, we broke up that monolithic database into what I was telling you before those entity aligned databases. Again, one set for items, one set for users, one set for transactions, you know, dot dot, dot, um, and that division of those databases was shared.You know, those databases were shared between. The three using those things and then V sorry, V2 using those things and V3 using those things. Um, and then, you know, so we've completely decoupled the rewrite of the database, you know, kind of data storage layer from the rewrite of the application layer, if that makes sense.[00:09:09] Jeremy: Yeah. So, so you had V2 that was connecting to these individual Oracle databases. You said like they were for different types of entities, like maybe for items and users and things like that. but it was a shared database situation where V2 was connected to the same database as V3. Is that right?[00:09:28] Randy: Correct and also in V3, even when done. Different V3 applications, were also connecting to the same database, again, like anybody who used user, anybody who used the user entity, which is a lot we're connecting to the user suite of databases and anybody who used the item entity, which again is a lot, um, you were connecting to the item databases, et cetera.So yeah, it was this many to many that's, I'm trying to say many to many relationship between applications in the V3 world and databases.[00:10:00] Jeremy: Okay. Yeah, I think I, I got it because[00:10:03] Randy: It's easier with a diagram.[00:10:04] Jeremy: yeah. W 'cause when you, when you think about services now, um, you think of services having dependencies on other services. Whereas in this case you would have multiple services that rather than talking to a different service, they would all just talk to the same database.They all needed users. So they all needed to connect to the user's database.[00:10:24] Randy: Right exactly. And so, uh, I don't want to jump ahead in this conversation, but like the problems that everybody has, everybody who's feeling uncomfortable at the moment. You're right. To feel uncomfortable because that wasn't unpleasant situation and microservices, or more generally the idea that individual services would own their own data.And only in the only are interactions to the service would be through the service interface and not like behind the services back to the, to the data storage layer. Um, that's better. And Amazon discovered that, you know, uh, lots of people discovered that around that same, around that same early two thousands period.And so yeah, we had that situation at eBay at the time. Uh, it was better than it was before. Right, right. Better than a monolithic database and a monolithic application layer, but it definitely also had issues. Uh, as you can imagine,[00:11:14] Jeremy: you know, thinking about back to that time where you were saying it's better than a monolith, um, what were sort of the trade-offs of, you know, you have a monolith connecting to all these databases versus you having all these applications, connecting to all these databases, like what were the things that you gained and what did you lose if that made sense?[00:11:36] Randy: Hmm. Yeah. Well, I mean, why we did it in the first place is develop is like isolation between development teams right? So we were looking for developer productivity or the phrase we used to use was feature velocity, you know, so how quickly would we be able to move? And to the extent that we could move independently, you know, the search team could move independently from the buying team, which could move independently from the selling team, et cetera.Um, that was what we were gaining. Um, what were we losing? Uh, you know, when you're in a monolith situation, If there's an issue, you know, where it is, it's in the monolith. You might not know where in the monolith. Um, but like there's only one place that could be. And so an issue that one has, uh, when you break things up into smaller units, uh, especially when they have this, you know, shared, shared mutable state, essentially in the form of these databases, like who changed that column?What, you know, what's the deal. Uh, actually we did have a solution for that or something that really helped us, which was, um, now 20, more than 20 years ago, we had something that we would now call distributed tracing where, uh, actually I talked about this way back in the 2007 thing, cause it was pretty cool, uh, at the time, uh, You know, just like the spans one would create using a modern distributed tracing, you know, open telemetry or, you know, any of the disruptive tracing vendors.Um, just like you would do that. We, we didn't use the term span, but that same idea where, um, we could, and the goal was the same to like debug stuff. So, uh, every time we were about to make a database call, we would say, Hey, I'm about to make this data, you know, we would log we about to make this database call and then it would happen.And then we would log whether it was successful or not successful. We could see how long it took, et cetera. Um, and so we built our own, you know, monitoring system, which, which we called central application logging or CAL, uh, totally proprietary to eBay. I'm happy to talk about whatever gory details you want to know about that, but it was pretty cool certainly way back in 2000.It was, and that was our mitigation against the thing I'm telling you, which is, you know, when something, when not. Something is weird in the database. We can kind of back up and figure out where it might've happened, or things are slow. What's, you know, what's the deal. And, uh, you know, cause sometimes the database is slow for reasons.Um, and what, which, what thing is, you know, from an application perspective, I'm talking to 20 different databases, but things are slow. Like what is it? And, um, CAL helped us to, to figure out both elements of that, right? Like what applications are talking to, what databases and what backend services and like debug and diagnose from that perspective.And then for a given application, what, you know, databases in backend services are you talking to? And, um, debug that. And then we have the whole, and then we, um, we, we had monitors on those things and we would notice when databases would, where be a lot of errors or where, when database is starting in slower than they used to be.Um, and then. We implemented what people would now call circuit breakers, where we would notice that, oh, you know, everybody who's trying to talk to database 1, 2, 3, 4 is seeing it slow down. I guess 1, 2, 3, 4 is unhappy. So now flip everybody to say, don't talk to 1, 2, 3, 4, and like, just that kind of stuff.You're not going to be able to serve. Uh, but whatever, that's better than stopping everything. So I hope that makes sense. Like, you know, so all these, all these like modern resilience techniques, um, we always had, we had our own proprietary names for them, but you know, we, we implemented a lot of them way back when,[00:15:22] Jeremy: Yeah. And, and I guess just to contextualize it for the audience, I mean, this was back in 2004. Oh it back in 2000.[00:15:32] Randy: Again, because we had this, sorry to interrupt you because we have, the problem is that we were just talking about where application many applications are talking to many services and databases and we didn't know what was going on. And so we needed some visibility into what was going on.Sorry, go ahead.[00:15:48] Jeremy: yeah. Okay. So all the way back in 2000, there's a lot less, Services out there, like nowadays you think about so many software as a service products. if you were building the same thing today, what are some of the services that people today would just go and say like, oh, I'll just, I'll just pay for this and have this company handle it for me. You know, that wasn't available, then[00:16:10] Randy: sure. Well, there. No, essentially, no. Well, there was no cloud cloud didn't happen until 2006. Um, and there were a few software as a service vendors like Salesforce existed at the time, but they weren't usable in the way you're thinking of where I could give you money and you would operate a technical or technological software service on my behalf.Do you know what I mean? So we didn't have any of the monitoring vendors. We didn't have any of the stuff today. So yeah. So what would we do, you know, to solve that specific problem today? Uh, I would, as we do today, I would, uh, instrument everything with open telemetry because that's generic. Thank you, Ben Siegelman and LightStep for starting that whole open sourcing process, uh, of that thing and, and, um, getting all the vendors to, you know, respect it.Um, and then I would shoot, you know, for my backend, I would choose one of the very many wonderful, uh, you know, uh, distributed tracing vendors of which there are so many, I can't remember, but like LightStep is one honeycomb... you know, there were a bunch of, uh, you know, backend, um, distributed tracing vendors in particular, you know, for that.Uh, what else do you have today? I mean, we could go on for hours on this one, but like, we didn't have distributed logging or we didn't have like logging vendors, you know? So there was no, uh, there was no Splunk, there was no, um, you know, any, any of those, uh, any of the many, uh, distributed log, uh, or centralized logging vendor, uh, vendors.So we didn't have any of those things. We didn't. like caveman, you know, we rent, we, uh, you know, had our own data. We built our own data centers. We racked our own servers. We installed all the OSS in them, you know, uh, by the way, we still do all that because it's way cheaper for us at our scale to do that.But happy to talk about that too. Uh, anyway, but yeah, no, the people who live in, I don't know if this is where you want to go in 2022, the software developer has this massive menu of options. You know, if you only have a credit card, uh, and it doesn't usually cost that much, you can get a lot of stuff done from the cloud vendors, from the software service vendors, et cetera, et cetera.And none of that existed in 2000.[00:18:31] Jeremy: it's really interesting to think about how different, I guess the development world is now. Like, cause you mentioned how cloud wasn't even really a thing until 2006, all these, these vendors that people take for granted. Um, none of them existed. And so it just, uh, it must've been a very, very different time.[00:18:52] Randy: Well, we didn't know. It was every, every year is better than the previous year, you know, in software every year. You know? So at that time we were really excited that we had all the tools and capabilities that, that we did have. Uh, and also, you know, you look back from, you know, 20 years in the future and, uh, you know, it looks caveman, you know, from that perspective.But, uh, it was, you know, all those things were cutting edge at the time. What happened really was the big companies rolled their own, right. Everybody, you know, everybody built their own data centers, rack their own servers. Um, so at least at scale and the best you could hope for the most you could pay anybody else to do is rack your servers for you.You know what I mean? Like there were external people, you know, and they still exist. A lot of them, you know, the Rackspaces you know Equinixes, et cetera of the world. Like they would. Have a co-location facility. Uh, and you, you know, you ask them please, you know, I'd like to buy the, these specific machines and please rack these specific machines for me and connect them up on the network in this particular way.Um, that was the thing you could pay for. Um, but you pretty much couldn't pay them to put software on there for you. That was your job. Um, and then operating. It was also your job, if that makes sense.[00:20:06] Jeremy: and then back then, would that be where. Employees would actually have to go to the data center and then, you know, put in their, their windows CD or their Linux CD and, you know, actually do everything right there.[00:20:18] Randy: Yeah. 100%. Yeah. In fact, um, again, anybody who operates data centers, I mean, there's more automation, but the conceptually, when we run three data centers ourselves at eBay right now, um, and all of our, all of our software runs on them. So like we have physical, we have those physical data centers. We have employees that, uh, physically work in those things, physical.Rack and stack the servers again, we're smarter about it now. Like we buy a whole rack, we roll the whole rack in and cable it, you know, with one big chunk, uh, sound, uh, as distinct from, you know, individual wiring and the networks are different and better. So there's a lot less like individual stuff, but you know, at the end of the day, but yeah, everybody in quotes, everybody at that time was doing that or paying somebody to do exactly that.Right. Yeah.[00:21:05] Jeremy: Yeah. And it's, it's interesting too, that you mentioned that it's still being done by eBay. You said you have three, three data centers. because it seems like now maybe it's just assumed that someone's using a cloud service or using AWS or whatnot. And so, oh, go ahead.[00:21:23] Randy: I was just going to say, well, I'm just going to riff off what you said, how the world has changed. I mean, so much, right? So. Uh, it's fine. You didn't need to say my whole LinkedIn, but like I used to work on Google cloud. So I've been, uh, I've been a cloud vendor, uh, at a bunch of previous companies I've been a cloud consumer, uh, at stitch fix and we work in other places.Um, so I'm fully aware, you know, fully, fully, personally aware of, of all that stuff. But yeah, I mean, there's this, um, you know, eBay is in the, uh, eBay is at the size where it is actually. Cost-effective very, cost-effective, uh, can't tell you more than that, uh, for us to operate our own, um, uh, our own infrastructure, right?So, you know, you know, one would expect if Google didn't operate their own infrastructure, nobody would expect Google to use somebody else's right. Like that, that doesn't make any economic sense. Um, and, uh, you know, Facebook is in the same category. Uh, for a while, Twitter and PayPal have been in that category.So there's like this clap, you know, there are the known hyperscalers, right. You know, the, the Google, Amazon, uh, Microsoft that are like cloud vendors in addition to consumers internally have their own, their own clouds. Um, and then there's a whole class of other, um, places that operate their own internal clouds in quotes.Uh, but don't offer them externally and again, uh, Facebook or Meta, uh, you know, is one example. eBay's another, you know, there's a, I'm making this up. Dropbox actually famously started in the cloud and then found it was much cheaper for them to operate their own infrastructure again, for the particular workloads that they had.Um, so yeah, there's probably, I'm making this up. Let's call it two dozen around the world of these, I'm making this term up many hyperscalers, right? Like self hyperscalers or something like that. And eBay's in that category.[00:23:11] Jeremy: I know this is kind of a, you know, a big what if, but you were saying how once you reach a certain scale, that's when it makes sense to move into your own data center. And, uh, I'm wondering if, if E-bay, had started more recently, like, let's say in the last, you know, 10 years, I wonder if it would've made sense for it to start on a public cloud and then move to, um, you know, its own infrastructure after it got bigger, or if you know, it really did make sense to just start with your own infrastructure from the start.[00:23:44] Randy: Oh, I'm so glad you asked that. Um, the, the answer is obvious, but like, I'm so glad you asked that because I love to make this point. No one should ever, ever start by building your own servers and your own (laughs) cloud. Like, No, there's be, uh, you should be so lucky (laughs) after years and years and years that you outgrow the cloud vendors.Right. Um, it happens, but it doesn't happen that often, you know, it happens so rarely that people write articles about it when it happens. Do you know what I mean? Like Dropbox is a good example. So yes, 100% anytime. Where are we? 2022. Any time in, more than the last 10 years? Um, yeah, let's call it. Let's call it 2010, 2012.Right. Um, when cloud had proved itself over and you know, many times over, um, anybody who starts since that time should absolutely start in the public cloud. There's no argument about it. Uh, and again, one should be so lucky that over time, you're seeing successive zeros added to your cloud bill, and it becomes so many zeros that it makes sense to shift your focus toward building and operating your own data centers.That's it. I haven't been part of that transition. I've been the other way, you know, at other places where, you know, I've migrated from owned data centers and colos into, into public cloud. Um, and that's the, that's the more common migration. And again, there are, there are a handful, maybe not even a handful of, uh, companies that have migrated away, but when they do, they've done all the math, right.I mean, uh, Dropbox has done some great, uh, talks and articles about, about their transition and boy, the math makes sense for them. So, yeah.[00:25:30] Jeremy: Yeah. And it also seems like maybe it's for certain types of businesses where moving off of public cloud. Makes sense. Like you mentioned Dropbox where so much of their business is probably centered around storage or centered around, you know, bandwidth and, you know, there's probably certain workloads that it's like need to leave public cloud earlier.[00:25:51] Randy: Um, yeah, I think that's fair. Um, I think that, I think that's a, I think that's an insightful comment. Again, it's all about the economics at some point, you know, it's a big investment to, uh, uh, and it takes years to develop the intern, forget the money that you're paying people, but like just to develop the internal capabilities.So they're very specialized skill sets around building an operating data centers. So like it's a big deal. Um, and, uh, yeah. So are there particular classes of workloads where you would for the same dollar figure or whatever, uh, migrate earlier or later? I'm sure that's probably true. And again, what can absolutely imagine?Well, when they say Dropbox in this example, um, yeah, it's because like they, they need to go direct to the storage. And then, I mean, like, they want to remove every middle person, you know, from the flow of the bytes that are coming into the storage media. Um, and it makes perfect sense for, for them. And when I understood what they were doing, which was a number of years ago, they were hybrid, right. So they had, they had completely, you know, they kept the top, you know, external layer, uh, in public cloud. And then the, the storage layer was all custom. I don't know what they do today, but people could check.[00:27:07] Jeremy: And I'm kind of coming back to your, your first time at eBay. is there anything you felt that you would've done differently with the knowledge you have now?but with the technology that existed, then.[00:27:25] Randy: Gosh, that's the 20, 20 hindsight. Um, the one that comes to mind is the one we touched on a little bit, but I'll say it more starkly, the. If I could, if I could go back in time 20 years and say, Hey, we're about to do this V3 transition at eBay. I would not. I would have had us move directly to what we would now call microservices in the sense that individual services own their own data storage and are only interacted with through the public interface.Um, there's a famous Amazon memo around that same time. So Amazon did the transition from a monolith into what we would now call microservices over about a four or five-year period, 2000 to 2005. And there was a famous Jeff Bezos memo from the early part of that, where, you know, seven, you know, requirements I can't remember them, but you know, essentially it was, you may, you may, you may never, you may never talk to anybody else's database. You may only interact with other services through their public interfaces. I don't care what those public interfaces are, so they didn't standardize around. You know, CORBA or JSON or GRPC, which didn't exist at the time, you know, like they didn't standardize around any, any particular, uh, interaction mechanism, but you did need to again, have this kind of microservice capability, that's modern terminology, um, uh, where, you know, the only services own their own data and nobody can talk in the back door.So that is the one architectural thing that I wish, you know, with 2020 hindsight, uh, that I would bring back in my time travel to 20 years ago, because that would help. That does help a lot. And to be fair, Amazon, um, Amazon was, um, pioneering in that approach and a lot of people internally and externally from Amazon, I'm told, didn't think it would work, uh, and it, and it did famously.So that's, that's the thing I would do.[00:29:30] Jeremy: Yeah. I'm glad you brought that up because, when you had mentioned that, I think you said there were 220 applications or something like that at certain scales, people might think like, oh, that sounds like microservices to me. But when you, you mentioned that microservice to you means it having its own data store.I think that's a good distinction.[00:29:52] Randy: Yeah. So, um, I talk a lot about microservices that have for, for a decade or so. Yeah. I mean, several of the distinguishing characteristics are the micro in microservices is size and scope of the interface, right? So you can have a service oriented architecture with one big service, um, or some very small number of very large services.But the micro in microservice means this thing does, maybe it doesn't have one operation, but it doesn't have a thousand. The several or the handful or several handfuls of operations are all about this one particular thing. So that's the one part of it. And then the other part of it that is critical to the success of that is owning the, owning your own data storage.Um, so each service, you know, again, uh, it's hard to do this with a diagram, but like imagine, imagine the bubble of the service surrounding the data storage, right? So like people, anybody from the outside, whether they're interacting synchronously, asynchronously, messaging, synchronous, whatever HTTP doesn't matter are only interacting to the bubble and never getting inside where the, uh, where the data is I hope that makes sense.[00:31:04] Jeremy: Yeah. I mean, I mean, it's a kind of in direct contrast to before you're talking about how you had all these databases that all of these services shared. So it was probably hard to kind of keep track of, um, who had modified data. Um, you know, one service could modify it, then another service control to get data out and it's been changed, but it didn't change it.So it could be kind of hard to track what's going on.[00:31:28] Randy: Yeah, exactly. Inner integration at the database level is something that people have been doing since probably the 1980s. Um, and so again, I, you know, in retrospect it looks like caveman approach. Uh, it was pretty advanced at the time, actually, even the idea of sharding of, you know, Hey, there are users and the users live in databases, but they don't all live in the same one.Uh, they live in 10 different databases or 20 different databases. And then there's this layer that. For this particular user, it figures out which of the 20 databases it's in and finds it and gets it back. And, um, you know, that was all pretty advanced. And by the way, that's all those capabilities still exist.They're just hidden from everybody behind, you know, nice, simple, uh, software as a service, uh, interfaces anyway, but that takes nothing away from your excellent point, which is, yeah. It's, you know, when you're, again, when there's many to many to relations, when there is this many to many relationship between, um, uh, applications and databases, uh, and there's shared mutable state in those databases that when is shared, like that's bad, you know, it's not bad to have state.It's not bad to have mutable state it's bad to have shared beautiful state.[00:32:41] Jeremy: Yeah. And I think anybody who's kind of interested in learning more about the, you had talked about sharding and things like that. If they go back and listen to your, your first appearance on software engineering radio, um, yeah. It kind of struck me how you were talking about sharding and how it was something that was kind of unique or unusual.Whereas today it feels like it's very, I don't know, if quaint is the right word, but it's like, um, it's something that, that people kind of are accustomed to now.[00:33:09] Randy: Yeah. Yeah. Um, it's obvious. Um, it seems obvious in retrospect. Yeah. You know, at the time, and by the way, he didn't invent charting. As I said, in 2007, you know, Google and Yahoo and, uh, Amazon, and, you know, it was the obvious, it took a while to reach it, but it's one of those things where once, once people have the, you know, brainwave to see, oh, you know what, we don't actually have to stop store this in one, uh, database.We can, we can chop that database up into, you know, into chunks. And that, that looks similar to that herself similar. Um, yeah, that was, uh, that was, uh, that was reinvented by lots of, uh, Lots of the big companies at the same time again, because everybody was solving that same problem at the same time. Um, but yeah, when you look back and you, I mean, like, and honestly, like everything that I said there, it's still like this, all the techniques about how you shard things.And there's lots of, you know, it's not interesting anymore because the problems have been solved, but all those solutions are still the solutions, if that makes any sense, but you know,[00:34:09] Jeremy: Yeah, for sure. I mean, I think anybody who goes back and listens to it. Yeah. Like you said, it's, it's, it's very interesting because it's. it all still applies and it's like, I think the, the solutions that are kind of interesting to me are ones where it's, it's things that could have been implemented long ago, but we just later on realized like, this is how we could do it.[00:34:31] Randy: Well part of it is, as we grow as an industry, we just, we discover new problems. You know, we, we get to the point where, you know, sharding over databases has only a problem when one database doesn't work. You know, when it, when you're the load that you put on that database is too big, or you want the availability of, you know, multiple.Um, and so that's not a, that's not a day one problem, right? That's a day two or day 2000 and kind of problem. Right. Um, and so a lot of these things, yeah, well, you know, it's software. So like we could have done, we could have done any of these things in older languages and older operating systems and with older technology.But for the most part, we didn't have those problems or we didn't have them at sufficiently enough. People didn't have the problem that we, you know, um, for us to have solved it as an industry, if that makes any sense.[00:35:30] Jeremy: yeah, no, that's a good point because you think about when Amazon first started and it was just a bookstore, right. And the number of people using the site where, uh, who knows it was, it might've been tens a day or hundreds a day. I don't, I don't know. And, and so, like you said, the problems that Amazon has now in terms of scale are just like, it's a completely different world than when they started.[00:35:52] Randy: Yeah. I mean, probably I'm making it up, but I don't think that's too off to say that it's a billion times more, their problems are a billion fold. You know, what they, what they were[00:36:05] Jeremy: the next thing I'd like to talk about is you came back to eBay I think about has it been about two years ago.[00:36:14] Randy: Two years yeah.[00:36:15] Jeremy: Yeah. And, and so, so tell me about the experience of coming back to an organization that you had been at, you know, 10 years prior or however long it was like, how is your onboarding different when it's somewhere you've been before?[00:36:31] Randy: Yeah. Sure. So, um, like, like you said, I worked at eBay from 2004 to 2011. Um, and I worked in a different role than I have today. I've worked mostly on eBay search engine. Um, and then, uh, I left to co-found a startup, which was in the 99%. So the one, you know, like didn't really do much. Uh, I joined, I worked at Google in the early days of Google cloud, as I mentioned on Google app engine and had a bunch of other roles including more recently, like you said, stitch fix and we work, um, leading those engineering teams.And, um, so yeah, coming back to eBay as chief architect and, and, you know, leading. Developer platform, essentially a part of eBay. Um, yeah. What was the onboarding like? I mean, lots of things had changed, you know, in the, in the intervening 10 years or so. Uh, and lots had stayed the same, you know, not in a bad way, but just, you know, uh, some of the technologies that we use today are still some of the technologies we used 10 years ago, a lot has changed though.Um, a bunch of the people are still around. So there's something about eBay that, um, people tend to stay a long time. You know, it's not really very strange for people to be at eBay for 20 years. Um, in my particular team of let's call it 150, there are four or five people that have crossed their 20 year anniversary at the company.Um, and I also re I rejoined with a bunch of other boomerangs as the term we use internally. So it's, you know, the, um, including the CEO, by the way. So sort of bringing the band back together, a bunch of people that had gone off and worked at it, but at other places have, have come back for various reasons over the last couple of.So it was both a lot of familiarity, a lot of unfamiliarity, a lot of familiar faces. Um, yup.[00:38:17] Jeremy: So, I mean, having these people who you work with still be there and actually coming back with some of those people, um, what were some of the big, I guess, advantages or benefits you got from, you know, those existing connections?[00:38:33] Randy: Yeah. Well, I mean, as with all things, you know, imagine, I mean, everybody can imagine like getting back together with friends that they had from high school or university, or like you had some people had some schooling at some point and like you get back together with those friends and there's this, you know, there's this implicit trust in most situations of, you know, because you went through a bunch of stuff together and you knew each other, uh, you know, a long time.And so that definitely helps, you know, when you're returning to a place where again, there are a lot of familiar faces where there's a lot of trust built up. Um, and then it's also helpful, you know, eBay's a pretty complicated place and it's 10 years ago, it was too big to hold in any one person's head and it's even harder to hold it in one person said now, but to be able to come back and have a little bit of that, well, more than a little bit of that context about, okay, here's how eBay works.And here, you know, here are the, you know, unique complexities of the marketplace cause it's very unique, you know, um, uh, in the world. Um, and so, yeah, no, I mean, it was helpful. It's helpful a lot. And then also, you know, in my current role, um, uh, my, my main goal actually is to just make all of eBay better, you know, so we have about 4,000 engineers and, you know, my team's job is to make all of them better and more productive and more successful and, uh, being able to combine.Knowing what eBay, knowing the context about eBay and having a bunch of connections to the people that, you know, a bunch of the leaders there, uh, here, um, combining that with 10 years of experience doing other things at other places, you know, that's helpful because you know, now there are things that we do at eBay that, okay, well there, you know, you know, that this other place is doing, this has that same problem and is solving it in a different way.And so maybe we should, you know, look into that option. So,[00:40:19] Jeremy: so, so you mentioned just trying to make developers, work or lives easier. you start the job. How do you decide what to tackle first? Like how do you figure out where the problems are or what to do next?[00:40:32] Randy: yeah, that's a great question. Um, so, uh, again, my, uh, I lead this thing that we internally called the velocity initiative, which is about just making us, giving us the ability to deliver. Features and bug fixes more quickly to customers. Right. And, um, so what do I figure for that problem? How can we deliver things more quickly to customers and improve, you know, get more customer value and business value?Uh, what I did, uh, with, in collaboration with a bunch of people is what one would call a value stream map. And that's a term from lean software and lean manufacturing, where you just look end to end at a process and like say all the steps and how long those steps take. So a value stream, as you can imagine, like all these steps that are happening at the end, there's some value, right?Like we produced some, you know, feature or, you know, hopefully gotten some revenue or like helped out the customer and the business in some way. And so value, you know, mapping that value stream. That's what it means. And, um, Looking for you look at that. And when you can see the end-to-end process, you know, and like really see it in some kind of diagram, uh, you can look for opportunities like, oh, okay, well, you know, if it takes us, I'm making this effort, it takes us a week from when we have an idea to when it shows up on the site.Well, you know, some of those steps take five minutes. That's not worth optimizing, but some of those steps take, you know, five days and that is worth optimizing. And so, um, getting some visibility into the system, you know, looking end to end with some, with a kind of view of the system systems thinking, uh, that will give you the, uh, the knowledge about, or the opportunities about we know what can be improved.And so that's, that's what we did. And we didn't talk with all 4,000, you know, uh, engineers are all, you know, whatever, half a thousand teams or whatever we had. Um, but we sampled. And after we talked with three teams who were already hearing a bunch of the same things, you know, so we were hearing in the whole product life cycle, which I like to divide into four stages.I'd like to say, there's planning. How does an idea become a project or a thing that people work on a software development? How does a project or become committed code software delivery? How does committed code become a feature that people actually use? And then what I call post release iteration, which is okay, it's now there are out there on the site and we're turning it on and off for individual users.We're learning in analytics and usage in the real world and, and experimenting. And so there were opportunities that eBay at all, four of those stages, um, which I'm happy to talk about, but what we ended up seeing again and again, uh, is that that software delivery part was our current bottleneck. So again, that's the, how long does it take from an engineer when she commits her code to, it shows up as a feature on the site.And, you know, before we started the work. You know, two years ago before we started the work that I've been doing for the last two years with a bunch of people, um, on average and eBay was like a week and a half. So, you know, it'd be a week and a half between when someone's finished and then, okay. It gets code reviewed and, you know, dot, dot, dot it gets rolled out.It gets tested, you know, all that stuff. Um, it was, you know, essentially 10 days. And now for the teams that we've been working with, uh, it's down to two. So we used a lot of, um, what people may be familiar with, uh, the accelerate book. So it's called accelerate by Nicole Forsgren. Um, Jez humble and Gene Kim, uh, 2018, like if there's one book anybody should read about software engineering, it's that?Uh, so please read accelerate. Um, it summarizes almost a decade of research from the state of DevOps reports, um, which the three people that I mentioned led. So Nicole Forsgren, you know, is, uh, is a doctor, uh, you know, she's a PhD and, uh, data science. She knows how to do all this stuff. Um, anyway, so, uh, that when your, when your problem happens to be software delivery.The accelerate book tells you all the kind of continuous delivery techniques, trunk based development, uh, all sorts of stuff that you can do to, to solve that, uh, solve those problems. And then there are also four metrics that they use to measure the effectiveness of an organization, software delivery. So people might be familiar with, uh, there's deployment frequency.How often are we deploying a particular application lead time for change? That's that time from when a developer commits her code to when it shows up on the site, uh, change failure rate, which is when we deploy code, how often do we roll it back or hot fix it, or, you know, there's some problem that we need to, you know, address.Um, and then, uh, meantime to re uh, meantime to restore, which is when we have one of those incidents or problems, how, how quickly can we, uh, roll it back or do that hot fix? Um, and again, the beauty of Nicole Forsgren research summarized in the accelerate book is that the science shows that companies cluster, in other words, Mostly the organizations that are not good at, you know, deployment frequency and lead time are also not good at the quality metrics of, uh, meantime to restore and change failure rate and the companies that are excellent at, you know, uh, deployment frequency and lead time are also excellent at meantime, to recover and, uh, change failure rate.Um, so companies or organizations, uh, divided into these four categories. So there's a low performers, medium performers, high performers, and then elite performers. And, uh, eBay was solidly eBay on average at the time. And still on average is solidly in that medium performer category. So, uh, and what we've been able to do with the teams that we've been working with is we've been able to move those teams to the high category.So just super brief. Uh, and I w we'll give you a chance to ask you some more questions, but like in the low category, all those things are kind of measured in months, right. So how long, how often are we deploying, you know, measure that in months? How long does it take us to get a commit to the site? You know, measure that in months, you know, um, where, and then the low performer, sorry.Uh, the medium performers are like, everything's measured in weeks, right? So like, if we were deploy, you know, couple, you know, once every couple of weeks or once a week, uh, lead time is measured in weeks, et cetera. The, uh, the high-performers things are measured in days and the elite performance things are measured in hours.And so you can see there's like order of magnitude improvements when you go from, you know, when you move from one of those kind of clusters to another cluster. Anyway. So what we were focused on again, because our problem was software delivery was moving a whole, a whole set of teams from that medium performer category where things are measured in weeks to the, uh, high-performer category, where things are missing.[00:47:21] Jeremy: throughout all this, you said the, the big thing that you focused on was the delivery time. So somebody wrote code and, they felt that it was ready for deployment, but for some reason it took 10 days to actually get out to the actual site. So I wonder if you could talk a little bit about, uh, maybe a specific team or a specific application, where, where, where was that time being spent?You know, you, you said you moved from 10 days to two days. What, what was happening in the meantime?[00:47:49] Randy: Yeah, no, that's a great question. Thank you. Um, yeah, so, uh, okay, so now, so we, we, we looked end to end of the process and we found that software delivery was the first place to focus, and then there are other issues in other areas, but we'll get to them later. Um, so then for, um, to improve software delivery, now we asked individual teams, we, we, we did something like, um, you know, some conversation like I'm about to say, so we said, hi, it looks like you're deploying kind of once or twice.If I, if I told you, you had to deploy once a day, tell me all the reasons why that's not going to work. And the teams are like, oh, of course, well, it's a build times take too long. And the deployments aren't automated and you know, our testing is flaky. So we have to retry it all the time and, you know, dot, dot, dot, dot, dot.And we said, great, you just gave my team, our backlog. Right. So rather than, you know, just coming and like, let's complain about it. Um, which the teams work it's legit for them to complain. Uh, I was a, you know, we were able, because again, the developer program or sorry, the developer platform, you know, is as part of my team, uh, we said, great, like you just gave us, you just told us all the, all your top, uh, issues or your impediments, as we say, um, and we're going to work on them with you.And so every time we had some idea about, well, I bet we can use Canary deployments to automate the deployment which we have now done. We would pilot that with a bunch of teams, we'd learn what works and doesn't work. And then we would roll that out to everybody. Um, So what were the impediments like? It was a little bit different for each individual team, but in some, it was, uh, the things we ended up focusing on or have been focusing on our build times, you know, so we build everything in Java still.Um, and, uh, even though we're generation five, as opposed to that generation three that I mentioned, um, still build times for a lot of applications we're taking way too long. And so we, we spend a bunch of time improving those things and we were able to take stuff from, you know, hours down to, you know, single digit minutes.So that's a huge improvement to developer productivity. Um, we made a lot of investment in our continuous delivery pipelines. Um, so making all the, making all the automation around, you know, deploying something to one environment and checking it there and then deploying it into a common staging environment and checking it there and then deploying it from there into the production environment.And, um, and then, you know, rolling it out via this Canary mechanism. We invested a lot in something that we call traffic mirroring, which is a, we didn't invent. Other T other places have a different name for this? I don't know if there's a standard industry name. Some people call it shadowing, but the idea is I have a change that I'm making, which is not intended to change the behavior.Like a lots of changes that we make, bug fixes, et cetera, uh, upgrading to new, you know, open source, dependencies, whatever, changing the version of the framework. There's a bunch of changes that we make regularly day-to-day as developers, which are like, refactorings kind of where we're not actually intending to change the behavior.And so a tra traffic mirroring was our idea of. You have the old code that's running in production and you, and you fire a request, a production request at that old code and it responds, but then you also fire that request at the new version and compare the results, you know, did the same, Jason come back, you know, between the old version and the new version.Um, and that's, that's a great way kind of from the outside to sort of black box detect any unintended changes in the, in the behavior. And so we definitely leveraged that very, very aggressively. Um, we've invested in a bunch of other bunch of other things, but, but all those investments are driven by what does the team, what do the particular teams tell us are getting in their way?And there are a bunch of things that the teams themselves have, you know, been motivated to do. So my team's not the only one that's making improvements. You know, teams have. Reoriented, uh, moved, moved from branching development to trunk based development, which makes a big difference. Um, making sure that, uh, PR approvals and like, um, you know, code reviews are happening much more regularly.So like right after, you know, a thing that some teams have started doing is like immediately after standup in the morning, everybody does all the code reviews that you know, are waiting. And so things don't drag on for, you know, two, three days, cause whatever. Um, so there's just like a, you know, everybody kind of works on that much more quickly.Um, teams are building their own automations for things like testing site speed and accessibility and all sorts of stuff. So like all the, all the things that, you know, a team goes through in the development and roll out of their software, they were been spending a lot of time automating and making, making a leaner, making more efficient.[00:52:22] Jeremy: So, so some of those, it sounds like the PR example is really, on the team. Like you're, you're telling them like, Hey, this is something that you internally should change how you work. for things like improving the build time and things like that. Did you have like a separate team that was helping these teams, you know, speed that process up? Or what, what was that [00:52:46] Randy: like?Yeah. Great. I mean, and you did give to those two examples are, are like you say, very different. So I'm going to start from, we just simply showed everybody. Here's your deployment frequency for this application? Here's your lead time for this application? Here's your change failure rate. And here's your meantime to restore.And again, as I didn't mention before. All of the state of DevOps research and the accelerate book prove that by improving those metrics, you get better engineering outcomes and you also get better business outcomes. So like it's scientifically proven that improving those four things matters. Okay. So now we've shown to teams, Hey, you're we would like you to improve, you know, for your own good, but you know, more broadly at eBay, we would like the deployment frequency to be faster.And we would like the lead time to be shorter. And the insight there is when we deploy smaller units of work, when we don't like batch up a week's worth of work, a month's worth of work, uh, it's much, much less risky to just deploy like an hour's worth of work. Right. And the, and the insight is the hours worth of work fits in your head.And if you roll it out and there's an issue. First off rolling backs, no big deal. Cause you only, you know, not, you've only lost an hour of work for a temporary period of time, but also like you never have this thing, like what in the world broke? Cause like with a month's worth of work, there's a lot of things that changed and a lot of stuff that could break, but with an hour's worth of work, it's only like one change that you made.So, you know, when, if something happens, like it's pretty much, pretty much guaranteed to be that thing anyway, that's the back. Uh, that's the backstory. And um, and so yeah, we were just working with individual teams. Oh yeah. So they were, the teams were motivated to like, see what's the biggest bang for the buck in order to improve those things.Like how can we improve those things? And again, some teams were saying, well, you know what, a huge component of our, of that lead time between when somebody commits and it's, it's a feature on the site, a huge percentage of that. Maybe multiple days, it's like waiting for somebody to code review. Okay, great.We can just change our team kind of agreements and our team behavior to make that happen. And then yes, to answer your question about. Were the other things like building the Canary capability and traffic mirroring and build time improvements. Those were done by central, uh, platform and infrastructure teams, you know, some of which were in my group and some of which are in peer peer groups, uh, in, in my part of the organization.So, yeah, so I mean like providing the generic tools and, you know, generic capabilities, those are absolutely things that a platform organization does. Like that's our job. Um, and you know, we did it. And, uh, and then there are a bunch of other things like that around kind of team behavior and how you approach building a particular application that are, are, and should be completely in the control of the individual teams.And we were trying not to be, not trying not to be, we were definitely not being super prescriptive. Like we didn't come in and we say, we didn't come in and say, alright, by next, by next Tuesday, we want you to be doing trunk based development by, you know, the Tuesday after that, we want to see test-driven development, you know, dot, dot, Um, we would just offer to teams, you know, hear it.Here's where you are. Here's where we know you can get, because like we work with other teams and we've seen that they can get there. Um, you know, they just work together on, well, what's the biggest bang for the buck and what would be most helpful for that team? So it's like a menu of options and you don't have to take everything off the menu, if that makes sense.[00:56:10] Jeremy: And, and how did that communication flow from you and your team down to the individual contributor? Like you have, I'm assuming you have engineering managers and technical leads and all these people sort of in the chain. How does it[00:56:24] Randy: Yeah, thanks for asking that. Yeah. I didn't really say how we work as an initiative. So every, um, so there are a bunch of teams that are involved. Um, and we have, uh, every Monday morning, so, uh, just so happens. It's late Monday morning today. So we already did this a couple of hours ago, but once a week we get all the teams that are involved, both like the platform kind of provider teams and also the product.Or we would say domain like consumer teams. And we do a quick scrum of scrums, like a big old kind of stand up. What have you all done this week? What are you working on next week? What are you blocked by kind of idea. And, you know, there are probably 20 or 30 teams again, across the individual platform capabilities and across the teams that, you know, uh, consume this stuff and everybody gives a quick update and they, and, uh, it's a great opportunity for people to say, oh, I have that same problem too.Maybe we should offline try to figure out how to solve that together. You built a tool that automates the site speed stuff. That's great. I would S I would so love to have that. And, um, so it, uh, this weekly meeting has been a great opportunity for us to share wins, share, um, you know, help that people need and then get, uh, get teams to help with each other.And also, similarly, one of the platform teams would say something like, Hey, we're about to be done or beta, let's say, you know, this new Canary capability, I'm making this up. Anybody wanna pilot that for us? And then you get a bunch of hands raised of yo, we would be very happy to pilot that that would be great.Um, so that's how we communicate back and forth. And, you know, it's a big enough. It's kind of like engineering managers are kind of are the kind of level that are involved in that typically. Um, so it's not individual developers, but it's like somebody on most, every team, if that makes any sense. So like, that's kind of how we do that, that like communication, uh, back to the individual developers.If that makes sense.[00:58:26] Jeremy: Yeah. So it sounds like you would have, like you said, the engineering manager go to the standup and um, you said maybe 20 to 30 teams, or like, I'm just trying to get a picture for how many people are in this meeting.[00:58:39] Randy: Yeah. It's like 30 or 40 people.[00:58:41] Jeremy: Okay. Yeah.[00:58:42] Randy: And again, it's quick, right? It's an hour. So we just go, boom, boom, boom, boom. And we've just developed a cadence of people. We have a shared Google doc and like people like write their little summaries, you know, of what they're, what they've worked on and what they're working on.So we've over time made it so that it's pretty efficient with people's time. And. Pretty dense in a good way of like information flow, back and forth. Um, and then also separately, we meet more in more detail with the individual teams that are involved. Again, try to elicit, okay, now, where are you now?Here's where you are. Please let us know what problems you're seeing with this part of the infrastructure or problems you're seeing in the pipelines or something like that. And we're, you know, we're constantly trying to learn and get better and, you know, solicit feedback from teams on what we can do differently.[00:59:29] Jeremy: earlier you had talked a little bit about how there were a few services that got brought over from V2 or V3, basically kind of more legacy or older services that are, have been a part of eBay for quite some time.And I was wondering if there were things about those services that made this process different, like, you know, in terms of how often you could deploy or, um, just what were some key differences between something that was made recently versus something that has been with the company for a long time?[01:00:06] Randy: Yeah, sure. I mean, the stuff that's been with the company for a long time was best in class. As of when we built it, you know, maybe 15 and sometimes 20 years ago. Um, there actually, I wouldn't even less than a handful. There are, as we speak, there are two or three of those V3. Uh, clusters or applications or services still around and they should be gone in a completely migrated away from, in the next a couple of months.So like, we're almost at the end of, um, you know, uh, moving all to more modern things. But yeah, you know, I mean, again, uh, stuff that was state-of-the-art, you know, 20 years ago, which was like deploying things once every two weeks, like that was a big deal in 2000 or 2004. Uh, and it's, you know, like that was fast in 2004 and is slow in 2022.So, um, yeah, I mean, what's the difference? Um, yeah, I mean, a lot of these things, you know, if they haven't already been migrated, there's a reason. And it's because often that they're way in the guts of something that's really important. You know, this is the, this is a core part. I'm making these examples up and they're not even right, but like it's a core part of the payments flow.It's a core part of, you know, uh, how, uh, sellers get paid. And those aren't examples. We have, those are modern, but you see what I'm saying? Like stuff that's like really core to the business and that's why it's kind of lasted.[01:01:34] Jeremy: And, uh, I'm kind of curious from the perspective of some of these new things you're introducing, like you're talking about, um, improving continuous delivery and things like that. Uh, when you're working with some of these services that have been around a long time, are the teams the rate at which they deploy or the rate at which you find defects is that noticeably different from services that are more recent?[01:02:04] Randy: I mean, and that's true of any legacy at any, at any place. Right? So, um, yeah, I mean, people are legitimately, uh, I have some trepidation that say about, you know, changing something that's

About ChrisChris Short has been a proponent of open source solutions throughout his over two decades in various IT disciplines, including systems, security, networks, DevOps management, and cloud native advocacy across the public and private sectors. He currently works on the Kubernetes team at Amazon Web Services and is an active Kubernetes contributor and Co-chair of OpenGitOps. Chris is a disabled US Air Force veteran living with his wife and son in Greater Metro Detroit. Chris writes about Cloud Native, DevOps, and other topics at ChrisShort.net. He also runs the Cloud Native, DevOps, GitOps, Open Source, industry news, and culture focused newsletter DevOps'ish.Links Referenced: DevOps'ish: https://devopsish.com/ EKS News: https://eks.news/ Containers from the Couch: https://containersfromthecouch.com opengitops.dev: https://opengitops.dev ChrisShort.net: https://chrisshort.net Twitter: https://twitter.com/ChrisShort TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Coming back to us since episode two—it's always nice to go back and see the where are they now type of approach—I am joined by Senior Developer Advocate at AWS Chris Short. Chris, been a few years. How has it been?Chris: Ha. Corey, we have talked outside of the podcast. But it's been good. For those that have been listening, I think when we recorded I wasn't even—like, when was season two, what year was that? [laugh].Corey: Episode two was first pre-pandemic and the rest. I believe—Chris: Oh. So, yeah. I was at Red Hat, maybe, when I—yeah.Corey: Yeah. You were doing Red Hat stuff, back when you got to work on open-source stuff, as opposed to now, where you're not within 1000 miles of that stuff, right?Chris: Actually well, no. So, to be clear, I'm on the EKS team, the Kubernetes team here at AWS. So, when I joined AWS in October, they were like, “Hey, you do open-source stuff. We like that. Do more.” And I was like, “Oh, wait, do more?” And they were like, “Yes, do more.” “Okay.”So, since joining AWS, I've probably done more open-source work than the three years at Red Hat that I did. So, that's kind of—you know, like, it's an interesting point when I talk to people about it because the first couple months are, like—you know, my friends are like, “So, are you liking it? Are you enjoying it? What's going on?” And—Corey: Do they beat you with reeds? Like, all the questions people have about companies? Because—Chris: Right. Like, I get a lot of random questions about Amazon and AWS that I don't know the answer to.Corey: Oh, when I started telling people, I fixed Amazon bills, I had to quickly pivot that to AWS bills because people started asking me, “Well, can you save me money on underpants?” It's I—Chris: Yeah.Corey: How do you—fine. Get the prime credit card. It docks 5% off the bill, so there you go. But other than that, no, I can't.Chris: No.Corey: It's—Chris: Like, I had to call my bank this morning about a transaction that I didn't recognize, and it was from Amazon. And I was like, that's weird. Why would that—Corey: Money just flows one direction, and that's the wrong direction from my employer.Chris: Yeah. Like, what is going on here? It shouldn't have been on that card kind of thing. And I had to explain to the person on the phone that I do work at Amazon but under the Web Services team. And he was like, “Oh, so you're in IT?”And I'm like, “No.” [laugh]. “It's actually this big company. That—it's a cloud company.” And they're like, “Oh, okay, okay. Yeah. The cloud. Got it.” [laugh]. So, it's interesting talking to people about, “I work at Amazon.” “Oh, my son works at Amazon distribution center,” blah, blah, blah. It's like, cool. “I know about that, but very little. I do this.”Corey: Your son works in Amazon distribution center. Is he a robot? Is normally my next question on that? Yeah. That's neither here nor there.So, you and I started talking a while back. We both write newsletters that go to a somewhat similar audience. You write DevOps'ish. I write Last Week in AWS. And recently, you also have started EKS News because, yeah, the one thing I look at when I'm doing these newsletters every week is, you know what I want to do? That's right. Write more newsletters.Chris: [laugh].Corey: So, you are just a glutton for punishment? And, yeah, welcome to the addiction, I suppose. How's it been going for you?Chris: It's actually been pretty interesting, right? Like, we haven't pushed it very hard. We're now starting to include it in things. Like we did Container Day; we made sure that EKS news was on the landing page for Container Day at KubeCon EU. And you know, it's kind of just grown organically since then.But it was one of those things where it's like, internally—this happened at Red Hat, right—when I started live streaming at Red Hat, the ultimate goal was to do our product management—like, here's what's new in the next version thing—do those live so anybody can see that at any point in time anywhere on Earth, the second it's available. Similar situation to here. This newsletter actually is generated as part of a report my boss puts together to brief our other DAs—or developer advocates—you know, our solutions architects, the whole nine yards about new EKS features. So, I was like, why can't we just flip that into a weekly newsletter, you know? Like, I can pull from the same sources you can.And what's interesting is, he only does the meeting bi-weekly. So, there's some weeks where it's just all me doing it and he ends up just kind of copying and pasting the newsletter into his document, [laugh] and then adds on for the week. But that report meeting for that team is now getting disseminated to essentially anyone that subscribes to eks.news. Just go to the site, there's a subscribe thing right there. And we've gotten 20 issues in and it's gotten rave reviews, right?Corey: I have been a subscriber for a while. I will say that it has less Chris Short personality—Chris: Mm-hm.Corey: —to it than DevOps'ish does, which I have to assume is by design. A lot of The Duckbill Group's marketing these days is no longer in my voice, rather intentionally, because it turns out that being a sarcastic jackass and doing half-billion dollar AWS contracts can not to be the most congruent thing in the world. So okay, we're slowly ameliorating that. It's professional voice versus snarky voice.Chris: Well, and here's the thing, right? Like, I realized this year with DevOps'ish that, like, if I want to take a week off, I have to do, like, what you did when your child was born. You hired folks to like, do the newsletter for you, or I actually don't do the newsletter, right? It's binary: hire someone else to do it, or don't do it. So, the way I structured this newsletter was that any developer advocate on my team could jump in and take over the newsletter so that, you know, if I'm off that week, or whatever may be happening, I, Chris Short, am not the voice. It is now the entire developer advocate team.Corey: I will challenge you on that a bit. Because it's not Chris Short voice, that's for sure, but it's also not official AWS brand voice either.Chris: No.Corey: It is clearly written by a human being who is used to communicating with the audience for whom it is written. And that is no small thing. Normally, when oh, there's a corporate newsletter; that's just a lot of words to say it's bad. This one is good. I want to be very clear on that.Chris: Yeah, I mean, we have just, like, DevOps'ish, we have sections, just like your newsletter, there's certain sections, so any new, what's new announcements, those go in automatically. So, like, that can get delivered to your inbox every Friday. Same thing with new blog posts about anything containers related to EKS, those will be in there, then Containers from the Couch, our streaming platform, essentially, for all things Kubernetes. Those videos go in.And then there's some ecosystem news as well that I collect and put in the newsletter to give people a broader sense of what's going on out there in Kubernetes-land because let's face it, there's upstream and then there's downstream, and sometimes those aren't in sync, and that's normal. That's how Kubernetes kind of works sometimes. If you're running upstream Kubernetes, you are awesome. I appreciate you, but I feel like that would cause more problems and it's worse sometimes.Corey: Thank you for being the trailblazers. The rest of us can learn from your misfortune.Chris: [laugh]. Yeah, exactly. Right? Like, please file your bugs accordingly. [laugh].Corey: EKS is interesting to me because I don't see a lot of it, which is, probably, going to get a whole lot of, “Wait, what?” Moments because wait, don't you deal with very large AWS bills? And I do. But what I mean by that is that EKS, until you're using its Fargate expression, charges for the control plane, which rounds to no money, and the rest is running on EC2 instances running in a company's account. From the billing perspective, there is no difference between, “We're running massive fleets of EKS nodes.” And, “We're managing a whole bunch of EC2 instances by hand.”And that feels like an interesting allegory for how Kubernetes winds up expressing itself to cloud providers. Because from a billing perspective, it just looks like one big single-tenant application that has some really strange behaviors internally. It gets very chatty across AZs when there's no reason to, and whatnot. And it becomes a very interesting study in how to expose aspects of what's going on inside of those containers and inside of the Kubernetes environment to the cloud provider in a way that becomes actionable. There are no good answers for this yet, but it's something I've been seeing a lot of. Like, “Oh, I thought you'd be running Kubernetes. Oh, wait, you are and I just keep forgetting what I'm looking at sometimes.”Chris: So, that's an interesting point. The billing is kind of like, yeah, it's just compute, right? So—Corey: And my insight into AWS and the way I start thinking about it is always from a billing perspective. That's great. It's because that means the more expensive the services, the more I know about it. It's like, “IAM. What is that?” Like, “Oh, I have no idea. It's free. How important could it be?” Professional advice: do not take that philosophy, ever.Chris: [laugh]. No. Ever. No.Corey: Security: it matters. Oh, my God. It's like you're all stars. Your IAM policy should not be. I digress.Chris: Right. Yeah. Anyways, so two points I want to make real quick on that is, one, we've recently released an open-source project called Carpenter, which is really cool in my purview because it looks at your Kubernetes file and says, “Oh, you want this to run on ARM instance.” And you can even go so far as to say, right, here's my limits, and it'll find an instance that fits those limits and add that to your cluster automatically. Run your pod on that compute as long as it needs to run and then if it's done, it'll downsize—eventually, kind of thing—your cluster.So, you can basically just throw a bunch of workloads at it, and it'll auto-detect what kind of compute you will need and then provision it for you, run it, and then be done. So, that is one-way folks are probably starting to save money running EKS is to adopt Carpenter as your autoscaler as opposed to the inbuilt Kubernetes autoscaler. Because this is instance-aware, essentially, so it can say, like, “Oh, your massive ARM application can run here,” because you know, thank you, Graviton. We have those processors in-house. And you know, you can run your ARM64 instances, you can run all the Intel workloads you want, and it'll right size the compute for your workloads.And I'll look at one container or all your containers, however you want to configure it. Secondly, the good folks over at Kubecost have opencost, which is the open-source version of Kubecost, basically. So, they have a service that you can run in your clusters that will help you say, “Hey, maybe this one notes too heavy; maybe this one notes too light,” and you know, give you some insights into Kubernetes spend that are a little bit more granular as far as usage and things like that go. So, those two projects right there, I feel like, will give folks an optimal savings experience when it comes to Kubernetes. But to your point, it's just compute, right? And that's really how we treat it, kind of, here internally is that it's a way to run… compute, Kubernetes, or ECS, or any of those tools.Corey: A fairly expensive one because ignoring entirely for a second the actual raw cost of compute, you also have the other side of it, which is in every environment, unless you are doing something very strange or pre-funding as a one-person startup in your spare time, your payroll costs will it—should—exceed your AWS bill by a fairly healthy amount. And engineering time is always more expensive than services time. So, for example, looking at EKS, I would absolutely recommend people use that rather than rolling their own because—Chris: Rolling their own? Yeah.Corey: —get out of that engineering space where your time is free. I assure you from a business context, it is not. So, there's always that question of what you can do to make things easier for people and do more of the heavy lifting.Chris: Yeah, and to your rather cheeky point that there's 17 ways to run a container on AWS, it is answering that question, right? Like those 17 ways, like, how much of this do you want to run yourself, you could run EKS distro on EC2 instances if you want full control over your environment.Corey: And then run IoT Greengrass core on top within that cluster—Chris: Right.Corey: So, I can run my own Lambda function runtime, so I'm not locked in. Also, DynamoDB local so I'm not locked into AWS. At which point I have gone so far around the bend, no one can help me.Chris: Well—Corey: Pro tip, don't do that. Just don't do that.Chris: But to your point, we have all these options for compute, and specifically containers because there's a lot of people that want to granularly say, “This is where my engineering team gets involved. Everything else you handle.” If I want EKS on Spot Instances only, you can do that. If you want EKS to use Carpenter and say only run ARM workloads, you can do that. If you want to say Fargate and not have anything to manage other than the container file, you can do that.It's how much does your team want to manage? That's the customer obsession part of AWS coming through when it comes to containers is because there's so many different ways to run those workloads, but there's so many different ways to make sure that your team is right-sized, based off the services you're using.Corey: I do want to change gears a bit here because you are mostly known for a couple of things: the DevOps'ish newsletter because that is the oldest and longest thing you've been doing the time that I've known you; EKS, obviously. But when prepping for this show, I discovered you are now co-chair of the OpenGitOps project.Chris: Yes.Corey: So, I have heard of GitOps in the context of, “Oh, it's just basically your CI/CD stuff is triggered by Git events and whatnot.” And I'm sitting here going, “Okay, so from where you're sitting, the two best user interfaces in the world that you have discovered are YAML and Git.” And I just have to start with the question, “Who hurt you?”Chris: [laugh]. Yeah, I share your sentiment when it comes to Git. Not so much with YAML, but I think it's because I'm so used to it. Maybe it's Stockholm Syndrome, maybe the whole YAML thing. I don't know.Corey: Well, it's no XML. We'll put it that way.Chris: Thankfully, yes because if it was, I would have way more, like, just template files laying around to build things. But the—Corey: And rage. Don't forget rage.Chris: And rage, yeah. So, GitOps is a little bit more than just Git in IaC—infrastructure as Code. It's more like Justin Garrison, who's also on my team, he calls it infrastructure software because there's four main principles to GitOps, and if you go to opengitops.dev, you can see them. It's version one.So, we put them on the website, right there on the page. You have to have a declared state and that state has to live somewhere. Now, it's called GitOps because Git is probably the most full-featured thing to put your state in, but you could use an S3 bucket and just version it, for example. And make it private so no one else can get to it.Corey: Or you could use local files: copy-of-copy-of-this-thing-restored-parentheses-use-this-one-dot-final-dot-doc-dot-zip. You know, my preferred naming convention.Chris: Ah, yeah. Wow. Okay. [laugh]. Yeah.Corey: Everything I touch is terrifying.Chris: Yes. Geez, I'm sorry. So first, it's declarative. You declare your state. You store it somewhere. It's versioned and immutable, like I said. And then pulled automatically—don't focus so much on pull—but basically, software agents are applying the desired state from source. So, what does that mean? When it's—you know, the fourth principle is implemented, continuously reconciled. That means those software agents that are checking your desired state are actually putting it back into the desired state if it's out of whack, right? So—Corey: You're talking about agents running it persistently on instances, validating—Chris: Yes.Corey: —a checkpoint on a cron. How is this meaningfully different than a Puppet agent running in years past? Having spent I learned to speak publicly by being a traveling trainer for Puppet; same type of model, and in fact, when I was at Pinterest, we wound up having a fair bit—like, that was their entire model, where they would have—the Puppet's code would live in an S3 bucket that was then copied down, I believe, via Git, and then applied to the instance on a schedule. Like, that sounds like this was sort of a early days GitOps.Chris: Yeah, exactly. Right? Like so it's, I like to think of that as a component of GitOps, right? DevOps, when you talk about DevOps in general, there's a lot of stuff out there. There's a lot of things labeled DevOps that maybe are, or maybe aren't sticking to some of those DevOps core things that make you great.Like the stuff that Nicole Forsgren writes about in books, you know? Accelerate is on my desk for a reason because there's things that good, well-managed DevOps practices do. I see GitOps as an actual implementation of DevOps in an open-source manner because all the tooling for GitOps these days is open-source and it all started as open-source. Now, you can get, like, Flux or Argo—Argo, specifically—there's managed services out there for it, you can have Flux and not maintain it, through an add-on, on EKS for example, and it will reconcile that state for you automatically. And the other thing I like to say about GitOps, specifically, is that it moves at the speed of the Kubernetes Audit Log.If you've ever looked at a Kubernetes audit log, you know it's rather noisy with all these groups and versions and kinds getting thrown out there. So, GitOps will say, “Oh, there's an event for said thing that I'm supposed to be watching. Do I need to change anything? Yes or no? Yes? Okay, go.”And the change gets applied, or, “Hey, there's a new Git thing. Pull it in. A change has happened inGit I need to update it.” You can set it to reconcile on events on time. It's like a cron or it's like an event-driven architecture, but it's combined.Corey: How does it survive the stake through the heart of configuration management? Because before I was doing all this, I wasn't even a T-shaped engineer: you're broad across a bunch of things, but deep in one or two areas, and one of mine was configuration management. I wrote part of SaltStack, once upon a time—Chris: Oh.Corey: —due to a bunch of very strange coincidences all hitting it once, like, I taught people how to use Puppet. But containers ultimately arose and the idea of immutable infrastructure became a thing. And these days when we were doing full-on serverless, well, great, I just wind up deploying a new code bundle to the Lambdas function that I wind up caring about, and that is a immutable version replacement. There is no drift because there is no way to log in and change those things other than through a clear deployment of this as the new version that goes out there. Where does GitOps fit into that imagined pattern?Chris: So, configuration management becomes part of your approval process, right? So, you now are generating an audit log, essentially, of all changes to your system through the approval process that you set up as part of your, how you get things into source and then promote that out to production. That's kind of the beauty of it, right? Like, that's why we suggest using Git because it has functions, like, requests and issues and things like that you can say, “Hey, yes, I approve this,” or, “Hey, no, I don't approve that. We need changes.” So, that's kind of natively happening with Git and, you know, GitLab, GitHub, whatever implementation of Git. There's always, kind of—Corey: Uh, JIF-ub is, I believe, the pronunciation.Chris: JIF-ub? Oh.Corey: Yeah. That's what I'm—Chris: Today, I learned. Okay.Corey: Exactly. And that's one of the things that I do for my lasttweetinaws.com Twitter client that I build—because I needed it, and if other people want to use it, that's great—that is now deployed to 20 different AWS commercial regions, simultaneously. And that is done via—because it turns out that that's a very long to execute for loop if you start down that path—Chris: Well, yeah.Corey: I wound up building out a GitHub Actions matrix—sorry a JIF-ub—actions matrix job that winds up instantiating 20 parallel builds of the CDK deploy that goes out to each region as expected. And because that gets really expensive with native GitHub Actions runners for, like, 36 cents per deploy, and I don't know how to test my own code, so every time I have a typo, that's another quarter in the jar. Cool, but that was annoying for me so I built my own custom runner system that uses Lambda functions as runners running containers pulled from ECR that, oh, it just runs in parallel, less than three minutes. Every time I commit something between I press the push button and it is out and running in the wild across all regions. Which is awesome and also terrifying because, as previously mentioned, I don't know how to test my code.Chris: Yeah. So, you don't know what you're deploying to 20 regions sometime, right?Corey: But it also means I have a pristine, re-composable build environment because I can—Chris: Right.Corey: Just automatically have that go out and the fact that I am making a—either merging a pull request or doing a direct push because I consider main to be my feature branch as whenever something hits that, all the automation kicks off. That was something that I found to be transformative as far as a way of thinking about this because I was very tired of having to tweak my local laptop environment to, “Oh, you didn't assume the proper role and everything failed again and you broke it. Good job.” It wound up being something where I could start developing on more and more disparate platforms. And it finally is what got me away from my old development model of everything I build is on an EC2 instance, and that means that my editor of choice was Vim. I use the VS Code now for these things, and I'm pretty happy with it.Chris: Yeah. So, you know, I'm glad you brought up CDK. CDK gives you a lot of the capabilities to implement GitOps in a way that you could say, like, “Hey, use CDK to declare I need four Amazon EKS clusters with this size, shape, and configuration. Go.” Or even further, connect to these EKS clusters to RDS instances and load balancers and everything else.But you put that state into Git and then you have something that deploys that automatically upon changes. That is infrastructure as code. Now, when you say, “Okay, main is your feature branch,” you know, things happen on main, if this were running in Kubernetes across a fleet of clusters or the globe-wide in 20 regions, something like Flux or Argo would kick in and say, “There's been a change to source, main, and we need to roll this out.” And it'll start applying those changes. Now, what do you get with GitOps that you don't get with your configuration?I mean, can you rollback if you ever have, like, a bad commit that's just awful? I mean, that's really part of the process with GitOps is to make sure that you can, A, roll back to the previous good state, B, roll forward to a known good state, or C, promote that state up through various environments. And then having that all done declaratively, automatically, and immutably, and versioned with an audit log, that I think is the real power of GitOps in the sense that, like, oh, so-and-so approve this change to security policy XYZ on this date at this time. And that to an auditor, you just hand them a log file on, like, “Here's everything we've ever done to our system. Done.” Right?Like, you could get to that state, if you want to, which I think is kind of the idea of DevOps, which says, “Take all these disparate tools and processes and procedures and culture changes”—culture being the hardest part to adopt in DevOps; GitOps kind of forces a culture change where, like, you can't do a CAB with GitOps. Like, those two things don't fly. You don't have a configuration management database unless you absolutely—Corey: Oh, you CAB now but they're all the comments of the pull request.Chris: Right. Exactly. Like, don't push this change out until Thursday after this other thing has happened, kind of thing. Yeah, like, that all happens in GitHub. But it's very democratizing in the sense that people don't have to waste time in an hour-long meeting to get their five minutes in, right?Corey: DoorDash had a problem. As their cloud-native environment scaled and developers delivered new features, their monitoring system kept breaking down. In an organization where data is used to make better decisions about technology and about the business, losing observability means the entire company loses their competitive edge. With Chronosphere, DoorDash is no longer losing visibility into their applications suite. The key? Chronosphere is an open-source compatible, scalable, and reliable observability solution that gives the observability lead at DoorDash business, confidence, and peace of mind. Read the full success story at snark.cloud/chronosphere. That's snark.cloud slash C-H-R-O-N-O-S-P-H-E-R-E.Corey: So, would it be overwhelmingly cynical to suggest that GitOps is the means to implement what we've all been pretending to have implemented for the last decade when giving talks at conferences?Chris: Ehh, I wouldn't go that far. I would say that GitOps is an excellent way to implement the things you've been talking about at all these conferences for all these years. But keep in mind, the technology has changed a lot in the, what 11, 12 years of the existence of DevOps, now. I mean, we've gone from, let's try to manage whole servers immutably to, “Oh, now we just need to maintain an orchestration platform and run containers.” That whole compute interface, you go from SSH to a Docker file, that's a big leap, right?Like, you don't have bespoke sysadmins; you have, like, a platform team. You don't have DevOps engineers; they're part of that platform team, or DevOps teams, right? Like, which was kind of antithetical to the whole idea of DevOps to have a DevOps team. You know, everybody's kind of in the same boat now, where we see skill sets kind of changing. And GitOps and Kubernetes-land is, like, a platform team that manages the cluster, and its state, and health and, you know, production essentially.And then you have your developers deploying what they want to deploy in when whatever namespace they've been given access to and whatever rights they have. So, now you have the potential for one set of people—the platform team—to use one set of GitOps tooling, and your applications teams might not like that, and that's fine. They can have their own namespaces with their own tooling in it. Like, Argo, for example, is preferred by a lot of developers because it has a nice UI with green and red dots and they can show people and it looks nice, Flux, it's command line based. And there are some projects out there that kind of take the UI of Argo and try to run Flux underneath that, and those are cool kind of projects, I think, in my mind, but in general, right, I think GitOps gives you the choice that we missed somewhat in DevOps implementations of the past because it was, “Oh, we need to go get cloud.” “Well, you can only use this cloud.” “Oh, we need to go get this thing.” “Well, you can only use this thing in-house.”And you know, there's a lot of restrictions sometimes placed on what you can use in your environment. Well, if your environment is Kubernetes, how do you restrict what you can run, right? Like you can't have an easily configured say, no open-source policy if you're running Kubernetes. [laugh] so it becomes, you know—Corey: Well, that doesn't stop some companies from trying.Chris: Yeah, that's true. But the idea of, like, enabling your developers to deploy at will and then promote their changes as they see fit is really the dream of DevOps, right? Like, same with production and platform teams, right? I want to push my changes out to a larger system that is across the globe. How do I do that? How do I manage that? How do I make sure everything's consistent?GitOps gives you those ways, with Kubernetes native things like customizations, to make consistent environments that are robust and actually going to be reconciled automatically if someone breaks the glass and says, “Oh, I need to run this container immediately.” Well, that's going to create problems because it's deviated from state and it's just that one region, so we'll put it back into state.Corey: It'll be dueling banjos, at some point. You'll try and doing something manually, it gets reverted automatically. I love that pattern. You'll get bored before the computer does, always.Chris: Yeah. And GitOps is very new, right? When you think about the lifetime of GitOps, I think it was coined in, like, 2018. So, it's only four years old, right? When—Corey: I prefer it to ChatOps, at least, as far as—Chris: Well, I mean—Corey: —implementation and expression of the thing.Chris: —ChatOps was a way to do DevOps. I think GitOps—Corey: Well, ChatOps is also a way to wind up giving whoever gets access to your Slack workspace root in production.Chris: Mmm.Corey: But that's neither here nor there.Chris: Mm-hm.Corey: It's yeah, we all like to pretend that's not a giant security issue in our industry, but that's a topic for another time.Chris: Yeah. And that's why, like, GitOps also depends upon you having good security, you know, and good authorization and approval processes. It enforces that upon—Corey: Yeah, who doesn't have one of those?Chris: Yeah. If it's a sole operation kind of deal, like in your setup, your case, I think you kind of got it doing right, right? Like, as far as GitOps goes—Corey: Oh, to be clear, we are 11 people and we do have dueling pull requests and all the rest.Chris: Right, right, right.Corey: But most of the stuff I talk about publicly is not our production stuff, so it really is just me. Just as a point of clarity there. I've n—the 11 people here do not all—the rest of you don't just sit there and clap as I do all the work.Chris: Right.Corey: Most days.Chris: No, I'm sure they don't. I'm almost certain they don't clap… for you. I mean, they would—Corey: No. No, they try and talk me out of it in almost every case.Chris: Yeah, exactly. So, the setup that you, Corey Quinn, have implemented to deploy these 20 regions is kind of very GitOps-y, in the sense that when main changes, it gets updated. Where it's not GitOps-y is what if the endpoint changes? Does it get reconciled? That's the piece you're probably missing is that continuous reconciliation component, where it's constantly checking and saying, “This thing out there is deployed in the way I want it. You know, the way I declared it to be in my source of truth.”Corey: Yeah, when you start having other people getting involved, there can—yeah, that's where regressions enter. And it's like, “Well, I know where things are so why would I change the endpoint?” Yeah, it turns out, not everyone has the state of the entire application in their head. Ideally it should live in—Chris: Yeah. Right. And, you know—Corey: —you know, Git or S3.Chris: —when I—yeah, exactly. When I think about interactions of the past coming out as a new DevOps engineer to work with developers, it's always been, will developers have access to prod or they don't? And if you're in that environment with—you're trying to run a multi-billion dollar operation, and your devs have direct—or one Dev has direct access to prod because prod is in his brain, that's where it's like, well, now wait a minute. Prod doesn't have to be only in your brain. You can put that in the codebase and now we know what is in your brain, right?Like, you can almost do—if you document your code, well, you can have your full lifecycle right there in one place, including documentation, which I think is the best part, too. So, you know, it encourages approval processes and automation over this one person has an entire state of the system in their head; they have to go in and fix it. And what if they're not on call, or in Jamaica, or on a cruise ship somewhere kind of thing? Things get difficult. Like, for example, I just got back from vacation. We were so far off the grid, we had satellite internet. And let me tell you, it was hard to write an email newsletter where I usually open 50 to 100 tabs.Corey: There's a little bit of internet out Californ-ie way.Chris: [laugh].Corey: Yeah it's… it's always weird going from, like, especially after pandemic; I have gigabit symmetric here and going even to re:Invent where I'm trying to upload a bunch of video and whatnot.Chris: Yeah. Oh wow.Corey: And the conference WiFi was doing its thing, and well, Verizon 5G was there but spotty. And well, yeah. Usual stuff.Chris: Yeah. It's amazing to me how connectivity has become so ubiquitous.Corey: To the point where when it's not there anymore, it's what do I do with myself? Same story about people pushing back against remote development of, “Oh, I'm just going to do it all on my laptop because what happens if I'm on a plane?” It's, yeah, the year before the pandemic, I flew 140,000 miles domestically and I was almost never hamstrung by my ability to do work. And my only local computer is an iPad for those things. So, it turns out that is less of a real world concern for most folks.Chris: Yeah I actually ordered the components to upgrade an old Nook that I have here and turn it into my, like, this is my remote code server, that's going to be all attached to GitHub and everything else. That's where I want to be: have Tailscale and just VPN into this box.Corey: Tailscale is transformative.Chris: Yes. Tailscale will change your life. That's just my personal opinion.Corey: Yep.Chris: That's not an AWS opinion or anything. But yeah, when you start thinking about your network as it could be anywhere, that's where Tailscale, like, really shines. So—Corey: Tailscale makes the internet work like we all wanted to believe that it worked.Chris: Yeah. And Wireguard is an excellent open-source project. And Tailscale consumes that and puts an amazingly easy-to-use UI, and troubleshooting tools, and routing, and all kinds of forwarding capabilities, and makes it kind of easy, which is really, really, really kind of awesome. And Tailscale and Kubernetes—Corey: Yeah, ‘network' and ‘easy' don't belong in the same sentence, but in this case, they do.Chris: Yeah. And trust me, the Kubernetes story in Tailscale, there is a lot of there. I understand you might want to not open ports in your VPC, maybe, but if you use Tailscale, that node is just another thing on your network. You can connect to that and see what's going on. Your management cluster is just another thing on the network where you can watch the state.But it's all—you're connected to it continuously through Tailscale. Or, you know, it's a much lighter weight, kind of meshy VPN, I would say, if I had to sum it up in one sentence. That was not on our agenda to talk about at all. Anyways. [laugh]Corey: No, no. I love how many different topics we talk about on these things. We'll have to have you back soon to talk again. I really want to thank you for being so generous with your time. If people want to learn more about what you're up to and how you view these things, where can they find you?Chris: Go to ChrisShort.net. So, Chris Short—I'm six-four so remember, it's Short—dot net, and you will find all the places that I write, you can go to devopsish.com to subscribe to my newsletter, which goes out every week. This year. Next year, there'll be breaks. And then finally, if you want to follow me on Twitter, Chris Short: at @ChrisShort on Twitter. All one word so you see two s's. Like, it's okay, there's two s's there.Corey: Links to all of that will of course be in the show notes. It's easier for people to do the clicky-clicky thing as a general rule.Chris: Clicky things are easier than the wordy things, yes.Corey: Says the Kubernetes guy.Chris: Yeah. Says the Kubernetes guy. Yeah, you like that, huh? Like I said, Argo gives you a UI. [laugh].Corey: Thank you [laugh] so much for your time. I really do appreciate it.Chris: Thank you. This has been fun. If folks have questions, feel free to reach out. Like, I am not one of those people that hides behind a screen all day and doesn't respond. I will respond to you eventually.Corey: I'm right here, Chris. Come on, come on. You're calling me out in front of myself. My God.Chris: Egh. It might take a day or two, but I will respond. I promise.Corey: Thanks again for your time. This has been Chris Short, senior developer advocate at AWS. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice and if it's YouTube, click the thumbs-up button. Whereas if you've hated this podcast, same thing, smash the buttons five-star review and leave an insulting comment that is written in syntactically correct YAML because it's just so easy to do.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Today, we decided we'd have a chat about the 8 principles or tenets for a high performance serverless first team. 1. 'Chase a business outcome or a KPI' A team should know what business KPI they're working towards. You should be able to tap a person on the shoulder and have them tell you what they're working on. And what business impact the work they're doing is going to have. If the team says 'I don't know', then you run a Northstar workshop. After the Northstar workshop if nobody can think of a KPI then the next step is to ask if the team should be doing this work. It's not that they are a bad team. They are being asked to do the wrong stuff. 2. 'Be secure by design' Don't do security afterwards. Bake it in from the start. It's everyone's job. It's such a difficult thing to retrofit. Use threat models and get it done early. Try to solve for what you can and what you know. Bake it into all your engineering practices. And bake it into your pipelines. Shift it all left and help to enable teams to be more secure. Security has a risk profile, if you don't do it right. And it can be an existential risk for the businesses if you don't have a secure solution. 3. 'Keep a high throughput of work' That is borrowed from the DORA metrics in the 'Accelerate' book by Nicole Forsgren. This principle looks at high throughput, which is deployment frequency and lead time. For serverless teams, it is key to make changes fast and frequent. And always be learning and driving observability. As Charity Majors says, "speed is stability". The more frequently you do something, the more you deploy to production. You're actually improving your stability. You smooth out the pathways and the error conditions. And you bake it into your pipelines. Which means that you automate a lot of the stuff that could go wrong. 4. 'Reliably run, high stability system' That's the other two DORA metrics of throughput and stability. A lot of discussions with test teams, QA and software engineers drive the need for investment in world class quality and testing capabilities/practices. If you're not stable, where's the gap? What scenarios and behaviours have you not covered? It drives the right evolution. When you look at three and four, you can't achieve any of them if you've got things in the middle. Or handoffs, or if you are dumping things over walls. It's about promoting ownership. To get elite scores, you need to know what you're doing and embrace that approach. 5. 'Rent or reuse with build as a final option' How do you do that? Serverless! With Serverless and SaaS and our background you're used to going straight to the workspace. And with the FORESEE diagram, we find out what we are doing and it is coding. It's a mindset thing. It's back to knowing your business purpose. And then knowing your business KPIs. If you can achieve business outcomes without doing code you are at your most optimal. If you can leverage a SaaS offering that does what you need, that's probably the next thing. And finally you need to build following a serverless first mindset and approach. Using all the serviceful offerings and managed services. 6. 'Continuously optimise the total cost' This is the best question to ask any team. Good teams will tell you how much their cloud costs are. But loads of teams have no idea. This is a great measure of a good team. They have a cost in mind. A good team will tell you the run cost. And a great team will tell you the total cost. But really good teams will get into a worst case development conversation about how much features cost. And how much revenue they're bringing in. In other words, how impactful they are to the business. 7. 'Build event driven via strong API's' This sounds very easy. But from talking to Sam Dengler, nobody is doing this properly. We've been talking about this for 20 years. Proper integration is still a mystery to most people. It is about making sure you've got the right things in the right places. But also at the right size. And having things that are composable. It's about breaking things up into their smallest constituent parts. And change things as frequently as possible. I find that this one takes a lot of evolution and yields through different levels of complexity. And it takes time. You should always be thinking about it. 8. 'Build solutions that fit in their heads'. This principle is borrowed from Dan North. In other words, don't build crazy systems that are too complicated. This has a nice nod towards Team Topologies and setting proper boundaries. We've seen teams become victims to crazy architectures. Where there's too much to fit in your head and the cognitive load breaks people. When we are getting teams going my manta is 'just enough design'. Some teams want to design everything up front and go into huge amounts of detail. But it is better to keep your world small. Focus on what you're doing today. Serverless Craic from The Serverless Edge theserverlessedge.com @ServerlessEdge

In this episode, I talk with Michael Stahnke, VP Platform at CircleCI about the “four core metrics”, and what's next. Michael was Director of Engineering at Puppet back when Dr. Nicole Forsgren, Jez Humble, and Gene Kim created the State of DevOps report which provided a rigorous basis for focusing on four key metrics for tracking the maturity of an engineering org. When taken together, Deployment Frequency, Lead Time for Changes, Mean Time to Recover, and Change Failure Rate provide a great starting point for engineering teams looking to improve their efficiency. But no metrics are perfect. In our chat, Michael shares how (and why) he defines DevOps for different audiences, how he thinks about each of the metrics, things to bear in mind when thinking about the survey responses, and the variability in developer experience even between companies that perform well on all four metrics. And he raises the question, “once you're performing well on the DORA Metrics, what do you prioritize improving next?”PARTNERThanks to our partner CloudZero — Cloud Cost Intelligence Platform. Control cost and drive better decisions with CloudZero cloud cost intelligence. The CloudZero platform provides visibility into cloud spend without the typical pitfalls of legacy cloud cost management tools, like endless tagging or clunky Kubernetes support. Optimize unit economics, decentralize cost data to engineering, and create a shared language between finance and technical teams. CloudZero helps you organize cloud spending better than anyone else.Join companies like Drift, Rapid7, and SeatGeek by visiting cloudzero.com/ctoconnection to get started.

Materiały dodatkowe..Prezentacje:Dissecting Bounded Contexts, prezentacja Nicka Tune z konferencji DDD Europe 2020Context Maps - a deep dive, prezentacja Michaela Plöd z konferencji KanDDDinsky 2019Książki:Accelerate: Building and Scaling High-Performing Technology Organizations, Nicole Forsgren,Jez Humble, Gene KimThe DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations, Gene Kim, Jez Humble, Patrick Debois, John WillisEscaping the Build Trap: How Effective Product Management Creates Real Value, Melissa PerriInspired: How to Create Tech Products Customers Love, Marty CaganEmpowered: Ordinary People, Extraordinary Products, Marty Cagan, Chris JonesThe Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win, Gene Kim, Kevin Behr, George SpaffordStrategic Microservices and Monoliths, Vaughn Vernon, Tomasz JaskułaLearning Domain-Driven Design: Aligning Software Architecture and Business Strategy, Vladik Khononov

Sorti en 2018, le livre Accelerate (écrit par Nicole Forsgren, Jez Humble et Gene Kim) est un peu la nouvelle référence littéraire dans le milieu technologique. L'ouvrage reprend et développe les conclusions de 4 années de sondage pour démontrer (chiffres à l'appui) que la performance des équipes de développement logiciel est déterminée par une multiplicité de facteurs techniques, mais aussi produits, managériaux et culturels.L'un d'entre eux, emblématique, est la faculté à livrer en production le plus souvent possible, car contrairement à la croyance populaire, la stabilité du produit n'en souffre pas, bien au contraire ! Pourtant, aujourd'hui, la mise en production est bien souvent une étape redoutée par les équipes et  se fait trop souvent dans la douleur...Alors, accélérer la livraison logicielle, est-ce vraiment possible ? Faire mieux, plus rapidement, comment m'y mettre ? Et est-ce que ça fait mal ? Plus globalement, qu'est-ce que proposent les auteurs d'Accelerate et peut-on leur faire confiance ? Au micro de Laure Constantinesco, Maria Mokbel, consultante chez OCTO, vous explique tout dans ce quatrième épisode de OCTO Buzzword Bingo !Ressources :Livres :Accelerate: The Science Behind Devops: Building and Scaling High Performing Technology Organizations, Nicole Forsgren, Jez Humble, Gene Kim, IT Revolution Press, 2017.The Goal,  Eliyahu M. GoldrattContinuous Delivery, Jez Humble et David FarleyThe principles of Product Development Flow,  Donald G. ReinertsenToyota Kata,  Mike RotherL' Étude Accelerate sur le site de Google : Accelerate State Of DevOps Report La Matinale OCTO consacrée à Accelerate, un talk à voir ou revoir sur notre blogLa formation dispensée par OCTO Academy, pour adopter les bonnes pratiques et être plus performant en delivery Crédits :Un podcast proposé par OCTO Technology, écrit, réalisé et enregistré par LACOLAB (Laure Constantinesco et Charlotte Abdelnour). Description du podcast : Natalie SchmitzSound Design et mixage : Paul Love (The Clean Sound). Musique originale : AudioJungle / Ocean_Studio.

What you'll learn: Focusing on predictability rather than velocity How the code base's complexity can affect estimation Balancing the effect of technology change on the team David references the following book: Nicole Forsgren's book: Accelerate Meet: David Kaplan is the head of engineering at Policygenius, the nation's leading online insurance marketplace. David has been an engineering leader for over a decade as a VP of Engineering at Yodle (Acquired by Web.com), a Senior Manager at Bloomberg L.P., and the co-founder of Helioscene, which was a SaaS startup in the video production space. If you have any questions for David, please feel free to reach out via LinkedIn: https://www.linkedin.com/in/davebkaplan/ I hope you enjoyed the episode, the best place to connect with me is on Linkedin - https://www.linkedin.com/in/amirbormand (Amir Bormand). Please send me a message if you would like me to cover certain topics with future guests.

SpringOne Platform is in just one week! Richard and Coté talk about the conference, some of their highlights, and some quick food recommendations (burgers!). Perhaps too quick! We also cover some recent news in .Net, serverless, and Azure security. Richard discusses a DevOps Report webinar he did with Dr. Nicole Forsgren, and Coté gives an overview of the new book he's working on (The Business Bottleneck) and preview of a two part webinar on the topic (part one, part two). There's still time to register for SpringOne Platform. It's Oct 7th to 10th in Austin, Texas - Coté's home town! Use the code S1P200_Cote to get $200 off.

Matt & Nell Shamrell-Harrington of the Food Fight Show chat with Nicole Forsgren (DORA).

Food Fight teamed up with Arrested DevOps to host a live DevOps call in show where Dr. Nicole Forsgren answered audience questions about devops, metrics, and more!

Bridget and Matt chat with Nicole Forsgren (DORA) and Tim Gross (Joyent).

Where is your DevOps at? Carl and Richard talk to Jez Humble and Nicole Forsgren about DORA, that is the DevOps Readiness Assessment. DORA helps you understand where your organization is at in the spectrum of DevOps, from low to medium to high. The conversation digs into what it takes to improve operational capability, focusing on understanding exactly how your organization delivers software so you can improve it. Along the way there are challenges, it is never easy to change an organization, but that's what it takes to actually improve software delivery. The goods news is that DevOps can work in any kind of organization: Big or small, startup or heavily regulated industry, it doesn't matter!Support this podcast at — https://redcircle.com/net-rocks/donations

Bridget chats about startups with Charity Majors (Honeycomb) and Nicole Forsgren (DORA)

Bridget chats about enterprise transformation and the democratizing effect of platforms with guests Charity Majors, Nicole Forsgren, Andrew Clay Shafer, and James Watters, in front of a live studio audience at devopsdays Minneapolis 2016.

Chef's Dr. Nicole Forsgren has a frank talk with Matt about the often neglected portions of CAMS theory - Measurement and Sharing. She gives real-world, practical tips on how to use data to drive a transformation...and even how culture can be measured.