Podcasts about S3

  • 1,156PODCASTS
  • 4,073EPISODES
  • 39mAVG DURATION
  • 1DAILY NEW EPISODE
  • Sep 21, 2023LATEST

POPULARITY

20162017201820192020202120222023

Categories



Best podcasts about S3

Show all podcasts related to s3

Latest podcast episodes about S3

Creeps & Crimes
156. Creepy Accounts Vol. 28

Creeps & Crimes

Play Episode Listen Later Sep 21, 2023 75:57


Last Creepy Account Ep for Season 3, Besties!!!! Next week, for our final ep of S3, we have a two parter with some of our new best friends!! So make sure you are looking on Insta to find out where to get part 1 on Tuesday!!Today we read the Creepy Accounts of: Brittany, Andrea, "Dot", Sophia, Claire, Hilary, Kate, and Amanda!!! Thank you all for sending in your stories for us to share and trusting us to share them!! If you have a Creepy Account of your own, you can submit it through the portal on our website, CREEPSANDCRIMES.COM, or email it to us at CREEPSANDCRIMES.CA@GMAIL.COMIf you need to CALL SUSAN of Angel Wings and Healing Things, the official psychic angelic medium of c&c, call or text her (704-562-3476) to schedule your appointment and tell her Morgan and Taylar sent you for a special discount code for only our listeners!! Love you besties! See you next weeeeekkkk!!This episode may contain paid endorsements and advertisements for products and services. Individuals on the show may have a direct, or indirect financial interest in products, or services referred to in this episode.Visit FactorMeals.com/creepsandcrimes50 and use code creepsandcrimes50 to get 50% off your first box.Who Killed Mia is for all game lovers and detectives. A new kind of mystery game from Relatable. Get 20% off with promo code CREEPSANDCRIMES at Relatable.com/whokilledmia Take your FREE in-depth hair consultation and get 15% off your first order today! Go to Prose.com/creepsandcrimesNew customers get $5 off a Lume Starter Pack with code Creepsandcrimes at Lumedeodorant.comProduced by Dear MediaSee Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

Starcourt Study Hall: A Stranger Things Podcast

Marina and Amanda reluctantly close out Season 3. We check in on our characters, revisit some themes, and go over some of our favorite post-S3 predictions. Stay Strange. --- Support this podcast: https://podcasters.spotify.com/pod/show/starcourtstudyhall/support

It's a Podcast, Charlie Brown
101: THE SNOOPY SHOW S3, Ep 4 - A CELEBRATION OF SNAIL MAIL, APPARENTLY

It's a Podcast, Charlie Brown

Play Episode Listen Later Sep 20, 2023 80:02


In 101, we check out "The Snoopy Show", S3, episode 4 - which is apparently a celebration of all things snail mail. Rememer snail mail? Kids, go ask your parents. The plot of each story in the episode hinges on at least one letter sent through regular ol' US Mail. Coincidence? Yeah, probably. Or maybe not...hmmm. We've also got a "Random Strip of the Month", plus your feedback on peanut-y things and news from the world of Peanuts! Good grief, that's a lot! Thanks to Kevin McLeod at Incompetech.com for creative commons use of his songs "Mining by Moonlight", "Bass Walker" and "Hidden Agenda".  Thanks to Nick Jones for the use of "25% Off". Thanks to Henry Pope for the use of "Linus & Lucy Remix". Support It's a Podcast, Charlie Brown on patreon.com  Find out more at carnivalofgleecreations.com   

Dishing Drama with Dana Wilkey UNCENSORED
Ep 148 RHOSLC & the Utah Paranormal: UAP, Area 52 & a Real Life Trip to Skinwalker Ranch P1 (feat. Angie Harrington)

Dishing Drama with Dana Wilkey UNCENSORED

Play Episode Listen Later Sep 17, 2023 23:41


Angie Harrington, RHOSLC Cast Member Alumni S2 & S3 joins me for a two part podcast.  This one will totally surprise you!  If you love Indian curses, UAPs, Interdimensional beings and secret military bases, you won't want to miss today and next weeks show. Angie and I will of course discuss her RHOSLC experience and gossip a little but more importantly we take on THE  STRANGE of UTAH!  You see, Angie Harrington and I became friends and we learned we not only have Bravo in common, but we also love mysteries, UAP and paranormal.  It was during one of our discussions about this that I discovered her trip to Skin Walker Ranch. Angie and Chris are friends with the owner, Brandon Fugal.  She shared with me her and Chris's experiences and impressions on the Ranch with Nine Inch Nails and now she will share them with you.  I tell Angie all about the new government UAP taskforce  AARO, they just announced their mission and I share what this new UAP oriented government organization has dropped. They are part of the Defense Department. BTW, the head Scientist at Skinwalker Ranch was actually on the old UAP Task force committee, Travis Taylor, so yes it all ties together.  This episode will take you on a worm whole to Bravo and and back as we take you all the strange of Utah in this 2 part series you won't want to miss!Sound Credit: Skinwalker Ranch Season 3 trailer , Head Like a HoleSupport the showDana is on Cameo!Get Dishing Drama Dana Merch!https://represent.com/store/dishing-drama-dana-wilkeyFollow Dana: @Wilkey_Dana$25,000 Song - Apple Music$25,000 Song - SpotifyTo support the show and listen to full episodes, become a member on PatreonTo learn more about sponsorships, email DDDWpodcast@gmail.comDana's YouTube Channel

North Meets South Web Podcast
Dads in Dev, cloud services, and static-site generation

North Meets South Web Podcast

Play Episode Listen Later Sep 13, 2023 40:46


In this episode, Jake and Michael discuss using Airdrop to simplify and speed up your asset compilation and application deployment, restricting access to S3 buckets based on username, and some of the complexities around building your frontend.This episode is brought to you by our friends at Workvivo - The leading employee communication app.Show links Laracon AU tickets on sale again Bun Baozi Once Ian Landsman on on-prem softwareDeploying Tighten's Jigsaw-powered sites with GitHub Actions

Chile Today Podcast
S3 episode 19: 50 years since the coup

Chile Today Podcast

Play Episode Listen Later Sep 11, 2023 99:58


S3 episode 19: 50 years since the coupOn our special episode, Bethany guides us through the events related to the coup d'état on September 11th, exactly 50 years ago when Chile lost its democracy. Lenny also shares news of recent events in Chile.Support the show

Confidence Mastery: Unlock Your Life Podcast
Confidence, Business, and Life: Insights With Gary Goldsmith

Confidence Mastery: Unlock Your Life Podcast

Play Episode Listen Later Sep 10, 2023 53:58


In this episode, I had the pleasure of interviewing Gary Goldsmith, a remarkable entrepreneur and recruitment expert. As someone who shares a passion for football and Chelsea, I was intrigued to learn about his incredible journey in the recruitment industry. From his humble beginnings to the creation of a multi-million pound business, Gary's story is truly inspiring.Overall, this episode has left me with valuable insights into achieving success, finding happiness, and making the most out of life. It's a must-listen for anyone looking to master their confidence and excel in their journey.KEY TAKEAWAYS Gary Goldsmith has had a successful career in recruitment, with over 40 years of experience. He emphasises the importance of work-life balance and finding happiness in what you do. Gary believes in setting goals and deadlines, celebrating successes along the way, and constantly challenging oneself to improve. He advocates for outsourcing tasks and surrounding oneself with a strong support team, including a personal assistant. Gary highlights the importance of building relationships, making phone calls, and being confident in sales conversations. He also emphasises the value of giving and sharing knowledge to establish oneself as a subject matter expert. BEST MOMENTS"I think I'm probably best known for being around the recruitment now for nearly 40 years, man and boy.""I'm a big believer in goals and deadlines and celebrating the success you have along the way.""If you're fearful and scared to pick up the phone, anybody who runs a good business is a salesperson.""If you don't ask, you don't get. What's the point of the phone call?" - Gary Goldsmith"I'm a massive advocate of being in the sun by the sea. I moved to Mallorca in 2011."ABOUT THE GUESTGary Goldsmith is a renowned figure in the recruitment industry, with nearly 40 years of experience under his belt. Known for his vibrant personality and sense of humour, Gary has built a successful career in recruitment, leading S3 to incredible growth during his tenure. Beyond his professional achievements, Gary is also a podcast host, author, and entrepreneur. He is passionate about helping recruitment CEOs maximise their shareholder value and achieve their goals.ABOUT THE HOSTNatalie Bailey, a Property Developer, Coach, and Mentor, boasts a decade of business acumen, from Bars to Gyms and eCommerce. Now partnered with her mother, Paula, in property development, she empowers others to find confidence and success in health, wealth, and happiness. Her Better Together initiative combats loneliness, aiding entrepreneurs through the Confident Entrepreneurs Club, Mastermind groups, and Retreats. Bridging Mallorca and London, Natalie embodies her teachings. Fitness, wealth, and happiness intertwine in her holistic approach. Dive deeper at www.nataliearabella.com for coaching plans and more info. CONTACT METHODFACEBOOK- https://www.facebook.com/nataliearabellabaileyLinkedIn- https://www.linkedin.com/in/nataliegoldstarbep/Instagram- https://www.instagram.com/nataliearabellabailey/Clubhouse- https://www.clubhouse.com/@nataliearabellaTikTok- https://www.tiktok.com/@nataliearabellabThis show was brought to you by Progressive Media

Born of Wonder
S6:5 EP81: Live like an Oxford Student: Michaelmas, Books, and New School Year Energy

Born of Wonder

Play Episode Listen Later Sep 5, 2023 23:58


It's September and this time of year means 'back to school' - whether you're a student or not, we can all benefit from the fresh start a new academic year can offer. Bring on the hot coffee, boots, books, course lists, blank notebooks, new pens... On this episode Katie looks to Oxford student life for a model on how to live like a student no matter your age or stage in life. We'll also discuss some of the charming unique traditions for Michaelmas (St. Michael's Day). Lots of good advice, ideas, and inspiration for capturing that new school year energy! ------ Subscribe to Born of Wonder on Substack https://bornofwonder.substack.com     Support Born of Wonder on Patreon https://www.patreon.com/bornofwonder    www.bornofwonder.com  ------- S3:14 EP39: Oxford Reminiscence With Chris Marquette https://podcasts.apple.com/us/podcast/s3-14-ep39-oxford-reminiscence-with-chris-marquette/id1521925425?i=1000558713612    Well Read Mom https://wellreadmom.com/    Dying Well to Live Well: Explorations from Tolstoy to Tolkien, á Kempis to Bellarmine https://collegiuminstitute.org/calendar/gcl-dying-well-event Michaelmas Traditions https://www.carrotsformichaelmas.com/2012/09/19/michaelmas-traditions-prayers-food-and-flowers/    Micron pens https://www.dickblick.com/products/sakura-pigma-micron-pen/?gclid=CjwKCAjwo9unBhBTEiwAipC11yJB-CEqLFpK4wbAPVHXMfzo2LVhler9nKQYZ8vVmWjt4QF3dhJI6RoCapIQAvD_BwE    "Gilmore girls but fall" https://open.spotify.com/playlist/0xPv1coTWtlsxuA5Z1SmbD?si=aJmDhZqdTdqSJKckfhbH0g  

Citizens of Pawnee
Ep. 87: S3E2 "Flu Season"

Citizens of Pawnee

Play Episode Listen Later Sep 5, 2023 50:12


On this special episode, I'm joined by my big brother, Rob, as we talk about the S3 fan-favorite episiode, "Flu Season! (12:35)" Rob is partially responsible for getting me into Parks and Rec (1:44) (so you can send all hate mail his way,) and has a great memory and attention to detail for the show. Also, some b.s. about fantasy football and baseball (5:14) *Recorded September 1st, 2023 (Not August, as stated in the episode) FILLER: Transformers: Rise of the Beasts (6:39) CONTACT: citizensofpawnee@gmail.com and Instagram @citizensofpawneepodcast and @parksrecmemes New episodes every Tuesday GO T.B.A.

Adafruit Industries
ESP32-S3 moon phase clock test on 2.1" round TFT display

Adafruit Industries

Play Episode Listen Later Sep 2, 2023 1:31


Now that we have somewhat-kinda-sorta working support for RGB TFT displays on the ESP32-S3 - shout out to Jepler, who is doing the hard work over in this PR https://github.com/adafruit/circuitpython/pull/8351 - its time to test it with wifi too! That's right, the S3 can do wifi and these big displays at once, and CircuitPython is a beautiful framework for it since its so fast to iterate. This code snippet is based on PaintYourDragon's moon clock code here https://learn.adafruit.com/moon-phase-clock-for-adafruit-matrixportal but pared down for testing. We get the geolocation from IP, then look up the moon phase. Currently we just hardcoded it to display today's phase, but the next step is generating ~28 different phase images, and we'll display the one for the current evening as the API informs us. Visit the Adafruit shop online - http://www.adafruit.com ----------------------------------------- LIVE CHAT IS HERE! http://adafru.it/discord Subscribe to Adafruit on YouTube: http://adafru.it/subscribe New tutorials on the Adafruit Learning System: http://learn.adafruit.com/ ----------------------------------------- #espressif #esp32 #espfriends #display #adafruit #electronics #opensource #opensourcehardware #circuitpython #tftdisplay #coding #rgbdisplay #round #moonphase #test #wifi #jepler #prsupport #framework #fastiterate #geolocation #api #eveningphase

pr ip clock display api s3 moon phases adafruit esp32 circuitpython adafruit learning system
Gratitude Blooming Podcast
Reciprocity: Beyond a 'debt of gratitude'

Gratitude Blooming Podcast

Play Episode Listen Later Sep 1, 2023 29:03 Transcription Available


There is more to reciprocity than giving back. Our guest, Christine Liboon, a PhD student in Education, is here to take us on a deep exploration of her findings. She shares her fascinating research around the Filipino concept of 'utang na loob', which she and her colleagues are redefining as a gift rather than a debt. We also explore the symbolism of currency as not merely monetary exchange but a representation of a promise or a commitment.The second part of our journey with Christine leads us into an important conversation about self-care, joy and reciprocal relationships. Drawing from her personal experiences, she underlines the essence of rekindling joy in activities we love, the significance of introspection, and the need to rebalance life. She makes us rethink kindness and gentleness, not just as virtues, but as tangible bridges connecting self-care and community. As a parting gift, we invite you  to embrace and feel the power of gentleness through music. So, get ready to be inspired and challenged in equal measure in this enriching episode.If you enjoyed this episode, please take a moment to leave us a 5-star rating and review. Your feedback is valuable to us and helps us grow. Share your thoughts and comments by emailing us at hello@gratitudeblooming.com. We love hearing from our listeners.And don't forget to check out our shop at www.gratitudeblooming.com to help us sustain this podcast or sponsor us here. Thank you for your continued support. We appreciate you!

R Weekly Highlights
Issue 2023-W35 Highlights

R Weekly Highlights

Play Episode Listen Later Sep 1, 2023 40:00


The next generation of object-oriented programming in R arrives on CRAN, a novel use of R to automate R scripts and documents for Tidy Tuesday analyses, and a terrific presentation de-mystifying the world of web APIs in R. Episode Links This week's curator: Eric Nantz - @theRcast (https://twitter.com/theRcast) (Twitter) & @rpodcast@podcastindex.social (https://podcastindex.social/@rpodcast) (Mastodon) {S7} 0.1.0 (https://cran.r-project.org/package=S7): An Object Oriented System Meant to Become a Successor to S3 and S4 Creating template files with R (https://nrennie.rbind.io/blog/script-templates-r/) A Gradual Introduction to Web APIs and JSON (https://laderast.github.io/intro_apis_json_cascadia/#/title-slide) Entire issue available at rweekly.org/2023-W35 (https://rweekly.org/2023-W35.html) Supplement Resources S7 - A new object-oriented system in R https://rconsortium.github.io/S7/ Name of the game (issue 262): https://github.com/RConsortium/OOP-WG/issues/262 TidyX Episode 109 - R Classes and Objects - Making an S3 Object Part 1 https://www.youtube.com/watch?v=k9PGOx9Oqjo Mike's boston R user group talk (https://github.com/mthomas-ketchbrook/boston_useR_talk) Standard Notes (End-to-End Encrypted Notes App) https://standardnotes.com/ HedgeDoc https://hedgedoc.org/ fs - Provide cross platform file operations based on libuv https://fs.r-lib.org/ Room by Room Temperature Tracking (Jared Lander) https://www.jaredlander.com/2021/02/room-by-room-temperature-tracking/ Ted Laderas - A gRadual introduction to Web APIs and JSON https://www.youtube.com/watch?v=HA7KfdEsdpo Open Call for rOpenSci Champions Program 2023 Applications (deadline September 4th, 2023) https://ropensci.org/champions/ Supporting the show Use the contact page at https://rweekly.fireside.fm/contact to send us your feedback R-Weekly Highlights on the Podcastindex.org (https://podcastindex.org/podcast/1062040) - You can send a boost into the show directly in the Podcast Index. First, top-up with Alby (https://getalby.com/), and then head over to the R-Weekly Highlights podcast entry on the index. A new way to think about value: https://value4value.info Get in touch with us on social media Eric Nantz: @theRcast (https://twitter.com/theRcast) (Twitter) and @rpodcast@podcastindex.social (https://podcastindex.social/@rpodcast) (Mastodon) Mike Thomas: @mike_ketchbrook (https://twitter.com/mike_ketchbrook) (Twitter) and @mike_thomas@fosstodon.org (https://fosstodon.org/@mike_thomas) (Mastodon) Music Credits powered by OverClocked Remix (https://ocremix.org/) Fiesta Amongst the Trees - Ristar - Southwestern College Afro-Cuban Jazz Ensemble - https://ocremix.org/remix/OCR02125 Suco de Melancia - Final Fantasy 7 Voices of the Lifestream - Red Tailed Fox - https://ocremix.org/remix/OCR01671

North Meets South Web Podcast
Airdrop, user-specific S3, and frontend complexity

North Meets South Web Podcast

Play Episode Listen Later Aug 31, 2023 39:19


In this episode, Jake and Michael discuss using Airdrop to simplify and speed up your asset compilation and application deployment, restricting access to S3 buckets based on username, and some of the complexities around building your frontend.This episode is brought to you by our friends at Workvivo - The leading employee communication app.Show links Nucleus Laracon AU Airdrop Deploying with Airdrop

Screaming in the Cloud
How Redpanda Extracts Business Value from Data Events with Alex Gallego

Screaming in the Cloud

Play Episode Listen Later Aug 31, 2023 34:43


Alex Gallego, CEO & Founder of Redpanda, joins Corey on Screaming in the Cloud to discuss his experience founding and scaling a successful data streaming company over the past 4 years. Alex explains how it's been a fun and humbling journey to go from being an engineer to being a founder, and how he's built a team he trusts to hand the production off to. Corey and Alex discuss the benefits and various applications of Redpanda's data streaming services, and Alex reveals why it was so important to him to focus on doing one thing really well when it comes to his product strategy. Alex also shares details on the Hack the Planet scholarship program he founded for individuals in underrepresented communities. About AlexAlex Gallego is the founder and CEO of Redpanda, the streaming data platform for developers. Alex has spent his career immersed in deeply technical environments, and is passionate about finding and building solutions to the challenges of modern data streaming. Prior to Redpanda, Alex was a principal engineer at Akamai, as well as co-founder and CTO of Concord.io, a high-performance stream-processing engine acquired by Akamai in 2016. He has also engineered software at Factset Research Systems, Forex Capital Markets and Yieldmo; and holds a bachelor's degree in computer science and cryptography from NYU. Links Referenced: Redpanda: https://redpanda.com/ Twitter: https://twitter.com/emaxerrno Redpanda community Slack: https://redpandacommunity.slack.com/join/shared_invite/zt-1xq6m0ucj-nI41I7dXWB13aQ2iKBDvDw Hack The Planet Scholarship: https://redpanda.com/scholarship TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Tired of slow database performance and bottlenecks on MySQL or PostgresSQL when using Amazon RDS or Aurora? How'd you like to reduce query response times by ninety percent? Better yet, how would you like to get me to pronounce database names correctly? Join customers like Zscaler, Intel, Booking.com, and others that use OtterTune's artificial intelligence to automatically optimize and keep their databases healthy. Go to ottertune dot com to learn more and start a free trial. That's O-T-T-E-R-T-U-N-E dot com.Corey: Welcome to Screaming in the Cloud, I'm Corey Quinn, and this promoted guest episode is brought to us by our friends at Redpanda, which I'm thrilled about because I have a personal affinity for companies that have cartoon mascots in the form of animals and are willing to at least be slightly creative with them. My guest is Alex Gallego, the founder and CEO over at Redpanda. Alex, thanks for joining me.Alex: Corey, thanks for having me.Corey: So, I'm not asking about the animal; I'm talking about the company, which I imagine is a frequent source of disambiguation when you meet people at parties and they don't quite understand what it is that you do. And you folks are big in the data streaming space, but data streaming can mean an awful lot of things to an awful lot of people. What is it for you?Alex: Largely it's about enabling developers to build applications that can extract value of every single event, every click, every mouse movement, every transaction, every event that goes through your network. This is what Redpanda is about. It's like how do we help you make more money with every single event? How do we help you be more successful? And you know, happy to give examples in finance, or IoT, or oil and gas, if it's helpful for the audience, but really, to me, it's like, okay, if we can give you the framework in which you can build a new application that allows you to extract value out of data, every single event that's going through your network, to me, that's what a streaming is about. It large, it's you know, data contextualized with a timestamp and largely, a sort of a database of event streaming.Corey: One of the things that I find curious about the space is that usually, companies wind up going one of two directions when you're talking about data streaming. Either there, “Oh, just send it all to us and we'll take care of it for you,” or otherwise, it's a, great they more or less ship something that you've run in your own environment. In the olden days of data centers, that usually resembled a box of some sort. You're one of those interesting split-the-difference companies where you offer both models. Do you find that one of those tends to be seeing more adoption these days or that there's an increasing trend toward one direction or the other?Alex: Yeah. So, right now, I think that to me, the future of all these data-intensive products—whether you're a database or a streaming engine—will, because simply of cost of networks transferred between the hybrid clouds and your accounts, sending a gigabyte a second of data between, let's say, you know, your data center and a vendor, it's just so expensive that at some point, from just a cost perspective, like, running the infrastructure, it's in the millions of dollars. And so, running the data inside your VPC, it's sort of the next logical evolution of how we've used to consume services. And so, I actually think it's just the evolution: people would self-host because of costs and then they would use services because of operational simplicity. “I don't want to spend team skills and time building this. I want to pay a vendor.”And so, BYOC, to be honest—which is what we call this offering—it was about [laugh] sidestepping the costs and of being stuck in the hybrid clouds, whether it's Google or Amazon, where you're paying egress and ingress costs and it's just so expensive, in addition to this whole idea of data residency or data sovereignty and privacy. It's like, yeah, why not both? Like, if I'm an engineer, I want low latency and I don't want to pay you to transfer this thing to the next rack. I mean, my computer's probably, like, you know, a hundred feet away from my customer's computer. Like, why [laugh] way is that so complicated? So, you know, my view is that the future of data-intensive products will be in this form of where it—like, data planes are actually owned by companies, and then you offer that as a Software as a Service.Corey: One of the things that catches an awful lot of companies with telemetry use cases—or data streaming as another example of that—by surprise when they start building their own cloud-hosted offering is that they're suddenly seeing a lot more cross-AZ data charges than they would have potentially expected. And that's because unlike cross-region or the really expensive version of this with egress, it's a penny in and a penny out per gigabyte in most of AWS regions. Which means that that isn't also bound strictly to an AWS organization. So, you have customers co-located with you and you're starting to pay ingress charges on customers throwing their data over to you. And, on some level, the most economical solution for you is well, we're just going to put our listeners somewhere else far away so that we can just have them pay the steep egress fee but then we can just reflect it back to ourselves for free.And that's a terrible pattern, but it's a byproduct of the absolutely byzantine cross-AZ data transfer pricing, in fact, all of the data transfer pricing that is at least AWS tends to present. And it shapes the architectural decisions you make as a result.Alex: You know, as a user, it just didn't make sense. When we launched this product, the number of people that says like, “Why wouldn't your charge for, you know, effectively renting [unintelligible 00:05:14], and giving a markup to your customers?” That's we don't add any value on that, you know? I think people should really just pay us for the value that we create for them. And so, you know, for us competing with other companies is relatively easy.Competing with MSK is it's harder because MSK just has this, you know, muscle where they don't charge you for some particular network traffic between you. And so, it forces companies like us that are trying to be innovative in the data space to, like, put our services in that so that we can actually compete in the market. And so, it's a forcing function of the hybrid clouds having this strong muscle of being able to discount their services in a way that companies just simply don't have access to. And then, you know, it becomes—for the others—latency and sovereignty.Corey: This is the way that effectively all of AWS has first-party offerings of other things go. Replication traffic between AZs is not chargeable. And when I asked them about that, they say, “Oh, yeah. We just price that into the cost of the service.” I don't know that I necessarily buy that because if I try and run this sort of thing on top of EC2, it would cost me more than using their crappy implementation of it, just in data transfer alone for an awful lot of use cases.No third party can touch that level of cost-effectiveness and discounting. It really is probably the clearest example I can think of actual anti-competitive behavior in the market. But it's also complex enough to explain, to, you know, regulators that it doesn't make for exciting exposés and the basis for lawsuits. Yet. Hope springs eternal.Alex: [laugh]. You know—okay, so here is how—if someone is listening to this podcast and is, like, “Okay, well, what can I do?” For us, S3 is the answer. S3 is basically you need to be able to lean in into S3 as a way of replication across [AZ 00:06:56], you need to be able to lean into S3 to read data. And so actually, when I wrote, originally, Redpanda, you know, it's just like this C++ thing using [unintelligible 00:07:04], geared towards super low latency.When we moved it into the cloud, what we realized is, this is cost prohibitive to run either on EBS volumes or local disk. I have to tier all the storage into S3, so that I can use S3's cross-AZ network transfer, which is basically free, to be able to then bring a separate cluster on a different AZ, and then read from the bucket at zero cost. And so, you end up really—like, there are fundamental technical things that you have to do to just be able to compete in a way that's cost-effective for you. And so, in addition to just, like, the muscle that they can enforce on the companies is—it—there are deep implications of what it translates to at the technical level. Like, at the code level.Corey: In the cloud, more than almost anywhere else, it really does become apparent that cost and architecture are fundamentally the same thing. And I have a bit of an advantage here in that I've seen what you do deployed at least one customer of mine. It's fun. When you have a bunch of logos on your site, it's, “Hey, I recognize some of those.” And what I found interesting was the way that multiple people, when I spoke to them, described what it is that you do because some of them talked about it purely as a cost play, but other people were just as enthusiastic about it being a means of improving feature velocity and unlocking capabilities that they didn't otherwise have or couldn't have gotten to without a whole lot of custom work on their part. Which is it? How do you view what it is that you're bringing to market? Is it a cost play or is it a capability story?Alex: From our customer base, I would say 40% is—of our customer base—is about Redpanda enabling them to do things that they simply couldn't do before. An example is, we have, you know, a Fortune 100 company that they basically run their hedge trading strategy on top of Redpanda. And the reason for that is because we give them a five-millisecond average latency with predictable flight latencies, right? And so, for them, that predictability of Redpanda, you know, and sort of like the architecture that came about from trying to invent a new storage engine, allows them to throw away a bunch of in-house, you know, custom-built pub/sub messaging that, you know, basically gave them the same or worse latency. And so, for them, there's that.For others, I think in the IoT space, or if you have flying vehicles around the world, we have some logos that, you know, I just can't mention them. But they have this, like, flying computers around the world and they want to measure that. And so, like, the profile of the footprint, like, the mechanical footprint of being able to run on a single Pthread with a few megs of memory allows these new deployment models that, you know, simply, it's just, it's not possible with the alternatives where let's say you have to have, you know, like, a zookeeper on the schema registry and an HTTP proxy and a broker and all of these things. That simply just, it cannot run on a single Pthread with a few megs of memory, if you put any sort of workload into that. And so, it's like, the computational efficiencies simply enable new things that you couldn't do before. And that's probably 40%. And then the other, it's just… money was really cheap last year [laugh] or the year before and I think now it's less cheap [unintelligible 00:10:08] yeah.Corey: Yeah, I couldn't help but notice that in my own business, too. It turns out that not giving a shit about the AWS bill was a zero-interest-rate phenomenon. Who knew?Alex: [laugh]. Yeah, exactly. And now people [unintelligible 00:10:17], you know, the CIOs in particular, it's like, help. And so, that's really 60%, and our business has boomed since.Corey: Yeah, one thing that I find interesting is that you've been around for only four years. I know that's weird to say ‘only,' but time moves differently in tech. And you've started showing up in some very strange places that I would not have expected. You recently—somewhat recently; time is, of course, a flat circle—completed $100 million Series C, and I also saw you in places where I didn't expect to see you in the form of, last week, one of your large competitor's earnings calls, where they were asked by an analyst about an unnamed company that had raised $100 million Series C, and the CEO [unintelligible 00:11:00], “Oh, you're probably talking about Redpanda.” And then they gave an answer that was fine.I mean, no one is going to be on an earnings call and not be prepared for questions like that and to not have an answer ready to go. No one's going to say, “Well, we're doomed if it works,” because I think that businesses are more sophisticated than that. But it was an interesting shout-out in a place where you normally don't see competitors validate that you're doing something interesting by name-checking you.Alex: What was fundamentally interesting for me about that, is that I feel that as an investor, if you're putting you know, 2, 3, 4, or $500 million check into a public position of a company, you want to know, is this money simply going to make returns? That's basically what an investor cares about. And so, the reason for that question is, “Hey, there's a Series C startup company that now has a bunch of these Fortune 2000 logos,” and you know, when we talked to them, like, their customer [unintelligible 00:11:51] phenomena, like, why is that the case? And then, you know, our competitor was forced to name, you know, [laugh] a single win. That's as far as I remember it. We don't know of any additional customers that have switched to that.And so, I think when you have, like, you know, your win rate is above, whatever, 95%, 97% ratio, then I think, you know, they're just sort of forced to answer that. And in a way, I just think that they focus on different things. And for me, it was like, “Okay, developer, hands on keyboard, behind the terminal, how do I make you successful?” And that seems to have worked out enough to be mentioned in the earnings call.Corey: On some level, it's a little bit of a dog-and-pony show. I think that as companies had a certain point of scale, they feel that they need to validate what they're doing to investors at various points—which is always, on some level, of concern—and validate themselves to analysts, both financial—which, okay, whatever—and also, industry analysts, where they come with checklists that they believe is what customers want and is often a little bit off of the mark. But the validation that I think that matters, that actually determines whether or not something has legs is what your customers—you know, people paying you money for a thing—have to say and what they take away from what you're doing. And having seen in a couple of cases now myself, that usage of Redpanda has increased after initial proofs of concept and putting things on to it, I already sort of know the answer to this, but it seems that you also have a vibrant community of boosters for people who are thrilled to use the thing you're selling them.Alex: You know, Jumptraders recently posted that there was a use case in the new stack where they, like, put for the most mission-critical. So, for those of you that listening, Jumptraders is financial company, and they're super technical company. One of, like, the hardest things, they'll probably put your [unintelligible 00:13:35] your product through some of the most rigorous testing [unintelligible 00:13:38]. So, when you start doing some of these logos, it gives confidence. And actually, the majority of our developers that we get to partner with, it was really a friend telling a friend, for [laugh] the longest time, my marketing department was super, super small.And then what's been fun, some, like, really different use case was the one I mentioned about on this, like, flying vehicles around the world. They fly both in outer space and in airplanes. That was really fun. And then the large one is when you have workloads at, like, 14-and-a-half gigabytes per second, where the alternative of using something like Kinesis in the case of Lacework—which, you know, they wrote a new stack article about—would be so exorbitantly expensive. And so, in a way, I think that, you know, just trying to make the developers successful, really focusing, honestly, on the person who just has to make things work. We don't—by the time we get to the CIO, really the champion was the engineer who had to build an application. “I was just trying to figure it out the whack-a-mole of trying to debug alternative systems.”Corey: One of the, I think, seductive problems with your entire space is that no one decides day one that they're going to implement a data streaming solution for a very scaled-out, high-traffic site. The early adoption is always a small thing that you're in the process of building. And at that scale at that speed, it just doesn't feel like it's that hard of a problem because scale introduces its own unique series of challenges, but it's often one that people only really find out themselves when the simple thing that works in theory but not in production starts to cause problems internally. I used to work with someone who was a deeply passionate believer in Apache Kafka to a point where it almost became a problem, just because their answer to every problem—it almost didn't matter if it was, “How do we get more coffee this morning?”—Kafka would be the answer for all of it.And that's great, but it turned out, they became one of these people that borderline took on a product or a technology as their identity. So, anything that would potentially take a workload away from that, I got a lot of internal resistance. I'm wondering if you find that you're being brought in to replace existing systems or for completely greenfield stuff. And if the former, are you seeing a lot of internal resistance to people who have built a little niche for themselves?Alex: It's true, the people that have built a career, especially at large banks, were a pretty good fit for, you know, they actually get a team, they got a promotion cycle because they brought this technology and the technology sort of helped them make money. I personally tend to love to talk to these people. And there was a ca—to me, like, technically, let's talk about, like, deeply technical. Let me help you. That obviously doesn't scale because I can't have the same conversation with ten people.So, we do tend to see some of that. Actually, from our customers' standpoint, I would say that the large part of our customer base, you know, if I'm trying to put numbers, maybe 65%, I probably rip and replace of, you know, either upstream Apache Software or private companies or hosted services, et cetera. And so, I think you're right in saying, “Hey, that resistance,” they probably handled the [unintelligible 00:16:38], but what changed in the last year is that the CIO now stepped in and says, “I am going to fire all of you or you have to come up with a $10 million savings. Help me.” [laugh]. And so, you know, then really, my job is to help them look like a hero.It's like, “Hey, look, try it tested, benchmark it in your with your own workload, and if it saves you money, then use it.” That's been, you know, to sort of super helpful kind of on the macroeconomic environment. And then the last one is sometimes, you know, you do have to go with a greenfield, right? Like, someone has built a career, they want to gain confidence, they want to ask you questions, they want to trust you that you don't lose data, they want to make sure that you do say the things that you want to say. And so, sometimes it's about building trust and building that relationship.And developers are right. Like, there's a bunch of products out there. Like, why should I trust you? And so, a little easier time, probably now, that you know, with the CIOs wanting to cut costs, and now you have an excuse to go back to the executive team and say, “Look, I made you look smart. We get to [unintelligible 00:17:35], you know, our systems can scale to this.” That's easy. Or the second one is we do, you know, we'll start with some side use case or a greenfield. But both exists, and I would say 65% is probably rip-outs.Corey: One question, I love to, I wouldn't call it ambush, but definitely come up with, the catches some folks by surprise is one of the ways I like to sort out zealots from people who are focused on business problems. Do have an example of a data streaming workload for which Redpanda would not be a great fit?Alex: Yeah. Database-style queries are not a fit. And so, think that there was a streaming engine before there was trying to build a database on top of it, and, like—and probably it does work in some low volume of traffic, like, say 5, 10 megabytes per second, but when you get to actual large scale, it just it doesn't work. And it doesn't work because but what Redpanda is, it gives you two properties as a developer. You can add data to the end or you can truncate the head, right?And so, because those are your only two operations on the log, then you have to build this entire caching level to be able to give this database semantics. And so, do you know, I think for that the future isn't for us to build a database, just as an example, it's really to almost invert it. It's like, hey, what if we make our format an open format like Apache Iceberg and then bring in your favorite database? Like, bring in, you know, Snowflake or Athena or Trina or Spark or [unintelligible 00:18:54] or [unintelligible 00:18:55] or whatever the other [unintelligible 00:18:56] of great databases that are better than we are, and doing, you know, just MPP, right, like a massively parallelizable database, do that, and then the job for us, for [unintelligible 00:19:05], let me just structure your log in a way that allows you to query, right? And so, for us, when we announced the $100 million dollar Series C funding, it's like, I'm going to put the data in an iceberg format so you can go and query it with the other ten databases. And there are a better job than we are at that than we are.Corey: It's frankly, refreshing to see a vendor that knows where, okay, this is where we start and this is where we stop because it just seems that there's been an industry-wide push for a while now to oh, you built a component in a larger system that works super well. Now, expand to do everything else in the architectural diagram. And you suddenly have databases trying to be network transport layers and queues trying to be data warehouses, and it just doesn't work that way. It just it feels like oh, this is a terrible approach to solving this particular problem. And what's worse, from my mind, is that people who hadn't heard of you before look at you through this lens that does not put you in your best light, and, “Oh, this is a terrible database.” Well, it's not supposed to be one.Alex: [laugh].Corey: But it also—it puts them off as a result. Have you faced pressure to expand beyond your core competency from either investors or customers or analysts or, I don't know, the voices late at night that I hear and I assume everyone else does, too?Alex: Exactly. The 3 a.m. voice that I have to take my phone and take a voice note because it's like, I don't want to lose this idea. Totally. For us. I think there's pressures, like, hey, you built this great engine. Why don't you add, like, the latest, you know, soup de jour in systems was like a vector database.I was like, “This doesn't even make any sense.” For me, it's, I want to do one thing really well. And I generally call it internally, ‘the ring zero.' It's, if you think of the internet, right, like, as a computer, especially with this mode to what we talked about earlier in a BYOC, like, we could be the best ring zero, the best sort of like, you know, messaging platform for people to build real-time applications. And then that's the case and there's just so much low-hanging fruit for us.Like, the developer experience wasn't great for other systems, like, why don't we focus on the last mile, like, making that developer, you know, successful at doing this one thing as opposed to be an average and a bunch of other a hundred products? And until we feel, honestly, that we've done a phenomenal job at that—I think we still have some roadmap to get there—I don't want to expand. And, like, if there's pressure, my answer is, like… look, the market is big enough. We don't have to do it. We're still, you know, growing.I think it's obviously not trivial and I'm kind of trivializing a bunch of problems from a business perspective. I'm not trying to degrade anyone else. But for us, it's just being focused. This is what we do well. And bring every other technology that makes you successful. I don't really care. I just want to make this part well.Corey: I think that that is something that's under-appreciated. I feel like I should get over at one point to something that's been nagging at the back of my mind. Some would call it a personal attack and I suppose I'll let them, but what I find interesting is your background. Historically, you were a distributed systems engineer at very large scale. And you apparently wrote the first version of Redpanda yourself in—was it C or C++?Alex: C++.Corey: Yeah. And now you are the CEO of a company that is clearly doing very well. Have you gotten the hell out of production yet? The reason I ask this is I have worked in a number of companies where the founder was also the initial engineer and then they invariably treated main as their feature branch and the rest of us all had to work around them to keep them from, you know, destroying everything we were trying to build around us, due to missing context. In other words, how annoyed with you are your engineers on any given afternoon?Alex: [laugh]. Yeah. I would say that as a company builder now, if I may say that, is the team is probably the thing I'm the most proud of. They're just so talented, such good [unintelligible 00:22:47] of humans. And so—group of humans—I stopped coding about two years ago, roughly.So, the company is four-and-a-half years old, really the first two-and-a-half years old, the first one, two years, definitely, I was personally putting in, like, tons and tons of hours working on the code. It was a ton of fun. To me, one of the most rewarding technical projects I've ever had a chance to do. I still read pull requests, though, just so that when I have a conversation with a technical leader, I don't be, like, I have no clue how the transactions work. So, I still have to read the code, but I don't write any more code and my heart was a little broken when my dev prod team removed my write access to the GitHub repo.We got SOC2 compliance, and they're like, “You can't have access to being an admin on Google domains, and you're no longer able to write into main.” And so, I think as a—I don't know, maybe my identity—myself identity is that of a builder, and I think as long as I personally feel like I'm building, today, it's not code, but you know, is the company and [unintelligible 00:23:41] sort of culture, then I feel okay [laugh]. But yeah, I no longer write code. And the last story on that, is this—an engineer of ours, his name is [Stefan 00:23:51], he's like, “Hey, so Alex wrote this semaphore”—this was actually two days ago—and so they posted a video, and I commented, I was like, “Hey, this was the context of semaphore. I'm sorry for this bug I caused.” But yeah, at least I still remember some context for them.Corey: What's fun is watching things continue to outpace and outgrow you. I mean, one of the hard parts of building a company is the realization that every person you hire for a thing that's now getting off of your plate is better at that thing than you are. It's a constant experience of being humbled. And at some point, things wind up outpacing you to the point where, at least in my case, I've been on calls with customers and I explained how we did some things and how it worked and had to be corrected by my team of, “Well. That used to be true, however…” like, “Oh, dear Lord. I'm falling behind.” And that's always been a weird feeling for me.Alex: Totally. You know, it's the feeling of being—before I think I became a CEO, I was a highly comped  engineer and did a competent, to the extent that it allowed me to build this product. And then you start doing all of these things and you're incompetent, obviously, by definition because you haven't done those things and so there's like that discomfort [laugh]. But I have to get it done because no one else wants to do, whatever, like say, like, you know, rev ops or marketing or whatever.And then you find somebody who's great and you're like, oh my God, I was like, I was so poor tactically at doing this thing. And it's definitely humbling every day. And it's almost it's, like, gosh, you're just—this year was kind of this role where you're just, like, mediocre at, like, a whole lot of things as a company, but you're the only person that has to do the job because you have the context and you just have to go and do it. And so, it's definitely humbling. And in some ways, I'm learning, so for me today, it's still a lot of fun to learn.Corey: This is a little more in the weeds, I suppose, but I always love to ask people these questions. Because I used to be naive, which meant that I had hope and I saw a brighter future in technology. I now know that was all a lie. But I used to believe that out there was some company whose internal infrastructure for what they'd built was glorious and it would be amazing. And I knew I would never work there, nor what I want to, because when everything's running perfectly, all I can really do is mess that up; there's no way to win and a bunch of ways to lose.But I found that place doesn't exist. Every time I talk to someone about how they built the thing that they built and I ask them, “If you were starting over from scratch, what would you do differently?” The answer often distills down to, “Oh, everything.” Because it's an organically evolving system that oh, yeah, everything's easier the second time. At least you get to find new failure modes go in that way. When you look back at how you designed it originally, are there any missteps that you could have saved yourself a whole lot of grief by not making the first time?Alex: Gosh, so many things. But if I were to give Hollywood highlights on these things, something that [unintelligible 00:26:35] is, does well is exposing these high-level data types of, like, streams, and lists and maps and et cetera. And I was like, “Well, why couldn't streams offer this as a first-class citizen?” And we got some things well which I think would still do, like the whole [thread recorder 00:26:49] could—like, the fundamentals of the engine I will still do the same. But, you know, exposing new programming models earlier in the life of the product, I think would have allowed us to capture even more wildly different use cases.But now we kind of have this production engine, we have to support Fortune 2000, so you know, it's kind of like a very delicate evolution of the product. Definitely would have changed—I would have added, like, custom data types upfront, I would have pushed a little harder on I think WebAssembly than we did originally. Man, I could just go on for—like, [added detail 00:27:21], I would definitely have changed things. Like, I would have pressed on the first—on the version of the cloud that we talked about early on, that as the first deployment mode. If we go back through the stack of all of the products you had, it's funny, like, 11 products that are surfaced to the customers to, like, business lines, I would change fundamental things about just [laugh], you know, everything else. I think that's maybe the curse of the expert. Like, you know, you could always find improvements.Corey: Oh, always. I still look back at my career before starting this place when I was working in a bunch of finance companies, and—I'll never forget this; it was over a decade ago—we were building out our architecture in AWS, and doing a deal with a large finance company. And they said, “Cool, where's your data center?” And I said, “Oh, it's AWS.” And they said, “Ha ha ha ha. Where's your data center?”And that was oh, okay, great. Now, it feels like if that's their reaction, they have not kept pace with the times. It feels it is easier to go to a lot of very serious enterprises with very serious businesses and serious workload concerns attendant to those and not get laughed out of the room because you didn't wind up doing a multi-million dollar data center build out that, with an eye toward making it look as enterprise-y as possible.Alex: Yeah. Okay, so here's, I think, maybe something a little bit controversial. I think that's true. People are moving to the cloud, and I don't think that that idea, especially when we go when we talk to banks, is true. They're like, “Hey, I have this contract with one of the hybrid clouds.”—you know, it's usually with two of them, and then you're like—“This is my workload. I want to spend $70 million or $100 million. Who could give me the biggest discount?” And then you kind of shop it around.But what we are seeing is that effectively, the data transfer costs are so expensive and running this for so much this large volume of traffic is still so, so expensive, that there is an inverse [unintelligible 00:29:09] to host from some category of the workload where you don't have dynamism. Actually hosted in your data center is, like, a huge boom in terms of cost efficiencies for the companies, especially where we are and especially in finances—you mentioned that—if you're trying to trade and you have this, like, steady state line from nine to five, whatever, eight to four, whenever the markets open, it's actually relatively cost-efficient because you can measure hey, look, you know, the New York Stock Exchange is 1.5 gigabytes per second at market close. Like, I could provision my hardware to beat this. And like, it'll be that I don't need this dynamism that the cloud gives me.And so yeah, it's kind of fascinating that for us because we offered the self-hosted Redpanda which can adapt to super low latencies with kernel parameter tuning, and the cloud due to the tiered storage, we talked about S3 being [unintelligible 00:29:52] to, so it's been really fun to participate in deployments where we have both. And you couldn't—they couldn't look more different. I mean, it's almost looks like two companies.Corey: One last question before we wind up calling it an episode. I think I saw something fly by on Twitter a while back as I slowly returned to the platform—no, I'm not calling it X—something you're doing involving a scholarship. Can you tell me a bit more about that?Alex: Yeah. So, you know, I'm a Latino CEO, first generation in the States, and some of the things that I felt really frustrated with, growing up that, like, I feel fortunate because I got to [unintelligible 00:30:25] that is that, you know, people were just—that look like me are probably given some bullshit QA jobs, so like, you know, behemoth job, I think, for a bank. And so, I wanted to change that. And so, we give money and mentorship to people and we release all of the intellectual property. And so, we mentor someone—actually, anyone from underrepresented backgrounds—for three months.We give then, like, 1200 bucks a month—or 1500, I can't remember—mentorship from our top principal level engineers that have worked at Amazon and Google and Facebook and basically the world's top companies. And so, they meet with them one hour a week, we give them money, they could sit in the couch if they want to. No one has to [unintelligible 00:31:06]. And all we're trying to do is, like, “Hey, if you are part of this group, go and try to build something super hard.” [laugh].And often their minds, which is great, and they're like, “I want to build an OpenAI competitor in three months, and here's the week-by-week progress.” Or, “I want to build a new storage engine, new database in three months.” And that's the kind of people that we want to help, these like, super ambitious, that just hasn't had a chance to be mentored by some of the world's best engineers. And I just want to help them. Like, we—this is a non-scalable project. I meet with them once a week. I don't want to have a team of, like, ten people.Like, to me, I feel like their most valuable thing I could do is to give them my time and to help them mentor. I was like, “Hey, let's think about this problem. Let's decompose this. How do you think about this?” And then bring you the best engineers that I, you know, that work for—with me, and let me help you think about problems differently and give you some money.And we just don't care how you use the time or the money; we just want people to work on hard problems. So, it's active. It runs once a year, and if anyone is listening to this, if you want to send it to your friends, we'd love to have that application. It's for anyone in the world, too, as long as we can send the person a check [laugh]. You know, my head of finance is not going to walk to a Moneygram—which we have done in the past—but other than that, as long as you have a bank account that we can send the check to, you should be able to apply.Corey: That is a compelling offer, particularly in the current macro environment that we find ourselves faced in. We'll definitely put a link to that into the [show notes 00:32:32]. I really want to thank you for taking the time to, I guess, get me up to speed on what it is you're doing. If people want to learn more where's the best place for them to go?Alex: On Twitter, my handle is @emaxerrno, which stands for the largest error in the kernel. I felt like that was apt for my handle. So, that's one. Feel free to find me on the community Slack. There's a Slack button on the website redpanda.com on the top right. I'm always there if you want to DM me. Feel free to stop by. And yeah, thanks for having me. This was a lot of fun.Corey: Likewise. I look forward to the next time. Alex Gallego, CEO and founder at Redpanda. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an insulting comment that I will almost certainly never read because they have not figured out how to get data from one place to another.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

Screaming in the Cloud
Reflecting on a Legendary Tech Career with Kelsey Hightower

Screaming in the Cloud

Play Episode Listen Later Aug 29, 2023 43:01


Kelsey Hightower joins Corey on Screaming in the Cloud to discuss his reflections on how the tech industry is progressing. Kelsey describes what he's been getting out of retirement so far, and reflects on what he learned throughout his high-profile career - including why feature sprawl is such a driving force behind the complexity of the cloud environment and the tactics he used to create demos that are engaging for the audience. Corey and Kelsey also discuss the importance of remaining authentic throughout your career, and what it means to truly have an authentic voice in tech. About KelseyKelsey Hightower is a former Distinguished Engineer at Google Cloud, the co-chair of KubeCon, the world's premier Kubernetes conference, and an open source enthusiast. He's also the co-author of Kubernetes Up & Running: Dive into the Future of Infrastructure. Recently, Kelsey announced his retirement after a 25-year career in tech.Links Referenced:Twitter: https://twitter.com/kelseyhightower TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Do you wish there were cheat codes for database optimization? Well, there are – no seriously. If you're using Postgres or MySQL on Amazon Aurora or RDS, OtterTune uses AI to automatically optimize your knobs and indexes and queries and other bits and bobs in databases. OtterTune applies optimal settings and recommendations in the background or surfaces them to you and allows you to do it. The best part is that there's no cost to try it. Get a free, thirty-day trial to take it for a test drive. Go to ottertune dot com to learn more. That's O-T-T-E-R-T-U-N-E dot com.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. You know, there's a great story from the Bible or Torah—Old Testament, regardless—that I was always a big fan of where you wind up with the Israelites walking the desert for 40 years in order to figure out what comes next. And Moses led them but could never enter into what came next. Honestly, I feel like my entire life is sort of going to be that direction. Not the biblical aspects, but rather always wondering what's on the other side of a door that I can never cross, and that door is retirement. Today I'm having returning guest Kelsey Hightower, who is no longer at Google. In fact, is no longer working and has joined the ranks of the gloriously retired. Welcome back, and what's it like?Kelsey: I'm happy to be here. I think retirement is just like work in some ways: you have to learn how to do it. A lot of people have no practice in their adult life what to do with all of their time. We have small dabs in it, like, you get the weekend off, depending on what your work, but you never have enough time to kind of unwind and get into something else. So, I'm being honest with myself. It's going to be a learning curve, what to do with that much time.You're probably still going to do work, but it's going to be a different type of work than you're used to. And so, that's where I am. 30 days into this, I'm in that learning mode, I'm on-the-job training.Corey: What's harder than you expected?Kelsey: It's not the hard part because I think mentally I've been preparing for, like, the last ten years, being a minimalist, learning how to kind of live within my means, learn to appreciate things that are just not work-related or status symbols. And so, to me, it felt like a smooth transition because I started to value my time more than anything else, right? Just waking up the next day became valuable to me. Spending time in the moment, right, you go to these conferences, there's, like, 10,000 people, but you learn to value those one-on-one encounters, those one-off, kind of, let's just go grab lunch situations. So, to me, retirement just makes more room for that, right? I no longer have this calendar that is super full, so I think for me, it was a nice transition in terms of getting more of that valuable time back.Corey: It seems to me that you're in a similar position to the one that I find myself in where the job that you were doing and I still am is tied, more or less, to a sense of identity as opposed to a particular task or particular role that you fill. You were Kelsey Hightower. That was a complete sentence. People didn't necessarily need to hear the rest of what you were working on or what you were going to be talking about at a given conference or whatnot. So, it seemed, at least from the outside, that an awful lot of what you did was quite simply who you were. Do you feel that your sense of identity has changed?Kelsey: So, I think when you have that much influence, when you have that much reputation, the words you say travel further, they tend to come with a little bit more respect, and so when you're working with a team on new product, and you say, “Hey, I think we should change some things.” And when they hear those words coming from someone that they trust or has a name that is attached to reputation, you tend to be able to make a lot of impact with very few words. But what you also find is that no matter what you get involved in—configuration management, distributed systems, serverless, working with customers—it all is helped and aided by the reputation that you bring into that line of work. And so yes, who you are matters, but one thing that I think helped me, kind of greatly, people are paying attention maybe to the last eight years of my career: containers, Kubernetes, but my career stretches back to the converting COBOL into Python days; the dawn of DevOps, Puppet, Chef, and Ansible; the Golang appearance and every tool being rewritten from Ruby to Golang; the Docker era.And so, my identity has stayed with me throughout those transitions. And so, it was very easy for me to walk away from that thing because I've done it three or four times before in the past, so I know who I am. I've never had, like, a Twitter bio that said, “Company X. X person from company X.” I've learned long ago to just decouple who I am from my current employer because that is always subject to change.Corey: I was fortunate enough to not find myself in the public eye until I owned my own company. But I definitely remember times in my previous incarnations where I was, “Oh, today I'm working at this company,” and I believed—usually inaccurately—that this was it. This was where I really found my niche. And then surprise I'm not there anymore six months later for, either their decision, my decision, or mutual agreement. And I was always hesitant about hanging a shingle out that was tied too tightly to any one employer.Even now, I was little worried about doing it when I went independent, just because well, what if it doesn't work? Well, what if, on some level? I think that there's an authenticity that you can bring with you—and you certainly have—where, for a long time now, whenever you say something, I take it seriously, and a lot of people do. It's not that you're unassailably correct, but I've never known you to say something you did not authentically believe in. And that is an opinion that is very broadly shared in this industry. So, if nothing else, you definitely were a terrific object lesson in speaking the truth, as you saw it.Kelsey: I think what you describe is one way that, whether you're an engineer doing QA, working in the sales department, when you can be honest with the team you're working with, when you can be honest with the customers you're selling into when you can be honest with the community you're part of, that's where the authenticity gets built, right? Companies, sometimes on the surface, you believe that they just want you to walk the party line, you know, they give you the lines and you just read them verbatim and you're doing your part. To be honest, you can do that with the website. You can do that with a well-placed ad in the search queries.What people are actually looking for are real people with real experiences, sharing not just fact, but I think when you mix kind of fact and opinion, you get this level of authenticity that you can't get just by pure strategic marketing. And so, having that leverage, I remember back in the day, people used to say, “I'm going to do the right thing and if it gets me fired, then that's just the way it's going to be. I don't want to go around doing the wrong thing because I'm scared I'm going to lose my job.” You want to find yourself in that situation where doing the right thing, is also the best thing for the company, and that's very rare, so when I've either had that opportunity or I've tried to create that opportunity and move from there.Corey: It resonates and it shows. I have never had a lot of respect for people who effectively are saying one thing today and another thing the next week based upon which way they think that the winds are blowing. But there's also something to be said for being able and willing to publicly recant things you have said previously as technology evolves, as your perspective evolves and, in light of new information, I'm now going to change my perspective on something. I've done that already with multi-cloud, for example. I thought it was ridiculous when I heard about it. But there are also expressions of it that basically every company is using, including my own. And it's a nuanced area. Where I find it challenging is when you see a lot of these perspectives that people are espousing that just so happen to deeply align with where their paycheck comes from any given week. That doesn't ring quite as true to me.Kelsey: Yeah, most companies actually don't know how to deal with it either. And now there has been times at any number of companies where my authentic opinion that I put out there is against party line. And you get those emails from directors and VPs. Like, “Hey, I thought we all agree to think this way or to at least say this.” And that's where you have to kind of have that moment of clarity and say, “Listen, that is undeniably wrong. It's so wrong in fact that if you say this in public, whether a small setting or large setting, you are going to instantly lose credibility going forward for yourself. Forget the company for a moment. There's going to be a situation where you will no longer be effective in your job because all of your authenticity is now gone. And so, what I'm trying to do and tell you is don't do that. You're better off saying nothing.”But if you go out there, and you're telling what is obviously misinformation or isn't accurate, people are not dumb. They're going to see through it and you will be classified as a person not to listen to. And so, I think a lot of people struggle with that because they believe that enterprise's consensus should also be theirs.Corey: An argument that I made—we'll call it a prediction—four-and-a-half years ago, was that in five years, nobody would really care about Kubernetes. And people misunderstood that initially, and I've clarified since repeatedly that I'm not suggesting it's going away: “Oh, turns out that was just a ridiculous fever dream and we're all going back to running bare metal with our hands again,” but rather that it would slip below the surface-level of awareness. And I don't know that I got the timing quite right on that, I think it's going to depend on the company and the culture that you find yourself in. But increasingly, when there's an application to run, it's easy to ask someone just, “Oh, great. Where's the Kubernetes cluster live so we can throw this on there and just add it to the rest of the pile?”That is sort of what I was seeing. My intention with that was not purely just to be controversial, as much fun as that might be, but also to act as a bit of a warning, where I've known too many people who let their identities become inextricably tangled with the technology. But technologies rise and fall, and at some point—like, you talk about configuration management days; I learned to speak publicly as a traveling trainer for Puppet. I wrote part of SaltStack once upon a time. But it was clear that that was not the direction the industry was going, so it was time to find something else to focus on. And I fear for people who don't keep an awareness or their feet underneath them and pay attention to broader market trends.Kelsey: Yeah, I think whenever I was personally caught up in linking my identity to technology, like, “I'm a Rubyist,” right?“, I'm a Puppeteer,” and you wear those names proudly. But I remember just thinking to myself, like, “You have to take a step back. What's more important, you or the technology?” And at some point, I realized, like, it's me, that is more important, right? Like, my independent thinking on this, my independent experience with this is far more important than the success of this thing.But also, I think there's a component there. Like when you talked about Kubernetes, you know, maybe being less relevant in five years, there's two things there. One is the success of all infrastructure things equals irrelevancy. When flights don't crash, when bridges just work, you do not think about them. You just use them because they're so stable and they become very boring. That is the success criteria.Corey: Utilities. No one's wondering if the faucet's going to work when they turn it on in the morning.Kelsey: Yeah. So, you know, there's a couple of ways to look at your statement. One is, you believe Kubernetes is on the trajectory that it's going to stabilize itself and hit that success criteria, and then it will be irrelevant. Or there's another part of the irrelevancy where something else comes along and replaces that thing, right? I think Cloud Foundry and Mesos are two good examples of Kubernetes coming along and stealing all of the attention from that because those particular products never gained that mass adoption. Maybe they got to the stable part, but they never got to the mass adoption part. So, I think when it comes to infrastructure, it's going to be irrelevant. It's just what side of that [laugh] coin do you land on?Corey: It's similar to folks who used to have to work at a variety of different companies on very specific Linux kernel subsystems because everyone had to care because there were significant performance impacts. Time went on and now there's still a few of those people that very much need to care, but for the rest of us, it is below the level of things that we have to care about. For me, the signs of the unsustainability were, oh, you can run Kubernetes effectively in production? That's a minimum of a quarter-million dollars a year in comp or up in some cases. Not every company is going to be able to field a team of those people and still remain a going concern in business. Nor frankly, should they have to.Kelsey: I'm going to pull on that thread a little bit because it's about—we're hitting that ten-year mark of Kubernetes. So, when Kubernetes comes out, why were people drawn to it, right? Why did it even get the time of day to begin with? And I think Docker kind of opened Pandora's box there. This idea of Chef, Puppet, Ansible, ten thousand package managers, and honestly, that trajectory was going to continue forever and it was helping no one. It was literally people doing duplicate work depending on the operating system you're dealing with and we were wasting time copying bits to servers—literally—in a very glorified way.So, Docker comes along and gives us this nicer, better abstraction, but it has gaps. It has no orchestration. It's literally this thing where now we've unified the packaging situation, we've learned a lot from Red Hat, YUM, Debian, and the various package repo combinations out there and so we made this universal thing. Great. We also learned a little bit about orchestration through brute force, bash scripts, config management, you name it, and so we serialized that all into this thing we call Kubernetes.It's pretty simple on the surface, but it was probably never worthy of such fanfare, right? But I think a lot of people were relieved that now we finally commoditized this expertise that the Googles, the Facebooks of the world had, right, building these systems that can copy bits to other systems very fast. There you go. We've gotten that piece. But I think what the market actually wants is in the mobile space, if you want to ship software to 300 million people that you don't even know, you can do it with the app store.There's this appetite that the boring stuff should be easy. Let's Encrypt has made SSL certificates beyond easy. It's just so easy to do the right thing. And I think for this problem we call deployments—you know, shipping apps around—at some point we have to get to a point where that is just crazy easy. And it still isn't.So, I think some of the frustration people express ten years later, they're realizing that they're trying to recreate a Rube Goldberg machine with Kubernetes is the base element and we still haven't understood that this whole thing needs to simplify, not ten thousand new pieces so you can build your own adventure.Corey: It's the idea almost of what I'm seeing AWS go through, and to some extent, its large competitors. But building anything on top of AWS from scratch these days is still reminiscent of going to Home Depot—or any hardware store—and walking up and down the aisles and getting all the different components to piece together what you want. Sometimes just want to buy something from Target that's already assembled and you have to do all of that work. I'm not saying there isn't value to having a Home Depot down the street, but it's also not the panacea that solves for all use cases. An awful lot of customers just want to get the job done and I feel that if we cling too tightly to how things used to be, we lose it.Kelsey: I'm going to tell you, being in the cloud business for almost eight years, it's the customers that create this. Now, I'm not blaming the customer, but when you start dealing with thousands of customers with tons of money, you end up in a very different situation. You can have one customer willing to pay you a billion dollars a year and they will dictate things that apply to no one else. “We want this particular set of features that only we will use.” And for a billion bucks a year times ten years, it's probably worth from a business standpoint to add that feature.Now, do this times 500 customers, each major provider. What you end up with is a cloud console that is unbearable, right? Because they also want these things to be first-class citizens. There's always smaller companies trying to mimic larger peers in their segment that you just end up in that chaos machine of unbound features forever. I don't know how to stop it. Unless you really come out maybe more Apple style and you tell people, “This is the one and only true way to do things and if you don't like it, you have to go find an alternative.” The cloud business, I think, still deals with the, “If you have a large payment, we will build it.”Corey: I think that that is a perspective that is not appreciated until you've been in the position of watching how large enterprises really interact with each other. Because it's, “Well, what customer the world is asking for yet another way to run containers?” “Uh, this specific one and their constraints are valid.” Every time I think I've seen everything there is to see in the world of cloud, I just have to go talk to one more customer and I'm learning something new. It's inevitable.I just wish that there was a better way to explain some of this to newcomers, when they're looking at, “Oh, I'm going to learn how this cloud thing works. Oh, my stars, look at how many services there are.” And then they wind up getting lost with analysis paralysis, and every time they get started and ask someone for help, they're pushed in a completely different direction and you keep spinning your wheels getting told to start over time and time again when any of these things can be made to work. But getting there is often harder than it really should be.Kelsey: Yeah. I mean, I think a lot of people don't realize how far you can get with, like, three VMs, a load balancer, and Postgres. My guess is you can probably build pretty much any clone of any service we use today with at least 1 million customers. Most people never reached that level—I don't even want to say the word scale—but that blueprint is there and most people will probably be better served by that level of simplicity than trying to mimic the behaviors of large customers—or large companies—with these elaborate use cases. I don't think they understand the context there. A lot of that stuff is baggage. It's not [laugh] even, like, best-of-breed or great design. It's like happenstance from 20 years of trying to buy everything that's been sold to you.Corey: I agree with that idea wholeheartedly. I was surprising someone the other day when I said that if you were to give me a task of getting some random application up and running by tomorrow, I do a traditional three-tier architecture, some virtual machines, a load balancer, and a database service. And is that the way that all the cool kids are doing it today? Well, they're not talking about it, but mostly. But the point is, is that it's what I know, it's where my background is, and the thing you already know when you're trying to solve a new problem is incredibly helpful, rather than trying to learn everything along that new path that you're forging down. Is that architecture the best approach? No, but it's perfectly sufficient for an awful lot of stuff.Kelsey: Yeah. And so, I mean, look, I've benefited my whole career from people fantasizing about [laugh] infrastructure—Corey: [laugh].Kelsey: And the truth is that in 2023, this stuff is so powerful that you can do almost anything you want to do with the simplest architecture that's available to us. The three-tier architecture has actually gotten better over the years. I think people are forgotten: CPUs are faster, RAM is much bigger quantities, the networks are faster, right, these databases can store more data than ever. It's so good to learn the fundamentals, start there, and worst case, you have a sound architecture people can reason about, and then you can go jump into the deep end, once you learn how to swim.Corey: I think that people would be depressed to understand just how much the common case for the value that Kubernetes brings is, “Oh yeah, now we can lose a drive or a server and the application stays up.” It feels like it's a bit overkill for that one somewhat paltry use case, but that problem has been hounding companies for decades.Kelsey: Yeah, I think at some point, the whole ‘SSH is my only interface into these kinds of systems,' that's a little low level, that's a little bare bones, and there will probably be a feature now where we start to have this not Infrastructure as Code, not cloud where we put infrastructure behind APIs and you pay per use, but I think what Kubernetes hints at is a future where you have APIs that do something. Right now the APIs give you pieces so you can assemble things. In the future, the APIs will just do something, “Run this app. I need it to be available and here's my money budget, my security budget, and reliability budget.” And then that thing will say, “Okay, we know how to do that, and here's roughly what is going to cost.”And I think that's what people actually want because that's how requests actually come down from humans, right? We say, “We want this app or this game to be played by millions of people from Australia to New York.” And then for a person with experience, that means something. You kind of know what architecture you need for that, you know what pieces that need to go there. So, we're just moving into a realm where we're going to have APIs that do things all of a sudden.And so, Kubernetes is the warm-up to that era. And that's why I think that transition is a little rough because it leaks the pieces part, so where you can kind of build all the pieces that you want. But we know what's coming. Serverless also hints at this. But that's what people should be looking for: APIs that actually do something.Corey: This episode is sponsored in part by Panoptica.  Panoptica simplifies container deployment, monitoring, and security, protecting the entire application stack from build to runtime. Scalable across clusters and multi-cloud environments, Panoptica secures containers, serverless APIs, and Kubernetes with a unified view, reducing operational complexity and promoting collaboration by integrating with commonly used developer, SRE, and SecOps tools. Panoptica ensures compliance with regulatory mandates and CIS benchmarks for best practice conformity. Privacy teams can monitor API traffic and identify sensitive data, while identifying open-source components vulnerable to attacks that require patching. Proactively addressing security issues with Panoptica allows businesses to focus on mitigating critical risks and protecting their interests. Learn more about Panoptica today at panoptica.app.Corey: You started the show by talking about how your career began with translating COBOL into Python. I firmly believe someone starting their career today listening to this could absolutely find that by the time their career starts drawing to their own close, that Kubernetes is right in there as far as sounding like the deprecated thing that no one really talks about or thinks about anymore. And I hope so. I want the future to be brighter than the past. I want getting a business or getting software together in a way that helps people to not require the amount of, “First, spend six weeks at a boot camp,” or, “Learn how to write just enough code that you can wind up getting funding and then have it torn apart.”What's the drag-and-drop story? What's the describe the application to a robot and it builds it for you? I'm optimistic about the future of infrastructure, just because based upon its power to potentially make reliability and scale available to folks who have no idea of what's involved with that. That's kind of the point. That's the end game of having won this space.Kelsey: Well, you know what? Kubernetes is providing the metadata to make that possible, right? Like in the early days, people were writing one-off scripts or, you know, writing little for loops to get things in the right place. And then we get config management that kind of formalizes that, but it still had no metadata, right? You'd have things like Puppet report information.But in the world of, like, Kubernetes, or any cloud provider, now you get semantic meaning. “This app needs this volume with this much space with this much memory, I need three of these behind this load balancer with these protocols enabled.” There is now so much metadata about applications, their life cycles, and how they work that if you were to design a new system, you can actually use that data to craft a much better API that made a lot of this boilerplate the defaults. Oh, that's a web application. You do not need to specify all of this boilerplate. Now, we can give you much better nouns and verbs to describe what needs to happen.So, I think this is that transition as all the new people coming up, they're going to be dealing with semantic meaning to infrastructure, where we were dealing with, like, tribal knowledge and intuition, right? “Run this script, pipe it to this thing, and then this should happen. And if it doesn't, run the script again with this flag.” Versus, “Oh, here's the semantic meaning to a working system.” That's a game-changer.Corey: One other topic I wanted to ask you about—I've it's been on my list of things to bring up the next time I ran into you and then you went ahead and retired, making it harder to run into you. But a little while back, I was at a tech conference and someone gave a demo, and it didn't go as well as they had hoped. And a few of us were talking about it afterwards. We've all been speakers, we've all lived that life. Zero shade.But someone brought you up in particular—unprompted; your legend does precede you—and the phrase that they used was that Kelsey's demos were always picture-perfect. He was so lucky with how the demos worked out. And I just have to ask—because you don't strike me as someone who is not careful, particularly when all eyes are upon you—and real experts make things look easy, did you have demos periodically go wrong that the audience just didn't see going wrong along the way? Or did you just actually YOLO all of your demos and got super lucky every single time for the last eight years?Kelsey: There was a musician who said, “Hey, your demos are like jazz. You improvise the whole thing.” There's no script, there's no video. The way I look at the demo is, like, you got this instrument, the command prompt, and the web browser. You can do whatever you want with them.Now, I have working code. I wrote the code, I wrote the deployment scenarios, I delete it all and I put it all back. And so, I know how it's supposed to work from the ground up. And so, what that means is if anything goes wrong, I can improvise. I could go into fixing the code. I can go into doing a redeploy.And I'll give you one good example. The first time Kubernetes came out, there was this small meetup in San Francisco with just the core contributors, right? So, there is no community yet, there's no conference yet, just people hacking on Kubernetes. And so, we decided, we're going to have the first Kubernetes meetup. And everyone got, like, six, seven minutes, max. That's it. You got to move.And so, I was like, “Hey, I noticed that in the lineup, there is no ‘What is Kubernetes?' talk. We're just getting into these nuts and bolts and I don't think that's fair to the people that will be watching this for the first time.” And I said, “All right, Kelsey, you should give maybe an intro to what it is.” I was like, “You know what I'll do? I'm going to build a Kubernetes cluster from the ground up, starting with VMs on my laptop.”And I'm in it and I'm feeling confident. So, confidence is the part that makes it look good, right? Where you're confident in the commands you type. One thing I learned to do is just use your history, just hit the up arrow instead of trying to copy all these things out. So, you hit the up arrow, you find the right command and you talk through it and no one looks at what's happening. You're cycling through the history.Or you have multiple tabs where you know the next up arrow is the right history. So, you give yourself shortcuts. And so, I'm halfway through this demo. We got three minutes left, and it doesn't work. Like, VMware is doing something weird on my laptop and there's a guy calling me off stage, like, “Hey, that's it. Cut it now. You're done.”I'm like, “Oh, nope. Thou shalt not go out like this.” It's time to improvise. And so, I said, “Hey, who wants to see me finish this?” And now everyone is locked in. It's dead silent. And I blow the whole thing away. I bring up the VMs, I [pixie 00:28:20] boot, I installed the kubelet, I install Docker. And everyone's clapping. And it's up, it's going, and I say, “Now, if all of this works, we run this command and it should start running the app.” And I do kubectl apply-f and it comes up and the place goes crazy.And I had more to the demo. But you stop. You've gotten the point across, right? This is what Kubernetes is, here's how it works, and look how you do it from scratch. And I remember saying, “And that's the end of my presentation.” You need to know when to stop, you need to know when to pivot, and you need to have confidence that it's supposed to work, and if you've seen it work a couple of times, your confidence is unshaken.And when I walked off that stage, I remember someone from Red Hat was like—Clayton Coleman; that's his name—Clayton Coleman walked up to me and said, “You planned that. You planned it to fail just like that, so you can show people how to go from scratch all the way up. That was brilliant.” And I was like, “Sure. That's exactly what I did.”Corey: “Yeah, I meant to do that.” I like that approach. I found there's always things I have to plan for in demos. For example, I can never count on having solid WiFi from a conference hall. The show has to go on. It's, okay, the WiFi doesn't work. I've at one point had to give a talk where the projector just wasn't working to a bunch of students. So okay, close the laptop. We're turning this into a bunch of question-and-answer sessions, and it was one of the better talks I've ever given.But the alternative is getting stuck in how you think a talk absolutely needs to go. Now, keynotes are a little harder where everything has been scripted and choreographed and at that point, I've had multiple fallbacks for demos that I've had to switch between. And people never noticed I was doing it for that exact reason. But it takes work to look polished.Kelsey: I will tell you that the last Next keynote I gave was completely irresponsible. No dry runs, no rehearsals, no table reads, no speaker notes. And I think there were 30,000 people at that particular Next. And Diane Greene was still CEO, and I remember when marketing was like, “Yo, at least a backup recording.” I was like, “Nah, I don't have anything.”And that demo was extensive. I mean, I was building an app from scratch, starting with Postgres, adding the schema, building an app, deploying the app. And something went wrong halfway. And there's this joke that I came up with just to pass over the time, they gave me a new Chromebook to do the demo. And so, it's not mine, so none of the default settings were there, I was getting pop-ups all over the place.And I came up with this joke on the way to the conference. I was like, “You know what'd be cool? When I show off the serverless stuff, I would just copy the code from Stack Overflow. That'd be like a really cool joke to say this is what senior engineers do.” And I go to Stack Overflow and it's getting all of these pop-ups and my mouse couldn't highlight the text.So, I'm sitting there like a deer in headlights in front of all of these people and I'm looking down, and marketing is, like, “This is what… this is what we're talking about.” And so, I'm like, “Man do I have to end this thing here?” And I remember I kept trying, I kept trying, and came to me. Once the mouse finally got in there and I cleared up all the popups, I just came up with this joke. I said, “Good developers copy.” And I switched over to my terminal and I took the text from Stack Overflow and I said, “Great developers paste,” and the whole room start laughing.And I had them back. And we kept going and continued. And at the end, there was like this Google Assistant, and when it was finished, I said, “Thank you,” to the Google Assistant and it was talking back through the live system. And it said, “I got to admit, that was kind of dope.” So, I go to the back and Diane Greene walks back there—the CEO of Google Cloud—and she pats me on the shoulder. “Kelsey, that was dope.”But it was the thrill because I had as much thrill as the people watching it. So, in real-time, I was going through all these emotions. But I think people forget, the demo is supposed to convey something. The demo is supposed to tell some story. And I've seen people overdo their demos with way too much code, way too many commands, almost if they're trying to show off their expertise versus telling a story. And so, when I think about the demo, it has to complement the entire narrative. And so, sometimes you don't need as many commands, you don't need as much code. You can keep things simple and that gives you a lot more ins and outs in case something does go crazy.Corey: And I think the key takeaway here that so many people lose sight of is you have to know the material well enough that whatever happens, well, things don't always go the way I planned during the day, either, and talking through that is something that I think serves as a good example. It feels like a bit more of a challenge when you're trying to demo something that a company is trying to sell someone, “Oh, yeah, it didn't work. But that's okay.” But I'm still reminded by probably one of the best conference demo fails I've ever seen on video. One day, someone was attempting to do a talk that hit Amazon S3 and it didn't work.And the audience started shouting at him that yeah, S3 is down right now. Because that was the big day that S3 took a nap for four hours. It was one of those foundational things you'd should never stop to consider. Like, well, what if the internet doesn't work tomorrow when I'm doing my demo? That's a tough one to work around. But rough timing.Kelsey: [breathy sound]Corey: He nailed the rest of the talk, though. You keep going. That's the thing that people miss. They get stuck in the demo that isn't working, they expect the audience knows as much as they do about what's supposed to happen next. You're the one up there telling a story. People forget it's storytelling.Kelsey: Now, I will be remiss to say, I know that the demo gods have been on my side for, like, ten, maybe fifteen years solid. So, I retired from doing live demos. This is why I just don't do them anymore. I know I'm overdue as an understatement. But the thing I've learned though, is that what I found more impressive than the live demo is to be able to convey the same narratives through story alone. No slides. No demo. Nothing. But you can still make people feel where you would try to go with that live demo.And it's insanely hard, especially for technologies people have never seen before. But that's that new challenge that I kind of set up for myself. So, if you see me at a keynote and you've noticed why I've been choosing these fireside chats, it's mainly because I'm also trying to increase my ability to share narrative, technical concepts, but now in a new form. So, this new storytelling format through the fireside chat has been my substitute for the live demo, normally because I think sometimes, unless there's something really to show that people haven't seen before, the live demo isn't as powerful to me. Once the thing is kind of known… the live demo is kind of more of the same. So, I think they really work well when people literally have never seen the thing before, but outside of that, I think you can kind of move on to, like, real-life scenarios and narratives that help people understand the fundamentals and the philosophy behind the tech.Corey: An awful lot of tools and tech that we use on a day-to-day basis as well are thankfully optimized for the people using them and the ergonomics of going about your day. That is orthogonal, in my experience, to looking very impressive on stage. It's the rare company that can have a product that not only works well but also presents well. And that is something I don't tend to index on when I'm selecting a tool to do something with. So, it's always a question of how can I make this more visually entertaining? For while I got out of doing demos entirely, just because talking about things that have more staying power than a screenshot that is going to wind up being irrelevant the next week when they decide to redo the console for some service yet again.Kelsey: But you know what? That was my secret to doing software products and projects. When I was at CoreOS, we used to have these meetups we would used to do every two weeks or so. So, when we were building things like etcd, Fleet was a container management platform that came before Kubernetes, we would always run through them as a user, start install them, use them, and ask how does it feel? These command line flags, they don't feel right. This isn't a narrative you can present with the software alone.But once we could, then the meetups were that much more engaging. Like hey, have you ever tried to distribute configuration to, like, a thousand servers? It's insanely hard. Here's how you do with Puppet. But now I'm going to show you how you do with etcd. And then the narrative will kind of take care of itself because the tool was positioned behind what people would actually do with it versus what the tool could do by itself.Corey: I think that's the missing piece that most marketing doesn't seem to quite grasp is, they talk about the tool and how awesome it is, but that's why I love customer demos so much. They're showing us how they use a tool to solve a real-world problem. And honestly, from my snarky side of the world and the attendant perspective there, I can make an awful lot of fun about basically anything a company decides to show me, but put a customer on stage talking about how whatever they've built is solving a real-world problem for them, that's the point where I generally shut up and listen because I'm going to learn something about a real-world story. Because you don't generally get to tell customers to go on stage and just make up a story that makes us sound good, and have it come off with any sense of reality whatsoever. I haven't seen that one happen yet, but I'm sure it's out there somewhere.Kelsey: I don't know how many founders or people building companies listen in to your podcast, but this is right now, I think the number one problem that especially venture-backed startups have. They tend to have great technology—maybe it's based off some open-source project—with tons of users who just know how that tool works, it's just an ingredient into what they're already trying to do. But that isn't going to ever be your entire customer base. Soon, you'll deal with customers who don't understand the thing you have and they need more than technology, right? They need a product.And most of these companies struggle painting that picture. Here's what you can do with it. Or here's what you can't do now, but you will be able to do if you were to use this. And since they are missing that, a lot of these companies, they produce a lot of code, they ship a lot of open-source stuff, they raise a lot of capital, and then it just goes away, it fades out over time because they can bring on no newcomers. The people who need help the most, they don't have a narrative for them, and so therefore, they're just hoping that the people who have all the skills in the world, the early adopters, but unfortunately, those people are tend to be the ones that don't actually pay. They just kind of do it themselves. It's the people who need the most help.Corey: How do we monetize the bleeding edge of adoption? In many cases you don't. They become your community if you don't hug them to death first.Kelsey: Exactly.Corey: Ugh. None of this is easy. I really want to thank you for taking the time to catch up and talk about how you seen the remains of a career well spent, and now you're going off into that glorious sunset. But I have a sneaking suspicion you'll still be around. Where should people go if they want to follow up on what you're up to these days?Kelsey: Right now I still use… I'm going to keep calling it Twitter.Corey: I agree.Kelsey: I kind of use that for my real-time interactions. And I'm still attending conferences, doing fireside chats, and just meeting people on those conference floors. But that's what where I'll be for now. So yeah, I'll still be around, but maybe not as deep. And I'll be spending more time just doing normal life stuff, maybe less building software.Corey: And we will, of course, put a link to that in the show notes. Thank you so much for taking the time to catch up and share your reflections on how the industry is progressing.Kelsey: Awesome. Thanks for having me, Corey.Corey: Kelsey Hightower, now gloriously retired. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment that you're going to type on stage as part of a conference talk, and then accidentally typo all over yourself while you're doing it.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

Legacy-Dads Podcast
"Students Standing Strong" with Terry Ann Kelly

Legacy-Dads Podcast

Play Episode Listen Later Aug 28, 2023 53:11


In this episode, Dave and Dante sit down with Terry Ann Kelly, the president and founder of Students Standing Strong. S3 is a ministry focused on providing students an opportunity to lead an on-campus Bible club which empowers students to stand strong in their faith, no matter the grade level. Terry Ann gives the background behind answering God's call to create the ministry and talks about why it is important for parents to encourage their kids to grow in their faith - to set them up for the future. God created us to be in community with each other...and that includes our school-age kids! Listen in to hear the passion behind Terry Ann's ministry and get involved today at https://www.mys3.org. If you have any questions, reach out to Terry Ann at terryann@mys3.org where she will respond! Legacy Dads is proud to be partnered with the Christian Parenting Network of Podcasts. For more practical and spiritual resources to help you become the perfectly imperfect parent you wish to be, visit https://www.christianparenting.org today!

AWS Podcast
#616: Mountpoint for Amazon S3

AWS Podcast

Play Episode Listen Later Aug 28, 2023 17:43


Mountpoint for Amazon S3 is a new open source file client that you can use to mount an S3 bucket on your compute instance and access it as a local file system. In this episode, Simon is joined by Devabrat Kumar, Sr. Product Manager for Amazon S3, to discuss the launch of Mountpoint for Amazon S3 and how customer feedback has driven the evolution of this feature. Mountpoint blog: https://bit.ly/3skIddG Mountpoint website: https://bit.ly/45qWZ15 Mountpoint user guide: https://bit.ly/45qK7YV

airhacks.fm podcast with adam bien
Why MicroStream is Faster

airhacks.fm podcast with adam bien

Play Episode Listen Later Aug 27, 2023 61:04


An airhacks.fm conversation with Florian Habermann (@FHHabermann) about: CPC Schneider / Amstrad, playing with Basic and sound, building an 3d engine in BASIC, from BASIC to Java, the private school: BSZ Wiesau, ObjectStore, Versant, Poet, Object database, moving the IDE to Eclipse, using Vaadin as frontend framework, RapidClipse, Markus Kett on airhacks.fm: "#36 Java Native Database", "#116 MicroStream: When a Java Application Becomes a DB", the object-relation impedance mismatch, Object-Relational Mapping is the Vietnam of Computer Science, JetStream became microstream, Java Serializer only supports a complete snapshot, MicroStream supports partial serialization, FileMaker - productivity for non-programmers, using sun.mics.Unsafe, id to object mapping with SwissLink, cloud-native storage with S3, DynamoDB and MicroStream, Universally Unique Lexicographically Sortable Identifier: ulid, managing object versions with microstream.one Florian Habermann on twitter: @FHHabermann

The Cloud Pod
224: The Cloud Pod Adopts the BS License

The Cloud Pod

Play Episode Listen Later Aug 25, 2023 54:46


Welcome to episode 224 of The CloudPod Podcast - where the forecast is always cloudy! This week, your hosts Justin, Jonathan, and Ryan discuss some major changes at Terraform, including switching from open source to a BSL License. Additionally, we cover updates to Amazon S3, goodies from Storage Day, and Google Gemini vs. Open AI.  Titles we almost went with this week: None! This week's title was ✨chef's kiss✨ A big thanks to this week's sponsor: Foghorn Consulting provides top-notch cloud and DevOps engineers to the world's most innovative companies. Initiatives stalled because you have trouble hiring?  Foghorn can be burning down your DevOps and Cloud backlogs as soon as next week.

Feddie Scum
S4E5 - S3 Episode 5

Feddie Scum

Play Episode Listen Later Aug 22, 2023 60:20


S3 episode 5 - Ghosts in the Dark of Space - Now that the murder was solved and the murderer found, It's time for a good ol' fashioned interrogation episode! Preorder the Zaku Bell shirt, now for a limited time! - https://deadsetmedia.bigcartel.com/ Edited by Dallas Welk and Chris Ramey - Feddie Scum - The Gundam RPG Podcast Twitter - https://twitter.com/FeddieScum Facebook - https://www.facebook.com/feddiescum Youtube - https://www.youtube.com/channel/UCWptBHBc3no9bht-EpZO8wQ Support us on Patreon! - https://www.patreon.com/deadsetmedia Check out our merch store! - https://deadsetmedia.bigcartel.com/ Find out more at https://feddiescum.pinecast.co

Chile Today Podcast
S3 episode 18: August News!

Chile Today Podcast

Play Episode Listen Later Aug 21, 2023 88:36


S3 episode 18: August News! While Lenny is out on vacation, Bethany and Pingüino take over to make sure you're up-to-date with the latest news in Chile. From Cabinet changes to DRAMA good enough to be its own telenovela. Tune in!Support the show

Strong Women
S4 1: Bringing Christ's Love and Light to Women in Strip Clubs With Rachelle Starr

Strong Women

Play Episode Listen Later Aug 16, 2023 40:58


Our guest today is Rachelle Starr, founder of Scarlet Hope. At 21, Rachelle felt God calling her to minister to women trapped working in strip clubs in her city of Louisville, KY. Today, she shares some of the stories she's witnessed over 16 years of building relationships through good food and introducing these women to the God who sees and loves them. Scarlet Hope is now in many major cities around the nation, working to bring the hope of the Gospel to places that are often overlooked.    Rachelle's Website Scarlet Hope  Outrageous Obedience: Answering God's Call to Shine in the Darkest Places by Rachelle Starr  Resilient: Restoring Your Weary Soul in These Turbulent Times by John Eldredge  Discernment by Henri J.M. Nouwen  The Way of the Heart by Henri J.M. Nouwen  Tramp for the Lord by Corrie Ten Boom  S3 27: Looking Back and Looking Forward: Reflecting on Three Years of the Strong Women Podcast With Erin Kunkle     In the fight against suicide, the Church has a vital role to play, but what does that look like in action? Our next Lighthouse Voices event on September 5 will help you answer that question. Our featured speaker is Dr. Matthew Sleeth, author of Hope Always: How to Be a Force for Life in a Culture of Suicide. Dr. Sleeth will help us understand the current suicide epidemic and share how Christians can fight against it and care for those who are struggling. This event will be offered both in-person and online, and you can register now at colsoncenter.org/lighthousevoices.  To help you live the Truth in a “my truth” world, we want to send you a copy of Alisa Childers' latest book, Live Your Truth and Other Lies. Request your copy by giving a gift of any amount to the Colson Center in August at colsoncenter.org/swchildersresource.  The Strong Women Podcast is a product of the Colson Center which equips Christians to live out their faith with clarity, confidence, and courage in this cultural moment. Through commentaries, podcasts, videos, and more, we help Christians better understand what's happening in the world, and champion what is true and good wherever God has called them.  Learn more about the Colson Center here: https://www.colsoncenter.org/   Visit our website and sign up for our email list so that you can stay up to date on what we are doing here and also receive our monthly book list: https://www.colsoncenter.org/strong-women  Join Strong Women on Social Media:   https://www.facebook.com/StrongWomenCC  https://www.facebook.com/groups/strongwomencommunitycc/  https://www.instagram.com/strongwomencc/ 

The Changelog
The relicensings will continue until morale improves

The Changelog

Play Episode Listen Later Aug 14, 2023 8:04 Transcription Available


HashiCorp adopts a Business Source license, Matt Rickard hypothesizes why Tailwind CSS won, WarpStream sets out to make a Kafka-compatible offering directly on S3, Vadim Kravcenko publishes an excellent guide for managing difficult software engineers & Russ Cox gives an update on Go 2.

The Changelog
Kaizen! S3 R2 B2 D2

The Changelog

Play Episode Listen Later Aug 11, 2023 79:15


Gerhard joins us for the 11th Kaizen and this one might contain the most improvements ever. We're on Fly Apps V2, we've moved from S3 to R2 & we have a status page now, just to name a few.

kaizen s3 gerhard r2 adam stacoviak jerod santo
Screaming in the Cloud
How Cloudflare is Working to Fix the Internet with Matthew Prince

Screaming in the Cloud

Play Episode Listen Later Aug 10, 2023 42:30


Matthew Prince, Co-founder & CEO at Cloudflare, joins Corey on Screaming in the Cloud to discuss how and why Cloudflare is working to solve some of the Internet's biggest problems. Matthew reveals some of his biggest issues with cloud providers, including the tendency to charge more for egress than ingress and the fact that the various clouds don't compete on a feature vs. feature basis. Corey and Matthew also discuss how Cloudflare is working to change those issues so the Internet is a better and more secure place. Matthew also discusses how transparency has been key to winning trust in the community and among Cloudflare's customers, and how he hopes the Internet and cloud providers will evolve over time.About MatthewMatthew Prince is co-founder and CEO of Cloudflare. Cloudflare's mission is to help build a better Internet. Today the company runs one of the world's largest networks, which spans more than 200 cities in over 100 countries. Matthew is a World Economic Forum Technology Pioneer, a member of the Council on Foreign Relations, winner of the 2011 Tech Fellow Award, and serves on the Board of Advisors for the Center for Information Technology and Privacy Law. Matthew holds an MBA from Harvard Business School where he was a George F. Baker Scholar and awarded the Dubilier Prize for Entrepreneurship. He is a member of the Illinois Bar, and earned his J.D. from the University of Chicago and B.A. in English Literature and Computer Science from Trinity College. He's also the co-creator of Project Honey Pot, the largest community of webmasters tracking online fraud and abuse.Links Referenced: Cloudflare: https://www.cloudflare.com/ Twitter: https://twitter.com/eastdakota TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. One of the things we talk about here, an awful lot is cloud providers. There sure are a lot of them, and there's the usual suspects that you would tend to expect with to come up, and there are companies that work within their ecosystem. And then there are the enigmas.Today, I'm talking to returning guest Matthew Prince, Cloudflare CEO and co-founder, who… well first, welcome back, Matthew. I appreciate your taking the time to come and suffer the slings and arrows a second time.Matthew: Corey, thanks for having me.Corey: What I'm trying to do at the moment is figure out where Cloudflare lives in the context of the broad ecosystem because you folks have released an awful lot. You had this vaporware-style announcement of R2, which was an S3 competitor, that then turned out to be real. And oh, it's always interesting, when vapor congeals into something that actually exists. Cloudflare Workers have been around for a while and I find that they become more capable every time I turn around. You have Cloudflare Tunnel which, to my understanding, is effectively a VPN without the VPN overhead. And it feels that you are coming at building a cloud provider almost from the other side than the traditional cloud provider path. Is it accurate? Am I missing something obvious? How do you see yourselves?Matthew: Hey, you know, I think that, you know, you can often tell a lot about a company by what they measure and what they measure themselves by. And so, if you're at a traditional, you know, hyperscale public cloud, an AWS or a Microsoft Azure or a Google Cloud, the key KPI that they focus on is how much of a customer's data are they hoarding, effectively? They're all hoarding clouds, fundamentally. Whereas at Cloudflare, we focus on something of it's very different, which is, how effectively are we moving a customer's data from one place to another? And so, while the traditional hyperscale public clouds are all focused on keeping your data and making sure that they have as much of it, what we're really focused on is how do we make sure your data is wherever you need it to be and how do we connect all of the various things together?So, I think it's exactly right, where we start with a network and are kind of building more functions on top of that network, whereas other companies start really with a database—the traditional hyperscale public clouds—and the network is sort of an afterthought on top of it, just you know, a cost center on what they're delivering. And I think that describes a lot of the difference between us and everyone else. And so oftentimes, we work very much in conjunction with. A lot of our customers use hyperscale public clouds and Cloudflare, but increasingly, there are certain applications, there's certain data that just makes sense to live inside the network itself, and in those cases, customers are using things like R2, they're using our Workers platform in order to be able to build applications that will be available everywhere around the world and incredibly performant. And I think that is fundamentally the difference. We're all about moving data between places, making sure it's available everywhere, whereas the traditional hyperscale public clouds are all about hoarding that data in one place.Corey: I want to clarify that when you say hoard, I think of this, from my position as a cloud economist, as effectively in an economic story where hoarding the data, they get to charge you for hosting it, they get to charge you serious prices for egress. I've had people mishear that before in a variety of ways, usually distilled down to, “Oh, and their data mining all of their customers' data.” And I want to make sure that that's not the direction that you intend the term to be used. If it is, then great, we can talk about that, too. I just want to make sure that I don't get letters because God forbid we get letters for things that we say in the public.Matthew: No, I mean, I had an aunt who was a hoarder and she collected every piece of everything and stored it somewhere in her tiny little apartment in the panhandle of Florida. I don't think she looked at any of it and for the most part, I don't think that AWS or Google or Microsoft are really using your data in any way that's nefarious, but they're definitely not going to make it easy for you to get it out of those places; they're going to make it very, very expensive. And again, what they're measuring is how much of a customer's data are they holding onto whereas at Cloudflare we're measuring how much can we enable you to move your data around and connected wherever you need it. And again, I think that that kind of gets to the fundamental difference between how we think of the world and how I think the hyperscale public clouds thing of the world. And it also gets to where are the places where it makes sense to use Cloudflare, and where are the places that it makes sense to use an AWS or Google Cloud or Microsoft Azure.Corey: So, I have to ask, and this gets into the origin story trope a bit, but what radicalized you? For me, it was the realization one day that I could download two terabytes of data from S3 once, and it would cost significantly more than having Amazon.com ship me a two-terabyte hard drive from their store.Matthew: I think that—so Cloudflare started with the basic idea that the internet's not as good as it should be. If we all knew what the internet was going to be used for and what we're all going to depend on it for, we would have made very different decisions in how it was designed. And we would have made sure that security was built in from day one, we would have—you know, the internet is very reliable and available, but there are now airplanes that can't land if the internet goes offline, they are shopping transactions shut down if the internet goes offline. And so, I don't think we understood—we made it available to some extent, but not nearly to the level that we all now depend on it. And it wasn't as fast or as efficient as it possibly could be. It's still very dependent on the geography of where data is located.And so, Cloudflare started out by saying, “Can we fix that? Can we go back and effectively patch the internet and make it what it should have been when we set down the original protocols in the '60s, '70s, and '80s?” But can we go back and say, can we build a new, sort of, overlay on the internet that solves those problems: make it more secure, make it more reliable, make it faster and more efficient? And so, I think that that's where we started, and as a result of, again, starting from that place, it just made fundamental sense that our job was, how do you move data from one place to another and do it in all of those ways? And so, where I think that, again, the hyperscale public clouds measure themselves by how much of a customer's data are they hoarding; we measure ourselves by how easy are we making it to securely, reliably, and efficiently move any piece of data from one place to another.And so, I guess, that is radical compared to some of the business models of the traditional cloud providers, but it just seems like what the internet should be. And that's our North Star and that's what just continues to drive us and I think is a big reason why more and more customers continue to rely on Cloudflare.Corey: The thing that irks me potentially the most in the entire broad strokes of cloud is how the actions of the existing hyperscalers have reflected mostly what's going on in the larger world. Moore's law has been going on for something like 100 years now. And compute continues to get faster all the time. Storage continues to cost less year over year in a variety of ways. But they have, on some level, tricked an entire generation of businesses into believing that network bandwidth is this precious, very finite thing, and of course, it's going to be ridiculously expensive. You know, unless you're taking it inbound, in which case, oh, by all means back the truck around. It'll be great.So, I've talked to founders—or prospective founders—who had ideas but were firmly convinced that there was no economical way to build it. Because oh, if I were to start doing real-time video stuff, well, great, let's do the numbers on this. And hey, that'll be $50,000 a minute, if I read the pricing page correctly, it's like, well, you could get some discounts if you ask nicely, but it doesn't occur to them that they could wind up asking for a 98% discount on these things. Everything is measured in a per gigabyte dimension and that just becomes one of those things where people are starting to think about and meter something that—from my days in data centers where you care about the size of the pipe and not what's passing through it—to be the wrong way of thinking about things.Matthew: A little of this is that everybody is colored by their experience of dealing with their ISP at home. And in the United States, in a lot of the world, ISPs are built on the old cable infrastructure. And if you think about the cable infrastructure, when it was originally laid down, it was all one-directional. So, you know, if you were turning on cable in your house in a pre-internet world, data fl—Corey: Oh, you'd watch a show and your feedback was yelling at the TV, and that's okay. They would drop those packets.Matthew: And there was a tiny, tiny, tiny bit of data that would go back the other direction, but cable was one-directional. And so, it actually took an enormous amount of engineering to make cable bi-directional. And that's the reason why if you're using a traditional cable company as your ISP, typically you will have a large amount of download capacity, you'll have, you know, a 100 megabits of down capacity, but you might only have a 10th of that—so maybe ten megabits—of upload capacity. That is an artifact of the cable system. That is not just the natural way that the internet works.And the way that it is different, that wholesale bandwidth works, is that when you sign up for wholesale bandwidth—again, as you phrase it, you're not buying this many bytes that flows over the line; you're buying, effectively, a pipe. You know, the late Senator Ted Stevens said that the internet is just a series of tubes and got mocked mercilessly, but the internet is just a series of tubes. And when Cloudflare or AWS or Google or Microsoft buys one of those tubes, what they pay for is the diameter of the tube, the amount that can fit through it. And the nature of this is you don't just get one tube, you get two. One that is down and one that is up. And they're the same size.And so, if you've got a terabit of traffic coming down and zero going up, that costs exactly the same as a terabit going up and zero going down, which costs exactly the same as a terabit going down and a terabit going up. It is different than your home, you know, cable internet connection. And that's the thing that I think a lot of people don't understand. And so, as you pointed out, but the great tragedy of the cloud is that for nothing other than business reasons, these hyperscale public cloud companies don't charge you anything to accept data—even though that is actually the more expensive of the two operations for that because writes are more expensive than reads—but the inherent fact that they were able to suck the data in means that they have the capacity, at no additional cost, to be able to send that data back out. And so, I think that, you know, the good news is that you're starting to see some providers—so Cloudflare, we've never charged for egress because, again, we think that over time, bandwidth prices go to zero because it just makes sense; it makes sense for ISPs, it makes sense for connectiv—to be connected to us.And that's something that we can do, but even in the cases of the cloud providers where maybe they're all in one place and somebody has to pay to backhaul the traffic around the world, maybe there's some cost, but you're starting to see some pressure from some of the more forward-leaning providers. So Oracle, I think has done a good job of leaning in and showing how egress fees are just out of control. But it's crazy that in some cases, you have a 4,000x markup on AWS bandwidth fees. And that's assuming that they're paying the same rates as what we would get at Cloudflare, you know, even though we are a much smaller company than they are, and they should be able to get even better prices.Corey: Yes, if there's one thing Amazon is known for, it as being bad at negotiating. Yeah, sure it is. I'm sure that they're just a terrific joy to be a vendor to.Matthew: Yeah, and I think that fundamentally what the price of bandwidth is, is tied very closely to what the cost of a port on a router costs. And what we've seen over the course of the last ten years is that cost has just gone enormously down where the capacity of that port has gone way up and the just physical cost, the depreciated cost that port has gone down. And yet, when you look at Amazon, you just haven't seen a decrease in the cost of bandwidth that they're passing on to customers. And so, again, I think that this is one of the places where you're starting to see regulators pay attention, we've seen efforts in the EU to say whatever you charge to take data out is the same as what you should charge it to put data in. We're seeing the FTC start to look at this, and we're seeing customers that are saying that this is a purely anti-competitive action.And, you know, I think what would be the best and healthiest thing for the cloud by far is if we made it easy to move between various cloud providers. Because right now the choice is, do I use AWS or Google or Microsoft, whereas what I think any company out there really wants to be able to do is they want to be able to say, “I want to use this feature at AWS because they're really good at that and I want to use this other feature at Google because they're really good at that, and I want to us this other feature at Microsoft, and I want to mix and match between those various things.” And I think that if you actually got cloud providers to start competing on features as opposed to competing on their overall platform, we'd actually have a much richer and more robust cloud environment, where you'd see a significantly improved amount of what's going on, as opposed to what we have now, which is AWS being mediocre at everything.Corey: I think that there's also a story where for me, the egress is annoying, but so is the cross-region and so is the cross-AZ, which in many cases costs exactly the same. And that frustrates me from the perspective of, yes, if you have two data centers ten miles apart, there is some startup costs to you in running fiber between them, however you want to wind up with that working, but it's a sunk cost. But at the end of that, though, when you wind up continuing to charge on a per gigabyte basis to customers on that, you're making them decide on a very explicit trade-off of, do I care more about cost or do I care more about reliability? And it's always going to be an investment decision between those two things, but when you make the reasonable approach of well, okay, an availability zone rarely goes down, and then it does, you get castigated by everyone for, “Oh it even says in their best practice documents to go ahead and build it this way.” It's funny how a lot of the best practice documents wind up suggesting things that accrue primarily to a cloud provider's benefit. But that's the way of the world I suppose.I just know, there's a lot of customer frustration on it and in my client environments, it doesn't seem to be very acute until we tear apart a bill and look at where they're spending money, and on what, at which point, the dawning realization, you can watch it happen, where they suddenly realize exactly where their money is going—because it's relatively impenetrable without that—and then they get angry. And I feel like if people don't know what they're being charged for, on some level, you've messed up.Matthew: Yeah. So, there's cost to running a network, but there's no reason other than limiting competition why you would charge more to take data out than you would put data in. And that's a puzzle. The cross-region thing, you know, I think where we're seeing a lot of that is actually oftentimes, when you've got new technologies that come out and they need to take advantage of some scarce resource. And so, AI—and all the AI companies are a classic example of this—right now, if you're trying to build a model, an AI model, you are hunting the world for available GPUs at a reasonable price because there's an enormous scarcity of them.And so, you need to move from AWS East to AWS West, to AWS, you know, Singapore, to AWS in Luxembourg and bounce around to find wherever there's GPU availability. And then that is crossed against the fact that these training datasets are huge. You know, I mean, they're just massive, massive, massive amounts of data. And so, what that is doing is you're having these AI companies that are really seeing this get hit in the face, where they literally can't get the capacity they need because of the fact that whatever cloud provider in whatever region they've selected to store their data isn't able to have that capacity. And so, they're getting hit not only by sort of a double whammy of, “I need to move my data to wherever there's capacity. And if I don't do that, then I have to pay some premium, an ever-escalating price for the underlying GPUs.” And God forbid, you have to move from AWS to Google to chase that.And so, we're seeing a lot of companies that are saying, “This doesn't make any sense. We have this enormous training set. If we just put it with Cloudflare, this is data that makes sense to live in the network, fundamentally.” And not everything does. Like, we're not the right place to store your long-term transaction logs that you're only going to look at if you get sued. There are much better places, much more effective places do it.But in those cases where you've got to read data frequently, you've got to read it from different places around the world, and you will need to decrease what those costs of each one of those reads are, what we're seeing is just an enormous amount of demand for that. And I think these AI startups are really just a very clear example of what company after company after company needs, and why R2 has had—which is our zero egress cost S3 competitor—why that is just seeing such explosive growth from a broad set of customers.Corey: Because I enjoy pushing the bounds of how ridiculous I can be on the internet, I wound up grabbing a copy of the model, the Llama 2 model that Meta just released earlier this week as we're recording this. And it was great. It took a little while to download here. I have gigabit internet, so okay, it took some time. But then I wound up with something like 330 gigs of models. Great, awesome.Except for the fact that I do the math on that and just for me as one person to download that, had they been paying the listed price on the AWS website, they would have spent a bit over $30, just for me as one random user to download the model, once. If you can express that into the idea of this is a model that is absolutely perfect for whatever use case, but we want to have it run with some great GPUs available at another cloud provider. Let's move the model over there, ignoring the data it's operating on as well, it becomes completely untenable. It really strikes me as an anti-competitiveness issue.Matthew: Yeah. I think that's it. That's right. And that's just the model. To build that model, you would have literally millions of times more data that was feeding it. And so, the training sets for that model would be many, many, many, many, many, many orders of magnitude larger in terms of what's there. And so, I think the AI space is really illustrating where you have this scarce resource that you need to chase around the world, you have these enormous datasets, it's illustrating how these egress fees are actually holding back the ability for innovation to happen.And again, they are absolutely—there is no valid reason why you would charge more for egress than you do for ingress other than limiting competition. And I think the good news, again, is that's something that's gotten regulators' attention, that's something that's gotten customers' attention, and over time, I think we all benefit. And I think actually, AWS and Google and Microsoft actually become better if we start to have more competition on a feature-by-feature basis as opposed to on an overall platform. The choice shouldn't be, “I use AWS.” And any big company, like, nobody is all-in only on one cloud provider. Everyone is multi-cloud, whether they want to be or not because people end up buying another company or some skunkworks team goes off and uses some other function.So, you are across multiple different clouds, whether you want to be or not. But the ideal, and when I talk to customers, they want is, they want to say, “Well, you know that stuff that they're doing over at Microsoft with AI, that sounds really interesting. I want to use that, but I really like the maturity and robustness of some of the EC2 API, so I want to use that at AWS. And Google is still, you know, the best in the world at doing search and indexing and everything, so I want to use that as well, in order to build my application.” And the applications of the future will inherently stitch together different features from different cloud providers, different startups.And at Cloudflare, what we see is our, sort of, purpose for being is how do we make that stitching as easy as possible, as cost-effective as possible, and make it just make sense so that you have one consistent security layer? And again, we're not about hording the data; we're about connecting all of those things together. And again, you know, from the last time we talked to now, I'm actually much more optimistic that you're going to see, kind of, this revolution where egress prices go down, you get competition on feature-by-features, and that's just going to make every cloud provider better over the long-term.Corey: This episode is sponsored in part by Panoptica.  Panoptica simplifies container deployment, monitoring, and security, protecting the entire application stack from build to runtime. Scalable across clusters and multi-cloud environments, Panoptica secures containers, serverless APIs, and Kubernetes with a unified view, reducing operational complexity and promoting collaboration by integrating with commonly used developer, SRE, and SecOps tools. Panoptica ensures compliance with regulatory mandates and CIS benchmarks for best practice conformity. Privacy teams can monitor API traffic and identify sensitive data, while identifying open-source components vulnerable to attacks that require patching. Proactively addressing security issues with Panoptica allows businesses to focus on mitigating critical risks and protecting their interests. Learn more about Panoptica today at panoptica.app.Corey: I don't know that I would trust you folks to the long-term storage of critical data or the store of record on that. You don't have the track record on that as a company the way that you do for being the network interchange that makes everything just work together. There are areas where I'm thrilled to explore and see how it works, but it takes time, at least from the sensible infrastructure perspective of trusting people with track records on these things. And you clearly have the network track record on these things to make this stick. It almost—it seems unfair to you folks, but I view you as Cloudflare is a CDN, that also dabbles in a few other things here in there, though, increasingly, it seems it's CDN and security company are becoming synonymous.Matthew: It's interesting. I remember—and this really is going back to the origin story, but when we were starting Cloudflare, you know, what we saw was that, you know, we watched as software—starting with companies like Salesforce—transition from something that you bought in the box to something that you bought as a service [into 00:23:25] the cloud. We watched as, sort of, storage and compute transition from something that you bought from Dell or HP to something that you rented as a service. And so the fundamental problem that Cloudflare started out with was if the software and the storage and compute are going to move, inherently the security and the networking is going to move as well because it has to be as a service as well, there's no way you can buy a you know, Cisco firewall and stick it in front of your cloud service. You have to be in the cloud as well.So, we actually started very much as a security company. And the objection that everybody had to us as we would sort of go out and describe what we were planning on doing was, “You know, that sounds great, but you're going to slow everything down.” And so, we became just obsessed with latency. And Michelle, my co-founder, and I were business students and we had an advisor, a guy named Tom [Eisenmann 00:24:26] in business school. And I remember going in and that was his objection as well and so we did all this work to figure it out.And obviously, you know, I'd say computer science, and anytime that you have a problem around latency or speed caching is an obvious part of the solution to that. And so, we went in and we said, “Here's how we're going to do it: [unintelligible 00:24:47] all this protocol optimization stuff, and here's how we're going to distribute it around the world and get close to where users are. And we're going to use caching in the places where we can do caching.” And Tom said, “Oh, you're building a CDN.” And I remember looking at him and then I'm looking at Michelle. And Michelle is Canadian, and so I was like, “I don't know that I'm building a Canadian, but I guess. I don't know.”And then, you know, we walked out in the hall and Michelle looked at me and she's like, “We have to go figure out what the CDN thing is.” And we had no idea what a CDN was. And even when we learned about it, we were like, that business doesn't make any sense. Like because again, the CDNs were the first ones to really charge for bandwidth. And so today, we have effectively built, you know, a giant CDN and are the fastest in the world and do all those things.But we've always given it away basically for free because fundamentally, what we're trying to do is all that other stuff. And so, we actually started with security. Security is—you know, my—I've been working in security now for over 25 years and that's where my background comes from, and if you go back and look at what the original plan was, it was how do we provide that security as a service? And yeah, you need to have caching because caching makes sense. What I think is the difference is that in order to do that, in order to be able to build that, we had to build a set of developer tools for our own team to allow them to build things as quickly as possible.And, you know, if you look at Cloudflare, I think one of the things we're known for is just the rapid, rapid, rapid pace of innovation. And so, over time, customers would ask us, “How do you innovate so fast? How do you build things fast?” And part of the answer to that, there are lots of ways that we've been able to do that, but part of the answer to that is we built a developer platform for our own team, which was just incredibly flexible, allowed you to scale to almost any level, took care of a lot of that traditional SRE functions just behind the scenes without you having to think about it, and it allowed our team to be really fast. And our customers are like, “Wow, I want that too.”And so, customer after customer after customer after customer was asking and saying, you know, “We have those same problems. You know, if we're a big e-commerce player, we need to be able to build something that can scale up incredibly quickly, and we don't have to think about spinning up VMs or containers or whatever, we don't have to think about that. You know, our customers are around the world. We don't want to have to pick a region for where we're going to deploy code.” And so, where we built Cloudflare Workers for ourself first, customers really pushed us to make it available to them as well.And that's the way that almost any good developer platform starts out. That's how AWS started. That's how, you know, the Microsoft developer platform, and so the Apple developer platform, the Salesforce developer platform, they all start out as internal tools, and then someone says, “Can you expose this to us as well?” And that's where, you know, I think that we have built this. And again, it's very opinionated, it is right for certain applications, it's never going to be the right place to run SAP HANA, but the company that builds the tool [crosstalk 00:27:58]—Corey: I'm not convinced there is a right place to run SAP HANA, but that's probably unfair of me.Matthew: Yeah, but there is a startup out there, I guarantee you, that's building whatever the replacement for SAP HANA is. And I think it's a better than even bet that Cloudflare Workers is part of their stack because it solves a lot of those fundamental challenges. And that's been great because it is now allowing customer after customer after customer, big and large startups and multinationals, to do things that you just can't do with traditional legacy hyperscale public cloud. And so, I think we're sort of the next generation of building that. And again, I don't think we set out to build a developer platform for third parties, but we needed to build it for ourselves and that's how we built such an effective tool that now so many companies are relying on.Corey: As a Cloudflare customer myself, I think that one of the things that makes you folks standalone—it's why I included security as well as CDN is one of the things I trust you folks with—has been—Matthew: I still think CDN is Canadian. You will never see us use that term. It's like, Gartner was like, “You have to submit something for the CDN-like ser—” and we ended up, like, being absolute top-right in it. But it's a space that is inherently going to zero because again, if bandwidth is free, I'm not sure what—this is what the internet—how the internet should work. So yeah, anyway.Corey: I agree wholeheartedly. But what I've always enjoyed, and this is probably going to make me sound meaner than I intend it to, it has been your outages. Because when computers inherently at some point break, which is what they do, you personally and you as a company have both taken a tone that I don't want to say gleeful, but it's sort of the next closest thing to it regarding the postmortem that winds up getting published, the explanation of what caused it, the transparency is unheard of at companies that are your scale, where usually they want to talk about these things as little as possible. Whereas you've turned these into things that are educational to those of us who don't have the same scale to worry about but can take things from that are helpful. And that transparency just counts for so much when we're talking about things as critical as security.Matthew: I would definitely not describe it as gleeful. It is incredibly painful. And we, you know, we know we let customers down anytime we have an issue. But we tend not to make the same mistake twice. And the only way that we really can reliably do that is by being just as transparent as possible about exactly what happened.And we hope that others can learn from the mistakes that we made. And so, we own the mistakes we made and we talk about them and we're transparent, both internally but also externally when there's a problem. And it's really amazing to just see how much, you know, we've improved over time. So, it's actually interesting that, you know, if you look across—and we measure, we test and measure all the big hyperscale public clouds, what their availability and reliability is and measure ourselves against it, and across the board, second half of 2021 and into the first half of 2022 was the worst for every cloud provider in terms of reliability. And the question is why?And the answer is, Covid. I mean, the answer to most things over the last three years is in one way, directly or indirectly, Covid. But what happened over that period of time was that in April of 2020, internet traffic and traffic to our service and everyone who's like us doubled over the course of a two-week period. And there are not many utilities that you can imagine that if their usage doubles, that you wouldn't have a problem. Imagine the sewer system all of a sudden has twice as much sewage, or the electrical grid as twice as much demand, or the freeways have twice as many cars. Like, things break down.And especially the European internet came incredibly close to just completely failing at that time. And we all saw where our bottlenecks were. And what's interesting is actually the availability wasn't so bad in 2020 because people were—they understood the absolute critical importance that while we're in the middle of a pandemic, we had to make sure the internet worked. And so, we—there were a lot of sleepless nights, there's a—and not just at with us, but with every provider that's out there. We were all doing Herculean tasks in order to make sure that things came online.By the time we got to the sort of the second half of 2021, what everybody did, Cloudflare included, was we looked at it, and we said, “Okay, here were where the bottlenecks were. Here were the problems. What can we do to rearchitect our systems to do that?” And one of the things that we saw was that we effectively treated large data centers as one big block, and if you had certain pieces of equipment that failed in a way, that you would take that entire data center down and then that could have cascading effects across traffic as it shifted around across our network. And so, we did the work to say, “Let's take that one big data center and divide it effectively into multiple independent units, where you make sure that they're all on different power suppliers, you make sure they're all in different [crosstalk 00:32:52]”—Corey: [crosstalk 00:32:51] harder than it sounds. When you have redundant things, very often, the thing that takes you down the most is the heartbeat that determines whether something next to it is up or not. It gets a false reading and suddenly, they're basically trying to clobber each other to death. So, this is a lot harder than it sounds like.Matthew: Yeah, and it was—but what's interesting is, like, we took it all that into account, but the act of fixing things, you break things. And that was not just true at Cloudflare. If you look across Google and Microsoft and Amazon, everybody, their worst availability was second half of 2021 or into 2022. But it both internally and externally, we talked about the mistakes we made, we talked about the challenges we had, we talked about—and today, we're significantly more resilient and more reliable because of that. And so, transparency is built into Cloudflare from the beginning.The earliest story of this, I remember, there was a 15-year-old kid living in Long Beach, California who bought my social security number off of a Russian website that had hacked a bank that I'd once used to get a mortgage. He then use that to redirect my cell phone voicemail to a voicemail box he controlled. He then used that to get into my personal email. He then used that to find a zero-day vulnerability in Google's corporate email where he could privilege-escalate from my personal email into Google's corporate email, which is the provider that we use for our email service. And then he used that as an administrator on our email at the time—this is back in the early days of Cloudflare—to get into another administration account that he then used to redirect one of Cloud Source customers to a website that he controlled.And thankfully, it wasn't, you know, the FBI or the Central Bank of Brazil, which were all Cloudflare customers. Instead, it was 4chan because he was a 15-year-old hacker kid. And we fix it pretty quickly and nobody knew who Cloudflare was at the time. And so potential—Corey: The potential damage that could have been caused at that point with that level of access to things, like, that is such a ridiculous way to use it.Matthew: And—yeah [laugh]—my temptation—because it was embarrassing. He took a bunch of stuff from my personal email and he put it up on a website, which just to add insult to injury, was actually using Cloudflare as well. And I wanted to sweep it under the rug. And our team was like, “That's not the right thing to do. We're fundamentally a security company and we need to talk about when we make mistakes on security.” And so, we wrote a huge postmortem on, “Here's all the stupid things that we did that caused this hack to happen.” And by the way, it wasn't just us. It was AT&T, it was Google. I mean, there are a lot of people that ended up being involved.Corey: It builds trust with that stuff. It's painful in the short term, but I believe with the benefit of hindsight, it was clearly the right call.Matthew: And it was—and I remember, you know, pushing ‘publish' on the blog post and thinking, “This is going to be the end of the company.” And quite the opposite happened, which was all of a sudden, we saw just an incredible amount of people who signed up the next day saying, “If you're going to be that transparent about something that was incredibly embarrassing when you didn't have to be, then that's the sort of thing that actually makes me trust that you're going to be transparent the future.” And I think learning that lesson early on, has been just an incredibly valuable lesson for us and made us the company that we are today.Corey: A question that I have for you about the idea of there being no reason to charge in one direction but not the other. There's something that I'm not sure that I understand on this. If I run a website, to use your numbers of a terabit out—because it's a web server—and effectively nothing in—because it's a webserver; other than the request, nothing really is going to come in—that ingress bandwidth becomes effectively unused and also free. So, if I have another use case where I'm paying for it anyway, if I'm primarily caring about an outward direction, sure, you can send things in for free. Now, there's a lot of nuance that goes into that. But I'm curious as to what the—is their fundamental misunderstanding in that analysis of the bandwidth market?Matthew: No. And I think that's exactly, exactly right. And it's actually interesting. At Cloudflare, our infrastructure team—which is the one that manages our connections to the outside world, manages the hardware we have—meets on a quarterly basis with our product team. It's called the Hot and Cold Meeting.And what they do is they go over our infrastructure, and they say, “Okay, where are we hot? Where do we have not enough capacity?” If you think of any given server, an easy way to think of a server is that it has, sort of, four resources that are available to it. This is, kind of, vast simplification, but one is the connectivity to the outside world, both transit in and out. The second is the—Corey: Otherwise it's just a complicated space heater.Matthew: Yeah [laugh]. The other is the CPU. The other is the longer-term storage. We use only SSDs, but sort of, you know, hard drives or SSD storage. And then the fourth is the short-term storage, or RAM that's in that server.And so, at any given moment, there are going to be places where we are running hot, where we have a sort of capacity level that we're targeting and we're over that capacity level, but we're also going to be running cold in some of those areas. And so, the infrastructure team and the product team get together and the product team has requests on, you know, “Here's some more places we would be great to have more infrastructure.” And we're really good at deploying that when we need to, but the infrastructure team then also says, “Here are the places where we're cold, where we have excess capacity.” And that turns into products at Cloudflare. So, for instance, you know, the reason that we got into the zero-trust space was very much because we had all this excess capacity.We have 100 times the capacity of something like Zscaler across our network, and we can add that—that is primar—where most of our older products are all about outward traffic, the zero-trust products are all about inward traffic. And the reason that we can do everything that Zscaler does, but for, you know, a much, much, much more affordable prices, we going to basically just layer that on the network that already exists. The reason we don't charge for the bandwidth behind DDoS attacks is DDoS attacks are always about inbound traffic and we have just a ton of excess capacity around that inbound traffic. And so, that unused capacity is a resource that we can then turn into products, and very much that conversation between our product team and our infrastructure team drives how we think about building new products. And we're always trying to say how can we get as much utilization out of every single piece of equipment that we run everywhere in the world.The way we build our network, we don't have custom machines or different networks for every products. We build all of our machines—they come in generations. So, we're on, I think, generation 14 of servers where we spec a server and it has, again, a certain amount of each of those four [bits 00:39:22] of capacity. But we can then deploy that server all around the world, and we're buying many, many, many of them at any given time so we can get the best cost on that. But our product team is very much in constant communication with our infrastructure team and saying, “What more can we do with the capacity that we have?” And then we pass that on to our customers by adding additional features that work across our network and then doing it in a way that's incredibly cost-effective.Corey: I really want to thank you for taking the time to, basically once again, suffer slings and arrows about networking, security, cloud, economics, and so much more. If people want to learn more, where's the best place for them to find you?Matthew: You know, used to be an easy question to answer because it was just, you know, go on Twitter and find me but now we have all these new mediums. So, I'm @eastdakota on Twitter. I'm eastdakota.com on Bluesky. I'm @real_eastdakota on Threads. And so, you know, one way or another, if you search for eastdakota, you'll come across me somewhere out there in the ether.Corey: And we will, of course, put links to that in the show notes. Thank you so much for your time. I appreciate it.Matthew: It's great to talk to you, Corey.Corey: Matthew Prince, CEO and co-founder of Cloudflare. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry, insulting comment that I will of course not charge you inbound data rates on.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

Feddie Scum
S3 Episode 4

Feddie Scum

Play Episode Listen Later Aug 8, 2023 79:01


S3 episode 4 - Murder on the Saint Sebastian - There's been a murder and only one man is fit to find the killer!  Thanks to Adam from Gundam Explained for voicing Greg! https://twitter.com/GundamExplained Edited by Dallas Welk and Chris Ramey - Feddie Scum - The Gundam RPG Podcast Twitter - https://twitter.com/FeddieScum Facebook - https://www.facebook.com/feddiescum Youtube - https://www.youtube.com/channel/UCWptBHBc3no9bht-EpZO8wQ Support us on Patreon! - https://www.patreon.com/deadsetmedia Check out our merch store! - https://feddiescum.bigcartel.comEpisode Notes Find out more at https://feddiescum.pinecast.co

Screaming in the Cloud
The Role of DevRel at Google with Richard Seroter

Screaming in the Cloud

Play Episode Listen Later Aug 8, 2023 34:07


Richard Seroter, Director of Outbound Product Management at Google, joins Corey on Screaming in the Cloud to discuss what's new at Google. Corey and Richard discuss how AI can move from a novelty to truly providing value, as well as the importance of people maintaining their skills and abilities rather than using AI as a black box solution. Richard also discusses how he views the DevRel function, and why he feels it's so critical to communicate expectations for product launches with customers. About RichardRichard Seroter is Director of Outbound Product Management at Google Cloud. He's also an instructor at Pluralsight, a frequent public speaker, and the author of multiple books on software design and development. Richard maintains a regularly updated blog (seroter.com) on topics of architecture and solution design and can be found on Twitter as @rseroter. Links Referenced: Google Cloud: https://cloud.google.com Personal website: https://seroter.com Twitter: https://twitter.com/rseroter LinkedIn: https://www.linkedin.com/in/seroter/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Human-scale teams use Tailscale to build trusted networks. Tailscale Funnel is a great way to share a local service with your team for collaboration, testing, and experimentation.  Funnel securely exposes your dev environment at a stable URL, complete with auto-provisioned TLS certificates. Use it from the command line or the new VS Code extensions. In a few keystrokes, you can securely expose a local port to the internet, right from the IDE.I did this in a talk I gave at Tailscale Up, their first inaugural developer conference. I used it to present my slides and only revealed that that's what I was doing at the end of it. It's awesome, it works! Check it out!Their free plan now includes 3 users & 100 devices. Try it at snark.cloud/tailscalescream Corey: Welcome to Screaming in the Cloud, I'm Corey Quinn. We have returning guest Richard Seroter here who has apparently been collecting words to add to his job title over the years that we've been talking to him. Richard, you are now the Director of Product Management and Developer Relations at Google Cloud. Do I have all those words in the correct order and I haven't forgotten any along the way?Richard: I think that's all right. I think my first job was at Anderson Consulting as an analyst, so my goal is to really just add more words to whatever these titles—Corey: It's an adjective collection, really. That's what a career turns into. It's really the length of a career and success is measured not by accomplishments but by word count on your resume.Richard: If your business card requires a comma, success.Corey: So, it's been about a year or so since we last chatted here. What have you been up to?Richard: Yeah, plenty of things here, still, at Google Cloud as we took on developer relations. And, but you know, Google Cloud proper, I think AI has—I don't know if you've noticed, AI has kind of taken off with some folks who's spending a lot the last year… juicing up services and getting things ready there. And you know, myself and the team kind of remaking DevRel for a 2023 sort of worldview. So, yeah we spent the last year just scaling and growing and in covering some new areas like AI, which has been fun.Corey: You became profitable, which is awesome. I imagined at some point, someone wound up, like, basically realizing that you need to, like, patch the hole in the pipe and suddenly the water bill is no longer $8 billion a quarter. And hey, that works super well. Like, wow, that explains our utility bill and a few other things as well. I imagine the actual cause is slightly more complex than that, but I am a simple creature.Richard: Yeah. I think we made more than YouTube last quarter, which was a good milestone when you think of—I don't think anybody who says Google Cloud is a fun side project of Google is talking seriously anymore.Corey: I misunderstood you at first. I thought you said that you're pretty sure you made more than I did last year. It's like, well, yes, if a multi-billion dollar company's hyperscale cloud doesn't make more than I personally do, then I have many questions. And if I make more than that, I have a bunch of different questions, all of which could be terrifying to someone.Richard: You're killing it. Yeah.Corey: I'm working on it. So, over the last year, another trend that's emerged has been a pivot away—thankfully—from all of the Web3 nonsense and instead embracing the sprinkle some AI on it. And I'm not—people are about to listen to this and think, wait a minute, is he subtweeting my company? No, I'm subtweeting everyone's company because it seems to be a universal phenomenon. What's your take on it?Richard: I mean, it's countercultural now to not start every conversation with let me tell you about our AI story. And hopefully, we're going to get past this cycle. I think the AI stuff is here to stay. This does not feel like a hype trend to me overall. Like, this is legit tech with real user interest. I think that's awesome.I don't think a year from now, we're going to be competing over who has the biggest model anymore. Nobody cares. I don't know if we're going to hopefully lead with AI the same way as much as, what is it doing for me? What is my experience? Is it better? Can I do this job better? Did you eliminate this complex piece of toil from my day two stuff? That's what we should be talking about. But right now it's new and it's interesting. So, we all have to rub some AI on it.Corey: I think that there is also a bit of a passing of the buck going on when it comes to AI where I've talked to companies that are super excited about how they have this new AI story that's going to be great. And, “Well, what does it do?” “It lets you query our interface to get an answer.” Okay, is this just cover for being bad UX?Richard: [laugh]. That can be true in some cases. In other cases, this will fix UXes that will always be hard. Like, do we need to keep changing… I don't know, I'm sure if you and I go to our favorite cloud providers and go through their documentation, it's hard to have docs for 200 services and millions of pages. Maybe AI will fix some of that and make it easier to discover stuff.So in some cases, UIs are just hard at scale. But yes, I think in some cases, this papers over other things not happening by just rubbing some AI on it. Hopefully, for most everybody else, it's actually interesting, new value. But yeah, that's a… every week it's a new press release from somebody saying they're about to launch some AI stuff. I don't know how any normal human is keeping up with it.Corey: I certainly don't know. I'm curious to see what happens but it's kind of wild, too, because there you're right. There is something real there where you ask it to draw you a picture of a pony or something and it does, or give me a bunch of random analysis of this. I asked one recently to go ahead and rank the US presidents by absorbency and with a straight face, it did it, which is kind of amazing. I feel like there's a lack of imagination in the way that people talk about these things and a certain lack of awareness that you can make this a lot of fun, and in some ways, make that a better showcase of the business value than trying to do the straight-laced thing of having it explain Microsoft Excel to you.Richard: I think that's fair. I don't know how much sometimes whimsy and enterprise mix. Sometimes that can be a tricky part of the value prop. But I'm with you this some of this is hopefully returns to some more creativity of things. I mean, I personally use things like Bard or what have you that, “Hey, I'm trying to think of this idea. Can you give me some suggestions?” Or—I just did a couple weeks ago—“I need sample data for my app.”I could spend the next ten minutes coming up with Seinfeld and Bob's Burgers characters, or just give me the list in two seconds in JSON. Like that's great. So, I'm hoping we get to use this for more fun stuff. I'll be fascinated to see if when I write the keynote for—I'm working on the keynote for Next, if I can really inject something completely off the wall. I guess you're challenging me and I respect that.Corey: Oh, I absolutely am. And one of the things that I believe firmly is that we lose sight of the fact that people are inherently multifaceted. Just because you are a C-level executive at an enterprise does not mean that you're not also a human being with a sense of creativity and a bit of whimsy as well. Everyone is going to compete to wind up boring you to death with PowerPoint. Find something that sparks the imagination and sparks joy.Because yes, you're going to find the boring business case on your own without too much in the way of prodding for that, but isn't it great to imagine what if? What if we could have fun with some of these things? At least to me, that's always been the goal is to get people's attention. Humor has been my path, but there are others.Richard: I'm with you. I think there's a lot to that. And the question will be… yeah, I mean, again, to me, you and I talked about this before we started recording, this is the first trend for me in a while that feels purely organic where our customers, now—and I'll tell our internal folks—our customers have much better ideas than we do. And it's because they're doing all kinds of wild things. They're trying new scenarios, they're building apps purely based on prompts, and they're trying to, you know, do this.And it's better than what we just come up with, which is awesome. That's how it should be, versus just some vendor-led hype initiative where it is just boring corporate stuff. So, I like the fact that this isn't just us talking; it's the whole industry talking. It's people talking to my non-technical family members, giving me ideas for what they're using this stuff for. I think that's awesome. So yeah, but I'm with you, I think companies can also look for more creative angles than just what's another way to left-align something in a cell.Corey: I mean, some of the expressions on this are wild to me. The Photoshop beta with its generative AI play has just been phenomenal. Because it's weird stuff, like, things that, yeah, I'm never going to be a great artist, let's be clear, but being able to say remove this person from the background, and it does it, as best I can tell, seamlessly is stuff where yeah, that would have taken me ages to find someone who knows what the hell they're doing on the internet somewhere and then pay them to do it. Or basically stumble my way through it for two hours and it somehow looks worse afterwards than before I started. It's the baseline stuff of, I'm never going to be able to have it—to my understanding—go ahead just build me a whole banner ad that does this and hit these tones and the rest, but it is going to help me refine something in that direction, until I can then, you know, hand it to a professional who can take it from my chicken scratching into something real.Richard: If it will. I think that's my only concern personally with some of this is I don't want this to erase expertise or us to think we can just get lazy. I think that I get nervous, like, can I just tell it to do stuff and I don't even check the output, or I don't do whatever. So, I think that's when you go back to, again, enterprise use cases. If this is generating code or instructions or documentation or what have you, I need to trust that output in some way.Or more importantly, I still need to retain the skills necessary to check it. So, I'm hoping people like you and me and all our —every—all the users out there of this stuff, don't just offload responsibility to the machine. Like, just always treat it like a kind of slightly drunk friend sitting next to you with good advice and always check it out.Corey: It's critical. I think that there's a lot of concern—and I'm not saying that people are wrong on this—but that people are now going to let it take over their jobs, it's going to wind up destroying industries. No, I think it's going to continue to automate things that previously required human intervention. But this has been true since the Industrial Revolution, where opportunities arise and old jobs that used to be critical are no longer centered in quite the same way. The one aspect that does concern me is not that kids are going to be used to cheat on essays like, okay, great, whatever. That seems to be floated mostly by academics who are concerned about the appropriate structure of academia.For me, the problem is, is there's a reason that we have people go through 12 years of English class in the United States and that is, it's not to dissect of the work of long-dead authors. It's to understand how to write and how to tell us a story and how to frame ideas cohesively. And, “The computer will do that for me,” I feel like that potentially might not serve people particularly well. But as a counterpoint, I was told when I was going to school my entire life that you're never going to have a calculator in your pocket all the time that you need one. No, but I can also speak now to the open air, ask it any math problem I can imagine, and get a correct answer spoken back to me. That also wasn't really in the bingo card that I had back then either, so I am a hesitant to try and predict the future.Richard: Yeah, that's fair. I think it's still important for a kid that I know how to make change or do certain things. I don't want to just offload to calculators or—I want to be able to understand, as you say, literature or things, not just ever print me out a book report. But that happens with us professionals, too, right? Like, I don't want to just atrophy all of my programming skills because all I'm doing is accepting suggestions from the machine, or that it's writing my emails for me. Like, that still weirds me out a little bit. I like to write an email or send a tweet or do a summary. To me, I enjoy those things still. I don't want to—that's not toil to me. So, I'm hoping that we just use this to make ourselves better and we don't just use it to make ourselves lazier.Corey: You mentioned a few minutes ago that you are currently working on writing your keynote for Next, so I'm going to pretend, through a vicious character attack here, that this is—you know, it's 11 o'clock at night, the day before the Next keynote and you found new and exciting ways to procrastinate, like recording a podcast episode with me. My question for you is, how is this Next going to be different than previous Nexts?Richard: Hmm. Yeah, I mean, for the first time in a while it's in person, which is wonderful. So, we'll have a bunch of folks at Moscone in San Francisco, which is tremendous. And I [unintelligible 00:11:56] it, too, I definitely have online events fatigue. So—because absolutely no one has ever just watched the screen entirely for a 15 or 30 or 60-minute keynote. We're all tabbing over to something else and multitasking. And at least when I'm in the room, I can at least pretend I'll be paying attention the whole time. The medium is different. So, first off, I'm just excited—Corey: Right. It feels a lot ruder to get up and walk out of the front row in the middle of someone's talk. Now, don't get me wrong, I'll still do it because I'm a jerk, but I'll feel bad about it as I do. I kid, I kid. But yeah, a tab away is always a thing. And we seem to have taken the same structure that works in those events and tried to force it into more or less a non-interactive Zoom call, and I feel like that is just very hard to distinguish.I will say that Google did a phenomenal job of online events, given the constraints it was operating under. Production value is great, the fact that you took advantage of being in different facilities was awesome. But yeah, it'll be good to be back in person again. I will be there with bells on in Moscone myself, mostly yelling at people, but you know, that's what I do.Richard: It's what you do. But we missed that hallway track. You missed this sort of bump into people. Do hands-on labs, purposely have nothing to do where you just walk around the show floor. Like we have been missing, I think, society-wise, a little bit of just that intentional boredom. And so, sometimes you need at conference events, too, where you're like, “I'm going to skip that next talk and just see what's going on around here.” That's awesome. You should do that more often.So, we're going to have a lot of spaces for just, like, go—like, 6000 square feet of even just going and looking at demos or doing hands-on stuff or talking with other people. Like that's just the fun, awesome part. And yeah, you're going to hear a lot about AI, but plenty about other stuff, too. Tons of announcements. But the key is that to me, community stuff, learn from each other stuff, that energy in person, you can't replicate that online.Corey: So, an area that you have expanded into has been DevRel, where you've always been involved with it, let's be clear, but it's becoming a bit more pronounced. And as an outsider, I look at Google Cloud's DevRel presence and I don't see as much of it as your staffing levels would indicate, to the naive approach. And let's be clear, that means from my perspective, all public-facing humorous, probably performative content in different ways, where you have zany music videos that, you know, maybe, I don't know, parody popular songs do celebrate some exec's birthday they didn't know was coming—[fake coughing]. Or creative nonsense on social media. And the the lack of seeing a lot of that could in part be explained by the fact that social media is wildly fracturing into a bunch of different islands which, on balance, is probably a good thing for the internet, but I also suspect it comes down to a common misunderstanding of what DevRel actually is.It turns out that, contrary to what many people wanted to believe in the before times, it is not getting paid as much as an engineer, spending three times that amount of money on travel expenses every year to travel to exotic places, get on stage, party with your friends, and then give a 45-minute talk that spends two minutes mentioning where you work and 45 minutes talking about, I don't know, how to pick the right standing desk. That has, in many cases, been the perception of DevRel and I don't think that's particularly defensible in our current macroeconomic climate. So, what are all those DevRel people doing?Richard: [laugh]. That's such a good loaded question.Corey: It's always good to be given a question where the answers are very clear there are right answers and wrong answers, and oh, wow. It's a fun minefield. Have fun. Go catch.Richard: Yeah. No, that's terrific. Yeah, and your first part, we do have a pretty well-distributed team globally, who does a lot of things. Our YouTube channel has, you know, we just crossed a million subscribers who are getting this stuff regularly. It's more than Amazon and Azure combined on YouTube. So, in terms of like that, audience—Corey: Counterpoint, you definitionally are YouTube. But that's neither here nor there, either. I don't believe you're juicing the stats, but it's also somehow… not as awesome if, say, I were to do it, which I'm working on it, but I have a face for radio and it shows.Richard: [laugh]. Yeah, but a lot of this has been… the quality and quantity. Like, you look at the quantity of video, it overwhelms everyone else because we spend a lot of time, we have a specific media team within my DevRel team that does the studio work, that does the production, that does all that stuff. And it's a concerted effort. That team's amazing. They do really awesome work.But, you know, a lot of DevRel as you say, [sigh] I don't know about you, I don't think I've ever truly believed in the sort of halo effect of if super smart person works at X company, even if they don't even talk about that company, that somehow presents good vibes and business benefits to that company. I don't think we've ever proven that's really true. Maybe you've seen counterpoints, where [crosstalk 00:16:34]—Corey: I can think of anecdata examples of it. Often though, on some level, for me at least, it's been okay someone I tremendously respect to the industry has gone to work at a company that I've never heard of. I will be paying attention to what that company does as a direct result. Conversely, when someone who is super well known, and has been working at a company for a while leaves and then either trashes the company on the way out or doesn't talk about it, it's a question of, what's going on? Did something horrible happen there? Should we no longer like that company? Are we not friends anymore? It's—and I don't know if that's necessarily constructive, either, but it also, on some level, feels like it can shorthand to oh, to be working DevRel, you have to be an influencer, which frankly, I find terrifying.Richard: Yeah. Yeah. I just—the modern DevRel, hopefully, is doing a little more of product-led growth style work. They're focusing specifically on how are we helping developers discover, engage, scale, become advocates themselves in the platform, increasing that flywheel through usage, but that has very discreet metrics, it has very specific ownership. Again, personally, I don't even think DevRel should do as much with sales teams because sales teams have hundreds and sometimes thousands of sales engineers and sales reps. It's amazing. They have exactly what they need.I don't think DevRel is a drop in the bucket to that team. I'd rather talk directly to developers, focus on people who are self-service signups, people who are developers in those big accounts. So, I think the modern DevRel team is doing more in that respect. But when I look at—I just look, Corey, this morning at what my team did last week—so the average DevRel team, I look at what advocacy does, teams writing code labs, they're building tutorials. Yes, they're doing some in person events. They wrote some blog posts, published some videos, shipped a couple open-source projects that they contribute to in, like gaming sector, we ship—we have a couple projects there.They're actually usually customer zero in the product. They use the product before it ships, provides bugs and feedback to the team, we run DORA workshops—because again, we're the DevOps Research and Assessment gang—we actually run the tutorial and Docs platform for Google Cloud. We have people who write code samples and reference apps. So, sometimes you see things publicly, but you don't see the 20,000 code samples in the docs, many written by our team. So, a lot of the times, DevRel is doing work to just enable on some of these different properties, whether that's blogs or docs, whether that's guest articles or event series, but all of this should be in service of having that credible relationship to help devs use the platform easier. And I love watching this team do that.But I think there's more to it now than years ago, where maybe it was just, let's do some amazing work and try to have some second, third-order effect. I think DevRel teams that can have very discrete metrics around leading indicators of long-term cloud consumption. And if you can't measure that successfully, you've probably got to rethink the team.[midroll 00:19:20]Corey: That's probably fair. I think that there's a tremendous series of… I want to call it thankless work. Like having done some of those ridiculous parody videos myself, people look at it and they chuckle and they wind up, that was clever and funny, and they move on to the next one. And they don't see the fact that, you know, behind the scenes for that three-minute video, there was a five-figure budget to pull all that together with a lot of people doing a bunch of disparate work. Done right, a lot of this stuff looks like it was easy or that there was no work at all.I mean, at some level, I'm as guilty of that as anyone. We're recording a podcast now that is going to be handed over to the folks at HumblePod. They are going to produce this into something that sounds coherent, they're going to fix audio issues, all kinds of other stuff across the board, a full transcript, and the rest. And all of that is invisible to me. It's like AI; it's the magic box I drop a file into and get podcast out the other side.And that does a disservice to those people who are actively working in that space to make things better. Because the good stuff that they do never gets attention, but then the company makes an interesting blunder in some way or another and suddenly, everyone's out there screaming and wondering why these people aren't responding on Twitter in 20 seconds when they're finding out about this stuff for the first time.Richard: Mm-hm. Yeah, that's fair. You know, different internal, external expectations of even DevRel. We've recently launched—I don't know if you caught it—something called Jump Start Solutions, which were executable reference architectures. You can come into the Google Cloud Console or hit one of our pages and go, “Hey, I want to do a multi-tier web app.” “Hey, I want to do a data processing pipeline.” Like, use cases.One click, we blow out the entire thing in the platform, use it, mess around with it, turn it off with one click. Most of those are built by DevRel. Like, my engineers have gone and built that. Tons of work behind the scenes. Really, like, production-grade quality type architectures, really, really great work. There's going to be—there's a dozen of these. We'll GA them at Next—but really, really cool work. That's DevRel. Now, that's behind-the-scenes work, but as engineering work.That can be some of the thankless work of setting up projects, deployment architectures, Terraform, all of them also dropped into GitHub, ton of work documenting those. But yeah, that looks like behind-the-scenes work. But that's what—I mean, most of DevRel is engineers. These are folks often just building the things that then devs can use to learn the platforms. Is it the flashy work? No. Is it the most important work? Probably.Corey: I do have a question I'd be remiss not to ask. Since the last time we spoke, relatively recently from this recording, Google—well, I'd say ‘Google announced,' but they kind of didn't—Squarespace announced that they'd be taking over Google domains. And there was a lot of silence, which I interpret, to be clear, as people at Google being caught by surprise, by large companies, communication is challenging. And that's fine, but I don't think it was anything necessarily nefarious.And then it came out further in time with an FAQ that Google published on their site, that Google Cloud domains was a part of this as well. And that took a lot of people aback, in the sense—not that it's hard to migrate a domain from one provider to another, but it brought up the old question of, if you're building something in cloud, how do you pick what to trust? And I want to be clear before you answer that, I know you work there. I know that there are constraints on what you can or cannot say.And for people who are wondering why I'm not hitting you harder on this, I want to be very explicit, I can ask you a whole bunch of questions that I already know the answer to, and that answer is that you can't comment. That's not constructive or creative. So, I don't want people to think that I'm not intentionally asking the hard questions, but I also know that I'm not going to get an answer and all I'll do is make you uncomfortable. But I think it's fair to ask, how do you evaluate what services or providers or other resources you're using when you're building in cloud that are going to be around, that you can trust building on top of?Richard: It's a fair question. Not everyone's on… let's update our software on a weekly basis and I can just swap things in left. You know, there's a reason that even Red Hat is so popular with Linux because as a government employee, I can use that Linux and know it's backwards compatible for 15 years. And they sell that. Like, that's the value, that this thing works forever.And Microsoft does the same with a lot of their server products. Like, you know, for better or for worse, [laugh] they will always kind of work with a component you wrote 15 years ago in SharePoint and somehow it runs today. I don't even know how that's possible. Love it. That's impressive.Now, there's a cost to that. There's a giant tax in the vendor space to make that work. But yeah, there's certain times where even with us, look, we are trying to get better and better at things like comms. And last year we announced—I checked them recently—you know, we have 185 Cloud products in our enterprise APIs. Meaning they have a very, very tight way we would deprecate with very, very long notice, they've got certain expectations on guarantees of how long you can use them, quality of service, all the SLAs.And so, for me, like, I would bank on, first off, for every cloud provider, whether they're anchor services. Build on those right? You know, S3 is not going anywhere from Amazon. Rock solid service. BigQuery Goodness gracious, it's the center of Google Cloud.And you look at a lot of services: what can you bet on that are the anchors? And then you can take bets on things that sit around it. There's times to be edgy and say, “Hey, I'll use Service Weaver,” which we open-sourced earlier this year. It's kind of a cool framework for building apps and we'll deconstruct it into microservices at deploy time. That's cool.Would I literally build my whole business on it? No, I don't think so. It's early stuff. Now, would I maybe use it also with some really boring VMs and boring API Gateway and boring storage? Totally. Those are going to be around forever.I think for me, personally, I try to think of how do I isolate things that have some variability to them. Now, to your point, sometimes you don't know there's variability. You would have just thought that service might be around forever. So, how are you supposed to know that that thing could go away at some point? And that's totally fair. I get that.Which is why we have to keep being better at comms, making sure more things are in our enterprise APIs, which is almost everything. So, you have some assurances, when I build this thing, I've got a multi-year runway if anything ever changes. Nothing's going to stay the same forever, but nothing should change tomorrow on a dime. We need more trust than that.Corey: Absolutely. And I agree. And the problem, too, is hidden dependencies. Let's say what is something very simple. I want to log in to [unintelligible 00:25:34] brand new AWS account and spin of a single EC2 instance. The end. Well, I can trust that EC2 is going to be there. Great. That's not one service you need to go through that critical path. It is a bare minimum six, possibly as many as twelve, depending upon what it is exactly you're doing.And it's the, you find out after the fact that oh, there was that hidden dependency in there that I wasn't fully aware of. That is a tricky and delicate balance to strike. And, again, no one is going to ever congratulate you—at all—on the decision to maintain a service that is internally painful and engineering-ly expensive to keep going, but as soon as you kill something, even it's for this thing doesn't have any customers, the narrative becomes, “They're screwing over their customers.” It's—they just said that it didn't have any. What's the concern here?It's a messaging problem; it is a reputation problem. Conversely, everyone knows that Amazon does not kill AWS services. Full stop. Yeah, that turns out everyone's wrong. By my count, they've killed ten, full-on AWS services and counting at the moment. But that is not the reputation that they have.Conversely, I think that the reputation that Google is going to kill everything that it touches is probably not accurate, though I don't know that I'd want to have them over to babysit either. So, I don't know. But it is something that it feels like you're swimming uphill on in many respects, just due to not even deprecation decisions, historically, so much as poor communication around them.Richard: Mm-hm. I mean, communication can always get better, you know. And that's, it's not our customers' problem to make sure that they can track every weird thing we feel like doing. It's not their challenge. If our business model changes or our strategy changes, that's not technically the customer's problem. So, it's always our job to make this as easy as possible. Anytime we don't, we have made a mistake.So, you know, even DevRel, hey, look, it puts teams in a tough spot. We want our customers to trust us. We have to earn that; you will never just give it to us. At the same time, as you say, “Hey, we're profitable. It's great. We're growing like weeds,” it's amazing to see how many people are using this platform. I mean, even services, you don't talk about having—I mean, doing really, really well. But I got to earn that. And you got to earn, more importantly, the scale. I don't want you to just kick the tires on Google Cloud; I want you to bet on it. But we're only going to earn that with really good support, really good price, stability, really good feeling like these services are rock solid. Have we totally earned that? We're getting there, but not as mature as we'd like to get yet, but I like where we're going.Corey: I agree. And reputations are tricky. I mean, recently InfluxDB deprecated two regions and wound up turning them off and deleting data. And they wound up getting massive blowback for this, which, to their credit, their co-founder and CTO, Paul Dix—who has been on the show before—wound up talking about and saying, “Yeah, that was us. We're taking ownership of this.”But the public announcement said that they had—that data in AWS was not recoverable and they're reaching out to see if the data in GCP was still available. At which point, I took the wrong impression from this. Like, whoa, whoa, whoa. Hang on. Hold the phone here. Does that mean that data that I delete from a Google Cloud account isn't really deleted?Because I have a whole bunch of regulators that would like a word if so. And Paul jumped onto that with, “No, no, no, no, no. I want to be clear, we have a backup system internally that we were using that has that set up. And we deleted the backups on the AWS side; we don't believe we did on the Google Cloud side. It's purely us, not a cloud provider problem.” It's like, “Okay, first, sorry for causing a fire drill.” Secondly, “Okay, that's great.” But the reason I jumped in that direction was just because it becomes so easy when a narrative gets out there to believe the worst about companies that you don't even realize you're doing it.Richard: No, I understand. It's reflexive. And I get it. And look, B2B is not B2C, you know? In B2B, it's not, “Build it and they will come.” I think we have the best cloud infrastructure, the best security posture, and the most sophisticated managed services. I believe that I use all the clouds. I think that's true. But it doesn't matter unless you also do the things around it, around support, security, you know, usability, trust, you have to go sell these things and bring them to people. You can't just sit back and say, “It's amazing. Everyone's going to use it.” You've got to earn that. And so, that's something that we're still on the journey of, but our foundation is terrific. We just got to do a better job on some of these intangibles around it.Corey: I agree with you, when you s—I think there's a spirited debate you could have on any of those things you said that you believe that Google Cloud is the best at, with the exception of security, where I think that is unquestionably. I think that is a lot less variable than the others. The others are more or less, “Who has the best cloud infrastructure?” Well, depends on who had what for breakfast today. But the simplicity and the approach you take to security is head and shoulders above the competition.And I want to make sure I give credit where due: it is because of that simplicity and default posturing that customers wind up better for it as a result. Otherwise, you wind up in this hell of, “You must have at least this much security training to responsibly secure your environment.” And that is never going to happen. People read far less than we wish they would. I want to make very clear that Google deserves the credit for that security posture.Richard: Yeah, and the other thing, look, I'll say that, from my observation, where we do something that feels a little special and different is we do think in platforms, we think in both how we build and how we operate and how the console is built by a platform team, you—singularly. How—[is 00:30:51] we're doing Duet AI that we've pre-announced at I/O and are shipping. That is a full platform experience covering a dozen services. That is really hard to do if you have a lot of isolation. So, we've done a really cool job thinking in platforms and giving that simplicity at that platform level. Hard to do, but again, we have to bring people to it. You're not going to discover it by accident.Corey: Richard, I will let you get back to your tear-filled late-night writing of tomorrow's Next keynote, but if people want to learn more—once the dust settles—where's the best place for them to find you?Richard: Yeah, hopefully, they continue to hang out at cloud.google.com and using all the free stuff, which is great. You can always find me at seroter.com. I read a bunch every day and then I've read a blog post every day about what I read, so if you ever want to tune in on that, just see what wacky things I'm checking out in tech, that is good. And I still hang out on different social networks, Twitter at @rseroter and LinkedIn and things like that. But yeah, join in and yell at me about anything I said.Corey: I did not realize you had a daily reading list of what you put up there. That is news to me and I will definitely track in, and then of course, yell at you from the cheap seats when I disagree with anything that you've chosen to include. Thank you so much for taking the time to speak with me and suffer the uncomfortable questions.Richard: Hey, I love it. If people aren't talking about us, then we don't matter, so I would much rather we'd be yelling about us than the opposite there.Corey: [laugh]. As always, it's been a pleasure. Richard Seroter, Director of Product Management and Developer Relations at Google Cloud. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment that you had an AI system write for you because you never learned how to structure a sentence.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

The Bike Shed
396: Build vs. Buy

The Bike Shed

Play Episode Listen Later Aug 8, 2023 33:57


Joël has been fighting a frustrating bug where he's integrating with a third-party database, and some queries just crash. Stephanie shares her own debugging story about a leaky stub that caused flaky tests. Additionally, they discuss the build vs. buy decision when integrating with third-party systems. They consider the time and cost implications of building their own integration versus using off-the-shelf components and conclude that the decision often depends on the specific needs and priorities of the project, including how quickly a solution is needed and whether the integration is core to the business's value proposition. Ruby class instance variables (https://www.codegram.com/blog/understanding-class-instance-variables-in-ruby/) Build vs Buy by Josh Clayton (https://thoughtbot.com/blog/build-vs-buy-considerations-for-new-products) Sustainable Rails (https://sustainable-rails.com/) Transcript: STEPHANIE: Hello and welcome to another episode of The Bike Shed, a weekly podcast from your friends at thoughtbot about developing great software. I'm Stephanie Minn. JOËL: And I'm Joël Quenneville. And together, we're here to share a bit of what we've learned along the way. STEPHANIE: So, Joël, what's new in your world? JOËL: My world has been kind of frustrating recently. I've been fighting a really frustrating bug where I'm integrating with a third-party database. And there are queries that just straight-up crash. Any query that instantiates an instance of an ActiveRecord object will just straight-up fail. And that's because before, we make the actual query, almost like a preflight query that fetches the schema of the database, particularly the list of tables that the database has, and there's something in this schema that the code doesn't like, and everything just crashes. Specifically, I'm using an ODBC connection. I forget exactly what the acronym stands for, Open Database connection, maybe? Which is a standard put up by Microsoft. The way I'm integrating it via Ruby is there's a gem that's a C extension. And somewhere deep in the C extension, this whole thing is crashing. So, I've had to sort of dust off some C a little bit to look through. And it's not super clear exactly why things are crashing. So, I've spent several days trying to figure out what's going on there. And it's been really cryptic. STEPHANIE: Yeah, that does sound frustrating. And it seems like maybe you are a little bit out of your depth in terms of your usual tools for figuring out a bug are not so helpful here. JOËL: Yeah, yeah. It's a lot harder to just go through and put in a print or a debug statement because now I have to recompile some C. And, you know, you can mess around with some things by passing different flags. But it is a lot more difficult than just doing, like, a bundle open and binding to RB in the code. My ultimate solution was asking for help. So, I got another thoughtboter to help me, and we paired on it. We got to a solution that worked. And then, right before I went to deploy this change, because this was breaking on the staging website, I refreshed the website just to make sure that everything was breaking before I pushed the fix to see that everything is working. This is a habit I've picked up from test-driven development. You always want to see your test break before you see it succeed. And this is a situation where this habit paid off because the website was just working. My changes were not deployed. It just started working again. Now it's gotten me just completely questioning whether my solution fixes anything. The difficulty is because I am integrating [inaudible 03:20] third-party database; it's non-deterministic. The schema on there is changing rather frequently. I think the reason things are crashing is because there's some kind of bad data or data that the ODBC adapter doesn't like in this third-party system. But it just got introduced one day; everything started breaking, and then somehow it got removed, and everything is working again without any input or code changes on my end. So, now I don't trust my fix. STEPHANIE: Oh no. Yeah, I would struggle with that because your reality has come crashing down, [laughs] or how you understood reality. That's tough. Where do you think you'll go from here? If it's no longer really an issue in this current state of the schema, is it worth pursuing further at this time? JOËL: So, that's interesting because it turns into a prioritization problem. And for this particular project, with the deadlines that we have, we've decided it's not worth it. I've opened up a PR with my fix, with some pretty in-depth documentation for why I thought that was the fix and what I think the underlying problem is. If this shows up again in the next few days, I'll have that PR that I can pull in and see if it fixes things, and if it doesn't, I'll probably just close that PR, but it'll be available for us if we ever run into this again. I've also looked at a few potential mitigating situations. Part of the problem is that this is a, like, massive system. The Rails app that I'm using really doesn't need to deal with this massive database. I think there's, you know, almost 1,000 tables, and I really only care about a subset of tables in, like, one underlying schema. And so, I think by reducing the permissions of my database user to only those tables that I care about, there's a lower chance of me triggering something like this. STEPHANIE: Interesting. What you mentioned about, you know, having that PR continue to exist will be really helpful for future folks who might come across the same problem, right? Because then they can see, like, all of the research and investigation you've already done. And you may have already done this, but if you do think it's a schema issue, I'm curious about whether the snapshot of the schema could be captured from when it was failing to when it has magically gotten fixed. And I wonder if there may be some clues there for some future investigator. JOËL: Yeah. I'm not sure what our backup situation is because this is a third-party system, so I'd have to figure out what things are like in the admin interface there. But yeah, if there is some kind of auditing, or snapshots, or backups, or something there, and I have rough, you know, if I know it's within a 24-hour period, maybe there's something there that would tell me what's happening. My best guess is that there's some string that is longer than expected or maybe being marked as a CHAR when it should be a VARCHAR, or maybe something that's not a non-UTF-8 encoded character, or something weird like that. So, I never know exactly what was wrong in the schema. There's some weird string thing happening that's causing the Ruby adapter to blow up. STEPHANIE: That also feels so unsatisfying [laughs] for you. I could imagine. JOËL: Yeah, there's no, like, clean resolution, right? It's a, well, the bug is gone for now. We're trying to make it less likely for it to pop up again in the future. I'm trying to leave some documentation for the next person who's going to come along, and I'm moving forward, fingers crossed. Is that something you've ever had to do on one of your projects? STEPHANIE: Given up? Yes. [laughter] I think I have definitely had to learn how to timebox debugging and have some action items for when I just can't figure it out. And, you know, like we mentioned, leaving some documentation for the next person to pick up, adding some additional logging so that maybe we can get more clues next time. But, you know, realizing that I do have to move on and that's the best that I can do is really challenging. JOËL: So, you used two words here to describe the situation: one was giving up, and the other one was timebox. I think I really like the idea of describing this as timeboxing. Giving up feels kind of like, defeatist. You know, there's so many things that we can do with our time, and we really have to be strategic with how we prioritize. So, I like the idea of describing this as a timeboxing situation. STEPHANIE: Yeah, I agree. Maybe I should celebrate every time that I successfully timebox something [laughs] according to how I planned to. [laughs] JOËL: There's always room to extend the timebox, right? STEPHANIE: [laughs] It's funny you bring up a debugging mystery because I have one of my own to share today. And I do have to say that it ended up being resolved, [chuckles] so it was a win in my book. But I will call this the case of the leaky stub. JOËL: That sounds slightly scary. STEPHANIE: It really was. The premise of what we were trying to figure out here was that we were having some flaky tests that were failing with a runtime error, so that was already kind of interesting. But it was quickly determined it was flaky because of the tests running in a certain order, so-- JOËL: Classic. STEPHANIE: Right. So, I knew something was happening, and any tests that came after it were running into this error. And I was taking a look, and I figured out how to recreate it. And we even isolated to the test itself that was running before everything else, that would then cause some problems. And so, looking into this test, I saw that it was stubbing the find method on an ActiveRecord model. JOËL: Interesting. STEPHANIE: Yeah. And the stubbed value that we were choosing to return ended up being referenced in the tests that followed. So, that was really strange to me because it went against everything I understood about how RSpec cleans up stubs between tests, right? JOËL: Yeah, that is really strange. STEPHANIE: Yeah, and I knew that it was referencing the stub value because we had set a really custom, like, ID value to it. So, when I was seeing this exact ID value showing up in a test that seemed totally unrelated, that was kind of a clue that there was some leakage happening. JOËL: So, what did you do next? STEPHANIE: The next discovery was that the error was actually raised in the factory setup for the failing tests and not even getting to running the examples at all. So, that was really strange. And digging into the factories was also its own adventure because there was a lot of complexity in the factories. A lot of them used hooks as well that then called some application code. And it was a wild goose chase. But ultimately, I realized that in the factory setup, we were calling some application code for that model where we had stubbed the find, and it had used the find method to memoize a class instance variable. JOËL: Oh no. I can see where this is going. STEPHANIE: Yeah. So, at some point, our model.find() returned our, you know, stub value that we had wanted in the previous test. And it got cached and just continued to leak into everything else that eventually would try to call that memoized method when it really should have tried to do that look-up for a separate record. JOËL: And class instance variables will persist between tests as long as they're on the same thread, right? STEPHANIE: Yeah, as far as I understand it. JOËL: That sounds like a really frustrating journey. And then that moment when you see the class instance variable, and you're like, oh no, I can't believe this is happening. STEPHANIE: Right? It was a real recipe for disaster, I think, where we had some, you know, really complicated factories. We had some sneaky caching issues, and this, you know, totally seemingly random runtime error that was being raised. And it was a real wild goose chase because there was not a lot of directness in going down the debugging path. I feel like I went around all over the codebase to get to the root of it. And, in the end, you know, we were trying to come up with some takeaways. And what was unfortunate was that you know, like, normally, stubbing find can be okay if you are, you know, really wanting to make sure that you are returning your mocked value that you may have, like, stubbed some other stuff on in your test. But because of all this, we were like, well, should we just not stub find on this really particular model? And that didn't seem particularly sustainable to make as a takeaway for other developers who want to avoid this problem. So, in the end, I think we scoped the stub to be a little more specific with the arguments that we wanted to target. And that was the way that we went forward with the particular flaky test at hand. JOËL: It sounds like the root cause of the problem was not so much the stub as it was the fact that this value is getting cached at the class level. Is that right? STEPHANIE: Yes and no. It seems like a real pain for running the tests. But I'm assuming that it was done for a good reason in production, maybe, maybe not. To be fair, I think we didn't need to cache it at all because it's calling a find, which is, you know, should be pretty quick and doesn't need to be cached. But who knows? It's hard to tell. It was really old code. And I think we were feeling also a little nervous to adjust something that we weren't sure what the impact would be. JOËL: I'm always really skeptical of caching. Caching has its place. But I think a lot of developers are a little too happy to introduce one, especially doing it preemptively that, oh well, we might need a cache here, so why not? Let's add that. Or even sometimes, just as a blind solution to any kind of slowness, oh, the site is slow; let's throw a cache here and hope for the best. And the, like, bedrock, like, rule zero of any kind of performance tuning is you've got to measure before and after and make sure that the change that you introduce actually makes things better. And then, also, is it better enough speed-wise that you're willing to pay any kind of costs associated to maintaining the code now that it's more complex? And a lot of caches can have some higher carrying costs. STEPHANIE: Yeah, that's a great point. This debugging mystery an example of one of them. JOËL: How long did it take you to figure out the solution here? STEPHANIE: So, like you, I actually was on a bit of the incorrect path for a little while. And it was only because this issue affected a different flaky test that someone else was investigating that they were able to connect the dots and be like, I think these, you know, two issues are related. And they were the ones who ultimately were able to point us out to the offending test if you will. So, you know, it took me a few days. And I imagine it took the other developer a few days. So, our combined effort was, like, over a week. JOËL: Yep. So, for all our listeners out there, you just heard that Stephanie and I [laughs] both went on multi-day debugging journeys. That happens to everyone. Just because we've been doing this job for years doesn't mean that every bug is, like, a thing that we figure out immediately. So, separately from this bug that I've been working on, a big issue that's been front of mind for me on this project has been the classic build versus buy decision. Because we're integrating with a third-party system, we have to look at either building our own integration or trying to use some off-the-shelf components. And there's a few different levels of this. There are some parts where you can actually, like, literally buy an integration and think through some of the decisions there. And then there's some situations where maybe there's an open-source component that we can use. And there's always trade-offs with both the commercial and the open-source situation. And we have to decide, are we willing to use this, or do we want to build our own? And those have been some really interesting discussions to have. STEPHANIE: Yeah. I think you actually expanded this decision-making problem into a build versus buy versus open source because they are kind of, you know, really different solutions with different outcomes in terms of, you know, maintenance and dependencies, right? And that all have, like, a little bit of a different way to engage with them. JOËL: Interesting. I think I tend to think of the buy category, including both like commercial off-the-shelf software and also open-source off-the-shelf software, things that we wouldn't build custom for ourselves but that are third-party components that we can pull in. STEPHANIE: Yeah, that's interesting because I had a bit of a different mental model because, in my head, when you're buying a commercial solution, you, you know, are maybe losing out on some opportunities for customization or even, like, forking it on your own. So, with an open-source solution, there could be an aspect of making it work for you. Whereas for a commercial solution, you really become dependent on that other company and whether they are willing to cater [laughs] to your needs or not. JOËL: That's fair. For something that's closed-source where you don't actually have access to the code, say it's more of a software as a service situation, then, yeah, you're kind of locked in and hoping that they can provide the needs that you have. On the flip side, you are generally paying for some level of support. The quality of that varies sometimes from one vendor to another. But if something goes wrong, usually, there's someone you can email, someone you can call, and they will tell you how to fix the problem, or they will fix it on their end. STEPHANIE: For the purposes of this conversation, should we talk about the differences, you know, building yourself or leaning on an existing built-out solution for you? JOËL: The project I'm working on is integrating with a Snowflake data warehouse, which is an external place that stores data accessible through something SQL-like. And one of the things that's attractive about this is that you can pull in data from a variety of different sources, transform it, and have it all stored in a kind of standardized structure that you can then integrate with. So, for pulling data in, you can build your own sort of ingestion pipeline, if you want, with code, and their APIs, and things. But there are also third-party vendors that will give you kind of off-the-shelf components that you can use for a lot of popular other data sources that you might want to pull. So, you're saying; I want to pull from this external service. They've probably got a pre-built connector for it. They can also do things like pull from an arbitrary Postgres database on some other server if that's something you have access to. It becomes really attractive because all you need to do is create an account on this website, plug in a few, like, API keys and URLs. And, all of a sudden, data is just flowing from one third-party system into your Snowflake data Warehouse, and it all just kind of works. And you don't have to bother with APIs, or ODBC, or any of that kind of stuff. STEPHANIE: Got it. Yeah, that does sound convenient. As you were talking about this, I was thinking about how if I were in the position of trying to decide how to make that integration happen, the idea of building it would seem kind of scary, especially if it's something that I don't have a lot of expertise in. JOËL: Yeah, so this was really interesting. In the beginning of the project, I looked into a little bit of what goes into building these, and it's fairly simple in terms of the architecture. You just need something that writes data files to typically something like an S3 bucket. And then you can point Snowflake to periodically pull from that bucket, and you write an import script to, you know, parse the columns and write them to the right tables in the structure that you want inside Snowflake. Where things get tricky is the actual integration on the other end. So, you have some sort of third-party service. And now, how do you sort of, on a timer maybe, pull data from that? And if there are data changes that you're synchronizing, is it just all append-only data? Or are you allowing the third-party service to say, "Hey, I deleted this record, and you should reflect that in Snowflake?" Or maybe dealing with an update. So, all of these things you have to think about, as well as synchronization. What you end up having to do is you probably boot up some kind of small service and, you know, maybe this is a small Ruby app that you have on Heroku, maybe this is, like, an AWS Lambda kind of thing. And you probably end up running this every so many seconds or so many minutes, do some work, potentially write some files to S3. And there's a lot of edge cases you have to think about to do it properly. And so, not having to think about all of those edge cases becomes really enticing when you're looking to potentially pay a third party to do this for you. STEPHANIE: Yeah, when you used the words new service, I bristled a little bit [laughs] because I've definitely seen this happen maybe on a bit of a bigger scale for a tool or solution for some need, right? Where some team is formed, or maybe we kind of add some more responsibilities to an existing team to spin up a new service with a new repo with its own pipeline, and it becomes yet another thing to maintain. And I have definitely seen issues with the longevity of that kind of approach. JOËL: The idea of maintaining a fleet of little services for each of our integrations seemed very unappealing to me, especially given that setting something like this up using the commercial approach probably takes 30 minutes per third-party service. There's no way I'm standing up an app and doing this whole querying every so many minutes, and getting data, and transforming it, and writing it to S3, and addressing all the edge cases in 30 minutes. And it's building something that's robust. And, you know, maybe if I want to go, like, really low tech, there's something fun I could do with, like, a Zapier hook and just, like, duct tape a few services together and make this, like, a no-code solution. I still don't know that it would have the robustness of the vendor. And I don't think that I could do it in the same amount of time. STEPHANIE: Yeah. I like the keyword robustness here because, at first, you were saying, like, you know, this looked relatively small in scope, right? The code that you had to write. But introducing all of the variables of things that could go wrong [laughs] beyond the custom part that you actually care about seemed quite cumbersome. JOËL: I think there's also, at this point, a lot of really interesting prioritization questions. There are money questions, but there are also time questions you have to think about. So, how much dev time do we want to devote upfront to building out these integrations? And if you're trying to move fast and get a proof of concept out, or even get, like, an MVP out in front of customers, it might be worth paying more money upfront to a third-party vendor because it allows you to ship something this week rather than next month. STEPHANIE: Yeah. The "How soon do you need it?" is a very good question to ask. Another one that I have learned to include in my arsenal of, you know, evaluating this kind of stuff comes from a thoughtbot blog by Josh Clayton, where he, you know, talks about the build versus buy problem. And his takeaway is that you should buy when your business is not dependent on it. JOËL: When it's not part of, like, the core, like, value-add that your business is doing. Why spend developer time on something that's not, like, the core thing that your product is when you can pay someone else to do it for you? And like we said earlier, a lot of that time ends up being sunk into edge cases and robustness and things like that to the point where now you have to build an expertise in a, like, secondary thing that your business doesn't really care about. STEPHANIE: Yeah, absolutely. I think this is also perhaps where very clear business goals or a vision would come in handy as well. Because if you're considering building something that doesn't quite support that vision, then it will likely end up continuing to be deprioritized over the long term until it becomes this thing that no one is accountable for maintaining and caring for. And just causes a lot of, honestly, morale issues is what I've seen when some service that was spun up to try to solve a particular problem is kind of on its last legs and has been really neglected, and no one wants to work on it. But it ends up causing issues for the rest of the development team. But then they're also really focused on initiatives that actually do provide the business value. That is a really hard balancing act that I've seen teams struggle with. JOËL: Earlier this year, we were talking about the book Sustainable Rails. And it really hammers home the idea of a carrying cost for the code, and I think that's exactly what we're talking about here. And that carrying cost can be time and money. But I like that you also mentioned the morale effects. You know, that's a carrying cost that just sort of depresses the productivity of your team when morale is low. STEPHANIE: Yeah, absolutely. I'm curious if we could discuss some of the carrying costs of buying a solution and where you've seen that become tricky. JOËL: The first thing to look at is the literal cost, the money aspect of things. And I think it's a really interesting situation for the business models for these types of Snowflake connectors because they typically charge by the amount of data that you're transmitting, so per row of data that you're transmitting. And so, that cost will fluctuate depending on whether the third-party service you're integrating with is, like, really chatty or not. When you contrast that to building, building typically has a relatively fixed cost. It's a big upfront cost, and then there's some maintenance cost to go with it. So, if I'm building some kind of integration for, let's say, Shopify, then there's the cost I need to build up front to integrate that. And if that takes me, I don't know, a week or two weeks, or however long it is, you know, that's a pretty big chunk of time. And my time is money. And so, you can actually do the math and say, "Well, if we know that we're getting so many rows per day at this rate from the commercial vendor, how many weeks do we have to pay for the commercial one before we break even and it becomes more expensive than building it upfront, just in terms of my time?" And sometimes you do that math, and you're like, wow, you know, we could be going on this commercial thing for, like, two years before we break even. In that case, from a purely financial point of view, it's probably worth paying for that connector. And so, now it becomes really interesting. You say, okay, well, which are the connectors that we have that are low volume, and which are the ones that are high volume? Because each of them is going to have a different break-even point. The ones that you break even after, you know, three or four weeks might be the ones that become more interesting to have a conversation about building. Whereas some of the others, it's clearly not worth our time to build it ourselves. STEPHANIE: The way you described this problem was really interesting to me because it almost sounds like you found the solution somewhere in the middle, potentially, where, you know, you may try building the ones that are highest priority, and you end up learning a lot from that experience, right? That could make it easier or at least, like, set you up to consider doing that moving forward in the future if you find, like, that is what is valuable. But it's interesting to me that you kind of have the best of both worlds of, like, getting the commercial solutions now for the things that are lower value and then doing what you can to get the most out of building a solution. JOËL: Yeah. So, my final recommendation ended up being, let's go all commercial for now. And then, once we've built out something, and because speed is also an issue here, once we've built out something and it's out with customers, and we're starting to see value from this, then we can start looking at how much are we paying per week for each of these connectors? And is it worth maybe going back and building our own for some of these higher-volume connectors? But starting with the commercial one for everything. STEPHANIE: Yeah, I actually think that's generally a pretty good path forward because then you are also learning about how you use the commercial solution and, you know, which features of it are critical so that if you do eventually find yourselves, like, maybe considering a shift to building in-house, like, you could start with a more clear MVP, right? Because you know how your team is using an existing product and can focus on the parts that your business are dependent on. JOËL: Yeah, it's that classic iterative development style. I think here it's also kind of inspired by a strategy I typically use for performance, which is make it work before you try to make it fast. And, actually, make it work, then profile, then measure, find the hotspots, and then focus on making those things fast. So, in this case, instead of speed, we're talking about money. So, it's make it work, then profile, find the parts that are expensive, and make the trade-off of, like, okay, is it worth investing into making that part less expensive in terms of resources? STEPHANIE: I like that as a framework a lot. JOËL: A lot of what we do as programmers is optimization, right? And sometimes, we're optimizing for execution time. Sometimes we're optimizing for memory cost, and sometimes we're optimizing for dollars. STEPHANIE: Yeah, that's really interesting because, with the buy solution, you know very clearly, like, how much the thing will cost. Whereas I've definitely seen teams go down the building route, and it always takes longer than expected [laughs], and that is money, right? In terms of the developer's time, for sure. JOËL: Yeah, definitely, like, add some kind of multiplier when you're budgeting out that build alternative because, quite likely, there are some edge cases that you haven't thought about that the commercial partner has, and you will have to spend more time on that than you expected. STEPHANIE: Yeah, in addition to whatever opportunity cost of not working on something that is driving revenue for the business right now. JOËL: Exactly. STEPHANIE: So, the direction of this conversation ended up going kind of towards, like, what is best for the team at, like, a product and company level. But I think that we make these decisions a lot more frequently, even when it comes to whether we pull in a gem or, you know, use an open-source tool or not. And I would be really interested in discussing more of that in another episode. JOËL: Yeah. That gets into some controversial takes, right? It's the evergreen topic of: do we build it ourselves, or do we pull in some kind of third-party package? STEPHANIE: Something for the future to look forward to. On that note, shall we wrap up? JOËL: Let's wrap up. STEPHANIE: Show notes for this episode can be found at bikeshed.fm. JOËL: This show has been produced and edited by Mandy Moore. STEPHANIE: If you enjoyed listening, one really easy way to support the show is to leave us a quick rating or even a review in iTunes. It really helps other folks find the show. JOËL: If you have any feedback for this or any of our other episodes, you can reach us @_bikeshed, or you can reach me @joelquen on Twitter. STEPHANIE: Or reach both of us at hosts@bikeshed.fm via email. JOËL: Thanks so much for listening to The Bike Shed, and we'll see you next week. ALL: Byeeeeeeeee!!!!! ANNOUNCER: This podcast is brought to you by thoughtbot, your expert strategy, design, development, and product management partner. We bring digital products from idea to success and teach you how because we care. Learn more at thoughtbot.com.

pr giving microsoft mvp id shopify api rb warehouses snowflakes apis rails char s3 sql zapier urls mandy moore utf heroku caching aws lambda postgres quenneville activerecord rspec odbc stephanie yeah josh clayton stephanie it stephanie so stephanie oh stephanie yes stephanie right stephanie for
Chile Today Podcast
S3, Episode 17: Chile's electrical system part II

Chile Today Podcast

Play Episode Listen Later Aug 7, 2023 97:12


S3, Episode 17: Chile's electrical system part IIThis week, Bethany and Pingüino will update you on recent news and upcoming events. Then follow Lenny as he continues his deep dive in to Chile's electrical system, this time focusing on the challenges that Chile is facing today.Support our pod by joining us in Patreon! Lots of extras just for you! Tip sheets, tickets to events, merch and more!Support the show