Podcasts about amazon rds

  • 53PODCASTS
  • 220EPISODES
  • 37mAVG DURATION
  • ?INFREQUENT EPISODES
  • Dec 23, 2024LATEST

POPULARITY

20172018201920202021202220232024


Best podcasts about amazon rds

Latest podcast episodes about amazon rds

AWS Morning Brief
A Legend Moves On

AWS Morning Brief

Play Episode Listen Later Dec 23, 2024 7:29


AWS Morning Brief for the week of December 23, with Corey Quinn. Links:Amazon AppStream 2.0 introduces client for macOSAmazon EC2 instances support bandwidth configurations for VPC and EBSAmazon Timestream for InfluxDB now supports Internet Protocol Version 6 (IPv6) connectivityAmazon WorkSpaces Thin Client now available to purchase in IndiaAWS Backup launches support for search and item-level recoveryAWS Mainframe Modernization now supports connectivity over Internet Protocol version 6 (IPv6)AWS Marketplace now supports self-service promotional media on seller product detail pagesAWS re:Post now supports Spanish and PortugueseAWS Resource Explorer supports 59 new resource typesAWS offers a self-service feature to update business names on AWS InvoicesAnnouncing CloudFormation support for AWS Parallel Computing ServiceAnnouncing Node Health Monitoring and Auto-Repair for Amazon EKS - AWSAnd that's a wrap!Best practices for creating a VPC for Amazon RDS for Db2How the Amazon TimeHub team handled disruption in AWS DMS CDC task caused by Oracle RESETLOGS: Part 3How to detect and monitor Amazon Simple Storage Service (S3) access with AWS CloudTrail and Amazon CloudWatchEnforce resource configuration to control access to new features with AWSMaximizing your cloud journey: Engaging an AWS Solutions Architect

AWS Morning Brief
The re:Invent Stragglers

AWS Morning Brief

Play Episode Listen Later Dec 16, 2024 5:21


AWS Morning Brief for the week of December 16th, 2024, with Corey Quinn. Links:Amazon Bedrock Guardrails reduces pricing by up to 85%Amazon CloudWatch now provides centralized visibility into telemetry configurationsAmazon EC2 F2 instances, featuring up to 8 FPGAs, are generally availableAmazon SES now offers Global Endpoints for multi-region sending resilienceAWS Toolkit for Visual Studio Code now includes Amazon CloudWatch Logs Live TailAccelerate your AWS Graviton adoption with the AWS Graviton Savings DashboardCapture data changes while restoring an Amazon DynamoDB tableUnderstand the benefits of physical replication in Amazon RDS for PostgreSQL Blue/Green DeploymentsHow AWS sales uses Amazon Q Business for customer engagementAWS Network Firewall Geographic IP Filtering launchIssue with DynamoDB local - CVE-2022-1471

AWS Morning Brief
Amazon Basics Corey Quinn

AWS Morning Brief

Play Episode Listen Later Jul 29, 2024 3:41


AWS Morning Brief for the week of Monday, July 29th, with Mike Julian.Links:Amazon VPC IPAM now supports BYOIP for IPs registered with any Internet RegistryAWS Cost Categories now supports “Billing Entity” dimension Enhance database performance with Amazon RDS dedicated log volumesQLDB deprecation--more on the way?

What's new in Cloud FinOps?
WNiCF - June 2024 - News

What's new in Cloud FinOps?

Play Episode Listen Later Jul 8, 2024 31:20


In this episode of What's New in Cloud FinOps, we (Frank and Stephen) discuss the latest news and updates in cloud computing impacting your wallet.We cover topics such as right-sizing recommendations for Amazon RDS, data exports for cost optimization, and improved billing and cost management. We also highlight talks from FinOps X in San Diego and share their favourite sessions. Overall, the conversation provides valuable insights into the latest developments in cloud FinOps, including a rant from Stephen on the hardship that is finding good content for Azure. 

Podcast AWS LATAM
EP207: Soporte Extendido de Amazon RDS

Podcast AWS LATAM

Play Episode Listen Later Jun 11, 2024 10:50


En este episodio, Victor Sampaio y Guido Mercado, expertos del equipo de Optimización de Costos y Aceleración para Latinoamérica, brindan una explicación detallada acerca de la inscripción automática al Soporte Extendido de Amazon RDS. Este servicio está diseñado para aquellas bases de datos cuyo período de Soporte Estándar de Amazon RDS ha finalizado. Los especialistas compartirán información valiosa para asegurar la continuidad y el rendimiento óptimo de tus bases de datos, evitando interrupciones en tus operaciones críticas.

AWS Developers Podcast
Migrate 600 Oracle databases from on-premises to Amazon RDS

AWS Developers Podcast

Play Episode Listen Later Jun 7, 2024 33:04


Join us as we dive into an inspiring conversation from the AWS Summit in Stockholm with Matt Houghton, an AWS Ambassador and Community Builder. Matt shares insights on his team at CDL and their monumental achievement of migrating 600 Oracle databases to RDS Postgres.

AWS Podcast
#669: How Figma scaled their databases to 100x growth with Amazon RDS

AWS Podcast

Play Episode Listen Later May 28, 2024 26:33


Join us as Figma shares their innovative strategies for staying ahead of rapid growth and scaling their database infrastructure seamlessly with the help of the Amazon RDS team.

Le Podcast AWS en Français
L'excellence opérationnelle pour les DB

Le Podcast AWS en Français

Play Episode Listen Later May 10, 2024 44:51


Qonto a fait le choix d'utiliser les bases de données managées Amazon RDS. Leur modèle business mutualisé les a conduit à développer des outils de monitoring complémentaires. Dans cet épisode, nous parlons de RDS Exporter pour Prometheus et du Database Monitoring Framework, deux projets open source créés et utilisés par Qonto.

Le Podcast AWS en Français
L'excellence opérationnelle pour les DB

Le Podcast AWS en Français

Play Episode Listen Later May 10, 2024 44:51


Qonto a fait le choix d'utiliser les bases de données managées Amazon RDS. Leur modèle business mutualisé les a conduit à développer des outils de monitoring complémentaires. Dans cet épisode, nous parlons de RDS Exporter pour Prometheus et du Database Monitoring Framework, deux projets open source créés et utilisés par Qonto.

AWS Morning Brief
Open S3 Buckets No Longer a Concern?

AWS Morning Brief

Play Episode Listen Later Apr 22, 2024 5:51


AWS Morning Brief for the week of April 22, 2024, with Corey Quinn. Links:AWS IAM Identity Center adds independent 90-days session duration for Amazon CodeWhisperer Deloitte and AWS Strategic Collaboration to Accelerate Cloud Adoption in Growth MarketsImprove cost visibility of Amazon EKS with AWS Split Cost Allocation Data Congratulations to the PartyRock generative AI hackathon winners Access Amazon RDS across AWS accounts using AWS PrivateLink, Network Load Balancer, and Amazon RDS ProxyProgrammatic approach to optimize the cost of Amazon RDS snapshots Reduce cost and improve performance by migrating to Amazon DocumentDB 5.0A secure approach to generative AI with AWS AWS celebrates big technology wins at NAB 2024 New AWS survey reveals the link between AI fluency and the next education revolutionCVE-2024-28056Creating shortcut links to AWS Management Console destinations - AWS IAM Identity Center 

amazon ai cloud concerns longer aws devops buckets nab corey quinn party rock amazon rds amazon eks aws management console amazon documentdb aws privatelink network load balancer last week in aws
AWS Podcast
#646: Amazon RDS for Db2

AWS Podcast

Play Episode Listen Later Dec 18, 2023 17:28


Amazon RDS is a fully managed, secure, and compliant service that simplifies setup, operation, and database scaling in the cloud. Joining this episode with Simon Elisha, is Karthik Gopalakrishnan, Sr. Technical Product Manager at AWS. They'll discuss the features, benefits, and use cases of Amazon RDS for Db2.

Postgres FM
Blue-green deployments

Postgres FM

Play Episode Listen Later Nov 10, 2023 43:45


Nikolay and Michael discuss blue-green deployments — specifically an RDS blog post, how similar this is (or not) to what they understand to be blue-green deployments, and how applicable the methodology might be in the database world more generally.  Here are some links to things they mentioned:Fully managed Blue/Green Deployment in Amazon Aurora PostgreSQL and Amazon RDS for PostgreSQL https://aws.amazon.com/blogs/database/new-fully-managed-blue-green-deployment-in-amazon-aurora-postgresql-and-amazon-rds-for-postgresql/  Blue-green deployment (blog post by Martin Fowler) https://martinfowler.com/bliki/BlueGreenDeployment.html  Our episode on logical replication https://postgres.fm/episodes/logical-replication pgroll https://github.com/xataio/pgroll ~~~What did you like or not like? What should we discuss next time? Let us know via a YouTube comment, on social media, or by commenting on our Google doc!~~~Postgres FM is brought to you by:Nikolay Samokhvalov, founder of Postgres.aiMichael Christofides, founder of pgMustardWith special thanks to:Jessie Draws for the amazing artwork 

The New Stack Podcast
PostgreSQL Takes a New Turn

The New Stack Podcast

Play Episode Listen Later Nov 8, 2023 21:07


Jonathan Katz, a principal product manager at Amazon Web Services, discusses the evolution of PostgreSQL in an episode of The New Stack Makers. He notes that PostgreSQL's uses have expanded significantly since its inception and now cover a wide range of applications and workloads. Initially considered niche, it faced competition from both open-source and commercial relational database systems. Katz's involvement in the PostgreSQL community began as an app developer, and he later contributed by organizing events.PostgreSQL originated from academic research at the University of California at Berkeley in the mid-1980s, becoming an open-source project in 1994. In the mid-1990s, proprietary databases like Oracle, IBM DB2, and Microsoft SQL dominated the market, while open-source alternatives like MySQL, MariaDB, and SQLite emerged.PostgreSQL 16 introduces logical replication from standby servers, enhancing scalability by offloading work from the primary server. The meticulous design process within the PostgreSQL community leads to stable and reliable features. Katz mentions the development of Direct I/O as a long-term feature to reduce latency and improve data writing performance, although it will take several years to implement.Amazon Web Services has built Amazon RDS on PostgreSQL to simplify application development for developers. This managed service handles operational tasks such as deployment, backups, and monitoring, allowing developers to focus on their applications. Amazon RDS supports multiple PostgreSQL releases, making it easier for businesses to manage and maintain their databases.Learn more from The New Stack about PostgreSQL and AWS:PostgreSQL 16 Expands Analytics CapabilitiesPowertools for AWS Lambda Grows with Help of VolunteersHow Donating Open Source Code Can Advance Your Career

Postgres FM
Over-indexing

Postgres FM

Play Episode Listen Later Oct 20, 2023 42:45


Nikolay and Michael discuss over-indexing — what we mean by it, the regular issues people discuss about it, as well as a novel one Nikolay has come across and benchmarked recently.  Here are some links to things they mentioned:Nikolay's tweet on over-indexing https://twitter.com/samokhvalov/status/1713101666629927112 Heap-Only Tuples (HOT) optimization https://www.postgresql.org/docs/current/storage-hot.html Our episode on index maintenance https://postgres.fm/episodes/index-maintenance PgBouncer now supports prepared statements https://github.com/pgbouncer/pgbouncer/releases/tag/pgbouncer_1_21_0 Our episode on connection poolers https://postgres.fm/episodes/connection-poolers Configurable FP_LOCK_SLOTS_PER_BACKEND (Hackers mailing list discussion) https://www.postgresql.org/message-id/flat/CAM527d-uDn5osa6QPKxHAC6srOfBH3M8iXUM%3DewqHV6n%3Dw1u8Q%40mail.gmail.com LWLock:lock_manager (Amazon RDS docs) https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/wait-event.lw-lock-manager.html ~~~What did you like or not like? What should we discuss next time? Let us know via a YouTube comment, on social media, or by commenting on our Google doc!~~~Postgres FM is brought to you by:Nikolay Samokhvalov, founder of Postgres.aiMichael Christofides, founder of pgMustardWith special thanks to:Jessie Draws for the amazing artwork 

AWS Morning Brief
AWS Guild Dinner & Tournament

AWS Morning Brief

Play Episode Listen Later Sep 11, 2023 6:29


AWS Morning Brief for the week of September 11, 2023, with Corey Quinn. Links: Amazon Aurora and Amazon RDS announces Extended Support for MySQL and PostgreSQL databases Amazon CloudWatch adds Amazon EKS control plane logs as Vended Logs Amazon CloudWatch Logs announces regular expression filter pattern syntax support As SwiftOnSecurity pointed out a week or two ago, a lot of folks can now discover firsthand just how many of their rules allow all 10* traffic Introducing Amazon EC2 R7iz instances  AWS Marketplace now supports AWS CloudTrail to improve procurement activity monitoring  AWS Step Functions launches enhanced error handling AWS Trusted Advisor adds 1 new fault tolerance check Announcing daily disbursements for AWS Marketplace sellers  Embracing FinOps to Maximize Cloud Value and Control Costs with the Deloitte FinOps Framework  Transforming Aviation Maintenance with the Infosys Generative AI Solution Built on Amazon Bedrock  How Vercel Shipped Cron Jobs in 2 Months Using Amazon EventBridge Scheduler How contact center leaders can prepare for generative AI  A Culture of Resilience  How generative AI is energizing the beauty industry Migrating AWS Direct Connect to a new location Reduce the security and compliance risks of messaging apps with AWS Wickr  AWS Guild Tournament builds cloud skills and innovative customer solutions From chocolate sales to a career in cloud with training from AWS re/Start Amazon to Discontinue Honeycode App-Building Service 

Screaming in the Cloud
The Evolution of OpenTelemetry with Austin Parker

Screaming in the Cloud

Play Episode Listen Later Sep 5, 2023 40:09


Austin Parker, Community Maintainer at OpenTelemetry, joins Corey on Screaming in the Cloud to discuss OpenTelemetry's mission in the world of observability. Austin explains how the OpenTelemetry community was able to scale the OpenTelemetry project to a commercial offering, and the way Open Telemetry is driving innovation in the data space. Corey and Austin also discuss why Austin decided to write a book on OpenTelemetry, and the book's focus on the evergreen applications of the tool. About AustinAustin Parker is the OpenTelemetry Community Maintainer, as well as an event organizer, public speaker, author, and general bon vivant. They've been a part of OpenTelemetry since its inception in 2019.Links Referenced: OpenTelemetry: https://opentelemetry.io/ Learning OpenTelemetry early release: https://www.oreilly.com/library/view/learning-opentelemetry/9781098147174/ Page with Austin's social links: https://social.ap2.io TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Look, I get it. Folks are being asked to do more and more. Most companies don't have a dedicated DBA because that person now has a full time job figuring out which one of AWS's multiple managed database offerings is right for every workload. Instead, developers and engineers are being asked to support, and heck, if time allows, optimize their databases. That's where OtterTune comes in. Their AI is your database co-pilot for MySQL and PostgresSQL on Amazon RDS or Aurora. It helps improve performance by up to four x OR reduce costs by 50 percent – both of those are decent options. Go to ottertune dot com to learn more and start a free trial. That's O-T-T-E-R-T-U-N-E dot com.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. It's been a few hundred episodes since I had Austin Parker on to talk about the things that Austin cares about. But it's time to rectify that. Austin is the community maintainer for OpenTelemetry, which is a CNCF project. If you're unfamiliar with, we're probably going to fix that in short order. Austin, Welcome back, it's been a month of Sundays.Austin: It has been a month-and-a-half of Sundays. A whole pandemic-and-a-half.Corey: So, much has happened since then. I tried to instrument something with OpenTelemetry about a year-and-a-half ago, and in defense to the project, my use case is always very strange, but it felt like—a lot of things have sharp edges, but it felt like this had so many sharp edges that you just pivot to being a chainsaw, and I would have been at least a little bit more understanding of why it hurts so very much. But I have heard from people that I trust that the experience has gotten significantly better. Before we get into the nitty-gritty of me lobbing passive-aggressive bug reports at you have for you to fix in a scenario in which you can't possibly refuse me, let's start with the beginning. What is OpenTelemetry?Austin: That's a great question. Thank you for asking it. So, OpenTelemetry is an observability framework. It is run by the CNCF, you know, home of such wonderful award-winning technologies as Kubernetes, and you know, the second biggest source of YAML in the known universe [clear throat].Corey: On some level, it feels like that is right there with hydrogen as far as unlimited resources in our universe.Austin: It really is. And, you know, as we all know, there are two things that make, sort of, the DevOps and cloud world go around: one of them being, as you would probably know, AWS bills; and the second being YAML. But OpenTelemetry tries to kind of carve a path through this, right, because we're interested in observability. And observability, for those that don't know or have been living under a rock or not reading blogs, it's a lot of things. It's a—but we can generally sort of describe it as, like, this is how you understand what your system is doing.I like to describe it as, it's a way that we can model systems, especially complex, distributed, or decentralized software systems that are pretty commonly found in larg—you know, organizations of every shape and size, quite often running on Kubernetes, quite often running in public or private clouds. And the goal of observability is to help you, you know, model this system and understand what it's doing, which is something that I think we can all agree, a pretty important part of our job as software engineers. Where OpenTelemetry fits into this is as the framework that helps you get the telemetry data you need from those systems, put it into a universal format, and then ship it off to some observability back-end, you know, a Prometheus or a Datadog or whatever, in order to analyze that data and get answers to your questions you have.Corey: From where I sit, the value of OTel—or OpenTelemetry; people in software engineering love abbreviations that are impenetrable from the outside, so of course, we're going to lean into that—but what I found for my own use case is the shining value prop was that I could instrument an application with OTel—in theory—and then send whatever I wanted that was emitted in terms of telemetry, be it events, be it logs, be it metrics, et cetera, and send that to any or all of a curation of vendors on a case-by-case basis, which meant that suddenly it was the first step in, I guess, an observability pipeline, which increasingly is starting to feel like a milit—like an industrial-observability complex, where there's so many different companies out there, it seems like a good approach to use, to start, I guess, racing vendors in different areas to see which performs better. One of the challenges I've had with that when I started down that path is it felt like every vendor who was embracing OTel did it from a perspective of their implementation. Here's how to instrument it to—send it to us because we're the best, obviously. And you're a community maintainer, despite working at observability vendors yourself. You have always been one of those community-first types where you care more about the user experience than you do this quarter for any particular employer that you have, which to be very clear, is intended as a compliment, not a terrifying warning. It's why you have this authentic air to you and why you are one of those very few voices that I trust in a space where normally I need to approach it with significant skepticism. How do you see the relationship between vendors and OpenTelemetry?Austin: I think the hard thing is that I know who signs my paychecks at the end of the day, right, and you always have, you know, some level of, you know, let's say bias, right? Because it is a bias to look after, you know, them who brought you to the dance. But I think you can be responsible with balancing, sort of, the needs of your employer, and the needs of the community. You know, the way I've always described this is that if you think about observability as, like, a—you know, as a market, what's the total addressable market there? It's literally everyone that uses software; it's literally every software company.Which means there's plenty of room for people to make their numbers and to buy and sell and trade and do all this sort of stuff. And by taking that approach, by taking sort of the big picture approach and saying, “Well, look, you know, there's going to be—you know, of all these people, there are going to be some of them that are going to use our stuff and there are some of them that are going to use our competitor's stuff.” And that's fine. Let's figure out where we can invest… in an OpenTelemetry, in a way that makes sense for everyone and not just, you know, our people. So, let's build things like documentation, right?You know, one of the things I'm most impressed with, with OpenTelemetry over the past, like, two years is we went from being, as a project, like, if you searched for OpenTelemetry, you would go and you would get five or six or ten different vendor pages coming up trying to tell you, like, “This is how you use it, this is how you use it.” And what we've done as a community is we've said, you know, “If you go looking for documentation, you should find our website. You should find our resources.” And we've managed to get the OpenTelemetry website to basically rank above almost everything else when people are searching for help with OpenTelemetry. And that's been really good because, one, it means that now, rather than vendors or whoever coming in and saying, like, “Well, we can do this better than you,” we can be like, “Well, look, just, you know, put your effort here, right? It's already the top result. It's already where people are coming, and we can prove that.”And two, it means that as people come in, they're going to be put into this process of community feedback, where they can go in, they can look at the docs, and they can say, “Oh, well, I had a bad experience here,” or, “How do I do this?” And we get that feedback and then we can improve the docs for everyone else by acting on that feedback, and the net result of this is that more people are using OpenTelemetry, which means there are more people kind of going into the tippy-tippy top of the funnel, right, that are able to become a customer of one of these myriad observability back ends.Corey: You touched on something very important here, when I first was exploring this—you may have been looking over my shoulder as I went through this process—my impression initially was, oh, this is a ‘CNCF project' in quotes, where—this is not true universally, of course, but there are cases where it clearly—is where this is an, effectively, vendor-captured project, not necessarily by one vendor, but by an almost consortium of them. And that was my takeaway from OpenTelemetry. It was conversations with you, among others, that led me to believe no, no, this is not in that vein. This is clearly something that is a win. There are just a whole bunch of vendors more-or-less falling all over themselves, trying to stake out thought leadership and imply ownership, on some level, of where these things go. But I definitely left with a sense that this is bigger than any one vendor.Austin: I would agree. I think, to even step back further, right, there's almost two different ways that I think vendors—or anyone—can approach OpenTelemetry, you know, from a market perspective, and one is to say, like, “Oh, this is socializing, kind of, the maintenance burden of instrumentation.” Which is a huge cost for commercial players, right? Like, if you're a Datadog or a Splunk or whoever, you know, you have these agents that you go in and they rip telemetry out of your web servers, out of your gRPC libraries, whatever, and it costs a lot of money to pay engineers to maintain those instrumentation agents, right? And the cynical take is, oh, look at all these big companies that are kind of like pushing all that labor onto the open-source community, and you know, I'm not casting any aspersions here, like, I do think that there's an element of truth to it though because, yeah, that is a huge fixed cost.And if you look at the actual lived reality of people and you look at back when SignalFx was still a going concern, right, and they had their APM agents open-sourced, you could go into the SignalFx repo and diff, like, their [Node Express 00:10:15] instrumentation against the Datadog Node Express instrumentation, and it's almost a hundred percent the same, right? Because it's truly a commodity. There's no—there's nothing interesting about how you get that telemetry out. The interesting stuff all happens after you have the telemetry and you've sent it to some back-end, and then you can, you know, analyze it and find interesting things. So, yeah, like, it doesn't make sense for there to be five or six or eight different companies all competing to rebuild the same wheels over and over and over and over when they don't have to.I think the second thing that some people are starting to understand is that it's like, okay, let's take this a step beyond instrumentation, right? Because the goal of OpenTelemetry really is to make sure that this instrumentation is native so that you don't need a third-party agent, you don't need some other process or jar or whatever that you drop in and it instruments stuff for you. The JVM should provide this, your web framework should provide this, your RPC library should provide this right? Like, this data should come from the code itself and be in a normalized fashion that can then be sent to any number of vendors or back ends or whatever. And that changes how—sort of, the competitive landscape a lot, I think, for observability vendors because rather than, kind of, what you have now, which is people will competing on, like, well, how quickly can I throw this agent in and get set up and get a dashboard going, it really becomes more about, like, okay, how are you differentiating yourself against every other person that has access to the same data, right? And you get more interesting use cases and how much more interesting analysis features, and that results in more innovation in, sort of, the industry than we've seen in a very long time.Corey: For me, just from the customer side of the world, one of the biggest problems I had with observability in my career as an SRE-type for years was you would wind up building your observability pipeline around whatever vendor you had selected and that meant emphasizing the things they were good at and de-emphasizing the things that they weren't. And sometimes it's worked to your benefit; usually not. But then you always had this question when it got things that touched on APM or whatnot—or Application Performance Monitoring—where oh, just embed our library into this. Okay, great. But a year-and-a-half ago, my exposure to this was on an application that I was running in distributed fashion on top of AWS Lambda.So great, you can either use an extension for this or you can build in the library yourself, but then there's always a question of precedence where when you have multiple things that are looking at this from different points of view, which one gets done first? Which one is going to see the others? Which one is going to enmesh the other—enclose the others in its own perspective of the world? And it just got incredibly frustrating. One of the—at least for me—bright lights of OTel was that it got away from that where all of the vendors receiving telemetry got the same view.Austin: Yeah. They all get the same view, they all get the same data, and you know, there's a pretty rich collection of tools that we're starting to develop to help you build those pipelines yourselves and really own everything from the point of generation to intermediate collection to actually outputting it to wherever you want to go. For example, a lot of really interesting work has come out of the OpenTelemetry collector recently; one of them is this feature called Connectors. And Connectors let you take the output of certain pipelines and route them as inputs to another pipeline. And as part of that connection, you can transform stuff.So, for example, let's say you have a bunch of [spans 00:14:05] or traces coming from your API endpoints, and you don't necessarily want to keep all those traces in their raw form because maybe they aren't interesting or maybe there's just too high of a volume. So, with Connectors, you can go and you can actually convert all of those spans into metrics and export them to a metrics database. You could continue to save that span data if you want, but you have options now, right? Like, you can take that span data and put it into cold storage or put it into, like, you know, some sort of slow blob storage thing where it's not actively indexed and it's slow lookups, and then keep a metric representation of it in your alerting pipeline, use metadata exemplars or whatever to kind of connect those things back. And so, when you do suddenly see it's like, “Oh, well, there's some interesting p99 behavior,” or we're hitting an alert or violating an SLO or whatever, then you can go back and say, like, “Okay, well, let's go dig through the slow da—you know, let's look at the cold data to figure out what actually happened.”And those are features that, historically, you would have needed to go to a big, important vendor and say, like, “Hey, here's a bunch of money,” right? Like, “Do this for me.” Now, you have the option to kind of do all that more interesting pipeline stuff yourself and then make choices about vendors based on, like, who is making a tool that can help me with the problem that I have? Because most of the time, I don't—I feel like we tend to treat observability tools as—it depends a lot on where you sit in the org—but you certainly seen this movement towards, like, “Well, we don't want a tool; we want a platform. We want to go to Lowe's and we want to get the 48-in-one kit that has a bunch of things in it. And we're going to pay for the 48-in-one kit, even if we only need, like, two things or three things out of it.”OpenTelemetry lets you kind of step back and say, like, “Well, what if we just got, like, really high-quality tools for the two or three things we need, and then for the rest of the stuff, we can use other cheaper options?” Which is, I think, really attractive, especially in today's macroeconomic conditions, let's say.Corey: One thing I'm trying to wrap my head around because we all find when it comes to observability, in my experience, it's the parable of three blind people trying to describe an elephant by touch; depending on where you are on the elephant, you have a very different perspective. What I'm trying to wrap my head around is, what is the vision for OpenTelemetry? Is it specifically envisioned to be the agent that runs wherever the workload is, whether it's an agent on a host or a layer in a Lambda function, or a sidecar or whatnot in a Kubernetes cluster that winds up gathering and sending data out? Or is the vision something different? Because part of what you're saying aligns with my perspective on it, but other parts of it seem to—that there's a misunderstanding somewhere, and it's almost certainly on my part.Austin: I think the long-term vision is that you as a developer, you as an SRE, don't even have to think about OpenTelemetry, that when you are using your container orchestrator or you are using your API framework or you're using your Managed API Gateway, or any kind of software that you're building something with, that the telemetry data from that software is emitted in an OpenTelemetry format, right? And when you are writing your code, you know, and you're using gRPC, let's say, you could just natively expect that OpenTelemetry is kind of there in the background and it's integrated into the actual libraries themselves. And so, you can just call the OpenTelemetry API and it's part of the standard library almost, right? You add some additional metadata to a span and say, like, “Oh, this is the customer ID,” or, “This is some interesting attribute that I want to track for later on,” or, “I'm going to create a histogram here or counter,” whatever it is, and then all that data is just kind of there, right, invisible to you unless you need it. And then when you need it, it's there for you to kind of pick up and send off somewhere to any number of back-ends or databases or whatnot that you could then use to discover problems or better model your system.That's the long-term vision, right, that it's just there, everyone uses it. It is a de facto and du jour standard. I think in the medium term, it does look a little bit more like OpenTelemetry is kind of this Swiss army knife agent that's running on—inside cars in Kubernetes or it's running on your EC2 instance. Until we get to the point of everyone just agrees that we're going to use OpenTelemetry protocol for the data and we're going to use all your stuff and we just natively emit it, then that's going to be how long we're in that midpoint. But that's sort of the medium and long-term vision I think. Does that track?Corey: It does. And I'm trying to equate this to—like the evolution back in the Stone Age was back when I was first getting started, Nagios was the gold standard. It was kind of the original Call of Duty. And it was awful. There were a bunch of problems with it, but it also worked.And I'm not trying to dunk on the people who built that. We all stand on the shoulders of giants. It was an open-source project that was awesome doing exactly what it did, but it was a product built for a very different time. It completely had the wheels fall off as soon as you got to things were even slightly ephemeral because it required this idea of the server needed to know where all of the things that was monitoring lived as an individual host basis, so there was this constant joy of, “Oh, we're going to add things to a cluster.” Its perspective was, “What's a cluster?” Or you'd have these problems with a core switch going down and suddenly everything else would explode as well.And even setting up an on-call rotation for who got paged when was nightmarish. And a bunch of things have evolved since then, which is putting it mildly. Like, you could say that about fire, the invention of the wheel. Yeah, a lot of things have evolved since the invention of the wheel, and here we are tricking sand into thinking. But we find ourselves just—now it seems that the outcome of all of this has been instead of one option that's the de facto standard that's kind of terrible in its own ways, now, we have an entire universe of different products, many of which are best-of-breed at one very specific thing, but nothing's great at everything.It's the multifunction printer conundrum, where you find things that are great at one or two things at most, and then mediocre at best at the rest. I'm excited about the possibility for OpenTelemetry to really get to a point of best-of-breed for everything. But it also feels like the money folks are pushing for consolidation, if you believe a lot of the analyst reports around this of, “We already pay for seven different observability vendors. How about we knock it down to just one that does all of these things?” Because that would be terrible. What do you land on that?Austin: Well, as I intu—or alluded to this earlier, I think the consolidation in the observability space, in general, is very much driven by that force you just pointed out, right? The buyers want to consolidate more and more things into single tools. And I think there's a lot of… there are reasons for that that—you know, there are good reasons for that, but I also feel like a lot of those reasons are driven by fundamentally telemetry-side concerns, right? So like, one example of this is if you were Large Business X, and you see—you are an engineering director and you get a report, that's like, “We have eight different metrics products.” And you're like, “That seems like a lot. Let's just use Brand X.”And Brand X will tell you very, very happily tell you, like, “Oh, you just install our thing everywhere and you can get rid of all these other tools.” And usually, there's two reasons that people pick tools, right? One reason is that they are forced to and then they are forced to do a bunch of integration work to get whatever the old stuff was working in the new way, but the other reason is because they tried a bunch of different things and they found the one tool that actually worked for them. And what happens invariably in these sort of consolidation stories is, you know, the new vendor comes in on a shining horse to consolidate, and you wind up instead of eight distinct metrics tools, now you have nine distinct metrics tools because there's never any bandwidth for people to go back and, you know—you're Nagios example, right, Nag—people still use Nagios every day. What's the economic justification to take all those Nagios installs, if they're working, and put them into something else, right?What's the economic justification to go and take a bunch of old software that hasn't been touched for ten years that still runs and still does what needs to do, like, where's the incentive to go and re-instrument that with OpenTelemetry or anything else? It doesn't necessarily exist, right? And that's a pretty, I think, fundamental decision point in everyone's observability journey, which is what do you do about all the old stuff? Because most of the stuff is the old stuff and the worst part is, most of the stuff that you make money off of is the old stuff as well. So, you can't ignore it, and if you're spending, you know, millions of millions of dollars on the new stuff—like, there was a story that went around a while ago, I think, Coinbase spent something like, what, $60 million on Datadog… I hope they asked for it in real money and not Bitcoin. But—Corey: Yeah, something I've noticed about all the vendors, and even Coinbase themselves, very few of them actually transact in cryptocurrency. It's always cash on the barrelhead, so to speak.Austin: Yeah, smart. But still, like, that's an absurd amount of money [laugh] for any product or service, I would argue, right? But that's just my perspective. I do think though, it goes to show you that you know, it's very easy to get into these sort of things where you're just spending over the barrel to, like, the newest vendor that's going to come in and solve all your problems for you. And just, it often doesn't work that way because most places aren't—especially large organizations—just aren't built in is sort of like, “Oh, we can go through and we can just redo stuff,” right? “We can just roll out a new agent through… whatever.”We have mainframes [unintelligible 00:25:09], mainframes to thinking about, you have… in many cases, you have an awful lot of business systems that most, kind of, cloud people don't like, think about, right, like SAP or Salesforce or ServiceNow, or whatever. And those sort of business process systems are actually responsible for quite a few things that are interesting from an observability point of view. But you don't see—I mean, hell, you don't even see OpenTelemetry going out and saying, like, “Oh, well, here's the thing to let you know, observe Apex applications on Salesforce,” right? It's kind of an undiscovered country in a lot of ways and it's something that I think we will have to grapple with as we go forward. In the shorter term, there's a reason that OpenTelemetry mostly focuses on cloud-native applications because that's a little bit easier to actually do what we're trying to do on them and that's where the heat and light is. But once we get done with that, then the sky is the limit.[midroll 00:26:11]Corey: It still feels like OpenTelemetry is evolving rapidly. It's certainly not, I don't want to say it's not feature complete, which, again, what—software is never done. But it does seem like even quarter-to-quarter or month-to-month, its capabilities expand massively. Because you apparently enjoy pain, you're in the process of writing a book. I think it's in early release or early access that comes out next year, 2024. Why would you do such a thing?Austin: That's a great question. And if I ever figure out the answer I will tell you.Corey: Remember, no one wants to write a book; they want to have written the book.Austin: And the worst part is, is I have written the book and for some reason, I went back for another round. I—Corey: It's like childbirth. No one remembers exactly how horrible it was.Austin: Yeah, my partner could probably attest to that. Although I was in the room, and I don't think I'd want to do it either. So, I think the real, you know, the real reason that I decided to go and kind of write this book—and it's Learning OpenTelemetry; it's in early release right now on the O'Reilly learning platform and it'll be out in print and digital next year, I believe, we're targeting right now, early next year.But the goal is, as you pointed out so eloquently, OpenTelemetry changes a lot. And it changes month to month sometimes. So, why would someone decide—say, “Hey, I'm going to write the book about learning this?” Well, there's a very good reason for that and it is that I've looked at a lot of the other books out there on OpenTelemetry, on observability in general, and they talk a lot about, like, here's how you use the API. Here's how you use the SDK. Here's how you make a trace or a span or a log statement or whatever. And it's very technical; it's very kind of in the weeds.What I was interested in is saying, like, “Okay, let's put all that stuff aside because you don't necessarily…” I'm not saying any of that stuff's going to change. And I'm not saying that how to make a span is going to change tomorrow; it's not, but learning how to actually use something like OpenTelemetry isn't just knowing how to create a measurement or how to create a trace. It's, how do I actually use this in a production system? To my point earlier, how do I use this to get data about, you know, these quote-unquote, “Legacy systems?” How do I use this to monitor a Kubernetes cluster? What's the important parts of building these observability pipelines? If I'm maintaining a library, how should I integrate OpenTelemetry into that library for my users? And so on, and so on, and so forth.And the answers to those questions actually probably aren't going to change a ton over the next four or five years. Which is good because that makes it the perfect thing to write a book about. So, the goal of Learning OpenTelemetry is to help you learn not just how to use OpenTelemetry at an API or SDK level, but it's how to build an observability pipeline with OpenTelemetry, it's how to roll it out to an organization, it's how to convince your boss that this is what you should use, both for new and maybe picking up some legacy development. It's really meant to give you that sort of 10,000-foot view of what are the benefits of this, how does it bring value and how can you use it to build value for an observability practice in an organization?Corey: I think that's fair. Looking at the more quote-unquote, “Evergreen,” style of content as opposed to—like, that's the reason for example, I never wind up doing tutorials on how to use an AWS service because one console change away and suddenly I have to redo the entire thing. That's a treadmill I never had much interest in getting on. One last topic I want to get into before we wind up wrapping the episode—because I almost feel obligated to sprinkle this all over everything because the analysts told me I have to—what's your take on generative AI, specifically with an eye toward observability?Austin: [sigh], gosh, I've been thinking a lot about this. And—hot take alert—as a skeptic of many technological bubbles over the past five or so years, ten years, I'm actually pretty hot on AI—generative AI, large language models, things like that—but not for the reasons that people like to kind of hold them up, right? Not so that we can all make our perfect, funny [sigh], deep dream, meme characters or whatever through Stable Fusion or whatever ChatGPT spits out at us when we ask for a joke. I think the real win here is that this to me is, like, the biggest advance in human-computer interaction since resistive touchscreens. Actually, probably since the mouse.Corey: I would agree with that.Austin: And I don't know if anyone has tried to get someone that is, you know, over the age of 70 to use a computer at any time in their life, but mapping human language to trying to do something on an operating system or do something on a computer on the web is honestly one of the most challenging things that faces interface design, face OS designers, faces anyone. And I think this also applies for dev tools in general, right? Like, if you think about observability, if you think about, like, well, what are the actual tasks involved in observability? It's like, well, you're making—you're asking questions. You're saying, like, “Hey, for this metric named HTTPrequestsByCode,” and there's four or five dimensions, and you say, like, “Okay, well break this down for me.” You know, you have to kind of know the magic words, right? You have to know the magic promQL sequence or whatever else to plug in and to get it to graph that for you.And you as an operator have to have this very, very well developed, like, depth of knowledge and math and statistics to really kind of get a lot of—Corey: You must be at least this smart to ride on this ride.Austin: Yeah. And I think that, like that, to me is the real—the short-term win for certainly generative AI around using, like, large language models, is the ability to create human language interfaces to observability tools, that—Corey: As opposed to learning your own custom SQL dialect, which I see a fair number of times.Austin: Right. And, you know, and it's actually very funny because there was a while for the—like, one of my kind of side projects for the past [sigh] a little bit [unintelligible 00:32:31] idea of, like, well, can we make, like, a universal query language or universal query layer that you could ship your dashboards or ship your alerts or whatever. And then it's like, generative AI kind of just, you know, completely leapfrogs that, right? It just says, like, well, why would you need a query language, if we can just—if you can just ask the computer and it works, right?Corey: The most common programming language is about to become English.Austin: Which I mean, there's an awful lot of externalities there—Corey: Which is great. I want to be clear. I'm not here to gatekeep.Austin: Yeah. I mean, I think there's a lot of externalities there, and there's a lot—and the kind of hype to provable benefit ratio is very skewed right now towards hype. That said, one of the things that is concerning to me as sort of an observability practitioner is the amount of people that are just, like, whole-hog, throwing themselves into, like, oh, we need to integrate generative AI, right? Like, we need to put AI chatbots and we need to have ChatGPT built into our products and da-da-da-da-da. And now you kind of have this perfect storm of people that really don't ha—because they're just using these APIs to integrate gen AI stuff with, they really don't understand what it's doing because a lot you know, it is very complex, and I'll be the first to admit that I really don't understand what a lot of it is doing, you know, on the deep, on the foundational math side.But if we're going to have trust in, kind of, any kind of system, we have to understand what it's doing, right? And so, the only way that we can understand what it's doing is through observability, which means it's incredibly important for organizations and companies that are building products on generative AI to, like, drop what—you know, walk—don't walk, run towards something that is going to give you observability into these language models.Corey: Yeah. “The computer said so,” is strangely dissatisfying.Austin: Yeah. You need to have that base, you know, sort of, performance [goals and signals 00:34:31], obviously, but you also need to really understand what are the questions being asked. As an example, let's say you have something that is tokenizing questions. You really probably do want to have some sort of observability on the hot path there that lets you kind of break down common tokens, especially if you were using, like, custom dialects or, like, vectors or whatever to modify the, you know, neural network model, like, you really want to see, like, well, what's the frequency of the certain tokens that I'm getting they're hitting the vectors versus not right? Like, where can I improve these sorts of things? Where am I getting, like, unexpected results?And maybe even have some sort of continuous feedback mechanism that it could be either analyzing the tone and tenor of end-user responses or you can have the little, like, frowny and happy face, whatever it is, like, something that is giving you that kind of constant feedback about, like, hey, this is how people are actually like interacting with it. Because I think there's way too many stories right now people just kind of like saying, like, “Oh, okay. Here's some AI-powered search,” and people just, like, hating it. Because people are already very primed to distrust AI, I think. And I can't blame anyone.Corey: Well, we've had an entire lifetime of movies telling us that's going to kill us all.Austin: Yeah.Corey: And now you have a bunch of, also, billionaire tech owners who are basically intent on making that reality. But that's neither here nor there.Austin: It isn't, but like I said, it's difficult. It's actually one of the first times I've been like—that I've found myself very conflicted.Corey: Yeah, I'm a booster of this stuff; I love it, but at the same time, you have some of the ridiculous hype around it and the complete lack of attention to safety and humanity aspects of it that it's—I like the technology and I think it has a lot of promise, but I want to get lumped in with that set.Austin: Exactly. Like, the technology is great. The fan base is… ehh, maybe something a little different. But I do think that, for lack of a better—not to be an inevitable-ist or whatever, but I do think that there is a significant amount of, like, this is a genie you can't put back in the bottle and it is going to have, like, wide-ranging, transformative effects on the discipline of, like, software development, software engineering, and white collar work in general, right? Like, there's a lot of—if your job involves, like, putting numbers into Excel and making pretty spreadsheets, then ooh, that doesn't seem like something that's going to do too hot when I can just have Excel do that for me.And I think we do need to be aware of that, right? Like, we do need to have that sort of conversation about, like… what are we actually comfortable doing here in terms of displacing human labor? When we do displace human labor, are we doing it so that we can actually give people leisure time or so that we can just cram even more work down the throats of the humans that are left?Corey: And unfortunately, I think we might know what that answer is, at least on our current path.Austin: That's true. But you know, I'm an optimist.Corey: I… don't do well with disappointment. Which the show has certainly not been. I really want to thank you for taking the time to speak with me today. If people want to learn more, where's the best place for them to find you?Austin: Welp, I—you can find me on most social media. Many, many social medias. I used to be on Twitter a lot, and we all know what happened there. The best place to figure out what's going on is check out my bio, social.ap2.io will give you all the links to where I am. And yeah, been great talking with you.Corey: Likewise. Thank you so much for taking the time out of your day. Austin Parker, community maintainer for OpenTelemetry. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment pointing out that actually, physicists say the vast majority of the universe's empty space, so that we can later correct you by saying ah, but it's empty whitespace. That's right. YAML wins again.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

Screaming in the Cloud
How Redpanda Extracts Business Value from Data Events with Alex Gallego

Screaming in the Cloud

Play Episode Listen Later Aug 31, 2023 34:43


Alex Gallego, CEO & Founder of Redpanda, joins Corey on Screaming in the Cloud to discuss his experience founding and scaling a successful data streaming company over the past 4 years. Alex explains how it's been a fun and humbling journey to go from being an engineer to being a founder, and how he's built a team he trusts to hand the production off to. Corey and Alex discuss the benefits and various applications of Redpanda's data streaming services, and Alex reveals why it was so important to him to focus on doing one thing really well when it comes to his product strategy. Alex also shares details on the Hack the Planet scholarship program he founded for individuals in underrepresented communities. About AlexAlex Gallego is the founder and CEO of Redpanda, the streaming data platform for developers. Alex has spent his career immersed in deeply technical environments, and is passionate about finding and building solutions to the challenges of modern data streaming. Prior to Redpanda, Alex was a principal engineer at Akamai, as well as co-founder and CTO of Concord.io, a high-performance stream-processing engine acquired by Akamai in 2016. He has also engineered software at Factset Research Systems, Forex Capital Markets and Yieldmo; and holds a bachelor's degree in computer science and cryptography from NYU. Links Referenced: Redpanda: https://redpanda.com/ Twitter: https://twitter.com/emaxerrno Redpanda community Slack: https://redpandacommunity.slack.com/join/shared_invite/zt-1xq6m0ucj-nI41I7dXWB13aQ2iKBDvDw Hack The Planet Scholarship: https://redpanda.com/scholarship TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: Tired of slow database performance and bottlenecks on MySQL or PostgresSQL when using Amazon RDS or Aurora? How'd you like to reduce query response times by ninety percent? Better yet, how would you like to get me to pronounce database names correctly? Join customers like Zscaler, Intel, Booking.com, and others that use OtterTune's artificial intelligence to automatically optimize and keep their databases healthy. Go to ottertune dot com to learn more and start a free trial. That's O-T-T-E-R-T-U-N-E dot com.Corey: Welcome to Screaming in the Cloud, I'm Corey Quinn, and this promoted guest episode is brought to us by our friends at Redpanda, which I'm thrilled about because I have a personal affinity for companies that have cartoon mascots in the form of animals and are willing to at least be slightly creative with them. My guest is Alex Gallego, the founder and CEO over at Redpanda. Alex, thanks for joining me.Alex: Corey, thanks for having me.Corey: So, I'm not asking about the animal; I'm talking about the company, which I imagine is a frequent source of disambiguation when you meet people at parties and they don't quite understand what it is that you do. And you folks are big in the data streaming space, but data streaming can mean an awful lot of things to an awful lot of people. What is it for you?Alex: Largely it's about enabling developers to build applications that can extract value of every single event, every click, every mouse movement, every transaction, every event that goes through your network. This is what Redpanda is about. It's like how do we help you make more money with every single event? How do we help you be more successful? And you know, happy to give examples in finance, or IoT, or oil and gas, if it's helpful for the audience, but really, to me, it's like, okay, if we can give you the framework in which you can build a new application that allows you to extract value out of data, every single event that's going through your network, to me, that's what a streaming is about. It large, it's you know, data contextualized with a timestamp and largely, a sort of a database of event streaming.Corey: One of the things that I find curious about the space is that usually, companies wind up going one of two directions when you're talking about data streaming. Either there, “Oh, just send it all to us and we'll take care of it for you,” or otherwise, it's a, great they more or less ship something that you've run in your own environment. In the olden days of data centers, that usually resembled a box of some sort. You're one of those interesting split-the-difference companies where you offer both models. Do you find that one of those tends to be seeing more adoption these days or that there's an increasing trend toward one direction or the other?Alex: Yeah. So, right now, I think that to me, the future of all these data-intensive products—whether you're a database or a streaming engine—will, because simply of cost of networks transferred between the hybrid clouds and your accounts, sending a gigabyte a second of data between, let's say, you know, your data center and a vendor, it's just so expensive that at some point, from just a cost perspective, like, running the infrastructure, it's in the millions of dollars. And so, running the data inside your VPC, it's sort of the next logical evolution of how we've used to consume services. And so, I actually think it's just the evolution: people would self-host because of costs and then they would use services because of operational simplicity. “I don't want to spend team skills and time building this. I want to pay a vendor.”And so, BYOC, to be honest—which is what we call this offering—it was about [laugh] sidestepping the costs and of being stuck in the hybrid clouds, whether it's Google or Amazon, where you're paying egress and ingress costs and it's just so expensive, in addition to this whole idea of data residency or data sovereignty and privacy. It's like, yeah, why not both? Like, if I'm an engineer, I want low latency and I don't want to pay you to transfer this thing to the next rack. I mean, my computer's probably, like, you know, a hundred feet away from my customer's computer. Like, why [laugh] way is that so complicated? So, you know, my view is that the future of data-intensive products will be in this form of where it—like, data planes are actually owned by companies, and then you offer that as a Software as a Service.Corey: One of the things that catches an awful lot of companies with telemetry use cases—or data streaming as another example of that—by surprise when they start building their own cloud-hosted offering is that they're suddenly seeing a lot more cross-AZ data charges than they would have potentially expected. And that's because unlike cross-region or the really expensive version of this with egress, it's a penny in and a penny out per gigabyte in most of AWS regions. Which means that that isn't also bound strictly to an AWS organization. So, you have customers co-located with you and you're starting to pay ingress charges on customers throwing their data over to you. And, on some level, the most economical solution for you is well, we're just going to put our listeners somewhere else far away so that we can just have them pay the steep egress fee but then we can just reflect it back to ourselves for free.And that's a terrible pattern, but it's a byproduct of the absolutely byzantine cross-AZ data transfer pricing, in fact, all of the data transfer pricing that is at least AWS tends to present. And it shapes the architectural decisions you make as a result.Alex: You know, as a user, it just didn't make sense. When we launched this product, the number of people that says like, “Why wouldn't your charge for, you know, effectively renting [unintelligible 00:05:14], and giving a markup to your customers?” That's we don't add any value on that, you know? I think people should really just pay us for the value that we create for them. And so, you know, for us competing with other companies is relatively easy.Competing with MSK is it's harder because MSK just has this, you know, muscle where they don't charge you for some particular network traffic between you. And so, it forces companies like us that are trying to be innovative in the data space to, like, put our services in that so that we can actually compete in the market. And so, it's a forcing function of the hybrid clouds having this strong muscle of being able to discount their services in a way that companies just simply don't have access to. And then, you know, it becomes—for the others—latency and sovereignty.Corey: This is the way that effectively all of AWS has first-party offerings of other things go. Replication traffic between AZs is not chargeable. And when I asked them about that, they say, “Oh, yeah. We just price that into the cost of the service.” I don't know that I necessarily buy that because if I try and run this sort of thing on top of EC2, it would cost me more than using their crappy implementation of it, just in data transfer alone for an awful lot of use cases.No third party can touch that level of cost-effectiveness and discounting. It really is probably the clearest example I can think of actual anti-competitive behavior in the market. But it's also complex enough to explain, to, you know, regulators that it doesn't make for exciting exposés and the basis for lawsuits. Yet. Hope springs eternal.Alex: [laugh]. You know—okay, so here is how—if someone is listening to this podcast and is, like, “Okay, well, what can I do?” For us, S3 is the answer. S3 is basically you need to be able to lean in into S3 as a way of replication across [AZ 00:06:56], you need to be able to lean into S3 to read data. And so actually, when I wrote, originally, Redpanda, you know, it's just like this C++ thing using [unintelligible 00:07:04], geared towards super low latency.When we moved it into the cloud, what we realized is, this is cost prohibitive to run either on EBS volumes or local disk. I have to tier all the storage into S3, so that I can use S3's cross-AZ network transfer, which is basically free, to be able to then bring a separate cluster on a different AZ, and then read from the bucket at zero cost. And so, you end up really—like, there are fundamental technical things that you have to do to just be able to compete in a way that's cost-effective for you. And so, in addition to just, like, the muscle that they can enforce on the companies is—it—there are deep implications of what it translates to at the technical level. Like, at the code level.Corey: In the cloud, more than almost anywhere else, it really does become apparent that cost and architecture are fundamentally the same thing. And I have a bit of an advantage here in that I've seen what you do deployed at least one customer of mine. It's fun. When you have a bunch of logos on your site, it's, “Hey, I recognize some of those.” And what I found interesting was the way that multiple people, when I spoke to them, described what it is that you do because some of them talked about it purely as a cost play, but other people were just as enthusiastic about it being a means of improving feature velocity and unlocking capabilities that they didn't otherwise have or couldn't have gotten to without a whole lot of custom work on their part. Which is it? How do you view what it is that you're bringing to market? Is it a cost play or is it a capability story?Alex: From our customer base, I would say 40% is—of our customer base—is about Redpanda enabling them to do things that they simply couldn't do before. An example is, we have, you know, a Fortune 100 company that they basically run their hedge trading strategy on top of Redpanda. And the reason for that is because we give them a five-millisecond average latency with predictable flight latencies, right? And so, for them, that predictability of Redpanda, you know, and sort of like the architecture that came about from trying to invent a new storage engine, allows them to throw away a bunch of in-house, you know, custom-built pub/sub messaging that, you know, basically gave them the same or worse latency. And so, for them, there's that.For others, I think in the IoT space, or if you have flying vehicles around the world, we have some logos that, you know, I just can't mention them. But they have this, like, flying computers around the world and they want to measure that. And so, like, the profile of the footprint, like, the mechanical footprint of being able to run on a single Pthread with a few megs of memory allows these new deployment models that, you know, simply, it's just, it's not possible with the alternatives where let's say you have to have, you know, like, a zookeeper on the schema registry and an HTTP proxy and a broker and all of these things. That simply just, it cannot run on a single Pthread with a few megs of memory, if you put any sort of workload into that. And so, it's like, the computational efficiencies simply enable new things that you couldn't do before. And that's probably 40%. And then the other, it's just… money was really cheap last year [laugh] or the year before and I think now it's less cheap [unintelligible 00:10:08] yeah.Corey: Yeah, I couldn't help but notice that in my own business, too. It turns out that not giving a shit about the AWS bill was a zero-interest-rate phenomenon. Who knew?Alex: [laugh]. Yeah, exactly. And now people [unintelligible 00:10:17], you know, the CIOs in particular, it's like, help. And so, that's really 60%, and our business has boomed since.Corey: Yeah, one thing that I find interesting is that you've been around for only four years. I know that's weird to say ‘only,' but time moves differently in tech. And you've started showing up in some very strange places that I would not have expected. You recently—somewhat recently; time is, of course, a flat circle—completed $100 million Series C, and I also saw you in places where I didn't expect to see you in the form of, last week, one of your large competitor's earnings calls, where they were asked by an analyst about an unnamed company that had raised $100 million Series C, and the CEO [unintelligible 00:11:00], “Oh, you're probably talking about Redpanda.” And then they gave an answer that was fine.I mean, no one is going to be on an earnings call and not be prepared for questions like that and to not have an answer ready to go. No one's going to say, “Well, we're doomed if it works,” because I think that businesses are more sophisticated than that. But it was an interesting shout-out in a place where you normally don't see competitors validate that you're doing something interesting by name-checking you.Alex: What was fundamentally interesting for me about that, is that I feel that as an investor, if you're putting you know, 2, 3, 4, or $500 million check into a public position of a company, you want to know, is this money simply going to make returns? That's basically what an investor cares about. And so, the reason for that question is, “Hey, there's a Series C startup company that now has a bunch of these Fortune 2000 logos,” and you know, when we talked to them, like, their customer [unintelligible 00:11:51] phenomena, like, why is that the case? And then, you know, our competitor was forced to name, you know, [laugh] a single win. That's as far as I remember it. We don't know of any additional customers that have switched to that.And so, I think when you have, like, you know, your win rate is above, whatever, 95%, 97% ratio, then I think, you know, they're just sort of forced to answer that. And in a way, I just think that they focus on different things. And for me, it was like, “Okay, developer, hands on keyboard, behind the terminal, how do I make you successful?” And that seems to have worked out enough to be mentioned in the earnings call.Corey: On some level, it's a little bit of a dog-and-pony show. I think that as companies had a certain point of scale, they feel that they need to validate what they're doing to investors at various points—which is always, on some level, of concern—and validate themselves to analysts, both financial—which, okay, whatever—and also, industry analysts, where they come with checklists that they believe is what customers want and is often a little bit off of the mark. But the validation that I think that matters, that actually determines whether or not something has legs is what your customers—you know, people paying you money for a thing—have to say and what they take away from what you're doing. And having seen in a couple of cases now myself, that usage of Redpanda has increased after initial proofs of concept and putting things on to it, I already sort of know the answer to this, but it seems that you also have a vibrant community of boosters for people who are thrilled to use the thing you're selling them.Alex: You know, Jumptraders recently posted that there was a use case in the new stack where they, like, put for the most mission-critical. So, for those of you that listening, Jumptraders is financial company, and they're super technical company. One of, like, the hardest things, they'll probably put your [unintelligible 00:13:35] your product through some of the most rigorous testing [unintelligible 00:13:38]. So, when you start doing some of these logos, it gives confidence. And actually, the majority of our developers that we get to partner with, it was really a friend telling a friend, for [laugh] the longest time, my marketing department was super, super small.And then what's been fun, some, like, really different use case was the one I mentioned about on this, like, flying vehicles around the world. They fly both in outer space and in airplanes. That was really fun. And then the large one is when you have workloads at, like, 14-and-a-half gigabytes per second, where the alternative of using something like Kinesis in the case of Lacework—which, you know, they wrote a new stack article about—would be so exorbitantly expensive. And so, in a way, I think that, you know, just trying to make the developers successful, really focusing, honestly, on the person who just has to make things work. We don't—by the time we get to the CIO, really the champion was the engineer who had to build an application. “I was just trying to figure it out the whack-a-mole of trying to debug alternative systems.”Corey: One of the, I think, seductive problems with your entire space is that no one decides day one that they're going to implement a data streaming solution for a very scaled-out, high-traffic site. The early adoption is always a small thing that you're in the process of building. And at that scale at that speed, it just doesn't feel like it's that hard of a problem because scale introduces its own unique series of challenges, but it's often one that people only really find out themselves when the simple thing that works in theory but not in production starts to cause problems internally. I used to work with someone who was a deeply passionate believer in Apache Kafka to a point where it almost became a problem, just because their answer to every problem—it almost didn't matter if it was, “How do we get more coffee this morning?”—Kafka would be the answer for all of it.And that's great, but it turned out, they became one of these people that borderline took on a product or a technology as their identity. So, anything that would potentially take a workload away from that, I got a lot of internal resistance. I'm wondering if you find that you're being brought in to replace existing systems or for completely greenfield stuff. And if the former, are you seeing a lot of internal resistance to people who have built a little niche for themselves?Alex: It's true, the people that have built a career, especially at large banks, were a pretty good fit for, you know, they actually get a team, they got a promotion cycle because they brought this technology and the technology sort of helped them make money. I personally tend to love to talk to these people. And there was a ca—to me, like, technically, let's talk about, like, deeply technical. Let me help you. That obviously doesn't scale because I can't have the same conversation with ten people.So, we do tend to see some of that. Actually, from our customers' standpoint, I would say that the large part of our customer base, you know, if I'm trying to put numbers, maybe 65%, I probably rip and replace of, you know, either upstream Apache Software or private companies or hosted services, et cetera. And so, I think you're right in saying, “Hey, that resistance,” they probably handled the [unintelligible 00:16:38], but what changed in the last year is that the CIO now stepped in and says, “I am going to fire all of you or you have to come up with a $10 million savings. Help me.” [laugh]. And so, you know, then really, my job is to help them look like a hero.It's like, “Hey, look, try it tested, benchmark it in your with your own workload, and if it saves you money, then use it.” That's been, you know, to sort of super helpful kind of on the macroeconomic environment. And then the last one is sometimes, you know, you do have to go with a greenfield, right? Like, someone has built a career, they want to gain confidence, they want to ask you questions, they want to trust you that you don't lose data, they want to make sure that you do say the things that you want to say. And so, sometimes it's about building trust and building that relationship.And developers are right. Like, there's a bunch of products out there. Like, why should I trust you? And so, a little easier time, probably now, that you know, with the CIOs wanting to cut costs, and now you have an excuse to go back to the executive team and say, “Look, I made you look smart. We get to [unintelligible 00:17:35], you know, our systems can scale to this.” That's easy. Or the second one is we do, you know, we'll start with some side use case or a greenfield. But both exists, and I would say 65% is probably rip-outs.Corey: One question, I love to, I wouldn't call it ambush, but definitely come up with, the catches some folks by surprise is one of the ways I like to sort out zealots from people who are focused on business problems. Do have an example of a data streaming workload for which Redpanda would not be a great fit?Alex: Yeah. Database-style queries are not a fit. And so, think that there was a streaming engine before there was trying to build a database on top of it, and, like—and probably it does work in some low volume of traffic, like, say 5, 10 megabytes per second, but when you get to actual large scale, it just it doesn't work. And it doesn't work because but what Redpanda is, it gives you two properties as a developer. You can add data to the end or you can truncate the head, right?And so, because those are your only two operations on the log, then you have to build this entire caching level to be able to give this database semantics. And so, do you know, I think for that the future isn't for us to build a database, just as an example, it's really to almost invert it. It's like, hey, what if we make our format an open format like Apache Iceberg and then bring in your favorite database? Like, bring in, you know, Snowflake or Athena or Trina or Spark or [unintelligible 00:18:54] or [unintelligible 00:18:55] or whatever the other [unintelligible 00:18:56] of great databases that are better than we are, and doing, you know, just MPP, right, like a massively parallelizable database, do that, and then the job for us, for [unintelligible 00:19:05], let me just structure your log in a way that allows you to query, right? And so, for us, when we announced the $100 million dollar Series C funding, it's like, I'm going to put the data in an iceberg format so you can go and query it with the other ten databases. And there are a better job than we are at that than we are.Corey: It's frankly, refreshing to see a vendor that knows where, okay, this is where we start and this is where we stop because it just seems that there's been an industry-wide push for a while now to oh, you built a component in a larger system that works super well. Now, expand to do everything else in the architectural diagram. And you suddenly have databases trying to be network transport layers and queues trying to be data warehouses, and it just doesn't work that way. It just it feels like oh, this is a terrible approach to solving this particular problem. And what's worse, from my mind, is that people who hadn't heard of you before look at you through this lens that does not put you in your best light, and, “Oh, this is a terrible database.” Well, it's not supposed to be one.Alex: [laugh].Corey: But it also—it puts them off as a result. Have you faced pressure to expand beyond your core competency from either investors or customers or analysts or, I don't know, the voices late at night that I hear and I assume everyone else does, too?Alex: Exactly. The 3 a.m. voice that I have to take my phone and take a voice note because it's like, I don't want to lose this idea. Totally. For us. I think there's pressures, like, hey, you built this great engine. Why don't you add, like, the latest, you know, soup de jour in systems was like a vector database.I was like, “This doesn't even make any sense.” For me, it's, I want to do one thing really well. And I generally call it internally, ‘the ring zero.' It's, if you think of the internet, right, like, as a computer, especially with this mode to what we talked about earlier in a BYOC, like, we could be the best ring zero, the best sort of like, you know, messaging platform for people to build real-time applications. And then that's the case and there's just so much low-hanging fruit for us.Like, the developer experience wasn't great for other systems, like, why don't we focus on the last mile, like, making that developer, you know, successful at doing this one thing as opposed to be an average and a bunch of other a hundred products? And until we feel, honestly, that we've done a phenomenal job at that—I think we still have some roadmap to get there—I don't want to expand. And, like, if there's pressure, my answer is, like… look, the market is big enough. We don't have to do it. We're still, you know, growing.I think it's obviously not trivial and I'm kind of trivializing a bunch of problems from a business perspective. I'm not trying to degrade anyone else. But for us, it's just being focused. This is what we do well. And bring every other technology that makes you successful. I don't really care. I just want to make this part well.Corey: I think that that is something that's under-appreciated. I feel like I should get over at one point to something that's been nagging at the back of my mind. Some would call it a personal attack and I suppose I'll let them, but what I find interesting is your background. Historically, you were a distributed systems engineer at very large scale. And you apparently wrote the first version of Redpanda yourself in—was it C or C++?Alex: C++.Corey: Yeah. And now you are the CEO of a company that is clearly doing very well. Have you gotten the hell out of production yet? The reason I ask this is I have worked in a number of companies where the founder was also the initial engineer and then they invariably treated main as their feature branch and the rest of us all had to work around them to keep them from, you know, destroying everything we were trying to build around us, due to missing context. In other words, how annoyed with you are your engineers on any given afternoon?Alex: [laugh]. Yeah. I would say that as a company builder now, if I may say that, is the team is probably the thing I'm the most proud of. They're just so talented, such good [unintelligible 00:22:47] of humans. And so—group of humans—I stopped coding about two years ago, roughly.So, the company is four-and-a-half years old, really the first two-and-a-half years old, the first one, two years, definitely, I was personally putting in, like, tons and tons of hours working on the code. It was a ton of fun. To me, one of the most rewarding technical projects I've ever had a chance to do. I still read pull requests, though, just so that when I have a conversation with a technical leader, I don't be, like, I have no clue how the transactions work. So, I still have to read the code, but I don't write any more code and my heart was a little broken when my dev prod team removed my write access to the GitHub repo.We got SOC2 compliance, and they're like, “You can't have access to being an admin on Google domains, and you're no longer able to write into main.” And so, I think as a—I don't know, maybe my identity—myself identity is that of a builder, and I think as long as I personally feel like I'm building, today, it's not code, but you know, is the company and [unintelligible 00:23:41] sort of culture, then I feel okay [laugh]. But yeah, I no longer write code. And the last story on that, is this—an engineer of ours, his name is [Stefan 00:23:51], he's like, “Hey, so Alex wrote this semaphore”—this was actually two days ago—and so they posted a video, and I commented, I was like, “Hey, this was the context of semaphore. I'm sorry for this bug I caused.” But yeah, at least I still remember some context for them.Corey: What's fun is watching things continue to outpace and outgrow you. I mean, one of the hard parts of building a company is the realization that every person you hire for a thing that's now getting off of your plate is better at that thing than you are. It's a constant experience of being humbled. And at some point, things wind up outpacing you to the point where, at least in my case, I've been on calls with customers and I explained how we did some things and how it worked and had to be corrected by my team of, “Well. That used to be true, however…” like, “Oh, dear Lord. I'm falling behind.” And that's always been a weird feeling for me.Alex: Totally. You know, it's the feeling of being—before I think I became a CEO, I was a highly comped  engineer and did a competent, to the extent that it allowed me to build this product. And then you start doing all of these things and you're incompetent, obviously, by definition because you haven't done those things and so there's like that discomfort [laugh]. But I have to get it done because no one else wants to do, whatever, like say, like, you know, rev ops or marketing or whatever.And then you find somebody who's great and you're like, oh my God, I was like, I was so poor tactically at doing this thing. And it's definitely humbling every day. And it's almost it's, like, gosh, you're just—this year was kind of this role where you're just, like, mediocre at, like, a whole lot of things as a company, but you're the only person that has to do the job because you have the context and you just have to go and do it. And so, it's definitely humbling. And in some ways, I'm learning, so for me today, it's still a lot of fun to learn.Corey: This is a little more in the weeds, I suppose, but I always love to ask people these questions. Because I used to be naive, which meant that I had hope and I saw a brighter future in technology. I now know that was all a lie. But I used to believe that out there was some company whose internal infrastructure for what they'd built was glorious and it would be amazing. And I knew I would never work there, nor what I want to, because when everything's running perfectly, all I can really do is mess that up; there's no way to win and a bunch of ways to lose.But I found that place doesn't exist. Every time I talk to someone about how they built the thing that they built and I ask them, “If you were starting over from scratch, what would you do differently?” The answer often distills down to, “Oh, everything.” Because it's an organically evolving system that oh, yeah, everything's easier the second time. At least you get to find new failure modes go in that way. When you look back at how you designed it originally, are there any missteps that you could have saved yourself a whole lot of grief by not making the first time?Alex: Gosh, so many things. But if I were to give Hollywood highlights on these things, something that [unintelligible 00:26:35] is, does well is exposing these high-level data types of, like, streams, and lists and maps and et cetera. And I was like, “Well, why couldn't streams offer this as a first-class citizen?” And we got some things well which I think would still do, like the whole [thread recorder 00:26:49] could—like, the fundamentals of the engine I will still do the same. But, you know, exposing new programming models earlier in the life of the product, I think would have allowed us to capture even more wildly different use cases.But now we kind of have this production engine, we have to support Fortune 2000, so you know, it's kind of like a very delicate evolution of the product. Definitely would have changed—I would have added, like, custom data types upfront, I would have pushed a little harder on I think WebAssembly than we did originally. Man, I could just go on for—like, [added detail 00:27:21], I would definitely have changed things. Like, I would have pressed on the first—on the version of the cloud that we talked about early on, that as the first deployment mode. If we go back through the stack of all of the products you had, it's funny, like, 11 products that are surfaced to the customers to, like, business lines, I would change fundamental things about just [laugh], you know, everything else. I think that's maybe the curse of the expert. Like, you know, you could always find improvements.Corey: Oh, always. I still look back at my career before starting this place when I was working in a bunch of finance companies, and—I'll never forget this; it was over a decade ago—we were building out our architecture in AWS, and doing a deal with a large finance company. And they said, “Cool, where's your data center?” And I said, “Oh, it's AWS.” And they said, “Ha ha ha ha. Where's your data center?”And that was oh, okay, great. Now, it feels like if that's their reaction, they have not kept pace with the times. It feels it is easier to go to a lot of very serious enterprises with very serious businesses and serious workload concerns attendant to those and not get laughed out of the room because you didn't wind up doing a multi-million dollar data center build out that, with an eye toward making it look as enterprise-y as possible.Alex: Yeah. Okay, so here's, I think, maybe something a little bit controversial. I think that's true. People are moving to the cloud, and I don't think that that idea, especially when we go when we talk to banks, is true. They're like, “Hey, I have this contract with one of the hybrid clouds.”—you know, it's usually with two of them, and then you're like—“This is my workload. I want to spend $70 million or $100 million. Who could give me the biggest discount?” And then you kind of shop it around.But what we are seeing is that effectively, the data transfer costs are so expensive and running this for so much this large volume of traffic is still so, so expensive, that there is an inverse [unintelligible 00:29:09] to host from some category of the workload where you don't have dynamism. Actually hosted in your data center is, like, a huge boom in terms of cost efficiencies for the companies, especially where we are and especially in finances—you mentioned that—if you're trying to trade and you have this, like, steady state line from nine to five, whatever, eight to four, whenever the markets open, it's actually relatively cost-efficient because you can measure hey, look, you know, the New York Stock Exchange is 1.5 gigabytes per second at market close. Like, I could provision my hardware to beat this. And like, it'll be that I don't need this dynamism that the cloud gives me.And so yeah, it's kind of fascinating that for us because we offered the self-hosted Redpanda which can adapt to super low latencies with kernel parameter tuning, and the cloud due to the tiered storage, we talked about S3 being [unintelligible 00:29:52] to, so it's been really fun to participate in deployments where we have both. And you couldn't—they couldn't look more different. I mean, it's almost looks like two companies.Corey: One last question before we wind up calling it an episode. I think I saw something fly by on Twitter a while back as I slowly returned to the platform—no, I'm not calling it X—something you're doing involving a scholarship. Can you tell me a bit more about that?Alex: Yeah. So, you know, I'm a Latino CEO, first generation in the States, and some of the things that I felt really frustrated with, growing up that, like, I feel fortunate because I got to [unintelligible 00:30:25] that is that, you know, people were just—that look like me are probably given some bullshit QA jobs, so like, you know, behemoth job, I think, for a bank. And so, I wanted to change that. And so, we give money and mentorship to people and we release all of the intellectual property. And so, we mentor someone—actually, anyone from underrepresented backgrounds—for three months.We give then, like, 1200 bucks a month—or 1500, I can't remember—mentorship from our top principal level engineers that have worked at Amazon and Google and Facebook and basically the world's top companies. And so, they meet with them one hour a week, we give them money, they could sit in the couch if they want to. No one has to [unintelligible 00:31:06]. And all we're trying to do is, like, “Hey, if you are part of this group, go and try to build something super hard.” [laugh].And often their minds, which is great, and they're like, “I want to build an OpenAI competitor in three months, and here's the week-by-week progress.” Or, “I want to build a new storage engine, new database in three months.” And that's the kind of people that we want to help, these like, super ambitious, that just hasn't had a chance to be mentored by some of the world's best engineers. And I just want to help them. Like, we—this is a non-scalable project. I meet with them once a week. I don't want to have a team of, like, ten people.Like, to me, I feel like their most valuable thing I could do is to give them my time and to help them mentor. I was like, “Hey, let's think about this problem. Let's decompose this. How do you think about this?” And then bring you the best engineers that I, you know, that work for—with me, and let me help you think about problems differently and give you some money.And we just don't care how you use the time or the money; we just want people to work on hard problems. So, it's active. It runs once a year, and if anyone is listening to this, if you want to send it to your friends, we'd love to have that application. It's for anyone in the world, too, as long as we can send the person a check [laugh]. You know, my head of finance is not going to walk to a Moneygram—which we have done in the past—but other than that, as long as you have a bank account that we can send the check to, you should be able to apply.Corey: That is a compelling offer, particularly in the current macro environment that we find ourselves faced in. We'll definitely put a link to that into the [show notes 00:32:32]. I really want to thank you for taking the time to, I guess, get me up to speed on what it is you're doing. If people want to learn more where's the best place for them to go?Alex: On Twitter, my handle is @emaxerrno, which stands for the largest error in the kernel. I felt like that was apt for my handle. So, that's one. Feel free to find me on the community Slack. There's a Slack button on the website redpanda.com on the top right. I'm always there if you want to DM me. Feel free to stop by. And yeah, thanks for having me. This was a lot of fun.Corey: Likewise. I look forward to the next time. Alex Gallego, CEO and founder at Redpanda. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an insulting comment that I will almost certainly never read because they have not figured out how to get data from one place to another.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

TestGuild Performance Testing and Site Reliability Podcast
Postgres Performance at Any Scale with Lukas Fittl

TestGuild Performance Testing and Site Reliability Podcast

Play Episode Listen Later Jul 27, 2023 28:31


Welcome to another episode of the DevOps Toolchain podcast! In this episode, we have a special guest, Lukas Fittl, an experienced engineer and entrepreneur known for his work in optimizing Postgres performance. Lukas will share his insights and tips on tuning Postgres databases for optimal performance. We'll explore the different tuning aspects, including using managed services like Amazon RDS or Amazon Aurora, general best practices, and understanding where bottlenecks occur. Lukas will also discuss his journey in developing a comprehensive tool called pganalyze, which provides valuable insights and recommendations for optimizing Postgres performance. But that's not all! We'll also dive into the evolving role of the traditional DBA, the impact of AI on decision-making processes, and the latest trends in database indexing. Whether you're a developer, a database administrator, or just interested in learning more about Postgres performance, this episode is packed with valuable information. So grab your headphones and get ready to dive deep into Postgres performance at any scale with our guest Lukas Fittl. Let's get started!  

Oracle University Podcast
MySQL Database Service and HeatWave

Oracle University Podcast

Play Episode Listen Later Jul 11, 2023 14:20


In this episode, Lois Houston and Nikita Abraham are joined by Autumn Black to discuss MySQL Database, a fully-managed database service powered by the integrated HeatWave in-memory query accelerator.   Oracle MyLearn: https://mylearn.oracle.com/ Oracle University Learning Community: https://education.oracle.com/ou-community LinkedIn: https://www.linkedin.com/showcase/oracle-university/ Twitter: https://twitter.com/Oracle_Edu   Special thanks to Arijit Ghosh, David Wright, Deepak Modi, Ranbir Singh, and the OU Studio Team for helping us create this episode.   ---------------------------------------------------------   Episode Transcript: 00;00;00;00 - 00;00;39;08 Welcome to the Oracle University Podcast, the first stop on your cloud journey. During this series of informative podcasts, we'll bring you foundational training on the most popular Oracle technologies. Let's get started. Hello and welcome to the Oracle University Podcast. You're listening to our second season Oracle Database Made Easy. I'm Lois Houston, Director of Product Innovation and Go to Market Programs with Oracle University.   00;00;39;10 - 00;01;08;03 And with me is Nikita Abraham, Principal Technical Editor. Hi, everyone. In our last episode, we had a really fascinating conversation about Oracle Machine Learning with Cloud Engineer Nick Commisso. Do remember to catch that episode if you missed it. Today, we have with us Autumn Black, who's an Oracle Database Specialist. Autumn is going to take us through MySQL, the free version and the Enterprise Edition, and MySQL Data Service.   00;01;08;05 - 00;01;39;16 We're also going to ask her about HeatWave. So let's get started. Hi, Autumn. So tell me, why is MySQL such a popular choice for developers? MySQL is the number one open-source database and the second most popular database overall after the Oracle Database. According to a Stack Overflow survey, MySQL has been for a long time and remains the number one choice for developers, primarily because of its ease of use, reliability, and performance.   00;01;39;17 - 00;02;08;22 And it's also big with companies? MySQL is used by the world's most innovative companies. This includes Twitter, Facebook, Netflix, and Uber. It is also used by students and small companies. There are different versions of MySQL, right? What are the main differences between them when it comes to security, data recovery, and support? MySQL comes in two flavors: free version or paid version.   00;02;08;24 - 00;02;45;05 MySQL Community, the free version, contains the basic components for handling data storage. Just download it, install it, and you're ready to go. But remember, free has costs. That stored data is not exactly secure and data recovery is not easy and sometimes impossible. And there is no such thing as free MySQL Community support. This is why MySQL Enterprise Edition was created, to provide all of those missing important pieces: high availability, security, and Oracle support from the people who build MySQL.   00;02;45;10 - 00;03;09;24 You said MySQL is open source and can be easily downloaded and run. Does it run on-premises or in the cloud? MySQL runs on a local computer, company's data center, or in the cloud. Autumn, can we talk more about MySQL in the cloud? Today, MySQL can be found in Amazon RDS and Aurora, Google Cloud SQL, and Microsoft Azure Database for MySQL.   00;03;09;27 - 00;03;35;23 They all offer a cloud-managed version of MySQL Community Edition with all of its limitations. These MySQL cloud services are expensive and it's not easy to move data away from their cloud. And most important of all, they do not include the MySQL Enterprise Edition advanced features and tools. And they are not supported by the Oracle MySQL experts.   00;03;35;25 - 00;04;07;03 So why is MySQL Database Service in Oracle Cloud Infrastructure better than other MySQL cloud offerings? How does it help data admins and developers? MySQL Database Service in Oracle Cloud Infrastructure is the only MySQL database service built on MySQL Enterprise Edition and 100% built, managed, and supported by the MySQL team. Let's focus on the three major categories that make MySQL Database Service better than the other MySQL cloud offerings: ease of use, security, and enterprise readiness.   00;04;07;03 - 00;04;44;24 MySQL DBAs tend to be overloaded with mundane database administration tasks. They're responsible for many databases, their performance, security, availability, and more. It is difficult for them to focus on innovation and on addressing the demands of lines of business. MySQL is fully managed on OCI. MySQL Database Service automates all those time-consuming tasks so they can improve productivity and focus on higher value tasks.   00;04;44;26 - 00;05;07;13 Developers can quickly get all the latest features directly from the MySQL team to deliver new modern apps. They don't get that on other clouds that rely on outdated or forked versions of MySQL. Developers can use the MySQL Document Store to mix and match SQL and NoSQL content in the same database as well as the same application.   00;05;07;19 - 00;05;30;26 Yes. And we're going to talk about MySQL Document Store in a lot more detail in two weeks, so don't forget to tune in to that episode. Coming back to this, you spoke about how MySQL Database Service or MDS on OCI is easy to use. What about its security? MDS security first means it is built on Gen 2 cloud infrastructure.   00;05;30;28 - 00;05;57;13 Data is encrypted for privacy. Data is on OCI block volume. So what does this Gen 2 cloud infrastructure offer? Is it more secure? Oracle Cloud is secure by design and architected very differently from the Gen 1 clouds of our competitors. Gen 2 provides maximum isolation and protection. That means Oracle cannot see customer data and users cannot access our cloud control computer.   00;05;57;15 - 00;06;27;09 Gen 2 architecture allows us to offer superior performance on our compute objects. Finally, Oracle Cloud is open. Customers can run Oracle software, third-party options, open source, whatever you choose without modifications, trade-offs, or lock-ins. Just to dive a little deeper into this, what kind of security features does MySQL Database Service offer to protect data? Data security has become a top priority for all organizations.   00;06;27;12 - 00;06;55;17 MySQL Database Service can help you protect your data against external attacks, as well as internal malicious users with a range of advanced security features. Those advanced security features can also help you meet industry and regulatory compliance requirements, including GDPR, PCI, and HIPPA. When a security vulnerability is discovered, you'll get the fix directly from the MySQL team, from the team that actually develops MySQL.   00;06;55;19 - 00;07;22;16 I want to talk about MySQL Enterprise Edition that you brought up earlier. Can you tell us a little more about it? MySQL Database Service is the only public cloud service built on MySQL Enterprise Edition, which includes 24/7 support from the team that actually builds MySQL, at no additional cost. All of the other cloud vendors are using the Community Edition of MySQL, so they lack the Enterprise Edition features and tools.   00;07;22;22 - 00;07;53;24 What are some of the default features that are available in MySQL Database Service? MySQL Enterprise scalability, also known as the thread pool plugin, data-at-rest encryption, native backup, and OCI built-in native monitoring. You can also install MySQL Enterprise Monitor to monitor MySQL Database Service remotely. MySQL works well with your existing Oracle investments like Oracle Data Integrator, Oracle Analytics Cloud, Oracle GoldenGate, and more.   00;07;53;27 - 00;08;17;20 MySQL Database Service customers can easily use Docker and Kubernetes for DevOps operations. So how much of this is managed by the MySQL team and how much is the responsibility of the user? MySQL Database Service is a fully managed database service. A MySQL Database Service user is responsible for logical schema modeling, query design and optimization, define data access and retention policies.   00;08;17;22 - 00;08;44;26 The MySQL team is responsible for providing automation for operating system installation, database and OS patching, including security patches, backup, and recovery. The system backs up the data for you, but in an emergency, you can restore it to a new instance with a click. Monitoring and log handling. Security with advanced options available in MySQL Enterprise Edition.   00;08;44;28 - 00;09;01;18 And of course, maintaining the data center for you. To use MDS, users must have OCI tenancy, a compartment, belong to a group with required policies.   00;09;01;21 - 00;09;28;28 Did you know that Oracle University offers free courses on Oracle Cloud Infrastructure? You'll find training on everything from cloud computing, database, and security to artificial intelligence and machine learning, all of which is available free to subscribers. So get going. Pick a course of your choice, get certified, join the Oracle University Learning Community, and network with your peers. If you're already an Oracle MyLearn user, go to MyLearn to begin your journey.   00;09;29;03 - 00;09;40;24 If you have not yet accessed Oracle MyLearn, visit mylearn.oracle.com and create an account to get started.   00;09;40;27 - 00;10;05;20 Welcome back! Autumn, tell us about the system architecture of MySQL Database Service. A database system is a logical container for the MySQL instance. It provides an interface enabling management of tasks, such as provisioning, backup and restore, monitoring, and so on. It also provides a read and write endpoint, enabling you to connect to the MySQL instance using the standard protocols.   00;10;05;28 - 00;10;31;27 And what components does a MySQL Database Service DB system consist of? A computer instance, an Oracle Linux operating system, the latest version of MySQL server Enterprise Edition, a virtual network interface card, VNIC, that attaches the DB system to a subnet of the virtual cloud network, network-attached higher performance block storage. Is there a way to monitor how the MySQL Database Service is performing?   00;10;31;29 - 00;10;59;29 You can monitor the health, capacity, and performance of your Oracle Cloud Infrastructure MySQL Database Service resources by using metrics, alarms, and notifications. The MySQL Database Service metrics enable you to measure useful quantitative data about your MySQL databases such as current connection information, statement activity, and latency, host CPU, memory, and disk I/O utilization, and so on.   00;11;00;03 - 00;11;23;15 You can use metrics data to diagnose and troubleshoot problems with MySQL databases. What should I keep in mind about managing the SQL database? Stopped MySQL Database Service system stops billing for OCPUs, but you also cannot connect to the DB system. During MDS automatic update, the operating system is upgraded along with patching of the MySQL server.   00;11;23;17 - 00;11;49;15 Metrics are used to measure useful data about MySQL Database Service system. Turning on automatic backups is an update to MDS to enable automatic backups. MDS backups can be removed by using the details pages and OCI and clicking Delete. Thanks for that detailed explanation on MySQL, Autumn. Can you also touch upon MySQL HeatWave? Why would you use it over traditional methods of running analytics on MySQL data?   00;11;49;18 - 00;12;18;01 Many organizations choose MySQL to store their valuable enterprise data. MySQL is optimized for Online Transaction Processing, OLTP, but it is not designed for Online Analytic Processing, OLAP. As a result, organizations that need to efficiently run analytics on data stored in MySQL database move their data to another database to run analytic applications such as Amazon Redshift.   00;12;18;04 - 00;12;41;22 MySQL HeatWave is designed to enable customers to run analytics on data that is stored in MySQL database without moving data to another database. What are the key features and components of HeatWave? HeatWave is built on an innovative in-memory analytics engine that is architected for scalability and performance, and is optimized for Oracle Cloud Infrastructure, OCI.   00;12;41;24 - 00;13;05;29 It is enabled when you add a HeatWave cluster to a MySQL database system. A HeatWave cluster comprises a MySQL DB system node and two or more HeatWave nodes. The MySQL DB system node includes a plugin that is responsible for cluster management, loading data into the HeatWave cluster, query scheduling, and returning query results to the MySQL database system.   00;13;06;02 - 00;13;29;15 The HeatWave nodes store data and memory and processed analytics queries. Each HeatWave node contains an instance of the HeatWave. The number of HeatWave nodes required depends on the size of your data and the amount of compression that is achieved when loading the data into the HeatWave cluster. Various aspects of HeatWave use machine-learning-driven automation that helps to reduce database administrative costs.   00;13;29;18 - 00;13;52;11 Thanks, Autumn, for joining us today. We're looking forward to having you again next week to talk to us about Oracle NoSQL Database Cloud Service. To learn more about MySQL Data Service, head over to mylearn.oracle.com and look for the Oracle Cloud Data Management Foundations Workshop. Until next time, this is Nikita Abraham and Lois Houston signing off.   00;13;52;14 - 00;16;33;05 That's all for this episode of the Oracle University Podcast. If you enjoyed listening, please click Subscribe to get all the latest episodes. We'd also love it if you would take a moment to rate and review us on your podcast app. See you again on the next episode of the Oracle University Podcast.

The Cloud Pod
216: The Cloud Pod is Feeling Elevated Enough to Record the Podcast

The Cloud Pod

Play Episode Listen Later Jun 30, 2023 30:53


Welcome to the newest episode of The Cloud Pod podcast - where the forecast is always cloudy! Today your hosts are Jonathan and Matt as we discuss all things cloud and AI, including Temporary Elevated Access Management (or TEAM, since we REALLY like acronyms today)  FTP servers, SQL servers and all the other servers, as well as pipelines, whether or not the government should regulate AI (spoiler alert: the AI companies don't think so) and some updates to security at Amazon and Google.  Titles we almost went with this week: The Cloud Pod's FTP server now with post-quantum keys support The CloudPod can now Team into your account, but only temporarily  The CloudPod dusts off their old floppy drive  The CloudPod dusts off their old SQL server disks The CloudPod is feeling temporarily elevated to do a podcast The CloudPod promise that AI will not take over the world The CloudPod duals with keys The CloudPod is feeling temporarily elevated. A big thanks to this week's sponsor: Foghorn Consulting, provides top-notch cloud and DevOps engineers to the world's most innovative companies. Initiatives stalled because you have trouble hiring?  Foghorn can be burning down your DevOps and Cloud backlogs as soon as next week.

AWS Morning Brief
Rated R for Ridiculousness

AWS Morning Brief

Play Episode Listen Later Jun 5, 2023 3:16


AWS Morning Brief for the week of June 5, 2023 with Corey Quinn. Links: Corey is off to Washington DC tomorrow for the Public Sector summit. If you're in town, he's hosting a drink up at Highline RxR from 6-8 PM tomorrow (Tuesday) evening. Let him buy you a drink! AWS Pricing Calculator now offers visibility of point in time cost estimations Invoice Summary is now available  Optimize your x86-based Amazon EC2 Workloads  New – AWS DMS Serverless: Automatically Provisions and Scales Capacity for Migration and Data Replication Build hypothetical indexes in Amazon RDS for PostgreSQL with HypoPG Version 1 of the AWS Cloud Development Kit (AWS CDK) has reached end-of-support The Retail Race: A Roadmap for Implementing a Smart Store Strategy  Get ready for AWS IPv6 day Introducing a cost control solution for Amazon Braket 

The Cloud Pod
208: Azure AI Lost in Space

The Cloud Pod

Play Episode Listen Later Apr 21, 2023 57:43


Welcome to the newest episode of The Cloud Pod podcast! Justin, Ryan and Matthew are your hosts this week as we discuss all the latest news and announcements in the world of the cloud and AI. Do people really love Matt's Azure know-how? Can Google make Bard fit into literally everything they make? What's the latest with Azure AI and their space collaborations? Let's find out! Titles we almost went with this week: Clouds in Space, Fictional Realms of Oracles, Oh My.  The cloudpod streams lambda to the cloud A big thanks to this week's sponsor:  Foghorn Consulting, provides top-notch cloud and DevOps engineers to the world's most innovative companies. Initiatives stalled because you have trouble hiring?  Foghorn can be burning down your DevOps and Cloud backlogs as soon as next week.

AWS Developers Podcast
Episode 073 - Fully Managed Blue Green Deployments in Amazon Aurora and Amazon RDS with Keyur Diwan

AWS Developers Podcast

Play Episode Listen Later Mar 3, 2023 29:36


In this episode, Emily and Dave chat with Keyur Diwan Sr. Product Manager, Amazon RDS. Amazon Relational Database Service (Amazon RDS) is a collection of managed services that makes it simple to set up, operate, and scale databases in the cloud. Amazon RDS recently launched fully managed Blue/Green Deployments to help you with safer, simpler, and faster updates to your Amazon Aurora and Amazon RDS databases. Blue/Green Deployments create a fully managed staging environment that allows you to deploy and test production changes, keeping your current production database safe. With a single click, you can promote the staging environment to be the new production system in as fast as a minute, with no changes to your application and no data loss. Keyur walks us through this new feature, how to get started, how customers are using it today, and what's next for RDS. [BLOG] Announcing Amazon RDS Blue/Green Deployments for safer, simpler, and faster updates - https://aws.amazon.com/about-aws/whats-new/2022/11/amazon-rds-blue-green-deployments-safer-simpler-faster-updates/ [BLOG] New – Fully Managed Blue/Green Deployments in Amazon Aurora and Amazon RDS - https://aws.amazon.com/blogs/aws/new-fully-managed-blue-green-deployments-in-amazon-aurora-and-amazon-rds/ [DOCS] Amazon RDS and Aurora Documentation - https://docs.aws.amazon.com/rds/ [PORTAL] Amazon Relational Database Service (RDS) https://aws.amazon.com/rds/ [YOUTUBE] Introduction to Amazon RDS Blue/Green Deployments - https://youtu.be/mGAjzAzBOsk Origins of the term "blue-green deployment" - https://gitlab.com/-/snippets/1846041 Martin Fowler - Blue Green Deployment - https://martinfowler.com/bliki/BlueGreenDeployment.html Subscribe: Amazon Music: https://music.amazon.com/podcasts/f8bf7630-2521-4b40-be90-c46a9222c159/aws-developers-podcast Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-developers-podcast/id1574162669 Google Podcasts: https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkcy5zb3VuZGNsb3VkLmNvbS91c2Vycy9zb3VuZGNsb3VkOnVzZXJzOjk5NDM2MzU0OS9zb3VuZHMucnNz Spotify: https://open.spotify.com/show/7rQjgnBvuyr18K03tnEHBI TuneIn: https://tunein.com/podcasts/Technology-Podcasts/AWS-Developers-Podcast-p1461814/ RSS Feed: https://feeds.soundcloud

The Cloud Pod
201: The CloudPod is assimilated and joins the Azure Collective

The Cloud Pod

Play Episode Listen Later Feb 28, 2023 36:04


On this episode of The Cloud Pod, the team discusses the AWS systems manager default enablement option for all EC2 instances in an account, different ideas from leveraging innovators plus subscription using $500 Google credits, the Azure Open Source Day, the new theme for the Oracle OCI Console, and lastly, different ways to migrate to a cloud provider. A big thanks to this week's sponsor, Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights

The Cloud Pod
200: Now you can make bad cloud decisions like running EKS on SNOW

The Cloud Pod

Play Episode Listen Later Feb 22, 2023 50:18


EKS on Snow Devices On this episode of The Cloud Pod, the team highlights the new Graviton3-based images for users of AWS, new ways provided by Google to pay for its cloud services, the new partnership between Azure and the Finops Foundation, as well as Oracle's new cloud banking, and the automation of CCOE. A big thanks to this week's sponsor, Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights

The Cloud Pod
199: All AI Products Agree, Earnings are down

The Cloud Pod

Play Episode Listen Later Feb 18, 2023 52:59


AI Products & Earnings On this episode of The Cloud Pod, the team talks about the announcement of Amazon VPC resource map, Google's new AI product, the new Bing AI-powered search engine, and why multiple accounts are necessary for data centers to carry out work seamlessly in the cloud. A big thanks to this week's sponsor, Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure.  This week's highlights

The Cloud Pod
198: Cloudtrail ingests activity events, CloudPod ingests Pizza

The Cloud Pod

Play Episode Listen Later Feb 10, 2023 59:02


On this episode of The Cloud Pod, the team discusses the upcoming 2023 in-person Google Cloud conference, the accessibility of AWS CloudTrail Lake for non-AWS activity events, the new updates from Azure Chaos studio, and the comparison between Oracle Cloud service and other Cloud providers. They also highlight the application and importance of VPCs in CCOE. A big thanks to this week's sponsor, Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure.  This week's highlights

The Cloud Pod
196: The Cloud Pod plays with all the stuff it found in the cleanroom

The Cloud Pod

Play Episode Listen Later Jan 28, 2023 40:43


On this episode of The Cloud Pod, the team sits to talk about AWS's new patching policies, the general availability of Azure OpenAI, and the role of addressing IM or access management challenges in ensuring the seamless transition to the Cloud. A big thanks to this week's sponsor, Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights

The Cloud Pod
194: The Cloud Pods New Years Resolution: Change everything!

The Cloud Pod

Play Episode Listen Later Jan 10, 2023 80:40


For our New Years Resolution, we decided to change some of our show. First, we have cut the lightning round in favor of our new Cloud Journey series, where we will talk about core cloud concepts over several episodes. We are also covering only the larger stories from the cloud providers, we still want to provide you with all of the news, so you'll find it in the show notes; if you enjoy the aggregation, subscribe to our newsletter to get the show notes to get your mailbox weekly.  Share your feedback through our website or join our slack team.  On this episode of The Cloud Pod, the team follows up on the news from Salesforce's last episode, as workforce cuts ensue as a fallout of the noted decline in productivity, with more on 2023 predictions from Peter, including general expectations in the tech space, while also highlighting the new Graph-explorer tool by Amazon Neptune, GCP security trends for the coming year, the CES Conference and CCOE from the new Cloud Journey Series. A big thanks to this week's sponsor, Foghorn Consulting, which provides full-stack cloud solutions focused on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights

AWS Podcast
#565: DevOps Guru for RDS Detects and Resolves Performance Bottlenecks for Amazon RDS

AWS Podcast

Play Episode Listen Later Jan 2, 2023 18:04


In this episode, Simon is joined by Shrey Luthra and Jacob Sullivan, Product Managers for Amazon RDS and Amazon DevOps Guru, to talk about a new capability through which customers can detect and diagnose performance issues for their Amazon RDS databases and get recommendations on how to fix them. For more information: https://go.aws/3I2HVhx

The Cloud Pod
191: The Cloud Pod Reinvents the Recap Show

The Cloud Pod

Play Episode Listen Later Dec 14, 2022 75:47


The Cloud Pod recaps all of the positives and negatives of Amazon ReInvent 2022, the annual conference in Las Vegas, bringing together 50,000 cloud computing professionals.  This year's keynote speakers include Adam Selpisky, CEO of Amazon Web Services, Swami Sivasubramanian, Vice President of Data and Machine Learning at AWS and Werner Vogels, Amazon's CTO.  Attendees and web viewers were treated to new features and products, such as AWS Lambda Snapstart for Java Functions, New Quicksight capabilities and quality-of-life improvements to hundreds of services.  Justin, Jonathan, Ryan, Peter and Special guest Joe Daly from the Finops foundation talk about the show and the announcements. Thank you to our sponsor, Foghorn Consulting, which provides top notch cloud and DevOps engineers to the world's most innovative companies. Initiatives stalled because you're having trouble hiring? Foghorn can be burning down your DevOps and Cloud backlogs as soon as next week. Episode Highlights ⏰ AWS Pricing Calculator now supports modernization cost estimates for Microsoft workloads. ⏰ AWS Re:Invent 2022 announcements and keynote updates. Top Quote

AWS Developers Podcast
Episode 063 - Announcing Trusted Language Extensions for PostgreSQL

AWS Developers Podcast

Play Episode Listen Later Dec 12, 2022 17:46


In this episode, Emily and Dave chat with John Dalton, Sr. Product Manager, Amazon RDS Open Source, and Jonathan Katz, Principal PMT, Amazon RDS Open Source, about the recent launch of Trusted Language Extensions PostgreSQL. Trusted Language Extensions for PostgreSQL is an open source development kit for building PostgreSQL extensions that allows developers to build high performance PostgreSQL extensions and safely run them on your RDS for PostgreSQL DB instance. By using Trusted Language Extensions (TLE) for PostgreSQL, developers can create PostgreSQL extensions that follow the documented approach for extending PostgreSQL functionality. Both Johns discuss building this new service update, and their approach to open source. [BLOG] New – Trusted Language Extensions for PostgreSQL on Amazon Aurora and Amazon RDS: https://aws.amazon.com/blogs/aws/new-trusted-language-extensions-for-postgresql-on-amazon-aurora-and-amazon-rds/ [BLOG] Announcing Trusted Language Extensions for PostgreSQL on Amazon Aurora and Amazon RDS: https://aws.amazon.com/about-aws/whats-new/2022/11/trusted-language-extensions-postgresql-amazon-aurora-rds/ [DOCS] Working with Trusted Language Extensions for PostgreSQL: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/PostgreSQL_trusted_language_extension.html [GITHUB] Open Source Framework for Building Trusted Language Extensions for PostgreSQL: https://github.com/aws/pg_tle [PORTAL] Amazon Aurora: https://aws.amazon.com/rds/aurora/ [PORTAL] Amazon RDS for PostgreSQL: https://aws.amazon.com/rds/postgresql/ Subscribe: Amazon Music: https://music.amazon.com/podcasts/f8bf7630-2521-4b40-be90-c46a9222c159/aws-developers-podcast Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-developers-podcast/id1574162669 Google Podcasts: https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkcy5zb3VuZGNsb3VkLmNvbS91c2Vycy9zb3VuZGNsb3VkOnVzZXJzOjk5NDM2MzU0OS9zb3VuZHMucnNz Spotify: https://open.spotify.com/show/7rQjgnBvuyr18K03tnEHBI TuneIn: https://tunein.com/podcasts/Technology-Podcasts/AWS-Developers-Podcast-p1461814/ RSS Feed: https://feeds.soundcloud

Hack és Lángos
HnL254 - Klaudia Schniffer

Hack és Lángos

Play Episode Listen Later Nov 24, 2022 58:27


Mai menü:eKréta follow-upSocial Engineers warnedWITSEC videókAccidental $70k Google Pixel Lock Screen BypassUsing Wi-FI to See through WallsDisneyland Malware Team: It's a Puny World After AllNMAP without NMAP - Port Testing and Scanning with PowerShell, (Mon, Oct 31st)Amazon RDS priv data made publicPacket Tuesday (!!!)Elérhetőségeink:TelegramTwitterInstagramFacebookMail: info@hackeslangos.show

The Cloud Pod
189: The CloudPod Celebrates AWS Becoming a New Time Lord

The Cloud Pod

Play Episode Listen Later Nov 23, 2022 36:48


RE:INVENT NOTICE Jonathan, Ryan and Justin will be live streaming the major keynotes starting Monday Night, followed by Adam's keynote on Tuesday, Swami's keynote on Wednesday and Wrap up our Re:Invent coverage with Werner's keynote on Thursday. Tune into our live stream here on the site or via Twitch/Twitter, etc.  On The Cloud Pod this week, Amazon Time Sync is now available over the internet as a public NTP service, Amazon announces ECS Task Scale-in protection, and Private Marketplace is now in preview. Thank you to our sponsor, Foghorn Consulting, which provides top notch cloud and DevOps engineers to the world's most innovative companies. Initiatives stalled because you're having trouble hiring? Foghorn can be burning down your DevOps and Cloud backlogs as soon as next week. Episode Highlights ⏰ Amazon Time Sync is now available over the internet as a public NTP service. ⏰ Amazon announces ECS Task Scale-in protection. ⏰ Private Marketplace is now in preview. Top Quote

AWS Morning Brief
IAM Over the Moon About Multiple MFA Devices

AWS Morning Brief

Play Episode Listen Later Nov 21, 2022 7:51


Links: Amazon NAT Gateway Now Allows You to Select Private IP Address for Network Address Translation Amazon S3 Glacier improves restore throughput by up to 10x when retrieving large volumes of archived data Amazon Time Sync is now available over the internet as a public NTP service AWS re:Post launches a community leaderboard Announcing the new Applications widget on AWS Console Home Amazon S3 request-level information on use of access control lists (ACLs) coming to S3 server access logs and AWS CloudTrail  Know Before You Go: An AWS Partner's Guide to re:Invent 2022 Introducing our final AWS Heroes of the year – November 2022 Now Open–AWS Region in Spain Introducing Amazon EventBridge Scheduler Migrate ROW CHANGE TIMESTAMP from IBM Db2 for z/OS to Amazon RDS for PostgreSQL or Amazon Aurora PostgreSQL-Compatible Edition You can now assign multiple MFA devices in IAM 

Cyber Security Today
Cyber Security Today, Nov. 18, 2022 - A warning about Amazon RDS snapshots, a new ransomware strain found, and more

Cyber Security Today

Play Episode Listen Later Nov 18, 2022 5:53


This episode reports on the risks of misconfigured, a warning on the Log4Shell vulnerability, ransomware reports and more

Cyber Security Headlines
Musk's ultimatum, Iran breaches government using Log4Shell, Amazon RDS data leak

Cyber Security Headlines

Play Episode Listen Later Nov 18, 2022 7:06


Musk's ultimatum to employees leaves Twitter at risk Iranian APT breaches government agency using Log4Shell Hundreds of Amazon RDS snapshots discovered leaking user data And now a word from our sponsor, AppOmni Can you name all the third party apps connected to your major SaaS platforms like Salseforce and Microsoft? What about the data these apps can access? After all, one compromised third party app could put your entire SaaS ecosystem at risk.  With AppOmni, you get visibility to all third party apps, including which end users have enabled them, and the level of data access they've been granted. Visit AppOmni.com to request a free risk assessment.

The CyberWire
Getting tangled up in the blockchain. RDS vulnerabilities. The language of fraud. An offer of help to the G19.Draft Episode for Nov 16, 2022

The CyberWire

Play Episode Listen Later Nov 16, 2022 30:44


Blockchains and cryptocurrency exchanges, and the risks they present. Vulnerabilities in Amazon RDS may expose PII. A study of the language of fraud. Tim Starks from Washington Post's Cybersecurity 202 on a lagging DHS cyber doomsday report. Our guest is Ashif Samnani of Cenovus Energy with insights from the world of OT cyber. And President Zelenskyy offers the benefit of Ukraine's experience with cyber warfare to the "G19”. For links to all of today's stories check out our CyberWire daily news briefing: https://thecyberwire.com/newsletters/daily-briefing/11/220 Selected reading. Cryptocurrency sector vulnerabilities. (CyberWire) Oops, I Leaked It Again — How Mitiga Found PII in Exposed Amazon RDS Snapshots (Mitiga) Amazon RDS may expose PII. (CyberWire) The specious language of fraud. (CyberWire) Zelensky offers G20 leaders to use Ukrainian experience in cyber defense (Ukrinform)  Ukraine at D+265: A missile campaign punctuates diplomacy. (CyberWire)

Security In Five Podcast
Episode 1310 - Hundreds Of Amazon RDS Snapshots Found Exposed, How To Avoid This

Security In Five Podcast

Play Episode Listen Later Nov 16, 2022 6:03


Mitiga, a cloud incident response company, recently found hundreds of Amazon RDS Snapshots exposed publicly. This led to the leak of users' PII and other information. This episode talks about what RDS and the snapshots are and why you need to understand how the cloud works end to end. Source - https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_CreateSnapshot.html Be aware, be safe. Support the show and get access to behind the scenes content as a patron - https://www.patreon.com/SecurityInFive *** Support the podcast with a cup of coffee *** - Ko-Fi Security In Five Mighty Mackenzie - https://www.facebook.com/mightymackie Where you can find Security In Five - https://linktr.ee/binaryblogger Email - bblogger@protonmail.com

The Cloud Pod
183: The Cloud Pod competes for the Google Cloud Fly Cup

The Cloud Pod

Play Episode Listen Later Sep 30, 2022 45:05


On The Cloud Pod this week, AWS Enterprise Support adds incident detection and response, the announcement of Google Cloud Spanner, and Oracle expands to Spain. Thank you to our sponsor, Foghorn Consulting, which provides top notch cloud and DevOps engineers to the world's most innovative companies. Initiatives stalled because you're having trouble hiring? Foghorn can be burning down your DevOps and Cloud backlogs as soon as next week. Episode Highlights ⏰ AWS Enterprise Support adds incident detection and response ⏰ You can now get a 90-day free trial of Google Cloud Spanner ⏰ Oracle opens its newest cloud infrastructure region in Spain Top Quote

The Cloud Pod
180: Azure Data Explorer Says ‘All Your S3 Data are Belong to Us'

The Cloud Pod

Play Episode Listen Later Sep 9, 2022 46:00


On The Cloud Pod this week, Amazon adds the ability to embed fine-grained visualizations directly onto web pages, Google offers pay-as-you-go pricing for Apigee customers, and Microsoft launches Arm-based Azure VMs that are powered by ampere chips. Thank you to our sponsor, Foghorn Consulting, which provides top notch cloud and DevOps engineers to the world's most innovative companies. Initiatives stalled because you're having trouble hiring? Foghorn can be burning down your DevOps and Cloud backlogs as soon as next week. Episode Highlights ⏰  Fine-grained visualizations can now be embedded directly into your webpages and applications ⏰  Google is now offering pay-as-you-go pricing for its Apigee API customers ⏰  Microsoft launches Arm-based Azure VMs powered by ampere chips Top Quote

The Cloud Pod
176: The Cloud Pod Earnings Continue To Be Steady

The Cloud Pod

Play Episode Listen Later Aug 11, 2022 67:15


On The Cloud Pod this week, the team discusses why Ryan's yelling all day (hint: he's learning). Plus: Peter misses the all-important cloud earnings, AWS Skill Builder subscriptions are now available, and Google Eventarc connects SaaS platforms.  A big thanks to this week's sponsor, Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights

Screaming in the Cloud
Cloud-Hosted Database Services with Benjamin Anderson

Screaming in the Cloud

Play Episode Listen Later Jul 21, 2022 35:39


About BenjaminBenjamin Anderson is CTO, Cloud at EDB, where he is responsible for developing and driving strategy for the company's Postgres-based cloud offerings. Ben brings over ten years' experience building and running distributed database systems in the cloud for multiple startups and large enterprises. Prior to EDB, he served as chief architect of IBM's Cloud Databases organization, built an SRE practice at database startup Cloudant, and founded a Y Combinator-funded hardware startup.Links Referenced: EDB: https://www.enterprisedb.com/ BigAnimal: biganimal.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.I come bearing ill tidings. Developers are responsible for more than ever these days. Not just the code that they write, but also the containers and the cloud infrastructure that their apps run on. Because serverless means it's still somebody's problem. And a big part of that responsibility is app security from code to cloud. And that's where our friend Snyk comes in. Snyk is a frictionless security platform that meets developers where they are - Finding and fixing vulnerabilities right from the CLI, IDEs, Repos, and Pipelines. Snyk integrates seamlessly with AWS offerings like code pipeline, EKS, ECR, and more! As well as things you're actually likely to be using. Deploy on AWS, secure with Snyk. Learn more at Snyk.co/scream That's S-N-Y-K.co/screamCorey: This episode is sponsored by our friends at Fortinet. Fortinet's partnership with AWS is a better-together combination that ensures your workloads on AWS are protected by best-in-class security solutions powered by comprehensive threat intelligence and more than 20 years of cybersecurity experience. Integrations with key AWS services simplify security management, ensure full visibility across environments, and provide broad protection across your workloads and applications. Visit them at AWS re:Inforce to see the latest trends in cybersecurity on July 25-26 at the Boston Convention Center. Just go over to the Fortinet booth and tell them Corey Quinn sent you and watch for the flinch. My thanks again to my friends at Fortinet.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. This promoted guest episode is brought to us by our friends at EDB. And not only do they bring us this promoted episode, they bring me their CTO for Cloud, Benjamin Anderson. Benjamin, thank you so much for agreeing to suffer the slings and arrows that I will no doubt throw at you in a professional context, because EDB is a database company, and I suck at those things.Benjamin: [laugh]. Thanks, Corey. Nice to be here.Corey: Of course. So, databases are an interesting and varied space. I think we can all agree—or agree to disagree—that the best database is, of course, Route 53, when you misuse TXT records as a database. Everything else is generally vying for number two. EDB was—back in the days that I was your customer—was EnterpriseDB, now rebranded as EDB, which is way faster to say, and I approve of that.But you were always the escalation point of last resort. When you're stuck with a really weird and interesting Postgres problem, EDB was where you went because if you folks couldn't solve the problem, it was likely not going to get solved. I always contextualized you folks as a consulting shop. That's not really what you do. You are the CTO for Cloud.And, ah, interesting. Do databases behave differently in cloud environments? Well, they do when you host them as a managed service, which is an area you folks have somewhat recently branched into. How'd you get there?Benjamin: Ah, that's interesting. So, there's a bunch of stuff to unpack there. I think EDB has been around for a long time. It's something like 13, 14, 15 years, something like that, and really it's just been kind of slowly growing, right? We did start very much as a product company. We built some technology to help customers get from Oracle database on to Postgres, way back in 2007, 2008.That business has just slowly been growing. It's been going quite well. Frankly, I only joined about 18 months ago, and it's really cool tech, right? We natively understand some things that Oracle is doing. Customers don't have to change their schemas to migrate from Oracle to Postgres. There's some cool technology in there.But as you point out, I think a lot of our position in the market has not been that product focused. There's been a lot of people seeing us as the Postgres experts, and as people who can solve Postgres problems, in general. We have, for a long time, employed a lot of really sharp Postgres people. We still employ a lot of really sharp Postgres people. That's very much, in a lot of ways, our bread and butter. That we're going to fix Postgres problems as they come up.Now, over the past few years, we've definitely tried to shift quite a bit into being more of a product company. We've brought on a bunch of people who've been doing more enterprise software product type development over the past few years, and really focusing ourselves more and more on building products and investing in ourselves as a product company. We're not a services company. We're not a consulting company. We do, I think, provide the best Postgres support in the market. But it's been a journey. The cloud has been a significant part of that as well, right? You can't get away.Corey: Oh, yeah. These days, when someone's spinning up a new workload, it's unlikely—in most cases—they're going to wind up spinning up a new data center, if they don't already have one. Yes, there's still a whole bunch of on-prem workloads. But increasingly, the default has become cloud. Instead of, “Why cloud?” The question's become, “Why not?”Benjamin: Right, exactly. Then, as people are more and more accepting of managed services, you have to be a product company. You have to be building products in order to support your database customers because what they want his managed services. I was working in managed databases and service, something like, ten years ago, and it was like pulling teeth. This is after RDS launched. This was still pulling teeth trying to get people to think about, oh, I'm going to let you run my database. Whereas, now obviously, it's just completely different. We have to build great products in order to succeed in the database business, in general.Corey: One thing that jumped out at me when you first announced this was the URL is enterprisedb.com. That doesn't exactly speak to, you know, non-large companies, and EDB is what you do. You have a very corporate logo, but your managed service is called BigAnimal, which I absolutely love. It actually expresses a sense of whimsy and personality that I can no doubt guess that a whole bunch of people argued against, but BigAnimal, it is. It won through. I love that. Was that as contentious as I'm painting it to be, or people actually have a sense of humor sometimes?Benjamin: [laugh]. Both, it was extremely contentious. I, frankly, was one of the people who was not in favor of it at first. I was in favor of something that was whimsical, but maybe not quite that whimsical.Corey: Well, I call it Postgres-squeal, so let's be very clear here that we're probably not going to see eye-to-eye on most anything in pronunciation things. But we can set those differences aside and have a conversation.Benjamin: Absolutely, no consider that. It was deliberate, though, to try to step away a little bit from the blue-suit-and-tie, enterprise, DB-type branding. Obviously, a lot of our customers are big enterprises. We're good at that. We're not trying to be the hip, young startup targeting business in a lot of ways. We have a wide range of customers, but we want to branch out a little bit.Corey: One of the challenges right now is if I spin up an environment inside of AWS, as one does, and I decide I certainly don't want to take the traditional approach of running a database on top of an EC2 instance—the way that we did in the olden days—because RDS was crappy. Now that it's slightly less crappy, that becomes a not ideal path. I start looking at their managed database offerings, and there are something like 15 distinct managed databases that they offer, and they never turn anything off. And they continue to launch things into the far future. And it really feels, on some level, like 20 years from now—what we call a DBA today—their primary role is going to look a lot more like helping a company figure out which of Amazon's 40 managed databases is the appropriate fit for this given workload. Yet, when I look around at what the industry has done, it seems that when we're talking about relational databases. Postgres has emerged back when I was, more or less, abusing servers in person in my data center days, it was always MySQL. These days, Postgres is the de facto standard, full stop. I admit that I was mostly keeping my aura away from any data that was irreplaceable at that time. What happened? What did I miss?Benjamin: It's a really good question. And I certainly am not a hundred percent on all the trends that went on there. I know there's a lot of folks that are not happy about the MySQL acquisition by Oracle. I think there's a lot of energy that was adopted by the NoSQL movement, as well. You have people who didn't really care about transactional semantics that were using MySQL because they needed a place to store their data. And then, things like MongoDB and that type of system comes along where it's significantly easier than MySQL, and that subset of the population just sort of drifts away from MySQL.Corey: And in turn, those NoSQL projects eventually turn into something where, okay, now we're trying to build a banking system on top of it, and it's, you know, I guess you can use a torque wrench as a hammer if you're really creative about it, but it seems like there's a better approach.Benjamin: Yeah, exactly. And those folks are coming back around to the relational databases, exactly. At the same time, the advancements in Postgres from the early eight series to today are significant, right? We shouldn't underestimate how much Postgres has really moved forward. It wasn't that long ago that replication was hardly a thing and Postgres, right? It's been a journey.Corey: One thing that your website talks about is that you accelerate your open-sourced database transformation. And this is a bit of a hobby horse I get on from time to time. I think that there are a lot of misunderstandings when people talk about this. You have the open-source purists—of which I shamefully admit I used to be one—saying that, “Oh, it's about the idea of purity and open and free as in software.” Great. Okay, awesome. But when I find that corporate customers are talking about when they say open-source database, they don't particularly care if they have access to the source code because they're not going to go in and patch a database engine, we hope. But what they do care about is regardless of where they are today—even if they're perfectly happy there—they don't want to wind up beholden to a commercial database provider, and/or they don't want to wind up beholden to the environment that is running within. There's a strategic Exodus that's available in theory, which on some level serves to make people feel better about not actually Exodus-ing, but it also means if they're doing a migration at some point, they don't also have to completely redo their entire data plan.Benjamin: Yeah, I think that's a really good point. I mean, I like to talk—there's a big rat's nest of questions and problems in here—but I generally like talk to about open APIs, talk about standards, talk about how much is going to have to change if you eliminate this vendor. We're definitely not open-source purists. Well, we employ a lot of open-source purists. I also used to be an open—Corey: Don't let them hear you say that, then. Fair enough. Fair enough.Benjamin: [laugh] we have proprietary software at EDB, as well. There's a kind of wide range of businesses that we participate in. Glad to hear you also mention this where-it's-hosted angle, as well. I think there's some degree to which people are—they figured out that having at least open APIs or an open-source-ish database is a good idea rather than being beholden to proprietary database. But then, immediately forget that when they're picking a cloud vendor, right? And realizing that putting their data in Cloud Vendor A versus Cloud Vendor B is also putting them in a similar difficult situation. They need to be really wary of when they're doing that. Now, obviously, I work at an independent software company, and I have some incentive to say this, but I do think it's true. And you know, there's meaningful data gravity risk.Corey: I assure you, I have no incentive. I don't care what cloud provider you're on. My guidance has been, for years, to—as a general rule—pick a provider, I care about which one, and go all in until there's a significant reason to switch. Trying to build an optionality, “Oh, everything we do should be fully portable at an instance notice.” Great. Unless you're actually doing it, you're more or less, giving up a whole bunch of shortcuts and feature velocity you could otherwise have, in the hopes of one day you'll do a thing, but all the assumptions you're surrounded by baked themselves in regardless. So, you're more or less just creating extra work for yourself for no defined benefit. This is not popular in some circles, where people try to sell something that requires someone to go multi-cloud, but here we are.Benjamin: No, I think you're right. I think people underestimate the degree to which the abstractions are just not very good, right, and the degree to which those cloud-specific details are going to leak in if you're going to try to get anything done, you end up in kind of a difficult place. What I see more frequently is situations where we have a big enterprise—not even big, even medium-sized companies where maybe they've done an acquisition or two, they've got business units that are trying to do things on their own. And they end up in two or three clouds, sort of by happenstance. It's not like they're trying to do replication live between two clouds, but they've got one business unit in AWS and one business unit and Azure, and somebody in the corporate—say enterprise architect or something like that—really would like to make things consistent between the two so they get a consistent security posture and things like that. So, there are situations where the multi-cloud is a reality at a certain level, but maybe not at a concrete technical level. But I think it's still really useful for a lot of customers.Corey: You position your cloud offering in two different ways. One of them is the idea of BigAnimal, and the other—well, it sort of harkens back to when I was in sixth grade going through the American public school system. They had a cop come in and talk to us and paint to this imaginary story of people trying to push drugs. “Hey, kid. You want to try some of this?” And I'm reading this and it says EDB, Postgres for Kubernetes. And I'm sent back there, where it's like, “Hey, kid. You want to run your stateful databases on top of Kubernetes?” And my default answer to that is good lord, no. What am I missing?Benjamin: That's a good question. Kubernetes has come a long way—I think is part of that.Corey: Oh, truly. I used to think of containers as a pure story for stateless things. And then, of course, I put state into them, and then, everything exploded everywhere because it turns out, I'm bad at computers. Great. And it has come a long way. I have been tracking a lot of that. But it still feels like the idea being that you'd want to have your database endpoints somewhere a lot less, I guess I'll call it fickle, if that makes sense.Benjamin: It's an interesting problem because we are seeing a lot of people who are interested in our Kubernetes-based products. It's actually based on—we recently open-sourced the core of it under a project called cloud-native PG. It's a cool piece of technology. If you think about sort of two by two. In one corner, you've got self-managed on-premise databases. So, you're very, very slow-moving, big-iron type, old-school database deployments. And on the opposite corner, you've got fully-managed, in the cloud, BigAnimal, Amazon RDS, that type of thing. There's a place on that map where you've got customers that want a self-service type experience. Whether that's for production, or maybe it's even for dev tests, something like that. But you don't want to be giving the management capability off to a third party.For folks that want that type of experience, trying to build that themselves by, like, wiring up EC2 instances, or doing something in their own data center with VMware, or something like that, can be extremely difficult. Whereas if you've go to a Kubernetes-based product, you can get that type of self-service experience really easily, right? And customers can get a lot more flexibility out of how they run their databases and operate their databases. And what sort of control they give to, say application developers who want to spin up a new database for a test or for some sort of small microservice, that type of thing. Those types of workloads tend to work really well with this first-party Kubernetes-based offering. I've been doing databases on Kubernetes in managed services for a long time as well. And I don't, frankly, have any concerns about doing it. There are definitely some sharp edges. And if you wanted to do to-scale, you need to really know what you're doing with Kubernetes because the naive thing will shoot you in the foot.Corey: Oh, yes. So, some it feels almost like people want to cosplay working for Google, but they don't want to pass the technical interview along the way. It's a bit of a weird moment for it.Benjamin: Yeah, I would agree.Corey: I have to go back to my own experiences with using RDS back at my last real job before I went down this path. We were migrating from EC2-Classic to VPC. So, you could imagine what dates me reasonably effectively. And the big problem was the database. And the joy that we had was, “Okay, we have to quiesce the application.” So, the database is now quiet, stop writes, take a snapshot, restore that snapshot into the environment. And whenever we talk to AWS folks, it's like, “So, how long is this going to take?” And the answer was, “Guess.” And that was not exactly reassuring. It went off without a hitch because every migration has one problem. We were sideswiped in an Uber on the way home. But that's neither here nor there. This was two o'clock in the morning, and we finished in half the maintenance time we had allotted. But it was the fact that, well, guess we're going to have to take the database down for many hours with no real visibility, and we hope it'll be up by morning. That wasn't great. But that was the big one going on, on an ongoing basis, there were maintenance windows with a database. We just stopped databasing for a period of time during a fairly broad maintenance window. And that led to a whole lot of unfortunate associations in my mind with using relational databases for an awful lot of stuff. How do you handle maintenance windows and upgrading and not tearing down someone's application? Because I have to assume, “Oh, we just never patch anything. It turns out that's way easier,” is in fact, the wrong answer.Benjamin: Yeah, definitely. As you point out, there's a bunch of fundamental limitations here, if we start to talk about how Postgres actually fits together, right? Pretty much everybody in RDS is a little bit weird. The older RDS offerings are a little bit weird in terms of how they do replication. But most folks are using Postgres streaming replication, to do high availability, Postgres in managed services. And honestly, of course—Corey: That winds up failing over, or the application's aware of both endpoints and switches to the other one?Benjamin: Yeah—Corey: Sort of a database pooling connection or some sort of proxy?Benjamin: Right. There's a bunch of subtleties that get into their way. You say, well, did the [vit 00:16:16] failover too early, did the application try to connect and start making requests before the secondaries available? That sort of thing.Corey: Or you misconfigure it and point to the secondary, suddenly, when there's a switchover of some database, suddenly, nothing can write, it can only read, then you cause a massive outage on the weekend?Benjamin: Yeah. Yeah.Corey: That may have been of an actual story I made up.Benjamin: [laugh] yeah, you should use a managed service.Corey: Yeah.Benjamin: So, it's complicated, but even with managed services, you end up in situations where you have downtime, you have maintenance windows. And with Postgres, especially—and other databases as well—especially with Postgres, one of the biggest concerns you have is major version upgrades, right? So, if I want to go from Postgres 12 to 13, 13 to 14, I can't do that live. I can't have a single cluster that is streaming one Postgres version to another Postgres version, right?So, every year, people want to put things off for two years, three years sometimes—which is obviously not to their benefit—you have this maintenance, you have some sort of downtime, where you perform a Postgres upgrade. At EDB, we've got—so this is a big problem, this is a problem for us. We're involved in the Postgres community. We know this is challenging. That's just a well-known thing. Some of the folks that are working EDB are folks who worked on the Postgres logical replication tech, which arrived in Postgres 10. Logical replication is really a nice tool for doing things like change data capture, you can do Walter JSON, all these types of things are based on logical replication tech.It's not really a thing, at least, the code that's in Postgres itself doesn't really support high availability, though. It's not really something that you can use to build a leader-follower type cluster on top of. We have some techs, some proprietary tech within EDB that used to be called bi-directional replication. There used to be an open-source project called bi-directional replication. This is a kind of a descendant of that. It's now called Postgres Distributed, or EDB Postgres Distributed is the product name. And that tech actually allows us—because it's based on logical replication—allows us to do multiple major versions at the same time, right? So, we can upgrade one node in a cluster to Postgres 14, while the other nodes in the clusters are at Postgres 13. We can then upgrade the next node. We can support these types of operations in a kind of wide range of maintenance operations without taking a cluster down from maintenance.So, there's a lot of interesting opportunities here when we start to say, well, let's step back from what your typical assumptions are for Postgres streaming replication. Give ourselves a little bit more freedom by using logical replication instead of physical streaming replication. And then, what type of services, and what type of patterns can we build on top of that, that ultimately help customers build, whether it's faster databases, more highly available databases, so on and so forth.Corey: Let's face it, on-call firefighting at 2am is stressful! So there's good news and there's bad news. The bad news is that you probably can't prevent incidents from happening, but the good news is that incident.io makes incidents less stressful and a lot more valuable. incident.io is a Slack-native incident management platform that allows you to automate incident processes, focus on fixing the issues and learn from incident insights to improve site reliability and fix your vulnerabilities. Try incident.io, recover faster and sleep more.Corey: One approach that I took for, I guess you could call it backup sort of, was intentionally staggering replication between the primary and the replica about 15 minutes or so. So, if I drop a production table or something like that, I have 15 short minutes to realize what has happened and sever the replication before it is now committed to the replica and now I'm living in hell. It felt like this was not, like, option A, B, or C, or the right way to do things. But given that meeting customers where they are as important, is that the sort of thing that you support with BigAnimal, or do you try to talk customers into not being ridiculous?Benjamin: That's not something we support now. It's not actually something that I hear that many asks for these days. It's kind of interesting, that's a pattern that I've run into a lot in the past.Corey: I was an ancient, grumpy sysadmin. Again, I'm dating myself here. These days, I just store everything at DNS text records, and it's way easier. But I digress.Benjamin: [laugh] yeah, it's something that we see a lot for and we had support for a point-in-time restore, like pretty much anybody else in the business at this point. And that's usually the, “I fat-fingered something,” type response. Honestly, I think there's room to be a bit more flexible and room to do some more interesting things. I think RDS is setting a bar and a lot of database services out there and kind of just meeting that bar. And we all kind of need to be pushing a little bit more into more interesting spaces and figuring out how to get customers more value, get customers to get more out of their money for the database, honestly.Corey: One of the problems we tend to see, in the database ecosystem at large, without naming names or companies or anything like that, is that it's a pretty thin and blurry line between database advocate, database evangelist, and database zealot. Where it feels like instead, we're arguing about religion more than actual technical constraints and concerns. So, here's a fun question that hopefully isn't too much of a gotcha. But what sort of workloads would you actively advise someone not to use BigAnimal for in the database world? But yes, again, if you try to run a DNS server, it's probably not fit for purpose without at least a shim in the way there. But what sort of workloads are you not targeting that a customer is likely to have a relatively unfortunate time with?Benjamin: Large-scale analytical workloads is the easy answer to that, right? If you've got a problem where you're choosing between Postgres and Snowflake, you're seriously considering—you actually have as much data that you seriously be considering Snowflake? You probably don't want to be using Postgres, right? You want to be using something that's column, or you want to be using a query planner that really understands a columnar layout that's going to get you the sorts of performance that you need for those analytical workloads. We don't try to touch that space.Corey: Yeah, we're doing some of that right now with just the sheer volume of client AWS bills we have. We don't really need a relational model for a lot of it. And Athena is basically fallen down on the job in some cases, and, “Oh, do you want to use Redshift, that's basically Postgres.” It's like, “Yeah, it's Postgres, if it decided to run on bars of gold.” No, thank you. It just becomes this ridiculously overwrought solution for what feels like it should be a lot similar. So, it's weird, six months ago or so I wouldn't have had much of an idea what you're talking about. I see it a lot better now. Generally, by virtue of trying to do something the precise wrong way that someone should.Benjamin: Right. Yeah, exactly. I think there's interesting room for Postgres to expand here. It's not something that we're actively working on. I'm not aware of a lot happening in the community that Postgres is, for better or worse, extremely extensible, right? And if you see the JSON-supported Postgres, it didn't exist, I don't know, five, six years ago. And now it's incredibly powerful. It's incredibly flexible. And you can do a lot of quote-unquote, schemaless stuff straight in Postgres. Or you look at PostGIS, right, for doing GIS geographical data, right? That's really a fantastic integration directly in the database.Corey: Yeah, before that people start doing ridiculous things almost looks similar to a graph database or a columnar store somehow, and yeah.Benjamin: Yeah, exactly. I think sometimes somebody will do a good column store that's an open-source deeply integrated into Postgres, rather than—Corey: I've seen someone build one on top of S3 bucket with that head, a quarter of a trillion objects in it. Professional advice, don't do that.Benjamin: [laugh]. Unless you're Snowflake. So, I mean, it's something that I'd like to see Postgres expand into. I think that's an interesting space, but not something that, at least especially for BigAnimal, and frankly, for a lot of EDB customers. It's not something we're trying to push people toward.Corey: One thing that I think we are seeing a schism around is the idea that some vendors are one side of it, some are on the other, where on the one side, you have, oh, every workload should have a bespoke, purpose-built database that is exactly for this type of workload. And the other school of thought is you should generally buy us for a general-purpose database until you have a workload that is scaled and significant to a point where running that on its own purpose-built database begins to make sense. I don't necessarily think that is a binary choice, where do you tend to fall on that spectrum?Benjamin: I think everybody should use Postgres. And I say not just because I work in a Postgres company.Corey: Well, let's be clear. Before this, you were at IBM for five years working on a whole bunch of database stuff over there, not just Postgres. And you, so far, have not struck me as the kind of person who's like, “Oh, so what's your favorite database?” “The one that pays me.” We've met people like that, let's be very clear. But you seem very even-handed in those conversations.Benjamin: Yeah, I got my start in databases, actually, with Apache CouchDB. I am a committer on CouchDB. I worked on a managed at CouchDB service ten years ago. At IBM, I worked on something in nine different open-source databases and managed services. But I love having conversations about, like, well, I've got this workload, should I use Postgres, rr should I use Mongo, should I use Cassandra, all of those types of discussions. Frankly, though, I think in a lot of cases people are—they don't understand how much power they're missing out on if they don't choose a relational database. If they don't understand the relational model well enough to understand that they really actually want that. In a lot of cases, people are also just over-optimizing too early, right? It's just going to be much faster for them to get off the ground, get product in customers hands, if they start with something that they don't have to think twice about. And they don't end up with this architecture with 45 different databases, and there's only one guy in the company that knows how to manage the whole thing.Corey: Oh, the same story of picking a cloud provider. It's, “Okay, you hire a team, you're going to build a thing. Which cloud provider do you pick?” Every cloud provider has a whole matrix and sales deck, and the rest. The right answer, of course, is the one your team's already familiar with because learning a new cloud provider while trying not to run out of money at your startup, can't really doesn't work super well.Benjamin: Exactly. Yeah.Corey: One thing that I think has been sort of interesting, and when I saw it, it was one of those, “Oh, I sort of like them.” Because I had that instinctive reaction and I don't think I'm alone in this. As of this recording a couple of weeks ago, you folks received a sizable investment from private equity. And default reaction to that is, “Oh, well, I guess I put a fork in the company, they're done.” Because the narrative is that once private equity takes an investment, well, that company's best days are probably not in front of it. Now, the counterpoint is that this is not the first time private equity has invested in EDB, and you folks from what I can tell are significantly better than you were when I was your customer a decade ago. So clearly, there is something wrong with that mental model. What am I missing?Benjamin: Yeah. Frankly, I don't know. I'm no expert in funding models and all of those sorts of things. I will say that my experience has been what I've seen at EDB, has definitely been that maybe there's private equity, and then there's private equity. We're in this to build better products and become a better product company. We were previously owned by a private equity firm for the past four years or so. And during the course of those four years, we brought on a bunch of folks who were very product-focused, new leadership. We made a significant acquisition of a company called 2ndQuadrant, which they employed a lot of the European best Postgres company. Now, they're part of EDB and most of them have stayed with us. And we built the managed cloud service, right? So, this is a pretty significant—private equity company buying us to invest in the company. I'm optimistic that that's what we're looking at going forward.Corey: I want to be clear as well, I'm not worried about what I normally would be in a private equity story about this, where they're there to save money and cut costs, and, “Do we really need all these database replicas floating around,” and, “These backups, seems like that's something we don't need.” You have, at last count, 32 Postgres contributors, 7 Postgres committers, and 3 core members. All of whom would run away screaming loudly and publicly, in the event that such a thing were taking place. Of all the challenges and concerns I might have about someone running a cloud service in the modern day. I do not have any fear that you folks are not doing what will very clearly be shown to be the right thing by your customers for the technology that you're building on top of. That is not a concern. There are companies I do not have that confidence in, to be clear.Benjamin: Yeah, I'm glad to hear that. I'm a hundred percent on board as well. I work here, but I think we're doing the right thing, and we're going to be doing great stuff going forward.Corey: One last topic I do want to get into a little bit is, on some level, launching in this decade, a cloud-hosted database offering at a time when Amazon—whose product strategy of yes is in full display—it seems like something ridiculous, that is not necessarily well thought out that why would you ever try to do this? Now, I will temper that by the fact that you are clearly succeeding in this direction. You have customers who say nice things about you, and the reviews have been almost universally positive anywhere I can see things. The negative ones are largely complaining about databases, which I admit might be coming from me.Benjamin: Right, it is a crowded space. There's a lot of things happening. Obviously, Amazon, Microsoft, Google are doing great things, both—Corey: Terrible things, but great, yes. Yes.Benjamin: [laugh] right, there's good products coming in. I think AlloyDB is not necessarily a great product. I haven't used it myself yet, but it's an interesting step in the direction. I'm excited to see development happening. But at the end of the day, we're a database company. Our focus is on building great databases and supporting great databases. We're not entering this business to try to take on Amazon from an infrastructure point of view. In fact, the way that we're structuring the product is really to try to get the strengths of both worlds. We want to give customers the ability to get the most out of the AWS or Azure infrastructure that they can, but come to us for their database.Frankly, we know Postgres better than anybody else. We have a greater ability to get bugs fixed in Postgres than anybody else. We've got folks working on the database in the open. We got folks working on the database proprietary for us. So, we give customers things like break/fix support on that database. If there is a bug in Postgres, there's a bug in the tech that sits around Postgres. Because obviously, Postgres is not a batteries-included system, really. We're going to fix that for you. That's part of the contract that we're giving to our customers. And I know a lot of smaller companies maybe haven't been burned by this sort of thing very much. We start to talk about enterprise customers and medium, larger-scale customers, this starts to get really valuable. The ability to have assurance on top of your open-source product. So, I think there's a lot of interesting things there, a lot of value that we can provide there.I think also that I talked a little bit about this earlier, but like the box, this sort of RDS-shaped box, I think is a bit too small. There's an opportunity for smaller players to come in and try to push the boundaries of that. For example, giving customers more support by default to do a good job using their database. We have folks on board that can help consult with customers to say, “No, you shouldn't be designing your schemas that way. You should be designing your schemas this way. You should be using indexes here,” that sort of stuff. That's been part of our business for a long time. Now, with a managed service, we can bake that right into the managed service. And that gives us the ability to kind of make that—you talk about shared responsibility between the service writer and the customer—we can change the boundaries of that shared responsibility a little bit, so that customers can get more value out of the managed database service than they might expect otherwise.Corey: There aren't these harsh separations and clearly defined lines across which nothing shall pass, when it makes sense to do that in a controlled responsible way.Benjamin: Right, exactly. Some of that is because we're a database company, and some of that is because, frankly, we're much smaller.Corey: I'll take it a step further beyond that, as well, that I have seen this pattern evolve a number of times where you have a customer running databases on EC2, and their AWS account managers suggests move to RDS. So, they do. Then, move to Aurora. So, they do. Then, I move this to DynamoDB. At which point, it's like, what do you think your job is here, exactly? Because it seems like every time we move databases, you show up in a nicer car. So, what exactly is the story here, and what are the incentives? Where it just feels like there is a, “Whatever you're doing is not the way that it should be done. So, it's time to do, yet, another migration.”There's something to be said for companies who are focused around a specific aspect of things. Then once that is up and working and running, great. Keep on going. This is fine. As opposed to trying to chase the latest shiny, on some level. I have a big sense of, I guess, affinity for companies that wind up knowing where they start, and most notably, where they stop.Benjamin: Yeah, I think that's a really good point. I don't think that we will be building an application platform anytime soon.Corey: “We're going to run Lambda functions on top of a database.” It's like, “Congratulations. That is the weirdest stored procedure I can imagine this week, but I'm sure we can come up with a worse one soon.”Benjamin: Exactly.Corey: I really want to thank you for taking the time to speak with me so much about how you're thinking about this, and what you've been building over there. If people want to learn more, where's the best place to go to find you?Benjamin: biganimal.com.Corey: Excellent. We will throw a link to that in the show notes and it only just occurred to me that the Postgres mascot is an elephant, and now I understand why it's called BigAnimal. Yeah, that's right. He who laughs last, thinks slowest, and today, that's me. I really want to thank you for being so generous with your time. I appreciate it.Benjamin: Thank you. I really appreciate it.Corey: Benjamin Anderson, CTO for Cloud at EDB. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment that you then wind up stuffing into a SQLite database, converting to Base64, and somehow stuffing into the comment field.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

The Cloud Pod
165: The Cloud Pod Angry That Amazon Describes Step Functions as Low Code

The Cloud Pod

Play Episode Listen Later May 20, 2022 33:33


On The Cloud Pod this week, the team discusses wholesome local Oakland toast for breakfast. Plus: Hybrid infrastructure is unsustainable, the AWS Proton template library expands, and Amazon angers the team by describing Step Functions as “low-code.” A big thanks to this week's sponsor, Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights

The Cloud Pod
164: The Cloud Pod SWIFT-ly Moves Its Money to Google Cloud

The Cloud Pod

Play Episode Listen Later May 16, 2022 42:45


On The Cloud Pod this week, Peter's been suspended without pay for two weeks for not filing his vacation requests in triplicate. Plus it's earnings season once again, there's a major Google and SWIFT collaboration afoot, and MSK Serverless is now generally available, making Kafka management fairly hassle-free. A big thanks to this week's sponsor, Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights

The Stack Overflow Podcast
Meet the design system that lets us customize and theme Stack Overflow

The Stack Overflow Podcast

Play Episode Listen Later Apr 26, 2022 32:17


If you're not familiar with Stacks, Stack Overflow's design system, it's a robust CSS and JavaScript Pattern library that helps users create coherent experiences in line with Stack Overflow's best practices and design principles. Explore more on Netlify or GitHub.Missed our April Fool's prank this year? Relive the hilarity and the pain.Atomic CSS is a CSS architecture approach favoring single-purpose classes named based on visual function.Today's Lifeboat badge goes to user ceejayoz for their answer to How do I do a database backup on Amazon RDS every hour?.Connect with Ben Kelley.Learn more about Aaron Shekey's work.

The Cloud Pod
161: The Cloud Pod Observes Its Databases With Google Cloud SQL Insights

The Cloud Pod

Play Episode Listen Later Apr 21, 2022 23:40


On The Cloud Pod this week and with half the team gone fishin', Justin and Peter hash it out short and sweet. Plus Google Cloud SQL Insights, Atlassian suffers an outage, and AWS finally offers accessible Lambda Function URLs. A big thanks to this week's sponsor, Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights

The Cloud Pod
159: The Cloud Pod Suspends Its (GCP) Hosts

The Cloud Pod

Play Episode Listen Later Apr 7, 2022 34:13


On The Cloud Pod this week, Ryan is in the doghouse and he's been suspended (with full pay). Plus, we're comfortably numb with AWS Cloud NGFW, GCP suspends hosts for big savings, and Azure is once again shutting the Front Door on us.  A big thanks to this week's sponsor, Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights

The Cloud Pod
158: The Cloud Pod Discloses All of Its Okta Breaches

The Cloud Pod

Play Episode Listen Later Apr 4, 2022 40:08


On The Cloud Pod this week, it's a brave new world for Ryan, who learns all kinds of things. Plus the Okta breach leads to customer outrage over not telling them for months, AWS announces its new Billing Conductor, and Google expands Contact Center AI for a reimagined customer experience.  A big thanks to this week's sponsor, Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights

The Cloud Pod
157: The Cloud Pod Goes on a Quest…. An AWS Cloud Quest

The Cloud Pod

Play Episode Listen Later Mar 24, 2022 56:35


On The Cloud Pod this week, the team discusses Peter's concept of fun. Plus digital adventures with AWS Cloud Quest game, much-wanted Google price increases, and a labyrinthine run-through of the details of Azure Health Data Services. A big thanks to this week's sponsor, Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights