Podcasts about cloud spanner

This week, we discuss Amazon embracing Microsoft Office 365, offer some SBF hot takes, and review the lessons Docker learned when building an open-source business. Plus, we share thoughts on the new Apple Pencil, USB-C, and some Tim Cook fan fiction. Watch the YouTube Live Recording of Episode (https://www.youtube.com/live/FR4HLs-xTOE?si=HsavpdEHYVF_FhYP) 437 (https://www.youtube.com/live/FR4HLs-xTOE?si=HsavpdEHYVF_FhYP) Runner-up Titles My enemy's Word Processor is my friend. You know what we should do, we should just meet about it. A downgrade would be an upgrade. Megadeal's a great word. It worked for Shingy Use my template. Rundown Amazon moves to the cloud Microsoft is preparing to bring on Amazon as a customer of its 365 cloud tools in a $1 billion megadeal, according to an internal document (https://www.businessinsider.com/microsoft-prepares-amazon-customer-365-cloud-tools-2023-10) Report: Amazon will use Microsoft 365 cloud productivity tools in $1B ‘megadeal' (https://www.geekwire.com/2023/report-amazon-will-use-microsoft-365-cloud-productivity-tools-in-1b-megadeal/) SBF Sam Bankman-Fried's legal peril deepens as his defense comes up short (https://www.washingtonpost.com/business/2023/10/17/bankman-fried-trial/?utm_campaign=wp_post_most&utm_medium=email&utm_source=newsletter&wpisrc=nl_most) Number Goes Up (https://www.amazon.com/Number-Go-Up-Cryptos-Staggering/dp/0593443810) Going Infinite: The Rise and Fall of a New Tycoon (https://www.amazon.com/Going-Infinite-Rise-Fall-Tycoon/dp/B0CD8V9SHD/ref=sr_1_1?crid=1YTBDKGIG9B2Y&keywords=going+infinity+michael+lewis&qid=1697580041&s=books&sprefix=Michael+Lewis+Infi%2Cstripbooks%2C156&sr=1-1) OSS Business Success with Open Source (https://pragprog.com/titles/vbfoss/business-success-with-open-source/) HashiCorp CEO predicts OSS-free Silicon Valley unless... (https://www.thestack.technology/hashicorp-ceo-predicts-oss-free-silicon-valley-unless-the-open-source-model-evolves/) Docker at 10 — 3 Things We Got Right, 3 Things We Got Wrong (https://thenewstack.io/docker-at-10-3-things-we-got-right-3-things-we-got-wrong/) How open source foundations protect the licensing integrity of open source projects (https://www.linuxfoundation.org/blog/how-open-source-foundations-protect-the-licensing-integrity-of-open-source-projects) VMware: What China Might Ask Of Broadcom Is Concerning Markets (NYSE:VMW) (https://seekingalpha.com/article/4641336-vmware-what-china-might-ask-broadcom-concerning-markets) Relevant to your Interests So Far, AI Is a Money Pit That Isn't Paying Off (https://gizmodo.com/github-copilot-ai-microsoft-openai-chatgpt-1850915549) IRS says Microsoft owes an additional $29 billion in back taxes (https://www.cnbc.com/2023/10/11/irs-says-microsoft-owes-an-additional-29-billion-in-back-taxes.html) Six Months Ago NPR Left Twitter. The Effects Have Been Negligible | Nieman Reports (https://niemanreports.org/articles/npr-twitter-musk/) Data transformation startup Prophecy lands $35M investment | TechCrunch (https://techcrunch.com/2023/10/11/data-transformation-startup-prophecy-lands-35m-investment/) Google turns up the heat on AWS, claims Cloud Spanner is half the cost of DynamoDB (https://techcrunch.com/2023/10/11/google-turns-up-the-heat-on-aws-claims-cloud-spanner-is-half-the-cost-of-dynamodb/) Apple reaches settlement with Caltech in $1 billion patent lawsuit - 9to5Mac (https://9to5mac.com/2023/10/12/apple-reaches-settlement-with-caltech-in-1-billion-patent-lawsuit/) We tried that, didn't work (https://world.hey.com/dhh/we-tried-that-didn-t-work-d9c42fe1) Engage a Wider Audience With ActivityPub on WordPress.com (https://wordpress.com/blog/2023/10/11/activitypub/) Apple wants to update iPhones in-store without opening the packaging (https://appleinsider.com/articles/23/10/15/apple-plans-to-update-iphones-in-store-without-opening-the-boxes) Atlassian content cloud migration will work. Users, less so (https://www.theregister.com/2023/10/16/atlassian_cloud_migration_server_deprecation/) Opinion | The Five-Day Office Week Is Dead (https://www.nytimes.com/2023/10/16/opinion/office-work-home-remote.html) Minecraft becomes first video game to hit 300m sales (https://www.bbc.com/news/technology-67105983) Marc Andreessen -- e/acc on X (https://x.com/pmarca/status/1713930459779129358?s=46&t=zgzybiDdIcGuQ_7WuoOX0A) Microsoft-owned LinkedIn lays off nearly 700 employees — read the memo here (https://www.cnbc.com/2023/10/16/microsoft-owned-linkedin-lays-off-nearly-700-read-the-memo-here.html) Apple introduces new Apple Pencil, bringing more value and choice to the lineup (https://www.apple.com/uk/newsroom/2023/10/apple-introduces-new-apple-pencil-bringing-more-value-and-choice-to-the-lineup/?utm_source=substack&utm_medium=email) SiFive Rolls Out RISC-V Cores Aimed at Generative AI and ML (https://www.allaboutcircuits.com/news/sifive-rolls-out-risc-v-cores-aimed-at-generative-ai-and-ml/) Apple introduces new Apple Pencil, bringing more value and choice to the lineup (https://www.apple.com/uk/newsroom/2023/10/apple-introduces-new-apple-pencil-bringing-more-value-and-choice-to-the-lineup/?utm_source=substack&utm_medium=email) Amazon quietly rolls out support for passkeys, with a catch | TechCrunch (https://techcrunch.com/2023/10/17/amazon-passkey-sign-in/) The price of managed cloud services (https://world.hey.com/dhh/the-price-of-managed-cloud-services-4f33d67e) Microsoft launches Radius, an open-source application platform for the cloud-native era (https://techcrunch.com/2023/10/18/microsoft-launches-radius-an-open-source-application-platform-for-the-cloud/?guccounter=1) UK Atlassian users complain of migration dead end (https://www.theregister.com/2023/10/18/atlassian_server_imgration_deadend/) Passwordless authentication startup SecureW2 raises $80M from Insight Partners (https://techcrunch.com/2023/10/18/passwordless-authentication-startup-securew2-raises-80m-from-insight-partners/?guccounter=1&guce_referrer=aHR0cHM6Ly9uZXdzLmdvb2dsZS5jb20v&guce_referrer_sig=AQAAAEg5u3LvXY_CzdVG2zQM-BixvZEUGH7W4PyZHAEyHEsInAVRmaxLjTPXHrs4ANq38SKj2Siv_yRyw2U4yR8SXfSjusCwmdqRjjscKA_XjYDMQrpLT0MhenCQfOiqmhCSCcx5PyfuW0Ga8dH4R8blCLZ8v176Pt-4IKPwZ1oQ54ph) Convicted Fugees rapper Pras Michel's lawyer used AI to draft bungled closing argument (https://www.nbcnews.com/news/us-news/convicted-fugees-rapper-pras-michels-lawyer-used-ai-draft-bungled-clos-rcna120992) IRS to offer a new option to file your tax return (https://www.washingtonpost.com/business/2023/10/17/irs-free-tax-filing-eligibility/) Welcoming Loom to the Atlassian team (https://www.atlassian.com/blog/announcements/atlassian-acquires-loom) Nonsense Costco sold $9B of clothing in 2022 (https://x.com/trungtphan/status/1712581893886181863?s=46&t=zgzybiDdIcGuQ_7WuoOX0A) United's new boarding system prioritizes window seats (https://www.yahoo.com/lifestyle/uniteds-boarding-system-prioritizes-window-211759965.html) Listener Feedback Software Engineering at Google (https://abseil.io/resources/swe-book) Sr. Product Marketing Manager, Platform Engineering (https://boards.greenhouse.io/harnessinc/jobs/4102778007) Conferences Nov 6-9, 2023, KubeCon NA (https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/), SDT's a sponsor, Matt's there. Use this VMware discount code for 20% off: KCNA23VMWEO20. Nov 6-9, 2023 VMware Explore Barcelona (https://www.vmware.com/explore/eu.html), Coté's attending Nov 7–8, 2023 RISC-V Summit | Linux Foundation Events (https://events.linuxfoundation.org/riscv-summit/) Jan 29, 2024 to Feb 1, 2024 That Conference Texas (https://that.us/events/tx/2024/schedule/) If you want your conference mentioned, let's talk media sponsorships. SDT news & hype Join us in Slack (http://www.softwaredefinedtalk.com/slack). Get a SDT Sticker! Send your postal address to stickers@softwaredefinedtalk.com (mailto:stickers@softwaredefinedtalk.com) and we will send you free laptop stickers! Follow us: Twitch (https://www.twitch.tv/sdtpodcast), Twitter (https://twitter.com/softwaredeftalk), Instagram (https://www.instagram.com/softwaredefinedtalk/), Mastodon (https://hachyderm.io/@softwaredefinedtalk), BlueSky (https://bsky.app/profile/softwaredefinedtalk.com), LinkedIn (https://www.linkedin.com/company/software-defined-talk/), TikTok (https://www.tiktok.com/@softwaredefinedtalk), Threads (https://www.threads.net/@softwaredefinedtalk) and YouTube (https://www.youtube.com/channel/UCi3OJPV6h9tp-hbsGBLGsDQ/featured). Use the code SDT to get $20 off Coté's book, Digital WTF (https://leanpub.com/digitalwtf/c/sdt), so $5 total. Become a sponsor of Software Defined Talk (https://www.softwaredefinedtalk.com/ads)! Recommendations Brandon: Sign up for Installer - The Verge (https://www.theverge.com/pages/installer-newsletter-sign-up) Matt: Dell customer support Coté: Evil Dead Rises (https://en.wikipedia.org/wiki/Evil_Dead_Rise). Also, this picture of Bruce Campbell (https://ew.com/thmb/Z-6NqxZMtIassHzw1Wgcs4LuntA=/750x0/filters:no_upscale():max_bytes(150000):strip_icc()/Bruce-Campbell-Evil-Dead-Rise-031623-392c8a22d985493583a1ccdcb11f1618.jpg), from here (https://ew.com/movies/bruce-campbell-shuts-down-evil-dead-rise-heckler-sxsw/). Photo Credits Header (https://unsplash.com/photos/black-tablet-computer-on-brown-wooden-table-aVP3ryIQKpM)

The RIAA calls for AI voice cloning sites to be added to the government piracy watchlist, the EU gives Meta 24 hours to respond to pro-Hamas content, Google unveils the Pixel 8 Pro and 8 Pro cameras, SiFive introduces new chip designs, the UN's Internet Governance Forum to host next international forum in Saudi Arabia, Google's Cloud Spanner becomes more efficient, Xsolla acquires Lightstream, Rainmaker, and API, KIT Plugins raises $1 million in funding, Sony establishes the Sony Innovation Fund: Africa, Dropbox releases new web interface and collaboration tools, SEC investigates MOVEit mass-hack, and Logitech launches Streamlabs plugin for Loupedeck consoles.

About AndiAndi Gutmans is the General Manager and Vice President for Databases at Google. Andi's focus is on building, managing and scaling the most innovative database services to deliver the industry's leading data platform for businesses. Before joining Google, Andi was VP Analytics at AWS running services such as Amazon Redshift. Before his tenure at AWS, Andi served as CEO and co-founder of Zend Technologies, the commercial backer of open-source PHP.Andi has over 20 years of experience as an open source contributor and leader. He co-authored open source PHP. He is an emeritus member of the Apache Software Foundation and served on the Eclipse Foundation's board of directors. He holds a bachelor's degree in Computer Science from the Technion, Israel Institute of Technology.Links Referenced: LinkedIn: https://www.linkedin.com/in/andigutmans/ Twitter: https://twitter.com/andigutmans TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Sysdig. Sysdig secures your cloud from source to run. They believe, as do I, that DevOps and security are inextricably linked. If you wanna learn more about how they view this, check out their blog, it's definitely worth the read. To learn more about how they are absolutely getting it right from where I sit, visit Sysdig.com and tell them that I sent you. That's S Y S D I G.com. And my thanks to them for their continued support of this ridiculous nonsense.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. This promoted episode is brought to us by our friends at Google Cloud, and in so doing, they have gotten a guest to appear on this show that I have been low-key trying to get here for a number of years. Andi Gutmans is VP and GM of Databases at Google Cloud. Andi, thank you for joining me.Andi: Corey, thanks so much for having me.Corey: I have to begin with the obvious. Given that one of my personal passion projects is misusing every cloud service I possibly can as a database, where do you start and where do you stop as far as saying, “Yes, that's a database,” so it rolls up to me and, “No, that's not a database, so someone else can deal with the nonsense?”Andi: I'm in charge of the operational databases, so that includes both the managed third-party databases such as MySQL, Postgres, SQL Server, and then also the cloud-first databases, such as Spanner, Big Table, Firestore, and AlloyDB. So, I suggest that's where you start because those are all awesome services. And then what doesn't fall underneath, kind of, that purview are things like BigQuery, which is an analytics, you know, data warehouse, and other analytics engines. And of course, there's always folks who bring in their favorite, maybe, lesser-known or less popular database and self-manage it on GCE, on Compute.Corey: Before you wound up at Google Cloud, you spent roughly four years at AWS as VP of Analytics, which is, again, one of those very hazy type of things. Where does it start? Where does it stop? It's not at all clear from the outside. But even before that, you were, I guess, something of a legendary figure, which I know is always a weird thing for people to hear.But you were partially at least responsible for the Zend Framework in the PHP world, which I didn't realize what the heck that was, despite supporting it in production at a couple of jobs, until after I, for better or worse, was no longer trusted to support production environments anymore. Which, honestly, if you can get out, I'm a big proponent of doing that. You sleep so much better without a pager. How did you go from programming languages all the way on over to databases? It just seems like a very odd mix.Andi: Yeah. No, that's a great question. So, I was one of the core developers of PHP, and you know, I had been in the PHP community for quite some time. I also helped ideate. The Zend Framework, which was the company that, you know, I co-founded Zend Technologies was kind of the company behind PHP.So, like Red Hat supports Linux commercially, we supported PHP. And I was very much focused on developers, programming languages, frameworks, IDEs, and that was, you know, really exciting. I had also done quite a bit of work on interoperability with databases, right, because behind every application, there's a database, and so a lot of what we focused on is a great connectivity to MySQL, to Postgres, to other databases, and I got to kind of learn the database world from the outside from the application builders. We sold our company in I think it was 2015 and so I had to kind of figure out what's next. And so, one option would have been, hey, stay in programming languages, but what I learned over the many years that I worked with application developers is that there's a huge amount of value in data.And frankly, I'm a very curious person; I always like to learn, so there was this opportunity to join Amazon, to join the non-relational database side, and take myself completely out of my comfort zone. And actually, I joined AWS to help build the graph database Amazon Neptune, which was even more out of my comfort zone than even probably a relational database. So, I kind of like to do different things and so I joined and I had to learn, you know how to build a database pretty much from the ground up. I mean, of course, I didn't do the coding, but I had to learn enough to be dangerous, and so I worked on a bunch of non-relational databases there such as, you know, Neptune, Redis, Elasticsearch, DynamoDB Accelerator. And then there was the opportunity for me to actually move over from non-relational databases to analytics, which was another way to get myself out of my comfort zone.And so, I moved to run the analytic space, which included services like Redshift, like EMR, Athena, you name it. So, that was just a great experience for me where I got to work with a lot of awesome people and learn a lot. And then the opportunity arose to join Google and actually run the Google transactional databases including their older relational databases. And by the way, my job actually have two jobs. One job is running Spanner and Big Table for Google itself—meaning, you know, search ads and YouTube and everything runs on these databases—and then the second job is actually running external-facing databases for external customers.Corey: How alike are those two? Is it effectively the exact same thing, just with different API endpoints? Are they two completely separate universes? It's always unclear from the outside when looking at large companies that effectively eat versions of their own dog food, where their internal usage of these things starts and stops.Andi: So, great question. So, Cloud Spanner and Cloud Big Table do actually use the internal Spanner and Big Table. So, at the core, it's exactly the same engine, the same runtime, same storage, and everything. However, you know, kind of, internally, the way we built the database APIs was kind of good for scrappy, you know, Google engineers, and you know, folks are kind of are okay, learning how to fit into the Google ecosystem, but when we needed to make this work for enterprise customers, we needed a cleaner APIs, we needed authentication that was an external, right, and so on, so forth. So, think about we had to add an additional set of APIs on top of it, and management, right, to really make these engines accessible to the external world.So, it's running the same engine under the hood, but it is a different set of APIs, and a big part of our focus is continuing to expose to enterprise customers all the goodness that we have on the internal system. So, it's really about taking these very, very unique differentiated databases and democratizing access to them to anyone who wants to.Corey: I'm curious to get your position on the idea that seems to be playing it's—I guess, a battle that's been playing itself out in a number of different customer conversations. And that is, I guess, the theoretical decision between, do we go towards general-purpose databases and more or less treat every problem as a nail in search of a hammer or do you decide that every workload gets its own custom database that aligns the best with that particular workload? There are trade-offs in either direction, but I'm curious where you land on that given that you tend to see a lot more of it than I do.Andi: No, that's a great question. And you know, just for the viewers who maybe aren't aware, there's kind of two extreme points of view, right? There's one point of view that says, purpose-built for everything, like, every specific pattern, like, build bespoke databases, it's kind of a best-of-breed approach. The problem with that approach is it becomes extremely complex for customers, right? Extremely complex to decide what to use, they might need to use multiple for the same application, and so that can be a bit daunting as a customer. And frankly, there's kind of a law of diminishing returns at some point.Corey: Absolutely. I don't know what the DBA role of the future is, but I don't think anyone really wants it to be, “Oh, yeah. We're deciding which one of these three dozen manage database services is the exact right fit for each and every individual workload.” I mean, at some point it feels like certain cloud providers believe that not only every workload should have its own database, but almost every workload should have its own database service. It's at some point, you're allowed to say no and stop building these completely, what feel like to me, Byzantine, esoteric database engines that don't seem to have broad applicability to a whole lot of problems.Andi: Exactly, exactly. And maybe the other extreme is what folks often talk about as multi-model where you say, like, “Hey, I'm going to have a single storage engine and then map onto that the relational model, the document model, the graph model, and so on.” I think what we tend to see is if you go too generic, you also start having performance issues, you may not be getting the right level of abilities and trade-offs around consistency, and replication, and so on. So, I would say Google, like, we're taking a very pragmatic approach where we're saying, “You know what? We're not going to solve all of customer problems with a single database, but we're also not going to have two dozen.” Right?So, we're basically saying, “Hey, let's understand that the main characteristics of the workloads that our customers need to address, build the best services around those.” You know, obviously, over time, we continue to enhance what we have to fit additional models. And then frankly, we have a really awesome partner ecosystem on Google Cloud where if someone really wants a very specialized database, you know, we also have great partners that they can use on Google Cloud and get great support and, you know, get the rest of the benefits of the platform.Corey: I'm very curious to get your take on a pattern that I've seen alluded to by basically every vendor out there except the couple of very obvious ones for whom it does not serve their particular vested interests, which is that there's a recurring narrative that customers are demanding open-source databases for their workloads. And when you hear that, at least, people who came up the way that I did, spending entirely too much time on Freenode, back when that was not a deeply problematic statement in and of itself, where, yes, we're open-source, I guess, zealots is probably the best terminology, and yeah, businesses are demanding to participate in the open-source ecosystem. Here in reality, what I see is not ideological purity or anything like that and much more to do with, “Yeah, we don't like having a single commercial vendor for our databases that basically plays the insert quarter to continue dance whenever we're trying to wind up doing something new. We want the ability to not have licensing constraints around when, where, how, and how quickly we can run databases.” That's what I hear when customers are actually talking about open-source versus proprietary databases. Is that what you see or do you think that plays out differently? Because let's be clear, you do have a number of database services that you offer that are not open-source, but are also absolutely not tied to weird licensing restrictions either?Andi: That's a great question, and I think for years now, customers have been in a difficult spot because the legacy proprietary database vendors, you know, knew how sticky the database is, and so as a result, you know, the prices often went up and was not easy for customers to kind of manage costs and agility and so on. But I would say that's always been somewhat of a concern. I think what I'm seeing changing and happening differently now is as customers are moving into the cloud and they want to run hybrid cloud, they want to run multi-cloud, they need to prove to their regulator that it can do a stressed exit, right, open-source is not just about reducing cost, it's really about flexibility and kind of being in control of when and where you can run the workloads. So, I think what we're really seeing now is a significant surge of customers who are trying to get off legacy proprietary database and really kind of move to open APIs, right, because they need that freedom. And that freedom is far more important to them than even the cost element.And what's really interesting is, you know, a lot of these are the decision-makers in these enterprises, not just the technical folks. Like, to your point, it's not just open-source advocates, right? It's really the business people who understand they need the flexibility. And by the way, even the regulators are asking them to show that they can flexibly move their workloads as they need to. So, we're seeing a huge interest there and, as you said, like, some of our services, you know, are open-source-based services, some of them are not.Like, take Spanner, as an example, it is heavily tied to how we build our infrastructure and how we build our systems. Like, I would say, it's almost impossible to open-source Spanner, but what we've done is we've basically embraced open APIs and made sure if a customer uses these systems, we're giving them control of when and where they want to run their workloads. So, for example, Big Table has an HBase API; Spanner now has a Postgres interface. So, our goal is really to give customers as much flexibility and also not lock them into Google Cloud. Like, we want them to be able to move out of Google Cloud so they have control of their destiny.Corey: I'm curious to know what you see happening in the real world because I can sit here and come up with a bunch of very well-thought-out logical reasons to go towards or away from certain patterns, but I spent years building things myself. I know how it works, you grab the closest thing handy and throw it in and we all know that there is nothing so permanent as a temporary fix. Like, that thing is load-bearing and you'll retire with that thing still in place. In the idealized world, I don't think that I would want to take a dependency on something like—easy example—Spanner or AlloyDB because despite the fact that they have Postgres-squeal—yes, that's how I pronounce it—compatibility, the capabilities of what they're able to do under the hood far exceed and outstrip whatever you're going to be able to build yourself or get anywhere else. So, there's a dataflow architectural dependency lock-in, despite the fact that it is at least on its face, Postgres compatible. Counterpoint, does that actually matter to customers in what you are seeing?Andi: I think it's a great question. I'll give you a couple of data points. I mean, first of all, even if you take a complete open-source product, right, running them in different clouds, different on-premises environments, and so on, fundamentally, you will have some differences in performance characteristics, availability characteristics, and so on. So, the truth is, even if you use open-source, right, you're not going to get a hundred percent of the same characteristics where you run that. But that said, you still have the freedom of movement, and with I would say and not a huge amount of engineering investment, right, you're going to make sure you can run that workload elsewhere.I kind of think of Spanner in the similar way where yes, I mean, you're going to get all those benefits of Spanner that you can't get anywhere else, like unlimited scale, global consistency, right, no maintenance downtime, five-nines availability, like, you can't really get that anywhere else. That said, not every application necessarily needs it. And you still have that option, right, that if you need to, or want to, or we're not giving you a reasonable price or reasonable price performance, but we're starting to neglect you as a customer—which of course we wouldn't, but let's just say hypothetically, that you know, that could happen—that you still had a way to basically go and run this elsewhere. Now, I'd also want to talk about some of the upsides something like Spanner gives you. Because you talked about, you want to be able to just grab a few things, build something quickly, and then, you know, you don't want to be stuck.The counterpoint to that is with Spanner, you can start really, really small, and then let's say you're a gaming studio, you know, you're building ten titles hoping that one of them is going to take off. So, you can build ten of those, you know, with very minimal spend on Spanner and if one takes off overnight, it's really only the database where you don't have to go and re-architect the application; it's going to scale as big as you need it to. And so, it does enable a lot of this innovation and a lot of cost management as you try to get to that overnight success.Corey: Yeah, overnight success. I always love that approach. It's one of those, “Yeah, I became an overnight success after only ten short years.” It becomes this idea people believe it's in fits and starts, but then you see, I guess, on some level, the other side of it where it's a lot of showing up and doing the work. I have to confess, I didn't do a whole lot of admin work in my production years that touched databases because I have an aura and I'm unlucky, and it turns out that when you blow away some web servers, everyone can laugh and we'll reprovision stateless things.Get too close to the data warehouse, for example, and you don't really have a company left anymore. And of course, in the world of finance that I came out of, transactional integrity is also very much a thing. A question that I had [centers 00:17:51] really around one of the predictions you gave recently at Google Cloud Next, which is your prediction for the future is that transactional and analytical workloads from a database perspective will converge. What's that based on?Andi: You know, I think we're really moving from a world where customers are trying to make real-time decisions, right? If there's model drift from an AI and ML perspective, want to be able to retrain their models as quickly as possible. So, everything is fast moving into streaming. And I think what you're starting to see is, you know, customers don't have that time to wait for analyzing their transactional data. Like in the past, you do a batch job, you know, once a day or once an hour, you know, move the data from your transactional system to analytical system, but that's just not how it is always-on businesses run anymore, and they want to have those real-time insights.So, I do think that what you're going to see is transactional systems more and more building analytical capabilities, analytical systems building, and more transactional, and then ultimately, cloud platform providers like us helping fill that gap and really making data movement seamless across transactional analytical, and even AI and ML workloads. And so, that's an area that I think is a big opportunity. I also think that Google is best positioned to solve that problem.Corey: Forget everything you know about SSH and try Tailscale. Imagine if you didn't need to manage PKI or rotate SSH keys every time someone leaves. That'd be pretty sweet, wouldn't it? With Tailscale SSH, you can do exactly that. Tailscale gives each server and user device a node key to connect to its VPN, and it uses the same node key to authorize and authenticate SSH.Basically you're SSHing the same way you manage access to your app. What's the benefit here? Built-in key rotation, permissions as code, connectivity between any two devices, reduce latency, and there's a lot more, but there's a time limit here. You can also ask users to reauthenticate for that extra bit of security. Sounds expensive?Nope, I wish it were. Tailscale is completely free for personal use on up to 20 devices. To learn more, visit snark.cloud/tailscale. Again, that's snark.cloud/tailscaleCorey: On some level, I've found that, at least in my own work, that once I wind up using a database for something, I'm inclined to try and stuff as many other things into that database as I possibly can just because getting a whole second data store, taking a dependency on it for any given workload tends to be a little bit on the, I guess, challenging side. Easy example of this. I've talked about it previously in various places, but I was talking to one of your colleagues, [Sarah Ellis 00:19:48], who wound up at one point making a joke that I, of course, took way too far. Long story short, I built a Twitter bot on top of Google Cloud Functions that every time the Azure brand account tweets, it simply quote-tweets that translates their tweet into all caps, and then puts a boomer-style statement in front of it if there's room. This account is @cloudboomer.Now, the hard part that I had while doing this is everything stateless works super well. Where do I wind up storing the ID of the last tweet that it saw on his previous run? And I was fourth and inches from just saying, “Well, I'm already using Twitter so why don't we use Twitter as a database?” Because everything's a database if you're either good enough or bad enough at programming. And instead, I decided, okay, we'll try this Firebase thing first.And I don't know if it's Firestore, or Datastore or whatever it's called these days, but once I wrap my head around it incredibly effective, very fast to get up and running, and I feel like I made at least a good decision, for once in my life, involving something touching databases. But it's hard. I feel like I'm consistently drawn toward the thing I'm already using as a default database. I can't shake the feeling that that's the wrong direction.Andi: I don't think it's necessarily wrong. I mean, I think, you know, with Firebase and Firestore, that combination is just extremely easy and quick to build awesome mobile applications. And actually, you can build mobile applications without a middle tier which is probably what attracted you to that. So, we just see, you know, huge amount of developers and applications. We have over 4 million databases in Firestore with just developers building these applications, especially mobile-first applications. So, I think, you know, if you can get your job done and get it done effectively, absolutely stick to them.And by the way, one thing a lot of people don't know about Firestore is it's actually running on Spanner infrastructure, so Firestore has the same five-nines availability, no maintenance downtime, and so on, that has Spanner, and the same kind of ability to scale. So, it's not just that it's quick, it will actually scale as much as you need it to and be as available as you need it to. So, that's on that piece. I think, though, to the same point, you know, there's other databases that we're then trying to make sure kind of also extend their usage beyond what they've traditionally done. So, you know, for example, we announced AlloyDB, which I kind of call it Postgres on steroids, we added analytical capabilities to this transactional database so that as customers do have more data in their transactional database, as opposed to having to go somewhere else to analyze it, they can actually do real-time analytics within that same database and it can actually do up to 100 times faster analytics than open-source Postgres.So, I would say both Firestore and AlloyDB, are kind of good examples of if it works for you, right, we'll also continue to make investments so the amount of use cases you can use these databases for continues to expand over time.Corey: One of the weird things that I noticed just looking around this entire ecosystem of databases—and you've been in this space long enough to, presumably, have seen the same type of evolution—back when I was transiting between different companies a fair bit, sometimes because I was consulting and other times because I'm one of the greatest in the world at getting myself fired from jobs based upon my personality, I found that the default standard was always, “Oh, whatever the database is going to be, it started off as MySQL and then eventually pivots into something else when that starts falling down.” These days, I can't shake the feeling that almost everywhere I look, Postgres is the answer instead. What changed? What did I miss in the ecosystem that's driving that renaissance, for lack of a better term?Andi: That's a great question. And, you know, I have been involved in—I'm going to date myself a bit—but in PHP since 1997, pretty much, and one of the things we kind of did is we build a really good connector to MySQL—and you know, I don't know if you remember, before MySQL, there was MS SQL. So, the MySQL API actually came from MS SQL—and we bundled the MySQL driver with PHP. And so, kind of that LAMP stack really took off. And kind of to your point, you know, the default in the web, right, was like, you're going to start with MySQL because it was super easy to use, just fun to use.By the way, I actually wrote—co-authored—the tab completion in the MySQL client. So like, a lot of these kinds of, you know, fun, simple ways of using MySQL were there, and frankly, was super fast, right? And so, kind of those fast reads and everything, it just was great for web and for content. And at the time, Postgres kind of came across more like a science project. Like the folks who were using Postgres were kind of the outliers, right, you know, the less pragmatic folks.I think, what's changed over the past, how many years has it been now, 25 years—I'm definitely dating myself—is a few things: one, MySQL is still awesome, but it didn't kind of go in the direction of really, kind of, trying to catch up with the legacy proprietary databases on features and functions. Part of that may just be that from a roadmap perspective, that's not where the owner wanted it to go. So, MySQL today is still great, but it didn't go into that direction. In parallel, right, customers wanting to move more to open-source. And so, what they found this, the thing that actually looks and smells more like legacy proprietary databases is actually Postgres, plus you saw an increase of investment in the Postgres ecosystem, also very liberal license.So, you have lots of other databases including commercial ones that have been built off the Postgres core. And so, I think you are today in a place where, for mainstream enterprise, Postgres is it because that is the thing that has all the features that the enterprise customer is used to. MySQL is still very popular, especially in, like, content and web, and mobile applications, but I would say that Postgres has really become kind of that de facto standard API that's replacing the legacy proprietary databases.Corey: I've been on the record way too much as saying, with some justification, that the best database in the world that should be used for everything is Route 53, specifically, TXT records. It's a key-value store and then anyone who's deep enough into DNS or databases generally gets a slightly greenish tinge and feels ill. That is my simultaneous best and worst database. I'm curious as to what your most controversial opinion is about the worst database in the world that you've ever seen.Andi: This is the worst database? Or—Corey: Yeah. What is the worst database that you've ever seen? I know, at some level, since you manage all things database, I'm asking you to pick your least favorite child, but here we are.Andi: Oh, that's a really good question. No, I would say probably the, “Worst database,” double-quotes is just the file system, right? When folks are basically using the file system as regular database. And that can work for, you know, really simple apps, but as apps get more complicated, that's not going to work. So, I've definitely seen some of that.I would say the most awesome database that is also file system-based kind of embedded, I think was actually SQLite, you know? And SQLite is actually still very, very popular. I think it sits on every mobile device pretty much on the planet. So, I actually think it's awesome, but it's, you know, it's on a database server. It's kind of an embedded database, but it's something that I, you know, I've always been pretty excited about. And, you know, their stuff [unintelligible 00:27:43] kind of new, interesting databases emerging that are also embedded, like DuckDB is quite interesting. You know, it's kind of the SQLite for analytics.Corey: We've been using it for a few things around a bill analysis ourselves. It's impressive. I've also got to say, people think that we had something to do with it because we're The Duckbill Group, and it's DuckDB. “Have you done anything with this?” And the answer is always, “Would you trust me with a database? I didn't think so.” So no, it's just a weird coincidence. But I liked that a lot.It's also counterintuitive from where I sit because I'm old enough to remember when Microsoft was teasing the idea of WinFS where they teased a future file system that fundamentally was a database—I believe it's an index or journal for all of that—and I don't believe anything ever came of it. But ugh, that felt like a really weird alternate world we could have lived in.Andi: Yeah. Well, that's a good point. And by the way, you know, if I actually take a step back, right, and I kind of half-jokingly said, you know, file system and obviously, you know, all the popular databases persist on the file system. But if you look at what's different in cloud-first databases, right, like, if you look at legacy proprietary databases, the typical setup is wright to the local disk and then do asynchronous replication with some kind of bounded replication lag to somewhere else, to a different region, or so on. If you actually start to look at what the cloud-first databases look like, they actually write the data in multiple data centers at the same time.And so, kind of joke aside, as you start to think about, “Hey, how do I build the next generation of applications and how do I really make sure I get the resiliency and the durability that the cloud can offer,” it really does take a new architecture. And so, that's where things like, you know, Spanner and Big Table, and kind of, AlloyDB databases are truly architected for the cloud. That's where they actually think very differently about durability and replication, and what it really takes to provide the highest level of availability and durability.Corey: On some level, I think one of the key things for me to realize was that in my own experiments, whenever I wind up doing something that is either for fun or I just want see how it works in what's possible, the scale of what I'm building is always inherently a toy problem. It's like the old line that if it fits in RAM, you don't have a big data problem. And then I'm looking at things these days that are having most of a petabyte's worth of RAM sometimes it's okay, that definition continues to extend and get ridiculous. But I still find that most of what I do in a database context can be done with almost any database. There's no reason for me not to, for example, uses a SQLite file or to use an object store—just there's a little latency, but whatever—or even a text file on disk.The challenge I find is that as you start scaling and growing these things, you start to run into limitations left and right, and only then it's one of those, oh, I should have made different choices or I should have built-in abstractions. But so many of these things comes to nothing; it just feels like extra work. What guidance do you have for people who are trying to figure out how much effort to put in upfront when they're just more or less puttering around to see what comes out of it?Andi: You know, we like to think about ourselves at Google Cloud as really having a unique value proposition that really helps you future-proof your development. You know, if I look at both Spanner and I look at BigQuery, you can actually start with a very, very low cost. And frankly, not every application has to scale. So, you can start at low cost, you can have a small application, but everyone wants two things: one is availability because you don't want your application to be down, and number two is if you have to scale you want to be able to without having to rewrite your application. And so, I think this is where we have a very unique value proposition, both in how we built Spanner and then also how we build BigQuery is that you can actually start small, and for example, on Spanner, you can go from one-tenth of what we call an instance, like, a small instance, that is, you know, under $65 a month, you can go to a petabyte scale OLTP environment with thousands of instances in Spanner, with zero downtime.And so, I think that is really the unique value proposition. We're basically saying you can hold the stick at both ends: you can basically start small and then if that application doesn't need to scale, does need to grow, you're not reengineering your application and you're not taking any downtime for reprovisioning. So, I think that's—if I had to give folks, kind of, advice, I say, “Look, what's done is done. You have workloads on MySQL, Postgres, and so on. That's great.”Like, they're awesome databases, keep on using them. But if you're truly building a new app, and you're hoping that app is going to be successful at some point, whether it's, like you said, all overnight successes take at least ten years, at least you built in on something like Spanner, you don't actually have to think about that anymore or worry about it, right? It will scale when you need it to scale and you're not going to have to take any downtime for it to scale. So, that's how we see a lot of these industries that have these potential spikes, like gaming, retail, also some use cases in financial services, they basically gravitate towards these databases.Corey: I really want to thank you for taking so much time out of your day to talk with me about databases and your perspective on them, especially given my profound level of ignorance around so many of them. If people want to learn more about how you view these things, where's the best place to find you?Andi: Follow me on LinkedIn. I tend to post quite a bit on LinkedIn, I still post a bit on Twitter, but frankly, I've moved more of my activity to LinkedIn now. I find it's—Corey: That is such a good decision. I envy you.Andi: It's a more curated [laugh], you know, audience and so on. And then also, you know, we just had Google Cloud Next. I recorded a session there that kind of talks about database and just some of the things that are new in database-land at Google Cloud. So, that's another thing that if folks more interested to get more information, that may be something that could be appealing to you.Corey: We will, of course, put links to all of this in the [show notes 00:34:03]. Thank you so much for your time. I really appreciate it.Andi: Great. Corey, thanks so much for having me.Corey: Andi Gutmans, VP and GM of Databases at Google Cloud. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry, insulting comment, then I'm going to collect all of those angry, insulting comments and use them as a database.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Stephanie Wong talks with guests Shachar Guz, Inna Weiner, and Gabe Weiss about Google's Database Migration Service and how it helps companies move data to Google Cloud. What typically is a complicated process, DMS simplifies everything from planning to security to validating database migrations. DMS has undergone some changes since last we spoke with Shachar and Gabe. It's gone GA and helped thousands of customers benefit from the service. Migrations are possible from any PostgreSQL database source to AlloyDB for PostgreSQL, which is designed to support HTAP data (transactional and analytical). One of the most exciting updates is the introduction of the DMS modernization journey, which allows customers to change database type during migration (heterogenous). In addition, migrations with DMS can be set up to continuously replicate data between the old and new database. With this feature, developers can compare the application performance against the old vs. new database. Inna talks about the benefits of keeping your data in the cloud, like secure, reliable, and scalable data storage. Google Cloud takes care of the maintenance work for you as well. DMS takes security seriously and supports multiple security methods to keep your data safe as it migrates. We talk about the different customers using DMS and how the process works for homogeneous and heterogeneous migrations. Before you even start, Gabe tells us, DMS helps you prepare for the migration. And tools like Dataflow can help when customers decide full migration would be too difficult. We talk about the difference between Datastream and DMS and use cases for each. We wrap up the show with a look at the future of DMS. Shachar Guz Shachar is a product manager at Google Cloud, he works on the Cloud Database Migration Service. Shachar worked in various product and engineering roles and shares a true passion about data and helping customers get the most out of their data. Shachar is passionate about building products that make cumbersome processes simple and straightforward and helping companies adopt Cloud technologies to accelerate their business. Inna Weiner Inna is a senior technical leader with 20+ years of global experience. She is a big data expert, specializing in deriving insights from data, product and user analytics. Currently, she leads engineering for Cloud DMS. Inna enjoys building diverse engineering organizations, with common vision, growth strategy and inclusive culture. Gabe Weiss Gabe leads the database advocacy team for the Google Cloud Platform team ensuring that developers can make awesome things, both inside and outside of Google. That could mean speaking at conferences, writing example code, running bootcamps, writing technical blogs or just doing some hand holding. Prior to Google he's worked in virtual reality production and distribution, source control, the games industry and professional acting. Cool things of the week Flexible committed use discounts — a simple new way to discount Compute Engine instances blog Understanding transactional locking in Cloud Spanner blog Interactive In-console Tutorial site Interview Database Migration Service site GCP Podcast Episode 262: Database Migration Service with Shachar Guz and Gabe Weiss podcast AlloyDB for PostgreSQL site PostgreSQL site Datastream site Dataflow site CloudSQL site Spanner site What's something cool you're working on? Gabe has been tinkering with new Google Cloud databases and managing a new team. Hosts Stephanie Wong

On the show this week, we're talking updated DevOps practices for 2022 with hosts Stephanie Wong and Chloe Condon and our guests Nathen Harvey and Derek DeBellis. Nathen and Derek start the show with a thorough discussion of DORA, the research program dedicated to helping organizations improve software delivery and operations, and the state of DevOps report that Google publishes every year. This year, the DevOps research team strengthened their focus on security and discovered that one of the biggest predictors in security practice adoption is company culture. Open, communicative, and trustful company cultures are some of the best for accepting and implementing optimized security practices. Derek tells us how company cultures are measured and scored for this purpose and Nathen talks about team and individual burnout and its affects on culture. Low, medium, high, and elite teams are another indicator of culture, and Nathen explains how teams earn their label through four keys of software delivery performance. Each year, they let the data show these four clusters of team performance. But this year there were only three, and Derek talks more about this phenomenon and why the elite cluster seems to have disappeared. When operational performance analysis was added, the four clusters reemerged and were renamed to better suit the new analysis metrics. Nathen details these four new clusters: starting, which performs neither well nor poorly and may be just starting out; flowing, teams that are performing well across throughput, stability, and operational performance; slowing teams, which don't have high throughput but excel in other areas; and retiring teams, which are reliable but not actively developing projects. We discuss how companies may shift from one cluster to another and how much context can affect this shift. We talk about key findings in the 2022 DevOps report, especially in the security space. Some of the most notable include the adoption of DevOps security practices and the decreased incidence of burnout on teams who leverage security practices. Nathen and Derek elaborate on how this year's research changed from last year and what remained the same. Nathen Harvey Nathen works with teams helping them learn about and apply the findings of our research into high performing teams. He's been involved in the DevOps community for more than a decade. Derek DeBellis Derek is a Quantitative User Experience Researcher at Google, where Derek focuses on survey research, logs analysis, and figuring out ways to measure concepts central to product development. Derek has published on Human-AI interaction, the impact of Covid-19's onset on smoking cessation, designing for NLP errors and the role of UX in ensuring privacy. Cool things of the week Try out Cloud Spanner databases at no cost with new free trial instances blog Chipotle Is Testing More Artificial Intelligence Solutions To Improve Operations article Gyfted uses Google Cloud AI/ML tools to match tech workers with the best jobs blog Interview 2022 Accelerate State of DevOps Report blog DevOps site 2022 State of the DevOps Report Report site DORA site DORA Community site SLSA site Security Software Development Framework site Westrum organizational culture site Google finds culture, not tech, is the biggest predictor of DevOps security outcomes article GCP Podcast Episode 205: DevOps with Nathen Harvey and Jez Humble podcast GCP Podcast Episode 284: State of DevOps Report 2021 with Nathen Harvey and Dustin Smith podcast GCP Podcast Episode 290: Resiliency at Shopify with Camilo Lopez and Tai Dickerson podcast What's something cool you're working on? Steph is working on talks for DevFest Nantes and a Google Cloud dev conference in London. She'll be talking about subsea fiber optics and Google Cloud networking products. Chloe is a Noogler, so she's been working on learning as much as she can! She is excited to make her podcast debut this week! Hosts Stephanie Wong and Chloe Condon

On The Cloud Pod this week, AWS Enterprise Support adds incident detection and response, the announcement of Google Cloud Spanner, and Oracle expands to Spain. Thank you to our sponsor, Foghorn Consulting, which provides top notch cloud and DevOps engineers to the world's most innovative companies. Initiatives stalled because you're having trouble hiring? Foghorn can be burning down your DevOps and Cloud backlogs as soon as next week. Episode Highlights ⏰ AWS Enterprise Support adds incident detection and response ⏰ You can now get a 90-day free trial of Google Cloud Spanner ⏰ Oracle opens its newest cloud infrastructure region in Spain Top Quote

On The Cloud Pod this week, the team discusses data sovereignty for future space-customers. Plus: There's a global cloud shortage, Google announces Apigee advanced API security, and GKE Autopilot gets new networking features. A big thanks to this week's sponsor, Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights

On The Cloud Pod this week, the team discusses the new Madrid region's midday siesta shutdown. Plus: Broadcom acquires VMWare for $61 billion, Azure gets paradigmatic with 5G, and you can now take the 2022 Google-DORA DevOps survey. A big thanks to this week's sponsor, Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights

This week, we're busting myths around Cloud Spanner with our guests Pritam Shah and Vaibhav Govil. Mark Mirchandani and Max Saltonstall host this episode and learn about the fantastic capabilities of Cloud Spanner. Our guests give us a quick run-down of Spanner database software and its fully-managed offerings. Spanner's unique take on the relational database has sparked some myths. We start by addressing cost and the idea that Spanner is expensive. With its high availability achieved through synchronously replicating data, failures are virtually a non-issue, making the cost well worth it. Our guests describe other features that add to the value of Spanner as well. Workloads of any size are a good fit for Spanner because of its scalability and pricing based on use. Despite rumors, Spanner is now very easy to start using. New additions like the PostgreSQL interface and ORM support have made the usability of Spanner much more familiar. Regional and multi-regional instances are supported, busting the myth that Spanner is only good for global workloads. Our guests offer examples of projects using local and global configurations with Spanner. In the database world, Vaibhav sees trends like the convergence of non-relational and relational databases as well as convergence in the OLTP and OLAP database semantics, and he tells us how Spanner is adapting and growing with these trends. Pritam points out that customers are paying more attention to total cost of ownership, the importance of scalable and reliable database solutions, and the peace of mind that comes with a managed database system. Spanner helps customers with these, freeing up business resources for other things. This year, Spanner has made many announcements about new capabilities coming soon, like PostgreSQL interface on spanner GA, Query Insights visualization tools, cross-regional backups GA, and more. We hear all about these awesome updates. Pritam Shah Pritam is the Director of Engineering for Cloud Spanner. He has been with Google for about four and a half years. Before Spanner, he was the Engineering Lead for observability libraries at Google. That included Distributed Tracing and Metrics at Google scale. His mission was to democratize the instrumentation libraries. That is when he launched Open Census and then took on Cloud Spanner. Vaibhav Govil Vaibhav is the Product lead for Spanner. He has been in this role for the past three years, and before this he was a Product Manager in Google Cloud Storage in Google. Overall, he has spent close to four years at Google, and it has been a great experience. Cool things of the week Our plans to invest $9.5 billion in the U.S. in 2022 blog A policy roadmap for 24⁄7 carbon-free energy blog SRE Prodcast site Meet the people of Google Cloud: Grace Mollison, solutions architect and professional problem solver blog GCP Podcast Episode 224: Solutions Engineering with Grace Mollison and Ann Wallace podcast Interview Spanner site Cloud Spanner myths busted blog PostgreSQL interface docs Cloud Spanner Ecosystem site Spanner: Google's Globally-Distributed Database white paper Spanner Docs docs Spanner Qwiklabs site Using the Cloud Spanner Emulator docs GCP Podcast Episode 62: Cloud Spanner with Deepti Srivastava podcast GCP Podcast Episode 248: Cloud Spanner Revisited with Dilraj Kaur and Christoph Bussler podcast Cloud Spanner federated queries docs What's something cool you're working on? Max is working on a new podcast platform and some spring break projects. Hosts Mark Mirchandani and Max Saltonstall

Original blog post More articles at cloud.google.com/blog

On The Cloud Pod this week, the team discusses Peter's concept of fun. Plus digital adventures with AWS Cloud Quest game, much-wanted Google price increases, and a labyrinthine run-through of the details of Azure Health Data Services. A big thanks to this week's sponsor, Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights

On The Cloud Pod this week, the team reminisces about dealing with awful database technologies, which Ryan luckily managed to avoid. Plus all things cybersecurity as Linux gets hit with a huge security emergency, Google acquires Mandiant for $5.4 billion, and Orca Security catches a major Azure cross-tenant vulnerability.  A big thanks to this week's sponsor, Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights

First time co-host Jan Kleinert joins Mark Mirchandani this week to talk about database observability and the cool tools that make it possible. Morgan McLean and Nimesh Bhagat describe database observability, which uses metrics, logs, and other tools to help users understand the health of your database. We talk about Object Relational Mappers and the challenges with using these for debugging database performance. SQL Commenter helps database observability in two ways: it is both a library and a standard, Nimesh tells us. He describes the process for us, detailing exactly how SQL Commenter effects projects. Recently, SQL Commenter was donated to OpenTelemetry to augment the observability offerings, create an application standard, and make it easier for developers to use a variety of different tools and languages. Engineers can get end-to-end traces no matter which database technologies they use. Morgan tells us about Splunk and how information from SQL Commenter is taken into Splunk and used. Backend data like metrics from Cloud Monitoring and client libraries can be correlated together with SQL Commenter and brought into Splunk for full stack observability. Nimesh offers client examples to help us understand how these useful tools integrate for optimal observability. He tells us about the databases and ORMs supported by SQL Commenter. Our guests and co-host Jan give tips to help our listeners get started with SQL Commenter and talk about what they're looking forward to in the future of observability. Nimesh Bhagat Nimesh is a product manager at Google Cloud, he leads Database Observability. He has worked across engineering and product roles, building highly available and high performance enterprise infrastructure used by Fortune 500 companies. His passion lies in combining powerful infrastructure with simple user experience so that every business and developer can build software at scale and velocity. Morgan McLean Morgan is ​​Director of Product Management at Splunk and co-creator of OpenCensus / OpenTelemetry. Cool things of the week Google Cloud Innovators site Redesigning the Cloud SDK + CLI for easier development blog GCP Podcast Episode 291: Redesigning the Cloud SDK and CLI with Wael Manasra and Cody Oss podcast What is Active Assist? video GCP Podcast Episode 235: Active Assist with Chris Law + MariaDB SkySQL with Robert Hedgepeth podcast Interview SQL Commenter site Sequelize site SQL Alchemy site ADO.net site GCP Podcast Episode 247: Cloud SQL Insights with Nimesh Bhagat podcast OpenTelemetry site Splunk site Cloud Monitoring site Cloud Spanner site Cloud SQL site Cloud Trace site Sqlcommenter now extending the vision of OpenTelemetry to databases blog Hosts Mark Mirchandani and Jan Kleinert

On The Cloud Pod this week, order in the court! Plus tackling those notorious latency issues with AWS Local Zones, things get quick and rusty with AWS s2n-quic, and GCP flexes with Dataplex data mesh. A big thanks to this week's sponsor, Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights

About YiftachYiftach is an experienced technologist, having held leadership engineering and product roles in diverse fields from application acceleration, cloud computing and software-as-a-service (SaaS), to broadband networks and metro networks. He was the founder, president and CTO of Crescendo Networks (acquired by F5, NASDAQ:FFIV), the vice president of software development at Native Networks (acquired by Alcatel, NASDAQ: ALU) and part of the founding team at ECI Telecom broadband division, where he served as vice president of software engineering.Yiftach holds a Bachelor of Science in Mathematics and Computer Science and has completed studies for Master of Science in Computer Science at Tel-Aviv University.Links: Redis, Inc.: https://redis.com/ Redis open source project: https://redis.io LinkedIn: https://www.linkedin.com/in/yiftachshoolman/ Twitter: https://twitter.com/yiftachsh TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Rising Cloud, which I hadn't heard of before, but they're doing something vaguely interesting here. They are using AI, which is usually where my eyes glaze over and I lose attention, but they're using it to help developers be more efficient by reducing repetitive tasks. So, the idea being that you can run stateless things without having to worry about scaling, placement, et cetera, and the rest. They claim significant cost savings, and they're able to wind up taking what you're running as it is, in AWS, with no changes, and run it inside of their data centers that span multiple regions. I'm somewhat skeptical, but their customers seem to really like them, so that's one of those areas where I really have a hard time being too snarky about it because when you solve a customer's problem, and they get out there in public and say, “We're solving a problem,” it's very hard to snark about that. Multus Medical, Construx.ai, and Stax have seen significant results by using them, and it's worth exploring. So, if you're looking for a smarter, faster, cheaper alternative to EC2, Lambda, or batch, consider checking them out. Visit risingcloud.com/benefits. That's risingcloud.com/benefits, and be sure to tell them that I said you because watching people wince when you mention my name is one of the guilty pleasures of listening to this podcast.Corey: This episode is sponsored in part by our friends at Sysdig. Sysdig is the solution for securing DevOps. They have a blog post that went up recently about how an insecure AWS Lambda function could be used as a pivot point to get access into your environment. They've also gone deep in-depth with a bunch of other approaches to how DevOps and security are inextricably linked. To learn more, visit sysdig.com and tell them I sent you. That's S-Y-S-D-I-G dot com. My thanks to them for their continued support of this ridiculous nonsense.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. This promoted episode is brought to us by a company that I would have had to introduce differently until toward the end of last year. Today, they're Redis, but for a while they've been Redis Labs, here to talk with me about that and oh, so much more is their co-founder and CT, Yiftach Shoolman. Yiftach, thank you for joining me.Yiftach: Hi, Corey. Nice to be a guest of you. This is a very interesting podcast, and I often happen to hear it.Corey: I'm always surprised when people tell me that they listen to this because unlike a newsletter or being obnoxious on Twitter, I don't wind up getting a whole lot of feedback from people via email or whatnot. My operating theory has been that it's like a—when I send an email out, people will get that, “Oh, an email. I know how to send one of those.” And they'll fire something back. But podcasts are almost like a radio show, and who calls into radio shows? Well, lunatics, generally, and if I give feedback, I'll feel like a lunatic.So, I get very little email response on stuff like this. But when I talk to people, they mention the show. It's, “Oh, right. Good. I did remember to turn the microphone on. People are out there listening.” Thank you.So you, back in August of 2021, the company that formerly known as Redis Labs, became known as Redis. What caused the name change? And sure, is a small change as opposed to, you know, completely rebranding a company like Square to Block, but what was it that really drove that, I guess, rebrand?Yiftach: Yeah, a great question. And by way, if you look at our history, we started the company under the name of Garantia Data, which is a terrible name. [laugh]. And initially, what we wanted to do is to accelerate databases with both technologies like memcached, and Redis. Eventually, we built a solution for both, and we found out that Redis is much more used by people. That was back in 2011.So, in 2021, we finally decided to say let's unify the brand because, you know, as a contributors to Redis from day one, and creator of Redis is also part of the company, Salvatore Sanfilippo. We believed that we should not confuse the market with multiple messages about Redis. Redis is not just the cache and we don't want people to definitely interpret this. Redis is more than a cache, it's actually, if you look at our customer, like, 66% of them are using it as a real-time database. And we wanted to unify everyone around this naming to avoid different interpretation. So, that was the motivation, maybe.Corey: It's interesting you talk about, I guess, the evolution of the use cases for Redis. Back in 2011, I was using Redis in an AWS environment, and, “Ah, disk persistence, we're going to turn that on.” And it didn't go so well back in those days because I found that the entire web app that we were using would periodically just slam to a halt for about three seconds whenever Redis wound up doing its disk persistent stuff, diving in far deeper than I really had any right to be doing, I figured out this was a regression in the Xen hypervisor and Xen kernel that AWS was using back then around the fork call. Not Redis's fault, to be very clear. But I looked at this and figured, “Ah. I know how to fix this.”And that's right. We badgered AWS into migrating to Nitro years later and not using Xen anymore, and that solve that particular problem. But this was early on in my technical career. It sort of led to the impression of, “Oh, Redis. That's a cache, I should never try and use it as anything approaching a database.” Today, that guidance no longer holds, you are positioning yourself as a data platform. What did that dawning awareness look like? How did you get to where you are from where Redis was once envisioned in the industry: Primarily as a cache?Yiftach: Yeah, very good question. So, I think we should look at this problem from the application perspective, or from the user perspective. Sounds like a marketing term, but we all know we are in the age of real-time. Like, you expect everything to be instantly. You don't want to wait, no one wants to wait, especially after Covid and everything's that brought to the you know, online services.And the expectation today from a real-time application is to be able to reply less than 100 milliseconds in order to feel the real-time. When I say 100 milliseconds, from the time you click the button until you get the first byte of the response. Now, if you do the math, you can see that, like, 50% of this goes to the network and 50% of this goes to the data center. And inside the data center, in order to complete the transaction in less than 50 milliseconds, you need a database that replies in no time, like, less than a millisecond. And today, I must say, only Redis can guarantee that.If you use Redis as a cache, every transaction—or there is a potential at least—that not all the information will be in Redis when the transaction is happening and you need to bring it probably from the main database, and you need to processing it, and you need to update Redis about it. And this takes a while. And eventually, it will help the end-user experience. And just to mention, if you look at our support tickets, like, I would say the majority of them is, why Redis replies—why Redis latency grew from 0.25 millisecond to 0.5 millisecond because there is a multiplier effect for the end-user. So, I hoping I managed to answer what are the new challenges that we see today in the market.Corey: Tell me a little bit more about the need for latency around things like that. Because as we look at modern web apps across the board, people are often accessing them through mobile devices, which, you know, we look at this spinning circle of regret as it winds up loading a site or whatnot, it takes a few seconds. So, the idea of oh, that the database call has to complete in millisecond or less time seems a little odd viewed purely from a perspective of, “Really? Because I spent a lot of time waiting for the webpage to load.” What drives that latency requirement?Yiftach: First of all, I agree with you. A lot of time, you know, application were not built for it then. This is why I think we still have an opportunity to improve existing application. But for those applications that weren't built for real-time, for instance, in the gaming space, it is clear that if you delay your reaction for your avatar, in more than two frame, like, I mean, 60 millisecond, the experience is very bad, and customers are not happy with this. Or, in transaction scoring example, when you swipe the card, you want the card issuer to approve or not approve it immediately. You don't want to wait. [unintelligible 00:07:19] is another example.But in addition to that there are systems like mobility as a service, like the Ubers of the world, or the Airbnb of the world. Or any e-commerce site. In order to be able to reply in second, they need to process behind the scene, thousand, and sometime millions of operations per second in order to get to the right decision. Yeah? You need to match between riders and drivers. Yeah, and you need to match between guests and free room in the hotel. And you need to see that the inventory is up-to-date with the shoppers.And all these takes a lot of transactions and a lot of processing behind the scene in order just to reply in second in a consistent manner. And this is why that this is useful in all these application. And by the way, just a note, you know, we recently look at how many operations per second actually happening in our cloud environment, and I must tell you that I was surprised to see that we have over one thousand clusters or databases with the speed of between 1 million to 10 million operation per second. And over 150 databases with over 10 million operations per second, which is huge. And if you ask yourself how come, this is exactly the reason. This is exactly the reason. For every user interaction, usually you need to do a lot of interaction with your data.Corey: That kind of transaction volume, it would never occur to me to try and get that on a single database. It would, “All right, let's talk about sharding for days and trying to break things out.” But it's odd because a lot of the constraints that I was used to in my formative years, back when I was building software—badly—are very much no longer the case. The orders of magnitude are different. And things that used to require incredibly expensive, dedicated hardware now just require, “Oh yeah, you can click the button and get one of those things in the cloud, and it's dirt cheap.”And it's been a very strange journey. Speaking of clicking buttons, and getting things available in the cloud, Redis has been a thing, and its rise has almost perfectly tracked the rise of the cloud itself. There's of course the Redis open-source project, which has been around for ages and is what you're based on top of. And then obviously AWS wind up launching—“Ah, we're going to wind up ‘collaborating'”—and the quotes should be visible from orbit on that—“With Redis by launching ElasticCache for Redis.” And they say, “Oh, no, no, it's not competition. It's validating your market.”And then last year, they looked at you folks again, like, “Ah, we're launching a second service: MemoryDB in this case.” It's like Redis, except bad. And I look at this, and I figure what is their story this time? It's like, “Oh, we're going to validate the shit out of your market now.” It's, on some level, you should be flattered having multiple services launched trying to compete slash offer the same types of things.Yet Redis is not losing money year-over-year. By all accounts, you folks are absolutely killing it in the market. What is it like to work both in cloud environments and with the cloud vendors themselves?Yiftach: This is a very good question. And we use the term frenemy, like, they're our friend, but sometimes they are our enemy. We try to collaborate and compete them fairly. And, you know, AWS is just one example. I think that the other cloud took a different approach.Like with GCP, we are fully integrated in the console, what is called, “Third-party first-class service.” You click the button through the GCP console and then you're redirected to our cloud, Redis Enterprise cloud. With Azure even, we took a one step further and we provide a fully integrated solution, which is managed by Azure, Azure Cache for Redis, and we are the enterprise tier. But we are also cooperating with AWS. We cooperating on the marketplace, and we cooperate in other activities, including the open-source itself.Now, to tell you that we do not have, you know, a competition in the market, the competition is good. And I think MemoryDB is a validation of your first question, like, how can you use Redis [more than occasion 00:11:33], and I encourage users to test the differences between these services and to decide what fits to their use case. I promise you my perspective, at least, that we provide a better solution. We believe that any real-time use case should eventually be served by Redis, and you don't need to create another database for that, and you don't need to create another caching layer; you have everything in a single data platform, including all the popular models, data models, not only key-value, but also JSON, and search, and graph, and time-series… and probably AI, and vector embedding, and other stuff. And this is our approach.Corey: Now, I know it's unpopular in AWS circles to point this out, but I have it on good authority that they are not the only large-scale cloud provider on the planet. And in fact, if I go to the Google Cloud Console, they will sell me Redis as well, but it's through a partner affinity as a first-party offering in the console called Redis Enterprise. And it just seems like a very different interaction model, as in, their approach is they're going to build their own databases that solve for a wide variety of problems, some of them common and some of them ridiculous, but if you want actual Redis or something that looks like Redis, their solution is, “Oh, well, why don't we just give you Redis directly, instead of making a crappy store-brand equivalent of Redis?” It just seems like a very different go to market approach. Have you seen significant uptake of Redis as a product, through partnering with Google Cloud in that way?Yiftach: I would do answer this politely and say that I can no more say that the big cloud momentum is only on AWS. [laugh]. We see a lot of momentum in other clouds in terms of growth. And I would challenge the AWS guys to think differently about partnership with ISV. I'm not saying that they're not partnering with us, but I think the partnerships that we have with other clouds are more… closer. Yeah. It's like there is less friction. And it's up to them, you know? It's up to any cloud vendor to decide the approach they wants to take in this market. And it's good.Corey: It's a common refrain that I hear is that AWS is where we see the eight-hundred-pound gorilla in the space, it's very hard to deny that. But it also has been very tricky to wind up working with them in a partnership sense. Partnering is not really a language that Amazon speaks super well, kind of like, you know, toddlers and sharing. Because today, they aren't competing directly with you, but what about tomorrow? And it's such a large distributed company that in many cases, your account manager or your partner manager there doesn't realize that they're launching a competitor until 12 hours before it launches. And that's—yeah, feels great. It just feels very odd.That said, you are a partner with AWS and you are seeing significant adoption via the AWS Marketplace, and the reason I know that is because I see it in my own customer accounts on the consulting side, I'm starting to see more and more spend via the marketplace, partially due to offset spend commitments that they've made contractually with AWS, but also, privately I tend to believe a lot of that is also an end-run around their own internal procurement department, who, “Oh, you want some Redis. Great. Give me nine months, and then find three other vendors to give me competitive bids on it.” And yeah, that's not how the world works anymore. Have you found that the marketplace is a discovery mechanism for customers to come to Redis, or are they mostly going into the marketplace saying, “I need Redis. I want it to be Redis Enterprise, from Redis, but this is the way I'm going to procure it.”Yiftach: My [unintelligible 00:15:17], you know, there are people that are seeing differently, that marketplace is how to be discovered through the marketplace. I still see it, I still see it as a billing mechanism for us, right? I mean, AWS helping us in sell. I mean, their sell are also sell partner and we have quite a few deals with them. And this mechanism works very nicely, I must say.And I know that all the marketplaces are trying to change it, for years. That customer whenever they look at something, they will go through the marketplace and find it there, but it's hard for us to see the momentum there. First of all, we don't have the metrics on the marketplace; we cannot say it works, it doesn't works. What we do see that works is that when we own the customer and when the customer is ascertaining how to pay, through the credit card or through the wire, they usually prefer to pay through the commit from the cloud, whether it is AWS, GCP, or Azure. And for that, we help them to do the transaction seamlessly.So, for me, the marketplace, the number one reason for that is to use your existing commit with the cloud provider and to pay for ourselves. That said, I must say that [with disregard 00:16:33] [laugh] AWS should improve something because not the entire deal is committed. It's like 50% or 60%, don't remember the exact number. But in other clouds when ISVs are interacting with them, the entire deal is credited for the commit, which is a big difference.Corey: I do point out, this is an increasing trend that I'm seeing across the board. For those who are unaware, when you have a large-scale commitment to spend a certain dollar amount per year on AWS Marketplace spend counts at a 50% rate. So, 50 cents of every dollar you spend to the marketplace counts toward your commit. And once upon a time, this was something that was advertised by AWS enterprise sales teams, as, “Ah. This is a benefit.”And they're talking about moving things over that at the time are great, you can move that $10,000 a year thing there. And it's, “You have a $50 million annual commit. You're getting awfully excited about knocking $5,000 off of it.” Now, as we see that pattern starting to gain momentum, we're talking millions a year toward a commit, and that is the game changer that they were talking about. It just looks ridiculous at the smaller-scale.Yiftach: Yeah. I agree. I agree. But anyway, I think this initiative—and I'm sure that AWS will change it one day because the other cloud, they decided not to play this game. They decided to give the entire—you know, whatever you pay for ISVs, it will be credited with your commit.Corey: We're all biased by our own experiences, so I have a certain point of view based upon what I see in my customer profile, but my customers don't necessarily look like the bulk of your customers. Your website talks a lot about Redis being available in all cloud providers, in on-prem environments, the hybrid and multi-cloud stories. Do you see significant uptake of workloads that span multiple clouds, or are they individual workloads that are on multiple providers? Like for example Workload A lives on Azure, Workload B lives on GCP? Or is it just Workload A is splattered across all four cloud providers?Yiftach: Did the majority of the workloads is splitted between application and each of them use different cloud. But we started to see more and more use cases in which you want to use the same data sets across cloud, and between hybrid and cloud, and we provide this solution as well. I don't want to promote myself so much because you worried me at the beginning, but we create these products that is called Active-Active Redis that is based on CRDT, Conflict-free Replicated Data Type. But in a nutshell, it allows you to run across multiple clouds, or multiple region in the same cloud, or hybrid environment with the speed the of Redis while guaranteeing that eventually all your rights will be converged to the same value, and while maintaining the speed of Redis. So, I would say quite a few customers have found it very attractive for them, and very easy to migrate between clouds or between hybrid to the cloud because in this approach of Active-Active, you don't need the single cut-off.A single cut-off is very complex process when you want to move a workload from one cloud to another. Think about it, it is not only data; you want to make sure that the whole entire application works. It never works in one shot and you need to return back, and if you don't have the data with you, you're stuck. So, that mechanism really helps. But the bigger picture, like you mentioned, we see a lot of [unintelligible 00:20:12] distribution need, like, to maintain the five nines availability and to be closer to the user to guarantee the real-time. Send dataset deployment across multiple clouds, and I agree, we see a growth there, but it is still not the mainstream, I would say.Corey: I think that my position on multi-cloud has been misconstrued in a variety of corners, which is doubtless my fault for failing to explain it properly. My belief has been when you're building something on day-one, greenfield pickup provider—I don't care which one—go all in. But I also am not a big fan of potentially closing off strategic or theoretical changes down the road. And if you're picking, let's say, DynamoDB, or Cloud Spanner, or Cosmos DB, and that is the core of your application, moving a workload from Cloud A to Cloud B is already very hard. If you have to redo the entire interface model for how it talks to his data store and the expectations built into that over a number of years, it becomes basically impossible.So, I'm a believer in going all-in but only to a certain point, in some cases, and for some workloads. I mean, I done a lot of work with DynamoDB, myself for my newsletter production pipeline, just because if I can't use AWS anymore, I don't really need to write Last Week in AWS. I have a hard time envisioning a scenario in which I need to go cross-cloud but still talk about the existing thing. But my use case is not other folks' use case. So, I'm a big believer in the theoretical exodus, just because not doing that in many corporate environments becomes a lot less defensible. And Redis seems like a way to go in that direction.Yiftach: Yeah. Totally with you. I think that this is a very important—and by the way, it is not… to say that multi-cloud is wrong, but it allows you to migrate workload from one cloud to another, once you decide to do it. And it's put you in a position as a consumer—no one wants—why no one likes [unintelligible 00:22:14]. You know, because of the pricing model [laugh], okay, right?You don't want to repeat this story, again with AWS, and with any of them. So, you want to provide enough choices, and in order to do that, you need to build your application on infrastructures that can be migrated from one cloud to another and will not be, you know, reliant on single cloud database that no one else has, I think it's clear.Corey: This episode is sponsored in part by our friends at Vultr. Spelled V-U-L-T-R because they're all about helping save money, including on things like, you know, vowels. So, what they do is they are a cloud provider that provides surprisingly high performance cloud compute at a price that—while sure they claim its better than AWS pricing—and when they say that they mean it is less money. Sure, I don't dispute that but what I find interesting is that it's predictable. They tell you in advance on a monthly basis what it's going to going to cost. They have a bunch of advanced networking features. They have nineteen global locations and scale things elastically. Not to be confused with openly, because apparently elastic and open can mean the same thing sometimes. They have had over a million users. Deployments take less that sixty seconds across twelve pre-selected operating systems. Or, if you're one of those nutters like me, you can bring your own ISO and install basically any operating system you want. Starting with pricing as low as $2.50 a month for Vultr cloud compute they have plans for developers and businesses of all sizes, except maybe Amazon, who stubbornly insists on having something to scale all on their own. Try Vultr today for free by visiting: vultr.com/screaming, and you'll receive a $100 in credit. Thats v-u-l-t-r.com slash screaming.Corey: Well, going greenfield story of building something out today, “I'm going to go back to my desk after this and go ahead and start building out a new application.” And great, I want to use Redis because this has been a great conversation, and it's been super compelling. I am probably not going to go to redis.com and sign up for an enterprise Redis agreement to start building out.It's much likelier that I would start down the open-source path because it turns out that I believe ‘docker pull redis' is pretty close to—or ‘docker run redis latest' or whatever it is, or however it is you want to get Redis—I have no judgment here—is going to get you the open-source tool super well. What is the nature of your relationship between the open-source Redis and the enterprise Redis that runs on money?Yiftach: So, first of all, we are, like, the number one contributor to the Redis open-source. So, I would say 98% of the code of Redis contributed by our team. Including the creator of Redis, Salvatore Sanfilippo, was part of our team. Salvatore has stepped back in, like—when was it? Like, one-and-a-half, almost two years ago because the project became, like, a monster, and he said, “Listen, this is too much. I worked, like, 10 years or 11 years. I want to rest a bit.”And the way we built the core team around Redis, we said we will allocate three people from the company according to their contribution. So, the leaders—the number two after Salvatore in terms of contribution, I mean, significant contribution, not typo and stuff [laugh] like this. And we also decided to make it, like, a community-driven project, and we invited people from other places, including AWS, Madelyn, and Zhao Zhao from Alibaba.And this is based on the past contribution to Redis, not because they are from famous cloud providers. And I think it works very well. We have a committee which is driven by consensus, and this is how we agree what we put in the open-source and what we do not. But in addition to the pure open-source, we also invested a lot in what we call Source Available. Source Available is a new approach that, I think, we were the first who started it, back in 2018, when we wanted to have a mechanism to be able to monetize the company.And what we did by then, we added all the modules which are extensions to the latest open-source that allow you to do the model, like JSON and search and graph and time series and AI and many others with Redis under the Source Available license. That mean you can use it like BSD; you can change everything, without copyleft, you don't need to contribute back. But there is one restriction. You cannot create a service or a product that compete directly with us. And so far, it works very well, and you can launch Docker containers with search, and with JSON—or with all the modules combined; we also having this—and get the experience from day zero.We also made sure that all your clients are now working very well with these modules, and we even created the object mapping client for each of the major language. So, we can easily integrate it with Spring, in Django, and Node.js platform, et cetera. This is called when OM .NET, OM Java, OM Node.js, OM Python, et cetera, very nicely. You don't need to know all the commands associated. We just speak [unintelligible 00:26:22] level with Redis and get all the functionality.Corey: It's a hard problem to solve for, and whenever we start talking about license changes for things like this, it becomes a passionate conversation. People with all sorts of different perspectives and assumptions baked in—and a remembrance of yesteryear—all have different thoughts on coulda, woulda, shoulda, et cetera, et cetera. But let's be very clear, running a business is hard. And building a business on top of an open-source model is way harder. Now, if we were launching an open-source company today in 2022, there are different things we would do; we would look at it very differently. But over a decade ago, it didn't work that way. If you were to be looking at how open-source companies can continue to thrive in a time of cloud, what guidance do you have for him?Yiftach: This is a great question, and I must say that the every month or every few weeks, I have a discussion with a new team of founders that want to create an open-source, and they asked me what is my opinion here. And I would say, today, that we and other ISV, we built a system for you to decide what you want to freely open-source, and take into account that if this goes very well, the cloud provider will pick it up and will make a service out of it. Because this is the way they work. And the way for you to protect yourself is to have different types of licenses, like we did. Like you can decide about Source Available and restrict it to the minimum.By the way, I think that Source Available is much better than AGPL with the copyleft and everything that it's provide. So, AGPL is a pure open-source, but it has so many other implications that people just don't want to touch it. So, it's available, you can do whatever you want, you just cannot create a competing product. And of course, if there are some code that you want to close, use closed-source. So, I would say think very seriously about your licensing model. This is my advice. It's not to say that open-source is not great. I truly believe that it helps you to get the adoption; there are a lot of other benefits that open-source creates.Corey: Historically, it feels that open-source was one of those things that people wanted the upside of the community, and the adoption, and getting people to work. Especially on a shoestring budget, and people can go in and fix these things. Like, that's the naive approach of, “Oh, it just because we get a whole bunch of free, unpaid labor.” Okay, fine, whatever. It also engenders trust and lets people iterate rapidly and build things to suit their use cases, and you learn so much more about the use cases as you continue to grow.But on the other side of it, there's always the Docker problem where they gave away the thing that added stupendous value. If they hadn't gone open-source with Docker, it never would have gotten the adoption that it did, but by going open-source, they wound up, effectively, being forced more or less than to say, “Okay, we're going to give away this awesome thing and then sell services around it.” And that's not really a venture-scaled business, in most cases. It's a hard market.Yiftach: And the [gate 00:29:26] should never be the cloud. Because people, like you mentioned, people doesn't start with the cloud. They start to develop with on the laptop or somewhere with Docker or whatever. And this is where Source Available can shine because it allows you to do the same thing like open-source—and be very clear, please do not confuse your user. Tells them that this is Source Available; they should know in advance, so they will be not surprise later on when they move to the production stage.Then if they have some question, legal questions, for Redis, we're able to answer, yeah. And if they don't, they need to deal with the implication of this. And so far, we found it suitable to most of the users. Of course, there will be always open-source gurus.Corey: If there's one thing people have on the internet, it's opinions.Yiftach: Yeah. I challenge the open-source gurus to change their mindset because the world has changed. You know, we cannot treat the open-source like we used to treat it there in 2008 or early-90s. It is a different world. And you want companies like Redis, you want people to invest in open-source. And we need to somehow survive, right? We need to create a business. So, I challenge these [OSI 00:30:38] committees to think differently. I hope they will, one day.Corey: One last topic that I want to cover is the explosion of AI—artificial intelligence—or machine-learning, or bias-laundering, depending upon who you ask. It feels in many ways like a marketing slogan, and I viewed it as more or less selling pickaxes into a digital gold rush on the part of the cloud providers, until somewhat recently, when I started encountering a bunch of customers who are in fact using it for very interesting and very appropriate use cases. Now, I'm seeing a bunch of databases that are touting their machine-learning capabilities on top of the existing database offerings. What's Redis's story around that?Yiftach: Great question. Great question. So, I think today, I have two story, which are related to the real-time AI, yeah, we are in the real-time world. One of them is what we call the online feature store. Just to explain the audience what is a feature store, usually, when you do inferencing, you need to enhance the transaction with some data, in order to get the right quality.Where do you store this data? So, for online transaction, usually you want to store it in Redis because you don't want to delay your application whenever you do inferencing. So, the way it works, you get a transaction, you bring the features, you combine them together, sends them to inferencing, and then whatever you want to do with the results. One of the things that we did with Redis, we combine AI inferencing inside with this, and we allow you to do that in one API call, which makes the real-time much, much faster. You can decide to use Redis just as a [unintelligible 00:32:16] feature store; this is also great.The other aspect of AI is vector embedding. Just to make sure that we are all aligned with vector embedding term, so vector embedding allows you to provide a context for your text, for your video, for your image in just 128-byte, or floating point. It really depends on the quality of vector. And think about is that tomorrow, every profile in your database will have a vector that explain the context of the product, the context of the user, everything, like, in one single object in your profile.So, Redis has it. So, what can you do once you have it? For instance, you can search where are the similar vector—this is called vector similarity search—for recommendation engines, and for many, many, many others implications. And you would like to combine it with metadata, like, not only bring me all the similar context, but also, you know, some information about the visitor, like the age, like the height, like where does the person live? So, it's not only vector similarity search, it's search with vector similarity search.Now, the question could be asked, do we want to create a totally different database just for this vector similarity search, and then I will make it fast as Redis because you need everything to run in real-time? And this is why I encourage people to look at what they have in Redis. And again, I don't want to be marketeer here, but they don't think that the single-feature deployment require a new database. And we added this capability because we do see the need to support it in real-time. I hope my answer was not too long.Corey: No, no, it's the right answer because the story that you're telling about this is not about how smart you are; it's not about hype-driven stuff. You're talking about using this to meet actual customer needs. And if you tell me that, “Oh, we built this thing because we're smart,” yeah, I can needle you to death on that and make fun of you until I'm blue in the face. But when you say, “I'm going to go ahead and do this because our customers have this pain,” well, that's a lot harder for me to criticize because, yeah, you have to meet your customers where they are; that's the right thing to do. So, this is the kind of story that is slowly but surely changing my view on the idea of machine-learning across the board.Yiftach: I'm happy that you like it. We like it as well. And we see a lot of traction today. Vector similarity search is becoming, like, a real thing. And also features store.Corey: I want to thank you so much for taking the time to speak with me today. If people want to learn more, where can they find you?Yiftach: Ah, I think first of all, you can go to redis.io or redis.com and look for our solution. And I'm available on LinkedIn and Twitter, and you can find me.Corey: And we will of course put links to all of that in the [show notes 00:35:10]. Thank you so much for your time today. I appreciate it.Yiftach: Thank you, Corey. It was very nice conversation. I really enjoy it. Thank you very much.Corey: Thank you. You as well. Yiftach Shoolman, CTO and co-founder at Redis. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with a long rambling angry comment about open-source licensing that no one is going to be bothered to read.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Genre: Cloud Services, Career Development. Cloud services are services provided by certain companies which allows remote accessibility of data with the help of internet. These services are very important as many companies have branches overseas which makes it difficult for employees to collaborate and work on certain projects. One of the emerging companies who provide reliable cloud resources is google cloud services which will be discussed in detail in this episode.   In the episode: Start 0:00:00 Pritam's career so far and top 3 things that helped him in his career. 0:02:55 What are different types of databases and how do they help the businesses? 0:04:22 What does google cloud offer to meet various DB requirements? 0:06:07 Data Warehousing options available on Google Cloud. 0:09:21 Google Cloud BigTable 0:12:56 How to get started with Database Migrations for homogeneous and heterogeneous scenarios? 0:15:51 Rapid fire questions 0:19:10 What would be your advice for those aspiring to make big in their careers? 0:23:21 Trivia on Websites 0:25:48  About the guest: Pritam has 15 years of IT industry experience, with Specialization on Cloud (IaaS, PaaS), & pre-sales experience in various geographies and market segments. His specialities include Google Cloud, AWS, Oracle Public Cloud, PaaS, IaaS, Hybrid Cloud, Cloud Security, Oracle Identity & Access Management, Oracle Database Technology related products, SAP Databases, SAP HANA. Important Links: Google Cloud Databases  https://cloud.google.com/products/databases Relational Databases           Cloud SQL https://cloud.google.com/sql          Cloud Spanner https://cloud.google.com/spanner Non relational Databases           BigTable https://cloud.google.com/bigtable          Firestore https://cloud.google.com/firestore Relational DB Migrations i.e. Database Migration Service https://cloud.google.com/database-migration Serverless Multicloud Datawarehouse Service on Google Cloud https://cloud.google.com/bigquery/   Connect with Pritam on LinkedIn: https://www.linkedin.com/in/pritam-sahoo-77a438a/ Naveen Samala https://www.linkedin.com/mwlite/in/naveensamala Sudhakar Nagandla: https://www.linkedin.com/in/nvsudhakar   Watch the video interview on YouTube: https://youtu.be/PO7Kk_yNkzM  

About RobertR2 advocates for Liquibase customers and provides technical architecture leadership. Prior to co-founding Datical (now Liquibase), Robert was a Director at the Austin Technology Incubator. Robert co-founded Phurnace Software in 2005. He invented and created the flagship product, Phurnace Deliver, which provides middleware infrastructure management to multiple Fortune 500 companies.Links: Liquibase: https://www.liquibase.com Liquibase Community: https://www.liquibase.org Liquibase AWS Marketplace: https://aws.amazon.com/marketplace/seller-profile?id=7e70900d-dcb2-4ef6-adab-f64590f4a967 Github: https://github.com/liquibase Twitter: https://twitter.com/liquibase TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: It seems like there is a new security breach every day. Are you confident that an old SSH key, or a shared admin account, isn't going to come back and bite you? If not, check out Teleport. Teleport is the easiest, most secure way to access all of your infrastructure. The open source Teleport Access Plane consolidates everything you need for secure access to your Linux and Windows servers—and I assure you there is no third option there. Kubernetes clusters, databases, and internal applications like AWS Management Console, Yankins, GitLab, Grafana, Jupyter Notebooks, and more. Teleport's unique approach is not only more secure, it also improves developer productivity. To learn more visit: goteleport.com. And not, that is not me telling you to go away, it is: goteleport.com. Corey: You know how Git works right?Announcer: Sorta, kinda, not really. Please ask someone else.Corey: That's all of us. Git is how we build things, and Netlify is one of the best ways I've found to build those things quickly for the web. Netlify's Git-based workflows mean you don't have to play slap-and-tickle with integrating arcane nonsense and web hooks, which are themselves about as well understood as Git. Give them a try and see what folks ranging from my fake Twitter for Pets startup, to global Fortune 2000 companies are raving about. If you end up talking to them—because you don't have to; they get why self-service is important—but if you do, be sure to tell them that I sent you and watch all of the blood drain from their faces instantly. You can find them in the AWS marketplace or at www.netlify.com. N-E-T-L-I-F-Y dot com.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. This is a promoted episode. What does that mean in practice? Well, it means the company who provides the guest has paid to turn this into a discussion that's much more aligned with the company than it is the individual.Sometimes it works, Sometimes it doesn't, but the key part of that story is I get paid. Why am I bringing this up? Because today's guest is someone I met in person at Monktoberfest, which is the RedMonk conference in Portland, Maine, one of the only reasons to go to Maine, speaking as someone who grew up there. And I spoke there, I met my guest today, and eventually it turned into this, proving that I am the envy of developer advocates everywhere because now I can directly tie me attending one conference to making a fixed sum of money, and right now they're all screaming and tearing off their headphones and closing this episode. But for those of you who are sticking around, thank you. My guest today is the CTO and co-founder of Liquibase. Please welcome Robert Reeves. Robert, thank you for joining me, and suffering the slings and arrows I'm about to hurled directly into your arse, as a warning shot.Robert: [laugh]. Man. Thanks for having me. Corey, I've been looking forward to this for a while. I love hanging out with you.Corey: One of the things I love about the Monktoberfest conference, and frankly, anything that RedMonk gets up to is, forget what's on stage, which is uniformly excellent; forget the people at RedMonk who are wonderful and I aspire to do more work with them in different ways; they're great, but the people that they attract are invariably interesting, they are invariably incredibly diverse in terms of not just demographics, but interests and proclivities. It's just a wonderful group of people, and every time I get the opportunity to spend time with those folks I do, and I've never once regretted it because I get to meet people like you. Snark and cynicism about sponsoring this nonsense aside—for which I do thank you—you've been a fascinating person to talk to you because you're better at a lot of the database-facing things than I am, so I shortcut to instead of forming my own opinions, I just skate off of yours in some cases. You're going to get letters now.Robert: Well, look, it's an occupational hazard, right? Releasing software, it's hard so you have to learn these platforms, and part of it includes the database. But I tell you, you're spot on about Monktoberfest. I left that conference so motivated. Really opened my eyes, certainly injecting empathy into what I do on a day-to-day basis, but it spurred me to action.And there's a lot of programs that we've started at Liquibase that the germination for that seed came from Monktoberfest. And certainly, you know, we were bummed out that it's been canceled two years in a row, but we can't wait to get back and sponsor it. No end of love and affection for that team. They're also really smart and right about a hundred percent of the time.Corey: That's the most amazing part is that they have opinions that generally tend to mirror my own—which, you know—Robert: [laugh].Corey: —confirmation bias is awesome, but they almost never get it wrong. And that is one of the impressive things is when I do it, I'm shooting from the hip and I already have an apology half-written and ready to go, whereas when dealing with them, they do research on this and they don't have the ‘I'm a loud, abrasive shitpostter on Twitter' defense to fall back on to defend opinions. And if they do, I've never seen them do it. They're right, and the fact that I am as aligned with them as I am, you'd think that one of us was cribbing from the other. I assure you that's not the case.But every time Steve O'Grady or Rachel Stephens, or Kelly—I forget her last name; my apologies is all Twitter, but she studied medieval history, I remember that—or James Governor writes something, I'm uniformly looking at this and I feel a sense of dismay, been, “Dammit. I should have written this. It's so well written and it makes such a salient point.” I really envy their ability to be so consistently on point.Robert: Well, they're the only analysts we pay money to. So, we vote with our dollars with that one. [laugh].Corey: Yeah. I'm only an analyst when people have analyst budget. Other than that, I'm whatever the hell you describe me. So, let's talk about that thing you're here to show. You know, that little side project thing you found and are the CTO of.I wasn't super familiar with what Liquibase does until I looked into it and then had this—I got to say, it really pissed me off because I'm looking at it, and it's how did I not know that this existed back when the exact problems that you solve are the things I was careening headlong into? I was actively annoyed. You're also an open-source project, which means that you're effectively making all of your money by giving things away and hoping for gratitude to come back on you in the fullness of time, right?Robert: Well, yeah. There's two things there. They're open-source component, but also, where was this when I was struggling with this problem? So, for the folks that don't know, what Liquibase does is automate database schema change. So, if you need to update a database—I don't care what it is—as part of your application deployment, we can help.Instead of writing a ticket or manually executing a SQL script, or generating a bunch of docs in a NoSQL database, you can have Liquibase help you out with that. And so I was at a conference years ago, at the booth, doing my booth thing, and a managing director of a very large bank came to me, like, “Hey, what do you do?” And saw what we did and got angry, started yelling at me. “Where were you three years ago when I was struggling with this problem?” Like, spitting mad. [laugh]. And I was like, “Dude, we just started”—this was a while ago—it was like, “We just started the company two years ago. We got here as soon as we could.”But I struggled with this problem when I was a release manager. And so I've been doing this for years and years and years—I don't even want to talk about how long—getting bits from dev to test to production, and the database was always, always, always the bottleneck, whether it was things didn't run the same in test as they did, eventually in production, environments weren't in sync. It's just really hard. And we've automated so much stuff, we've automated application deployment, lowercase a compiled bits; we're building things with containers, so everything's in that container. It's not a J2EE app anymore—yay—but we haven't done a damn thing for the database.And what this means is that we have a whole part of our industry, all of our database professionals, that are frankly struggling. I always say we don't sell software Liquibase. We sell piano recitals, date nights, happy hours, all the stuff you want to do but you can't because you're stuck dealing with the database. And that's what we do at Liquibase.Corey: Well, you're talking about database people. That's not how I even do it. I would never call myself that, for very good reason because you know, Route 53 remains the only database I use. But the problem I always had was that, “Great. I'm doing a deployment. Oh, I'm going to put out some changes to some web servers. Okay, what's my rollback?” “Well, we have this other commit we can use.” “Oh, we're going to be making a database schema change. What's your rollback strategy,” “Oh, I've updated my resume and made sure that any personal files I had on my work laptop been backed up somewhere else when I immediately leave the company when we can't roll back.” Because there's not really going to be a company anymore at that point.It's one of those everyone sort of holds their breath and winces when it comes to anything that resembles a schema change—or an ALTER TABLE as we used to call it—because that is the mistakes will show territory and you can hope and plan for things in pre-prod environments, but it's always scary. It's always terrifying because production is not like other things. That's why I always call my staging environment ‘theory' because things work in theory but not in production. So, it's how do you avoid the mess of winding up just creating disasters when you're dealing with the reality of your production environments? So, let's back up here. How do you do it? Because it sounds like something people would love to sell me but doesn't exist.Robert: [laugh]. Well, it's real simple. We have a file, we call it the change log. And this is a ledger. So, databases need to be evolved. You can't drop everything and recreate it from scratch, so you have to apply changes sequentially.And so what Liquibase will do is it connects to the database, and it says, “Hey, what version are you?” It looks at the change log, and we'll see, ehh, “There's ten change sets”—that's what components of a change log, we call them change sets—“There's ten change sets in there and the database is telling me that only five had been executed.” “Oh, great. Well, I'll execute these other five.” Or it asks the database, “Hey, how many have been executed?” And it says, “Ten.”And we've got a couple of meta tables that we have in the database, real simple, ANSI SQL compliant, that store the changes that happen to the database. So, if it's a net new database, say you're running a Docker container with the database in it on your local machine, it's empty, you would run Liquibase, and it says, “Oh, hey. It's got that, you know, new database smell. I can run everything.”And so the interesting thing happens when you start pointing it at an environment that you haven't updated in a while. So, dev and test typically are going to have a lot of releases. And so there's going to be little tiny incremental changes, but when it's time to go to production, Liquibase will catch it up. And so we speak SQL to the database, if it's a NoSQL database, we'll speak their API and make the changes requested. And that's it. It's very simple in how it works.The real complex stuff is when we go a couple of inches deeper, when we start doing things like, well, reverse engineering of your database. How can I get a change log of an existing database? Because nobody starts out using Liquibase for a project. You always do it later.Corey: No, no. It's one of those things where when you're doing a project to see if it works, it's one of those, “Great, I'll run a database in some local Docker container or something just to prove that it works.” And, “Todo: fix this later.” And yeah, that todo becomes load-bearing.Robert: [laugh]. That's scary. And so, you know, we can help, like, reverse engineering an entire database schema, no problem. We also have things called quality checks. So sure, you can test your Liquibase change against an empty database and it will tell you if it's syntactically correct—you'll get an error if you need to fix something—but it doesn't enforce things like corporate standards. “Tables start with T underscore.” “Do not create a foreign key unless those columns have an ID already applied.” And that's what our quality checks does. We used to call it rules, but nobody likes rules, so we call it quality checks now.Corey: How do you avoid the trap of enumerating all the bad things you've seen happen because at some point, it feels like that's what leads to process ossification at large companies where, “Oh, we had this bad thing happen once, like, a disk filled up, so now we have a check that makes sure that all the disks are at least 20, empty.” Et cetera. Great. But you keep stacking those you have thousands and thousands and thousands of those, and even a one-line code change then has to pass through so many different tests to validate that this isn't going to cause the failure mode that happened that one time in a unicorn circumstance. How do you avoid the bloat and the creep of stuff like that?Robert: Well, let's look at what we've learned from automated testing. We certainly want more and more tests. Look, DevOp's algorithm is, “All right, we had a problem here.” [laugh]. Or SRE algorithm, I should say. “We had a problem here. What happened? What are we going to change in the future to make sure this doesn't happen?” Typically, that involves a new standard.Now, ossification occurs when a person has to enforce that standard. And what we should do is seek to have automation, have the machine do it for us. Have the humans come up and identify the problem, find a creative way to look for the issue, and then let the machine enforce it. Ossification happens in large organizations when it's people that are responsible, not the machine. The machines are great at running these things over and over again, and they're never hung over, day after Super Bowl Sunday, their kid doesn't get sick, they don't get sick. But we want humans to look at the things that we need that creative energy, that brain power on. And then the rote drudgery, hand that off to the machine.Corey: Drudgery seems like sort of a job description for a lot of us who spend time doing operation stuff.Robert: [laugh].Corey: It's drudgery and it's boring, punctuated by moments of sheer terror. On some level, you're more or less taking some of the adrenaline high of this job away from people. And you know, when it comes to databases, I'm kind of okay with that as it turns out.Robert: Yeah. Oh, yeah, we want no surprises in database-land. And that is why over the past several decades—can I say several decades since 1979?Corey: Oh, you can s—it's many decades, I'm sorry to burst your bubble on that.Robert: [laugh]. Thank you, Corey. Thank you.Corey: Five, if we're being honest. Go ahead.Robert: So, it has evolved over these many decades where change is the enemy of stability. And so we don't want change, and we want to lock these things down. And our database professionals have become changed from sentinels of data into traffic cops and TSA. And as we all know, some things slip through those. Sometimes we speed, sometimes things get snuck through TSA.And so what we need to do is create a system where it's not the people that are in charge of that; that we can set these policies and have our database professionals do more valuable things, instead of that adrenaline rush of, “Oh, my God,” how about we get the rush of solving a problem and saving the company millions of dollars? How about that rush? How about the rush of taking our old, busted on-prem databases and figure out a way to scale these up in the cloud, and also provide quick dev and test environments for our developer and test friends? These are exciting things. These are more fun, I would argue.Corey: You have a list of reference customers on your website that are awesome. In fact, we share a reference customer in the form of Ticketmaster. And I don't think that they will get too upset if I mention that based upon my work with them, at no point was I left with the impression that they played fast and loose with databases. This was something that they take very seriously because for any company that, you know, sells tickets to things you kind of need an authoritative record of who's bought what, or suddenly you don't really have a ticket-selling business anymore. You also reference customers in the form of UPS, which is important; banks in a variety of different places.Yeah, this is stuff that matters. And you support—from the looks of it—every database people can name except for Route 53. You've got RDS, you've got Redshift, you've got Postgres-squeal, you've got Oracle, Snowflake, Google's Cloud Spanner—lest people think that it winds up being just something from a legacy perspective—Cassandra, et cetera, et cetera, et cetera, CockroachDB. I could go on because you have multiple pages of these things, SAP HANA—whatever the hell that's supposed to be—Yugabyte, and so on, and so forth. And it's like, some of these, like, ‘now you're just making up animals' territory.Robert: Well, that goes back to open-source, you know, you were talking about that earlier. There is no way in hell we could have brought out support for all these database platforms without us being open-source. That is where the community aligns their goals and works to a common end. So, I'll give you an example. So, case in point, recently, let me see Yugabyte, CockroachDB, AWS Redshift, and Google Cloud Spanner.So, these are four folks that reached out to us and said, either A) “Hey, we want Liquibase to support our database,” or B) “We want you to improve the support that's already there.” And so we have what we call—which is a super creative name—the Liquibase test harness, which is just genius because it's an automated way of running a whole suite of tests against an arbitrary database. And that helped us partner with these database vendors very quickly and to identify gaps. And so there's certain things that AWS Redshift—certain objects—that AWS Redshift doesn't support, for all the right reasons. Because it's data warehouse.Okay, great. And so we didn't have to run those tests. But there were other tests that we had to run, so we create a new test for them. They actually wrote some of those tests. Our friends at Yugabyte, CockroachDB, Cloud Spanner, they wrote these extensions and they came to us and partnered with us.The only way this works is with open-source, by being open, by being transparent, and aligning what we want out of life. And so what our friends—our database friends—wanted was they wanted more tooling for their platform. We wanted to support their platform. So, by teaming up, we help the most important person, [laugh] the most important person, and that's the customer. That's it. It was not about, “Oh, money,” and all this other stuff. It was, “This makes our customers' lives easier. So, let's do it. Oop, no brainer.”Corey: There's something to be said for making people's lives easier. I do want to talk about that open-source versus commercial divide. If I Google Liquibase—which, you know, I don't know how typing addresses in browsers works anymore because search engines are so fast—I just type in Liquibase. And the first thing it spits me out to is liquibase.org, which is the Community open-source version. And there's a link there to the Pro paid version and whatnot. And I was just scrolling idly through the comparison chart to see, “Oh, so ‘Community' is just code for shitty and you're holding back advanced features.” But it really doesn't look that way. What's the deal here?Robert: Oh, no. So, Liquibase open-source project started in 2006 and Liquibase the company, the commercial entity, started after that, 2012; 2014, first deal. And so, for—Nathan Voxland started this, and Nathan was struggling. He was working at a company, and he had to have his application—of course—you know, early 2000s, J2EE—support SQL Server and Oracle and he was struggling with it. And so he open-sourced it and added more and more databases.Certainly, as open-source databases grew, obviously he added those: MySQL, Postgres. But we're never going to undo that stuff. There's rollback for free in Liquibase, we're not going to be [laugh] we're not going to be jerks and either A) pull features out or, B) even worse, make Stephen O'Grady's life awful by changing the license [laugh] so he has to write about it. He loves writing about open-source license changes. We're Apache 2.0 and so you can do whatever you want with it.And we believe that the things that make sense for a paying customer, which is database-specific objects, that makes sense. But Liquibase Community, the open-source stuff, that is built so you can go to any database. So, if you have a change log that runs against Oracle, it should be able to run against SQL Server, or MySQL, or Postgres, as long as you don't use platform-specific data types and those sorts of things. And so that's what Community is about. Community is about being able to support any database with the same change log. Pro is about helping you get to that next level of DevOps Nirvana, of reaching those four metrics that Dr. Forsgren tells us are really important.Corey: Oh, yes. You can argue with Nicole Forsgren, but then you're wrong. So, why would you ever do that?Robert: Yeah. Yeah. [laugh]. It's just—it's a sucker's bet. Don't do it. There's a reason why she's got a PhD in CS.Corey: She has been a recurring guest on this show, and I only wish she would come back more often. You and I are fun to talk to, don't get me wrong. We want unbridled intellect that is couched in just a scintillating wit, and someone is great to talk to. Sorry, we're both outclassed.Robert: Yeah, you get entertained with us; you learn with her.Corey: Exactly. And you're still entertained while doing it is the best part.Robert: [laugh]. That's the difference between Community and Pro. Look, at the end of the day, if you're an individual developer just trying to solve a problem and get done and away from the computer and go spend time with your friends and family, yeah, go use Liquibase Community. If it's something that you think can improve the rest of the organization by teaming up and taking advantage of the collaboration features? Yes, sure, let us know. We're happy to help.Corey: Now, if people wanted to become an attorney, but law school was too expensive, out of reach, too much time, et cetera, but they did have a Twitter account, very often, they'll find that they can scratch that itch by arguing online about open-source licenses. So, I want to be very clear—because those people are odious when they email me—that you are licensed under the Apache License. That is a bonafide OSI approved open-source license. It is not everyone except big cloud companies, or service providers, which basically are people dancing around—they mean Amazon. So, let's be clear. One, are you worried about Amazon launching a competitive service with a dumb name? And/or have you really been validated as a product if AWS hasn't attempted and failed to launch a competitor?Robert: [laugh]. Well, I mean, we do have a very large corporation that has embedded Liquibase into one of their flagship products, and that is Oracle. They have embedded Liquibase in SQLcl. We're tickled pink because that means that, one, yes, it does validate Liquibase is the right way to do it, but it also means more people are getting help. Now, for Oracle users, if you're just an Oracle shop, great, have fun. We think it's a great solution. But there's not a lot of those.And so we believe that if you have Liquibase, whether it's open-source or the Pro version, then you're going to be able to support all the databases, and I think that's more important than being tied to a single cloud. Also—this is just my opinion and take it for what it's worth—but if Amazon wanted to do this, well, they're not the only game in town. So, somebody else is going to want to do it, too. And, you know, I would argue even with Amazon's backing that Liquibase is a little stronger brand than anything they would come out with.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance accelerator for the Oracle MySQL Database Service. Although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLTP and OLAP, don't ask me to ever say those acronyms again, workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense. Corey: So, I want to call out though, that on some level, they have already competed with you because one of database that you do not support is DynamoDB. Let's ignore the Route 53 stuff because, okay. But the reason behind that, having worked with it myself, is that, “Oh, how do you do a schema change in DynamoDB?” The answer is that you don't because it doesn't do schemas for one—it is schemaless, which is kind of the point of it—as well as oh, you want to change the primary, or the partition, or the sort key index? Great. You need a new table because those things are immutable.So, they've solved this Gordian Knot just like Alexander the Great did by cutting through it. Like, “Oh, how do you wind up doing this?” “You don't do this. The end.” And that is certainly an approach, but there are scenarios where those were first, NoSQL is not a acceptable answer for some workloads.I know Rick [Horahan 00:26:16] is going to yell at me for that as soon as he hears me, but okay. But there are some for which a relational database is kind of a thing, and you need that. So, Dynamo isn't fit for everything. But there are other workloads where, okay, I'm going to just switch over. I'm going to basically dump all the data and add it to a new table. I can't necessarily afford to do that with anything less than maybe, you know, 20 milliseconds of downtime between table one and table two. And they're obnoxious and difficult ways to do it, but for everything else, you do kind of need to make ALTER TABLE changes from time to time as you go through the build and release process.Robert: Yeah. Well, we certainly have plans for DynamoDB support. We are working our way through all the NoSQLs. Started with Mongo, and—Corey: Well, back that out a second then for me because there's something I'm clearly not grasping because it's my understanding, DynamoDB is schemaless. You can put whatever you want into various arbitrary fields. How would Liquibase work with something like that?Robert: Well, that's something I struggled with. I had the same question. Like, “Dude, really, we're a schema change tool. Why would we work with a schemaless database?” And so what happened was a soon-to-be friend of ours in Europe had reached out to me and said, “I built an extension for MongoDB in Liquibase. Can we open-source this, and can y'all take care of the care and feeding of this?” And I said, “Absolutely. What does it do?” [laugh].And so I looked at it and it turns out that it focuses on collections and generating data for test. So, you're right about schemaless because these are just documents and we're not going to go through every single document and change the structure, we're just going to have the application create a new doc and the new format. Maybe there's a conversion log logic built into the app, who knows. But it's the database professionals that have to apply these collections—you know, indices; that's what they call them in Mongo-land: collections. And so being able to apply these across all environments—dev, test, production—and have consistency, that's important.Now, what was really interesting is that this came from MasterCard. So, this engineer had a consulting business and worked for MasterCard. And they had a problem, and they said, “Hey, can you fix this with Liquibase?” And he said, “Sure, no problem.” And he built it.So, that's why if you go to the MongoDB—the liquibase-mongodb repository in our Liquibase org, you'll see that MasterCard has the copyright on all that code. Still Apache 2.0. But for me, that was the validation we needed to start expanding to other things: Dynamo, Couch. And same—Corey: Oh, yeah. For a lot of contributors, there's a contributor license process you can go through, assign copyright. For everything else, there's MasterCard.Robert: Yeah. Well, we don't do that. Look, you know, we certainly have a code of conduct with our community, but we don't have a signing copyright and that kind of stuff. Because that's baked into Apache 2.0. So, why would I want to take somebody's ability to get credit and magical internet points and increase the rep by taking that away? That's just rude.Corey: The problem I keep smacking myself into is just looking at how the entire database space across the board goes, it feels like it's built on lock-in, it's built on it is super finicky to work with, and it generally feels like, okay, great. You take something like Postgres-squeal or whatever it is you want to run your database on, yeah, you could theoretically move it a bunch of other places, but moving databases is really hard. Back when I was at my last, “Real job,” quote-unquote, years ago, we were late to the game; we migrated the entire site from EC2 Classic into a VPC, and the biggest pain in the ass with all of that was the RDS instance. Because we had to quiesce the database so it would stop taking writes; we would then do snapshot it, shut it down, and then restore a new database from that RDS snapshot.How long does it take, at least in those days? That is left as an experiment for the reader. So, we booked a four hour maintenance window under the fear that would not be enough. It completed in 45 minutes. So okay, there's that. Sparked the thing up and everything else was tested and good to go. And yay. Okay.It took a tremendous amount of planning, a tremendous amount of work, and that wasn't moving it very far. It is the only time I've done a late-night deploy, where not a single thing went wrong. Until I was on the way home and the Uber driver sideswiped a city vehicle. So, there we go—Robert: [laugh].Corey: —that's the one. But everything else was flawless on this because we planned these things out. But imagine moving to a different provider. Oh, forget it. Or imagine moving to a different database engine? That's good. Tell another one.Robert: Well, those are the problems that we want our database professionals to solve. We do not want them to be like janitors at an elementary school, cleaning up developer throw-up with sawdust. The issue that you're describing, that's a one time event. This is something that doesn't happen very often. You need hands on the keyboard, you want people there to look for problems.If you can take these database releases away from those folks and automate them safely—you can have safety and speed—then that frees up their time to do these other herculean tasks, these other feats of strength that they're far better at. There is no silver bullet panacea for database issues. All we're trying to do is take about 70% of DBAs time and free it up to do the fun stuff that you described. There are people that really enjoy that, and we want to free up their time so they can do that. Moving to another platform, going from the data center to the cloud, these sorts of things, this is what we want a human on; we don't want them updating a column three times in a row because dev couldn't get it right. Let's just give them the keys and make sure they stay in their lane.Corey: There's something glorious about being able to do that. I wish that there were more commonly appreciated ways of addressing those pains, rather than, “Oh, we're going to sell you something big and enterprise-y and it's going to add a bunch of process and not work out super well for you.” You integrate with existing CI/CD systems reasonably well, as best I can tell because the nice thing about CI/CD—and by nice I mean awful—is that there is no consensus. Every pipeline you see, in a release engineering process inherently becomes this beautiful bespoke unicorn.Robert: Mm-hm. Yeah. And we have to. We have to integrate with whatever CI/CD they have in place. And we do not want customers to just run Liquibase by itself. We want them to integrate it with whatever is driving that application deployment.We're Switzerland when it comes to databases, and CI/CD. And I certainly have my favorite of those, and it's primarily based on who bought me drinks at the last conference, but we cannot go into somebody's house and start rearranging the furniture. That's just rude. If they're deploying the app a certain way, what we tell that customer is, “Hey, we're just going to have that CI/CD tool call Liquibase to update the database. This should be an atomic unit of deployment.” And it should be hidden from the person that pushes that shiny button or the automation that does it.Corey: I wish that one day that you could automate all of the button pushing, but the thing that always annoyed me in release engineering was the, “Oh, and here's where we stop to have a human press the button.” And I get it. That stuff's scary for some folks, but at the same time, this is the nature of reality. So, you're not going to be able to technology your way around people. At least not successfully and not for very long.Robert: It's about trust. You have to earn that database professional's trust because if something goes wrong, blaming Liquibase doesn't go very far. In that company, they're going to want a person [laugh] who has a badge to—with a throat to choke. And so I've seen this pattern over and over again.And this happened at our first customer. Major, major, big, big, big bank, and this was on the consumer side. They were doing their first production push, and they wanted us ready. Not on the call, but ready if there was an issue they needed to escalate and get us to help them out. And so my VP of Engineering and me, we took it. Great. Got VP of engineering and CTO. Right on.And so Kevin and I, we stayed home, stayed sober [laugh], you know—a lot of places to party in Austin; we fought that temptation—and so we stayed and I'm texting with Kevin, back and forth. “Did you get a call?” “No, I didn't get a call.” It was Friday night. Saturday rolls around. Sunday. “Did you get a—what's going on?” [laugh].Monday, we're like, “Hey. Everything, okay? Did you push to the next weekend?” They're like, “Oh, no. We did. It went great. We forgot to tell you.” [laugh]. But here's what happened. The DBAs push the Liquibase ‘make it go' button, and then they said, “Uh-Oh.” And we're like, “What do you mean, uh-oh?” They said, “Well, something went wrong.” “Well, what went wrong?” “Well, it was too fast.” [laugh]. Something—no way. And so they went through the whole thing—Corey: That was my downtime when I supposed to be compiling.Robert: Yeah. So, they went through the whole thing to verify every single change set. Okay, so that was weekend one. And then they go to weekend two, they do it the same thing. All right, all right. Building trust.By week four, they called a meeting with the release team. And they said, “Hey, process change. We're no longer going to be on these calls. You are going to push the Liquibase button. Now, if you want to integrate it with your CI/CD, go right ahead, but that's not my problem.” Dev—or, the release team is tier one; dev is tier two; we—DBAs—are tier three support, but we'll call you because we'll know something went wrong. And to this day, it's all automated.And so you have to earn trust to get people to give that up. Once they have trust and you really—it's based on empathy. You have to understand how terrible [laugh] they are sometimes treated, and to actively take care of them, realize the problems they're struggling with, and when you earn that trust, then and only then will they allow automation. But it's hard, but it's something you got to do.Corey: You mentioned something a minute ago that I want to focus on a little bit more closely, specifically that you're in Austin. Seems like that's a popular choice lately. You've got companies that are relocating their headquarters there, presumably for tax purposes. Oracle's there, Tesla's there. Great. I mean, from my perspective, terrific because it gets a number of notably annoying CEOs out of my backyard. But what's going on? Why is Austin on this meteoric rise and how'd it get there?Robert: Well, a lot of folks—overnight success, 40 years in the making, I guess. But what a lot of people don't realize is that, one, we had a pretty vibrant tech hub prior to all this. It all started with MCC, Microcomputer Consortium, which in the '80s, we were afraid of the Japanese taking over and so we decided to get a bunch of companies together, and Admiral Bobby Inman who was director planted it in Austin. And that's where it started. You certainly have other folks that have a huge impact, obviously, Michael Dell, Austin Ventures, a whole host of folks that have really leaned in on tech in Austin, but it actually started before that.So, there was a time where Willie Nelson was in Nashville and was just fed up with RCA Records. They would not release his albums because he wanted to change his sound. And so he had some nice friends at Atlantic Records that said, “Willie, we got this. Go to New York, use our studio, cut an album, we'll fix it up.” And so he cut an album called Shotgun Willie, famous for having “Whiskey River” which is what he uses to open and close every show.But that album sucked as far as sales. It's a good album, I like it. But it didn't sell except for one place in America: in Austin, Texas. It sold more copies in Austin than anywhere else. And so Willie was like, “I need to go check this out.”And so he shows up in Austin and sees a bunch of rednecks and hippies hanging out together, really geeking out on music. It was a great vibe. And then he calls, you know, Kris, and Waylon, and Merle, and say, “Come on down.” And so what happened here was a bunch of people really wanted to geek out on this new type of country music, outlaw country. And it started a pattern where people just geek out on stuff they really like.So, same thing with Austin film. You got Robert Rodriguez, you got Richard Linklater, and Slackers, his first movie, that's why I moved to Austin. And I got a job at Les Amis—a coffee shop that's closed—because it had three scenes in that. There was a whole scene of people that just really wanted to make different types of films. And we see that with software, we see that with film, we see it with fashion.And it just seems that Austin is the place where if you're really into something, you're going to find somebody here that really wants to get into it with you, whether it's board gaming, D&D, noise punk, whatever. And that's really comforting. I think it's the community that's just welcoming. And I just hope that we can continue that creativity, that sense of community, and that we don't have large corporations that are coming in and just taking from the system. I hope they inject more.I think Oracle's done a really good job; their new headquarters is gorgeous, they've done some really good things with the city, doing a land swap, I think it was forty acres for nine acres. They coughed up forty for nine. And it was nine acres the city wasn't even using. Great. So, I think they're being good citizens. I think Tesla's been pretty cool with building that factory where it is. I hope more come. I hope they catch what is ever in the water and the breakfast tacos in Austin.Corey: [laugh]. I certainly look forward to this pandemic ending; I can come over and find out for myself. I'm looking forward to it. I always enjoyed my time there, I just wish I got to spend more of it.Robert: How many folks from Duckbill Group are in Austin now?Corey: One at the moment. Tim Banks. And the challenge, of course, is that if you look across the board, there really aren't that many places that have more than one employee. For example, our operations person, Megan, is here in San Francisco and so is Jesse DeRose, our manager of cloud economics. But my business partner is in Portland; we have people scattered all over the country.It's kind of fun having a fully-distributed company. We started this way, back when that was easy. And because all right, travel is easy; we'll just go and visit whenever we need to. But there's no central office, which I think is sort of the dangerous part of full remote because then you have this idea of second-class citizens hanging out in one part of the country and then they go out to lunch together and that's where the real decisions get made. And then you get caught up to speed. It definitely fosters a writing culture.Robert: Yeah. When we went to remote work, our lease was up. We just didn't renew. And now we have expanded hiring outside of Austin, we have folks in the Ukraine, Poland, Brazil, more and more coming. We even have folks that are moving out of Austin to places like Minnesota and Virginia, moving back home where their family is located.And that is wonderful. But we are getting together as a company in January. We're also going to, instead of having an office, we're calling it a ‘Liquibase Lounge.' So, there's a number of retail places that didn't survive, and so we're going to take one of those spots and just make a little hangout place so that people can come in. And we also want to open it up for the community as well.But it's very important—and we learned this from our friends at GitLab and their culture. We really studied how they do it, how they've been successful, and it is an awareness of those lunch meetings where the decisions are made. And it is saying, “Nope, this is great we've had this conversation. We need to have this conversation again. Let's bring other people in.” And that's how we're doing at Liquibase, and so far it seems to work.Corey: I'm looking forward to seeing what happens, once this whole pandemic ends, and how things continue to thrive. We're long past due for a startup center that isn't San Francisco. The whole thing is based on the idea of disruption. “Oh, we're disruptive.” “Yes, we're so disruptive, we've taken a job that can be done from literally anywhere with internet access and created a land crunch in eight square miles, located in an earthquake zone.” Genius, simply genius.Robert: It's a shame that we had to have such a tragedy to happen to fix that.Corey: Isn't that the truth?Robert: It really is. But the toothpaste is out of the tube. You ain't putting that back in. But my bet on the next Tech Hub: Kansas City. That town is cool, it has one hundred percent Google Fiber all throughout, great university. Kauffman Fellows, I believe, is based there, so VC folks are trained there. I believe so; I hope I'm not wrong with that. I know Kauffman Foundation is there. But look, there's something happening in that town. And so if you're a buy low, sell high kind of person, come check us out in Austin. I'm not trying to dissuade anybody from moving to Austin; I'm not one of those people. But if the housing prices [laugh] you don't like them, check out Kansas City, and get that two-gig fiber for peanuts. Well, $75 worth of peanuts.Corey: Robert, I want to thank you for taking the time to speak with me so extensively about Liquibase, about how awesome RedMonk is, about Austin and so many other topics. If people want to learn more, where can they find you?Robert: Well, I think the best place to find us right now is in AWS Marketplace. So—Corey: Now, hand on a second. When you say the best place for anything being the AWS Marketplace, I'm naturally a little suspicious. Tell me more.Robert: [laugh]. Well, best is, you know, it's—[laugh].Corey: It is a place that is there and people can find you through it. All right, then.Robert: I have a list. I have a list. But the first one I'm going to mention is AWS Marketplace. And so that's a really easy way, especially if you're taking advantage of the EDP, Enterprise Discount Program. That's helpful. Burn down those dollars, get a discount, et cetera, et cetera. Now, of course, you can go to liquibase.com, download a trial. Or you can find us on Github, github.com/liquibase. Of course, talking smack to us on Twitter is always appreciated.Corey: And we will, of course, include links to that in the [show notes 00:46:37]. Robert Reeves, CTO and co-founder of Liquibase. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment complaining about how Liquibase doesn't support your database engine of choice, which will quickly be rendered obsolete by the open-source community.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

On The Cloud Pod this week, the team wishes for time-traveling data. Also, GCP announces Data Lakehouse, Azure hosts Ignite 2021, and Microsoft is out for the metaverse.  A big thanks to this week's sponsors: Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. JumpCloud, which offers a complete platform for identity, access, and device management — no matter where your users and devices are located.  This week's highlights

About EdEd Boyajian, President and CEO of EDB, drives the development and execution of EDB's strategic vision and growth strategy in the database industry, steering the company through 47 consecutive quarters of recurring revenue growth. He also led EDB's acquisition of 2ndQuadrant, a deal that brought together the world's top PostgreSQL experts and positioned EDB as the largest dedicated provider of PostgreSQL products and solutions worldwide. A 15+ year veteran of the open source software movement, Ed is a seasoned enterprise software executive who emphasizes that EDB must be a technology-first business in order to lead the open source data management ecosystem. Ed joined EDB in 2008 after serving at Red Hat, where he rose to Vice President and General Manager of North America. While there, he played a central leadership role in the development of the modern business model for bringing open source to enterprises.Links:EDB: https://enterprisedb.com TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by Honeycomb. When production is running slow, it's hard to know where problems originate: is it your application code, users, or the underlying systems? I've got five bucks on DNS, personally. Why scroll through endless dashboards, while dealing with alert floods, going from tool to tool to tool that you employ, guessing at which puzzle pieces matter? Context switching and tool sprawl are slowly killing both your team and your business. You should care more about one of those than the other, which one is up to you. Drop the separate pillars and enter a world of getting one unified understanding of the one thing driving your business: production. With Honeycomb, you guess less and know more. Try it for free at Honeycomb.io/screaminginthecloud. Observability, it's more than just hipster monitoring. Corey: This episode is sponsored in part by our friends at Jellyfish. So, you're sitting in front of your office chair, bleary eyed, parked in front of a powerpoint and—oh my sweet feathery Jesus its the night before the board meeting, because of course it is! As you slot that crappy screenshot of traffic light colored excel tables into your deck, or sift through endless spreadsheets looking for just the right data set, have you ever wondered, why is it that sales and marketing get all this shiny, awesome analytics and inside tools? Whereas, engineering basically gets left with the dregs. Well, the founders of Jellyfish certainly did. That's why they created the Jellyfish Engineering Management Platform, but don't you dare call it JEMP! Designed to make it simple to analyze your engineering organization, Jellyfish ingests signals from your tech stack. Including JIRA, Git, and collaborative tools. Yes, depressing to think of those things as your tech stack but this is 2021. They use that to create a model that accurately reflects just how the breakdown of engineering work aligns with your wider business objectives. In other words, it translates from code into spreadsheet. When you have to explain what you're doing from an engineering perspective to people whose primary IDE is Microsoft Powerpoint, consider Jellyfish. Thats Jellyfish.co and tell them Corey sent you! Watch for the wince, thats my favorite part. Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Today's promoted episode is a treasure and a delight. Longtime listeners of this show know that it's not really a database—unless of course, it's Route 53—and of course, I don't solve pronunciation problems with answers that make absolutely everyone hate me. Longtime listeners of the show know that if there's one thing I adore when it comes to databases—you know, other than Route 53—it is solving pronunciation holy wars in such a way that absolutely everyone is furious with me as a result, and today is no exception. My guest is Ed Boyajian, the CEO of EDB, a company that effectively is the driving force behind the Postgres-squeal database. Ed, thank you for joining me.Ed: Hey, Corey.Corey: So, I know that other people pronounce it ‘post-gree,' ‘Postgresql,' ‘Postgres-Q-L,' all kinds of other things. We know it's decidedly not ‘Postgres-squeal,' which is how I go for it. How do you pronounce it?Ed: We say ‘Postgres,' and this is one of the great branding challenges this fantastic open-source project has endured over many years.Corey: So, I want to start at the very beginning because when I say that you folks are the driving force behind Postgres—or Postgres-squeal—I mean it. I've encountered folks from EDB—formerly EnterpriseDB—in the wild in consulting engagements before, and it's great because whenever we found an intractable database problem, back at my hands-on keyboard engineering implementation days, very quickly after you folks got involved, it stopped being a problem, which is kind of the entire point. A lot of companies will get up there and say, “Oh, it's an open-source project,” with an asterisk next to it and 15 other things that follow from it, or, “Now, we're changing our license so the big companies can't compete with us.” Your company's not named after Postgres-squeal and you're also—when you say you have people working on it, we're not talking just one or two folks; your fingerprints are all over the codebase. How do you engage with an open-source project in that sense?Ed: First and foremost, Postgres itself is, as you know, an independent open-source project, a lot like Linux. And that means it's not controlled by a company. I think that's inherently one of Postgres's greatest strengths and assets. With that in mind, it means that a company like EDB—and this started when I came to the company; I came from Red Hat, so I've been in open-source for 20 years—when I came to the company back in 2008, it starts with a commitment and investment in bringing technology leaders in and around Postgres into a business like EDB, to help enterprises and customers. And that dynamic intersection between building the core database in the community and addressing customer needs in a business, at that intersection is where the magic happens. And we've been doing that since I joined EDB in 2008; it was really an explicit focus for the company.Corey: I'd like to explore a little bit, well first and foremost, this story of is there a future for running databases in cloud environments yourself? And I have my own angry, loud opinion on this that I'm sure we'll get to momentarily, but I want to start with yours. Who is writing their own databases in the Year of our Lord 2021, rather than just using whatever managed thing is their cloud provider of choice today is offering for them?Ed: Well, let me give you context, Corey, because I think it matters. We've been bringing enterprise Postgres solutions to companies now, since the inception of the company, which dates back to 2004, and over that trajectory, we've been helping companies as they've done really two things: migrate away, in particular from Oracle, and land on Postgres, and then write new apps. Probably the first ten of the last 13 years since I've been in the company, the focus was in traditional on-prem database transformations that companies were going through. In the last three years, we've really seen an acceleration of that intersection of their traditional deployments and their cloud deployments. Our customers now, who are represented mostly in the Fortune 500 and Global 2000, 40% of our customers report they're deploying EDB's Postgres in the cloud, not in a managed context, but in a traditional EC2 or GCP self-managed cloud deployment.Corey: And that aligns with what I've seen, a fair bit. Years ago, I wound up getting the AWS Cloud Practitioner Certification—did a whole blog post on it—not because it was opening any doors for me, but because it let me get into the certified lounge at re:Invent, and ideally charge a battery and have some mostly crappy coffee. The one question I got wrong was I was honest when I answered, “How long does it take to restore an RDS database from snapshot backup?” Rather than giving the by-the-book answer, which is way shorter than I found in practice a fair bit of the time. And that's the problem I always ran into is that when you're starting out and building something that needs a database, and it needs a relational database that runs in that model so all the no SQL options are not viable for whatever reason, great, RDS is great for getting you started, but there's only so much that you can tune and tweak before you start to run into issues were, for particular workloads as they scale-out, it's no longer a fit for a variety of reasons.And most of the large companies that I work with that are heavily relational-database-driven have either started off or migrated to the idea of, “Oh, we're going to run our own databases on top of EC2 instances,” for a variety of reasons that, again, the cloud providers will say, “Oh, that's not accurate, and they're doing the wrong thing.” But, you know, it takes a certain courage to tell a large-scale customer, “You're doing it wrong.” “Well, why is that?” “Because I have things to sell you,” is kind of a terrible answer. How do you see it? Let's not pick on RDS, necessarily, because all of the cloud providers offered managed database offerings. Where do those make sense and where do they fall down?Ed: Yeah, I think many of our customers who made their first step into cloud picked a single vendor to do it, and we often hear AWS is been that early, early—Corey: Yeah, a five-year head start makes a pretty compelling story.Ed: That's right. And let's remember what these vendors are mostly. They are mostly infrastructure companies, they build massive data centers and set those up, and they do that beautifully well. And they lean on software, but they're not software companies themselves. And I think the early implementation of many of our customers in cloud relied on what I'll call relatively lightweight software offerings from their cloud vendor, including database.They traded convenience, ease of use, an easy on-ramp, and they traded some capability in some depth for that. And it was a good trade, in fact. And for a large number of workloads it may still be a good trade. But our more sophisticated customers, enterprise customers who are running Postgres or databases at scale in their traditional environments have long depended on a very intimate relationship with their database technology vendor. And that relationship is the intersection of their evolving and emerging needs and the actual development of the database capabilities in support of that.And that's the heart of who we are at EDB and what we do with Postgres and the many people we have committed to doing that. And we don't see our customers changing that appetite. So, I think for those customers, they've emerged more aware of the need to have a primary relationship with a database vendor and still be in cloud. And so I think that's how this evolves to see two different kinds of services side-by-side, what they really want is a Database as a Service from the database vendor, which is what we just announced here at Microsoft Ignite event.Corey: So, talk to me a little bit more about that, where it's interesting in 2021 to see a company launching a managed service offering, especially in the database space, when there's been so much pushback in different ways against the large cloud providers—[cough] Amazon—who tend to effectively lose sleep at night over the haunting fear that someone who isn't them is making money, somehow. And they will take whatever is available to them and turn it into a managed service offering. That's always been the fear, so people play games with licenses and the rest. Well, they've been running Postgres offerings for a long time. It is an independent open-source project.I don't think you can wind up forcing a license change through that says everyone except big companies can run this themselves and don't do a managed service with it because that cat is very much out of the bag. How is it that you're taking something to market now and expecting that to fare competitively?Ed: So, I think there's a few things that our customers are clearly telling us they want, and I think this is the most important thing: they want control of their data. And if you step back, Corey, look at it historically, they made a huge trade to big proprietary database companies, companies like Oracle, and they made that trade actually for convenience. They traded data to that database vendor. And we all know the successes Oracle's had, and the sheer extraordinary expense of those technologies. So, it felt like a walled garden.And that's where EDB and Postgres entered to really change that equation. What's interesting is the re-platforming that happened and the transformation to cloud actually had the same, kind of, binding effect; we now moved all that data over to the public cloud vendors, arguably in an even stickier context, and now I think customers are realizing that's created a dimension of inflexibility. It's also created some—as you rightly pointed out—some deficiencies in technical depth, in database, and in software. So, our customers have sorted that out and are kind of coming back to middle. And what they're saying is, “Well, we want all the advantages of an open-source database like a Postgres, but we want control of the data.”And so what control looks like is more the ability to take one version of that software—in our case, we're worrying about Postgres—and deploy the same thing everywhere they go. And that opens the door up for EDB to be their partner as a traditional on-prem partner, in the cloud where they run our Postgres and they manage it themselves, and as their managed service, Postgres Database as a Service Provider, which is what we're doing.Corey: I've been something of a bear on the idea of, “I'm going to build a workload to run everywhere in every cloud provider,” which I get. I think that's generally foolish, and people chasing that, with remarkably few exceptions, are often going after the wrong thing. That said, I'm also a fan of having a path to strategic Exodus, where Google's Cloud Spanner is fascinating, DynamoDB is revelatory, Cosmos DB is a security nightmare, which is neither here nor there, but the idea that I can take a provider's offering that even if it solves a bunch of problems for me, well, if I ever need to move this somewhere else for any reason, I'm re-architecting, my data model and re-architecting the built-in assumptions around how the database acts and behaves, and that is a very heavy lift. We have proof of that from Amazon, who got up on stage and told a story about how much they hate Oracle, and they're migrating everything off of Oracle to Aurora, which they had to build in order to get off of Oracle, and it took them three years to migrate things. And Oracle loves telling that story, too.And it's, you realize you both sound terrible when you tell that story? It's, “This is a massive undertaking that even we struggle with, so you should probably not attempt it.” Well, what I hear from that is good God, don't wind up getting locked into a particular database that is only available from one source. So, if you're all-in on a cloud provider, which I'm a fan of, personally—I don't care which one but pick a cloud provider—having a database that is not only going to work in that environment is just a reasonable step as far as how I view things. Trading up that optionality has got to pay serious dividends, and in many database use cases, I've just don't see it.Ed: Yeah, I think you're bringing up a really important point. So, let's unpack it for a minute.Corey: Please.Ed: Because I think you brought up some really prominent specialty database technologies, and I'm not sure there's ever a way out of that intersection and commitment to a single vendor if you pick their specialty database. But underneath this is exactly one of the things that we've worried about here at EDB, which is to make Postgres a more capable, robust database in its entirety. A Postgres superpower is its ability to run a vast array of workloads. Guess what, it's not sexy. It's not sexy not to be that specialty database, but it's incredibly powerful in the hands of an enterprise who can do more.And that really creates an opportunity, so we're trying to make Postgres apply to a much broader set of workloads, from traditional systems of record, like your ERP systems; systems of analysis, where people are doing lightweight analytic workloads or reporting, you can think in the world of data warehouse; and then systems of engagement, where customers are interacting with a website and have a database on the backend. All areas Postgres has done incredibly well in and we have customer experience with. So, when you separate out that core capability and then you look at it on a broader scale like Postgres, you realize that customers who want to make Postgres strategic, by definition need to be able to deploy it wherever they want to deploy it, and not be gated or bound by one cloud vendor. And all the cloud vendors picked up Postgres offerings, and that's been great for Postgres and great for enterprises. But that corresponding lock-in is what people want to get away from, at this point.Corey: There's something to be said for acknowledging that there is a form of lock-in as far as technology selection goes. If you have a team of folks who are terrific at one database engine and suddenly you're switching over to an entirely different database, well, folks who spent their entire career working on one particular database that's still in widespread use are probably not super thrilled to stick around for that. Having something that can migrate from environment to environment is valuable and important. When you say you're launching this as a database as a service offering, how does that actually work? Is that going to be running in your own cloud environment somewhere and people just make queries across the wire through standard connections to the database like they would something locally? Are you running inside of their account or environment? Is it something else?Ed: So, this is a fully-managed database as a service, just like you'd get from any cloud vendor or DBAAS vendor that you've worked with in the past, just being managed and run by EDB. And with that, you get lot of the goodies that we bring, including our compatibility, and all our deep Postgres expertise, but I think one of the other important attributes is we're going to run that service in our clients' account, which gives them a level of isolation and a level of independence that we think is really important. And as different as that is, it's not heroic; it's exactly what our customers told us they wanted.Corey: There's something to be said for building the thing that your customers have said that they want and make sense for you to build as opposed to, “We're going to build this ridiculous thing and we're sure folks are going to love it.” It's nice to see that shaping up in the proper order. And I've fallen victim to that myself; I think most technologists have to some extent. How big is EDB these days?Ed: So, we have over 650 employees. Now, around the world, we have 6000 customers. And of the 650 employees, about 300 of those are focused on Postgres. A subset of that are 30-odd core team members in the Postgres community, committers in the Postgres community, major contributors, and contributors in the Postgres community. So, we have a density of technical depth that is really unparalleled in Postgres.Corey: You're not, for lack of a better term, pulling an Amazon, insofar as you're, “Well, we have three people working on open-source projects, so we're going to go ahead and claim we're an open-source company,” in other words. Conversely, you're also not going down the path of this is a project that you folks have launched, and it claims to be open-source because we love it when people volunteer for for-profit entities, but we exercise total control over the project. You have a lot of contributors, but you're also still a minority, I think the largest minority, but still a minority of people contributing to Postgres.Ed: That's right. And, look, we're all-in on Postgres, and it's been that way since I got here. As I mentioned earlier, I came from Red Hat where I was—I was at Red Hat for a little over six years, so I've been an open-source now for 20 years. So, my orientation is towards really powerful, independent open-source projects. And I think we'll see Postgres really be the most transformative open-source technology since Linux.I think we'll see that as we look forward. And you're right, though, I think what's powerful about Postgres is it's an independent project, which means it's supported by thousands of contributors who aren't tied to single companies, around the world. And it just makes the software—we develop innovation faster, and I think it makes the software better. Now, EDB plays a big part in there. Roughly, a little less than a third of the last res—actually, the 13 release—were contributions that came from contributors who came from EDB.So, that's not a majority, and that's healthy. But it's a big part of what helps move Postgres along and there aren't—you know, the next set of companies are much, much—next set of combined contributors add up to quite small numbers. But the cloud vendors are virtually non-existent in that contribution.Corey: This episode is sponsored in part by something new. Cloud Academy is a training platform built on two primary goals. Having the highest quality content in tech and cloud skills, and building a good community the is rich and full of IT and engineering professionals. You wouldn't think those things go together, but sometimes they do. Its both useful for individuals and large enterprises, but here's what makes it new. I don't use that term lightly. Cloud Academy invites you to showcase just how good your AWS skills are. For the next four weeks you'll have a chance to prove yourself. Compete in four unique lab challenges, where they'll be awarding more than $2000 in cash and prizes. I'm not kidding, first place is a thousand bucks. Pre-register for the first challenge now, one that I picked out myself on Amazon SNS image resizing, by visiting cloudacademy.com/corey. C-O-R-E-Y. That's cloudacademy.com/corey. We're gonna have some fun with this one!Corey: Something else that does strike me as, I guess, strange, just because I've seen so many companies try to navigate this in different ways with varying levels of success. I always encountered EDB—even back when it was EnterpriseDB, which was, given their love of acronyms, I'm still somewhat partial to. I get it; branding, it's a thing—but the folks that I engaged with were always there in a consulting service's capacity, and they were great at this. Is EDB a services company or a product company?Ed: Yeah, we are unashamedly a product technology company. Our business is over 90% of our revenue is annually recurring subscription revenue that comes from technical products, database server, mostly, but then various adjacent capabilities in replication and other areas that we add around the database server itself. So no, we're a database technology company selling a subscription. Now, we help our customers, so we do have a really talented team of consultants who help our customers with their business strategy for Postgres, but also with migrations and all the things they need to do to get Postgres up and running.Corey: And the screaming, “Help, help, help, fix it, fix it, fix it now,” emergencies as well.Ed: I think we have the best Postgres support operation in the world. It is a global 24/7 organization, and I think a lot of what you likely experienced, Corey, came out of our support organization. So, our support guys, these guys aren't just handling lightweight issues. I mean, they wade into the gnarly questions and challenges that customers face. But that's a support business for us. So, that's part and parcel. You get that, it's included with the subscription.Corey: I would not be remembering this for 11 years later, if it hadn't been an absolutely stellar experience—or a horrible experience, for that matter; one or the other. You remember the superlatives, not the middle of the road ones—and if it hadn't been important. And it was. It also noteworthy; with many vendors that are product-focused, their services may have an asterisk next to it because it's either a, “Buy our product and then we'll support it,” or it's, “Ohh, we're going to sell you a whole thing just to get us on the phone.” And as I recall, there wasn't a single aspect of upsell involved in this.It was, “Let's get you back up and running and solve the problem.” Sure, later in time, there were other conversations, as all good businesses will have, but there was no point during those crisis moments where it felt like, “Oh, if you had gone ahead and bought this thing that we sell, this wouldn't happen,” or, “You need to buy this or we won't help you.” I guess that's why I've contextualized you folks as a services company, first and foremost.Ed: Well, I'm glad you have that [laugh] experience because that's our goal. And I think—look, this is an interesting point where customers want us to bring that capability to their managed DBAAS world. Step back again, go back to what I said about the big cloud vendors; they are, at their core, infrastructure companies. I mean, they're really good at that. They're not particularly well-positioned to take your Postgres call, and I don't think they want that call.We're the other guys; we want to help you run your Postgres, at scale, on-prem, in the cloud, fully managed in the cloud, by EDB, and solve those problems at the same time. And I think that's missing in the market today. And we can step back and look at this overall cloud evolution, and I think some might think, “Gee, we're into the mature phase of cloud adoption.” I would tell you, since the Red Sox have done well this year, I think in a nine-inning baseball game—for those of your listeners who follow American baseball—we're in, like, the top of the second inning, maybe. Maybe the bottom of the second inning. So, we've been able to listen and learn from the experiences our customers have had. I think that's an incredible advantage as we now firmly plant ourselves in the cloud DBAAS market alongside our robust Postgres capabilities that you experienced.Corey: The world isn't generating less data, and it's important that we're able to access that in a bunch of different ways. And the last time I really was playing with relational databases, you can view my understanding of it as Excel with a weirder interface, and you're mostly there. One thing that really struck me since the last time I went deep into database-land over in the Postgres-squeal world has been just the sheer variety of native data types that it winds up supporting. The idea of, “Here's some JSON. Take this and store it that way,” or it's GIS data that it can represent, or the idea of having data types that are beyond just string or var or whatever other somewhat limited boolean values or whatnot. Without having just that traditional list, which is of course all there as well. It also seems to have extensively improved its coverage that just can only hint to my small mind about these things and what sort of use cases people are really putting these things into.Ed: Yeah, I think this is one of Postgres' superpowers. And it started with Mike Stonebraker's original development of Postgres as an object-relational database. Mike is an adviser to EDB, which has been incredibly helpful as we've continued to evolve our thinking about what's possible in Postgres. But I think because of that core technology, or that core—because of that core technical capability within Postgres, we have been able to build a whole host of data types. And so now you see Postgres being used not just as the context of a traditional relational database, but we see it used as a time-series database. You pointed out a geospatial database, more and more is a document-oriented database with JSON and JSONB.These are all the things that make Postgres have much more universal appeal, universal appeal to developers—which is worth talking about in the recent StackOverflow developer survey, but we can come back to that—and I think universal applicability for new applications. This is what's bringing Postgres forward faster, unlike many of the specialty database companies that you mentioned earlier.Corey: Now, this is something that you can use for your traditional CRUD app, the my first hello world app that returns something from a database, yeah, that stuff works. But it also, for example, has [cyter 00:25:09] data types, where you can say, give me the results where the IP range contains this address, and it'll do that. Before that, you're trying to solve a whole bunch of very messy things in application logic that's generally awful. The database now does that for you automatically, and there's something—well, it would if I were smart and used it instead of storing it as strings because I make terrible life choices, but for sensible people, it solves a lot of those problems super well. And it's taken the idea of where logic should live in application versus database, and sort of turn a lot of those assumptions I was starting my career with on their head.Ed: Yeah, I think if you look now at the appeal of Postgres to developers, which we've paid a lot of attention to—one of our stated strategies at EDB is to make Postgres easier. That's been true for many years, so a drive for engineering and development here has been that call to action. And if you measure that, over time, we've been contributing—not alone, but contributing to making Postgres more approachable, easier to use, easier to engage with. Some of those things we do just through edb.com, and the way we handle EDB docs is a great example of that, and our developer advocacy and outreach into adjacent communities that care about Postgres. But here's where that's landed us. If you looked at the last Stack Overflow developer survey—the 2021 Stack Overflow developer survey, which I love because I think it's very independent-oriented—and they surveyed, I think this past year was 80,000 developers.Corey: Oh yeah, if Stack Overflow is captured by any particular constituency, it's got to be ‘Big Copy and Paste' that is really behind them. But yeah, other than the cabal of keyboard manufacturers for those copy-and-paste stories, yeah, they're fairly objective when it comes to stuff like this.Ed: And if you look at that survey, Corey, if you just took and summed it because it's helpful to sum it, most used, most loved, and most wanted database: Postgres wins. And I find it fascinating that if you—having been here, in this company for 13 years and watch the evolution from—you know, 13 years ago, Postgres needed help, both in terms of its awareness in the market and some technical capabilities it just lacked, we've come so far. For that to be the new standard for developers, I think, is a remarkable achievement. And I think it's a representation of why Postgres is doing so well in the market that we've long served, in the cloud market that we are now serving, and I think it speaks to what's ahead as a transformational database for the future.Corey: There really is something to be said for a technology as—please don't take this term the wrong way—old. As a relational database, Postgres has been around for a very long time, but it's also not your grandparents' Postgres. It is continuing to evolve. It continues to be there in a bunch of really interesting ways for developers in a variety of different capacities, and it's not the sort of thing that you're only using in, “Legacy environments,” quote-unquote. Instead, it's something that you'll see all over the place. It is rare that I see an environment that doesn't have Postgres in it somewhere these days.Ed: Yeah, I think quite the contrary to the old-school database, which I love that; I love that shade because when you step away from it, you realize, the Postgres community represents the very best of what's possible with open-source. And that's why Postgres continues to accelerate and move forward at the rate that it does. And obviously, we're proud to be a contributor to that, so we don't just watch that outcome happen; we're actually part of creating it. But I also think that when you see all that Postgres has become and where it's going, you really start to understand why the market is adopting open-source.Corey: It's one of those areas where even if some company comes out with something that is amazing and transformatively better, and you should jump into it with both feet and never look back, yeah, it turns out that it takes a long time to move databases, even when they're terrible. And you can lobby an awful lot of accusations at Postgres—or Postgres-squeal—but you can't call it terrible. It's used in enough interesting applications by enough large-scale companies out there—and small as well—that it's very hard to find a reason not to explore it. It's my default relational database when Route 53 loses steam. It just makes sense in a bunch of ways that other things really didn't for me before.Ed: Yeah, and I think we'll continue to see that. And we're just going to keep making Postgres better. And it gets better because of that intersection, as I mentioned, that intimate intersection between enterprise users, and the project, and the community, and the bridge that a company like EDB provides for that. That's why it'll get better faster; the breadth of use of Postgres will keep it accelerating. And I think it's different than many of the specialty databases.Look, I've been in open-source now for 20 years and it's intriguing to me how many new specialty open-source databases have come to market. We tend to forget the amount of roadkill we've had over the course of the past ten years of some of those open-source projects and companies. We certainly are tuned into some of the more prolific ones, even today. And I think again, here again, this is where Postgres shines, and where I think Postgres is a better call for a long-term. Just like Linux was.Corey: I want to thank you for taking so much time out of your day to talk to me about databases, which given my proclivities, is probably like pulling teeth for you. If people want to learn more, where can they find you?Ed: So, come to enterprisedb.com. You still get EnterpriseDB, Corey. Just come to enterprise—Corey: There we go. It's hidden in the URL, right in plain sight.Ed: Come to enterprisedb.com. You can learn all the things you need about the technology, and certainly more that we can do to help you.Corey: And we will, of course, put links to that in the [show notes 00:31:10]. Thank you once again for your time. I really do appreciate it.Ed: Thanks, Corey. My pleasure.Corey: Ed Boyajian, CEO of EDB. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with a long angry comment because you are one of the two Amazonian developers working on open-source databases.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

On The Cloud Pod this week, Jonathan reveals his love for “Twilight.” Plus GCP kicks off Google Cloud Next and announces Google Distributed Cloud, and Azure admits to a major DDoS attack.  A big thanks to this week's sponsors: Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. JumpCloud, which offers a complete platform for identity, access, and device management — no matter where your users and devices are located.  This week's highlights

Mark Mirchandani is back this week with cohost Bukola Ayodele. We’re talking with Eric Schmidt and Bruno Aziza about all the awesome new analytics, data, and AI launches from last week’s Google Cloud Next conference. Our guests start the show outlining the challenges clients face when storing, organizing, and analyzing data in the cloud. These needs have inspired Google solutions that focus on simplifying data management for customers. Next announcements like BigQuery Omni, which helps customers achieve full data visibility with cross-cloud analytics, and DataPlex, which facilitates data management at scale, will change the way companies think about their data. BigQuery integration with AppSheets and the new Cloud Looker LookML let customers build once and access from anywhere. The new Looker and Tableau integration revolutionizes the use of the semantic model in Tableau, allowing things like company-established data governance and the Looker Blocks ecosystem to pull into Tableau analysis. New Looker Blocks specifically targeted to the healthcare industry were also introduced at Next. We talk about the ML announcements including Vertex AI Workbench, a fully-managed service used for data exploration aimed at simplifying the workloads of data engineers. Serverless Spark on Google Cloud shares these goals by making performance tuning, scaling, infrastructure provisioning, and other tasks fully-managed. The new PostgreSQL interface for Spanner lets clients use tools already developed in PostgreSQL while leveraging the global scaling and other benefits of Spanner. Bruno and Eric share some favorite customer stories as we wrap up this week’s episode. Albertson’s, Renault, and others have interesting data journeys on Google Cloud and our listeners can learn more in the YouTube series hosted by Bruno. Eric Schmidt Eric is the the Head of Advocacy for Data Analytics at Google and has been at Google for almost eight years. Previously, he was with Microsoft, where he led Advocacy and Evangelism there, too. He focuses on products like BigQuery, Dataflow, Dataproc and leads a team of advocates who help customers turn data into value. In his downtime, Eric is a DJ at 90.3 KEXP here in Seattle or online at kexp.org where he focuses on global music culture. You can find Eric on Twitter. His handle is “not that eric” - not to be confused with the ‘other Eric Schmidt' here at Google. In fact, internally, people affectionately call him “cloud E”. Bruno Aziza Bruno is the Head of Data & Analytics at Google Cloud. He specializes in everything data, from data analytics, to business intelligence, data science, and artificial intelligence. Before working at Google, he worked at companies like Business Objects when it went IPO and Oracle, where his team led a big turnarounds in the business analytics industry. Bruno also had the opportunity to help launch startups like Alpine Data (now part of Tibco). Sisense and AtScale and helped Microsoft grow its Data unit into a $1B business. He has been educated in the US, France, the UK, and Germany and has written two books on Data Analytics and Performance Management. In his spare time, Bruno writes a monthly column on Forbes.com on everything Data, AI and Analytics. Cool things of the week Next Reaction: Security and zero-trust announcements blog Next Reaction: New Data Cloud launches blog Next Reaction: Making multicloud easier for all blog Next Reaction: Features to reduce IT carbon emissions and build climate-related solutions blog Next Reaction: Monitor your conversations, get started with CCAI Insights blog Interview GCP Podcast Episode 266: Data Analytics Launches with Bruno Aziza and Eric Schmidt podcast BigQuery site Bringing multi-cloud analytics to your data with BigQuery Omni blog Google Cloud Next—Day 1 livestream - WalMart video Dataplex site AppSheet site Cloud Looker LookML site Tableau site Vertex AI site Vertex AI Workbench site TensorFlow site Apache Spark on Google Cloud site New PostgreSQL Interface makes Cloud Spanner's scalability and availability more open and accessible blog PostgreSQL site Cloud Spanner site Google Earth Engine site Google Maps Platform site Inside Industry Data Management 4.0 at Renault site Chess.com site Google Next Opening Keynote site Data Journeys with Bruno Aziza videos Cloud Next Catalog site Bruno’s Cloud Next Playlist videos Cloud Next Data Analytics Playlist videos Bruno on Linkedin site Lak on Twitter site What’s something cool you’re working on? Bukola is working on the Click to Deploy video series.

Your hosts Stephanie Wong and Alicia Williams talk about BigQuery Admin Reference Guides with guest Leigha Jarett. Leigha tells us a bit about the origins of the Admin Reference Guide, which was developed to address frequent customer questions. The series of guides and videos covers onboarding topics from resource hierarchy and APIs to BigQuery tables and storage in an effort to help new admins get started. The team’s Reference Guide on tables helps admins understand the difference between federated and native tables, and Leigha tells our listeners some tips for using each type. Slots and reservations for workload management in BigQuery can be challenging to understand, but these Reference Guides and accompanying videos offer clear explanations. Optimizing projects for speed and monetary efficiency are important topics for any client, and Leigha and the optimization team have addressed these concerns as well. Tips for monitoring, data governance, and the secure sharing of data are also included in their video series, BigQuery Spotlight. We wrap up with a discussion on BigQuery APIs and how easy it is to integrate BigQuery and other Google products. Leigha Jarett Leigha is a developer advocate on the Google Cloud Data & Analytics team. She focuses on making sure developers using tools like BigQuery and Looker are getting the most possible value from their data. Cool things of the week Analyze Cloud Spanner data in BigQuery with federated queries docs Release notes dataset in BigQuery docs and XML feed Google Cloud release notes docs and XML feed Release notes in Cloud Console site Top 25 Google Search terms, now in BigQuery blog Interview BigQuery Admin Reference Guide Blog Recap site BigQuery Admin Reference Guide blog posts site BigQuery Spotlight Video Series videos BigQuery site BigQuery Documentation docs Cloud Spanner site Data Catalog site Apps Script site Looker site What’s something cool you’re working on? Alicia is building new BigQuery architectures in order to understand the journey and identify potential pain areas that may need more support.

The adoption of cloud databases is accelerating, driven by business transformation and the need for database modernization. In this episode of the Cloud Database Report Podcast, founding editor John Foley talks with Andi Gutmans, Google Cloud's GM and VP of Engineering for Databases, about the platforms and technologies that organizations are using to build and manage these new data environments. Gutmans is responsible for development of Google Cloud's databases and related technologies, including Bigtable, Cloud SQL, Spanner, and Firestore. In this conversation, he discusses the three steps of cloud database adoption: migration, modernization, and transformation. "We're definitely seeing a tremendous acceleration," he says. Gutmans talks about the different types of database migrations, from "homogenous" migrations that are relatively fast and simple to more complex ones that involve different database sources and target platforms. He reviews the tools and services available to help with the process, including Google Cloud's Database Migration Service and Datastream for change data capture. Gutmans provides an overview of the "data cloud" model as a comprehensive data environment that connects multiple databases and reduces the need for organizations to build their own plumbing. Data clouds can "democratize" data while providing security and governance. Looking ahead, Google Cloud will continue to focus on database migrations, developing new enterprise capabilities, and providing a better experience for developers. 

Brian Dorsey and Mark Mirchandani are talking intro to Firestore this week with fellow Googler Allison Kornher. Allison, a Cloud Technical Resident, starts the show telling us about the program and how it brought her to Firestore. Allison takes us through the differences between SQL and NoSQL databases and describes the four categories of NoSQL databases: family, document, key value, and graph. Firestore is a scalable, flexible NoSQL document database. To illustrate the uses and benefits of Firestore, Allison walks us through a delicious pizza example. Each document in the database belongs to a collection, which is used to organize these documents. Firestore documents are assigned an identifier and can be quickly changed and called within their collections. Because these documents are stored in an implicit schema in key value pairs, developers have control over the details of database organization and data change and growth are easy to manage. The availability of subcollections further adds to the flexibility of Firestore database design. Choosing a database type will depend on the situation, and Allison suggests this starts with a look at CAP theorem. If a document database is your database of choice, Allison gives our listeners tips for getting started with Firestore and clearing any hurdles along the way. Allison Kornher Allison is a Cloud Technical Resident and has worked helping startups looking to join GCP and in the Premium Tier Cloud Support organization with a focus on Storage. Cool things of the week BigQuery admin reference guide: Tables & routines blog Top 25 Google Search terms, now in BigQuery blog Three security and scalability improvements for Cloud SQL for SQL Server blog GCP Podcast Episode 247: Cloud SQL Insights with Nimesh Bhagat podcast GCP Podcast Episode 163: Cloud SQL with Amy Krishnamohan podcast Interview Cloud Firestore site Cloud Firestore Documentation docs Cloud Firestore explained: for users who never used Firestore before blog Gabi on Twitter site Datastore site BigTable site Firebase Realtime Database site Memorystore site Cloud Spanner site GCP Podcast Episode 248: Cloud Spanner Revisited with Dilraj Kaur and Christoph Bussler podcast All you need to know about Firestore: A cheatsheet blog What’s something cool you’re working on? Brian has been working on sharing a persistent disk between Google Compute Engine VMs. Cloud Storage site Cloud Filestore site Cloud SQL site

This week on The Cloud Pod, Ryan is stuck somewhere in a tent under a broken-down motorcycle but is apparently still having fun.         A big thanks to this week's sponsors: Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights Amazon went back to school to become a detective. Google was voted prom queen at the virtual homecoming. Oracle shocks everyone with its new look. General News: Great Partners Hashicorp has partnered with AWS to launch support for predictive scaling policy in the Terraform AWS provider. This will be hugely popular for people new to the cloud.     Amazon Web Services: Dropping Stories For No Reason AWS Lambda Extensions are now generally available with new performance improvements. This has pretty limited regional availability, though.    Amazon releases the AWS Shield threat landscape 2020 year in review. One of our favourite blogs. AWS EKS Add-Ons now supports CoreDNS and kube-proxy. This is neat! Introducing the AWS Application Cost Profiler — there have been a few complaints about this on Twitter. AWS Compute Optimizer launches updates to its EC2 instance type recommendations. This is awesome. AWS Outposts launches support for EC2 Capacity Reservations. Being able to use the same tool regardless of where you are is a good thing!   An AWS Region in the United Arab Emirates (UAE) is in the works. Great!       Google Cloud Platform: Prom Queen 2021 Google VM Manager with OS configuration management is now in Preview. This is basically patch and agent management.   Forrester names Google Cloud a leader in Unstructured Data Security Platforms. Good job, Google! Google has released a better way to manage firewall rules with Firewall Insights. We just want a firewall manager that does everything for us.  Google announces new BigQuery user-friendly SQL launches. Thanks but no thanks.  Azure: Selling No-Code To Developers Azure gains 100th compliance offering — protecting data with EU Cloud Code of Conduct. Now we know why France was so happy last week. Azure announces preview capabilities of Azure Application Services to run on K8 anywhere. We're really surprised by how quickly the cloud providers have embraced hybrid infrastructure.  Azure releases several new features to empower developers to innovate with Azure Database services. We need to bring the tumbleweed sound effect back.  Accenture, GitHub, Microsoft and ThoughtWorks launch the Green Software Foundation with the Linux Foundation. So they're anti-Bitcoin mining? Microsoft uses GPT-3 to add AI features to Power Apps. For developers who don't code.  Microsoft's new research lab studies developer productivity and well-being. We'll see what happens.    Oracle: One We're Actually Excited About Introducing Arm on Oracle Cloud Infrastructure. The free tier is amazing! TCP Lightning Round Justin really appreciates Jonathan for handing him an easy win and takes this week's point, leaving scores at Justin (9), Ryan (4), Jonathan (7).  Other headlines mentioned: Amazon Forecast now supports generating predictions for 5X more items using 3X more historic data points Amazon Elastic File System now supports longer resource identifiers AWS X-Ray now supports VPC endpoints Announcing enhancements to Amazon Rekognition text detection — support for more words, higher accuracy and lower latency Amazon CloudWatch Application Insights now supports container monitoring Customizations for AWS Control Tower v2.1 adds more scaling optimizations and improves compatibility with AWS CodeBuild Amazon EventBridge now supports sharing events between event buses in the same account and Region Amazon SageMaker Pipelines is now integrated with Amazon SageMaker Experiments Amazon Braket introduces quantum circuit noise simulator, DM1  AWS Transfer Family now supports Microsoft Active Directory Amazon EMR now supports Amazon EC2 On-Demand Capacity Reservations The Microsoft Build of OpenJDK is now generally available Public preview: Azure Confidential Ledger  Google now allows you to Test Dataflow pipelines with the Cloud Spanner emulator  Things Coming Up Announcing Google Cloud 2021 Summits [frequently updated] Harness Unscripted Conference — June 16–17 Google Cloud Next — Not announced yet (one site says Moscone is reserved June 28–30) Amazon re:Inforce — August 24-25 — Houston TX Google Cloud Next 2021 — October 12–14, 2021 AWS re:Invent — November 29–December 3 — Las Vegas

On The Cloud Pod this week, the team admits to using the podcast as a way to figure out what day it is. Justin also relents and includes Azure news because he couldn't handle any more Oracle mobile apps announcements.  A big thanks to this week's sponsors: Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights Social media influencers can breathe a sigh of relief.  Amazon is dangling a carrot in front of one of its partners.  Azure is throwing a spanner in the works. General News: Not Cool News The FBI arrests a man for his plan to kill “70% of the internet” in an AWS bomb attack. 70% is quite a stretch but we're sure it would have caused a crappy day for a lot of people.   Hashicorp has released its Boundary 0.2 release with several new features. We're really excited about this.  Announcing HashiCorp Terraform 0.15 General Availability. If you believe it, this is really great news.  Amazon Web Services: Good At Compromising AWS announces AQUA is now generally available. Justin should have gotten a prediction point for this one.   Amazon Managed Service for Grafana now offers more support. We'll see if Grafana can actually make money out of its partnership with Amazon. Amazon RDS for PostgreSQL now integrates with AWS Lambda. This is really cool!  Decrease machine learning costs with instance price reductions and savings plans for Amazon SageMaker. Some pretty significant savings here.   Google Cloud Platform: Colossal Google takes a deep dive into its scalable storage solution, Colossus. Nothing new here.  Google announces tracking index backfill operation progress in Cloud Spanner. This is super important.   The new Google Cloud region in Warsaw is open. Nice to see Eastern Europe getting another region.  Azure: Someone Out There Cares User data through Azure Instance Metadata Service is now generally available. It would be great to use this with VMWare.  Microsoft announces encryption is now supported at the host level with AKS. Compliance people will be happy with this one. Microsoft announces plans to establish its first datacenter region in Malaysia. Just an announcement — don't get too excited because it's not opening yet.  TCP Lightning Round Jonathan takes the cake and this week's point, leaving scores at Justin (6), Ryan (3), Jonathan (6).  Other headlines mentioned: Amazon Pinpoint is now FedRAMP High Compliant You can now use macros and transforms in CloudFormation templates to create AWS CloudFormation StackSets Amazon Macie adds CloudWatch logging for job status and health monitoring of sensitive data discovery jobs  Amazon Textract achieves FedRAMP compliance Now visualize and report patch compliance using AWS Systems Manager Patch Manager     Things Coming Up Discover cloud storage solutions at Azure Storage Day — April 29 Save the date: AWS Containers events in May AWS Regional Summits — May 10–19 AWS Summit Online Americas — May 12–13 Microsoft Build — May 19–21 (Digital) Google Financial Services Summit — May 27th  Harness Unscripted Conference — June 16–17 Google Cloud Next — Not announced yet (one site says Moscone is reserved June 28–30) Google Cloud Next 2021 — October 12–14, 2021 AWS re:Invent — November 29–December 3 — Las Vegas Oracle Open World (no details yet) 

On The Cloud Pod this week, the team discusses the future of the podcast and how they'll know they've made it when listeners use Twitter to bombard Ryan with hatred when he's wrong.  A big thanks to this week's sponsors: Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights Amazon gives Justin a long overdue birthday present. Google wants to educate the people. Azure has a new best friend but could they be a wolf in sheep’s clothing? General News: Goodbye, Friend The Apache foundation has decided to send Mesos to the attic. This makes us sad because we loved the concept. Amazon Web Services: Happy Birthday, Justin New AWS WAF Bot Control to reduce unwanted website traffic. This is great! AWS is releasing the Amazon Route 53 Resolver DNS firewall to defend against DNS-level threats. Pricing is interesting on this one. AWS launches CloudWatch Metric Streams. After years of complaints, they're finally fixing this issue.  AWS Lambda@Edge changes duration billing granularity from 50ms down to 1ms. Nice price cut! AWS Direct Connect announces MACsec encryption for dedicated 10Gbps and 100Gbps connections at select locations. AWS has fulfilled their promise to Justin — three years later. Amazon announces new predictable pricing model up to 90% lower and Python Support moves to GA for CodeGuru Reviewer. If this goes down next week, blame Ryan.  Google Cloud Platform: So Pretty Google is releasing an open-source set of JSON dashboards. This is super important.   Google announces free AI and machine learning training for fraud detection, chatbots and more. We recommend you check these out.   Google Clouds Database Migration Service is now generally available. Everything is so beautiful on paper.    Google introduces request priorities for Cloud Spanner APIs. This just reinforces the fact that we don't know how Cloud Spanner works.   Azure: Best Friends Microsoft’s new low-code programming language, Power FX, is in public preview. Terrible name. Microsoft announces new solutions for Oracle WebLogic on Azure Virtual Machines. They're running WebLogic on Azure because of some product requirement.    The U.S. Army moves Microsoft HoloLens-based headset from prototyping to production phase. You don't get JEDI, but you get HoloLens! Microsoft launches Azure Orbital to deepen the value chain for geospatial earth imagery on cloud. Reminded us to watch Lord of War again, it's a good movie. Oracle: Win Dinner With Larry Oracle offers free cloud migration to lure new customers. Oracle CEO Larry Ellison will fly you to his private island — but if you don't sign up, you have to make your own way back.  Oracle and Microsoft expand interconnection to Frankfurt, adding a third location in EMEA. Don't invite Oracle into your data center.   TCP Lightning Round Anyone who makes fun of the Canadian accent wins so Justin takes this week's point and the lead, leaving scores at Justin (5), Ryan (3), Jonathan (5).  Other headlines mentioned: Azure Kubernetes Service (AKS) now supports node image autoupgrade in public preview Public preview of Azure Kubernetes Service (AKS) run-command feature  Amazon WorkSpaces webcam support now generally available Amazon VPC Flow Logs announces out-of-the-box integration with Amazon Athena AWS WAF now supports Labels to improve rule customization and reporting Amazon EKS is now FedRAMP-High Compliant AWS Budgets announces CloudFormation support for budget actions AWS Systems Manager Parameter Store now supports easier public parameter discoverability AWS Systems Manager Run Command now displays more logs and enables log download from the console Amazon EC2 now allows you to copy Amazon Machine Images across AWS GovCloud, AWS China and other AWS Regions AWS Systems Manager Parameter Store now supports removal of parameter labels  Announcing Amazon Forecast Weather Index for Canada  Things Coming Up Public Sector Summit Online — April 15–16 Discover cloud storage solutions at Azure Storage Day — April 29 AWS Regional Summits — May 10–19 AWS Summit Online Americas — May 12–13 Microsoft Build — May 19–21 (Digital) Google Financial Services Summit — May 27th  Harness Unscripted Conference — June 16–17 Google Cloud Next — Not announced yet (one site says Moscone is reserved June 28–30) Google Cloud Next 2021 — October 12–14, 2021 AWS re:Invent — November 29–December 3 — Las Vegas Oracle Open World (no details yet) 

Google Cloud, AWS y Azure son los proveedores de servicios en la nube dominantes en el mercado actual. El mercado está altamente competitivo y existe un solapo significativo entre los servicios ofrecidos por estos tres proveedores principales. En función del dominio, los profesionales de los datos buscan normalmente una plataforma que se ajuste a sus necesidades y a sus casos de uso. En el episodio de SaaS Product Chat de esta semana hablamos específicamente de las capacidades de Google Cloud Platform y de la variedad de productos y servicios que ofrece en la nube.No olvides comentar aquí en YouTube y sugerir temas o invitados que desearíamos tener en el show.Estos son los enlaces a los temas de los que hemos hablado:Google Cloud: https://cloud.google.com/Andi Gutmans es el director general y VP de ingeniería para bases de datos en Google: https://softwareengineeringdaily.com/2021/03/16/google-cloud-databases-with-andi-gutmans/Estructurando Datos en Cloud Firestore: https://medium.com/canariasjs/estructurando-datos-en-cloud-firestore-3dce2cb1ceaeModelo de datos de Cloud Firestore: https://firebase.google.com/docs/firestore/data-model?hl=esLessons learned from evaluating Cloud Spanner at Uber scale: https://youtu.be/bNqmEnx6ESEFrancisco Solsona, líder de relación con desarrolladores de Google en Latinoamérica y el mundo de habla hispana, en el podcast de TheVentureCity, "Citizen": https://cuonda.com/citizen/cap-5-francisco-solsona-google-devs-splatam-y-la-situacion-del-ecosistema-emprendedorTimnit Gebru sobre desarrollos e investigaciones en torno al reconocimiento facial: https://soundcloud.com/the-impact-podcast/epsiode-93-facial-recognition-demographic-analysis-more-with-timnit-gebruTuit de Jeff Dean: https://twitter.com/jeffdean/status/1270961033616617473Bigquery, el almacén de datos de Google Cloud (Hablemos en Cloud): https://www.youtube.com/watch?v=kud8YDvBKHEGoogle Cloud Platform Podcast: https://podcasts.google.com/?feed=aHR0cHM6Ly9mZWVkcy5mZWVkYnVybmVyLmNvbS9HY3BQb2RjYXN0¿Qué es Kubernetes? https://kubernetes.io/es/docs/concepts/overview/what-is-kubernetes/Despliegue de aplicaciones sobre Google Cloud Platform con Kubernetes: https://youtu.be/XORU80znx9QSíguenos en Twitter:Danny Prol: https://twitter.com/DannyProl/Claudio Cossio: https://twitter.com/ccossioEstamos en todas estas plataformas:Apple Podcasts: https://podcasts.apple.com/ca/podcast/saas-product-chat/id1435000409ListenNotes: https://www.listennotes.com/podcasts/saas-product-chat-daniel-prol-y-claudio-CABZRIjGVdP/Spotify: https://open.spotify.com/show/36KIhM0DM7nwRLuZ1fVQy3Google Podcasts: https://podcasts.google.com/?feed=aHR0cHM6Ly9mZWVkcy5zaW1wbGVjYXN0LmNvbS8zN3N0Mzg2dg%3D%3D&hl=esBreaker: https://www.breaker.audio/saas-product-chatWeb: https://saasproductchat.com/

Mark Mirchandani and Stephanie Wong are back this week as we learn about all the new things happening with Google Cloud Spanner. Our guests this week, Dilraj Kaur and Christoph Bussler, describe Cloud Spanner as a fully managed relational database that boasts unlimited scaling and advanced consistency and availability. Unlimited scaling truly means unlimited, and Chris explains why Cloud Spanner offers this feature and how it’s making database design and development easier. Dilraj and Chris tell us all about the cool new features Spanner has developed, like generated columns and foreign keys, and how customer needs influenced these developments. Chris walks us through the process of using some of these new features, including how developers can monitor their database systems. Managed backups and multi-region configuration are additional recent additions to Cloud Spanner, and our guests explain how these are used by current enterprise clients. Dilraj and Chris explain the automatically managed features of Spanner versus the customer managed features and how people set up and manage database projects. We hear examples of companies using Cloud Spanner and how it has improved their businesses. Dilraj Kaur Dilraj Kaur is an Enterprise Customer Engineer with specialization in Data Management. She has been with Google for about 2.5 years and is based in Atlanta. Christoph Bussler As a Solutions Architect Chris is focusing on databases, data migration and data integration in enterprise customer settings. See his professional work and background on his website. Cool things of the week New to Google Cloud? Here are a few free trainings to help you get started blog Start your skills challenge today site Service Directory is generally available: Simplify your service inventory blog Interview Google Cloud Spanner site GCP Podcast Episode 62: Cloud Spanner with Deepti Srivastava podcast Using the Cloud Spanner Emulator docs Cloud Spanner Ecosystem site Cloud Spanner Qwiklabs site Google Cloud Platform Community On Slack site Creating and managing generated columns docs WITH Clause docs Foreign Keys docs Numeric Data Type docs Information schema docs Overview of introspection tools docs Backup and Restore docs Multi-region configurations docs ShareChat: Building a scalable data-driven social network for non-English speakers globally site Blockchain.com: Streamlining infrastructure for the world’s most dynamic financial market site What is Cloud Spanner? video What’s something cool you’re working on? Mark has been working on budgeting blog posts, including Protect your Google Cloud spending with budgets. Stephanie is working on her data center animation series

For the last eight years, Jaana Dogan has been building developer products at Google. Currently, she is a staff engineer and technical advisor working on Spanner, a relational database that’s globally scalable. Previously, Jaana worked as a software architect at Tart, a software engineer at Tikle, and a software engineer at Microsoft. She also founded Rootapi, a company that built rich editing tools for internet publishing companies. Join Corey and Jaana as they talk about Spanner and all things database, why Jaana believes that five nines is extreme for most businesses, the CAP theorem and what it actually means, the difference between Google’s internal Spanner product and the Cloud Spanner product you can buy with someone else’s credit card, how Google designs all of its major releases with scalability in mind, the role Jaana played in the Go community, what Jaana loves about working at Google, Jaana’s career advice, and more.

In this episode of The Podlets Podcast, we are talking about the very important topic of recovery from a disaster! A disaster can take many forms, from errors in software and hardware to natural disasters and acts of God. That being said that are better and worse ways of preparing for and preventing the inevitable problems that arise with your data. The message here is that issues will arise but through careful precaution and the right kind of infrastructure, the damage to your business can be minimal. We discuss some of the different ways that people are backing things up to suit their individual needs, recovery time objectives and recovery point objectives, what high availability can offer your system and more! The team offers a bunch of great safety tips to keep things from falling through the cracks and we get into keeping things simple avoiding too much mutation of infrastructure and why testing your backups can make all the difference. We naturally look at this question with an added focus on Kubernetes and go through a few tools that are currently available. So for anyone wanting to ensure safe data and a safe business, this episode is for you! Follow us: https://twitter.com/thepodlets Website: https://thepodlets.io Feeback: info@thepodlets.io https://github.com/vmware-tanzu/thepodlets/issues Hosts: https://twitter.com/carlisiahttps://twitter.com/bryanlhttps://twitter.com/joshrossohttps://twitter.com/opowero Key Points From This Episode: • A little introduction to Olive and her background in engineering, architecture, and science. • Disaster recovery strategies and the portion of customers who are prepared.• What is a disaster? What is recovery? The fundamentals of the terms we are using.• The physicality of disasters; replication of storage for recovery.• The simplicity of recovery and keeping things manageable for safety.• What high availability offers in terms of failsafes and disaster avoidance.• Disaster recovery for Kubernetes; safety on declarative systems.• The state of the infrastructure and its interaction with good and bad code.• Mutating infrastructure and the complications in terms of recovery and recreation. • Plug-ins and tools for Kubertnetes such as Velero.• Fire drills, testing backups and validating your data before a disaster!• The future of backups and considering what disasters might look like. Quotes: “It is an exciting space, to see how different people are figuring out how to back up distributed systems in a reliable manner.” — @opowero [0:06:01] “I can assure you, careers and fortunes have been made on helping people get this right!” — @bryanl [0:07:31] “Things break all the time, it is how that affects you and how quickly you can recover.” —@opowero [0:23:57] “We do everything through the Kubernetes API, that's one reason why we can do selectivebackups and restores.” — @carlisia [0:32:41] Links Mentioned in Today’s Episode: The Podlets — https://thepodlets.io/The Podlets on Twitter — https://twitter.com/thepodletsVMware — https://www.vmware.com/Olive Power — https://uk.linkedin.com/in/olive-power-488870138Kubernetes — https://kubernetes.io/PostgreSQL — https://www.postgresql.org/AWS — https://aws.amazon.com/Azure — https://azure.microsoft.com/Google Cloud — https://cloud.google.com/Digital Ocean — https://www.digitalocean.com/SoftLayer — https://www.ibm.com/cloudOracle — https://www.oracle.com/HackIT — https://hackit.org.uk/Red Hat — https://www.redhat.com/Velero — https://blog.kubernauts.io/backup-and-restore-of-kubernetes-applications-using- heptios-velero-with-restic-and-rook-ceph-as-2e8df15b1487CockroachDB — https://www.cockroachlabs.com/Cloud Spanner — https://cloud.google.com/spanner/ Transcript: EPISODE 08[INTRODUCTION] [0:00:08.7] ANNOUNCER: Welcome to The Podlets Podcast, a weekly show that explores Cloud Native one buzzword at a time. Each week, experts in the field will discuss and contrast distributed systems concepts, practices, tradeoffs and lessons learned to help you on your cloud native journey. This space moves fast and we shouldn’t reinvent the wheel. If you’re an engineer, operator or technically minded decision maker, this podcast is for you. [EPISODE] [00:00:41] CC: Hi, everybody. We are back. This is episode number 8. Today we have on the show myself, Carlisia Campos and Josh. [00:00:51] JR: Hello, everyone. [00:00:52] CC: That was Josh Rosso. And Olive Power. [00:00:55] OP: Hello. [00:00:57] CC: And also Brian Lyles. [00:00:59] BL: Hello. [00:00:59] CC: Olive, this is your first time, and I didn’t even give you a heads-up. But tell us a little bit about your background. [00:01:06] OP: Yeah, sure. I’m based in the UK. I joined VMware as part of the Heptio acquisition, which I joined Heptio way back last year in October. The acquisition happened pretty quickly for me. Before that, I was at Red Hat working on some of their cloud management tooling and a bit of OpenShift as well. Before that, I worked with HP and Fujitsu. I kind of work in enterprise management a lot, so things like desired state and automation are kind of things that have followed me around through most of my career. Coming in here to VMware, working in the cloud native applications business unit is kind of a good fit for me. I’m a mom of two and I’m based in the UK, which I have to point out, currently undergoing a heat wave. We’ve had about like 3 weeks of 25 to 30 degrees, which is warm, very warm for us. Everybody is in a great mood. [00:01:54] CC: You have a science background, right? [00:01:57] OP: Yeah, I studied chemistry in university and then I went on to do a PhD in cancer research. I was trying to figure out ways where we could predict how different people will going to respond to radiation treatments and then with a view to tailoring everybody’s treatment to make it unique for them rather than giving the same treatment to different who present you with the same disease but were response very, very different. Yeah, that was really, really interesting. [00:02:22] CC: What is your role at VMware? [00:02:23] OP: I’m a cloud native architect. I help customers predominantly focus on their Kubernetes platforms and how to build them either from scratch or help them get more production-ready depending on where they are in their Kubernetes journey. It’s been really exciting part of being part of Heptio and following through into the VMware acquisition. We’re going to speak to customers a lot at very exciting times for them. They’re kind of embarking on their Kubernetes journey a lot of them. We’re with them from the start and every step of the way. That’s really rewarding and exciting. [00:02:54] CC: Let me pick up on that thread actually, because one thing that I love about this group for me, because I don’t get to do that. You all meet customers and you know what they are doing. Get that knowledge first-hand. What would you say the percentage of the clients that you see, how disaster recovery strategy, which by the way is a topic of today’s show. [00:03:19] OP: I speak to customers a lot. As I mentioned earlier, a lot of them are like in different stages of their journey in terms of automation, in terms of infrastructure of code, in terms of where they want to go for their next platform. But there generally in the room a team that is responsible for backup and recovery, and that’s generally sort of leads into this storage team really because you’re trying to backup state predominantly. When we’re speaking to customers, we’ll have the automation people in the room. We’ll have the developers in the room and we’ll have the storage people in the room, and they are the ones that are primarily – Out of those three sort of folks I’ve mentioned, they’re the ones that are primarily concerned about backup. How to back up their data. How to restore it in a way that satisfies the SLAs or the time to get your systems back online in a timely manner. They are the force concerned with that. [00:04:10] JR: I think it’s interesting, because it’s almost scary how many of our customers don’t actually have a disaster recovery strategy of any sort. I think it’s often times just based on the maturity of the platform. A lot of the applications and such, they’re worried about downtime, but not necessarily like it’s going to devastate the business in a lot of these apps. I’m not trying to say that people don’t run mission critical apps on things like Kubernetes. It’s just a lot of people are very new and they’re just kind of ramping up. It’s a really complicated thing that we work with our customers on, and there’re so many like layers to this. I’m sure layers that we’ll get into. There are things like disaster recovery of the actual platform. If Kubernetes, as an example, goes down. Getting it back up, backing up its data store that we call etcd. There’s obviously like the applications disaster recovery. If a cluster of some sort goes own, be it Kubernetes or otherwise, shifting some CI system and redeploying that into some B cluster to bring it back up. Then to Olive’s point, what she said, it all comes back to storage. Yeah. I mean, that’s where it gets extremely complicated. Well, at least in my mind, it’s complicated for me, I should say. When you’re thinking about, “Okay, I’m running this PostgreS as a service thing on this cluster.” It’s not that simple to just move the app from cluster A to cluster B anymore. I have to consider what do I do with the data? How do I make sure I don’t lose it out? Then that’s a pretty complicated question to answer. [00:05:32] OP: I think a lot of the storage providers, vendors playing in that storage space are kind of looking at novel ways to solve that and have adapted their current thinking maybe that was maybe slightly older thinking to new ways of interacting with Kubernetes cluster to provide that ongoing replication of data around different systems outside of the Kubernetes and then allowing it to be ported back in when a Kubernetes cluster – If we’re talking about Kubernetes in this instance as a platform, porting that data back in. There’re a lot of vendors playing in that space. It’s kind of an exciting space really to see how different people are figuring out how to back up distributed systems in reliable manner, because different people want different levels of backup. Because of the microservices nature of the cloud native architectures that we predominantly deal with, your application is not just one thing anymore. Certain parts of that application need to be recovered fairly quickly, and other parts don’t need to recover that quickly. It’s all about functionality ultimately that your end customers or your end users see. If you think about visually as like a banking application, for example, where if you’re looking at things like – The customer is interacting with that and they can check their financial details and they can check the current stages of their account, then they are two different services. But the actual service to transfer money into their account is down. It’s still a pretty functional system to the end user. But in the background, all those great systems are in place to recover that transfer of money functionality, but it’s not detrimental to your business if that’s down. There’ll be different SLAs and different objectives in terms of recovery, in terms of the amount of time that it takes for you to restore. All of that has to be factored in into disaster recovery plans and it’s up to the company and we can help as much as possible for them to figure out which feats of the applications and which feats of your business need to conform to certain SLAs in terms of recovery, because different feats will have different standards and different times in and around that space. It’s a complicated thing. It definite is. [00:07:29] BL: I want to take a step back and unpack this term, disaster recovery, because I can assure you, careers and fortunes have been made on helping people get this right. Before we get super deep into this, what’s a disaster and then what’s a recovery for that? Have you thought about that at a fundamental level? [00:07:45] OP: Just for me, if we would kind of take it at face value. A physical disaster, they could be physical ones or software-based ones. Physical ones can be like earthquakes or floodings, fires, things like that that are happening either in your region or can be fairly widespread across the area that you’re in, or software, cyber attacks that are perhaps to your own internal systems, like your system has been compromised. That’s fairly local to you. There are two different design strategies there. Physical disaster, you have to have a recover plan that is outside of that physical boundary that you can recover your system from somewhere that’s not affected by that physical disaster. For the recovery in terms of software in terms of your system has been compromised, then the recovery from that is different. I’m not an expert on cyber attacks and vulnerabilities, but the recovery from there for companies trying to recover from that, they plan for it as much as possible. So they down their systems and try and get patches and fixes to them as quickly as possible and spin the system backups. [00:08:49] BL: I’m understanding what you’re saying. I’m trying to unpack it for those of us listening who don’t really understand it. I’m going to go through what you said and we’ll unpack it a little bit. Physical from my assumption is we’re running workloads. Let’s say we’re just going to say in a cloud, not on-premise. We’re running workloads in let’s say AWS, and in the United States, we can take care local diversity by running in East and West regions. Also, we can take care of local diversity by running in availability, but they don’t reach it, because AWS is guaranteed that AZ1 and AZ3 have different network connections, are not in the same building, and things like that. Would you agree? Do you see that? I mean, this is for everyone out there. I’m going to go from super high-level down to more specific. [00:09:39] OP: I personally wouldn’t argue that, except not everybody is on AWS. [00:09:43] BL: Okay. AWS, or Azure, or Google Cloud, DigitalOcean, or SoftLayer, or Oracle, or Packet. If I thought about this, probably we could do 20 more. [00:09:55] JR: IBM. [00:09:56] BL: IBM. That’s why I said SoftLayer. They all practice in the physical diversity. They all have different regions that you can deploy software. Whether it’s be data locality, but also for data protection. If you’re thinking about creating a planet for this, this would be something you could think about. Where does my rest? What could happen to that data? Building could actually just fall over on to itself. All the hard drives are gone. What do I do? [00:10:21] OP: You’re saying that replication is a form of backup? [00:10:26] BL: I’m actually saying way more than that. Before you even think about things when it comes to disaster recovery, you got to define what a disaster is. Some applications can actually run out of multiple physical locations. Let’s go back to my AWS example, because it’s everywhere and everyone understands how AWS works at a high-level. Sometimes people are running things out of US-East-1 and US-West-2, and they could run both of the applications. The reason they can do that is because the individual transactions of whatever they’re doing don’t need to talk to one another. They connect just websites out of places. To your point, when you talk about now you have the issue where maybe you’re doing inventory management, because you have a large store and you’re running it out of multiple countries. You’re in the EU and you’re somewhere on APAC as well. What do you do about that? Well, there are a couple of ways that – I could think about how we would do that. We could actually just have all the database connections go back to one single main service. Then what we could do with that main service is that we could have it replicated in their local place and then we can replicate it in a remote place too. If the local place goes up, at least you can point all the other sites back to this one. That’s the simplest way. The reason I wanted to bring this up, is because I don’t like acronyms all that much, but disaster recovery has two of my favorite ones and they’re called RPO and RTO. Really, what it comes down to is you need to think about when you have a disaster, no matter that disaster is or how you define it, you have RTO. Basically, it’s the time that you can be down before there’s a huge issue. Then you have something called DPO, which is without going into all the names, is how far you can go since your last backup before you have business problems. Just thinking about those things is how we should think about our backup disaster recovery, and it’s all based on how your business works or how your project works and how long you can be down and how much data you have. [00:12:27] CC: Which goes to what Olive was saying. Please spell out to us what RTO and RPO stand for. [00:12:35] BL: I’m going to look them up real quick, because I literally pushed those acronym meanings out. I just know what they mean. [00:12:40] OP: I think it’s recovery time objective and recovery data objective. [00:12:45] BL: Yeah. I don’t know what the P stands for, but it is for data. [00:12:49] OP: Recovery. [00:12:51] BL: It’s the recovery points. Yeah. That’s what it is. It is the recovery point objective, RPO; and recovery time objective, RTO. You could tell that I’ve spent a lot of time in enterprise, because we don’t even define words. The acronym means what it is. Do you know what the acronym stands for anymore? [00:13:09] OP: How far back in terms of data can we go that was still okay? How far back in time can we be down, basically, until we’re okay? [00:13:17] CC: It is true though, and as Josh was saying, some teams or companies or products, especially companies that are starting their journey, their cloud native journey. They don’t have a backup, because there are many complicated things to deal with, and backup is super complicated, I mean, the disaster recovery strategy. Doing that is not trivial. But shouldn’t you start with that or at least because it is so complex? It’s funny to me when people say I don’t have that kind of a strategy. Maybe just like what Bryan said why utilizing, spreading out your data through regions, that is a strategy in itself, and there’s more to it. [00:14:00] JR: Yeah. I think I oversimplified too much. Disaster recovery could theoretically be anything I suppose. Going back to what you were saying, Brian, the recovery aspect of it. Recovery for some of the customers I work with is literally to stand on a brand-new cluster, whatever that cluster is, a cluster, that is their platform. Then redeploy all the applications on top of it. That is a recovery strategy. It might not be the most elegant and it might make assumptions about the apps that run on it, but it is a recovery strategy that somewhat simple, simple to kind of conceptualize and get started with. I think a lot of the customers that I work with when they’re first getting their bearings with distributed system of sorts, they’re a lot more concerned about solving for high availability, which is what you just said, Carlisia, where we’re spreading across maybe multiple sites. There’s the notion of different parts of the world, but there’s also the idea of like what I think Amazon has coined availability zones. Making sure if there is a disaster, you’re somewhat resilient to that disaster like Brian was saying with moving connections over and so on. Then once we’ve done high-availability somewhat well, depending on the workloads that are running, we might try to get a more fancy recovery solution in place. One that’s not just rebuild everything and redeploy, because the downtime might not be acceptable. [00:15:19] BL: I’m actually going to give some advice to all the people out there who might be listening to this and thinking about disaster recovery. First of all, all that complex stuff, that book you read, forget about it. Not because you don’t need to know. It’s because you should only think about what’s in scope at any given time. When you’re starting an application, let’s say I’m actually making a huge assumption that you’re using someone else’s cloud. You’re using public cloud. Whenever you’re in your data center, there’s a different problem. Whenever you’re using public cloud, think about what you already have. All the major public clouds had a durable object storage. Many 9s of durability and then fewer 9s, but still a lot of 9s of availability too. The canonical example there is S3. When you’re designing your applications and you know that you’re going to have disaster issues, realize that S3 is almost always going to be there, unless it was 2017 and it goes down, or the other two failures that it had. Pretty much, it will be there. Think about how do I get that data into S3. I’m just saying, you can use it for storage. It’s fairly cheap for how much storage you can get. You can make it sure it’s encrypted, and using IM, you can definitely make sure that people who have the right pillages can see it. The same goes with Azure and the same goes with Google. That’s the first phase. The second phase is that now you’re going to say, “Well, what is a relational database?” Once again, use your cloud provider. All the major cloud providers have great relational databases, and actually key value stores as well. The neat thing about them is you can actually set them up sometimes to run in a whole region. You can set them up to do automated backups. At least the minimum that you have, you actually use your cloud provider for what it’s valuable for. Now, you’re not using a cloud provider and you’re doing it on-premise, I’m going to tell you, the simple answer is I hope you have a little bit of money, because you’re going to have to pay somebody either one of Kubernetes architects or you’re going to pay somebody else to do it. There’s no easy button for this kind of solution. Just for this little mini-rant, I’m going to leave everyone with the biggest piece of advice, the best piece of advice that I can ever leave you if you’re running relational databases. If you are running a relational database, whether it’d be PostgreS, MySQL, Aurora, have it replicated. But here’s the kicker, have another replica that you delay and make it delay 10 minutes, 15 minutes, not much longer than that. Because what’s going to happen, especially in a young company, especially if you’re using Rails or something like that, you’re going to have somebody who is going to have access to production, because you’re a small company, you haven’t really federated this out yet. Who’s going to drop your main database table? They’re just going to do it and it’s going to happen and you’re going to panic. If you have it in a replica, that databases go in a replica, you have a 10-minute delay replica – 10 minutes to figure it out before the world ends. Hopefully someone deletes the master database. You’re going to know pretty quickly and you can just cut that replica out, pull that other one over. I’m not going to say where i learned this trick. We had to employ it multiple times, and it saves our butts multiple times. That’s my favorite thing to share. [00:18:24] OP: Is that replica on separate system? [00:18:26] BL: It was on a separate system. I actually don’t say, because it will be telling on who did it. Let’s say that it was physically separate from the other one in a different location as well. [00:18:37] OP: I think we’ve all been there. We’ve all have deleted something that maybe – [00:18:41] CC: I’m going to tell who did it. It was me. [00:18:45] BL: Oh no! It definitely wasn’t me. [00:18:46] OP: We mentioned HA. Will the panel think that there’s now a slightly inverse relationship between the amount of HA that you architect for versus the disaster recovery plan that you have implemented on the back of that? More you’re architecting around HA, like the less you architect or plan for DR. Not eliminating ether of them. [00:19:08] BL: I see it more. Mean, it used to be 15 years ago. [00:19:11] CC: Sorry. HA, we’re talking about high availability. [00:19:15] BL: When you think about high availability, a lot of sites were hosted. This is really before you had public cloud and a lot of people were hosting things on WebHost or they’re hosting themselves. Even if you are a company who had like a big equinox of level 3, you probably didn’t have two facilities at two different equinoxes or level 3, which probably does had one big cage and you just had diversity in the systems in there. We found people had these huge tape backups and we’re very diligent about swapping our tapes out. One thing you did was we made sure that – I mean, lots of practice of bringing this huge system down, because we assumed that the database would die and we would just spend a few hours bringing it back up, or days. Now with high availability, we can architect systems where that is less of a problem, because we could run more things that manage our data. Then we can also do high availability in the backend on the database side too. We can do things like multi-writes and multi-reads. We can actually write our data in multiple places. What we find when we do this is that the loss of a single database or a slice of processing/webhosts just means that our services degraded, which means we don’t really have a disaster in this point and we’re trying to avoid disasters. [00:20:28] JR: I think on that point, the way I’ve always thought about it, and I’ll admit this is super overly simplified, but like successful high availability or HA could make your lead to perform disaster recovery less likely, can, maybe, right? It’s possible. [00:20:45] BL: Also realize that everybody is running in public cloud. In that case, well, you can still back your stuff up to public cloud even if you’re not running in public cloud. There are still people out there who are running big tape arrays, and I’ve seen them. I’ve seen tape arrays that are wider. I’m sitting in an 80-inch wide table, bigger than this table with robotic arms and takes the restic and you had to make sure that you got the text right for that particular day doing your implementation. I guess what I’m saying is that there is a balance. HA, high availability, if you’re doing it in a truly high available way, you can’t miss whole classes of disaster. But I’m not saying that you will not have disaster, because if that was the case, we won’t be having this discussion right now. I’d like to move the conversation just a little bit to more cloud native. If you’re running on Kubernetes, what should you think about for disaster recovery? What are the types of disasters we could have? How could we recover them? [00:21:39] JR: Yeah. I think one thing that comes to mind, I was actually reading the Kubernetes Best Practices book last night, but I just got an O’Reilly membership. Awesome. Really cool book. One of the things that they had recommended early on, which I thought was a really good pull out is that since Kubernetes is a declarative system where we write these manifests to describe the desired state of our application and how it should run, recommending that we make sure to keep that declarative state in source control, just like we would our code so that if something were to go wrong, it is somewhat more trivial to redeploy the application should we need to recover. That does assume we’re not worried about like data and things like that, but it is a good call out I think. I think the book made a good call out. [00:22:22] OP: That’s on the declarative system and enable to bring your systems back up to the exact way they were before kind of itself adds comfort to the whole notion that they could be disaster. If they was, we can spin up backup relatively quickly. That’s back from the days of automation where the guys originally – I came from Red Hat, so fork at Ansible. We’re kind of trying to do the infrastructure as a code, being able to deploy, redeploy, redeploy in the same manner as the previous installation, because I’ve been in this game long-time now and I’ve spent a lot of time working with processes in and around building physical servers. That process will get handled over to lots of different teams. It was a huge thing to build these things, to get one of these things built and signed off, because it literally has to pass through the different teams to do their own different bits of things. The idea that you would get a language that had the functionality that suited the needs of all those different teams, of the store team, could automate their piece, which they were doing. They just wasn’t interactive with any of the other teams. The network people would automate theirs and the application install people would do their bit. The server OS people would do their bit. Having a process that could tie those teams together in terms of a language, so Ansible, Puppet, Chef, those kinds of things try to unite those teams and it can all do your automation, but we have a tool that can take that code and run it as one system end-to-end. At the end of that, you get an up and running system. If you run it again, you get all the systems exactly the same as the previous one. If you run it again, you get another one. Reducing the time to build these things plays very importantly into this space. Disaster is only disaster in terms of time, because things break all the time. How that affects you and how quickly you can recover. If you can recover in like seconds, in minutes and it hasn’t affected your business at all, then it wasn’t really a disaster. The time it takes you to recover, to build your things back is key. All that automation and then leading on to Kubernetes, which is the next step, I think, this whole declarative, self-healing and implementing the desired state on a regular basis really plays well into this space. [00:24:25] CC: That makes me think, I don’t completely understand because I’m not out there architecting people’s systems. The one thing that I do is building this backup tool, which happens to be for Kubernetes. I don’t completely get the limitations and use cases, but my question is, is it enough to have the declarations of how your infrastructure should be in source control? Because what if you’re running applications on the platform and your applications are interacting with a platform, change in the state of the platform. Is that not something that happens? Of course, ideally, having those declarations and source control of course is a great backup, but don’t you also want to back up the changes to state as they keep happening? [00:25:14] BL: Yeah, of course. That has been used for a long-time. That’s how replication works. Literally, you take the change and you push it over the wire and it gets applied to the remote system. The problem is, is that there isn’t just one way to do this, because if you do only transaction-based. If you only do the changes, you need a good base to start with, because you have to apply those changes to something. How do you get that piece? I’m not asking you to answer that. It’s just something to think about. [00:25:44] JR: I think you’ve hit a fatal flaw too, Carlisia, and like what that simplified just like having source control model kind of falls over. I think having that declarative kind of stamped out, this is the ideal nature of the world to this deployment and source control has benefits beyond just that of disaster recovery scenario, right? For stateless applications especially, like we talked about in the previous podcast, it can actually be all lead potentially, which is so great. Move your CI system over to cluster B. Boom! You’re back up and running. That’s really neat. A lot of our customers we work with, once we get them to a point where they’re at that stage, they then go, “Well, what about all these persisted volumes?” which by the way is evolving on a computer, which is a Kubernetes term. But like what about all these parts on like disk that I don’t want to lose if I lose my cluster? That it totally feeds into why tools like the one you work on are so helpful. Maybe I don’t know if now would be a good time. But maybe, Carlisia, you could expand on that tool. What it tries to solve for? [00:26:41] CC: I want to back up a little though. Let’s put aside stateful workloads and volumes and databases. I was talking about the infrastructure itself, the state of the infrastructure. I mean, isn’t that common? I don’t know the answer to this. I might be completely off. Isn’t that common for you to develop a cloud native application that is changing the state of the infrastructure, or is this something that’s not good to do? [00:27:05] JR: It’s possible that you can write applications that can change infrastructure, but think about that. What happens when you have bad code? We all have bad code. Our people like to separate those two things. You can still have infrastructure as code, but it’s separated from the application itself, and that’s just to protect your app people from your not app people and vice versa. A lot of that is being handled through systems that people are writing right now. You have Ansible from IBM. You have things like HashiCorp and all the things that they’re doing. They have their hosted thing. They have their own premise thing. They have their local thing. People are looking at that problem. The good thing is that that problem hasn’t been solved. I guess good and bad at the same time, because it hasn’t been solved. So someone can solve it better. But the bad thing is that if we’re looking for good infrastructure as code software, that has not been solved yet. [00:27:57] OP: I think if we’re talking about containerized applications, I think if there was systems that interacted or affected or changed the infrastructure, they would be separate from the applications. As you were saying, Brian, you just expanded a little bit [inaudible 00:28:11] containerized or sandboxed, processes that were running separate to the main application. You’re separating out what’s actually running and doing function in terms of application versus systems that have to edit that infrastructure first before that main application runs. They’re two separate things. If you had to restore the infrastructure back to the way it was without rebuilding it, but perhaps have a system whereby if you have something editing the infrastructure, you would always have something that would edit it back. If you have the process that runs to stop something, you’d also have a process that start at something. If you’re trying to [inaudible 00:28:45] your applications and if it needs to interact with other things, then that application design should include the consideration of what do I need to do to interact with the infrastructure. If I’m doing something left-wise, I have to do the opposite in equal reaction right-wise to have an effectively clean application. That’s the kind of stuff I’ve seen anyway. [00:29:04] JR: I think it maybe even fold into a whole other topic that we could even cover on another podcast, which is like the notion of the concern of mutating infrastructure. If you have a ton of hands in those cookie jars and they’re like changing things all over the place, you’re losing that potential single source of declarative truth even, right? It just could become very complicated. I think maybe to the crux of your original point, Carlisia. Hopefully I’m not super off. If that is happening a lot, I think it could actually make recover more complicated, or maybe recovery is not the way to put it, but recreating the infrastructure, if that makes sense. [00:29:36] BL: Your infrastructure should be deterministic, and that’s why I said you could. I know we talked about this before about having applications modify infrastructure. Think about that. Can and should are two different things. If you have it happen within your application due to input of any kind, then you’re no longer deterministic, unless you can figure out what that input is going to be. Be very careful about that. That’s why people split infrastructure as code from their other code. You could still have CI, continuous integration and continuous delivery/deployment for both, but they’re on different pipelines with different release metrics and different monitoring and different validation to make sure they work correctly. [00:30:18] OP: Application design plays a very important role now, especially in terms of cloud native architecture. We’re talking a lot about microservices. A lot of companies are looking to re-architect their applications. Maybe mistakes that were made in the past, or maybe not mistakes. It’s perhaps a strong word. But maybe things that were allowed in the past perhaps are now best practices going forward. If we’re looking to be able to run things independently of each other, and by definition, applications independent on the infrastructure, that should be factored in into the architecture of those applications going forward. [00:30:50] CC: Josh asked me to talk a little bit about Velerao. I will touch up on it quickly. First of all, we’d love to have a whole show just about infrastructure code, GitOps. Maybe that would be two episodes. Velero doesn’t do any backup of the infrastructure itself. It works at the Kubernetes level. We back up the Kubernetes clusters including the volumes. If you have any sort of stateful app attached to a pod that can get backed up as well. If you want to restore that to even a different service provider, then the one you backed up from, we have a restic plugin that you can use. It’s embedded in the Velero tool. So you can do that using this plugin. There are few really cool things that I find really cool about Velero is, one, you can do selective backups, which really, really don’t recommend. We recommend you always back up everything, but you can do selective restores. That would be – If you don’t need to restore a whole cluster, why would you do it? You can just do parts of it. It’s super simple to use. Why would you not have a backup? Because this is ridiculously simple. You do it through a command line, and we have a scheduler. You can just put your backup on scheduler. Determine the expiration date of each backup. A lot of neat simple features and we are actively developing things all the time. Velero is not the only one. It’d be fair to mention, and I’m not a super well versed on the tools out there, but etcd itself has a backup tool. I’m not familiar with any of these other tools. One thing to highlight is that we do everything through the Kubernetes API. That’s for example one reason why we can do selective backup or restores. Yes, you can backup etcd completely yourself, but you have to back up the whole thing. If you’re on a managed service, you wouldn’t be able to do that, because you just wouldn’t have access. All the tools like we use to back up to the etcd offers or a service provider. PX-motion. I’m not sure what this is. I’m reading the documentation here. There is this K10 from [inaudible 00:33:13] Canister. I haven’t used any of these tools. [inaudible 00:33:16]. [00:33:17] OP: I just want to say, Velero, the last customer I worked on, they wanted to use Velero in its capacity to be able to back up a whole cluster and then restore that whole cluster on a different cloud provider, as you mentioned. They weren’t thoroughly using it as – Well, they were using it as backup, but their primary function was that they wanted to populate the cluster as it was on a brand-new cloud provider. [00:33:38] CC: Yeah. It’s a migration. One thing that, like I said, Velero does, is back up the cluster, like all the Kubernetes objects, because why would we want to do that? Because if you’re declaring – Someone explain to everybody who’s listening, including myself. Some people bring this up and they say, “Well, I don’t need to back up the Kubernetes objects if all of that is declared and I have the declaration is source control. If something happens, I can just do it again. [00:34:10] BL: Untrue, because just for any given Kubernetes object, there is a configuration that you created. Let’s say if you’re creating an appointment, you need spec replicas, you need the spec templates, you need labels and selectors. But if you actually go and pull down that object afterwards, what you’ll see is there is other things inside of that object. If you didn’t specify any replicas, you get the defaults or other things that you should get defaults for. You don’t want to have a lousy backup and restore, because then you get yourself into a place where if I go back this thing up and then I restore it to a different cluster to actually test it out to see if it works, it will be different. Just keep that in mind when you’re doing that. [00:34:51] JR: I think it just comes down to knowing exactly what Brian just said, because there certainly are times where when I’m working with a customer, there’s just such a simple use case at the notion of redeploying the application and potentially losing some of those factors that may have mutated overtime. They just shrug to it and go, “Whatever.” It is so awesome that tools like Velero and other tools are bridging that gap, and I think to a point that Olive made, not only just backing that stuff up and capturing it state as it was in the cluster, but providing us with a good way to section out one namespace or one group of applications and just move those potentially over and so on. Yeah, it just kind of comes to knowing what exactly are you going to have to solve for and how complex your solution should be. [00:35:32] BL: Yeah. We’re getting towards the end, and I wanted to make sure that we talked about testing your backup, because that’s a popular thing here. People take backups. I’ve done my backups, whether I dump to S3, or I have Velero dumping to S3, or I have some other method that is in an invalid backup, it’s not valid until someone comes and takes that backup, restore it somewhere and actually verifies that it works, because there’ll be nothing worse than having yourself in a situation where you need a backup and you’re in some kind of disaster, whether small or large, and going to find out that, “Oh my gosh! We didn’t even backup the important thing.” [00:36:11] CC: That is so true. I have only been in this backup world for a minute, but I mean I’ve needed to backup things before. I don’t think I’ve learned this concept after coming here. I think I’ve known this concept. It just became stronger in my mind, so I always tell people, if you haven’t done that restore, you don’t have a backup. [00:36:29] JR: One thing I love to add on to that concept too is having my customers run like fire drills if they’re open to it. Effectively, having a list of potential terrible things that can happen, from losing a cluster to just like losing an important component. Unlike one person the team, let’s say, once a week or one a month, depending on their tolerance, just chooses something from that list and does it, not in production, but does it. It gives you the opportunity to test everything end-to-end. Did your learning fire off? When you did restore to your points, was the backup valid? Did the application come back online? It’s kind of a lot of like semi-fun, using the word fun loosely there. Fun ways that you can approach it, and it really is a good way to kind of stress test. [00:37:09] BL: I do have one small follow up on that. You’re doing backups, and no matter how you’re doing them, think about your strategy and then how long to keep data. I mean, whether it’s due to regulation or just physical space and it costs money. You just don’t backup yesterday and then you’d backup again. Backup every day and keep the last 8 days and then, like old school, would actually then have a full backup and keep that for a while just in case, because you never know. [00:37:37] CC: Good point too. Yeah. I think a lot of what we said goes to what – It was Olive I think who said it first. You have to understand your needs. [00:37:46] OP: Yeah, just which bits have different varying degrees of importance in terms of application functionality for your end user. Which bits are absolutely critical and which bits can buy you a little bit more time to recover. [00:37:58] CC: Yeah. That would definitely vary from product to product. As we are getting into this idea of ephemeral clusters and automation and we get really good at automating things and bringing things back up, is it possible that we get to a point where we don’t even talk about disasters anymore, or you just have to grow, bring this up cluster or this system, and does it even matter why [inaudible 00:38:25]. We’re not going to talk about this aspect, because what I’m thinking is in the past, in a long, long time ago, or maybe not so long time ago. When I was working with application, and that was a disaster, it was a disaster, because it felt like a disaster. Somebody had to go in manually and find out what happened and what to fix and fix it manually. It was complete chaos and stress. Now if they just like keep rolling and automate it, something goes down, you bring it back up. Do you know what I mean? It won’t matter why. Are we going to talk about this in terms of it was a disaster? Does it even matter what caused it? Maybe it was a – Recovery from a disaster wouldn’t look any different than a planned update, for example. [00:39:12] BL: I think we’re getting to a place – And I don’t know whether we’re 5 years away or 10 years away or 20 years away, a place where we won’t have the same class of disaster that we have now. Think about where we’ve come over the past 20 years. Over the past 20 years, be basically looked at hardware in a rack is replace. I can think about 1988, 1999 and 2000. We rack a whole bunch of servers, and that server will be special. Now, at these scales, we don’t care about that anymore. When a server goes away, we have 50 more just like it. The reason we were able to do that across large platforms is because of Linux. Now with Kubernetes, if Kubernetes keeps on going in the same trajectory, we’re going to basically codify these patterns that makes hardware loss not a thing. We don’t really care if we lose a server. You have 50 more nodes that look just like it. We’re going to start having the software – The software is always available. Think about like the Google Spanner. Google Spanner is multi-location, and it can lose notes and it doesn’t lose data, and it’s relational as well. That’s what CockroachDB is about as well, about Spanner, and we’re going into the place where this kind of technology is available for anyone and we’re going to see that we’re not going to have these kinds of disasters that we’re having now. I think what we’ll have now is bigger distributed systems things where we have timing issues and things like that and leader election issues. But I think those cool stuff can’t be phased out at least over the next computing generation. [00:40:39] OP: It’s maybe more around architectures these days and applications designers and infrastructure architects in the container space and with Kubernetes orchestrating and maintaining your desired state. You’re thinking that things will fail, and that’s okay, because it will go back to the way it was before. The concept of something stopping in mid-run is not so scary anymore, because it would get put back to its state. Maybe you might need to investigate if it keeps stopping and starting and Kubernetes keeps bringing it back. The system is actually still fully functional in terms of end users. You as the operator might need to investigate why that’s so. But the actual endpoint is still that your application is still up and running. Things fail and it’s okay. That’s maybe a thing that’s changed from maybe 5 years ago, 10 years ago. [00:41:25] CC: This is a great conversation. I want to thank everybody, Olive Power, Josh Rosso, Brian Lyles. I’m Carlisia Campos singing off. Make sure to subscribe. This was Episode 8. We’ll be back next week. See you. [END OF EPISODE] [0:50:00.3] KN: Thank you for listening to The Podlets Cloud Native Podcast. Find us on Twitter at https://twitter.com/ThePodlets and on the http://thepodlets.io/ website, where you'll find transcripts and show notes. We'll be back next week. Stay tuned by subscribing. [END]See omnystudio.com/listener for privacy information.

Gabi Ferrara and Jon Foust are joined today by fellow Googler Zack Akil to discuss machine learning and AI advances at Google. First up, Zack explains some of the ways AutoML Vision and Video can be used to make life easier. One example is how Google Photos are automatically tagged, allowing them to be searchable thanks to AutoML. Developers can also train their own AutoML to detect specific scenarios, such as laughing in a video. We also talk Cloud Next 2019 and learn how Zack comes up with ideas for his cool demos. His goal is to inspire people to incorporate machine learning into their projects, so he tries to combine hardware and exciting technology to think of fun, creative ways developers can use ML. Recently, he made a smart AI bicycle that alerts riders of possible danger behind them through a system of lights and a project to track and photograph balls as they fly through the air after being kicked. To wrap it all up, Zack tells us about some cool projects he’s heard people use AutoML for (like bleeping out tv show spoilers in online videos!) and the future of the software. Zack Akil When he’s not teaching machine learning at Google, Zack likes to teach machine learning at his hands-on data science meetup, Central London Data Science Project Nights. Although he works in the cloud, most of his hobby projects look at different ways you can embed machine learning into low-power devices like Raspberry Pis and Arduinos. He also likes to have a bit of banter with his mixed tag rugby teams. Cool things of the week Stackdriver Logging comes to Cloud Code in Visual Studio Code blog Open Match v0.8 was released last month site Cloud Spanner now supports the WITH clause blog Interview Zack’s Website site Cloud AutoML site AutoML Video docs AutoML Vision site AutoML Vision Object Detection docs Coral site TensorFlow.js site Central London Data Science Meetup site Question of the week How do I run Cloud Functions in a local environment? Where can you find us next? Zack will be at DevRelCon. Gabi will be taking time to recharge after conference season, then visiting family. Jon will be attending several baby showers. Sound Effect Attribution “Small Group Laugh 4, 5 & 6” by Tim.Kahn of Freesound.org “Sparkling Effect A” by CetSoundCrew of Freesound.org

Mark Mirchandani hosts solo today but is later joined by fellow Googler and Developer Advocate Ray Tsang to talk Java! Ray tells us what’s new with Java 11, including more memory and fewer restrictions for developers. One of the greatest things for Ray is using Java 11 in App Engine because of the management support that it provides. Later, we talk about Spring Boot on GCP. Ray explains the many benefits of using this framework. Developers can get their projects started much more quickly, for example, and with Spring Cloud GCP, it’s easy to integrate GCP services like Spanner and run your project in the cloud. For users looking to containerize their Java projects, JIB can help you do this without having to write a Dockerfile. At the end of the show, Ray and Mark pull it all together by explaining how Spring Boot, Cloud Code, Skaffold, and proper dev-ops can work together for a seamless Java project. Ray Tsang Ray is a Developer Advocate for the Google Cloud Platform and a Java Champion. Ray works with engineering and product teams to improve Java developer productivity on GCP. He also helps Alphabet companies migrate and adopt cloud native architecture. Prior to Google, Ray worked at Red Hat, Accenture, and other consulting companies, where he focused on enterprise architecture, managed solutions delivery, and contributed to open source projects. Aside from technology, Ray enjoys traveling and adventures. Cool things of the week Cloud Run is now GA blog Budget API in Beta blog Interview App Engine site Micronaut site Quarkus site Java 11 on App Engine blog and docs Spring Boot and Spring Cloud site Spring Cloud GCP Projects site Cloud Spanner site Spring Cloud Sleuth site Stackdriver site Bootiful GCP: To Production! blog Effective Cloud Native Spring Boot on Kubernetes & Google Cloud Platform blog JDBC drivers site Hibernate ORM with Cloud Spanner docs Effective Cloud Native Spring Boot on Kubernetes & Google Cloud Platform blog Dev to Prod with Spring on GCP in 20 Minutes (Cloud Next ‘19) video Cloud Code site JIB site Skaffold site Debugger site Troubleshooting & Debugging Microservices in Kubernetes blog Cloud Code Quickstart docs Spring (or Java) to Kubernetes Faster and Easier blog GCP Podcast Episode 58: Java with Ray Tsang and Rajeev Dayal podcast Question of the week How do I dockerize my Java app? video github Where can you find us next? Ray is taking a break for the holidays, but in the future, you can find him at Java and JUG conferences. Mark is hanging out in the Bay Area, but Google Cloud Next in London and KubeCon and CloudNativeCon are happening now! Sound Effect Attribution “Small Group Laugh 4, 5 & 6” by Tim.Kahn of Freesound.org “Tre-Loco1” by Sonipro of Freesound.org “Mens Sincere Laughter” by Urupin of Freesound.org “Party Pack” by InspectorJ of Freesound.org “DrumRoll” by HolyGhostParty of Freesound.org “Tension” by ERH of Freesound.org

Gabi Ferrara and Jon Foust are back today and joined by fellow Googler Manuel Lima. In this episode, Manuel tells us all about data visualization, what it means, why it’s important, and the best ways to do it effectively. For Google and its mission, data visualization is especially necessary in faciliatating the accesibility of information. It “makes the invisible visible” because of the way it can decode meaningful data patterns. Working across multiple GCP products, Manuel and his team build advanced visualization models that go beyond graphs and bar charts to things like sophisticated time lines that aid in the progression from data to usable knowledge. They have also created guidelines for things like what kind of graphical language to use, what type of charts users might need, and more. These guidelines, originally used only internally, have now been adjusted and released for use by developers outside Google with the help of the Material.io team. The guidelines are based around the six data visualazation princples that help users get started. They can be employed to plan and inspire an entire project or to evaluate a specific data visualation chart. Some of the most important principles are to be honest and to lend a helping hand. You can read more in their Medium article, Six Principles for Designing Any Chart. Manuel Lima A Fellow of the Royal Society of Arts and nominated by Creativity magazine as “one of the 50 most creative and influential minds of 2009,” Manuel Lima is the founder of VisualComplexity.com, Design Lead at Google, and a regular teacher of data visualization at Parsons School of Design. Manuel is a leading voice on information visualization and has spoken at numerous conferences, universities, and festivals around the world, including TED, Lift, OFFF, Eyeo, Ars Electronica, IxDA Interaction, Harvard, Yale, MIT, Columbia, the Royal College of Art, NYU Tisch School of the Arts, ENSAD Paris, the University of Amsterdam, and MediaLab-Prado Madrid. He has also been featured in various publications and media outlets, such as Wired, the New York Times, Science, Nature, Businessweek, Fast Company, Forbes, The Guardian, BBC, CNN, Design Observer, Creative Review, Eye, Grafik, étapes, and El País. His first book, Visual Complexity: Mapping Patterns of Information, has been translated into French, Chinese, and Japanese. His latest, The Book of Circles: Visualizing Spheres of Knowledge, covers 1,000 hundred years of humanity’s long-lasting obsession with all things circular. With more than twelve years of experience designing digital products, Manuel has worked for Codecademy, Microsoft, Nokia, R/GA, and Kontrapunkt. He holds a BFA in Industrial Design and a MFA in Design & Technology from Parsons School of Design. During the course of his MFA program, Manuel worked for Siemens Corporate Research Center, the American Museum of Moving Image, and Parsons Institute for Information Mapping in research projects for the National Geospatial-Intelligence Agency. Cool things of the week Compute Engine or Kubernetes Engine? New trainings teach you the basics of architecting on Google Cloud blog Stadia comes next month site Google Cloud named a Leader in the 2019 Gartner Magic Quadrant for Full Life Cycle API Management for the fourth consecutive time blog Google Hardware Event Pixel 4 is here to help blog Meet the new Google Pixel Buds blog Nest Mini brings twice the bass and an upgraded Assistant blog More affordable and portable: let’s Pixelbook Go blog Interview Material.io site Data Visualization Guides site Six Principles for Designing Any Chart article Google’s six rules for great data design article BigQuery site Stackdriver site Google Analytics site Question of the week What are the most common products used in cloud gaming? Cloud Spanner for storing player authentication and inventory or long-term state storage site Redis is used in Open Match VM’s have been the most commonly used product for game servers but there has been a shift to Kubernetes Pub/Sub Where can you find us next? Gabi will be at Full Stack Europe. Jon will be at Kubecon in November to run a workshop on Open Match. Sound Effect Attribution “Small Group Laugh 6” by Tim.Kahn of Freesound.org “Jingle Romantic” by Jay_You of Freesound.org

Google’s own Billy Jacobson joins hosts Mark Mandel and Mark Mirchandani this week to dive deeper into Cloud Bigtable. Bigtable is Google’s petabyte scale, fully managed, NoSQL database. Billy elaborates on what projects Bigtable works best with, like time-series data user analytics, and why it’s such a great tool. It offers huge scalability with the benefits of a managed system, and it’s flexible and easily customized so users can turn on and off the pieces they need. Later, we learn about other programs that are compatible with Bigtable, such as JanusGraph, Open TSDB, and GeoMesa. Bigtable also supports the API for HBase, an open-source project similar to Bigtable. Because of this, it’s easy for HBase users to move to Bigtable, and the Bigtable community has access to many open source libraries. Billy also talks more about the nine clients available, and when customers might want to use Bigtable instead of, or in conjunction with, other Google services such as Spanner and BigQuery. Billy Jacobson Billy Jacobson is a developer programs engineer focusing on Cloud Bigtable. Cool things of the week Introducing Cloud Run Button: Click-to-deploy your git repos to Google Cloud blog Firebase Unity Solutions: Update game behavior without deploying with Remote Config blog Introducing the BigQuery Terraform module blog Macy’s uses Google Cloud to streamline retail operations blog Interview Cloud Bigtable site GCP Podcast Episode 18: Bigtable with Ian Lewis podcast BigQuery site Bigtable Documentation docs Codelab: Introduction to Cloud Bigtable site Key Visualizer docs Bigtable Replication Documentation docs Bigtable and HBase Documentation docs HBase site JanusGraph site Open TSDB site GeoMesa site Bigtable Client Libraries docs Cloud Spanner site Managing IoT Storage with Google’s Cloud Platform (Google I/O’19) video Cloud Datastore site Cloud Firestore site Mapping the invisible: Street View cars add air pollution sensors site Breathing Easy with Bigtable article Question of the week If I have an organization, how do I break down my billing data by folder? Where can you find us next? Mark Mirch is working around town but will be headed to LA soon. Mark Mandel will be at Pax Dev, Pax West, Kubecon, and the GDC Online Games Technology Summit.

Blockchain takes the spotlight as new host Carter Morgan joins veteran Mark Mandel in a fascinating interview with Allen Day. Allen is a developer advocate with Google, specializing in streaming analytics for blockchain, biomedical, and agricultural applications. This week Allen reveals how blockchain and cryptocurrencies can be applied to a variety of applications like distributed file storage and video services. We also discuss the hype and merits of blockchain + projects that Allen has worked on to analyze cryptocurrency transactions using Google Cloud’s big data platforms. The results may just surprise you. Allen Day Allen Day is a developer advocate with Google in Singapore. He specializes in streaming analytics for blockchain, biomedical, and agricultural applications. Allen studied at the UCLA Geffen School of Medicine and earned his PhD in Human Genetics. Allen’s blockchain work is focused on interoperability between smart contract platforms and cloud platforms. He created Google Cloud’s blockchain public datasets program, which allows non-specialist engineers and data scientists to search and analyze public blockchain data. Cool things of the week Blockchain.com, scaling and saving with Cloud Spanner blog Cloud TPU Pods break AI training records blog Cloud Memorystore adds import-export and Redis 4.0 blog To run or not to run a database on Kubernetes: What to consider blog Google to acquire Elastifile blog Interview Blockchain site Bitcoin site Coinbase site Ethereum site $24 million iced tea company says it’s pivoting to the blockchain, and its stock jumps 200% news article Blockchain ETL project on GitHub site BigQuery site Kubernetes site Cloud Composer site Pub/Sub site Bigtable site Tensorflow site Bitcoin in BigQuery: blockchain analytics on public data blog BigQuery public blockchain datasets on GCP site Ethereum in BigQuery: how we built this dataset blog Ethereum in BigQuery: a Public Dataset for smart contract analytics blog Introducing six new cryptocurrencies in BigQuery Public Datasets—and how to analyze them blog Building hybrid blockchain/cloud applications with Ethereum and Google Cloud blog Bitcoin in BigQuery: blockchain analytics on public data blog Unchained Podcast podcast Off the Chain Podcast podcast Question of the week What are the four (or six?) types of VMs that exist on Google Cloud Platform? blog and docs Where can you find us next? Mark Mandel is going to Tokyo Next, Open Source in Gaming Day , and the North American Open Source Summit, as well as Pax Dev and Pax West. Carter will be at the Edinburgh Fringe Festival and working on new videos. Allen will be at Strike Two Summit (Amsterdam), Singularity Festival (Heraklion), and Ethereum Devcon (Osaka). Sound Effect Attribution “mysterypeak1.wav” by FoolBoyMedia of Freesound.org “crowd laugh.wav” by Tom_Woysky of Freesound.org

Google Developer Advocate Jen Person talks with Mark Mandel and Mark Mirchandani today about developments in Firebase. Firebase is a suite of products that helps developers build apps. According to Jen, it’s equivalent to the client-side of Google Cloud. Firebase works across platforms, including Android, web, iOS and offers many growth features, setting it apart from other Google products. It helps site and app owners interact with and reach customers with services like notifications, remote configurations to optimize the app, testing, and more. Cloud Firestore has come out of beta, and it is available both through Firebase and Google Cloud Platform, making it easy for developers to move from one to the other if their needs change. Recently, the Firebase team has been working to refine their products based on user feedback. Firebase Authentication has been upgraded with the additions of phone authentication, email link authentication, and multiple email actions. They’ve also added a generic authentication option so developers can use any provider they choose. ML Kit makes machine learning much easier for client apps or on the server. With on-device ML features, users can continue using the app without internet service. Things like face recognition can still be done quickly without a wifi connection. ML Kit is adding new features all the time, including smart reply and translation, image labeling , facial feature detection, etc. Cloud Functions for Firebase is also out of beta. It includes new features like a crash-litics trigger that can notify you if your site or app crashes and scheduled functions. An emulator is new as well, so you can test without touching your live code. Jen Person Jen is a Developer Advocate at Google. She worked with Firebase for 2.5 years prior to recently joining Google Cloud. She loves building iOS apps with Swift and planning the ideal data structures for various apps using Cloud Firestore. Jen is currently co-starring with JavaScript in a buddy cop comedy where the two don’t see eye to eye but are forced to work together, eventually forming a strong loving bond through a series of hilarious misadventures. Cool things of the week Uploading images directly to Cloud Storage using Signed URL blog Build your own event-sourced system using Cloud Spanner blog Cloud Shell on the Cloud Console app site Google Cloud networking in depth: Cloud Load Balancing deconstructed blog Interview Firebase site Firestore site Cloud Storage site Firebase Authentication site ML Kit site TensorFlow Lite site Cloud Functions for Firebase site Cloud Functions Samples site I/O 2019 Talk: Zero to App video Guide - Cloud Firestore collection group queries docs Guide - Scheduled Cloud Functions docs YouTube - #AskFirebase Playlist videos Codelab - Recognize text, facial features, and objects in images with ML Kit for Firebase: iOS site Codelab - Train and deploy on-device image classification model with AutoML Vision in ML Kit site Codelab - Recognize text, facial features, and objects in images with ML Kit for Firebase: Android site Codelab - Identify objects in images using custom machine learning models with ML Kit for Firebase site Codelab - Detect objects in images with ML Kit for Firebase: Android site Previous episodes on Firebase: GCP Podcast Episode 13: Firebase with Sara Robinson and Vikrum Nijjar podcast GCP Podcast Episode 29: The New Firebase with Abe Haskins and Doug Stevenson podcast GCP Podcast Episode 78: Firebase at I/O 2017 with James Tamplin and Andrew Lee podcast GCP Podcast Episode 97: Cloud Firestore with Dan McGrath and Alex Dufetel podcast GCP Podcast Episode 99: Cloud Functions and Firebase Hosting with David East podcast Question of the week How do I save money on my GCP resources? Where can you find us next? Mark Man will be at Tokyo Next! Watch him live code on Twitch. Mark Mirch is going on vacation!

Mark and Brian Dorsey spend today talking Python with Dustin Ingram. Python is an interpreted, dynamically typed language, which encourages very readable code. Python is popular for web applications, data science, and much more! Python works great on Google Cloud, especially with App Engine, Compute Engine, and Cloud Functions. To learn more about best (and worst) use cases, listen in! Dustin Ingram Dustin Ingram is a Developer Advocate at Google, focused on supporting the Python community on Google Cloud. He’s also a member of the Python Packaging Authority, maintainer of PyPI, and organizer for the PyTexas conference. Cool things of the week Machine learning can boost the value of wind energy blog Compute Engine Guest Attributes site Colopl open sourced a Cloud Spanner driver for Laravel framework site Running Redis on GCP: four deployment scenarios blog Interview GCP Podcast Episode 3: Kubernetes and Google Container Engine podcast Python site Extending Python with C or C++ docs PyPy site PyPI site App Engine site Compute Engine site Cloud Functions site Ubuntu site Flask site Flask documentation docs Docker site Python documentation docs PyCon site PyCaribbean site Question of the week How can I manipulate images with Cloud Functions? Where can you find us next? Mark will be at GDC, Cloud NEXT, and ECGC in April. Dustin will be at Cloud Next and PyCon. Brian will be lecturing at Cloud Next: ‘Where should I run my code?’

We’re learning all about Cloud SQL this week with our guest, Amy Krishnamohan. Amy’s main job is to teach customers about the products she represents. Today, she explains to Mark and Gabi that Cloud SQL manages services for open source databases, and she spends a little time elaborating on the other database management services Google has to offer. Cloud SQL is a relational data storage solution. Relational data storage is very structured, almost like a table or spreadsheet, making it easier to analyze the data. Cloud SQL is capable of scaling out and up, meaning it can scale for traffic patterns and for storage. In comparison, NoSQL databases are very unstructured. If you’re not sure what kind of data is coming in, you can sort the data first and analyze it later. Each approach has its pros and cons and each is suitable for different types of projects. Recently, Cloud SQL released a feature making it easy to move from on-prem to the cloud. In the future, they will continue to streamline the process of moving between the two spaces. Amy Krishnamohan Amy is Product Marketing Manager at Google Cloud responsible for Databases. She has diverse experience across product marketing, marketing strategy and product management from leading enterprise software companies such as MariaDB, Teradata, SAP, Accenture, Cisco and Intuit. Amy received her Masters in Software Management from Carnegie Mellon University. Cool things of the week Process Workflows with the new Google Docs API blog Jib 1.0.0 is GA—building Java Docker images has never been easier blog GCP Podcast Episode 151: Java & Jib with Patrick Flynn and Mike Eltsufin podcast A guided tour in Google Earth that explores Black history blog Author: Gabe Weiss - Publishing series: Cloud IoT step-by-step Cloud IoT step-by-step: Connecting Raspberry PI + Python site Cloud IoT step-by-step: Cloud to device communication site Cloud IoT step-by-step: Quality of life tip - The command line site Interview Cloud SQL site Cloud SQL Features site MySQL site PostgreSQLsite Cloud MemoryStore site Cloud Bigtable site Cloud Firestore site Cloud Spanner site GCP Podcast Episode 62: Cloud Spanner with Deepti Srivastava podcast Mongo site Getting to know Google Cloud SQL video Question of the week What is a virtual column in a database? Generated columns blog and docs Where can you find us next? Amy will be at the Postgres Conference in New York on March 19. Gabi will be at PHP UK in London and Cloud NEXT in April. Mark will be at GDC in March, Cloud NEXT, and ECG in April. Diamond Partner Q&A: Google’s Mark Mandel Has The Tools To Help You Make Great Games article

An airhacks.fm conversation with Alexis (@alexismp) about: java -jar glassfish.jar, Community Management at Sun, Developer Relations, how to talk to developers, Texas Instruments 4a, a circle qualifies as "Hello World", Prolog to Java Applets migration for National French Space Agency, Java Center of Excellence at Sun Microsystems, Sun / JavaSoft / IBM as dream jobs, Scott McNealy and the ability of predicting the future - a reference to airhacks.fm episode #19 - interview with Scott McNealy, starting at Sun in 1998, Sun Netscape Alliance, iPlanet Appserver, moving a Reference Implementation to a product called "GlassFish", HK2, GlassFish started faster than Tomcat, moving the industry with GlassFish, fascination with modularity, NetBeans as platform, plugins as quality asurance, lightweight runtimes with 500 MB WARS, making servers bigger and deployables smaller, docker changed the conversation, dealing with boring technologies, different language communities at Google, Java is less ceremonial, than people think, the popularity of Java at Google, AppEngines 10th anniversary, Apache Beam and Google Dataflow, how Sun lost the engineers at Java 5 timeframe, a huge amount of Google projects is based on Java, AppEngine is "serverless", Sun and Google have a lot in common, JAX-RS is Google Cloud Endpoints, Managed PubSub service, PubSub is like JMS, AppEngine as PubSub message listener, Cloud Spanner -- a distributable scalable persistence, DataStore supports versioning is a document, key value store, canary deployments, Objectify an ORM for DataStore, Cloud SQL and PostgreSQL, BigTable, exports to BigQuery, istio , Kubernetes, Helidon on Google Cloud, Kubernetes Engine, you can find Alexis at twitter: @alexismp, LinkedIn, medium: @alexismp and his: blog.

Melanie and Mark talk with Google Cloud’s VP of Engineering, Melody Meckfessel, this week. In her time with Google Cloud, she and her team have worked to uncover what makes developers more productive. The main focus of their work is DevOps, defined by Melody as automation around the developer workflow and culture. In other words, Melody and her team are discovering new ways for developers to interact and how those interactions can encourage their productive peak. Melody and her team have used their internal research and expanded it to collaborate with Google Cloud partners and open source projects. The sharing of research and products has created even faster innovation as Google learns from these outside projects and vice versa. In the future, Melody sees amazing engagement with the community and even better experiences with containers on GCP. She is excited to see the Go community growing and evolving as more people use it and give feedback. Melody also speaks about diversity, encouraging everyone to be open-minded and try to build diverse teams to create products that are useful for all. Melody Meckfessel Melody Meckfessel is a hands-on technology leader with more than 20 years experience building and maintaining large-scale distributed systems and solving problems at scale. As VP of Engineering, she leads the team building DevOps tools and sharing DevOps best practices across Google and with software development and operations teams around the world. Her team powers the world’s most advanced continuously delivered software, enabling development teams to turn ideas into reliable, scalable production systems. After graduating from UC Berkeley, Melody programmed for startups and enterprise companies. Since joining Google in 2004, Melody has led teams in Google’s core search systems, search quality and cluster management. Melody is passionate about making software development fast, scalable, and fun. Cool things of the week Mark is back from vacation! We are at 2 million downloads! tweet Greg Wilson twitter and github Open source gaming: Agones - 0.6.0 - site Open Match - 0.2.0 RC - site What’s new at Firebase Summit 2018 blog Interview GCP Podcast Episode 137: Next Day 1 podcast Stackdriver site GitLab site Google SRE site Borg site Cloud Spanner site Go site GKE On-Prem site Skaffold site Minikube site DORA site Cloud Build site Bazel site Question of the week If I want to configure third party notifications (such as Slack or Github) into my Cloud Build configuration - how can I do that? Sending build notifications Configuring notifications for third-party services Where can you find us next? Mark will be at KubeCon next week. Melanie will be at NeurIPS this week. She’ll be attending Queer in AI, Black in AI, and LatinX this week as well.

It’s the third and final day for us at NEXT, and Mark and Melanie are wrapping up with some great interviews! First, we spoke with Stephanie Cueto and Vivian San of Techtonica, a San Francisco non-profit. Next, Liz Fong-Jones and Nikhita Raghunath joined us for a quick discussion about open source and Stackdriver and last but not least, Robert Kubis helped us close things sharing what it means to do DevRel at this event. Stephanie Cueto and Vivian San Stephanie Cueto is a Software Engineer and advocate for the Latinx & women community. She has been involved in the Tech community since 2016. Playing with code at an early age and working in education led to my interest in becoming a Software Engineer. Currently she is a Software Engineer Apprentice at Techtonica, where she has gained the skills to build projects in MongoDb, MySQL, Express.js, React, and Node.js. During the program, she created Salient Alert, a platform for reporting ICE Raids and Checkpoints. Vivian San is a highly analytical full-stack software engineer with an educational background in the hard sciences. She is strongly motivated by writing clean, efficient code, and passionate about teaching and giving back to underrepresented individuals and communities. Liz Fong-Jones and Nikhita Raghunath Liz Fong-Jones is a Staff Site Reliability Engineer at Google and works on the Google Cloud Customer Reliability Engineering team in New York. In her 10+ years at Google she has worked across eight different teams spanning the stack from Google Flights to Cloud Bigtable. She lives with her wife, Metamour, and a Samoyed/Golden Retriever mix in Brooklyn. In her spare time she plays classical piano, leads an EVE Online alliance, and advocates for transgender rights. Nikhita Raghunath is an intern at Red Hat and works on the extensibility of Kubernetes. Previously, she was a Google Summer of Code (2017) student for the Cloud Native Computing Foundation (CNCF) and also worked on Kubernetes. She is interested in backend applications, distributed systems and Linux. Nikhita likes programming in Go, C++, C, and Python. She also likes to give talks at conferences and speak about her work. Robert Kubis Robert Kubis is a developer advocate for the Google Cloud Platform based in London, UK, specializing in container, storage, and scalable technologies. Before joining Google, Robert collected over 10 years of experience in software development and architecture. He has driven multiple full-stack application developments at SAP with a passion for distributed systems, containers, and databases. In his spare time he enjoys following tech trends, trying new restaurants, traveling, and improving his photography skills. Interviews Made Here Together: NEXT Developer Keynote video Techtonica site I am Remarkable Workshop site Haben Girma’s accessibility presentation at NEXT video GCPPodcast Episode 127: SRE vs Devops with Liz Fong-Jones and Seth Vargo podcast Red Hat site Kubernetes site Introducing Agones blog Stackdriver site OpenCensus site GCPPodcast Episode 118: OpenCensus with Morgan McLean and JBD podcast Edge TPU site GCPPodcast Episode 135: VirusTotal with Emi Martínez podcast Cloud Spanner site

On this very special episode of the Google Cloud Platform Podcast, we have live interviews from the first day of NEXT! Melanie and Mark had the chance to chat with Melody MeckFessel, VP of Engineering at Google Cloud and Pavan Srivastava of Deloitte. Next we spoke with Sandeep Dinesh about Open Service Broker and Raejeanne Skillern of Intel. Melody Meckfessel Melody Meckfessel is a hands-on technology leader with more than 20 years experience building and maintaining large-scale distributed systems and solving problems at scale. As VP of Engineering, she leads the team building DevOps tools and sharing DevOps best practices across Google and with software development and operations teams around the world. Her team powers the world’s most advanced continuously delivered software, enabling development teams to turn ideas into reliable, scalable production systems. After graduating from UC Berkeley, Melody programmed for startups and enterprise companies. Since joining Google in 2004, Melody has led teams in Google’s core search systems, search quality and cluster management. Melody is passionate about making software development fast, scalable and fun. Pavan Srivastava Pavan is a technology leader with 20 years of experience in developing strategies and implementation of SAP focused technology solutions. Pavan leads Deloitte’s SAP technology capability that focuses on helping clients adopt innovative technology solutions such as cloud and SAP HANA to improve business efficiencies. Pavan has led several engagements helping clients develop strategy, architecture and implement SAP on the cloud and SAP HANA platform. Sandeep Dinesh Sandeep Dinesh is a Developer Advocate for Google Cloud. He blends and creates new opportunities for businesses and people by leveraging the best technology possible. Raejeanne Skillern Raejeanne Skillern is the VP of Data Center and General Manager of Intel’s cloud service provider (CSP) business. Her goal is to make it easier, more cost-effective and more efficient for CSPs to build new infrastructure and services. She is privileged to lead an exceptional team that manages Intel’s business, products and technologies for cloud infrastructure deployments and works closely with the world’s largest cloud providers to ensure Intel’s data center products are optimized for their unique needs. Interviews Cloud AutoML site GKE On-Prem site Melody Meckfessel’s Speaking Schedule at NEXT site DevOps site Google Open Source site Cloud Build site Spinnaker site Kubernetes site Stackdriver site Application Performance Management site OpenCensus site Deloitte site SAP site Deloitte and Google Cloud blog Google Cloud Platform Service Broker site Open Service Broker site Pub/Sub site Cloud Spanner site Intel Cloud Computing site Intel Xeon site Intel Optane DC Persistent Memory site Partnering with Intel and SAP on Intel Optane DC Persistent Memory for SAP HANA blog Where can you find us next? We’ll both be at Cloud NEXT in Moscone West on the first floor! Come by and say hi!

On this episode of the podcast, Melanie and Mark talk with Emiliano (Emi) Martínez to learn more about how VirusTotal is helping to create a safer internet by providing tools and building a community for security researchers. Emiliano (Emi) Martínez Emiliano has been with VirusTotal for over 10 years. He has seen the business grow from a small startup in southern Spain into a Google X moonshot under the new Chronicle bet. He is a software engineer acting as the Tech Lead for VirusTotal. Throughout the past 10 years, not only has he been immersed in coding and architecting the platform, but he has also participated at all levels of the business: from bootstrapping the very first sales to working close with marketing and other teams in order to take the project to the next level. His main interests are IT security (more specifically malware) and designing products and services from scratch. VirusTotal and Chronicle are Hiring VirusTotal is part of Chronicle, and Chronicle is hiring! Come join our team experts to help build out the next generation of security intelligence solutions. We are looking for talent that is comfortable operating in an organization that is scaling quickly, that loves variety in their work and wants to get their hands dirty with all things cyber security, cloud computing, and machine learning. We are a dynamic organization that likes to run experiments so we are looking for colleagues that are excited about trying new things and offering a creative yet efficient, and client-centric approach to engineering solutions. You are scrappy and resourceful, creative and driven – and excited to share in the magic of working at Chronicle Cool things of the week BigQuery in June: a new data type, new data import formats, and finer cost controls blog Dataflow Stream Processing now supports Python blog Associate Cloud Engineer blog Six AI & ML Sessions to Attend at NEXT blog Interview VirusTotal site VirusTotal Use Cases site and videos VirusTotal Intelligence site VirusTotal Malware Hunting site VirusTotal Monitor site VirusTotal APIs site VirusTotal Community site VirusTotal Contact site Data Connectors San Jose on July 12, 2018 site Data Connectors Raleigh on July 26, 2018 site BSides Las Vegas on August 7-8, 2018 site If you are interested in a 1:1 meeting with VirusTotal, please email info@virustotal.com Google Cloud App Engine site Google Compute Engine site Google Cloud Kubernetes Engine site BigQuery site Google Cloud Data Studio site Google Cloud MemoryStore site Google Cloud SQL site G Suite site Question of the week This week’s question comes from Andrew Sheridan, with a special guest answer from Robert Kubis. What is the best practice for multi tenancy in Google Cloud Spanner, especially if customers are not of the same size and have unequal load? What DBAs need to know about Cloud Spanner, part 1: Keys and indexes blog Cloud Spanner - Choosing the Right Primary Keys video More questions about Spanner? Robert will be presenting on it at Cloud NEXT. Where can you find us next? We’ll both be at Cloud NEXT! Melanie will speak at CERN July 17th and PyCon Russia July 22nd

Julien Anguenot d'Iland Cloud - https://www.iland.com/ - nous parle de CockroachDB - https://www.cockroachlabs.com/ - le clone open source de Cloud Spanner - https://cloud.google.com/spanner/) qu'il a mis en production en complément d'Apache Cassandra, pour des besoins internes.Julien et Alexander participeront en tant que speakers au prochain Paris Cassandra Meetup le 31 Janvier 2018 chez Deezer : https://www.meetup.com/fr-FR/Cassandra-Paris-Meetup/events/246902065/On a ouvert un Slack : bigdatahebdo.slack.cominvitation par DM @bigdatahebdo ou sur contact@bigdatahebdo.comLisez le blog D'affini-Techhttp://blog.affini-tech.com----------------------------------------------------------------------------------------http://www.bigdatahebdo.comhttps://twitter.com/bigdatahebdoVincent : https://twitter.com/vhe74Alexander : https://twitter.com/alexanderdeja & http://thelastpickle.com/blogJulien : https://twitter.com/anguenotCette publication est sponsorisée par Affini-Tech ( http://affini-tech.com https://twitter.com/affinitech )On recrute ! venez cruncher de la data avec nous ! écrivez nous à recrutement@affini-tech.com

Julien Anguenot d'Iland Cloud - https://www.iland.com/ - nous parle de CockroachDB - https://www.cockroachlabs.com/ - le clone open source de Cloud Spanner - https://cloud.google.com/spanner/) qu'il a mis en production en complément d'Apache Cassandra, pour des besoins internes.Julien et Alexander participeront en tant que speakers au prochain Paris Cassandra Meetup le 31 Janvier 2018 chez Deezer : https://www.meetup.com/fr-FR/Cassandra-Paris-Meetup/events/246902065/On a ouvert un Slack : bigdatahebdo.slack.cominvitation par DM @bigdatahebdo ou sur contact@bigdatahebdo.comLisez le blog D'affini-Techhttp://blog.affini-tech.com----------------------------------------------------------------------------------------http://www.bigdatahebdo.comhttps://twitter.com/bigdatahebdoVincent : https://twitter.com/vhe74Alexander : https://twitter.com/alexanderdeja & http://thelastpickle.com/blogJulien : https://twitter.com/anguenotCette publication est sponsorisée par Affini-Tech ( http://affini-tech.com https://twitter.com/affinitech )On recrute ! venez cruncher de la data avec nous ! écrivez nous à recrutement@affini-tech.com

This week we get the band back together! Francesc Campoy Flores rejoins the show along with Director of Google Cloud Developer Relations Greg Wilson to talk all about 2017 and Google Cloud with Mark and Melanie About Francesc Campoy Flores Francesc Campoy Flores is the VP of Developer Relations at source{d}, He's also a Gopher, Catalan, LGBTQIA advocate, previous Google employee (and Podcast host), and creator of the Just For Func YouTube series! About Greg Wilson Greg Wilson is the Director of Google Cloud Developer Relations, overseeing developer relations work across both G Suite and Google Cloud Platform. Cool things of the week Jeff Dean's talk at NIPS on ML for Systems and Systems for ML sides The Case for Learned Index Structures paper KubeFlow github hackernews Manage Google Kubernetes Engine from Cloud Console dashboard, now generally available blog Interview Top 5 Downloaded Episode of 2017 #88 Kubernetes 1.7 with Tim Hockin #91 The Future of Media with Machine Learning with Amit Pande #93 What's AI with Melanie Warrick #75 Container Engine with Chen Goldberg #100 Vint Cerf: past, present, and future of the internet Greg's Favourites #57 Pokémon GO with Edward Wu, Director of Software Engineering at Niantic #68 The Home Depot with William Bonnell #86 Broad Institute and Platinum Customers with Lukas Karlsson and Mike Altarace Francesc's Favourites #62 Cloud Spanner with Deepti Srivastava Mark's Favourites The SRE Category on GCP Podcast Melanie's Favourites #57 Pokémon GO with Edward Wu, Director of Software Engineering at Niantic Favourite announcements, products and more at Google Cloud Platform Cloud Spanner Cloud Machine Learning Engine TensorFlow GCE Virtual Machines, e.g. Pre-emptible VMs Go 1.8 on App Engine Cheaper GPUs Kubernetes Question of the week What were your personal highlights for 2017? Mark Getting involved with SIG API Machinery with Kubernetes Melanie Watching Haben Girma, the first Deafblind Graduate of Harvard Law School, speak about accessibility in tech. Where can you find us next? It's the end of the year! So we'll be taking a break, and returning in January 2018!

In this episode of the ARCHITECHT Show, Deepti Srivastava, lead product manager for Cloud Spanner at Google, discusses that service's new multi-region capabilities and Google's promise of 99.999% availability. She also shares some thoughts on the best use cases for Spanner and other cloud databases; the importance of database options in choosing a cloud provider; and the evolution of database technology that got us here -- "here" being geographically distributed, ACID-compliant SQL databases configurable with just a few clicks.

Colt McAnalis joins the podcast this week to talk about his Performance Atlas series where he dives into how to make Google Cloud applications faster and cheaper. In his words, his job is to help get someone promoted. About Colt McAnlis Colt McAnlis is a Developer Advocate at Google focusing on performance & compression. Before that, he was a graphics programmer in the games industry working at Blizzard, Microsoft (Ensemble), and Petroglyph. He's been an Adjunct Professor at SMU Guildhall, a UDACITY instructor (twice), and a Book Author, (twice). When he's not working with developers, Colt spends his time preparing for an invasion of giant ants from outer space. Cool things of the week With Multi-Region Support in Cloud Spanner, have your cake and eat it too blog The State of Data Science & Machine Learning by Kaggle blog and podcast Introducing Certified Kubernetes (and Google Kubernetes Engine!) blog Interview Performance Atlas series Profiling App Engine (Standard) Boot Time video TCP BBR site Cloud Functions site docs Understanding Compression book Google SRE book TCP/IP Illustrated book Ilya Grigorik site Perf Like a Pirate III site Question of the week What are the differences between sustained and committed use discounts? Sustained Use Discounts docs Committed Use Discounts docs Where can you find us next? Mark will be Montreal in December to speak at Montreal International Games Summit.

This is a special crossover episode of Cross Cutting Concerns with the Eat Sleep Code podcast, hosted by Ed Charbeneau (Microsoft MVP). This was recorded at the Stir Trek conference. Show Notes: Eric Brewer: One of his recent blog posts was about Cloud Spanner and the CAP Theorum Check out the blogs at Telerik, and check out Ed on Telerik's developer portal Couchcase: Github repo, blog posts Ed's website, EdCharbeneau.com Machine Learning for Developers This episode was published to Microsoft's Channel 9 and also Telerik's Develper Portal Ed Charbeneau is on Twitter Want to be on the next episode? You can! All you need is the willingness to talk about something technical. Theme music is "Crosscutting Concerns" by The Dirty Truckers, check out their music on Amazon or iTunes.

This week brings us back to an interview that we did while at Cloud Next last week. Mark and Francesc talk to William Bonnell, Senior Director of SRE at The Home Depot all about SRE culture, and the CRE team as well. About William Bonnell William Bonnell is Senior Director of Site Reliability Engineering at The Home Depot - managing the e-commerce and order management systems, support millions of customers per day! Cool things of the week 100 announcements (!) from Google Cloud Next ‘17 blog Identity-Aware Proxy (IAP) for Google Cloud Platform (Beta) site Cloud.google.com/community site Cloud SQL for Postgre SQL (Beta) site 64 Core machines + more memory blog A new issue tracker for Google Cloud Platform blog Happy Pi Day! site Interviews 24⁄7 resiliency (Google Cloud Next ‘17) youtube Smart, Secure, and Modern app delivery for enterprises and cloud-natives (Google Cloud Next ‘17) youtube Building Microservices book Production-Ready Microservices book Site Reliability Engineering book Introducing Google Customer Reliability Engineering blog Managed Instance Groups docs Question of the week Why should I be using Cloud Spanner, rather than Cloud SQL? (Thanks AJ!) What's the difference between Google Cloud Spanner and Cloud SQL? quora Cloud Spanner docs Cloud Spanner Pricing docs Where can you find us next? Mark will be heading to Polyglot Vancouver Meetup in April, and then on to East Coast Games Conference and Vector Francesc will be presenting at Gophercon China in April.

On the heels of the Cloud Spanner launch, Deepti Srivastava joins your hosts Francesc and Mark on this week's podcast to talk all about this globally distributed, horizontally scalable, relational database that also provides global consistency and ACID transactions! About Deepti Deepti Srivastava is passionate about technology and its ability to be a positive change enabler. As Product Manager for Cloud Spanner, on Google's Cloud Platform, Srivastava works on best in class Cloud Databases and Storage technologies. Srivastava is an enthusiastic member of Women@Google and a passionate advocate of STEM education, especially for girls. She also enjoys dancing, snowboarding and all things fashion. Cool thing of the week Each Google Cloud Product described in 4 words or less tweet doc Google Cloud and YouTube-8M Challenge blog Interview Cloud Spanner site docs Introducing Cloud Spanner: a global database service for mission-critical applications blog Inside Cloud Spanner and the CAP Theorem blog Quizlet Tests Cloud Spanner — The Most Sophisticated Cloud Database blog Don't Give Up on Serializability Just Yet • Neha Nerula youtube CAP Theorom wikipedia Spanner: Google's Globally-Distributed Database white paper Spanner, TrueTime and the CAP Theorem white paper Cloud Next: Cloud Spanner 101: Google's mission-critical relational database schedule Cloud Next: Cloud Spanner 201: getting the most out of Cloud Spanner schedule Ben Sigelman will present Spanner: Google's Globally-Distributed Database youtube Spanner: No-Compromise Relational Database Service Question of the week How do I run a mail server on the cloud? Sending Mail from a Virtual Machine docs Sending Email with SendGrid docs Sending Email with Mailgun docs Sending Email with Mailjet docs SMTP relay: Route outgoing non-Gmail messages through Google docs Were will we be? Mark will be at GDC and afterwards he'll be speaking at Cloud NEXT, both in San Francisco. Francesc will be at Gophercon India, at Cloud NEXT, and then Gopher China.

Coconauts es un podcast sobre tecnologia, desarrollo, gamedev, making y cosas frikis en general. En este episodio hablamos de: - Noticias: el outage de Gitlab, Steam cierra Greenlight, RethinkDB y Cloud Spanner, Gameband. - Yo he venido aquí a hablar de mi libro: la psicología de los objetos cotidianos de Don Norman - Debate: frameworks de desarrollo de videojuegos Enlaces: - Gitlab outage postmortem: https://about.gitlab.com/2017/02/10/postmortem-of-database-outage-of-january-31/ - Steam greenlight: http://steamcommunity.com/greenlight/discussions/18446744073709551615/133256758580075301/ - RethinkDB: https://rethinkdb.com/blog/rethinkdb-joins-linux-foundation/ - Google Cloud Spanner: https://cloudplatform.googleblog.com/2017/02/introducing-Cloud-Spanner-a-global-database-service-for-mission-critical-applications.html - Gameband: https://www.kickstarter.com/projects/gameband/gameband-the-first-smartwatch-for-gamers?ref=category_newest - Watchduino: https://www.youtube.com/watch?v=CtgR1YiwnEY - La psicologia de los objetos cotidianos: https://www.amazon.es/psicolog%C3%ADa-objetos-cotidianos-Serie-Media/dp/8415042019 - Comparativa de game frameworks: http://coconauts.net/blog/2017/01/09/2d-game-framework-comparison/ Music from Jukedeck - create your own at http://jukedeck.com

In this episode, we discuss quotes and CPQ, using font-end frameworks with Salesforce, Chris Rock's controversial routine at the Salesforce Annual Sales Kick-off Meeting, IsNull vs IsBlank, leaving code better than how you found it, and Google's Spanner database service. Studio Neat Ice Kit Wintersmith's Ice Baller Chris Rock Draws Laughs, Controversy at Salesforce Sales Meeting Dropbox Paper Salesforce launches Quip Connect app for Sales and Service Clouds Introducing Cloud Spanner: a global database service for mission-critical applications CLOUD SPANNER Why does it cost 20 times as much to protect Mark Zuckerberg as Tim Cook?

Eépisode chargé en sujets: langages, web, data, big data, sécurité, organisation sans oublier Donald (pas le neveu de Picsou). Merci à Saagie pour leur aide ! Enregistré le 15 février 2017 Téléchargement de l’épisode LesCastCodeurs-Episode–163.mp3 News Langages Les nouveautés de Java Time dans Java SE 9 Jigsaw et les automodules on vous aura prévenu … Groovy is the new black Francesc Campoy qui donne une overview de Go 1.8 Impact de la structure de l’API sur les performance - Go et logging Les tags du week-end sur Stack Overflow Web Le top 2016 du Javascript : Basé sur les étoiles github Basé sur un sondage Front: Vue.Js au top Build : Grunt est mort, Gulp en baisse et Webpack devient la référence. IDE : Visual Studio Code et Atom La : Jasmine et Mocha toujours là, AVA et Jest sont les nouveaux. Gros impact de facebook! Déployer une application Ratpack sur Google App Engine Flex Les 10 meilleurs frameworks web Java de 2017 Performance de démarrage de JavaScript (optionnel) Retour sur React Native par Instagram Middleware Hazelcast lance Jet, un stream processing engine OSS MiniShift: deploiement local pour OpenShift Les produits Google résumés en 4 mots gRPC chez la Cloud Native Computing Foundation Bean Validation 2.0 early draft La spec MVC, transférée à Ivar Grimstad Data Ransomware contre Elasticsearch Google lance son Cloud Spanner, sa supra base de données distribuée ACID Google Cloud Spanner viole-t-il le théorême de CAP? Google Cloud Spanner Post-mortem ReThinkDB ReThinkDB rejoins la fondation Linux et CNCF Réécrire son appli RethinkDB avec PostgreSQL PostgreSQL capable de traiter plein de use cases différents Spark 2.1 Kudu 1.2 Investissement en cours sur la Data Gouvernance Intel Big DL, grosse bataille Intel versus NVidia sur le Deep Learning : Les slides de Intel AI la réponse de Nvidia Construiser votre CSS avec du Deep Learning Jeff Dean sur l’état du Machine Learning aujourd’hui Jeff Dean sur Tensorflow Article du New York Times sur l’intelligence artificielle et Google Brain Podcast Big Data (et autre) : Roaring Elephant Podcast Software Engineering Daily Saagie Saagie est une start-up éditeur dans le Big Data. Bon tout le monde parle de Big Data et c’est un peu le mot à la mode, mais chez Saagie nous faisons vraiment du BIG DATA. Nous éditons Saagie Data Fabric pour industrialiser les mises en production de big data et de data science (Data Fabric ~ Plateforme de développement Data) avec l’option Saagie Data Governance pour organiser votre lac de données. L’ensemble est déployable sur notre cloud, sur amazon ou azure et via notre appliance avec option Deep Learning et HPC. Les postes ouverts Sécurité Retour de la FIC DevOps Rise of the ChatOps : Netflix Hub Commander Hubot GitLab et ses backups Open Source Github annonce les guides OSS pour aider à monter/contribuer à l’OSS Open Source Guides Organisation/Agile Talk Beyond breaking bad noproject Je n’embauche jamais de poisson panné par Quentin Adam Rise of the Data Engineer Outillage Utiliser Ngrok, Google Cloud Functions, API.AI pour faire des bots Microsoft annonce un Git File System Jenkins Declarative Pipeline 1.0 (+ Pipeline Editor Preview) et SCM API 2.0 Des slides HTML en Markdown exportés en PDF Société La France introduit des visas particuliers pour les entrepreneurs D’un trait de plume Donald Trump manque de faire exploser le Privacy Shield Outil de l’épisode JVM mon Conférences Quelques conseils pour écrire une proposition de conférence 10-ways-for-a-conference-to-upset-their-speakers - Troy Hunt 10-ways-for-speaker-to-upset-conference - Nicolas Deloof Le ParisJUG se lance dans adopt-a-JSR: un message en crowdcast 15 mars 2017 : soirée Hands-on / Hackergarten sur Jigsaw, animée par Rémi Forax et l’équipe du Paris JUG. le site du Paris JUG le twitter : @ParisJUG les détails des soirées et la liste de ce qu’il faudra installer sur son portable pour la partie Hands-on / Hackergarten seront publiés sur le site du Paris JUG. page communautaire d’écrivant le programme Adopt a JSR blog d’oracle annonçant le programme Adopt a JSR Salon Big Data Paris les 5–6 mars 2017 Printemps Agile le 9 mars 2017 Devoxx France les 5–7 avril 2017 Mix-IT les 20–21 avril 2017 Breizhcamp les 19–21 avril 2017 RivieraDev les 11–12 mai 2017 DevFest Lille 9 juin - inscriptions et CfP ouvert Voxxed Days au Luxembourg Nous contacter Faire un crowdcast ou une crowdquestion Contactez-nous via twitter https://twitter.com/lescastcodeurs sur le groupe Google https://groups.google.com/group/lescastcodeurs ou sur le site web https://lescastcodeurs.com/ Flattr-ez nous (dons) sur https://lescastcodeurs.com/ En savoir plus sur le sponsoring? sponsors@lescastcodeurs.com