Podcasts about PostgreSQL

Links: Elaastic Throughput for EFS Front any existing MySQL or PostgresQL with a GraphQL API Security Hub Multi-Region and Multi-Account Amazon Detective announces investigations for IAM Stay Up To Date with re:Quinnvent Sign up for the re:Quinnvent Newsletter Join the re:Quinnvent Scavenger Hunt Check out the re:Quinnvent playlist on YouTube Help the show Share your feedback Follow wherever you get your podcasts Buy our merch What's Corey up to? Follow Corey on Twitter (@quinnypig) See our recent work at the Duckbill Group Apply to work with Corey and the Duckbill Group to help lower your AWS bill

In this episode of Startup Hustle, Matt and Lauren talk about Pittsburg's Top Startups! Tune in to hear what's happening in Steel City's startup scene and meet 10 emerging growth companies poised for a disruptive 2024.    Find Startup Hustle Everywhere: https://gigb.co/l/YEh5   This episode is sponsored by Full Scale: https://fullscale.io   Learn more about InnovateHER KC: https://www.innovateherkc.com   Meet all the Top Startups we've featured around the US: https://gigb.co/l/x1fD   Episode Highlights:  Intro to the Pttisburg Top Startups episode (0:31) Koop Technologies - Insurance for all automation risks (2:58) Mach9 - Automated geospatial production platform (4:53) Mapless AI - One-stop, plug-in solution for remote car operation (7:56) Netail - AI SaaS solutions for the retail and consumer industry (11:03) OtterTune - AI-powered automatic PostgreSQL and MySQL tuning (16:38) Resilient Lifescience - Wearable medical device to prevent opioid overdose deaths (21:31) Stack AV - Autonomous trucking solutions (25:49) Stratus Materials - Next-generation Lithium-Ion Cathode active materials (27:35) Thoro.ai - Providing superior autonomy solutions (31:37) TRAiNED - Harnessing the skills of a Neurodivergent Workforce (36:11) Lauren's favorite Pittsburg top startups: (43:24) Matt's favorite Pittsburg startup: (44:20) AI is here to stay (47:50)   Read the feature on Inc.com: https://www.inc.com/inc-masters/pittsburgh-startups-to-watch-in-2024.html  See omnystudio.com/listener for privacy information.

Summary: FOSS4GNA 2023 presentation on using open source software at the Census Bureau, focusing on the Geographic Update Partnership Software (GUPS), a GIS solution built on QGIS framework. The presentation covers the benefits of open source for federal government, including security, flexibility, free support, and cost savings. The GUPS system integrates QGIS, PostgreSQL, GeoServer, open layers, and more. A demo of GUPS Web showcases features like map management, user invitation, role-based access, and the boundary review tool for automatic change detection. Highlights:

AWS Morning Brief for the week of November 13, 2023, with Corey Quinn. Show Notes:Links: Amazon Aurora Global Database for PostgreSQL now supports write forwarding Amazon SQS announces support for JSON protocol AWS Cost Management now provides purchase recommendations for Amazon MemoryDB Reserved Nodes Introducing the Generative AI Center of Excellence for AWS Partners: The Path to AI Expertise New – Block Public Sharing of Amazon EBS Snapshots New for Amazon Comprehend – Toxicity Detection AWS CodeBuild adds support for AWS Lambda compute mode An Overview of Bulk Sender Changes at Yahoo/Gmail Creating a correction of errors document Know Before You Go – AWS re:Invent 2023 | AWS Management Console Unhoused individuals gain shelter, prove their identity using AWS-powered solution Kiip How VMware partnered with AWS to nurture a culture of sustainability How to Stop Feeding AWS's AI With Your Data

Jonathan Katz, a principal product manager at Amazon Web Services, discusses the evolution of PostgreSQL in an episode of The New Stack Makers. He notes that PostgreSQL's uses have expanded significantly since its inception and now cover a wide range of applications and workloads. Initially considered niche, it faced competition from both open-source and commercial relational database systems. Katz's involvement in the PostgreSQL community began as an app developer, and he later contributed by organizing events.PostgreSQL originated from academic research at the University of California at Berkeley in the mid-1980s, becoming an open-source project in 1994. In the mid-1990s, proprietary databases like Oracle, IBM DB2, and Microsoft SQL dominated the market, while open-source alternatives like MySQL, MariaDB, and SQLite emerged.PostgreSQL 16 introduces logical replication from standby servers, enhancing scalability by offloading work from the primary server. The meticulous design process within the PostgreSQL community leads to stable and reliable features. Katz mentions the development of Direct I/O as a long-term feature to reduce latency and improve data writing performance, although it will take several years to implement.Amazon Web Services has built Amazon RDS on PostgreSQL to simplify application development for developers. This managed service handles operational tasks such as deployment, backups, and monitoring, allowing developers to focus on their applications. Amazon RDS supports multiple PostgreSQL releases, making it easier for businesses to manage and maintain their databases.Learn more from The New Stack about PostgreSQL and AWS:PostgreSQL 16 Expands Analytics CapabilitiesPowertools for AWS Lambda Grows with Help of VolunteersHow Donating Open Source Code Can Advance Your Career

Summary Databases are the core of most applications, but they are often treated as inscrutable black boxes. When an application is slow, there is a good probability that the database needs some attention. In this episode Lukas Fittl shares some hard-won wisdom about the causes and solution of many performance bottlenecks and the work that he is doing to shine some light on PostgreSQL to make it easier to understand how to keep it running smoothly. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. You specify the customer traits, then Profiles runs the joins and computations for you to create complete customer profiles. Get all of the details and try the new product today at dataengineeringpodcast.com/rudderstack (https://www.dataengineeringpodcast.com/rudderstack) You shouldn't have to throw away the database to build with fast-changing data. You should be able to keep the familiarity of SQL and the proven architecture of cloud warehouses, but swap the decades-old batch computation model for an efficient incremental engine to get complex queries that are always up-to-date. With Materialize, you can! It's the only true SQL streaming database built from the ground up to meet the needs of modern data products. Whether it's real-time dashboarding and analytics, personalization and segmentation or automation and alerting, Materialize gives you the ability to work with fresh, correct, and scalable results — all in a familiar SQL interface. Go to dataengineeringpodcast.com/materialize (https://www.dataengineeringpodcast.com/materialize) today to get 2 weeks free! Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics. Trusted by teams of all sizes, including Comcast and Doordash, Starburst is a data lake analytics platform that delivers the adaptability and flexibility a lakehouse ecosystem promises. And Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data. Want to see Starburst in action? Go to dataengineeringpodcast.com/starburst (https://www.dataengineeringpodcast.com/starburst) and get $500 in credits to try Starburst Galaxy today, the easiest and fastest way to get started using Trino. This episode is brought to you by Datafold – a testing automation platform for data engineers that finds data quality issues before the code and data are deployed to production. Datafold leverages data-diffing to compare production and development environments and column-level lineage to show you the exact impact of every code change on data, metrics, and BI tools, keeping your team productive and stakeholders happy. Datafold integrates with dbt, the modern data stack, and seamlessly plugs in your data CI for team-wide and automated testing. If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold (https://www.dataengineeringpodcast.com/datafold) Your host is Tobias Macey and today I'm interviewing Lukas Fittl about optimizing your database performance and tips for tuning Postgres Interview Introduction How did you get involved in the area of data management? What are the different ways that database performance problems impact the business? What are the most common contributors to performance issues? What are the useful signals that indicate performance challenges in the database? For a given symptom, what are the steps that you recommend for determining the proximate cause? What are the potential negative impacts to be aware of when tuning the configuration of your database? How does the database engine influence the methods used to identify and resolve performance challenges? Most of the database engines that are in common use today have been around for decades. How have the lessons learned from running these systems over the years influenced the ways to think about designing new engines or evolving the ones we have today? What are the most interesting, innovative, or unexpected ways that you have seen to address database performance? What are the most interesting, unexpected, or challenging lessons that you have learned while working on databases? What are your goals for the future of database engines? Contact Info LinkedIn (https://www.linkedin.com/in/lfittl/) @LukasFittl (https://twitter.com/LukasFittl) on Twitter Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. Podcast.__init__ (https://www.pythonpodcast.com) covers the Python language, its community, and the innovative ways it is being used. The Machine Learning Podcast (https://www.themachinelearningpodcast.com) helps you go from idea to production with machine learning. Visit the site (https://www.dataengineeringpodcast.com) to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@dataengineeringpodcast.com (mailto:hosts@dataengineeringpodcast.com)) with your story. To help other people find the show please leave a review on Apple Podcasts (https://podcasts.apple.com/us/podcast/data-engineering-podcast/id1193040557) and tell your friends and co-workers Links PGAnalyze (https://pganalyze.com/) Citus Data (https://www.citusdata.com/) Podcast Episode (https://www.dataengineeringpodcast.com/citus-data-with-ozgun-erdogan-and-craig-kerstiens-episode-13/) ORM == Object Relational Mapper (https://en.wikipedia.org/wiki/Object%E2%80%93relational_mapping) N+1 Query (https://docs.sentry.io/product/issues/issue-details/performance-issues/n-one-queries/) Autovacuum (https://www.postgresql.org/docs/current/routine-vacuuming.html#AUTOVACUUM) Write-ahead Log (https://en.wikipedia.org/wiki/Write-ahead_logging) pgstatio (https://pgpedia.info/p/pg_stat_io.html) randompagecost (https://postgresqlco.nf/doc/en/param/random_page_cost/) pgvector (https://github.com/pgvector/pgvector) Vector Database (https://en.wikipedia.org/wiki/Vector_database) Ottertune (https://ottertune.com/) Podcast Episode (https://www.dataengineeringpodcast.com/ottertune-database-performance-optimization-episode-197/) Citus Extension (https://github.com/citusdata/citus) Hydra (https://github.com/hydradatabase/hydra) Clickhouse (https://clickhouse.tech/) Podcast Episode (https://www.dataengineeringpodcast.com/clickhouse-data-warehouse-episode-88/) MyISAM (https://en.wikipedia.org/wiki/MyISAM) MyRocks (http://myrocks.io/) InnoDB (https://en.wikipedia.org/wiki/InnoDB) Great Expectations (https://greatexpectations.io/) Podcast Episode (https://www.dataengineeringpodcast.com/great-expectations-data-contracts-episode-352) OpenTelemetry (https://opentelemetry.io/) The intro and outro music is from The Hug (http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/Love_death_and_a_drunken_monkey/04_-_The_Hug) by The Freak Fandango Orchestra (http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/) / CC BY-SA (http://creativecommons.org/licenses/by-sa/3.0/)

Timestamps:- 0:00 Intro- 1:48 Releases 428-430- 6:30 Introducing Denis Magda from @YugabyteDB - 7:56 JDBC, Trino's JDBC driver, and the Postgres connector- 14:08 Introducing YugabyteDB- 21:33 Demo time! Trino with PostgreSQL- 29:56 Demoing Trino with YugabyteDB- 44:57 Failover and resiliency- 56:05 Upcoming events and Trino Summit soon!

In this episode, Julie begins by expressing concern about technical issues with Zoom and her camera being off, Andrew discusses the shortage of certain ADHD medication in the U.S. due to supply issues, and then the topic of keeping tidy beds and using chopsticks for eating comes up.   The conversation then shifts to Andrew introducing asdf as a runtime version manager and he dives into the complexity of managing different versions of tools.  The discussion touches on Docker, version consistency, and Podia's smooth transition to a new database version. They emphasize the importance of planning and backups during database upgrades and share insights into moving local databases. We end with a reflection on microservices and their evolving role in the tech industry. Hit download now to hear more! [00:00:13] Julie expresses concern about Zoom not working and her camera being off, and Andrew shares his thoughts on the shortage of ADHD medication in the U.S and how it's been affecting his life. [00:02:59] The topic of keeping their beds tidy comes up, bed habits, and converting to chopsticks for eating. [00:08:15] Andrew explains that he was helping a designer upgrade his Postgres database from version 13 to 15. He mentions the challenge of not knowing how the designer was running Postgres initially and the decision to use GUI app (Postgres app) for ease of use. [00:11:38] What is asdf?  Andrew asks Julie if she uses rbenv or RVM, and he explains that rbenv is a runtime version manager for Ruby and discusses the need for managing multiple versions of Ruby due to the default outdated version in macOS. Also, Andrew explains that it's a tool that manages various services, including Redis, Yarn, Node, Ruby, and more. [00:13:49] Julie brings up if Andrew had any issues with asdf, and he acknowledges some past issues, but has a strong preference for using asdf consistently.[00:14:12] Julie asks how Andrew's use of asdf at work compares to others who might not use it, and he relates the discussion to Docker, explaining that Docker containers can help standardize environments and avoid issues related to different versions of tools and services on different machines. [00:16:48] Julie asks whether applications can still run smoothly even if developers are using different version of tools like Redis.  Andrew confirms it's possible but explains the potential pitfalls and the importance of version consistency. [00:20:00] Andrew discusses the switch from database version 13 to 15 in production and mentions the challenges of upgrading databases compared to simpler updates like changing a Node.js version. [00:20:43] Julie asks about Podia's staging environment for testing changes before deploying to production, and Andrew mentions using Heroku pipelines and how the company relies on feature flags and aims to keep the main branch always deployable.[00:22:05] Andrew explains the potential for breaking changes when upgrading database versions, mentioning the PostgreSQL 16 is coming out and Heroku plans to support version 14 through 16 while dropping support for version 13. [00:23:14] Andrew tells us their backup plan in case the database upgrade to version 15 fails and discusses the importance of having backups and a fast recovery plan. [00:25:43] Julie asks Andrew what he meant by SSH into a box and about moving a local database to another computer, and he explains that it's possible using commands like PG dump and restore, as well as providing options for moving a local database.  [00:29:55] Why doesn't Podia use Docker? Andrew explains, citing previous experiences with Docker and the complexity it can introduce.[00:32:22] Julie wonders if Andrew uses microservices, to which he explains that when he began his career, microservices were popular, but the industry's perspective has since shifted.  Docker was initially associated with microservices, but the trend has changed over time.[00:33:01] What are the benefits of using microservices and why have some companies moved away from them? Andrew tells us the complexity of microservices became more apparent as they were adopted more widely, and some now consider them an anti-pattern. Panelists:Andrew MasonJulie J.Sponsor:GoRailsLinks:Andrew Mason TwitterAndrew Mason WebsiteJulie J. TwitterJulie J. WebsitePostgres.apprbenvRuby Version Manager (RVM)asdfPostgreSQL 16

How do you choose an Azure data storage solution? Richard talks to Nicole Stevens about her experiences helping companies move data into the cloud - typically in SQL Server. The obvious answer is Azure SQL, but that doesn't always make it the best solution! Nicole talks about a customer moving to Cosmos DB for the unstructured data capabilities and a lot of speed. But does the price make sense? All these factors are in play in choosing a data storage solution, and there is never one right choice - often, a mix of services makes the most sense!Links:Azure SQLAzure Cosmos DBAPIs in Cosmos DBAzure Table StorageAzure Queue StorageAzure Messaging Servicesdapr - Distributed Application RuntimeMicrosoft CertificationsData Architecture GuideData Landing Zones Azure Security BaselinesRecorded September 6, 2023

The Elixir community has a new OpenSource CMS thanks to DockYard and Leandro Pereira! We talk with Leandro to better understand what the Beacon project is and what it can do. It's built using Phoenix LiveView and can be deployed standalone or as part of an existing Elixir Phoenix application. The Admin features include the ability for non-developers to edit content and immediately deploy changes without re-deploying the app. We talk about how BeaconCMS is positioned against Wordpress and static sites and where the greatest benefits are. A feature in development is the HEEx template editor which promises to be a very exciting tool for non-developers. Learn along with us! Show Notes online - http://podcast.thinkingelixir.com/174 (http://podcast.thinkingelixir.com/174) Elixir Community News - https://twitter.com/chris_mccord/status/1713894354962534808 (https://twitter.com/chris_mccord/status/1713894354962534808?utm_source=thinkingelixir&utm_medium=shownotes) – LiveView optimized DOM patching announced, resulting in significant speed improvements in browser's DOM patching time. - https://twitter.com/basilenouvellet/status/1713981828028133847 (https://twitter.com/basilenouvellet/status/1713981828028133847?utm_source=thinkingelixir&utm_medium=shownotes) – Real world test results demonstrating faster speeds due to LiveView's optimized DOM patching. - https://dashbit.co/blog/latency-rendering-liveview (https://dashbit.co/blog/latency-rendering-liveview?utm_source=thinkingelixir&utm_medium=shownotes) – New post on performance optimization in Phoenix LiveView by José Valim. - https://youtu.be/Ckgl9KO4E4M?si=UNf5sNShzl1oTZQS&t=1731 (https://youtu.be/Ckgl9KO4E4M?si=UNf5sNShzl1oTZQS&t=1731?utm_source=thinkingelixir&utm_medium=shownotes) – Chris McCord's ElixirConf keynote demo on LiveView's new dev tool features. Time signature for dev tools example - https://twitter.com/josevalim/status/1711756969814426066 (https://twitter.com/josevalim/status/1711756969814426066?utm_source=thinkingelixir&utm_medium=shownotes) – José Valim's explanation on LiveView's unreleased feature. - https://github.com/elixir-saas/clicktocomponent (https://github.com/elixir-saas/click_to_component?utm_source=thinkingelixir&utm_medium=shownotes) – An external library called "clicktocomponent" to support the Cmd+Click functionality in LiveView. - https://news.livebook.dev/remote-execution-smart-cell---launch-week-2---day-1-m3dv2 (https://news.livebook.dev/remote-execution-smart-cell---launch-week-2---day-1-m3dv2?utm_source=thinkingelixir&utm_medium=shownotes) – Announcement of Livebook's new feature - Remote execution Smart cell. - https://news.livebook.dev/speech-to-text-with-whisper-timestamping-streaming-and-parallelism-oh-my---launch-week-2---day-2-36osSY (https://news.livebook.dev/speech-to-text-with-whisper-timestamping-streaming-and-parallelism-oh-my---launch-week-2---day-2-36osSY?utm_source=thinkingelixir&utm_medium=shownotes) – Livebook's new features in their Whisper integration for improved speech-to-text performance. - https://news.livebook.dev/introducing-file-integration---launch-week-2---day-3-2HoFfa (https://news.livebook.dev/introducing-file-integration---launch-week-2---day-3-2HoFfa?utm_source=thinkingelixir&utm_medium=shownotes) – Livebook's introduction to File Integration. - https://news.livebook.dev/integration-with-snowflake-and-microsoft-sql-server---launch-week-2---day-4-2o4z9C (https://news.livebook.dev/integration-with-snowflake-and-microsoft-sql-server---launch-week-2---day-4-2o4z9C?utm_source=thinkingelixir&utm_medium=shownotes) – Livebook adds support for Snowflake and SQL Server. - https://news.livebook.dev/vim-and-emacs-key-bindings---launch-week-2---day- (https://news.livebook.dev/vim-and-emacs-key-bindings---launch-week-2---day-?utm_source=thinkingelixir&utm_medium=shownotes) – Livebook support for VIM & Emacs key bindings. - https://hexdocs.pm/ash_sqlite/get-started-with-sqlite.html (https://hexdocs.pm/ash_sqlite/get-started-with-sqlite.html?utm_source=thinkingelixir&utm_medium=shownotes) – Ash Framework's new AshSqlite library release. - https://oban.pro/releases/web/v2.10 (https://oban.pro/releases/web/v2.10?utm_source=thinkingelixir&utm_medium=shownotes) – Release of Oban Web 2.10.0. - https://github.com/emmanueltouzery/elixir-extras.nvim (https://github.com/emmanueltouzery/elixir-extras.nvim?utm_source=thinkingelixir&utm_medium=shownotes) – Release of a new Neovim Elixir plugin called elixir-extras.nvim. - https://twitter.com/samokhvalov/status/1714153676212949355 (https://twitter.com/samokhvalov/status/1714153676212949355?utm_source=thinkingelixir&utm_medium=shownotes) – PostgreSQL tip for setting a human-readable label for a connection session. - https://github.com/pgbouncer/pgbouncer/releases/tag/pgbouncer121_0 (https://github.com/pgbouncer/pgbouncer/releases/tag/pgbouncer_1_21_0?utm_source=thinkingelixir&utm_medium=shownotes) – Upcoming PgBouncer 1.21 release with support for prepared statements. - https://twitter.com/ElixirConfEU/status/1713929804062273663 (https://twitter.com/ElixirConfEU/status/1713929804062273663?utm_source=thinkingelixir&utm_medium=shownotes) – Announcement for call for talks for ElixirConf Europe 2024. - https://www.elixirconf.eu/ (https://www.elixirconf.eu/?utm_source=thinkingelixir&utm_medium=shownotes) – ElixirConf Europe 2024 conference details. Do you have some Elixir news to share? Tell us at @ThinkingElixir (https://twitter.com/ThinkingElixir) or email at show@thinkingelixir.com (mailto:show@thinkingelixir.com) Discussion Resources - https://github.com/BeaconCMS/beacon (https://github.com/BeaconCMS/beacon?utm_source=thinkingelixir&utm_medium=shownotes) – The main Beacon CMS project - https://github.com/BeaconCMS/beacon_demo (https://github.com/BeaconCMS/beacon_demo?utm_source=thinkingelixir&utm_medium=shownotes) – Beacon CMS demo project - https://github.com/BeaconCMS/beaconliveadmin (https://github.com/BeaconCMS/beacon_live_admin?utm_source=thinkingelixir&utm_medium=shownotes) – Beacon CMS admin project - https://beaconcms.org/ (https://beaconcms.org/?utm_source=thinkingelixir&utm_medium=shownotes) - https://www.youtube.com/watch?v=5jk0fIJOFuc&list=PLqj39LCvnOWbHaZldxw_g02RaTQ4vQ1eY&index=16 (https://www.youtube.com/watch?v=5jk0fIJOFuc&list=PLqj39LCvnOWbHaZldxw_g02RaTQ4vQ1eY&index=16?utm_source=thinkingelixir&utm_medium=shownotes) – ElixirConf 2023 - Beacon - The next generation of CMS in Phoenix LiveView - https://mdxjs.com/ (https://mdxjs.com/?utm_source=thinkingelixir&utm_medium=shownotes) - https://hex.pm/packages/earmark (https://hex.pm/packages/earmark?utm_source=thinkingelixir&utm_medium=shownotes) - https://github.com/leandrocp/mdex (https://github.com/leandrocp/mdex?utm_source=thinkingelixir&utm_medium=shownotes) - https://github.com/BeaconCMS/livemonacoeditor (https://github.com/BeaconCMS/live_monaco_editor?utm_source=thinkingelixir&utm_medium=shownotes) – The code editor, based on the Livebook editor - https://github.com/leandrocp/mdex (https://github.com/leandrocp/mdex?utm_source=thinkingelixir&utm_medium=shownotes) – Markdown parser and formatter - https://github.com/leandrocp/autumn (https://github.com/leandrocp/autumn?utm_source=thinkingelixir&utm_medium=shownotes) – Syntax highlighter for code blocks - https://github.com/TheFirstAvenger/safe_code (https://github.com/TheFirstAvenger/safe_code?utm_source=thinkingelixir&utm_medium=shownotes) – Inspect HEEx for unsafe code - https://github.com/BeaconCMS/beacon/milestone/1 (https://github.com/BeaconCMS/beacon/milestone/1?utm_source=thinkingelixir&utm_medium=shownotes) – Beacon v0.1 release milestone - https://github.com/BeaconCMS/beaconliveadmin/milestone/1 (https://github.com/BeaconCMS/beacon_live_admin/milestone/1?utm_source=thinkingelixir&utm_medium=shownotes) Guest Information - https://twitter.com/leandrocesquini (https://twitter.com/leandrocesquini?utm_source=thinkingelixir&utm_medium=shownotes) – on Twitter - https://github.com/leandrocp/ (https://github.com/leandrocp/?utm_source=thinkingelixir&utm_medium=shownotes) – on Github - hhttps://leandrocp.com.br (hhttps://leandrocp.com.br?utm_source=thinkingelixir&utm_medium=shownotes) – Blog - https://github.com/leandrocp/mdex (https://github.com/leandrocp/mdex?utm_source=thinkingelixir&utm_medium=shownotes) – Other project MDEx - A fast 100% CommonMark-compatible GitHub Flavored Markdown parser and formatter for Elixir. Find us online - Message the show - @ThinkingElixir (https://twitter.com/ThinkingElixir) - Message the show on Fediverse - @ThinkingElixir@genserver.social (https://genserver.social/ThinkingElixir) - Email the show - show@thinkingelixir.com (mailto:show@thinkingelixir.com) - Mark Ericksen - @brainlid (https://twitter.com/brainlid) - Mark Ericksen on Fediverse - @brainlid@genserver.social (https://genserver.social/brainlid) - David Bernheisel - @bernheisel (https://twitter.com/bernheisel) - David Bernheisel on Fediverse - @dbern@genserver.social (https://genserver.social/dbern) - Cade Ward - @cadebward (https://twitter.com/cadebward) - Cade Ward on Fediverse - @cadebward@genserver.social (https://genserver.social/cadebward)

Summary Building streaming applications has gotten substantially easier over the past several years. Despite this, it is still operationally challenging to deploy and maintain your own stream processing infrastructure. Decodable was built with a mission of eliminating all of the painful aspects of developing and deploying stream processing systems for engineering teams. In this episode Eric Sammer discusses why more companies are including real-time capabilities in their products and the ways that Decodable makes it faster and easier. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. You specify the customer traits, then Profiles runs the joins and computations for you to create complete customer profiles. Get all of the details and try the new product today at dataengineeringpodcast.com/rudderstack (https://www.dataengineeringpodcast.com/rudderstack) This episode is brought to you by Datafold – a testing automation platform for data engineers that finds data quality issues before the code and data are deployed to production. Datafold leverages data-diffing to compare production and development environments and column-level lineage to show you the exact impact of every code change on data, metrics, and BI tools, keeping your team productive and stakeholders happy. Datafold integrates with dbt, the modern data stack, and seamlessly plugs in your data CI for team-wide and automated testing. If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold (https://www.dataengineeringpodcast.com/datafold) You shouldn't have to throw away the database to build with fast-changing data. You should be able to keep the familiarity of SQL and the proven architecture of cloud warehouses, but swap the decades-old batch computation model for an efficient incremental engine to get complex queries that are always up-to-date. With Materialize, you can! It's the only true SQL streaming database built from the ground up to meet the needs of modern data products. Whether it's real-time dashboarding and analytics, personalization and segmentation or automation and alerting, Materialize gives you the ability to work with fresh, correct, and scalable results — all in a familiar SQL interface. Go to dataengineeringpodcast.com/materialize (https://www.dataengineeringpodcast.com/materialize) today to get 2 weeks free! As more people start using AI for projects, two things are clear: It's a rapidly advancing field, but it's tough to navigate. How can you get the best results for your use case? Instead of being subjected to a bunch of buzzword bingo, hear directly from pioneers in the developer and data science space on how they use graph tech to build AI-powered apps. . Attend the dev and ML talks at NODES 2023, a free online conference on October 26 featuring some of the brightest minds in tech. Check out the agenda and register today at Neo4j.com/NODES (https://Neo4j.com/NODES). Your host is Tobias Macey and today I'm interviewing Eric Sammer about starting your stream processing journey with Decodable Interview Introduction How did you get involved in the area of data management? Can you describe what Decodable is and the story behind it? What are the notable changes to the Decodable platform since we last spoke? (October 2021) What are the industry shifts that have influenced the product direction? What are the problems that customers are trying to solve when they come to Decodable? When you launched your focus was on SQL transformations of streaming data. What was the process for adding full Java support in addition to SQL? What are the developer experience challenges that are particular to working with streaming data? How have you worked to address that in the Decodable platform and interfaces? As you evolve the technical and product direction, what is your heuristic for balancing the unification of interfaces and system integration against the ability to swap different components or interfaces as new technologies are introduced? What are the most interesting, innovative, or unexpected ways that you have seen Decodable used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Decodable? When is Decodable the wrong choice? What do you have planned for the future of Decodable? Contact Info esammer (https://github.com/esammer) on GitHub LinkedIn (https://www.linkedin.com/in/esammer/) Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. Podcast.__init__ (https://www.pythonpodcast.com) covers the Python language, its community, and the innovative ways it is being used. The Machine Learning Podcast (https://www.themachinelearningpodcast.com) helps you go from idea to production with machine learning. Visit the site (https://www.dataengineeringpodcast.com) to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@dataengineeringpodcast.com (mailto:hosts@dataengineeringpodcast.com)) with your story. To help other people find the show please leave a review on Apple Podcasts (https://podcasts.apple.com/us/podcast/data-engineering-podcast/id1193040557) and tell your friends and co-workers Links Decodable (https://www.decodable.co/) Podcast Episode (https://www.dataengineeringpodcast.com/decodable-streaming-data-pipelines-sql-episode-233/) Flink (https://flink.apache.org/) Podcast Episode (https://www.dataengineeringpodcast.com/apache-flink-with-fabian-hueske-episode-57/) Debezium (https://debezium.io/) Podcast Episode (https://www.dataengineeringpodcast.com/debezium-change-data-capture-episode-114/) Kafka (https://kafka.apache.org/) Redpanda (https://redpanda.com/) Podcast Episode (https://www.dataengineeringpodcast.com/vectorized-red-panda-streaming-data-episode-152/) Kinesis (https://aws.amazon.com/kinesis/) PostgreSQL (https://www.postgresql.org/) Podcast Episode (https://www.dataengineeringpodcast.com/postgresql-with-jonathan-katz-episode-42/) Snowflake (https://www.snowflake.com/en/) Podcast Episode (https://www.dataengineeringpodcast.com/snowflakedb-cloud-data-warehouse-episode-110/) Databricks (https://www.databricks.com/) Startree (https://startree.ai/) Pinot (https://pinot.apache.org/) Podcast Episode (https://www.dataengineeringpodcast.com/pinot-embedded-analytics-episode-273/) Rockset (https://rockset.com/) Podcast Episode (https://www.dataengineeringpodcast.com/rockset-serverless-analytics-episode-101/) Druid (https://druid.apache.org/) InfluxDB (https://www.influxdata.com/) Samza (https://samza.apache.org/) Storm (https://storm.apache.org/) Pulsar (https://pulsar.apache.org/) Podcast Episode (https://www.dataengineeringpodcast.com/pulsar-fast-and-scalable-messaging-with-rajan-dhabalia-and-matteo-merli-episode-17) ksqlDB (https://ksqldb.io/) Podcast Episode (https://www.dataengineeringpodcast.com/ksqldb-kafka-stream-processing-episode-122/) dbt (https://www.getdbt.com/) GitHub Actions (https://github.com/features/actions) Airbyte (https://airbyte.com/) Singer (https://www.singer.io/) Splunk (https://www.splunk.com/) Outbox Pattern (https://debezium.io/blog/2019/02/19/reliable-microservices-data-exchange-with-the-outbox-pattern/) The intro and outro music is from The Hug (http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/Love_death_and_a_drunken_monkey/04_-_The_Hug) by The Freak Fandango Orchestra (http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/) / CC BY-SA (http://creativecommons.org/licenses/by-sa/3.0/)

Topics covered in this episode: Psycopg 3 dacite RIP: Fast, barebones pip implementation in Rust Flaky Tests follow up Extras Joke Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training The Complete pytest Course Patreon Supporters Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Brian #1: Psycopg 3 Psycopg folks recommend starting with 3 for new projects 2 is still actively maintained, but no new features are planned recommend staying with 2 for legacy projects Psycopg 3 project 2 vs 3 feature comparison A few Psycopg 3 highlights native asyncio support native support for more Python types (such as Enums) and PostgreSQL types (such as multirange) Default server-side parameters binding Allows binary parameters and query results (and text, of course) Pipeline/batch mode support Static typing support Michael #2: dacite via Raymond Peck Simple creation of data classes from dictionaries Dacite supports following features: nested structures (basic) types checking optional fields (i.e. typing.Optional) unions forward references collections custom type hooks It's important to mention that dacite is not a data validation library. Type hooks are interesting too. Brian #3: RIP: Fast, barebones pip implementation in Rust list of current and planned features of RIP, the biggest are listed below: Downloading and aggressive caching of PyPI metadata. (done) Resolving of PyPI packages using Resolvo. (done) Installation of wheel files (planned) Support sdist files (planned) new project, just a couple weeks old. … “We would love to have you contribute!” Michael #4: Flaky Tests follow up by Marwan Sarieddine I was inspired by the Talk Python podcast on "Taming flaky tests" with Gregory Kapfhammer and Owain Parry so I wrote up an article on my blog titled "How not to footgun yourself when writing tests - a showcase of flaky tests” Extras Brian: Just wrapping up some personal projects, which means… Python People episodes soon Python Test episodes soon (but later) More course chapters coming Michael: PyBay 2023 was fun Switched to Spark Mail, recommended Dust (what science fiction story telling should be), try: FTL Oceanus Joke: There are more hydrogen atoms in a single molecule of water than there are stars in the entire Solar System. - mas.to/@SmudgeTheInsultCat/111174610011921264 The Big Rewrite

In this episode, Gayatri Kalyanaraman is in conversation with Saikat Chakrabarty who is a Technologist, Director, Engineering at Mihup and he's a AI/ML and Computer Vision Enthusiast.In his career spanning over a decade, he has been a Telecom engineer, prompt engineering, Interaction designer and conversational AI. Saikat is working in the pioneering world of building AI tools that map the man machine and the personHe's working in the contact center engineering by understanding the questions and asks from our customersSaikat started his career in creating software for LTE within 4G Telecom providers and says that he has not looked back from thereSaikat's team in Mihup creates the analysis of the several thousand conversations between the customer and contact center and understand the sentiment of the userListen to the conversation on how the vernacular language support is being created in the AI platform to build accuracyUnderstand that initial days you have to choose the right place to work and always stay invested in oneselfSaikat also talks about his mantra of “Don't choose an easy path and continue to learn”Saikat talks about his interests in Medical and Computer science and Brain networks are working. Saikat believes that the future bet is lying on medical access across the globeSaikat says that software changes in months not years - you have to keep yourself updated daily and that's the only way to stay relevant He's also deep believer of the growth of the technology in Tier 3 cities within India given the organizations changing to remote first or HybridSaikat has been instrumental in Mihup's success as one of the leading Conversational AI companies in India. He leads the entire tech organisation at Mihup, with a team of almost 20 people working under him, ranging from Devops to MLEs, overseeing the end to end development and shipping of the Mihup products.Saikat Chakrabarty is a dynamic leader and Senior Engineering Manager at Mihup, a pioneering conversational AI platform founded in 2016. Mihup's revolutionary approach empowers individuals to seamlessly interact with the digital world while prioritizing inclusivity and accessibility, regardless of language, accent, or dialect.Technical Leadership:At Mihup, Saikat has successfully managed and mentored cross-functional teams in machine learning, R&D, product development and DevOps. His expertise in managing the development of Mihup's conversational AI products has positioned him as a trusted leader in the industry. With a deep understanding of a wide range of technologies, including Java (Spring Boot), NodeJs, Golang, Angular, Python, RabbitMQ, PostgreSQL, Redis, Kubernetes, and more, Saikat has played a pivotal role in building highly scalable cloud microservices and optimizing premise-based applications.Diverse Experience:Saikat's wealth of experience extends beyond his role at Mihup. He has previously worked with renowned companies like Lexmark and Polaris Network Inc., where he honed his skills and passion for AI/ML and computer vision. His contributions to these cutting-edge fields have resulted in 2 patents and 6 paper publications, consolidating his exceptional knowledge and expertise.Recognition and Awards:Saikat's exceptional contributions and dedication to his craft have earned him notable accolades throughout his career. Notably, he was awarded the prestigious Digital Transformation Award at Lexmark Focus-2-Future, alongside the Manager Appreciation Award and several other commendations. At Mihup, he has regularly been the recipient of special rewards for his undying passion and dedication towards delivering products that customers love to use.Saikat can be contacted at Saikat Chakrabarty:

News this week includes the release of ElixirConf 2023 video playlist featuring keynote videos, a guide on Web Application Security Best Practices for BEAM languages from the EEF Security Working Group, the release of NextLS v0.12 with exciting new features, and a new library to integrate the JavaScript all-in-one toolkit, Bun with Phoenix. A new LangChain library makes it easy to integrate Elixir applications with an LLM like ChatGPT. Regarding the open source community, we discuss the major update in the Lodash JS project, the latest PostgreSQL 16 release, and a noteworthy article from ZigLang on bounties potentially damaging open source projects, and more! Show Notes online - http://podcast.thinkingelixir.com/170 (http://podcast.thinkingelixir.com/170) Elixir Community News - https://www.youtube.com/playlist?list=PLqj39LCvnOWbHaZldxw_g02RaTQ4vQ1eY (https://www.youtube.com/playlist?list=PLqj39LCvnOWbHaZldxw_g02RaTQ4vQ1eY?utm_source=thinkingelixir&utm_medium=shownotes) – ElixirConf 2023 video playlist, currently only includes the keynote videos. - https://twitter.com/bernheisel/status/1704019930515919092 (https://twitter.com/bernheisel/status/1704019930515919092?utm_source=thinkingelixir&utm_medium=shownotes) – Chris McCord's keynote for ElixirConf 2023. - https://erlef.github.io/security-wg/webappsecuritybestpractices_beam/ (https://erlef.github.io/security-wg/web_app_security_best_practices_beam/?utm_source=thinkingelixir&utm_medium=shownotes) – Guide on Web Application Security Best Practices for BEAM languages - https://twitter.com/paraxialio/status/1703146204404535467 (https://twitter.com/paraxialio/status/1703146204404535467?utm_source=thinkingelixir&utm_medium=shownotes) – Update on the same topic on Twitter. - https://podcast.thinkingelixir.com/134 (https://podcast.thinkingelixir.com/134?utm_source=thinkingelixir&utm_medium=shownotes) – Podcast episode discussing inside the Security Working Group - https://podcast.thinkingelixir.com/93 (https://podcast.thinkingelixir.com/93?utm_source=thinkingelixir&utm_medium=shownotes) – Preventing Service Abuse with Michael Lubas podcast episode - https://github.com/elixir-tools/next-ls (https://github.com/elixir-tools/next-ls?utm_source=thinkingelixir&utm_medium=shownotes) – NextLS v0.12 released with new features. - https://www.elixir-tools.dev/news/the-elixir-tools-update-vol-3/ (https://www.elixir-tools.dev/news/the-elixir-tools-update-vol-3/?utm_source=thinkingelixir&utm_medium=shownotes) – More information on the Elixir Tools update. - https://twitter.com/crbelaus/status/1702703595236331668 (https://twitter.com/crbelaus/status/1702703595236331668?utm_source=thinkingelixir&utm_medium=shownotes) – Phoenix's new library to integrate Bun, the JavaScript all-in-one toolkit. - https://hex.pm/packages/elixir_bun (https://hex.pm/packages/elixir_bun?utm_source=thinkingelixir&utm_medium=shownotes) – Link to the new Phoenix library on Hex. - https://podcast.thinkingelixir.com/83 (https://podcast.thinkingelixir.com/83?utm_source=thinkingelixir&utm_medium=shownotes) – More info about Zig with a previous guest Isaac Yonemoto. - https://github.com/brainlid/langchain (https://github.com/brainlid/langchain?utm_source=thinkingelixir&utm_medium=shownotes) – LangChain for Elixir library was released. - https://twitter.com/RudManusachi/status/1702093261530403223 (https://twitter.com/RudManusachi/status/1702093261530403223?utm_source=thinkingelixir&utm_medium=shownotes) – Brian Cardarella's LiveViewNative keynote showed the ExDoc documentation. - https://hexdocs.pm/ex_doc/readme.html#tabsets (https://hexdocs.pm/ex_doc/readme.html#tabsets?utm_source=thinkingelixir&utm_medium=shownotes) – Explanation on the use of tabsets in ExDocs. - https://twitter.com/danielcroe/status/1703127430523703432 (https://twitter.com/danielcroe/status/1703127430523703432?utm_source=thinkingelixir&utm_medium=shownotes) – Lodash JS project declared "Issue bankruptcy" - https://github.com/lodash/lodash (https://github.com/lodash/lodash?utm_source=thinkingelixir&utm_medium=shownotes) – Lodash, a modern JavaScript utility library, went through some changes. - https://ziglang.org/news/bounties-damage-open-source-projects/ (https://ziglang.org/news/bounties-damage-open-source-projects/?utm_source=thinkingelixir&utm_medium=shownotes) – ZigLang blog stating that "Bounties Damage Open Source Projects" - https://www.postgresql.org/about/news/postgresql-16-released-2715/ (https://www.postgresql.org/about/news/postgresql-16-released-2715/?utm_source=thinkingelixir&utm_medium=shownotes) – PostgreSQL 16 released with performance improvements and new features. - https://www.crunchydata.com/blog/active-active-postgres-16 (https://www.crunchydata.com/blog/active-active-postgres-16?utm_source=thinkingelixir&utm_medium=shownotes) – Understanding of "Active-Active" replication in PostgreSQL 16. Do you have some Elixir news to share? Tell us at @ThinkingElixir (https://twitter.com/ThinkingElixir) or email at show@thinkingelixir.com (mailto:show@thinkingelixir.com) Find us online - Message the show - @ThinkingElixir (https://twitter.com/ThinkingElixir) - Message the show on Fediverse - @ThinkingElixir@genserver.social (https://genserver.social/ThinkingElixir) - Email the show - show@thinkingelixir.com (mailto:show@thinkingelixir.com) - Mark Ericksen - @brainlid (https://twitter.com/brainlid) - Mark Ericksen on Fediverse - @brainlid@genserver.social (https://genserver.social/brainlid) - David Bernheisel - @bernheisel (https://twitter.com/bernheisel) - David Bernheisel on Fediverse - @dbern@genserver.social (https://genserver.social/dbern) - Cade Ward - @cadebward (https://twitter.com/cadebward) - Cade Ward on Fediverse - @cadebward@genserver.social (https://genserver.social/cadebward)

This week, we discuss why everyone is envious of Google's Internal Dev Tools, examine the state of Git, speculate about how 37 Signals plans to reinvent software licensing with ONCE, and share a few thoughts on the Salesforce CEO's recent comments about work from home. Watch the YouTube Live Recording of Episode (https://www.youtube.com/watch?v=AaX-PgF86bY) 433 (https://www.youtube.com/watch?v=AaX-PgF86bY) Runner-up Titles Lost in an acquisition hole. Headless Robot Dog. It's not better enough. GoogHub Why are you on the sad path Once version 2 is a paid upgrade You win interesting bingo Rundown The Full Circle on Developer Productivity with Steve Yegge (https://newsletter.pragmaticengineer.com/p/steve-yegge) Git is awful. GitHub isn't good enough. It's killing us! (Steve Yegge) (https://www.youtube.com/watch?v=EReooAZoMO0) Introducing ONCE (https://once.com/) Salesforce CEO takes a bold stand on remote work (https://www.thestreet.com/investing/salesforce-ceo-bold-stand-on-remote-work) Salesforce to Hire 3,300 People After Layoffs Earlier This Year (https://www.bloomberg.com/news/articles/2023-09-14/salesforce-to-hire-3-300-in-sales-engineering-data-after-earlier-job-cuts#xj4y7vzkg) Relevant to your Interests David Sacks has a new SaaS startup for other SaaS startups (https://www.axios.com/2023/09/06/david-sacks-has-a-new-saas-startup-for-other-saas-startups) Results of Major Technical Investigations for Storm-0558 Key Acquisition (https://msrc.microsoft.com/blog/2023/09/results-of-major-technical-investigations-for-storm-0558-key-acquisition/) Now it's PostgreSQL's turn to have a bogus CVE (https://opensourcewatch.beehiiv.com/p/now-postgresqls-turn-bogus-cve) HashiCorp Retools Licenses And Software To Grow Its Business - The Next Platform (https://www.nextplatform.com/2023/09/05/hashicorp-retools-licenses-and-software-to-grow-its-business/) Clouded Judgement 9.8.23 (https://cloudedjudgement.substack.com/p/clouded-judgement-9823?utm_source=post-email-title&publication_id=56878&post_id=136822157&isFreemail=true&r=2l9&utm_medium=email) Inside Hollywood's SBF Mad Scramble (https://theankler.com/p/inside-hollywoods-sbf-mad-scramble-c04?utm_source=newsletter&utm_medium=email&utm_campaign=newsletter_axiosprorata&stream=top) Tubi The Free Streaming Service, Hits 74 Million Monthly Active Users & Almost 250 Free Live Channels As Cord Cutting Grows | Cord Cutters News (https://cordcuttersnews.com/tubi-the-free-streaming-service-hits-74-million-monthly-active-users-almost-250-free-live-channels-as-cord-cutting-grows/) IBM Software mandates return to office for those within 80km (https://www.theregister.com/2023/09/11/ibm_software_tells_workers_to/) Cloud is here to stay, but at what cost, ask customers (https://www.theregister.com/2023/09/11/cloud_costs_feature/) Disney and Charter reach deal to end cable blackout in time for 'Monday Night Football' (https://www.cnbc.com/2023/09/11/disney-charter-near-carriage-deal-that-would-end-cable-blackout-sources-say.html) Microsoft to kill off third-party printer drivers in Windows (https://www.theregister.com/2023/09/11/go_native_or_go_home/) Oracle revenue misses estimates as tough economy hurts cloud spending (https://www.reuters.com/technology/oracle-reports-quarterly-revenue-narrowly-below-estimates-2023-09-11/) No privacy in cars (https://foundation.mozilla.org/en/privacynotincluded/articles/its-official-cars-are-the-worst-product-category-we-have-ever-reviewed-for-privacy/) Former CEO of China's Alibaba quits cloud business in surprise move during its leadership reshuffle (https://abcnews.go.com/Business/wireStory/former-ceo-chinas-alibaba-quits-cloud-business-surprise-103078368) A Look Back at Q2 '23 Public Cloud Software Earnings (https://cloudedjudgement.substack.com/p/a-look-back-at-q2-23-public-cloud?utm_source=post-email-title&publication_id=56878&post_id=136950716&utm_campaign=email-post-title&isFreemail=true&r=2l9&utm_medium=email) 1 big thing: A long-term plan to secure open-source software (https://www.axios.com/newsletters/axios-codebook-8200e5c5-aed7-4f42-a40e-117a390b57e3.html?chunk=0&utm_term=emshare#story0) MGM takes systems offline after cyberattack (https://www.axios.com/newsletters/axios-codebook-8200e5c5-aed7-4f42-a40e-117a390b57e3.html?chunk=1&utm_term=emshare#story1) Disney-Charter deal represents new era for TV bundles (https://www.axios.com/newsletters/axios-media-trends-fe1295c8-9b83-4403-bae2-06de14fede11.html?chunk=2&utm_term=emshare#story2) Salesforce introduces Einstein Copilot Studio to help customers customize their AI | TechCrunch (https://techcrunch.com/2023/09/12/salesforce-introduces-einstein-copilot-studio-to-customers-customize-their-ai/) Arm prices IPO at $51 per share, valuing company at over $54 billion (https://www.cnbc.com/2023/09/13/arm-prices-ipo-at-51-per-share.html) Tim Gurner's spray about ‘arrogant' workers lays bare the economic sadism of our time (https://www.theguardian.com/commentisfree/2023/sep/14/tim-gurner-ceo-comments-more-unemployment-millionaire-property-developer-workers-neoliberals) Cisco discontinues Hyperflex hyperconverged infrastructure (https://www.theregister.com/2023/09/14/cisco_discontinues_hyperflex_hci/) CloudBees Announces New Cloud Native DevSecOps Platform (https://www.cloudbees.com/newsroom/cloudbees-announces-new-cloud-native-devsecops-platform) Jet: Prepare For Liftoff (https://www.jetporch.com/) Artifact's new Links feature makes it much more than a news app (https://www.theverge.com/2023/9/13/23871561/artifact-links-news-reading-app-tiktok) TriggerMesh, RIP (https://triggermesh-community.slack.com/archives/C02GHUAQDCH/p1695048539668859) Clorox says last month's cyberattack is still disrupting production (https://www.cnbc.com/2023/09/18/clorox-says-last-months-cyberattack-is-still-disrupting-production.html) Excel clone built for Uber China exposed Microsoft mistake (https://www.theregister.com/2023/09/19/matt_uber_china_excel_clone/) Seattle startup MotherDuck raises $52.5M at a $400M valuation to fuel DuckDB analytics platform (https://www.geekwire.com/2023/seattle-startup-motherduck-raises-52-5m-at-a-400m-valuation-to-fuel-duckdb-analytics-platform/) Google's Bard chatbot can now find answers in your Gmail, Docs, Drive (https://www.theverge.com/2023/9/19/23878999/google-bard-ai-chatbot-gmail-docs-drive-extensions) Elon Musk says X may go behind a paywall for everyone so he can 'combat vast armies of bots' (https://www.businessinsider.com/elon-musk-x-twitter-paywall-for-everyone-2023-9) Restricted Source Licensing Is Here (https://www.forrester.com/blogs/restricted-source-licensing-is-here/) OpenTofu (https://opentofu.org/) RoboFab is ready to build 10,000 humanoid robots per year | TechCrunch (https://techcrunch.com/2023/09/18/the-robots-are-coming/) Unified Acceleration Foundation Forms to Drive Open Accelerated Compute and Cross-Platform Performance (https://www.linuxfoundation.org/press/announcing-unified-acceleration-foundation-uxl) Google gets its way, bakes a user-tracking ad platform directly into Chrome (https://arstechnica.com/gadgets/2023/09/googles-widely-opposed-ad-platform-the-privacy-sandbox-launches-in-chrome/) What is a service mesh? Why do you need a service mesh? And which is the best service mesh? (https://newsletter.cote.io/p/what-is-a-service-mesh-why-do-you) Did I Make a Mistake Selling My Social-Media Darling to Yahoo? (https://nymag.com/intelligencer/2018/10/did-i-make-a-mistake-selling-del-icio-us-to-yahoo.html?utm_source=substack&utm_medium=email) A new way of thinking about open source sustainability (https://www.infoworld.com/article/3706508/a-new-way-of-thinking-about-open-source-sustainability.html) Elon Musk moving servers himself shows his 'maniacal sense of urgency' at X, formerly Twitter (https://www.cnbc.com/2023/09/11/elon-musk-moved-twitter-servers-himself-in-the-night-new-biography-details-his-maniacal-sense-of-urgency.html) Cable TV Is on Life Support, but a New Bundle Is Coming Alive (https://www.nytimes.com/2023/09/14/business/media/cable-tv-bundle-streaming.html) Nonsense McDonald's is getting rid of self-serve soda machines | CNN Business (https://www.cnn.com/2023/09/12/business/mcdonalds-self-serve-soda-machines/index.html) Delta SkyMiles changes: Delta overhauls how you earn Medallion status in biggest change yet (https://thepointsguy.com/news/delta-skymiles-changes/) Australian baby named Methamphetamine Rules (https://www.1news.co.nz/2023/09/20/australian-baby-named-methamphetamine-rules/) ‘Take the Money and Run' Artist Must Repay Danish Museum (https://www.nytimes.com/2023/09/19/arts/design/jens-haaning-take-the-money-and-run.html?smid=nytcore-ios-share&referringSource=articleShare) Listener Feedback Jan recommends this Rich Roll interview: Mindset SECRETS From The World's Best Ultrarunner: Courtney Dauwalter (https://www.youtube.com/watch?v=WOtSvYSnzNk) Conferences October 6, 2023, KCD Texas 2023 (https://community.cncf.io/events/details/cncf-kcd-texas-presents-kcd-texas-2023/), CFP Closes: August 30, 2023 November 6-9, 2023, KubeCon NA (https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/), SDT's a sponsor, Matt's there November 6-9, 2023 VMware Explore Barcelona (https://www.vmware.com/explore/eu.html), Coté's attending Jan 29, 2024 to Feb 1, 2024 That Conference Texas (https://that.us/events/tx/2024/schedule/) If you want your conference mentioned, let's talk media sponsorships. SDT news & hype Join us in Slack (http://www.softwaredefinedtalk.com/slack). Get a SDT Sticker! Send your postal address to stickers@softwaredefinedtalk.com (mailto:stickers@softwaredefinedtalk.com) and we will send you free laptop stickers! Follow us: Twitch (https://www.twitch.tv/sdtpodcast), Twitter (https://twitter.com/softwaredeftalk), Instagram (https://www.instagram.com/softwaredefinedtalk/), Mastodon (https://hachyderm.io/@softwaredefinedtalk), BlueSky (https://bsky.app/profile/softwaredefinedtalk.com), LinkedIn (https://www.linkedin.com/company/software-defined-talk/), TikTok (https://www.tiktok.com/@softwaredefinedtalk), Threads (https://www.threads.net/@softwaredefinedtalk) and YouTube (https://www.youtube.com/channel/UCi3OJPV6h9tp-hbsGBLGsDQ/featured). Use the code SDT to get $20 off Coté's book, Digital WTF (https://leanpub.com/digitalwtf/c/sdt), so $5 total. Become a sponsor of Software Defined Talk (https://www.softwaredefinedtalk.com/ads)! Recommendations Brandon: YouTube TV (https://tv.youtube.com/welcome/) and NFL Sunday Ticket (https://tv.youtube.com/learn/nflsundayticket/) An Endgame for YouTube TV, Big Disney Decisions (And Whether Bob Iger Should Make Them), The Era Beyond Peak TV (https://sharptech.fm/member/episode/an-endgame-for-you-tube-tv-big-disney-decisions-and-whether-bob-iger-should-make-them-the-era-beyond-peak-tv) Matt: Airline wifi chat with Support Coté: Do Interesting (https://thedobook.co/products/do-interesting-notice-collect-share) book by Russel Davis. Photo Credits Header (https://unsplash.com/photos/m-Yot4dUd6s) Artwork (https://unsplash.com/photos/I7iJOE4fsYo)

* Modern Data Stack Meetup : Streamlit + Kestra### Gen AI* Deploy Your LLM Chatbot With Retrieval Augmented Generation (RAG), llama2-70B (MosaicML inferences) and Vector Search -> https://www.databricks.com/resources/demos/tutorials/data-science-and-ai/lakehouse-ai-deploy-your-llm-chatbot?itm_data=demo_center* LLMs in Action: A Practical Guide for Software Architects and Developers -> https://www.linkedin.com/pulse/llms-action-practical-guide-software-architects-s%C3%A9bastien-brasseur?utm_source=share&utm_medium=member_ios&utm_campaign=share_via* Lantern — a PostgreSQL vector database for building AI applications -> https://docs.lantern.dev/blog/2023/09/13/hello-world### Data science* Announcing Python in Excel: Combining the power of Python and the flexibility of Excel. -> https://techcommunity.microsoft.com/t5/excel-blog/announcing-python-in-excel-combining-the-power-of-python-and-the/ba-p/3893439* NEW Python in Excel - PYTHON + EXCEL + ChatGPT = Easy! -> https://youtu.be/-_1IaUjO-hk?si=ZA8-ztkfzQfA3cc0* Excel Just Got Python | Prime Reacts -> https://youtu.be/iAQJhYQEx-s?si=l_v3YC8o01Vyyar6### Cloud Native* Kubernetes OWASP Top 10: Secrets Management -> https://itnext.io/kubernetes-owasp-top-10-secrets-management-c996faa87b47Agenda* Bigdatapero à Paris 27/09/2023Cette publication est sponsorisée par Affini-Tech et CerenIT.CerenIT vous accompagne pour concevoir, industrialiser ou automatiser vos plateformes mais aussi pour faire parler vos données temporelles. Ecrivez nous à contact@cerenit.fr et retrouvez-nous aussi au Time Series France.Affini-Tech vous accompagne dans tous vos projets Cloud et Data, pour Imaginer, Expérimenter et Executer vos services ! (Affini-Tech, Datatask) Consulter le blog d'Affini-Tech et le blog de Datatask pour en savoir plus. On recrute ! Venez cruncher de la data avec nous ! Ecrivez nous à recrutement@affini-tech.comLe générique a été composé et réalisé par Maxence Lecointe

Tony Baer, Principal at dbInsight, joins Corey on Screaming in the Cloud to discuss his definition of what is and isn't a database, and the trends he's seeing in the industry. Tony explains why it's important to try and have an outsider's perspective when evaluating new ideas, and the growing awareness of the impact data has on our daily lives. Corey and Tony discuss the importance of working towards true operational simplicity in the cloud, and Tony also shares why explainability in generative AI is so crucial as the technology advances. About TonyTony Baer, the founder and CEO of dbInsight, is a recognized industry expert in extending data management practices, governance, and advanced analytics to address the desire of enterprises to generate meaningful value from data-driven transformation. His combined expertise in both legacy database technologies and emerging cloud and analytics technologies shapes how clients go to market in an industry undergoing significant transformation. During his 10 years as a principal analyst at Ovum, he established successful research practices in the firm's fastest growing categories, including big data, cloud data management, and product lifecycle management. He advised Ovum clients regarding product roadmap, positioning, and messaging and helped them understand how to evolve data management and analytic strategies as the cloud, big data, and AI moved the goal posts. Baer was one of Ovum's most heavily-billed analysts and provided strategic counsel to enterprises spanning the Fortune 100 to fast-growing privately held companies.With the cloud transforming the competitive landscape for database and analytics providers, Baer led deep dive research on the data platform portfolios of AWS, Microsoft Azure, and Google Cloud, and on how cloud transformation changed the roadmaps for incumbents such as Oracle, IBM, SAP, and Teradata. While at Ovum, he originated the term “Fast Data” which has since become synonymous with real-time streaming analytics.Baer's thought leadership and broad market influence in big data and analytics has been formally recognized on numerous occasions. Analytics Insight named him one of the 2019 Top 100 Artificial Intelligence and Big Data Influencers. Previous citations include Onalytica, which named Baer as one of the world's Top 20 thought leaders and influencers on Data Science; Analytics Week, which named him as one of 200 top thought leaders in Big Data and Analytics; and by KDnuggets, which listed Baer as one of the Top 12 top data analytics thought leaders on Twitter. While at Ovum, Baer was Ovum's IT's most visible and publicly quoted analyst, and was cited by Ovum's parent company Informa as Brand Ambassador in 2017. In raw numbers, Baer has 14,000 followers on Twitter, and his ZDnet “Big on Data” posts are read 20,000 – 30,000 times monthly. He is also a frequent speaker at industry conferences such as Strata Data and Spark Summit.Links Referenced:dbInsight: https://dbinsight.io/ TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is brought to us in part by our friends at RedHat.As your organization grows, so does the complexity of your IT resources. You need a flexible solution that lets you deploy, manage, and scale workloads throughout your entire ecosystem. The Red Hat Ansible Automation Platform simplifies the management of applications and services across your hybrid infrastructure with one platform. Look for it on the AWS Marketplace.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Back in my early formative years, I was an SRE sysadmin type, and one of the areas I always avoided was databases, or frankly, anything stateful because I am clumsy and unlucky and that's a bad combination to bring within spitting distance of anything that, you know, can't be spun back up intact, like databases. So, as a result, I tend not to spend a lot of time historically living in that world. It's time to expand horizons and think about this a little bit differently. My guest today is Tony Baer, principal at dbInsight. Tony, thank you for joining me.Tony: Oh, Corey, thanks for having me. And by the way, we'll try and basically knock down your primal fear of databases today. That's my mission.Corey: We're going to instill new fears in you. Because I was looking through a lot of your work over the years, and the criticism I have—and always the best place to deliver criticism is massively in public—is that you take a very conservative, stodgy approach to defining a database, whereas I'm on the opposite side of the world. I contain information. You can ask me about it, which we'll call querying. That's right. I'm a database.But I've never yet found myself listed in any of your analyses around various database options. So, what is your definition of databases these days? Where do they start and stop? Tony: Oh, gosh.Corey: Because anything can be a database if you hold it wrong.Tony: [laugh]. I think one of the last things I've ever been called as conservative and stodgy, so this is certainly a way to basically put the thumbtack on my share.Corey: Exactly. I'm trying to normalize my own brand of lunacy, so we'll see how it goes.Tony: Exactly because that's the role I normally play with my clients. So, now the shoe is on the other foot. What I view a database is, is basically a managed collection of data, and it's managed to the point where essentially, a database should be transactional—in other words, when I basically put some data in, I should have some positive information, I should hopefully, depending on the type of database, have some sort of guidelines or schema or model for how I structure the data. So, I mean, database, you know, even though you keep hearing about unstructured data, the fact is—Corey: Schemaless databases and data stores. Yeah, it was all the rage for a few years.Tony: Yeah, except that they all have schemas, just that those schemaless databases just have very variable schema. They're still schema.Corey: A question that I have is you obviously think deeply about these things, which should not come as a surprise to anyone. It's like, “Well, this is where I spend my entire career. Imagine that. I might think about the problem space a little bit.” But you have, to my understanding, never worked with databases in anger yourself. You don't have a history as a DBA or as an engineer—Tony: No.Corey: —but what I find very odd is that unlike a whole bunch of other analysts that I'm not going to name, but people know who I'm talking about regardless, you bring actual insights into this that I find useful and compelling, instead of reverting to the mean of well, I don't actually understand how any of these things work in reality, so I'm just going to believe whoever sounds the most confident when I ask a bunch of people about these things. Are you just asking the right people who also happen to sound confident? But how do you get away from that very common analyst trap?Tony: Well, a couple of things. One is I purposely play the role of outside observer. In other words, like, the idea is that if basically an idea is supposed to stand on its own legs, it has to make sense. If I've been working inside the industry, I might take too many things for granted. And a good example of this goes back, actually, to my early days—actually this goes back to my freshman year in college where I was taking an organic chem course for non-majors, and it was taught as a logic course not as a memorization course.And we were given the option at the end of the term to either, basically, take a final or  do a paper. So, of course, me being a writer I thought, I can BS my way through this. But what I found—and this is what fascinated me—is that as long as certain technical terms were defined for me, I found a logic to the way things work. And so, that really informs how I approach databases, how I approach technology today is I look at the logic  on how things work. That being said, in order for me to understand that, I need to know twice as much as the next guy in order to be able to speak that because I just don't do this in my sleep.Corey: That goes a big step toward, I guess, addressing a lot of these things, but it also feels like—and maybe this is just me paying closer attention—that the world of databases and data and analytics have really coalesced or emerged in a very different way over the past decade-ish. It used to be, at least from my perspective, that oh, that the actual, all the data we store, that's a storage admin problem. And that was about managing NetApps and SANs and the rest. And then you had the database side of it, which functionally from the storage side of the world was just a big file or series of files that are the backing store for the database. And okay, there's not a lot of cross-communication going on there.Then with the rise of object store, it started being a little bit different. And even the way that everyone is talking about getting meaning from data has really seem to be evolving at an incredibly intense clip lately. Is that an accurate perception, or have I just been asleep at the wheel for a while and finally woke up?Tony: No, I think you're onto something there. And the reason is that, one, data is touching us all around ourselves, and the fact is, I mean, I'm you can see it in the same way that all of a sudden that people know how to spell AI. They may not know what it means, but the thing is, there is an awareness the data that we work with, the data that is about us, it follows us, and with the cloud, this data has—well, I should say not just with the cloud but with smart mobile devices—we'll blame that—we are all each founts of data, and rich founts of data. And people in all walks of life, not just in the industry, are now becoming aware of it and there's a lot of concern about can we have any control, any ownership over the data that should be ours? So, I think that phenomenon has also happened in the enterprise, where essentially where we used to think that the data was the DBAs' issue, it's become the app developers' issue, it's become the business analysts' issue. Because the answers that we get, we're ultimately accountable for. It all comes from the data.Corey: It also feels like there's this idea of databases themselves becoming more contextually aware of the data contained within them. Originally, this used to be in the realm of, “Oh, we know what's been accessed recently and we can tier out where it lives for storage optimization purposes.” Okay, great, but what I'm seeing now almost seems to be a sense of, people like to talk about pouring ML into their database offerings. And I'm not able to tell whether that is something that adds actual value, or if it's marketing-ware.Tony: Okay. First off, let me kind of spill a couple of things. First of all, it's not a question of the database becoming aware. A database is not sentient.Corey: Niether are some engineers, but that's neither here nor there.Tony: That would be true, but then again, I don't want anyone with shotguns lining up at my door after this—Corey: [laugh].Tony: —after this interview is published. But [laugh] more of the point, though, is that I can see a couple roles for machine learning in databases. One is a database itself, the logs, are an incredible font of data, of operational data. And you can look at trends in terms of when this—when the pattern of these logs goes this way, that is likely to happen. So, the thing is that I could very easily say we're already seeing it: machine learning being used to help optimize the operation of databases, if you're Oracle, and say, “Hey, we can have a database that runs itself.”The other side of the coin is being able to run your own machine-learning models in database as opposed to having to go out into a separate cluster and move the data, and that's becoming more and more of a checkbox feature. However, that's going to be for essentially, probably, like, the low-hanging fruit, like the 80/20 rule. It'll be like the 20% of an ana—of relatively rudimentary, you know, let's say, predictive analyses that we can do inside the database. If you're going to be doing something more ambitious, such as a, you know, a large language model, you probably do not want to run that in database itself. So, there's a difference there.Corey: One would hope. I mean, one of the inappropriate uses of technology that I go for all the time is finding ways to—as directed or otherwise—in off-label uses find ways of tricking different services into running containers for me. It's kind of a problem; this is probably why everyone is very grateful I no longer write production code for anyone.But it does seem that there's been an awful lot of noise lately. I'm lazy. I take shortcuts very often, and one of those is that whenever AWS talks about something extensively through multiple marketing cycles, it becomes usually a pretty good indicator that they're on their back foot on that area. And for a long time, they were doing that about data and how it's very important to gather data, it unlocks the key to your business, but it always felt a little hollow-slash-hypocritical to me because you're going to some of the same events that I have that AWS throws on. You notice how you have to fill out the exact same form with a whole bunch of mandatory fields every single time, but there never seems to be anything that gets spat back out to you that demonstrates that any human or system has ever read—Tony: Right.Corey: Any of that? It's basically a, “Do what we say, not what we do,” style of story. And I always found that to be a little bit disingenuous.Tony: I don't want to just harp on AWS here. Of course, we can always talk about the two-pizza box rule and the fact that you have lots of small teams there, but I'd rather generalize this. And I think you really—what you're just describing is been my trip through the healthcare system. I had some sports-related injuries this summer, so I've been through a couple of surgeries to repair sports injuries. And it's amazing that every time you go to the doctor's office, you're filling the same HIPAA information over and over again, even with healthcare systems that use the same electronic health records software. So, it's more a function of that it's not just that the technologies are siloed, it's that the organizations are siloed. That's what you're saying.Corey: That is fair. And I think at some level—I don't know if this is a weird extension of Conway's Law or whatnot—but these things all have different backing stores as far as data goes. And there's a—the hard part, it seems, in a lot of companies once they hit a certain point of maturity is not just getting the data in—because they've already done that to some extent—but it's also then making it actionable and helping various data stores internal to the company reconcile with one another and start surfacing things that are useful. It increasingly feels like it's less of a technology problem and more of a people problem.Tony: It is. I mean, put it this way, I spent a lot of time last year, I burned a lot of brain cells working on data fabrics, which is an idea that's in the idea of the beholder. But the ideal of a data fabric is that it's not the tool that necessarily governs your data or secures your data or moves your data or transforms your data, but it's supposed to be the master orchestrator that brings all that stuff together. And maybe sometime 50 years in the future, we might see that.I think the problem here is both technical and organizational. [unintelligible 00:11:58] a promise, you have all these what we used call island silos. We still call them silos or islands of information. And actually, ironically, even though in the cloud we have technologies where we can integrate this, the cloud has actually exacerbated this issue because there's so many islands of information, you know, coming up, and there's so many different little parts of the organization that have their hands on that. That's also a large part of why there's such a big discussion about, for instance, data mesh last year: everybody is concerned about owning their own little piece of the pie, and there's a lot of question in terms of how do we get some consistency there? How do we all read from the same sheet of music? That's going to be an ongoing problem. You and I are going to get very old before that ever gets solved.Corey: Yeah, there are certain things that I am content to die knowing that they will not get solved. If they ever get solved, I will not live to see it, and there's a certain comfort in that, on some level.Tony: Yeah.Corey: But it feels like this stuff is also getting more and more complicated than it used to be, and terms aren't being used in quite the same way as they once were. Something that a number of companies have been saying for a while now has been that customers overwhelmingly are preferring open-source. Open source is important to them when it comes to their database selection. And I feel like that's a conflation of a couple of things. I've never yet found an ideological, purity-driven customer decision around that sort of thing.What they care about is, are there multiple vendors who can provide this thing so I'm not going to be using a commercially licensed database that can arbitrarily start playing games with seat licenses and wind up distorting my cost structure massively with very little notice. Does that align with your—Tony: Yeah.Corey: Understanding of what people are talking about when they say that, or am I missing something fundamental? Which is again, always possible?Tony: No, I think you're onto something there. Open-source is a whole other can of worms, and I've burned many, many brain cells over this one as well. And today, you're seeing a lot of pieces about the, you know, the—that are basically giving eulogies for open-source. It's—you know, like HashiCorp just finally changed its license and a bunch of others have in the database world. What open-source has meant is been—and I think for practitioners, for DBAs and developers—here's a platform that's been implemented by many different vendors, which means my skills are portable.And so, I think that's really been the key to why, for instance, like, you know, MySQL and especially PostgreSQL have really exploded, you know, in popularity. Especially Postgres, you know, of late. And it's like, you look at Postgres, it's a very unglamorous database. If you're talking about stodgy, it was born to be stodgy because they wanted to be an adult database from the start. They weren't the LAMP stack like MySQL.And the secret of success with Postgres was that it had a very permissive open-source license, which meant that as long as you don't hold University of California at Berkeley, liable, have at it, kids. And so, you see, like, a lot of different flavors of Postgres out there, which means that a lot of customers are attracted to that because if I get up to speed on this Postgres—on one Postgres database, my skills should be transferable, should be portable to another. So, I think that's a lot of what's happening there.Corey: Well, I do want to call that out in particular because when I was coming up in the naughts, the mid-2000s decade, the lingua franca on everything I used was MySQL, or as I insist on mispronouncing it, my-squeal. And lately, on same vein, Postgres-squeal seems to have taken over the entire universe, when it comes to the de facto database of choice. And I'm old and grumpy and learning new things as always challenging, so I don't understand a lot of the ways that thing gets managed from the context coming from where I did before, but what has driven the massive growth of mindshare among the Postgres-squeal set?Tony: Well, I think it's a matter of it's 30 years old and it's—number one, Postgres always positioned itself as an Oracle alternative. And the early years, you know, this is a new database, how are you going to be able to match, at that point, Oracle had about a 15-year headstart on it. And so, it was a gradual climb to respectability. And I have huge respect for Oracle, don't get me wrong on that, but you take a look at Postgres today and they have basically filled in a lot of the blanks.And so, it now is a very cre—in many cases, it's a credible alternative to Oracle. Can it do all the things Oracle can do? No. But for a lot of organizations, it's the 80/20 rule. And so, I think it's more just a matter of, like, Postgres coming of age. And the fact is, as a result of it coming of age, there's a huge marketplace out there and so much choice, and so much opportunity for skills portability. So, it's really one of those things where its time has come.Corey: I think that a lot of my own biases are simply a product of the era in which I learned how a lot of these things work on. I am terrible at Node, for example, but I would be hard-pressed not to suggest JavaScript as the default language that people should pick up if they're just entering tech today. It does front-end, it does back-end—Tony: Sure.Corey: —it even makes fries, apparently. There's a—that is the lingua franca of the modern internet in a bunch of different ways. That doesn't mean I'm any good at it, and it doesn't mean at this stage, I'm likely to improve massively at it, but it is the right move, even if it is inconvenient for me personally.Tony: Right. Right. Put it this way, we've seen—and as I said, I'm not an expert in programming languages, but we've seen a huge profusion of programming languages and frameworks. But the fact is that there's always been a draw towards critical mass. At the turn of the millennium, we thought is between Java and .NET. Little did we know that basically JavaScript—which at that point was just a web scripting language—[laugh] we didn't know that it could work on the server; we thought it was just a client. Who knew?Corey: That's like using something inappropriately as a database. I mean, good heavens.Tony: [laugh]. That would be true. I mean, when I could have, you know, easily just use a spreadsheet or something like that. But so, I mean, who knew? I mean, just like for instance, Java itself was originally conceived for a set-top box. You never know how this stuff is going to turn out. It's the same thing happen with Python. Python was also a web scripting language. Oh, by the way, it happens to be really powerful and flexible for data science. And whoa, you know, now Python is—in terms of data science languages—has become the new SaaS.Corey: It really took over in a bunch of different ways. Before that, Perl was great, and I go, “Why would I use—why write in Python when Perl is available?” It's like, “Okay, you know, how to write Perl, right?” “Yeah.” “Have you ever read anything a month later?” “Oh…” it's very much a write-only language. It is inscrutable after the fact. And Python at least makes that a lot more approachable, which is never a bad thing.Tony: Yeah.Corey: Speaking of what you touched on toward the beginning of this episode, the idea of databases not being sentient, which I equate to being self-aware, you just came out very recently with a report on generative AI and a trip that you wound up taking on this. Which I've read; I love it. In fact, we've both been independently using the phrase [unintelligible 00:19:09] to, “English is the new most common programming language once a lot of this stuff takes off.” But what have you seen? What have you witnessed as far as both the ground truth reality as well as the grandiose statements that companies are making as they trip over themselves trying to position as the forefront leader and all of this thing that didn't really exist five months ago?Tony: Well, what's funny is—and that's a perfect question because if on January 1st you asked “what's going to happen this year?” I don't think any of us would have thought about generative AI or large language models. And I will not identify the vendors, but I did some that had— was on some advanced briefing calls back around the January, February timeframe. They were talking about things like server lists, they were talking about in database machine learning and so on and so forth. They weren't saying anything about generative.And all of a sudden, April, it changed. And it's essentially just another case of the tail wagging the dog. Consumers were flocking to ChatGPT and enterprises had to take notice. And so, what I saw, in the spring was—and I was at a conference from SaaS, I'm [unintelligible 00:20:21] SAP, Oracle, IBM, Mongo, Snowflake, Databricks and others—that they all very quickly changed their tune to talk about generative AI. What we were seeing was for the most part, position statements, but we also saw, I think, the early emphasis was, as you say, it's basically English as the new default programming language or API, so basically, coding assistance, what I'll call conversational query.I don't want to call it natural language query because we had stuff like Tableau Ask Data, which was very robotic. So, we're seeing a lot of that. And we're also seeing a lot of attention towards foundation models because I mean, what organization is going to have the resources of a Google or an open AI to develop their own foundation model? Yes, some of the Wall Street houses might, but I think most of them are just going to say, “Look, let's just use this as a starting point.”I also saw a very big theme for your models with your data. And where I got a hint of that—it was a throwaway LinkedIn post. It was back in, I think like, February, Databricks had announced Dolly, which was kind of an experimental foundation model, just to use with your own data. And I just wrote three lines in a LinkedIn post, it was on Friday afternoon. By Monday, it had 65,000 hits.I've never seen anything—I mean, yes, I had a lot—I used to say ‘data mesh' last year, and it would—but didn't get anywhere near that. So, I mean, that really hit a nerve. And other things that I saw, was the, you know, the starting to look with vector storage and how that was going to be supported was it was going be a new type of database, and hey, let's have AWS come up with, like, an, you know, an [ADF 00:21:41] database here or is this going to be a feature? I think for the most part, it's going to be a feature. And of course, under all this, everybody's just falling in love, falling all over themselves to get in the good graces of Nvidia. In capsule, that's kind of like what I saw.Corey: That feels directionally accurate. And I think databases are a great area to point out one thing that's always been more a little disconcerting for me. The way that I've always viewed databases has been, unless I'm calling a RAND function or something like it and I don't change the underlying data structure, I should be able to run a query twice in a row and receive the same result deterministically both times.Tony: Mm-hm.Corey: Generative AI is effectively non-deterministic for all realistic measures of that term. Yes, I'm sure there's a deterministic reason things are under the hood. I am not smart enough or learned enough to get there. But it just feels like sometimes we're going to give you the answer you think you're going to get, sometimes we're going to give you a different answer. And sometimes, in generative AI space, we're going to be supremely confident and also completely wrong. That feels dangerous to me.Tony: [laugh]. Oh gosh, yes. I mean, I take a look at ChatGPT and to me, the responses are essentially, it's a high school senior coming out with an essay response without any footnotes. It's the exact opposite of an ACID database. The reason why we're very—in the database world, we're very strongly drawn towards ACID is because we want our data to be consistent and to get—if we ask the same query, we're going to get the same answer.And the problem is, is that with generative, you know, based on large language models, computers sounds sentient, but they're not. Large language models are basically just a series of probabilities, and so hopefully those probabilities will line up and you'll get something similar. That to me, kind of scares me quite a bit. And I think as we start to look at implementing this in an enterprise setting, we need to take a look at what kind of guardrails can we put on there. And the thing is, that what this led me to was that missing piece that I saw this spring with generative AI, at least in the data and analytics world, is nobody had a clue in terms of how to extend AI governance to this, how to make these models explainable. And I think that's still—that's a large problem. That's a huge nut that it's going to take the industry a while to crack.Corey: Yeah, but it's incredibly important that it does get cracked.Tony: Oh, gosh, yes.Corey: One last topic that I want to get into. I know you said you don't want to over-index on AWS, which, fair enough. It is where I spend the bulk of my professional time and energy—Tony: [laugh].Corey: Focusing on, but I think this one's fair because it is a microcosm of a broader industry question. And that is, I don't know what the DBA job of the future is going to look like, but increasingly, it feels like it's going to primarily be picking which purpose-built AWS database—or larger [story 00:24:56] purpose database is appropriate for a given workload. Even without my inappropriate misuse of things that are not databases as databases, they are legitimately 15 or 16 different AWS services that they position as database offerings. And it really feels like you're spiraling down a well of analysis paralysis, trying to pick between all these things. Do you think the future looks more like general-purpose databases, or very purpose-built and each one is this beautiful, bespoke unicorn?Tony: [laugh]. Well, this is basically a hit on a theme that I've been—you know, we've been all been thinking about for years. And the thing is, there are arguments to be made for multi-model databases, you know, versus a for-purpose database. That being said, okay, two things. One is that what I've been saying, in general, is that—and I wrote about this way, way back; I actually did a talk at the [unintelligible 00:25:50]; it was a throwaway talk, or [unintelligible 00:25:52] one of those conferences—I threw it together and it's basically looking at the emergence of all these specialized databases.But how I saw, also, there's going to be kind of an overlapping. Not that we're going to come back to Pangea per se, but that, for instance, like, a relational database will be able to support JSON. And Oracle, for instance, does has some fairly brilliant ideas up the sleeve, what they call a JSON duality, which sounds kind of scary, which basically says, “We can store data relationally, but superimpose GraphQL on top of all of this and this is going to look really JSON-y.” So, I think on one hand, you are going to be seeing databases that do overlap. Would I use Oracle for a MongoDB use case? No, but would I use Oracle for a case where I might have some document data? I could certainly see that.The other point, though, and this is really one I want to hammer on here—it's kind of a major concern I've had—is I think the cloud vendors, for all their talk that we give you operational simplicity and agility are making things very complex with its expanding cornucopia of services. And what they need to do—I'm not saying, you know, let's close down the patent office—what I think we do is we need to provide some guided experiences that says, “Tell us the use case. We will now blend these particular services together and this is the package that we would suggest.” I think cloud vendors really need to go back to the drawing board from that standpoint and look at, how do we bring this all together? How would he really simplify the life of the customer?Corey: That is, honestly, I think the biggest challenge that the cloud providers have across the board. There are hundreds of services available at this point from every hyperscaler out there. And some of them are brand new and effectively feel like they're there for three or four different customers and that's about it and others are universal services that most people are probably going to use. And most things fall in between those two extremes, but it becomes such an analysis paralysis moment of trying to figure out what do I do here? What is the golden path?And what that means is that when you start talking to other people and asking their opinion and getting their guidance on how to do something when you get stuck, it's, “Oh, you're using that service? Don't do it. Use this other thing instead.” And if you listen to that, you get midway through every problem for them to start over again because, “Oh, I'm going to pick a different selection of underlying components.” It becomes confusing and complicated, and I think it does customers largely a disservice. What I think we really need, on some level, is a simplified golden path with easy on-ramps and easy off-ramps where, in the absence of a compelling reason, this is what you should be using.Tony: Believe it or not, I think this would be a golden case for machine learning.Corey: [laugh].Tony: No, but submit to us the characteristics of your workload, and here's a recipe that we would propose. Obviously, we can't trust AI to make our decisions for us, but it can provide some guardrails.Corey: “Yeah. Use a graph database. Trust me, it'll be fine.” That's your general purpose—Tony: [laugh].Corey: —approach. Yeah, that'll end well.Tony: [laugh]. I would hope that the AI would basically be trained on a better set of training data to not come out with that conclusion.Corey: One could sure hope.Tony: Yeah, exactly.Corey: I really want to thank you for taking the time to catch up with me around what you're doing. If people want to learn more, where's the best place for them to find you?Tony: My website is dbinsight.io. And on my homepage, I list my latest research. So, you just have to go to the homepage where you can basically click on the links to the latest and greatest. And I will, as I said, after Labor Day, I'll be publishing my take on my generative AI journey from the spring.Corey: And we will, of course, put links to this in the [show notes 00:29:39]. Thank you so much for your time. I appreciate it.Tony: Hey, it's been a pleasure, Corey. Good seeing you again.Corey: Tony Baer, principal at dbInsight. I'm Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry, insulting comment that we will eventually stitch together with all those different platforms to create—that's right—a large-scale distributed database.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

Josh and Kurt talk about why CVE is making the news lately. Things are not well in the CVE program, and it's not looking like anything will get fixed anytime soon. Josh and Kurt have a unique set of knowledge around CVE. There's a lot of confusion and difficulty in understanding how CVE works. Show Notes Curl blog post Now it's PostgreSQL's turn to have a bogus CVE GitHub Advisory Database Josh's "CVE tried to get me fired" story

This week, we discuss Netflix's DVD deprecation, the remote work debate, and how to fork an open-source project. Plus, thoughts on why Europe needs more ice. Watch the YouTube Live Recording of Episode (https://www.youtube.com/watch?v=lFr-ysPYxnA) 431 (https://www.youtube.com/watch?v=lFr-ysPYxnA) Runner-up Titles Try Harder It's a necessary luxury Someone's drinking too much water here A culture of ice Where are the high performers, at home or at work Quit using your Gmail address Thou shalt export to CSV Rundown Netflix Says You Can Keep Their DVDs (and Request More, Too) (https://www.nytimes.com/2023/08/24/arts/netflix-dvds.html?smid=nytcore-ios-share&referringSource=articleShare) Zoom's CEO thinks Zoom sucks for building trust, leaked audio reveals (https://arstechnica.com/tech-policy/2023/08/leaked-audio-reveals-zoom-ceo-believes-its-hard-to-build-trust-on-zoom/) Meta is back in the office three days a week, as WFH continues to die (https://www.theverge.com/2023/9/5/23860073/meta-return-to-office-three-days-wfh-work-from-home) Can you trust 'open source' companies? (https://www.theregister.com/2023/08/18/opinion_column/) OpenTF created a fork of Terraform! (https://opentf.org/announcement) OpenTF pulls the trigger on its open-source Terraform fork (https://opensourcewatch.beehiiv.com/p/opentf-pulls-trigger-opensource-terraform-fork) Relevant to your Interests VMware's future: Navigating multicloud complexity and generative AI (https://siliconangle.com/2023/08/19/vmwares-future-navigating-multicloud-complexity-generative-ai-broadcoms-wing/) VMware Tanzu portfolio reshuffled ahead of Broadcom close | TechTarget (https://www.techtarget.com/searchitoperations/news/366549332/VMware-Tanzu-portfolio-reshuffled-ahead-of-Broadcom-close) Nvidia's blowout offers a giddy whiff of 1995 (https://www.axios.com/newsletters/axios-ai-plus-937b329c-8072-4f8a-a5d6-1039a0e794a5.html?chunk=0&utm_term=emshare#story0) Announcing AWS Dedicated Local Zones (https://aws.amazon.com/about-aws/whats-new/2023/08/aws-dedicated-local-zones/?utm_source=newsletter&utm_medium=email&utm_campaign=newsletter_axioslogin&stream=top) Top Ten social media platforms we spend the most time on (https://www.traveldailymedia.com/top-ten-social-media-platforms-we-spend-the-most-time-on/) Max will launch a 24/7 CNN stream for all subscribers next month (https://www.theverge.com/2023/8/24/23844121/cnn-max-warnerbros-discovery-news) Meta launches own AI code-writing tool: Code Llama (https://www.theverge.com/2023/8/24/23843487/meta-llama-code-generation-generative-ai-llm?stream=top) As TikTok Ban Looms, ByteDance Battles Oracle For Control Of Its Algorithm (https://www.forbes.com/sites/emilybaker-white/2023/08/24/tiktok-ban-oracle-bytedance-algorithm-fight/?sh=6cf5105e3ef0) Slack's Migration to a Cellular Architecture - Slack Engineering (https://slack.engineering/slacks-migration-to-a-cellular-architecture/) The Cloud 100 2023 (https://www.forbes.com/lists/cloud100/) Data isn't everything. Judgement counts too. (https://www.tiktok.com/t/ZT8YFUFju/) Amazon Elastic Block Store at 15 Years (https://perspectives.mvdirona.com/2023/08/amazon-elastic-block-store-at-15-years/?ck_subscriber_id=512840665) Instacart is the Best and Worst Grocery Business Imaginable (https://www.thediff.co/archive/instacart-is-the-best-and-worst-grocery-business-imaginable/) Amazon CEO Andy Jassy tells employees it's 'past' time to commit to the company's RTO mandate and their jobs are at stake (https://www.businessinsider.com/amazon-andy-jassy-rto-office-policy-employee-jobs-2023-8?op=1) Duet AI, Google's AI assistant suite, expands across Google Cloud (https://techcrunch.com/2023/08/29/duet-ai-googles-ai-assistant-suite-expands-across-google-cloud/) Halloween creeps a little closer: Seasonal supply chains accelerate (https://www.spglobal.com/marketintelligence/en/mi/research-analysis/halloween-creeps-closer-seasonal-supply-chains-accelerate.html) What's new with GKE at Google Cloud Next | Google Cloud Blog (https://cloud.google.com/blog/products/containers-kubernetes/whats-new-with-gke-at-google-cloud-next) Duet AI in Google Cloud Preview | Google Cloud Blog (https://cloud.google.com/blog/products/ai-machine-learning/duet-ai-in-google-cloud-preview) What's new in Oracle to PostgreSQL database migrations with DMS | Google Cloud Blog (https://cloud.google.com/blog/products/databases/whats-new-in-oracle-to-postgresql-database-migrations-with-dms) US AI startup Poolside raises $126m seed round and relocates to France (https://sifted.eu/articles/poolside-raises-126m-relocated-france-news) Ping, ForgeRock, Thoma Bravo, the power of open source, and the madness of IAM (https://callmeleach.substack.com/p/ping-forgerock-thoma-bravo-the-power?utm_medium=web) Thoma Bravo Completes Acquisition of ForgeRock; Combines ForgeRock into Ping Identity (https://www.prnewswire.com/news-releases/thoma-bravo-completes-acquisition-of-forgerock-combines-forgerock-into-ping-identity-301908059.html) Interoperability between Google Chat and other messaging platforms — powered by Mio (https://workspaceupdates.googleblog.com/2023/08/goolge-chat-slack-interoperability-mio.html) Broadcom boss dismisses notion China could derail VMware buy (https://www.theregister.com/2023/09/01/broadcom_vmware_nutanix_results/) Microsoft blames outage on small staff, automation failures (https://www.theregister.com/2023/09/04/microsoft_australia_outage_incident_report/) Amazon QuickSight adds scheduled and programmatic export to Excel format (https://aws.amazon.com/about-aws/whats-new/2023/08/amazon-quicksight-scheduled-programmatic-export-excel-format/?ck_subscriber_id=512840665) Google unveils AI tools for enterprise customers at $30 a month (https://www.reuters.com/technology/google-unveil-ai-tools-corporate-gmail-customers-30-month-wsj-2023-08-29/) Chip design firm Arm seeks up to $52 billion valuation in blockbuster U.S. IPO (https://www.cnbc.com/2023/09/05/chip-design-firm-arm-sets-share-price-between-47-and-51-for-blockbuster-us-ipo.html) Birmingham City Council goes under after Oracle disaster (https://www.theregister.com/2023/09/05/birmingham_city_council_oracle/?s=08) IBM Introduces 'Watsonx Your Business' (https://finance.yahoo.com/news/ibm-introduces-watsonx-business-160000392.html) Meta May Allow Instagram, Facebook Users in Europe to Pay and Avoid Ads (https://www.nytimes.com/2023/09/01/technology/meta-instagram-facebook-ads-europe.html?smid=nytcore-ios-share&referringSource=articleShare) Announcing Kubecost Cloud in General Availability: The Easiest Way to Optimize Your Kubernetes Costs (https://blog.kubecost.com/blog/kubecost-cloud-general-availability/) Platform Engineering - What You Need To Know Now (https://tanzu.vmware.com/content/ebooks/platformengineering-whatyouneedtoknownow?utm_source=cote&utm_campaign=devrel&utm_medium=newsletter&utm_content=newsletter20230830) The lifespans of technological adoptions in the US (http://www.asymco.com/2022/01/10/the-lifespans-of-technological-adoptions-in-the-us/) Introducing ONCE (https://once.com/) Nonsense The fight for the right to repair McFlurry machines (https://www.morningbrew.com/daily/stories/2023/08/31/the-fight-for-the-right-to-repair-mcflurry-machines) Delta Airlines Offers Woman $1,800 After Losing Her Dog (https://www.yahoo.com/entertainment/delta-airlines-offers-woman-1-142849291.html) Conferences Sep 18th to 19th SHIFT (https://shift.infobip.com/) in Zadar, Coté speaking. October 6, 2023, KCD Texas 2023 (https://community.cncf.io/events/details/cncf-kcd-texas-presents-kcd-texas-2023/), CFP Closes: August 30, 2023 November 6-9, 2023, KubeCon NA (https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/), SDT's a sponsor, Matt's there November 6-9, 2023 VMware Explore Barcelona (https://www.vmware.com/explore/eu.html), Coté's attending Jan 29, 2024 to Feb 1, 2024 That Conference Texas (https://that.us/events/tx/2024/schedule/) If you want your conference mentioned, let's talk media sponsorships. SDT news & hype Join us in Slack (http://www.softwaredefinedtalk.com/slack). Get a SDT Sticker! Send your postal address to stickers@softwaredefinedtalk.com (mailto:stickers@softwaredefinedtalk.com) and we will send you free laptop stickers! Follow us: Twitch (https://www.twitch.tv/sdtpodcast), Twitter (https://twitter.com/softwaredeftalk), Instagram (https://www.instagram.com/softwaredefinedtalk/), Mastodon (https://hachyderm.io/@softwaredefinedtalk), BlueSky (https://bsky.app/profile/softwaredefinedtalk.com), LinkedIn (https://www.linkedin.com/company/software-defined-talk/), TikTok (https://www.tiktok.com/@softwaredefinedtalk), Threads (https://www.threads.net/@softwaredefinedtalk) and YouTube (https://www.youtube.com/channel/UCi3OJPV6h9tp-hbsGBLGsDQ/featured). Use the code SDT to get $20 off Coté's book, Digital WTF (https://leanpub.com/digitalwtf/c/sdt), so $5 total. Become a sponsor of Software Defined Talk (https://www.softwaredefinedtalk.com/ads)! Recommendations Brandon: JUST ONE MILE | Official Trailer (https://www.youtube.com/watch?v=80V5o06yEZ4) Matt: Deadloch (https://www.imdb.com/title/tt14671678/) Coté: Rick Rubin interviews Rory Sutherland (https://www.youtube.com/watch?v=VnYlChfORRw). I doubt much of the airport business book stuff in here is “true,” but that's sort of the whole point, and it's fantastic listening. His book (https://amzn.to/462Mvov) Alchemy (https://amzn.to/462Mvov) has a great one word review right there in the title. But, again: it's fun! When you've listened to too much If Books Could Kill (https://en.wikipedia.org/wiki/If_Books_Could_Kill) you can check in on Rory if you need to take the cure (https://idioms.thefreedictionary.com/take+the+cure). Photo Credits Header (https://unsplash.com/photos/PsBTqRHVilU) Artwork (https://labs.openai.com/e/bKjqW8kPJyI2wuzBA0FogiKb/UJeLhuIFmvkrNFbfcCc4jE29)

If you love taking about databases, this is the episode for you. Ahead of the launch of his new book, High Performance PostgreSQL for Rails, Andrew Atkinson joined the show with special guest co-host, Pat Bair, to talk about why he wrote a book, why he focused on PostgreSQL and his favorite feature from the upcoming 7.1 release. Show Notes: Landing page and newsletter signup for the book “High Performance PostgreSQL for Rails” (https://pgrailsbook.com/) Andrew's Tweet Announcing Beta for the Book (https://twitter.com/andatki/status/1696933498219569615) Andrew's Personal blog on Postgres/Rails/general topics (https://andyatkinson.com/) Rideshare Rails API app used for book examples/exercises (https://github.com/andyatkinson/rideshare) Pg_scripts repo (https://github.com/andyatkinson/pg_scripts) Andrew's Postgres and Rails presentations and podcast appearances on YouTube (https://www.youtube.com/@andatki/playlists) Andrew Atkinson (@andatki) / X (https://twitter.com/andatki?lang=en) Andrew on GitHub (https://github.com/andyatkinson) Pat on Github (https://github.com/pjb4752) Sponsored By: Honeybadger (https://www.honeybadger.io/) You won't know if Honeybadger will really save you time and trouble until you see how it works in your own toolchain. With two lines of code and five minutes, you can see for yourself. Honeybadger automatically hooks into popular web frameworks, job systems, authentication libraries, and front-end JavaScript. Get started today in as little as 5 minutes at Honeybadger.io (https://www.honeybadger.io/) with plans starting at free! Scout APM (http://scoutapm.com/rubyonrails) Experience the perfect blend of efficiency and accuracy with Scout APM. Our performance monitoring solution is tailor-made for Rails developers, providing fast and effective troubleshooting with an intuitive UI and advanced tracing logic. With real-time anomaly detection and instant alerts, you can swiftly resolve issues like N+1 queries and memory bloat, and prevent customer impact. Don't wait any longer to optimize your Rails app performance - sign up for our free 14-day trial today at scoutapm.com/rubyonrails (http://scoutapm.com/rubyonrails).

Summary Data persistence is one of the most challenging aspects of computer systems. In the era of the cloud most developers rely on hosted services to manage their databases, but what if you are a cloud service? In this episode Vignesh Ravichandran explains how his team at Cloudflare provides PostgreSQL as a service to their developers for low latency and high uptime services at global scale. This is an interesting and insightful look at pragmatic engineering for reliability and scale. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. You specify the customer traits, then Profiles runs the joins and computations for you to create complete customer profiles. Get all of the details and try the new product today at dataengineeringpodcast.com/rudderstack (https://www.dataengineeringpodcast.com/rudderstack) This episode is brought to you by Datafold – a testing automation platform for data engineers that finds data quality issues before the code and data are deployed to production. Datafold leverages data-diffing to compare production and development environments and column-level lineage to show you the exact impact of every code change on data, metrics, and BI tools, keeping your team productive and stakeholders happy. Datafold integrates with dbt, the modern data stack, and seamlessly plugs in your data CI for team-wide and automated testing. If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold (https://www.dataengineeringpodcast.com/datafold) You shouldn't have to throw away the database to build with fast-changing data. You should be able to keep the familiarity of SQL and the proven architecture of cloud warehouses, but swap the decades-old batch computation model for an efficient incremental engine to get complex queries that are always up-to-date. With Materialize, you can! It's the only true SQL streaming database built from the ground up to meet the needs of modern data products. Whether it's real-time dashboarding and analytics, personalization and segmentation or automation and alerting, Materialize gives you the ability to work with fresh, correct, and scalable results — all in a familiar SQL interface. Go to dataengineeringpodcast.com/materialize (https://www.dataengineeringpodcast.com/materialize) today to get 2 weeks free! Your host is Tobias Macey and today I'm interviewing Vignesh Ravichandran about building an internal database as a service platform at Cloudflare Interview Introduction How did you get involved in the area of data management? Can you start by describing the different database workloads that you have at Cloudflare? What are the different methods that you have used for managing database instances? What are the requirements and constraints that you had to account for in designing your current system? Why Postgres? optimizations for Postgres simplification from not supporting multiple engines limitations in postgres that make multi-tenancy challenging scale of operation (data volume, request rate What are the most interesting, innovative, or unexpected ways that you have seen your DBaaS used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on your internal database platform? When is an internal database as a service the wrong choice? What do you have planned for the future of Postgres hosting at Cloudflare? Contact Info LinkedIn (https://www.linkedin.com/in/vigneshravichandran28/) Website (https://viggy28.dev/) Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. Podcast.__init__ (https://www.pythonpodcast.com) covers the Python language, its community, and the innovative ways it is being used. The Machine Learning Podcast (https://www.themachinelearningpodcast.com) helps you go from idea to production with machine learning. Visit the site (https://www.dataengineeringpodcast.com) to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@dataengineeringpodcast.com (mailto:hosts@dataengineeringpodcast.com)) with your story. To help other people find the show please leave a review on Apple Podcasts (https://podcasts.apple.com/us/podcast/data-engineering-podcast/id1193040557) and tell your friends and co-workers Links Cloudflare (https://www.cloudflare.com/) PostgreSQL (https://www.postgresql.org/) Podcast Episode (https://www.dataengineeringpodcast.com/postgresql-with-jonathan-katz-episode-42/) IP Address Data Type in Postgres (https://www.postgresql.org/docs/current/datatype-net-types.html) CockroachDB (https://www.cockroachlabs.com/) Podcast Episode (https://www.dataengineeringpodcast.com/cockroachdb-with-peter-mattis-episode-35/) Citus (https://www.citusdata.com/) Podcast Episode (https://www.dataengineeringpodcast.com/citus-data-with-ozgun-erdogan-and-craig-kerstiens-episode-13/) Yugabyte (https://www.yugabyte.com/) Podcast Episode (https://www.dataengineeringpodcast.com/yugabytedb-planet-scale-sql-episode-115/) Stolon (https://github.com/sorintlab/stolon) pg_rewind (https://www.postgresql.org/docs/current/app-pgrewind.html) PGBouncer (https://www.pgbouncer.org/) HAProxy Presentation (https://www.youtube.com/watch?v=HIOo4j-Tiq4) Etcd (https://etcd.io/) Patroni (https://patroni.readthedocs.io/en/latest/) pg_upgrade (https://www.postgresql.org/docs/current/pgupgrade.html) Edge Computing (https://en.wikipedia.org/wiki/Edge_computing) The intro and outro music is from The Hug (http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/Love_death_and_a_drunken_monkey/04_-_The_Hug) by The Freak Fandango Orchestra (http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/) / CC BY-SA (http://creativecommons.org/licenses/by-sa/3.0/)

Evan and Sujit cover a whole bunch of updates to various Azure services such as AKS, ACA, Functions, VMs, PostgerSQL, VNETs and Storage. Media file: https://azpodcast.blob.core.windows.net/episodes/Episode470.mp3 YouTube: https://youtu.be/eGsnmw5WjHI  Resources: VMs https://azure.microsoft.com/en-us/updates/generally-available-cross-subscription-restore-for-azure-virtual-machines/ https://techcommunity.microsoft.com/t5/azure-high-performance-computing/introducing-new-performance-tiers-for-azure-managed-lustre/ba-p/3900329   Storage https://azure.microsoft.com/en-us/updates/azure-elastic-san-updates-private-endpoints-shared-volumes/   Functions https://azure.microsoft.com/en-us/updates/generally-available-sdk-type-bindings-in-azure-functions/ https://azure.microsoft.com/en-us/updates/generally-available-application-insights-integration-for-the-azure-functions-net-worker/   AKS/ACA https://azure.microsoft.com/en-us/updates/generally-available-kubernetes-127-support-in-aks/ https://azure.microsoft.com/en-us/updates/generally-available-azure-key-vault-references-for-secrets-in-azure-container-apps/ https://azure.microsoft.com/en-us/updates/generally-available-secrets-volume-mounts-for-azure-container-apps/ https://azure.microsoft.com/en-us/updates/generally-available-session-affinity-for-azure-container-apps/ https://azure.microsoft.com/en-us/updates/generally-available-init-containers-in-azure-container-apps/ https://azure.microsoft.com/en-us/updates/generally-available-cross-origin-resource-sharing-cors-in-azure-container-apps/   PostgreSQL https://azure.microsoft.com/en-us/updates/public-preview-storage-autogrow-online-disk-scaling-for-azure-database-for-postgresql-flexible-server/ Major Version Upgrade of a flexible server - Azure portal | Microsoft Learn   VNET https://azure.microsoft.com/en-us/updates/public-preview-azure-vnet-flow-logs/

Nectain je low-code platforma původem z Ukrajiny, kde ji znají úplně všichni. A to buď pod jménem mateřské firmy Intecracy, nebo sesterského DMS Megapolis, které používá velká část státní správy. Součástí je i databáze optimalizovaná pro Big Data, připravená na miliony záznamů a zpracování tisíců requestů za vteřinu. Díky expanzi do světa má dnes Nectain přes 100 tisíc uživatelů, akvizice dalších je ale docela dřina. CEO Roman Olejnikov totiž nedávno na meetingu slyšel, že jejich nabídka je „too good to be true“

Sabine talks to Michele Cimmino, CEO of Lasting Dynamics which, far from being a generic software firm, epitomises precision and coding to partners' distinct needs in order to ensure an exclusive synergy for mutual growth. Michael discusses visionary leadership at Lasting Dynamics, championing quality and innovation, ensuring a diverse tech mastery, and building a global force in the startup ecosystem. KEY TAKEAWAYSLasting Dynamics was started in 2013 to provide the IT market with quality. Large enterprises usually target a span of 12 months to deliver an MVP (minimum viable product), by the time an MVP is delivered to market in 12 months it's already too late. When you have engineers who truly understand cybersecurity then it's easier to be compliant with policies. When you build something from the ground up, or start integrating the pieces into a digital product, with a forward-looking mindset you will not waste months in additional testing. This is an engineering mindset called “bottom-up”. BEST MOMENTS‘Quality assurance is also about analysing the requirements, it's not only about executing some tests, it's about understanding what will be needed and how this can be provided.'‘The most important part of the game is to be transparent and loyal to your partner or customer. That always pays back.'‘I like the idea that the most important goal and value for everyone on a project should be the final result of the product and everyone should sacrifice a bit of personal interest towards that goal.' ABOUT THE GUESTIn the dynamic world of software engineering, few names resonate as powerfully as Michele Cimmino. As a master of full stack web and mobile development, Michele's leadership at Lasting Dynamics has carved a niche in the tech spaces of Spain, Italy, and Norway. Dive deeper at Lasting Dynamics and you'll discover a hub where innovation meets commitment. A champion of quality, Michele's passion drives him and his adept team to consistently re-evaluate and refine their methodologies. They are agile, experimental, and always ahead of the curve, drawing insights from the latest in automation, innovation, and QA processes. His current explorations into Agile, OKRs, SCRUM, TDD, and the principles of Shu Ha Ri demonstrate a ceaseless thirst for knowledge. Lasting Dynamics, Michele's brainchild, isn't just any software company; it's a boutique, meticulously crafting code to fit partners' unique needs. This exclusivity ensures mutual growth, as they partner with only a select few each year. But there's more: The Academy, another feather in Michele's cap, streamlines the onboarding process, a testament to his visionary approach. Moreover, under his guidance, the launch of SaaS platforms like Roundrush for workflow management and VetrinaLive for e-commerce adds to his impressive portfolio. With a tech stack that's as expansive as it is impressive, ranging from Node.js to Python, React to Vue.js, PostgreSQL to AWS integrations, Michele's proficiency is unquestionable. He's been at the helm of AI-driven mobile apps since 2015, dabbled in the realm of blockchain and cryptocurrency from 2017, and from 2019, has steered Lasting Dynamics towards Growth Hacking. ABOUT THE HOSTSabine is a corporate strategist turned entrepreneur. She is the CEO and Managing Partner of Alchemy Crew, a venture lab that accelerates the curation, validation, and commercialization of new tech business models. Sabine is renowned within the insurance sector for building some of the most renowned tech startup accelerators around the world working with over 30 corporate insurers and accelerating over 100 startup ventures. Sabine is the co-editor of the bestseller The INSURTECH Book, a top 50 Women in Tech, a FinTech and InsurTech Influencer, an investor & multi-award winner.  Twitter: SabineVdLLinkedIn: Sabine VanderLindenInstagram: sabinevdLofficialFacebook: SabineVdLOfficialTikTok: sabinevdlofficialEmail: podcast@sabinevdl.comWebsite: www.sabinevdl.comThis show was brought to you by Progressive Media

To get the show notes as well as get notified of new episodes, visit:  https://www.scalingpostgres.com/episodes/277-postgres-releases-postgresql-survey-partitioning-sharding-bulk-loading/ In this episode of Scaling Postgres, we discuss new Postgres releases, taking the 2023 State of PostgreSQL survey, partitioning vs. sharding and the fastest way to do bulk loads.

Simon takes you on a tour of your GenAI options. From software development, to AI policy, to trialling FMs, to new instance types, and CPUs. Referenced Links: - URL: https://aws.amazon.com/blogs/devops/optimize-software-development-with-amazon-codewhisperer/ Title: Optimize software development with Amazon CodeWhisperer | AWS DevOps Blog - URL: https://aws.amazon.com/blogs/devops/10-ways-to-build-applications-faster-with-amazon-codewhisperer/ Title: 10 ways to build applications faster with Amazon CodeWhisperer | AWS DevOps Blog - URL: https://aws.amazon.com/blogs/big-data/build-data-integration-jobs-with-ai-companion-on-aws-glue-studio-notebook-powered-by-amazon-codewhisperer/ Title: Build data integration jobs with AI companion on AWS Glue Studio Notebook powered by Amazon CodeWhisperer - URL: https://aws.amazon.com/blogs/machine-learning/use-generative-ai-foundation-models-in-vpc-mode-with-no-internet-connectivity-using-amazon-sagemaker-jumpstart/ Title: Use generative AI foundation models in VPC mode with no internet connectivity using Amazon SageMaker JumpStart | AWS Machine Learning Blog - URL: https://aws.amazon.com/blogs/machine-learning/llama-2-foundation-models-from-meta-are-now-available-in-amazon-sagemaker-jumpstart/ Title: Llama 2 foundation models from Meta are now available in Amazon SageMaker JumpStart | AWS Machine Learning Blog - URL: https://aws.amazon.com/blogs/machine-learning/use-stable-diffusion-xl-with-amazon-sagemaker-jumpstart-in-amazon-sagemaker-studio/ Title: Use Stable Diffusion XL with Amazon SageMaker JumpStart in Amazon SageMaker Studio | AWS Machine Learning Blog - URL: https://aws.amazon.com/blogs/machine-learning/efficiently-train-tune-and-deploy-custom-ensembles-using-amazon-sagemaker/ Title: Efficiently train, tune, and deploy custom ensembles using Amazon SageMaker | AWS Machine Learning Blog - URL: https://aws.amazon.com/blogs/machine-learning/quickly-build-high-accuracy-generative-ai-applications-on-enterprise-data-using-amazon-kendra-langchain-and-large-language-models/ Title: Quickly build high-accuracy Generative AI applications on enterprise data using Amazon Kendra, LangChain, and large language models | AWS Machine Learning Blog - URL: https://aws.amazon.com/bedrock/features/ Title: Foundation Model API Service – Amazon Bedrock Features – AWS - URL: https://aws.amazon.com/blogs/aws/preview-enable-foundation-models-to-complete-tasks-with-agents-for-amazon-bedrock/ Title: Preview – Enable Foundation Models to Complete Tasks With Agents for Amazon Bedrock | AWS News Blog - URL: https://aws.amazon.com/blogs/business-intelligence/announcing-generative-bi-capabilities-in-amazon-quicksight/ Title: Announcing Generative BI capabilities in Amazon QuickSight | AWS Business Intelligence Blog - URL: https://aws.amazon.com/about-aws/whats-new/2023/05/amazon-rds-postgresql-pgvector-ml-model-integration/ Title: Amazon RDS for PostgreSQL now supports pgvector for simplified ML model integration - URL: https://aws.amazon.com/blogs/big-data/introducing-the-vector-engine-for-amazon-opensearch-serverless-now-in-preview/ Title: Introducing the vector engine for Amazon OpenSearch Serverless, now in preview | AWS Big Data Blog - URL: https://aws.amazon.com/blogs/aws/new-amazon-ec2-p5-instances-powered-by-nvidia-h100-tensor-core-gpus-for-accelerating-generative-ai-and-hpc-applications/ Title: New – Amazon EC2 P5 Instances Powered by NVIDIA H100 Tensor Core GPUs for Accelerating Generative AI and HPC Applications | AWS News Blog - URL: https://aws.amazon.com/blogs/machine-learning/maximize-stable-diffusion-performance-and-lower-inference-costs-with-aws-inferentia2/ Title: Maximize Stable Diffusion performance and lower inference costs with AWS Inferentia2 | AWS Machine Learning Blog - URL: https://aws.amazon.com/blogs/machine-learning/new-technical-deep-dive-course-generative-ai-foundations-on-aws/ Title: New technical deep dive course: Generative AI Foundations on AWS | AWS Machine Learning Blog - URL: https://aws.amazon.com/blogs/machine-learning/aws-offers-new-artificial-intelligence-machine-learning-and-generative-ai-guides-to-plan-your-ai-strategy/ Title: AWS offers new artificial intelligence, machine learning, and generative AI guides to plan your AI strategy | AWS Machine Learning Blog - URL: https://aws.amazon.com/blogs/machine-learning/aws-reaffirms-its-commitment-to-responsible-generative-ai/ Title: AWS Reaffirms its Commitment to Responsible Generative AI | AWS Machine Learning Blog

Benedikt talks to Andrew Atkinson about different types of database constraints and his upcoming book. Read the tweet that triggered this interview Learn more about High Performance PostgreSQL for Rails Check out Andrew's blog Follow Andrew on Twitter Andrew Atkinson is a staff software engineer at Fountain who is very passionate about Ruby on Rails and PostgreSQL (just like Benedikt!). He's so passionate about them that he's actually writing a book called “High Performance PostgreSQL for Rails” that's coming out very soon.Benedikt and Andrew also talk about dealing with constraints and ActiveRecord, Rails validations, and more.

Dans cet épisode estival Guillaume, Emmanuel et Arnaud parcourent les nouvelles du début d'été. Du Java, du Rust, du Go du coté des langages, du Micronaut, du Quarkus pour les frameworks, mais aussi du WebGPU, de l'agilité, du DDD, des sondages, de nombreux outils et surtout de l'intelligence artificielle à toutes les sauces (dans les bases de données, dans les voitures…). Enregistré le 21 juillet 2023 Téléchargement de l'épisode LesCastCodeurs-Episode-298.mp3 News Langages La release candidate de Go 1.21 supporte WASM et WASI nativement https://go.dev/blog/go1.21rc StringBuilder ou contatenation de String https://reneschwietzke.de/java/the-stringbuilder-advise-is-dead-or-isnt-it.html StringBuilder était la recommendation ca cela créait moins d'objects notamment. Mais la JVM a évolué et le compilateur ou JIT remplace cela par du code efficace Quelques petites exceptions le code froid (e.g. startup time) qui est encore interprété peut beneficier de StringBuilder autre cas, la concatenation dans des boucles où le JIT ne pourrait peut etre pas optimiser le StringBuilder “fluid” est plus efficace (inliné?) ces regles ne changement pas si des objects sont stringifié pour etre concaténés GPT 4 pas une revolution https://thealgorithmicbridge.substack.com/p/gpt-4s-secret-has-been-revealed rumeur ca beaucou de secret pas u modele a 1 trillion de parametres maus 8 a 220 Milliards combinés intelligeament les chercheurs attendaient un breakthrough amis c'est une envolution et pas particulierement nouveau methode deja implem,entee par des cherchers chez google (maintenant chez ooenai ils ont retarde la competition avec ces rumeurs de breakthrough amis 8 LLaMA peut peut etre rivaliser avec GPT4 Le blog Open Source de Google propose un article sur 5 mythes ou non sur l'apprentissage et l'utilisation de Rust https://opensource.googleblog.com/2023/06/rust-fact-vs-fiction-5-insights-from-googles-rust-journey-2022.html Il faut plus de 6 mois pour apprendre Rust : plutôt faux; quelques semaines à 3-4 mois max Le compilateur Rust est pas aussi rapide qu'on le souhaiterait — vrai ! Le code unsafe et l'interop sont les plus gros challanges — faux, c'est plutôt les macros, l'owernship/borrowing, et la programmation asynchrone Rust fournit des messages d'erreur de compilation géniaux — vrai Le code Rust est de haute qualité — vrai InfoQ sort un nouveau guide sur le Pattern Matching pour le switch de Java https://www.infoq.com/articles/pattern-matching-for-switch/ Le pattern matching supporte tous les types de référence L'article parle du cas de la valeur null L'utilisation des patterns “guarded” avec le mot clé when L'importance de l'ordre des cases Le pattern matching peut être utilisé aussi avec le default des switchs Le scope des variables du pattern Un seul pattern par case label Un seul case match-all dans un bloc switch L'exhaustivité de la couverture des types L'utilisation des generics La gestion d'erreur avec MatchException Librairies Sortie de Micronaut 4 https://micronaut.io/2023/07/14/micronaut-framework-4-0-0-released/ Langage minimal : Java 17, Groovy 4 et Kotlin 1.8 Support de la dernière version de GraalVM Utilisation des GraalVM Reachability Metadata Repository pour faciliter l'utilisation de Native Image Gradle 8 Nouveau Expression Language, à la compilation, pas possible au runtime (pour des raisons de sécurité et de support de pré-compilation) Support des Virtual Threads Nouvelle couche HTTP, éliminant les stack frames réactives quand on n'utilise pas le mode réactif Support expérimental de IO Uring et HTTP/3 Des filtres basés sur les annotations Le HTTP Client utilise maintenant le Java HTTP Client Génération de client et de serveur en Micronaut à partir de fichier OpenAPI L'utilisation YAML n'utilise plus la dépendance SnakeYAML (qui avait des problèmes de sécurité) Transition vers Jackarta terminé Et plein d'autres mises à jour de modules Couverture par InfoQ https://www.infoq.com/news/2023/07/micronaut-brings-virtual-thread/ Quarkus 3.2 et LTS https://quarkus.io/blog/quarkus-3-2-0-final-released/ https://quarkus.io/blog/quarkus-3-1-0-final-released/ https://quarkus.io/blog/lts-releases/ Infrastructure Red Hat partage les sources de sa distribution au travers de son Customer Portal, et impacte la communauté qui se base dessus https://almalinux.org/blog/impact-of-rhel-changes/ RedHat a annoncé un autre changement massif qui affecte tous les rebuilds et forks de Red Hat Enterprise Linux. À l'avenir, Red Hat publiera uniquement le code source pour les RHEL RPMs derrière leur portail client. Comme tous les clones de RHEL dépendent des sources publiées, cela perturbe encore une fois l'ensemble de l'écosystème Red Hat. Une analyse du choix de red hat sur la distribution du code source de rhel https://dissociatedpress.net/2023/06/24/red-hat-and-the-clone-wars/ Une reponse de red hat aux feux démarrés par l'annonce de la non distribution des sources de RHEL en public https://www.redhat.com/en/blog/red-hats-commitment-open-source-response-gitcentosorg-changes et un lien vers une de ces feux d'une personne proheminente dans la communauté Ansible https://www.jeffgeerling.com/blog/2023/im-done-red-hat-enterprise-linux Oracle demande a garder un Linux ouvert et gratuit https://www.oracle.com/news/announcement/blog/keep-linux-open-and-free-2023-07-10/ Suite à l'annonce d'IBM/RedHat, Oracle demande à garder Linux ouvert et gratuit IBM ne veut pas publier le code de RHEL car elle doit payer ses ingénieurs Alors que RedHat a pu maintenir son modèle économique durante des années L'article revient sur CentOS qu'IBM “a tué” en 2020 Oracle continue ses éfforts de rendre Linux ouvert et libre Oracle Linux continuera à être compatible avec RHEL jusqu'à la version 9.2, après ça sera compliqué de maintenir une comptabilité Oracle embauche des dev Linux Oracle demande à IBM de récupérer le downstream d'Oracle et de le distribuer SUSE forke RHEL https://www.suse.com/news/SUSE-Preserves-Choice-in-Enterprise-Linux/ SUSE est la société derrière Rancher, NeuVector, et SUSE Linux Enterprise (SLE) Annonce un fork de RHEL $10M d'investissement dans le projet sur les prochaines années Compatibilité assurée de RHEL et CentOS Web Google revent sont service de nom de domaine a Squarespace https://www.reddit.com/r/webdev/comments/14agag3/squarespace_acquires_google_domains/ et c'était pas gratuit donc on n'est pas censé etre le produit :wink: Squarespace est une entreprise américaine spécialisée dans la création de site internet Squarespace est un revendeur de Google Workspace depuis longtemps La vente devrait se finaliser en Q3 2023 Petite introduction à WebGPU en français https://blog.octo.com/connaissez-vous-webgpu/ Data Avec la mode des Large Language Models, on parle de plus en plus de bases de données vectorielles, pour stocker des “embeddings” (des vecteurs de nombre flottant représentant sémantiquement du texte, ou même des images). Un article explique que les Vecteurs sont le nouveau JSON dans les bases relationnelles comme PostgreSQL https://jkatz05.com/post/postgres/vectors-json-postgresql/ L'article parle en particulier de l'extension pgVector qui est une extension pour PostgreSQL pour rajouter le support des vectors comme type de colonne https://github.com/pgvector/pgvector Google Cloud annonce justement l'intégration de cette extension vectorielle à CloudSQL pour PostgreSQL et à AlloyDB pour PostgreSQL https://cloud.google.com/blog/products/databases/announcing-vector-support-in-postgresql-services-to-power-ai-enabled-applications Il y a également une vidéo, un notebook Colab, et une article plus détaillé techniquement utilisant LangChain https://cloud.google.com/blog/products/databases/using-pgvector-llms-and-langchain-with-google-cloud-databases Mais on voit aussi également Elastic améliorer Lucene pour utiliser le support des instructions SIMD pour accélérer les calculs vectoriels (produit scalaire, distance euclidienne, similarité cosinus) https://www.elastic.co/fr/blog/accelerating-vector-search-simd-instructions Outillage Le sondage de StackOverflow 2023 https://survey.stackoverflow.co/2023/ L'enquête a été réalisée auprès de 90 000 développeurs dans 185 pays. Les développeurs sont plus nombreux (+2%) que l'an dernier à travailler sur site (16% sur site, 41% remote, 42% hybrid) Les développeurs sont également de plus en plus nombreux à utiliser des outils d'intelligence artificielle, avec 70 % d'entre eux déclarant les utiliser (44%) ou prévoyant de les utiliser (25) dans leur travail. Les langages de programmation les plus populaires sont toujours JavaScript, Python et HTML/CSS. Les frameworks web les plus populaires sont Node, React, JQuery. Les bases de données les plus populaires sont PostgreSQL, MySQL, et SQLite. Les systèmes d'exploitation les plus populaires sont Windows puis macOS et Linux. Les IDE les plus populaires sont Visual Studio Code, Visual Studio et IDEA IntelliJ. Les différents types de déplacement dans Vim https://www.barbarianmeetscoding.com/boost-your-coding-fu-with-vscode-and-vim/moving-blazingly-fast-with-the-core-vim-motions/ JetBrains se mets aussi à la mode des assistants IA dans l'IDE https://blog.jetbrains.com/idea/2023/06/ai-assistant-in-jetbrains-ides/ une intégration avec OpenAI mais aussi de plus petits LLMs spécifiques à JetBrains un chat intégré pour discuter avec l'assistant, puis la possibilité d'intégrer les snippets de code là où se trouve le curseur possibilité de sélectionner du code et de demander à l'assistant d'expliquer ce que ce bout de code fait, mais aussi de suggérer un refactoring, ou de régler les problèmes potentiels on peut demander à générer la JavaDoc d'une méthode, d'une classe, etc, ou à suggérer un nom de méthode (en fonction de son contenu) génération de message de commit il faut avoir un compte JetBrains AI pour y avoir accès Des commandes macOS plus ou moins connues https://saurabhs.org/advanced-macos-commands caffeinate — pour garder le mac éveillé pbcopy / pbpaste — pour interagir avec le clipboard networkQuality — pour mesurer la rapidité de l'accès à internet sips — pour manipuler / redimensionner des images textutil — pour covertir des fichers word, texte, HTML screencapture — pour faire un screenshot say — pour donner une voix à vos commandes Le sondage de la communauté ArgoCD https://blog.argoproj.io/cncf-argo-cd-rollouts-2023-user-survey-results-514aa21c21df Un client d'API open-source et cross-platform pour GraphQL, REST, WebSockets, Server-sent events et gRPC https://github.com/Kong/insomnia Architecture Moderniser l'architecture avec la decouverte via le domain driven discovery https://www.infoq.com/articles/architecture-modernization-domain-driven-discovery/?utm_source=twitter&utm_medium=link&utm_campaign=calendar Un article très détaillé pour moderniser son architecture en utilisant une approche Domain-Driven Discovery qui se fait en 5 étapes: Encadrer le problème – Clarifier le problème que vous résolvez, les personnes touchées, les résultats souhaités et les contraintes de solution. Analyser l'état actuel – Explorer les processus opérationnels et l'architecture des systèmes existants afin d'établir une base de référence pour l'amélioration. Explorer l'état futur – Concevoir une architecture modernisée fondée sur des contextes délimités, établir des priorités stratégiques, évaluer les options et créer des solutions pour l'état futur. Créer une feuille de route – Créer un plan pour moderniser l'architecture au fil du temps en fonction des flux de travail ou des résultats souhaités. Récemment, Sfeir a lancé son blog de développement sur https://www.sfeir.dev/ plein d'articles techniques sur de nombreux thèmes : front, back, cloud, data, AI/ML, mobile aussi des tendances, des success stories par exemple dans les derniers articles : on parle d'Alan Turing, du Local Storage en Javascript, des la préparation de certifications React, l'impact de la cybersécurité sur le cloud Demis Hassabis annonce travailler sur une IA nommée Gemini qui dépassera ChatGPT https://www.wired.com/story/google-deepmind-demis-hassabis-chatgpt/ Demis Hassabis CEO de Google DeepMind créateur de AlphaGOet AlphaFold Travaille sur une IA nommé Gemini qui dépasserait ChatGPT de OpenAI Similair à GPT-4 mais avec des techniques issues de AlphaGO Encore en developpement, va prendre encore plusieurs mois Un remplaçant a Bard? Méthodologies Approcher l'agilité par les traumatismes (de developement) passés des individus https://www.infoq.com/articles/trauma-informed-agile/?utm_campaign=infoq_content&utm_source=twitter&utm_medium=feed&utm_term=culture-methods Nous subissons tous un traumatisme du développement qui rend difficile la collaboration avec d'autres - une partie cruciale du travail dans le développement de logiciels agiles. Diriger d'une manière tenant compte des traumatismes n'est pas pratiquer la psychothérapie non sollicitée, et ne justifie pas les comportements destructeurs sans les aborder. Être plus sensible aux traumatismes dans votre leadership peut aider tout le monde à agir de façon plus mature et plus disponible sur le plan cognitif, surtout dans des situations émotionnellement difficiles. Dans les milieux de travail tenant compte des traumatismes, les gens accordent plus d'attention à leur état physique et émotionnel. Ils s'appuient aussi davantage sur le pouvoir de l'intention, fixent des objectifs d'une manière moins manipulatrice et sont capables d'être empathiques sans s'approprier les problèmes des autres. Loi, société et organisation Mercedes va rajouter de l'intelligence artificielle dans ses voitures https://azure.microsoft.com/en-us/blog/mercedes-benz-enhances-drivers-experience-with-azure-openai-service/ Programme béta test de 3 mois pour le moment Assistance vocale “Hey Mercedes” Permet de discuter avec la voiture pour trouver son chemin, concocter une recette, ou avoir tout simplement des discussions Ils travaillent sur des plugin pour reserver un resto, acheter des tickets de cinéma Free software vs Open Source dans le contexte de l'intelligence artificielle par Sacha Labourey https://medium.com/@sachalabourey/ai-free-software-is-essential-to-save-humanity-86b08c3d4777 on parle beaucoup d'AI et d'open source mais il manque la dimension de controle des utilisateurs finaux Stallman a crée la FSF par peur de la notion d'humain augmenté par des logiciels qui sont controllés par d'autres (implants dans le cerveau etc) d'ou la GPL et sa viralité qui propage la capacité a voir et modifier le conde que l'on fait tourner dans le debat AI, ce n'est pas seulement open source (casser oligopolie) mais aissu le free software qui est en jeu La folie du Cyber Resilience Act (CRA) europeen https://news.apache.org/foundation/entry/save-open-source-the-impending-tragedy-of-the-cyber-resilience-act Au sein de l'UE, la loi sur la cyber-résilience (CRA) fait maintenant son chemin à travers les processus législatifs (et doit faire l'objet d'un vote clé le 19 juillet 2023). Cette loi s'appliquera à un large éventail de logiciels (et de matériel avec logiciel intégré) dans l'UE. L'intention de ce règlement est bonne (et sans doute attendue depuis longtemps) : rendre le logiciel beaucoup plus sûr. Le CRA a une approche binaire: oui/non et considère tout le monde de la même manière Le CRA réglementerait les projets à source ouverte à moins qu'ils n'aient « un modèle de développement entièrement décentralisé ». Mais les modèles OSS sont de complexes mélanges de pur OSS et éditeurs de logiciels les entreprises commerciales et les projets open source devront être beaucoup plus prudents quant à ce que les participants peuvent travailler sur le code, quel financement ils prennent, et quels correctifs ils peuvent accepter. Certaines des obligations sont pratiquement impossibles à respecter, par exemple l'obligation de « livrer un produit sans vulnérabilités exploitables connues ». Le CRA exige la divulgation de vulnérabilités graves non corrigées et exploitées à l'ENISA (une institution de l'UE) dans un délai mesuré en heures, avant qu'elles ne soient corrigées. (complètement opposé aux bonnes pratiques de sécu) Une fois de plus une bonne idée à l'origine mais très mal implémentée qui risque de faire beaucoup de dommages Octave Klaba, avec Miro, son frère, et la Caisse des Dépôts, finalisent la création de Synfonium qui va maintenant racheter 100% de Qwant et 100% fe Shadow. Synfonium est détenue à 75% par Jezby Venture & Deep Code et à 25% par la CDC. https://twitter.com/i/web/status/1673555414938427392 L'un de rôles de Synfonium est de créer la masse critique des utilisateurs et des clients B2C & B2B qui vont pouvoir utiliser tous ces services gratuits et payants Vous y retrouverez le moteur de recherche, les services gratuits, la suite collaborative, le social login, mais aussi les services de nos partenaires tech. Le but est de créer une plateforme dans le Cloud SaaS EU qui respectent nos valeurs et nos lois européennes Yann LeCun : «L'intelligence artificielle va amplifier l'intelligence humaine» https://www.europe1.fr/emissions/linterview-politique-dimitri-pavlenko/yann-lecun-li[…]gence-artificielle-va-amplifier-lintelligence-humaine-4189120 Conférences La liste des conférences provenant de Developers Conferences Agenda/List par Aurélie Vache et contributeurs : 2-3 septembre 2023 : SRE France SummerCamp - Chambéry (France) 6 septembre 2023 : Cloud Alpes - Lyon (France) 8 septembre 2023 : JUG Summer Camp - La Rochelle (France) 14 septembre 2023 : Cloud Sud - Remote / Toulouse (France) 18 septembre 2023 : Agile Tour Montpellier - Montpellier (France) 19-20 septembre 2023 : Agile en Seine - Paris (France) 19 septembre 2023 : Salon de la Data Nantes - Nantes (France) & Online 21-22 septembre 2023 : API Platform Conference - Lille (France) & Online 22 septembre 2023 : Agile Tour Sophia Antipolis - Valbonne (France) 25-26 septembre 2023 : BIG DATA & AI PARIS 2023 - Paris (France) 28-30 septembre 2023 : Paris Web - Paris (France) 2-6 octobre 2023 : Devoxx Belgium - Antwerp (Belgium) 6 octobre 2023 : DevFest Perros-Guirec - Perros-Guirec (France) 10 octobre 2023 : ParisTestConf - Paris (France) 11-13 octobre 2023 : Devoxx Morocco - Agadir (Morocco) 12 octobre 2023 : Cloud Nord - Lille (France) 12-13 octobre 2023 : Volcamp 2023 - Clermont-Ferrand (France) 12-13 octobre 2023 : Forum PHP 2023 - Marne-la-Vallée (France) 19-20 octobre 2023 : DevFest Nantes - Nantes (France) 19-20 octobre 2023 : Agile Tour Rennes - Rennes (France) 26 octobre 2023 : Codeurs en Seine - Rouen (France) 25-27 octobre 2023 : ScalaIO - Paris (France) 26-27 octobre 2023 : Agile Tour Bordeaux - Bordeaux (France) 26-29 octobre 2023 : SoCraTes-FR - Orange (France) 10 novembre 2023 : BDX I/O - Bordeaux (France) 15 novembre 2023 : DevFest Strasbourg - Strasbourg (France) 16 novembre 2023 : DevFest Toulouse - Toulouse (France) 23 novembre 2023 : DevOps D-Day #8 - Marseille (France) 30 novembre 2023 : PrestaShop Developer Conference - Paris (France) 30 novembre 2023 : WHO run the Tech - Rennes (France) 6-7 décembre 2023 : Open Source Experience - Paris (France) 7 décembre 2023 : Agile Tour Aix-Marseille - Gardanne (France) 8 décembre 2023 : DevFest Dijon - Dijon (France) 7-8 décembre 2023 : TechRocks Summit - Paris (France) Nous contacter Pour réagir à cet épisode, venez discuter sur le groupe Google https://groups.google.com/group/lescastcodeurs Contactez-nous via twitter https://twitter.com/lescastcodeurs Faire un crowdcast ou une crowdquestion Soutenez Les Cast Codeurs sur Patreon https://www.patreon.com/LesCastCodeurs Tous les épisodes et toutes les infos sur https://lescastcodeurs.com/

Welcome episode 219 of The Cloud Pod podcast - where the forecast is always cloudy! Today your hosts are Justin and Jonathan, and they discuss all things cloud, including clickstream analytics, databricks, Microsoft Entra, virtual machines, Outlook threats, and some major changes over at the Google Cloud team.  Titles we almost went with this week: TCP is not Entranced with Entra ID The Cave you Fear to Entra, Holds the Treasure you Seek Microsoft should rethink Entra rules for their Email A big thanks to this week's sponsor: Foghorn Consulting, provides top-notch cloud and DevOps engineers to the world's most innovative companies. Initiatives stalled because you have trouble hiring?  Foghorn can be burning down your DevOps and Cloud backlogs as soon as next week.

On this episode, Andrew Atkinson returns for a conversation centering on partitioning in PostgreSQL.  We also discuss his upcoming book High Performance PostgreSQL for Rails, Sin City Ruby and some of the benefits of attending conferences, and "getting reps in." High Performance PostgreSQL for Rails by Andrew Atkinson (coming August 2023)The Rails Way by Obie FernandezpgslicePartitioning Billions of Rows Without DowntimeAndy Atkinson.comAndrew Atkinson on TwitterAndrew Atkinson on GitHubSin City Ruby

After talking to Ryan Berry a little while back about OneStream's journey to Azure, the team catches back up with him to hear about how their architecture will evolve in the future to be more Cloud Native. In addition, they discuss the challenges of rearchitecting a pre-existing product when you cannot afford the have dual stacks running at the same time. Media File: https://azpodcast.blob.core.windows.net/episodes/Episode467.mp3 YouTube: https://www.youtube.com/watch?v=0D4x7jDjs-8 Resources Corporate Performance Management Software | OneStream Software Links to updates mentioned at the top of the show Azure AD is becoming Microsoft Entra ID Public preview: Ingest events from Azure Event Hubs to Azure Monitor Logs General availability: Azure Data Explorer adds support for PostgreSQL, MySQL, and CosmosDB SQL external tables Azure cross-region Load Balancer is now generally available Public preview: Azure Virtual Network encryption

Guy and Eitan discuss several interesting topics and recent experiences that you'd want to hear about! Relevant links: Trigram Wildcard String Search in SQL Server The Story of DBeaver, PostgreSQL, and the silly data engineer SQL Server Error Log Management High VLF Counts Got You Down? Here's How to Keep Them Low and Performance High

The promise of digital mapping is to provide a shared and real-time view of the state of the underlying system. pg_eventserv is a free and open-source component that helps fulfill the promise of real-time event modeling and shared views in PostgreSQL. By connecting to PostgreSQL and listening on specified channels, pg_eventserv captures database notifications and forwards them to web clients, enabling real-time updates and synchronization of data displayed on maps or other web interfaces. pg_eventserv does one thing and one thing only: take events generated by the PostgreSQL NOTIFY command and passes the payload along to waiting WebSockets clients. pg_eventserv is free and easy to install and you can find it here: https://github.com/CrunchyData/pg_eventserv What this means is that any client can watch for notifications and update as changes in the database happen. Real-time data! Here is a link to a Youtube video demonstration of pg_eventserv in action! https://youtu.be/UakRtYmoWow  I will let Paul Ramsey the creator of pg_eventserv explain all this in more detail in this episode.  If you want to reach out to Paul the best place to do that is http://blog.cleverelephant.ca/  Or if you want to listen to previous episodes with Paul you might find these interesting Raster in the database? https://mapscaping.com/podcast/rasters-in-a-database/ Dynamic Vector Tiles Straight From The Database https://mapscaping.com/podcast/dynamic-vector-tiles-straight-from-the-database/ Spatial SQL- GIS Without The GIS https://mapscaping.com/podcast/spatial-sql-gis-without-the-gis/   also ...  If you are interested in spatial databases at scale ... you might find this episode interesting https://mapscaping.com/podcast/distributing-geospatial-data/