Free and open-source relational database management system
Links: Elaastic Throughput for EFS Front any existing MySQL or PostgresQL with a GraphQL API Security Hub Multi-Region and Multi-Account Amazon Detective announces investigations for IAM Stay Up To Date with re:Quinnvent Sign up for the re:Quinnvent Newsletter Join the re:Quinnvent Scavenger Hunt Check out the re:Quinnvent playlist on YouTube Help the show Share your feedback Follow wherever you get your podcasts Buy our merch What's Corey up to? Follow Corey on Twitter (@quinnypig) See our recent work at the Duckbill Group Apply to work with Corey and the Duckbill Group to help lower your AWS bill
In this episode of Startup Hustle, Matt and Lauren talk about Pittsburg's Top Startups! Tune in to hear what's happening in Steel City's startup scene and meet 10 emerging growth companies poised for a disruptive 2024. Find Startup Hustle Everywhere: https://gigb.co/l/YEh5 This episode is sponsored by Full Scale: https://fullscale.io Learn more about InnovateHER KC: https://www.innovateherkc.com Meet all the Top Startups we've featured around the US: https://gigb.co/l/x1fD Episode Highlights: Intro to the Pttisburg Top Startups episode (0:31) Koop Technologies - Insurance for all automation risks (2:58) Mach9 - Automated geospatial production platform (4:53) Mapless AI - One-stop, plug-in solution for remote car operation (7:56) Netail - AI SaaS solutions for the retail and consumer industry (11:03) OtterTune - AI-powered automatic PostgreSQL and MySQL tuning (16:38) Resilient Lifescience - Wearable medical device to prevent opioid overdose deaths (21:31) Stack AV - Autonomous trucking solutions (25:49) Stratus Materials - Next-generation Lithium-Ion Cathode active materials (27:35) Thoro.ai - Providing superior autonomy solutions (31:37) TRAiNED - Harnessing the skills of a Neurodivergent Workforce (36:11) Lauren's favorite Pittsburg top startups: (43:24) Matt's favorite Pittsburg startup: (44:20) AI is here to stay (47:50) Read the feature on Inc.com: https://www.inc.com/inc-masters/pittsburgh-startups-to-watch-in-2024.html See omnystudio.com/listener for privacy information.
Summary: FOSS4GNA 2023 presentation on using open source software at the Census Bureau, focusing on the Geographic Update Partnership Software (GUPS), a GIS solution built on QGIS framework. The presentation covers the benefits of open source for federal government, including security, flexibility, free support, and cost savings. The GUPS system integrates QGIS, PostgreSQL, GeoServer, open layers, and more. A demo of GUPS Web showcases features like map management, user invitation, role-based access, and the boundary review tool for automatic change detection. Highlights:
AWS Morning Brief for the week of November 13, 2023, with Corey Quinn. Show Notes:Links: Amazon Aurora Global Database for PostgreSQL now supports write forwarding Amazon SQS announces support for JSON protocol AWS Cost Management now provides purchase recommendations for Amazon MemoryDB Reserved Nodes Introducing the Generative AI Center of Excellence for AWS Partners: The Path to AI Expertise New – Block Public Sharing of Amazon EBS Snapshots New for Amazon Comprehend – Toxicity Detection AWS CodeBuild adds support for AWS Lambda compute mode An Overview of Bulk Sender Changes at Yahoo/Gmail Creating a correction of errors document Know Before You Go – AWS re:Invent 2023 | AWS Management Console Unhoused individuals gain shelter, prove their identity using AWS-powered solution Kiip How VMware partnered with AWS to nurture a culture of sustainability How to Stop Feeding AWS's AI With Your Data
Jonathan Katz, a principal product manager at Amazon Web Services, discusses the evolution of PostgreSQL in an episode of The New Stack Makers. He notes that PostgreSQL's uses have expanded significantly since its inception and now cover a wide range of applications and workloads. Initially considered niche, it faced competition from both open-source and commercial relational database systems. Katz's involvement in the PostgreSQL community began as an app developer, and he later contributed by organizing events.PostgreSQL originated from academic research at the University of California at Berkeley in the mid-1980s, becoming an open-source project in 1994. In the mid-1990s, proprietary databases like Oracle, IBM DB2, and Microsoft SQL dominated the market, while open-source alternatives like MySQL, MariaDB, and SQLite emerged.PostgreSQL 16 introduces logical replication from standby servers, enhancing scalability by offloading work from the primary server. The meticulous design process within the PostgreSQL community leads to stable and reliable features. Katz mentions the development of Direct I/O as a long-term feature to reduce latency and improve data writing performance, although it will take several years to implement.Amazon Web Services has built Amazon RDS on PostgreSQL to simplify application development for developers. This managed service handles operational tasks such as deployment, backups, and monitoring, allowing developers to focus on their applications. Amazon RDS supports multiple PostgreSQL releases, making it easier for businesses to manage and maintain their databases.Learn more from The New Stack about PostgreSQL and AWS:PostgreSQL 16 Expands Analytics CapabilitiesPowertools for AWS Lambda Grows with Help of VolunteersHow Donating Open Source Code Can Advance Your Career
Summary Databases are the core of most applications, but they are often treated as inscrutable black boxes. When an application is slow, there is a good probability that the database needs some attention. In this episode Lukas Fittl shares some hard-won wisdom about the causes and solution of many performance bottlenecks and the work that he is doing to shine some light on PostgreSQL to make it easier to understand how to keep it running smoothly. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. You specify the customer traits, then Profiles runs the joins and computations for you to create complete customer profiles. Get all of the details and try the new product today at dataengineeringpodcast.com/rudderstack (https://www.dataengineeringpodcast.com/rudderstack) You shouldn't have to throw away the database to build with fast-changing data. You should be able to keep the familiarity of SQL and the proven architecture of cloud warehouses, but swap the decades-old batch computation model for an efficient incremental engine to get complex queries that are always up-to-date. With Materialize, you can! It's the only true SQL streaming database built from the ground up to meet the needs of modern data products. Whether it's real-time dashboarding and analytics, personalization and segmentation or automation and alerting, Materialize gives you the ability to work with fresh, correct, and scalable results — all in a familiar SQL interface. Go to dataengineeringpodcast.com/materialize (https://www.dataengineeringpodcast.com/materialize) today to get 2 weeks free! Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics. Trusted by teams of all sizes, including Comcast and Doordash, Starburst is a data lake analytics platform that delivers the adaptability and flexibility a lakehouse ecosystem promises. And Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data. Want to see Starburst in action? Go to dataengineeringpodcast.com/starburst (https://www.dataengineeringpodcast.com/starburst) and get $500 in credits to try Starburst Galaxy today, the easiest and fastest way to get started using Trino. This episode is brought to you by Datafold – a testing automation platform for data engineers that finds data quality issues before the code and data are deployed to production. Datafold leverages data-diffing to compare production and development environments and column-level lineage to show you the exact impact of every code change on data, metrics, and BI tools, keeping your team productive and stakeholders happy. Datafold integrates with dbt, the modern data stack, and seamlessly plugs in your data CI for team-wide and automated testing. If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold (https://www.dataengineeringpodcast.com/datafold) Your host is Tobias Macey and today I'm interviewing Lukas Fittl about optimizing your database performance and tips for tuning Postgres Interview Introduction How did you get involved in the area of data management? What are the different ways that database performance problems impact the business? What are the most common contributors to performance issues? What are the useful signals that indicate performance challenges in the database? For a given symptom, what are the steps that you recommend for determining the proximate cause? What are the potential negative impacts to be aware of when tuning the configuration of your database? How does the database engine influence the methods used to identify and resolve performance challenges? Most of the database engines that are in common use today have been around for decades. How have the lessons learned from running these systems over the years influenced the ways to think about designing new engines or evolving the ones we have today? What are the most interesting, innovative, or unexpected ways that you have seen to address database performance? What are the most interesting, unexpected, or challenging lessons that you have learned while working on databases? What are your goals for the future of database engines? Contact Info LinkedIn (https://www.linkedin.com/in/lfittl/) @LukasFittl (https://twitter.com/LukasFittl) on Twitter Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. Podcast.__init__ (https://www.pythonpodcast.com) covers the Python language, its community, and the innovative ways it is being used. The Machine Learning Podcast (https://www.themachinelearningpodcast.com) helps you go from idea to production with machine learning. Visit the site (https://www.dataengineeringpodcast.com) to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email email@example.com (mailto:firstname.lastname@example.org)) with your story. To help other people find the show please leave a review on Apple Podcasts (https://podcasts.apple.com/us/podcast/data-engineering-podcast/id1193040557) and tell your friends and co-workers Links PGAnalyze (https://pganalyze.com/) Citus Data (https://www.citusdata.com/) Podcast Episode (https://www.dataengineeringpodcast.com/citus-data-with-ozgun-erdogan-and-craig-kerstiens-episode-13/) ORM == Object Relational Mapper (https://en.wikipedia.org/wiki/Object%E2%80%93relational_mapping) N+1 Query (https://docs.sentry.io/product/issues/issue-details/performance-issues/n-one-queries/) Autovacuum (https://www.postgresql.org/docs/current/routine-vacuuming.html#AUTOVACUUM) Write-ahead Log (https://en.wikipedia.org/wiki/Write-ahead_logging) pgstatio (https://pgpedia.info/p/pg_stat_io.html) randompagecost (https://postgresqlco.nf/doc/en/param/random_page_cost/) pgvector (https://github.com/pgvector/pgvector) Vector Database (https://en.wikipedia.org/wiki/Vector_database) Ottertune (https://ottertune.com/) Podcast Episode (https://www.dataengineeringpodcast.com/ottertune-database-performance-optimization-episode-197/) Citus Extension (https://github.com/citusdata/citus) Hydra (https://github.com/hydradatabase/hydra) Clickhouse (https://clickhouse.tech/) Podcast Episode (https://www.dataengineeringpodcast.com/clickhouse-data-warehouse-episode-88/) MyISAM (https://en.wikipedia.org/wiki/MyISAM) MyRocks (http://myrocks.io/) InnoDB (https://en.wikipedia.org/wiki/InnoDB) Great Expectations (https://greatexpectations.io/) Podcast Episode (https://www.dataengineeringpodcast.com/great-expectations-data-contracts-episode-352) OpenTelemetry (https://opentelemetry.io/) The intro and outro music is from The Hug (http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/Love_death_and_a_drunken_monkey/04_-_The_Hug) by The Freak Fandango Orchestra (http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/) / CC BY-SA (http://creativecommons.org/licenses/by-sa/3.0/)
Timestamps:- 0:00 Intro- 1:48 Releases 428-430- 6:30 Introducing Denis Magda from @YugabyteDB - 7:56 JDBC, Trino's JDBC driver, and the Postgres connector- 14:08 Introducing YugabyteDB- 21:33 Demo time! Trino with PostgreSQL- 29:56 Demoing Trino with YugabyteDB- 44:57 Failover and resiliency- 56:05 Upcoming events and Trino Summit soon!
In this episode, Julie begins by expressing concern about technical issues with Zoom and her camera being off, Andrew discusses the shortage of certain ADHD medication in the U.S. due to supply issues, and then the topic of keeping tidy beds and using chopsticks for eating comes up. The conversation then shifts to Andrew introducing asdf as a runtime version manager and he dives into the complexity of managing different versions of tools. The discussion touches on Docker, version consistency, and Podia's smooth transition to a new database version. They emphasize the importance of planning and backups during database upgrades and share insights into moving local databases. We end with a reflection on microservices and their evolving role in the tech industry. Hit download now to hear more! [00:00:13] Julie expresses concern about Zoom not working and her camera being off, and Andrew shares his thoughts on the shortage of ADHD medication in the U.S and how it's been affecting his life. [00:02:59] The topic of keeping their beds tidy comes up, bed habits, and converting to chopsticks for eating. [00:08:15] Andrew explains that he was helping a designer upgrade his Postgres database from version 13 to 15. He mentions the challenge of not knowing how the designer was running Postgres initially and the decision to use GUI app (Postgres app) for ease of use. [00:11:38] What is asdf? Andrew asks Julie if she uses rbenv or RVM, and he explains that rbenv is a runtime version manager for Ruby and discusses the need for managing multiple versions of Ruby due to the default outdated version in macOS. Also, Andrew explains that it's a tool that manages various services, including Redis, Yarn, Node, Ruby, and more. [00:13:49] Julie brings up if Andrew had any issues with asdf, and he acknowledges some past issues, but has a strong preference for using asdf consistently.[00:14:12] Julie asks how Andrew's use of asdf at work compares to others who might not use it, and he relates the discussion to Docker, explaining that Docker containers can help standardize environments and avoid issues related to different versions of tools and services on different machines. [00:16:48] Julie asks whether applications can still run smoothly even if developers are using different version of tools like Redis. Andrew confirms it's possible but explains the potential pitfalls and the importance of version consistency. [00:20:00] Andrew discusses the switch from database version 13 to 15 in production and mentions the challenges of upgrading databases compared to simpler updates like changing a Node.js version. [00:20:43] Julie asks about Podia's staging environment for testing changes before deploying to production, and Andrew mentions using Heroku pipelines and how the company relies on feature flags and aims to keep the main branch always deployable.[00:22:05] Andrew explains the potential for breaking changes when upgrading database versions, mentioning the PostgreSQL 16 is coming out and Heroku plans to support version 14 through 16 while dropping support for version 13. [00:23:14] Andrew tells us their backup plan in case the database upgrade to version 15 fails and discusses the importance of having backups and a fast recovery plan. [00:25:43] Julie asks Andrew what he meant by SSH into a box and about moving a local database to another computer, and he explains that it's possible using commands like PG dump and restore, as well as providing options for moving a local database. [00:29:55] Why doesn't Podia use Docker? Andrew explains, citing previous experiences with Docker and the complexity it can introduce.[00:32:22] Julie wonders if Andrew uses microservices, to which he explains that when he began his career, microservices were popular, but the industry's perspective has since shifted. Docker was initially associated with microservices, but the trend has changed over time.[00:33:01] What are the benefits of using microservices and why have some companies moved away from them? Andrew tells us the complexity of microservices became more apparent as they were adopted more widely, and some now consider them an anti-pattern. Panelists:Andrew MasonJulie J.Sponsor:GoRailsLinks:Andrew Mason TwitterAndrew Mason WebsiteJulie J. TwitterJulie J. WebsitePostgres.apprbenvRuby Version Manager (RVM)asdfPostgreSQL 16
How do you choose an Azure data storage solution? Richard talks to Nicole Stevens about her experiences helping companies move data into the cloud - typically in SQL Server. The obvious answer is Azure SQL, but that doesn't always make it the best solution! Nicole talks about a customer moving to Cosmos DB for the unstructured data capabilities and a lot of speed. But does the price make sense? All these factors are in play in choosing a data storage solution, and there is never one right choice - often, a mix of services makes the most sense!Links:Azure SQLAzure Cosmos DBAPIs in Cosmos DBAzure Table StorageAzure Queue StorageAzure Messaging Servicesdapr - Distributed Application RuntimeMicrosoft CertificationsData Architecture GuideData Landing Zones Azure Security BaselinesRecorded September 6, 2023
The Elixir community has a new OpenSource CMS thanks to DockYard and Leandro Pereira! We talk with Leandro to better understand what the Beacon project is and what it can do. It's built using Phoenix LiveView and can be deployed standalone or as part of an existing Elixir Phoenix application. The Admin features include the ability for non-developers to edit content and immediately deploy changes without re-deploying the app. We talk about how BeaconCMS is positioned against Wordpress and static sites and where the greatest benefits are. A feature in development is the HEEx template editor which promises to be a very exciting tool for non-developers. Learn along with us! Show Notes online - http://podcast.thinkingelixir.com/174 (http://podcast.thinkingelixir.com/174) Elixir Community News - https://twitter.com/chris_mccord/status/1713894354962534808 (https://twitter.com/chris_mccord/status/1713894354962534808?utm_source=thinkingelixir&utm_medium=shownotes) – LiveView optimized DOM patching announced, resulting in significant speed improvements in browser's DOM patching time. - https://twitter.com/basilenouvellet/status/1713981828028133847 (https://twitter.com/basilenouvellet/status/1713981828028133847?utm_source=thinkingelixir&utm_medium=shownotes) – Real world test results demonstrating faster speeds due to LiveView's optimized DOM patching. - https://dashbit.co/blog/latency-rendering-liveview (https://dashbit.co/blog/latency-rendering-liveview?utm_source=thinkingelixir&utm_medium=shownotes) – New post on performance optimization in Phoenix LiveView by José Valim. - https://youtu.be/Ckgl9KO4E4M?si=UNf5sNShzl1oTZQS&t=1731 (https://youtu.be/Ckgl9KO4E4M?si=UNf5sNShzl1oTZQS&t=1731?utm_source=thinkingelixir&utm_medium=shownotes) – Chris McCord's ElixirConf keynote demo on LiveView's new dev tool features. Time signature for dev tools example - https://twitter.com/josevalim/status/1711756969814426066 (https://twitter.com/josevalim/status/1711756969814426066?utm_source=thinkingelixir&utm_medium=shownotes) – José Valim's explanation on LiveView's unreleased feature. - https://github.com/elixir-saas/clicktocomponent (https://github.com/elixir-saas/click_to_component?utm_source=thinkingelixir&utm_medium=shownotes) – An external library called "clicktocomponent" to support the Cmd+Click functionality in LiveView. - https://news.livebook.dev/remote-execution-smart-cell---launch-week-2---day-1-m3dv2 (https://news.livebook.dev/remote-execution-smart-cell---launch-week-2---day-1-m3dv2?utm_source=thinkingelixir&utm_medium=shownotes) – Announcement of Livebook's new feature - Remote execution Smart cell. - https://news.livebook.dev/speech-to-text-with-whisper-timestamping-streaming-and-parallelism-oh-my---launch-week-2---day-2-36osSY (https://news.livebook.dev/speech-to-text-with-whisper-timestamping-streaming-and-parallelism-oh-my---launch-week-2---day-2-36osSY?utm_source=thinkingelixir&utm_medium=shownotes) – Livebook's new features in their Whisper integration for improved speech-to-text performance. - https://news.livebook.dev/introducing-file-integration---launch-week-2---day-3-2HoFfa (https://news.livebook.dev/introducing-file-integration---launch-week-2---day-3-2HoFfa?utm_source=thinkingelixir&utm_medium=shownotes) – Livebook's introduction to File Integration. - https://news.livebook.dev/integration-with-snowflake-and-microsoft-sql-server---launch-week-2---day-4-2o4z9C (https://news.livebook.dev/integration-with-snowflake-and-microsoft-sql-server---launch-week-2---day-4-2o4z9C?utm_source=thinkingelixir&utm_medium=shownotes) – Livebook adds support for Snowflake and SQL Server. - https://news.livebook.dev/vim-and-emacs-key-bindings---launch-week-2---day- (https://news.livebook.dev/vim-and-emacs-key-bindings---launch-week-2---day-?utm_source=thinkingelixir&utm_medium=shownotes) – Livebook support for VIM & Emacs key bindings. - https://hexdocs.pm/ash_sqlite/get-started-with-sqlite.html (https://hexdocs.pm/ash_sqlite/get-started-with-sqlite.html?utm_source=thinkingelixir&utm_medium=shownotes) – Ash Framework's new AshSqlite library release. - https://oban.pro/releases/web/v2.10 (https://oban.pro/releases/web/v2.10?utm_source=thinkingelixir&utm_medium=shownotes) – Release of Oban Web 2.10.0. - https://github.com/emmanueltouzery/elixir-extras.nvim (https://github.com/emmanueltouzery/elixir-extras.nvim?utm_source=thinkingelixir&utm_medium=shownotes) – Release of a new Neovim Elixir plugin called elixir-extras.nvim. - https://twitter.com/samokhvalov/status/1714153676212949355 (https://twitter.com/samokhvalov/status/1714153676212949355?utm_source=thinkingelixir&utm_medium=shownotes) – PostgreSQL tip for setting a human-readable label for a connection session. - https://github.com/pgbouncer/pgbouncer/releases/tag/pgbouncer121_0 (https://github.com/pgbouncer/pgbouncer/releases/tag/pgbouncer_1_21_0?utm_source=thinkingelixir&utm_medium=shownotes) – Upcoming PgBouncer 1.21 release with support for prepared statements. - https://twitter.com/ElixirConfEU/status/1713929804062273663 (https://twitter.com/ElixirConfEU/status/1713929804062273663?utm_source=thinkingelixir&utm_medium=shownotes) – Announcement for call for talks for ElixirConf Europe 2024. - https://www.elixirconf.eu/ (https://www.elixirconf.eu/?utm_source=thinkingelixir&utm_medium=shownotes) – ElixirConf Europe 2024 conference details. Do you have some Elixir news to share? Tell us at @ThinkingElixir (https://twitter.com/ThinkingElixir) or email at email@example.com (mailto:firstname.lastname@example.org) Discussion Resources - https://github.com/BeaconCMS/beacon (https://github.com/BeaconCMS/beacon?utm_source=thinkingelixir&utm_medium=shownotes) – The main Beacon CMS project - https://github.com/BeaconCMS/beacon_demo (https://github.com/BeaconCMS/beacon_demo?utm_source=thinkingelixir&utm_medium=shownotes) – Beacon CMS demo project - https://github.com/BeaconCMS/beaconliveadmin (https://github.com/BeaconCMS/beacon_live_admin?utm_source=thinkingelixir&utm_medium=shownotes) – Beacon CMS admin project - https://beaconcms.org/ (https://beaconcms.org/?utm_source=thinkingelixir&utm_medium=shownotes) - https://www.youtube.com/watch?v=5jk0fIJOFuc&list=PLqj39LCvnOWbHaZldxw_g02RaTQ4vQ1eY&index=16 (https://www.youtube.com/watch?v=5jk0fIJOFuc&list=PLqj39LCvnOWbHaZldxw_g02RaTQ4vQ1eY&index=16?utm_source=thinkingelixir&utm_medium=shownotes) – ElixirConf 2023 - Beacon - The next generation of CMS in Phoenix LiveView - https://mdxjs.com/ (https://mdxjs.com/?utm_source=thinkingelixir&utm_medium=shownotes) - https://hex.pm/packages/earmark (https://hex.pm/packages/earmark?utm_source=thinkingelixir&utm_medium=shownotes) - https://github.com/leandrocp/mdex (https://github.com/leandrocp/mdex?utm_source=thinkingelixir&utm_medium=shownotes) - https://github.com/BeaconCMS/livemonacoeditor (https://github.com/BeaconCMS/live_monaco_editor?utm_source=thinkingelixir&utm_medium=shownotes) – The code editor, based on the Livebook editor - https://github.com/leandrocp/mdex (https://github.com/leandrocp/mdex?utm_source=thinkingelixir&utm_medium=shownotes) – Markdown parser and formatter - https://github.com/leandrocp/autumn (https://github.com/leandrocp/autumn?utm_source=thinkingelixir&utm_medium=shownotes) – Syntax highlighter for code blocks - https://github.com/TheFirstAvenger/safe_code (https://github.com/TheFirstAvenger/safe_code?utm_source=thinkingelixir&utm_medium=shownotes) – Inspect HEEx for unsafe code - https://github.com/BeaconCMS/beacon/milestone/1 (https://github.com/BeaconCMS/beacon/milestone/1?utm_source=thinkingelixir&utm_medium=shownotes) – Beacon v0.1 release milestone - https://github.com/BeaconCMS/beaconliveadmin/milestone/1 (https://github.com/BeaconCMS/beacon_live_admin/milestone/1?utm_source=thinkingelixir&utm_medium=shownotes) Guest Information - https://twitter.com/leandrocesquini (https://twitter.com/leandrocesquini?utm_source=thinkingelixir&utm_medium=shownotes) – on Twitter - https://github.com/leandrocp/ (https://github.com/leandrocp/?utm_source=thinkingelixir&utm_medium=shownotes) – on Github - hhttps://leandrocp.com.br (hhttps://leandrocp.com.br?utm_source=thinkingelixir&utm_medium=shownotes) – Blog - https://github.com/leandrocp/mdex (https://github.com/leandrocp/mdex?utm_source=thinkingelixir&utm_medium=shownotes) – Other project MDEx - A fast 100% CommonMark-compatible GitHub Flavored Markdown parser and formatter for Elixir. Find us online - Message the show - @ThinkingElixir (https://twitter.com/ThinkingElixir) - Message the show on Fediverse - @ThinkingElixir@genserver.social (https://genserver.social/ThinkingElixir) - Email the show - email@example.com (mailto:firstname.lastname@example.org) - Mark Ericksen - @brainlid (https://twitter.com/brainlid) - Mark Ericksen on Fediverse - @email@example.com (https://genserver.social/brainlid) - David Bernheisel - @bernheisel (https://twitter.com/bernheisel) - David Bernheisel on Fediverse - @firstname.lastname@example.org (https://genserver.social/dbern) - Cade Ward - @cadebward (https://twitter.com/cadebward) - Cade Ward on Fediverse - @email@example.com (https://genserver.social/cadebward)
Summary Building streaming applications has gotten substantially easier over the past several years. Despite this, it is still operationally challenging to deploy and maintain your own stream processing infrastructure. Decodable was built with a mission of eliminating all of the painful aspects of developing and deploying stream processing systems for engineering teams. In this episode Eric Sammer discusses why more companies are including real-time capabilities in their products and the ways that Decodable makes it faster and easier. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. You specify the customer traits, then Profiles runs the joins and computations for you to create complete customer profiles. Get all of the details and try the new product today at dataengineeringpodcast.com/rudderstack (https://www.dataengineeringpodcast.com/rudderstack) This episode is brought to you by Datafold – a testing automation platform for data engineers that finds data quality issues before the code and data are deployed to production. Datafold leverages data-diffing to compare production and development environments and column-level lineage to show you the exact impact of every code change on data, metrics, and BI tools, keeping your team productive and stakeholders happy. Datafold integrates with dbt, the modern data stack, and seamlessly plugs in your data CI for team-wide and automated testing. If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold (https://www.dataengineeringpodcast.com/datafold) You shouldn't have to throw away the database to build with fast-changing data. You should be able to keep the familiarity of SQL and the proven architecture of cloud warehouses, but swap the decades-old batch computation model for an efficient incremental engine to get complex queries that are always up-to-date. With Materialize, you can! It's the only true SQL streaming database built from the ground up to meet the needs of modern data products. Whether it's real-time dashboarding and analytics, personalization and segmentation or automation and alerting, Materialize gives you the ability to work with fresh, correct, and scalable results — all in a familiar SQL interface. Go to dataengineeringpodcast.com/materialize (https://www.dataengineeringpodcast.com/materialize) today to get 2 weeks free! As more people start using AI for projects, two things are clear: It's a rapidly advancing field, but it's tough to navigate. How can you get the best results for your use case? Instead of being subjected to a bunch of buzzword bingo, hear directly from pioneers in the developer and data science space on how they use graph tech to build AI-powered apps. . Attend the dev and ML talks at NODES 2023, a free online conference on October 26 featuring some of the brightest minds in tech. Check out the agenda and register today at Neo4j.com/NODES (https://Neo4j.com/NODES). Your host is Tobias Macey and today I'm interviewing Eric Sammer about starting your stream processing journey with Decodable Interview Introduction How did you get involved in the area of data management? Can you describe what Decodable is and the story behind it? What are the notable changes to the Decodable platform since we last spoke? (October 2021) What are the industry shifts that have influenced the product direction? What are the problems that customers are trying to solve when they come to Decodable? When you launched your focus was on SQL transformations of streaming data. What was the process for adding full Java support in addition to SQL? What are the developer experience challenges that are particular to working with streaming data? How have you worked to address that in the Decodable platform and interfaces? As you evolve the technical and product direction, what is your heuristic for balancing the unification of interfaces and system integration against the ability to swap different components or interfaces as new technologies are introduced? What are the most interesting, innovative, or unexpected ways that you have seen Decodable used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Decodable? When is Decodable the wrong choice? What do you have planned for the future of Decodable? Contact Info esammer (https://github.com/esammer) on GitHub LinkedIn (https://www.linkedin.com/in/esammer/) Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. Podcast.__init__ (https://www.pythonpodcast.com) covers the Python language, its community, and the innovative ways it is being used. The Machine Learning Podcast (https://www.themachinelearningpodcast.com) helps you go from idea to production with machine learning. Visit the site (https://www.dataengineeringpodcast.com) to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email firstname.lastname@example.org (mailto:email@example.com)) with your story. To help other people find the show please leave a review on Apple Podcasts (https://podcasts.apple.com/us/podcast/data-engineering-podcast/id1193040557) and tell your friends and co-workers Links Decodable (https://www.decodable.co/) Podcast Episode (https://www.dataengineeringpodcast.com/decodable-streaming-data-pipelines-sql-episode-233/) Flink (https://flink.apache.org/) Podcast Episode (https://www.dataengineeringpodcast.com/apache-flink-with-fabian-hueske-episode-57/) Debezium (https://debezium.io/) Podcast Episode (https://www.dataengineeringpodcast.com/debezium-change-data-capture-episode-114/) Kafka (https://kafka.apache.org/) Redpanda (https://redpanda.com/) Podcast Episode (https://www.dataengineeringpodcast.com/vectorized-red-panda-streaming-data-episode-152/) Kinesis (https://aws.amazon.com/kinesis/) PostgreSQL (https://www.postgresql.org/) Podcast Episode (https://www.dataengineeringpodcast.com/postgresql-with-jonathan-katz-episode-42/) Snowflake (https://www.snowflake.com/en/) Podcast Episode (https://www.dataengineeringpodcast.com/snowflakedb-cloud-data-warehouse-episode-110/) Databricks (https://www.databricks.com/) Startree (https://startree.ai/) Pinot (https://pinot.apache.org/) Podcast Episode (https://www.dataengineeringpodcast.com/pinot-embedded-analytics-episode-273/) Rockset (https://rockset.com/) Podcast Episode (https://www.dataengineeringpodcast.com/rockset-serverless-analytics-episode-101/) Druid (https://druid.apache.org/) InfluxDB (https://www.influxdata.com/) Samza (https://samza.apache.org/) Storm (https://storm.apache.org/) Pulsar (https://pulsar.apache.org/) Podcast Episode (https://www.dataengineeringpodcast.com/pulsar-fast-and-scalable-messaging-with-rajan-dhabalia-and-matteo-merli-episode-17) ksqlDB (https://ksqldb.io/) Podcast Episode (https://www.dataengineeringpodcast.com/ksqldb-kafka-stream-processing-episode-122/) dbt (https://www.getdbt.com/) GitHub Actions (https://github.com/features/actions) Airbyte (https://airbyte.com/) Singer (https://www.singer.io/) Splunk (https://www.splunk.com/) Outbox Pattern (https://debezium.io/blog/2019/02/19/reliable-microservices-data-exchange-with-the-outbox-pattern/) The intro and outro music is from The Hug (http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/Love_death_and_a_drunken_monkey/04_-_The_Hug) by The Freak Fandango Orchestra (http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/) / CC BY-SA (http://creativecommons.org/licenses/by-sa/3.0/)
Topics covered in this episode: Psycopg 3 dacite RIP: Fast, barebones pip implementation in Rust Flaky Tests follow up Extras Joke Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training The Complete pytest Course Patreon Supporters Connect with the hosts Michael: @firstname.lastname@example.org Brian: @email@example.com Show: @firstname.lastname@example.org Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Brian #1: Psycopg 3 Psycopg folks recommend starting with 3 for new projects 2 is still actively maintained, but no new features are planned recommend staying with 2 for legacy projects Psycopg 3 project 2 vs 3 feature comparison A few Psycopg 3 highlights native asyncio support native support for more Python types (such as Enums) and PostgreSQL types (such as multirange) Default server-side parameters binding Allows binary parameters and query results (and text, of course) Pipeline/batch mode support Static typing support Michael #2: dacite via Raymond Peck Simple creation of data classes from dictionaries Dacite supports following features: nested structures (basic) types checking optional fields (i.e. typing.Optional) unions forward references collections custom type hooks It's important to mention that dacite is not a data validation library. Type hooks are interesting too. Brian #3: RIP: Fast, barebones pip implementation in Rust list of current and planned features of RIP, the biggest are listed below: Downloading and aggressive caching of PyPI metadata. (done) Resolving of PyPI packages using Resolvo. (done) Installation of wheel files (planned) Support sdist files (planned) new project, just a couple weeks old. … “We would love to have you contribute!” Michael #4: Flaky Tests follow up by Marwan Sarieddine I was inspired by the Talk Python podcast on "Taming flaky tests" with Gregory Kapfhammer and Owain Parry so I wrote up an article on my blog titled "How not to footgun yourself when writing tests - a showcase of flaky tests” Extras Brian: Just wrapping up some personal projects, which means… Python People episodes soon Python Test episodes soon (but later) More course chapters coming Michael: PyBay 2023 was fun Switched to Spark Mail, recommended Dust (what science fiction story telling should be), try: FTL Oceanus Joke: There are more hydrogen atoms in a single molecule of water than there are stars in the entire Solar System. - mas.to/@SmudgeTheInsultCat/111174610011921264 The Big Rewrite
In this episode, Gayatri Kalyanaraman is in conversation with Saikat Chakrabarty who is a Technologist, Director, Engineering at Mihup and he's a AI/ML and Computer Vision Enthusiast.In his career spanning over a decade, he has been a Telecom engineer, prompt engineering, Interaction designer and conversational AI. Saikat is working in the pioneering world of building AI tools that map the man machine and the personHe's working in the contact center engineering by understanding the questions and asks from our customersSaikat started his career in creating software for LTE within 4G Telecom providers and says that he has not looked back from thereSaikat's team in Mihup creates the analysis of the several thousand conversations between the customer and contact center and understand the sentiment of the userListen to the conversation on how the vernacular language support is being created in the AI platform to build accuracyUnderstand that initial days you have to choose the right place to work and always stay invested in oneselfSaikat also talks about his mantra of “Don't choose an easy path and continue to learn”Saikat talks about his interests in Medical and Computer science and Brain networks are working. Saikat believes that the future bet is lying on medical access across the globeSaikat says that software changes in months not years - you have to keep yourself updated daily and that's the only way to stay relevant He's also deep believer of the growth of the technology in Tier 3 cities within India given the organizations changing to remote first or HybridSaikat has been instrumental in Mihup's success as one of the leading Conversational AI companies in India. He leads the entire tech organisation at Mihup, with a team of almost 20 people working under him, ranging from Devops to MLEs, overseeing the end to end development and shipping of the Mihup products.Saikat Chakrabarty is a dynamic leader and Senior Engineering Manager at Mihup, a pioneering conversational AI platform founded in 2016. Mihup's revolutionary approach empowers individuals to seamlessly interact with the digital world while prioritizing inclusivity and accessibility, regardless of language, accent, or dialect.Technical Leadership:At Mihup, Saikat has successfully managed and mentored cross-functional teams in machine learning, R&D, product development and DevOps. His expertise in managing the development of Mihup's conversational AI products has positioned him as a trusted leader in the industry. With a deep understanding of a wide range of technologies, including Java (Spring Boot), NodeJs, Golang, Angular, Python, RabbitMQ, PostgreSQL, Redis, Kubernetes, and more, Saikat has played a pivotal role in building highly scalable cloud microservices and optimizing premise-based applications.Diverse Experience:Saikat's wealth of experience extends beyond his role at Mihup. He has previously worked with renowned companies like Lexmark and Polaris Network Inc., where he honed his skills and passion for AI/ML and computer vision. His contributions to these cutting-edge fields have resulted in 2 patents and 6 paper publications, consolidating his exceptional knowledge and expertise.Recognition and Awards:Saikat's exceptional contributions and dedication to his craft have earned him notable accolades throughout his career. Notably, he was awarded the prestigious Digital Transformation Award at Lexmark Focus-2-Future, alongside the Manager Appreciation Award and several other commendations. At Mihup, he has regularly been the recipient of special rewards for his undying passion and dedication towards delivering products that customers love to use.Saikat can be contacted at Saikat Chakrabarty:
This week, we discuss why everyone is envious of Google's Internal Dev Tools, examine the state of Git, speculate about how 37 Signals plans to reinvent software licensing with ONCE, and share a few thoughts on the Salesforce CEO's recent comments about work from home. Watch the YouTube Live Recording of Episode (https://www.youtube.com/watch?v=AaX-PgF86bY) 433 (https://www.youtube.com/watch?v=AaX-PgF86bY) Runner-up Titles Lost in an acquisition hole. Headless Robot Dog. It's not better enough. GoogHub Why are you on the sad path Once version 2 is a paid upgrade You win interesting bingo Rundown The Full Circle on Developer Productivity with Steve Yegge (https://newsletter.pragmaticengineer.com/p/steve-yegge) Git is awful. GitHub isn't good enough. It's killing us! (Steve Yegge) (https://www.youtube.com/watch?v=EReooAZoMO0) Introducing ONCE (https://once.com/) Salesforce CEO takes a bold stand on remote work (https://www.thestreet.com/investing/salesforce-ceo-bold-stand-on-remote-work) Salesforce to Hire 3,300 People After Layoffs Earlier This Year (https://www.bloomberg.com/news/articles/2023-09-14/salesforce-to-hire-3-300-in-sales-engineering-data-after-earlier-job-cuts#xj4y7vzkg) Relevant to your Interests David Sacks has a new SaaS startup for other SaaS startups (https://www.axios.com/2023/09/06/david-sacks-has-a-new-saas-startup-for-other-saas-startups) Results of Major Technical Investigations for Storm-0558 Key Acquisition (https://msrc.microsoft.com/blog/2023/09/results-of-major-technical-investigations-for-storm-0558-key-acquisition/) Now it's PostgreSQL's turn to have a bogus CVE (https://opensourcewatch.beehiiv.com/p/now-postgresqls-turn-bogus-cve) HashiCorp Retools Licenses And Software To Grow Its Business - The Next Platform (https://www.nextplatform.com/2023/09/05/hashicorp-retools-licenses-and-software-to-grow-its-business/) Clouded Judgement 9.8.23 (https://cloudedjudgement.substack.com/p/clouded-judgement-9823?utm_source=post-email-title&publication_id=56878&post_id=136822157&isFreemail=true&r=2l9&utm_medium=email) Inside Hollywood's SBF Mad Scramble (https://theankler.com/p/inside-hollywoods-sbf-mad-scramble-c04?utm_source=newsletter&utm_medium=email&utm_campaign=newsletter_axiosprorata&stream=top) Tubi The Free Streaming Service, Hits 74 Million Monthly Active Users & Almost 250 Free Live Channels As Cord Cutting Grows | Cord Cutters News (https://cordcuttersnews.com/tubi-the-free-streaming-service-hits-74-million-monthly-active-users-almost-250-free-live-channels-as-cord-cutting-grows/) IBM Software mandates return to office for those within 80km (https://www.theregister.com/2023/09/11/ibm_software_tells_workers_to/) Cloud is here to stay, but at what cost, ask customers (https://www.theregister.com/2023/09/11/cloud_costs_feature/) Disney and Charter reach deal to end cable blackout in time for 'Monday Night Football' (https://www.cnbc.com/2023/09/11/disney-charter-near-carriage-deal-that-would-end-cable-blackout-sources-say.html) Microsoft to kill off third-party printer drivers in Windows (https://www.theregister.com/2023/09/11/go_native_or_go_home/) Oracle revenue misses estimates as tough economy hurts cloud spending (https://www.reuters.com/technology/oracle-reports-quarterly-revenue-narrowly-below-estimates-2023-09-11/) No privacy in cars (https://foundation.mozilla.org/en/privacynotincluded/articles/its-official-cars-are-the-worst-product-category-we-have-ever-reviewed-for-privacy/) Former CEO of China's Alibaba quits cloud business in surprise move during its leadership reshuffle (https://abcnews.go.com/Business/wireStory/former-ceo-chinas-alibaba-quits-cloud-business-surprise-103078368) A Look Back at Q2 '23 Public Cloud Software Earnings (https://cloudedjudgement.substack.com/p/a-look-back-at-q2-23-public-cloud?utm_source=post-email-title&publication_id=56878&post_id=136950716&utm_campaign=email-post-title&isFreemail=true&r=2l9&utm_medium=email) 1 big thing: A long-term plan to secure open-source software (https://www.axios.com/newsletters/axios-codebook-8200e5c5-aed7-4f42-a40e-117a390b57e3.html?chunk=0&utm_term=emshare#story0) MGM takes systems offline after cyberattack (https://www.axios.com/newsletters/axios-codebook-8200e5c5-aed7-4f42-a40e-117a390b57e3.html?chunk=1&utm_term=emshare#story1) Disney-Charter deal represents new era for TV bundles (https://www.axios.com/newsletters/axios-media-trends-fe1295c8-9b83-4403-bae2-06de14fede11.html?chunk=2&utm_term=emshare#story2) Salesforce introduces Einstein Copilot Studio to help customers customize their AI | TechCrunch (https://techcrunch.com/2023/09/12/salesforce-introduces-einstein-copilot-studio-to-customers-customize-their-ai/) Arm prices IPO at $51 per share, valuing company at over $54 billion (https://www.cnbc.com/2023/09/13/arm-prices-ipo-at-51-per-share.html) Tim Gurner's spray about ‘arrogant' workers lays bare the economic sadism of our time (https://www.theguardian.com/commentisfree/2023/sep/14/tim-gurner-ceo-comments-more-unemployment-millionaire-property-developer-workers-neoliberals) Cisco discontinues Hyperflex hyperconverged infrastructure (https://www.theregister.com/2023/09/14/cisco_discontinues_hyperflex_hci/) CloudBees Announces New Cloud Native DevSecOps Platform (https://www.cloudbees.com/newsroom/cloudbees-announces-new-cloud-native-devsecops-platform) Jet: Prepare For Liftoff (https://www.jetporch.com/) Artifact's new Links feature makes it much more than a news app (https://www.theverge.com/2023/9/13/23871561/artifact-links-news-reading-app-tiktok) TriggerMesh, RIP (https://triggermesh-community.slack.com/archives/C02GHUAQDCH/p1695048539668859) Clorox says last month's cyberattack is still disrupting production (https://www.cnbc.com/2023/09/18/clorox-says-last-months-cyberattack-is-still-disrupting-production.html) Excel clone built for Uber China exposed Microsoft mistake (https://www.theregister.com/2023/09/19/matt_uber_china_excel_clone/) Seattle startup MotherDuck raises $52.5M at a $400M valuation to fuel DuckDB analytics platform (https://www.geekwire.com/2023/seattle-startup-motherduck-raises-52-5m-at-a-400m-valuation-to-fuel-duckdb-analytics-platform/) Google's Bard chatbot can now find answers in your Gmail, Docs, Drive (https://www.theverge.com/2023/9/19/23878999/google-bard-ai-chatbot-gmail-docs-drive-extensions) Elon Musk says X may go behind a paywall for everyone so he can 'combat vast armies of bots' (https://www.businessinsider.com/elon-musk-x-twitter-paywall-for-everyone-2023-9) Restricted Source Licensing Is Here (https://www.forrester.com/blogs/restricted-source-licensing-is-here/) OpenTofu (https://opentofu.org/) RoboFab is ready to build 10,000 humanoid robots per year | TechCrunch (https://techcrunch.com/2023/09/18/the-robots-are-coming/) Unified Acceleration Foundation Forms to Drive Open Accelerated Compute and Cross-Platform Performance (https://www.linuxfoundation.org/press/announcing-unified-acceleration-foundation-uxl) Google gets its way, bakes a user-tracking ad platform directly into Chrome (https://arstechnica.com/gadgets/2023/09/googles-widely-opposed-ad-platform-the-privacy-sandbox-launches-in-chrome/) What is a service mesh? Why do you need a service mesh? And which is the best service mesh? (https://newsletter.cote.io/p/what-is-a-service-mesh-why-do-you) Did I Make a Mistake Selling My Social-Media Darling to Yahoo? (https://nymag.com/intelligencer/2018/10/did-i-make-a-mistake-selling-del-icio-us-to-yahoo.html?utm_source=substack&utm_medium=email) A new way of thinking about open source sustainability (https://www.infoworld.com/article/3706508/a-new-way-of-thinking-about-open-source-sustainability.html) Elon Musk moving servers himself shows his 'maniacal sense of urgency' at X, formerly Twitter (https://www.cnbc.com/2023/09/11/elon-musk-moved-twitter-servers-himself-in-the-night-new-biography-details-his-maniacal-sense-of-urgency.html) Cable TV Is on Life Support, but a New Bundle Is Coming Alive (https://www.nytimes.com/2023/09/14/business/media/cable-tv-bundle-streaming.html) Nonsense McDonald's is getting rid of self-serve soda machines | CNN Business (https://www.cnn.com/2023/09/12/business/mcdonalds-self-serve-soda-machines/index.html) Delta SkyMiles changes: Delta overhauls how you earn Medallion status in biggest change yet (https://thepointsguy.com/news/delta-skymiles-changes/) Australian baby named Methamphetamine Rules (https://www.1news.co.nz/2023/09/20/australian-baby-named-methamphetamine-rules/) ‘Take the Money and Run' Artist Must Repay Danish Museum (https://www.nytimes.com/2023/09/19/arts/design/jens-haaning-take-the-money-and-run.html?smid=nytcore-ios-share&referringSource=articleShare) Listener Feedback Jan recommends this Rich Roll interview: Mindset SECRETS From The World's Best Ultrarunner: Courtney Dauwalter (https://www.youtube.com/watch?v=WOtSvYSnzNk) Conferences October 6, 2023, KCD Texas 2023 (https://community.cncf.io/events/details/cncf-kcd-texas-presents-kcd-texas-2023/), CFP Closes: August 30, 2023 November 6-9, 2023, KubeCon NA (https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/), SDT's a sponsor, Matt's there November 6-9, 2023 VMware Explore Barcelona (https://www.vmware.com/explore/eu.html), Coté's attending Jan 29, 2024 to Feb 1, 2024 That Conference Texas (https://that.us/events/tx/2024/schedule/) If you want your conference mentioned, let's talk media sponsorships. SDT news & hype Join us in Slack (http://www.softwaredefinedtalk.com/slack). Get a SDT Sticker! Send your postal address to email@example.com (mailto:firstname.lastname@example.org) and we will send you free laptop stickers! Follow us: Twitch (https://www.twitch.tv/sdtpodcast), Twitter (https://twitter.com/softwaredeftalk), Instagram (https://www.instagram.com/softwaredefinedtalk/), Mastodon (https://hachyderm.io/@softwaredefinedtalk), BlueSky (https://bsky.app/profile/softwaredefinedtalk.com), LinkedIn (https://www.linkedin.com/company/software-defined-talk/), TikTok (https://www.tiktok.com/@softwaredefinedtalk), Threads (https://www.threads.net/@softwaredefinedtalk) and YouTube (https://www.youtube.com/channel/UCi3OJPV6h9tp-hbsGBLGsDQ/featured). Use the code SDT to get $20 off Coté's book, Digital WTF (https://leanpub.com/digitalwtf/c/sdt), so $5 total. Become a sponsor of Software Defined Talk (https://www.softwaredefinedtalk.com/ads)! Recommendations Brandon: YouTube TV (https://tv.youtube.com/welcome/) and NFL Sunday Ticket (https://tv.youtube.com/learn/nflsundayticket/) An Endgame for YouTube TV, Big Disney Decisions (And Whether Bob Iger Should Make Them), The Era Beyond Peak TV (https://sharptech.fm/member/episode/an-endgame-for-you-tube-tv-big-disney-decisions-and-whether-bob-iger-should-make-them-the-era-beyond-peak-tv) Matt: Airline wifi chat with Support Coté: Do Interesting (https://thedobook.co/products/do-interesting-notice-collect-share) book by Russel Davis. Photo Credits Header (https://unsplash.com/photos/m-Yot4dUd6s) Artwork (https://unsplash.com/photos/I7iJOE4fsYo)
* Modern Data Stack Meetup : Streamlit + Kestra### Gen AI* Deploy Your LLM Chatbot With Retrieval Augmented Generation (RAG), llama2-70B (MosaicML inferences) and Vector Search -> https://www.databricks.com/resources/demos/tutorials/data-science-and-ai/lakehouse-ai-deploy-your-llm-chatbot?itm_data=demo_center* LLMs in Action: A Practical Guide for Software Architects and Developers -> https://www.linkedin.com/pulse/llms-action-practical-guide-software-architects-s%C3%A9bastien-brasseur?utm_source=share&utm_medium=member_ios&utm_campaign=share_via* Lantern — a PostgreSQL vector database for building AI applications -> https://docs.lantern.dev/blog/2023/09/13/hello-world### Data science* Announcing Python in Excel: Combining the power of Python and the flexibility of Excel. -> https://techcommunity.microsoft.com/t5/excel-blog/announcing-python-in-excel-combining-the-power-of-python-and-the/ba-p/3893439* NEW Python in Excel - PYTHON + EXCEL + ChatGPT = Easy! -> https://youtu.be/-_1IaUjO-hk?si=ZA8-ztkfzQfA3cc0* Excel Just Got Python | Prime Reacts -> https://youtu.be/iAQJhYQEx-s?si=l_v3YC8o01Vyyar6### Cloud Native* Kubernetes OWASP Top 10: Secrets Management -> https://itnext.io/kubernetes-owasp-top-10-secrets-management-c996faa87b47Agenda* Bigdatapero à Paris 27/09/2023Cette publication est sponsorisée par Affini-Tech et CerenIT.CerenIT vous accompagne pour concevoir, industrialiser ou automatiser vos plateformes mais aussi pour faire parler vos données temporelles. Ecrivez nous à email@example.com et retrouvez-nous aussi au Time Series France.Affini-Tech vous accompagne dans tous vos projets Cloud et Data, pour Imaginer, Expérimenter et Executer vos services ! (Affini-Tech, Datatask) Consulter le blog d'Affini-Tech et le blog de Datatask pour en savoir plus. On recrute ! Venez cruncher de la data avec nous ! Ecrivez nous à firstname.lastname@example.orgLe générique a été composé et réalisé par Maxence Lecointe
Josh and Kurt talk about why CVE is making the news lately. Things are not well in the CVE program, and it's not looking like anything will get fixed anytime soon. Josh and Kurt have a unique set of knowledge around CVE. There's a lot of confusion and difficulty in understanding how CVE works. Show Notes Curl blog post Now it's PostgreSQL's turn to have a bogus CVE GitHub Advisory Database Josh's "CVE tried to get me fired" story
This week, we discuss Netflix's DVD deprecation, the remote work debate, and how to fork an open-source project. Plus, thoughts on why Europe needs more ice. Watch the YouTube Live Recording of Episode (https://www.youtube.com/watch?v=lFr-ysPYxnA) 431 (https://www.youtube.com/watch?v=lFr-ysPYxnA) Runner-up Titles Try Harder It's a necessary luxury Someone's drinking too much water here A culture of ice Where are the high performers, at home or at work Quit using your Gmail address Thou shalt export to CSV Rundown Netflix Says You Can Keep Their DVDs (and Request More, Too) (https://www.nytimes.com/2023/08/24/arts/netflix-dvds.html?smid=nytcore-ios-share&referringSource=articleShare) Zoom's CEO thinks Zoom sucks for building trust, leaked audio reveals (https://arstechnica.com/tech-policy/2023/08/leaked-audio-reveals-zoom-ceo-believes-its-hard-to-build-trust-on-zoom/) Meta is back in the office three days a week, as WFH continues to die (https://www.theverge.com/2023/9/5/23860073/meta-return-to-office-three-days-wfh-work-from-home) Can you trust 'open source' companies? (https://www.theregister.com/2023/08/18/opinion_column/) OpenTF created a fork of Terraform! (https://opentf.org/announcement) OpenTF pulls the trigger on its open-source Terraform fork (https://opensourcewatch.beehiiv.com/p/opentf-pulls-trigger-opensource-terraform-fork) Relevant to your Interests VMware's future: Navigating multicloud complexity and generative AI (https://siliconangle.com/2023/08/19/vmwares-future-navigating-multicloud-complexity-generative-ai-broadcoms-wing/) VMware Tanzu portfolio reshuffled ahead of Broadcom close | TechTarget (https://www.techtarget.com/searchitoperations/news/366549332/VMware-Tanzu-portfolio-reshuffled-ahead-of-Broadcom-close) Nvidia's blowout offers a giddy whiff of 1995 (https://www.axios.com/newsletters/axios-ai-plus-937b329c-8072-4f8a-a5d6-1039a0e794a5.html?chunk=0&utm_term=emshare#story0) Announcing AWS Dedicated Local Zones (https://aws.amazon.com/about-aws/whats-new/2023/08/aws-dedicated-local-zones/?utm_source=newsletter&utm_medium=email&utm_campaign=newsletter_axioslogin&stream=top) Top Ten social media platforms we spend the most time on (https://www.traveldailymedia.com/top-ten-social-media-platforms-we-spend-the-most-time-on/) Max will launch a 24/7 CNN stream for all subscribers next month (https://www.theverge.com/2023/8/24/23844121/cnn-max-warnerbros-discovery-news) Meta launches own AI code-writing tool: Code Llama (https://www.theverge.com/2023/8/24/23843487/meta-llama-code-generation-generative-ai-llm?stream=top) As TikTok Ban Looms, ByteDance Battles Oracle For Control Of Its Algorithm (https://www.forbes.com/sites/emilybaker-white/2023/08/24/tiktok-ban-oracle-bytedance-algorithm-fight/?sh=6cf5105e3ef0) Slack's Migration to a Cellular Architecture - Slack Engineering (https://slack.engineering/slacks-migration-to-a-cellular-architecture/) The Cloud 100 2023 (https://www.forbes.com/lists/cloud100/) Data isn't everything. Judgement counts too. (https://www.tiktok.com/t/ZT8YFUFju/) Amazon Elastic Block Store at 15 Years (https://perspectives.mvdirona.com/2023/08/amazon-elastic-block-store-at-15-years/?ck_subscriber_id=512840665) Instacart is the Best and Worst Grocery Business Imaginable (https://www.thediff.co/archive/instacart-is-the-best-and-worst-grocery-business-imaginable/) Amazon CEO Andy Jassy tells employees it's 'past' time to commit to the company's RTO mandate and their jobs are at stake (https://www.businessinsider.com/amazon-andy-jassy-rto-office-policy-employee-jobs-2023-8?op=1) Duet AI, Google's AI assistant suite, expands across Google Cloud (https://techcrunch.com/2023/08/29/duet-ai-googles-ai-assistant-suite-expands-across-google-cloud/) Halloween creeps a little closer: Seasonal supply chains accelerate (https://www.spglobal.com/marketintelligence/en/mi/research-analysis/halloween-creeps-closer-seasonal-supply-chains-accelerate.html) What's new with GKE at Google Cloud Next | Google Cloud Blog (https://cloud.google.com/blog/products/containers-kubernetes/whats-new-with-gke-at-google-cloud-next) Duet AI in Google Cloud Preview | Google Cloud Blog (https://cloud.google.com/blog/products/ai-machine-learning/duet-ai-in-google-cloud-preview) What's new in Oracle to PostgreSQL database migrations with DMS | Google Cloud Blog (https://cloud.google.com/blog/products/databases/whats-new-in-oracle-to-postgresql-database-migrations-with-dms) US AI startup Poolside raises $126m seed round and relocates to France (https://sifted.eu/articles/poolside-raises-126m-relocated-france-news) Ping, ForgeRock, Thoma Bravo, the power of open source, and the madness of IAM (https://callmeleach.substack.com/p/ping-forgerock-thoma-bravo-the-power?utm_medium=web) Thoma Bravo Completes Acquisition of ForgeRock; Combines ForgeRock into Ping Identity (https://www.prnewswire.com/news-releases/thoma-bravo-completes-acquisition-of-forgerock-combines-forgerock-into-ping-identity-301908059.html) Interoperability between Google Chat and other messaging platforms — powered by Mio (https://workspaceupdates.googleblog.com/2023/08/goolge-chat-slack-interoperability-mio.html) Broadcom boss dismisses notion China could derail VMware buy (https://www.theregister.com/2023/09/01/broadcom_vmware_nutanix_results/) Microsoft blames outage on small staff, automation failures (https://www.theregister.com/2023/09/04/microsoft_australia_outage_incident_report/) Amazon QuickSight adds scheduled and programmatic export to Excel format (https://aws.amazon.com/about-aws/whats-new/2023/08/amazon-quicksight-scheduled-programmatic-export-excel-format/?ck_subscriber_id=512840665) Google unveils AI tools for enterprise customers at $30 a month (https://www.reuters.com/technology/google-unveil-ai-tools-corporate-gmail-customers-30-month-wsj-2023-08-29/) Chip design firm Arm seeks up to $52 billion valuation in blockbuster U.S. IPO (https://www.cnbc.com/2023/09/05/chip-design-firm-arm-sets-share-price-between-47-and-51-for-blockbuster-us-ipo.html) Birmingham City Council goes under after Oracle disaster (https://www.theregister.com/2023/09/05/birmingham_city_council_oracle/?s=08) IBM Introduces 'Watsonx Your Business' (https://finance.yahoo.com/news/ibm-introduces-watsonx-business-160000392.html) Meta May Allow Instagram, Facebook Users in Europe to Pay and Avoid Ads (https://www.nytimes.com/2023/09/01/technology/meta-instagram-facebook-ads-europe.html?smid=nytcore-ios-share&referringSource=articleShare) Announcing Kubecost Cloud in General Availability: The Easiest Way to Optimize Your Kubernetes Costs (https://blog.kubecost.com/blog/kubecost-cloud-general-availability/) Platform Engineering - What You Need To Know Now (https://tanzu.vmware.com/content/ebooks/platformengineering-whatyouneedtoknownow?utm_source=cote&utm_campaign=devrel&utm_medium=newsletter&utm_content=newsletter20230830) The lifespans of technological adoptions in the US (http://www.asymco.com/2022/01/10/the-lifespans-of-technological-adoptions-in-the-us/) Introducing ONCE (https://once.com/) Nonsense The fight for the right to repair McFlurry machines (https://www.morningbrew.com/daily/stories/2023/08/31/the-fight-for-the-right-to-repair-mcflurry-machines) Delta Airlines Offers Woman $1,800 After Losing Her Dog (https://www.yahoo.com/entertainment/delta-airlines-offers-woman-1-142849291.html) Conferences Sep 18th to 19th SHIFT (https://shift.infobip.com/) in Zadar, Coté speaking. October 6, 2023, KCD Texas 2023 (https://community.cncf.io/events/details/cncf-kcd-texas-presents-kcd-texas-2023/), CFP Closes: August 30, 2023 November 6-9, 2023, KubeCon NA (https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/), SDT's a sponsor, Matt's there November 6-9, 2023 VMware Explore Barcelona (https://www.vmware.com/explore/eu.html), Coté's attending Jan 29, 2024 to Feb 1, 2024 That Conference Texas (https://that.us/events/tx/2024/schedule/) If you want your conference mentioned, let's talk media sponsorships. SDT news & hype Join us in Slack (http://www.softwaredefinedtalk.com/slack). Get a SDT Sticker! Send your postal address to email@example.com (mailto:firstname.lastname@example.org) and we will send you free laptop stickers! Follow us: Twitch (https://www.twitch.tv/sdtpodcast), Twitter (https://twitter.com/softwaredeftalk), Instagram (https://www.instagram.com/softwaredefinedtalk/), Mastodon (https://hachyderm.io/@softwaredefinedtalk), BlueSky (https://bsky.app/profile/softwaredefinedtalk.com), LinkedIn (https://www.linkedin.com/company/software-defined-talk/), TikTok (https://www.tiktok.com/@softwaredefinedtalk), Threads (https://www.threads.net/@softwaredefinedtalk) and YouTube (https://www.youtube.com/channel/UCi3OJPV6h9tp-hbsGBLGsDQ/featured). Use the code SDT to get $20 off Coté's book, Digital WTF (https://leanpub.com/digitalwtf/c/sdt), so $5 total. Become a sponsor of Software Defined Talk (https://www.softwaredefinedtalk.com/ads)! Recommendations Brandon: JUST ONE MILE | Official Trailer (https://www.youtube.com/watch?v=80V5o06yEZ4) Matt: Deadloch (https://www.imdb.com/title/tt14671678/) Coté: Rick Rubin interviews Rory Sutherland (https://www.youtube.com/watch?v=VnYlChfORRw). I doubt much of the airport business book stuff in here is “true,” but that's sort of the whole point, and it's fantastic listening. His book (https://amzn.to/462Mvov) Alchemy (https://amzn.to/462Mvov) has a great one word review right there in the title. But, again: it's fun! When you've listened to too much If Books Could Kill (https://en.wikipedia.org/wiki/If_Books_Could_Kill) you can check in on Rory if you need to take the cure (https://idioms.thefreedictionary.com/take+the+cure). Photo Credits Header (https://unsplash.com/photos/PsBTqRHVilU) Artwork (https://labs.openai.com/e/bKjqW8kPJyI2wuzBA0FogiKb/UJeLhuIFmvkrNFbfcCc4jE29)
Summary Data persistence is one of the most challenging aspects of computer systems. In the era of the cloud most developers rely on hosted services to manage their databases, but what if you are a cloud service? In this episode Vignesh Ravichandran explains how his team at Cloudflare provides PostgreSQL as a service to their developers for low latency and high uptime services at global scale. This is an interesting and insightful look at pragmatic engineering for reliability and scale. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Introducing RudderStack Profiles. RudderStack Profiles takes the SaaS guesswork and SQL grunt work out of building complete customer profiles so you can quickly ship actionable, enriched data to every downstream team. You specify the customer traits, then Profiles runs the joins and computations for you to create complete customer profiles. Get all of the details and try the new product today at dataengineeringpodcast.com/rudderstack (https://www.dataengineeringpodcast.com/rudderstack) This episode is brought to you by Datafold – a testing automation platform for data engineers that finds data quality issues before the code and data are deployed to production. Datafold leverages data-diffing to compare production and development environments and column-level lineage to show you the exact impact of every code change on data, metrics, and BI tools, keeping your team productive and stakeholders happy. Datafold integrates with dbt, the modern data stack, and seamlessly plugs in your data CI for team-wide and automated testing. If you are migrating to a modern data stack, Datafold can also help you automate data and code validation to speed up the migration. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold (https://www.dataengineeringpodcast.com/datafold) You shouldn't have to throw away the database to build with fast-changing data. You should be able to keep the familiarity of SQL and the proven architecture of cloud warehouses, but swap the decades-old batch computation model for an efficient incremental engine to get complex queries that are always up-to-date. With Materialize, you can! It's the only true SQL streaming database built from the ground up to meet the needs of modern data products. Whether it's real-time dashboarding and analytics, personalization and segmentation or automation and alerting, Materialize gives you the ability to work with fresh, correct, and scalable results — all in a familiar SQL interface. Go to dataengineeringpodcast.com/materialize (https://www.dataengineeringpodcast.com/materialize) today to get 2 weeks free! Your host is Tobias Macey and today I'm interviewing Vignesh Ravichandran about building an internal database as a service platform at Cloudflare Interview Introduction How did you get involved in the area of data management? Can you start by describing the different database workloads that you have at Cloudflare? What are the different methods that you have used for managing database instances? What are the requirements and constraints that you had to account for in designing your current system? Why Postgres? optimizations for Postgres simplification from not supporting multiple engines limitations in postgres that make multi-tenancy challenging scale of operation (data volume, request rate What are the most interesting, innovative, or unexpected ways that you have seen your DBaaS used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on your internal database platform? When is an internal database as a service the wrong choice? What do you have planned for the future of Postgres hosting at Cloudflare? Contact Info LinkedIn (https://www.linkedin.com/in/vigneshravichandran28/) Website (https://viggy28.dev/) Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. Podcast.__init__ (https://www.pythonpodcast.com) covers the Python language, its community, and the innovative ways it is being used. The Machine Learning Podcast (https://www.themachinelearningpodcast.com) helps you go from idea to production with machine learning. Visit the site (https://www.dataengineeringpodcast.com) to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email email@example.com (mailto:firstname.lastname@example.org)) with your story. To help other people find the show please leave a review on Apple Podcasts (https://podcasts.apple.com/us/podcast/data-engineering-podcast/id1193040557) and tell your friends and co-workers Links Cloudflare (https://www.cloudflare.com/) PostgreSQL (https://www.postgresql.org/) Podcast Episode (https://www.dataengineeringpodcast.com/postgresql-with-jonathan-katz-episode-42/) IP Address Data Type in Postgres (https://www.postgresql.org/docs/current/datatype-net-types.html) CockroachDB (https://www.cockroachlabs.com/) Podcast Episode (https://www.dataengineeringpodcast.com/cockroachdb-with-peter-mattis-episode-35/) Citus (https://www.citusdata.com/) Podcast Episode (https://www.dataengineeringpodcast.com/citus-data-with-ozgun-erdogan-and-craig-kerstiens-episode-13/) Yugabyte (https://www.yugabyte.com/) Podcast Episode (https://www.dataengineeringpodcast.com/yugabytedb-planet-scale-sql-episode-115/) Stolon (https://github.com/sorintlab/stolon) pg_rewind (https://www.postgresql.org/docs/current/app-pgrewind.html) PGBouncer (https://www.pgbouncer.org/) HAProxy Presentation (https://www.youtube.com/watch?v=HIOo4j-Tiq4) Etcd (https://etcd.io/) Patroni (https://patroni.readthedocs.io/en/latest/) pg_upgrade (https://www.postgresql.org/docs/current/pgupgrade.html) Edge Computing (https://en.wikipedia.org/wiki/Edge_computing) The intro and outro music is from The Hug (http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/Love_death_and_a_drunken_monkey/04_-_The_Hug) by The Freak Fandango Orchestra (http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/) / CC BY-SA (http://creativecommons.org/licenses/by-sa/3.0/)
Evan and Sujit cover a whole bunch of updates to various Azure services such as AKS, ACA, Functions, VMs, PostgerSQL, VNETs and Storage. Media file: https://azpodcast.blob.core.windows.net/episodes/Episode470.mp3 YouTube: https://youtu.be/eGsnmw5WjHI Resources: VMs https://azure.microsoft.com/en-us/updates/generally-available-cross-subscription-restore-for-azure-virtual-machines/ https://techcommunity.microsoft.com/t5/azure-high-performance-computing/introducing-new-performance-tiers-for-azure-managed-lustre/ba-p/3900329 Storage https://azure.microsoft.com/en-us/updates/azure-elastic-san-updates-private-endpoints-shared-volumes/ Functions https://azure.microsoft.com/en-us/updates/generally-available-sdk-type-bindings-in-azure-functions/ https://azure.microsoft.com/en-us/updates/generally-available-application-insights-integration-for-the-azure-functions-net-worker/ AKS/ACA https://azure.microsoft.com/en-us/updates/generally-available-kubernetes-127-support-in-aks/ https://azure.microsoft.com/en-us/updates/generally-available-azure-key-vault-references-for-secrets-in-azure-container-apps/ https://azure.microsoft.com/en-us/updates/generally-available-secrets-volume-mounts-for-azure-container-apps/ https://azure.microsoft.com/en-us/updates/generally-available-session-affinity-for-azure-container-apps/ https://azure.microsoft.com/en-us/updates/generally-available-init-containers-in-azure-container-apps/ https://azure.microsoft.com/en-us/updates/generally-available-cross-origin-resource-sharing-cors-in-azure-container-apps/ PostgreSQL https://azure.microsoft.com/en-us/updates/public-preview-storage-autogrow-online-disk-scaling-for-azure-database-for-postgresql-flexible-server/ Major Version Upgrade of a flexible server - Azure portal | Microsoft Learn VNET https://azure.microsoft.com/en-us/updates/public-preview-azure-vnet-flow-logs/
Nectain je low-code platforma původem z Ukrajiny, kde ji znají úplně všichni. A to buď pod jménem mateřské firmy Intecracy, nebo sesterského DMS Megapolis, které používá velká část státní správy. Součástí je i databáze optimalizovaná pro Big Data, připravená na miliony záznamů a zpracování tisíců requestů za vteřinu. Díky expanzi do světa má dnes Nectain přes 100 tisíc uživatelů, akvizice dalších je ale docela dřina. CEO Roman Olejnikov totiž nedávno na meetingu slyšel, že jejich nabídka je „too good to be true“
Sabine talks to Michele Cimmino, CEO of Lasting Dynamics which, far from being a generic software firm, epitomises precision and coding to partners' distinct needs in order to ensure an exclusive synergy for mutual growth. Michael discusses visionary leadership at Lasting Dynamics, championing quality and innovation, ensuring a diverse tech mastery, and building a global force in the startup ecosystem. KEY TAKEAWAYSLasting Dynamics was started in 2013 to provide the IT market with quality. Large enterprises usually target a span of 12 months to deliver an MVP (minimum viable product), by the time an MVP is delivered to market in 12 months it's already too late. When you have engineers who truly understand cybersecurity then it's easier to be compliant with policies. When you build something from the ground up, or start integrating the pieces into a digital product, with a forward-looking mindset you will not waste months in additional testing. This is an engineering mindset called “bottom-up”. BEST MOMENTS‘Quality assurance is also about analysing the requirements, it's not only about executing some tests, it's about understanding what will be needed and how this can be provided.'‘The most important part of the game is to be transparent and loyal to your partner or customer. That always pays back.'‘I like the idea that the most important goal and value for everyone on a project should be the final result of the product and everyone should sacrifice a bit of personal interest towards that goal.' ABOUT THE GUESTIn the dynamic world of software engineering, few names resonate as powerfully as Michele Cimmino. As a master of full stack web and mobile development, Michele's leadership at Lasting Dynamics has carved a niche in the tech spaces of Spain, Italy, and Norway. Dive deeper at Lasting Dynamics and you'll discover a hub where innovation meets commitment. A champion of quality, Michele's passion drives him and his adept team to consistently re-evaluate and refine their methodologies. They are agile, experimental, and always ahead of the curve, drawing insights from the latest in automation, innovation, and QA processes. His current explorations into Agile, OKRs, SCRUM, TDD, and the principles of Shu Ha Ri demonstrate a ceaseless thirst for knowledge. Lasting Dynamics, Michele's brainchild, isn't just any software company; it's a boutique, meticulously crafting code to fit partners' unique needs. This exclusivity ensures mutual growth, as they partner with only a select few each year. But there's more: The Academy, another feather in Michele's cap, streamlines the onboarding process, a testament to his visionary approach. Moreover, under his guidance, the launch of SaaS platforms like Roundrush for workflow management and VetrinaLive for e-commerce adds to his impressive portfolio. With a tech stack that's as expansive as it is impressive, ranging from Node.js to Python, React to Vue.js, PostgreSQL to AWS integrations, Michele's proficiency is unquestionable. He's been at the helm of AI-driven mobile apps since 2015, dabbled in the realm of blockchain and cryptocurrency from 2017, and from 2019, has steered Lasting Dynamics towards Growth Hacking. ABOUT THE HOSTSabine is a corporate strategist turned entrepreneur. She is the CEO and Managing Partner of Alchemy Crew, a venture lab that accelerates the curation, validation, and commercialization of new tech business models. Sabine is renowned within the insurance sector for building some of the most renowned tech startup accelerators around the world working with over 30 corporate insurers and accelerating over 100 startup ventures. Sabine is the co-editor of the bestseller The INSURTECH Book, a top 50 Women in Tech, a FinTech and InsurTech Influencer, an investor & multi-award winner. Twitter: SabineVdLLinkedIn: Sabine VanderLindenInstagram: sabinevdLofficialFacebook: SabineVdLOfficialTikTok: sabinevdlofficialEmail: email@example.comWebsite: www.sabinevdl.comThis show was brought to you by Progressive Media
To get the show notes as well as get notified of new episodes, visit: https://www.scalingpostgres.com/episodes/277-postgres-releases-postgresql-survey-partitioning-sharding-bulk-loading/ In this episode of Scaling Postgres, we discuss new Postgres releases, taking the 2023 State of PostgreSQL survey, partitioning vs. sharding and the fastest way to do bulk loads.
Simon takes you on a tour of your GenAI options. From software development, to AI policy, to trialling FMs, to new instance types, and CPUs. Referenced Links: - URL: https://aws.amazon.com/blogs/devops/optimize-software-development-with-amazon-codewhisperer/ Title: Optimize software development with Amazon CodeWhisperer | AWS DevOps Blog - URL: https://aws.amazon.com/blogs/devops/10-ways-to-build-applications-faster-with-amazon-codewhisperer/ Title: 10 ways to build applications faster with Amazon CodeWhisperer | AWS DevOps Blog - URL: https://aws.amazon.com/blogs/big-data/build-data-integration-jobs-with-ai-companion-on-aws-glue-studio-notebook-powered-by-amazon-codewhisperer/ Title: Build data integration jobs with AI companion on AWS Glue Studio Notebook powered by Amazon CodeWhisperer - URL: https://aws.amazon.com/blogs/machine-learning/use-generative-ai-foundation-models-in-vpc-mode-with-no-internet-connectivity-using-amazon-sagemaker-jumpstart/ Title: Use generative AI foundation models in VPC mode with no internet connectivity using Amazon SageMaker JumpStart | AWS Machine Learning Blog - URL: https://aws.amazon.com/blogs/machine-learning/llama-2-foundation-models-from-meta-are-now-available-in-amazon-sagemaker-jumpstart/ Title: Llama 2 foundation models from Meta are now available in Amazon SageMaker JumpStart | AWS Machine Learning Blog - URL: https://aws.amazon.com/blogs/machine-learning/use-stable-diffusion-xl-with-amazon-sagemaker-jumpstart-in-amazon-sagemaker-studio/ Title: Use Stable Diffusion XL with Amazon SageMaker JumpStart in Amazon SageMaker Studio | AWS Machine Learning Blog - URL: https://aws.amazon.com/blogs/machine-learning/efficiently-train-tune-and-deploy-custom-ensembles-using-amazon-sagemaker/ Title: Efficiently train, tune, and deploy custom ensembles using Amazon SageMaker | AWS Machine Learning Blog - URL: https://aws.amazon.com/blogs/machine-learning/quickly-build-high-accuracy-generative-ai-applications-on-enterprise-data-using-amazon-kendra-langchain-and-large-language-models/ Title: Quickly build high-accuracy Generative AI applications on enterprise data using Amazon Kendra, LangChain, and large language models | AWS Machine Learning Blog - URL: https://aws.amazon.com/bedrock/features/ Title: Foundation Model API Service – Amazon Bedrock Features – AWS - URL: https://aws.amazon.com/blogs/aws/preview-enable-foundation-models-to-complete-tasks-with-agents-for-amazon-bedrock/ Title: Preview – Enable Foundation Models to Complete Tasks With Agents for Amazon Bedrock | AWS News Blog - URL: https://aws.amazon.com/blogs/business-intelligence/announcing-generative-bi-capabilities-in-amazon-quicksight/ Title: Announcing Generative BI capabilities in Amazon QuickSight | AWS Business Intelligence Blog - URL: https://aws.amazon.com/about-aws/whats-new/2023/05/amazon-rds-postgresql-pgvector-ml-model-integration/ Title: Amazon RDS for PostgreSQL now supports pgvector for simplified ML model integration - URL: https://aws.amazon.com/blogs/big-data/introducing-the-vector-engine-for-amazon-opensearch-serverless-now-in-preview/ Title: Introducing the vector engine for Amazon OpenSearch Serverless, now in preview | AWS Big Data Blog - URL: https://aws.amazon.com/blogs/aws/new-amazon-ec2-p5-instances-powered-by-nvidia-h100-tensor-core-gpus-for-accelerating-generative-ai-and-hpc-applications/ Title: New – Amazon EC2 P5 Instances Powered by NVIDIA H100 Tensor Core GPUs for Accelerating Generative AI and HPC Applications | AWS News Blog - URL: https://aws.amazon.com/blogs/machine-learning/maximize-stable-diffusion-performance-and-lower-inference-costs-with-aws-inferentia2/ Title: Maximize Stable Diffusion performance and lower inference costs with AWS Inferentia2 | AWS Machine Learning Blog - URL: https://aws.amazon.com/blogs/machine-learning/new-technical-deep-dive-course-generative-ai-foundations-on-aws/ Title: New technical deep dive course: Generative AI Foundations on AWS | AWS Machine Learning Blog - URL: https://aws.amazon.com/blogs/machine-learning/aws-offers-new-artificial-intelligence-machine-learning-and-generative-ai-guides-to-plan-your-ai-strategy/ Title: AWS offers new artificial intelligence, machine learning, and generative AI guides to plan your AI strategy | AWS Machine Learning Blog - URL: https://aws.amazon.com/blogs/machine-learning/aws-reaffirms-its-commitment-to-responsible-generative-ai/ Title: AWS Reaffirms its Commitment to Responsible Generative AI | AWS Machine Learning Blog
Benedikt talks to Andrew Atkinson about different types of database constraints and his upcoming book. Read the tweet that triggered this interview Learn more about High Performance PostgreSQL for Rails Check out Andrew's blog Follow Andrew on Twitter Andrew Atkinson is a staff software engineer at Fountain who is very passionate about Ruby on Rails and PostgreSQL (just like Benedikt!). He's so passionate about them that he's actually writing a book called “High Performance PostgreSQL for Rails” that's coming out very soon.Benedikt and Andrew also talk about dealing with constraints and ActiveRecord, Rails validations, and more.
Welcome episode 219 of The Cloud Pod podcast - where the forecast is always cloudy! Today your hosts are Justin and Jonathan, and they discuss all things cloud, including clickstream analytics, databricks, Microsoft Entra, virtual machines, Outlook threats, and some major changes over at the Google Cloud team. Titles we almost went with this week: TCP is not Entranced with Entra ID The Cave you Fear to Entra, Holds the Treasure you Seek Microsoft should rethink Entra rules for their Email A big thanks to this week's sponsor: Foghorn Consulting, provides top-notch cloud and DevOps engineers to the world's most innovative companies. Initiatives stalled because you have trouble hiring? Foghorn can be burning down your DevOps and Cloud backlogs as soon as next week.
On this episode, Andrew Atkinson returns for a conversation centering on partitioning in PostgreSQL. We also discuss his upcoming book High Performance PostgreSQL for Rails, Sin City Ruby and some of the benefits of attending conferences, and "getting reps in." High Performance PostgreSQL for Rails by Andrew Atkinson (coming August 2023)The Rails Way by Obie FernandezpgslicePartitioning Billions of Rows Without DowntimeAndy Atkinson.comAndrew Atkinson on TwitterAndrew Atkinson on GitHubSin City Ruby
After talking to Ryan Berry a little while back about OneStream's journey to Azure, the team catches back up with him to hear about how their architecture will evolve in the future to be more Cloud Native. In addition, they discuss the challenges of rearchitecting a pre-existing product when you cannot afford the have dual stacks running at the same time. Media File: https://azpodcast.blob.core.windows.net/episodes/Episode467.mp3 YouTube: https://www.youtube.com/watch?v=0D4x7jDjs-8 Resources Corporate Performance Management Software | OneStream Software Links to updates mentioned at the top of the show Azure AD is becoming Microsoft Entra ID Public preview: Ingest events from Azure Event Hubs to Azure Monitor Logs General availability: Azure Data Explorer adds support for PostgreSQL, MySQL, and CosmosDB SQL external tables Azure cross-region Load Balancer is now generally available Public preview: Azure Virtual Network encryption
Guy and Eitan discuss several interesting topics and recent experiences that you'd want to hear about! Relevant links: Trigram Wildcard String Search in SQL Server The Story of DBeaver, PostgreSQL, and the silly data engineer SQL Server Error Log Management High VLF Counts Got You Down? Here's How to Keep Them Low and Performance High
The promise of digital mapping is to provide a shared and real-time view of the state of the underlying system. pg_eventserv is a free and open-source component that helps fulfill the promise of real-time event modeling and shared views in PostgreSQL. By connecting to PostgreSQL and listening on specified channels, pg_eventserv captures database notifications and forwards them to web clients, enabling real-time updates and synchronization of data displayed on maps or other web interfaces. pg_eventserv does one thing and one thing only: take events generated by the PostgreSQL NOTIFY command and passes the payload along to waiting WebSockets clients. pg_eventserv is free and easy to install and you can find it here: https://github.com/CrunchyData/pg_eventserv What this means is that any client can watch for notifications and update as changes in the database happen. Real-time data! Here is a link to a Youtube video demonstration of pg_eventserv in action! https://youtu.be/UakRtYmoWow I will let Paul Ramsey the creator of pg_eventserv explain all this in more detail in this episode. If you want to reach out to Paul the best place to do that is http://blog.cleverelephant.ca/ Or if you want to listen to previous episodes with Paul you might find these interesting Raster in the database? https://mapscaping.com/podcast/rasters-in-a-database/ Dynamic Vector Tiles Straight From The Database https://mapscaping.com/podcast/dynamic-vector-tiles-straight-from-the-database/ Spatial SQL- GIS Without The GIS https://mapscaping.com/podcast/spatial-sql-gis-without-the-gis/ also ... If you are interested in spatial databases at scale ... you might find this episode interesting https://mapscaping.com/podcast/distributing-geospatial-data/