Podcasts about zhamak dehghani

I'm joined this week by one of the authors of Apache Kafka In Action, to take a look at the state of Kafka, event systems & stream-processing technology. It's an approach (and a whole market) that's had at least a decade to mature, so how has it done? What does Kafka offer to developers and businesses, and which parts do they actually care about? What have streaming data systems promised and what have they actually delivered? What's still left to build?–Apache Kafka in Action: https://www.manning.com/books/apache-kafka-in-actionPat Helland, Data on the Inside vs Data on the Outside: https://queue.acm.org/detail.cfm?id=3415014Out of the Tar Pit: https://curtclifton.net/papers/MoseleyMarks06a.pdfMartin Kleppmann, Turning the Database Inside-Out: https://martin.kleppmann.com/2015/11/05/database-inside-out-at-oredev.htmlData Mesh by Zhamak Dehghani: https://www.amazon.co.uk/Data-Mesh-Delivering-Data-Driven-Value/dp/1492092398Quix Streams: https://github.com/quixio/quix-streamsXTDB: https://xtdb.com/Support Developer Voices on Patreon: https://patreon.com/DeveloperVoicesSupport Developer Voices on YouTube: https://www.youtube.com/@developervoices/joinAnatoly's Website: https://zelenin.de/Kris on Mastodon: http://mastodon.social/@krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/Kris on Twitter: https://twitter.com/krisajenkins

Zhamak Dehghani joins the Practical Data Community for an open-ended Q&A about autonomous data products, Data Mesh, NextData and much more.NextData: https://www.nextdata.com/Practical Data Community: https://discord.gg/gNfw5AKWSK

In this episode, we dive deep into the world of data decentralization with Zhamak Dehghani, founder of Data Mesh and CEO of Next Data. Zhamak, a pioneer in the field, shares her vast experience and insights into how data mesh is reshaping the way organizations manage and leverage their data. Zhamak explains the key concepts behind data mesh and its importance in solving data scalability challenges. With her expertise in designing data-driven solutions, she walks us through real-world use cases and the core principles that make data mesh a game-changer for modern businesses. Host: Jake Aaron Villarreal, leads the top AI Recruitment Firm in Silicon Valley www.matchrelevant.com, uncovering stories of funded startups and goes behinds to scenes to tell their founders journey.  If you are growing AI Startup or have a great storytelling, email us at: jake.villarreal@matchrelevant.com  

"The genesis of data mesh was really to address some of the deepest darkest problems that the data management had for decades."

Fredrik snackar data med Benny Andrén och Hugo Hjertén - expereter på data och röster i podden Datastudion. Data man har i sin organisation och vill göra bättre saker med - strukturera, göra tillgänglig, utvinna information och insikter ur och rent allmänt skapa värde. Data är där agile var - företag säljer datapaketet, men förståelsen finns inte än. Vi diskuterar också hypehantering, och inte minst AI-hantering. Att komma in och få företag att göra bra saker istället för att jaga den senaste trenden utan att ta reda på varför. Vad är en dataplattform, och varför behöver Fredrik en? Se plattformen och jobb med data som en produkt som levererar värde. Ifrågasätt data - man måste inte bara tänka inom den data man råkar ha precis nu. Ett stort tack till Cloudnet som sponsrar vår VPS! Har du kommentarer, frågor eller tips? Vi är @kodsnack, @thieta, @krig, och @bjoreman på Mastodon, har en sida på Facebook och epostas på info@kodsnack.se om du vill skriva längre. Vi läser allt som skickas. Gillar du Kodsnack får du hemskt gärna recensera oss i iTunes! Du kan också stödja podden genom att ge oss en kaffe (eller två!) på Ko-fi, eller handla något i vår butik. Länkar Benny Hugo Amiga 500 Generation 500 Jayway - numera Devoteam Modern data stack Data dao - företaget Benny och Hugo jobbar Google testade sig fram till rätt nyans av blått A/B-testning Q group BI architect KPI:er OKR Capgemini LLM Generativ AI Data lake Data warehouse Data lakehouse Data mesh Zhamak Dehghani - upphovsperson till data mesh Matlab R Dataops ML-modeller Creedence Shoreline Hubspot DBT Datastudion Avsnitt av Datastudion om AI GDPR Målstyrning Øredev Kallbadhuset Titlar En smålänning i exil Världen med data Då kommer inte data hjälpa dig Ett databolag Så ni jobbar med data? Behöver vi bli datadrivna? Seanser och workshopar Låt oss framstå som duktiga på AI AI-tåget Vad är det ni faktiskt vill få ut? Bra data AI är moroten Samma sak för en ML-modell Jobba mot perfektion Sin nisch inom datastacken Få ut värde ur data Vi behöver två Hugo Precis det som är IKEA

Joe and Matt are going to take the summer off from the Monday Morning Data Chat. They'll chat about what's on their minds, summer plans, and more. Plus, special guest Zhamak Dehghani discusses the current state of Data Mesh, and what she's up to at NextData.

In this podcast episode, host Ole Olesen-Bagneux welcomes Simon Harrer, celebrated for his influential work in the realm of software engineering and his pivotal role in championing the Data Mesh movement in Germany.Simon takes us on a journey from teaching Java programming and co-writing the book "Java by Comparison" to transitioning into the data world, spurred by his fascination with the transformative potential of Data Mesh. He recounts the challenges and triumphs of translating Zhamak Dehghani's "Data Mesh" book into German.As the conversation unfolds, Simon elucidates the relationship between software engineering principles and Data Mesh, emphasizing the significance of data contracts and the process of fostering a data-centric culture within traditionally software-focused teams. His insights shed light on the early adoption stages of Data Mesh and the first data decentralization success stories in Germany.

Follow: https://stree.ai/podcast | Sub: https://stree.ai/sub | New episodes every Monday! In this episode of "The Real Time Analytics Podcast," Tim Berglund is joined by returning guest Peter Corless (Director of Product Marketing, StarTree) to delve into the complex world of federated data systems. They discuss the evolution of data architectures, the challenges of federated identity and data governance, and the implications for modern businesses. Tune in for an insightful conversation on the intricacies and future directions of federated data in an era of diverse and interconnected systems.

Highlights from this week's conversation include:Defining data mesh (6:37)Addressing the scale of organizational complexity and usage (9:04)The shift from monolithic to microservices (12:24)The sociological structure in data mesh (13:59)Data product generation and sharing in data mesh (17:27)Data Mesh: Simplifying Data Work (24:09)Getting Started with Data Mesh (29:14)Building products for Data Mesh (36:42)Building a customizable and extensible platform to shape data practice (39:28)The characteristics of a data product (48:40)Defining what a data product is not (50:45)The origin of the term "mesh" in data mesh (53:32)The Data Stack Show is a weekly podcast powered by RudderStack, the CDP for developers. Each week we'll talk to data engineers, analysts, and data scientists about their experience around building and maintaining data infrastructure, delivering data and data products, and driving better outcomes across their businesses with data.RudderStack helps businesses make the most out of their customer data while ensuring data privacy and security. To learn more about RudderStack visit rudderstack.com.

#DataManagement #Korrelationen #DataOwnershipwww.iotusecase.comIn der 119. Folge des IoT Use Case Podcasts geht es um die innovative Integration und Nutzung von Daten in Unternehmen, illustriert anhand konkreter Use Cases wie dem "Product Carbon Footprint", Kundendatenabgleich und AI-basiertem Modelltraining.Die Episode beleuchtet, wie die Firmen Steadforce und Starburst Data gemeinsam innovative Datenlösungen entwickeln und umsetzen. Madeleine Mickeleit begrüßt zwei Experten: Stephan Schiffner, CTO von Steadforce, und Roland Mackert, Head Alliances & Ecosystem; Partnermanager bei Starburst Data.Folge 119 auf einen Blick (und Klick):[11:12] Herausforderungen, Potenziale und Status quo – So sieht der Use Case in der Praxis aus[26:56] Lösungen, Angebote und Services – Ein Blick auf die eingesetzten TechnologienZusammenfassung der PodcastfolgeEin zentrales Thema der Podcastfolge ist das Konzept des Data Mesh, entwickelt von Zhamak Dehghani. Data Mesh bricht traditionelle, monolithische und zentralisierte Datenstrukturen auf und betrachtet Daten aus einer neuen Perspektive. Das Konzept basiert auf vier Prinzipien: Domain Ownership, Betrachtung von Daten als Produkt, Förderung von Self-Service und geförderter Governance.Beide Experten diskutieren verschiedene Use Cases, die durch diese Art von Datenintegration ermöglicht werden. Ein Beispiel ist die Korrelation von Vertriebs- und Produktionsdaten, um Kundeninformationen abzugleichen und Analysen durchzuführen. Ein weiteres Beispiel ist der "Product Carbon Footprint", bei dem Daten aus verschiedenen Quellen wie ERP-Systemen und Produktionsdaten zusammengeführt werden müssen.Abschließend betonen Stephan Schiffner und Roland Mackert die Vorteile der dezentralisierten Datenansätze. Diese ermöglichen es Unternehmen, flexibel und effizient auf ihre Daten zuzugreifen und diese für geschäftliche Entscheidungen und Analysen zu nutzen. Die Technologie von Starburst Data, die auf der Trino SQL-Query-Engine basiert, spielt dabei eine entscheidende Rolle.---Relevante Folgenlinks:Stephan (https://www.linkedin.com/in/stephan-schiffner/)Roland (https://www.linkedin.com/in/roland-mackert/)Madeleine (https://www.linkedin.com/in/madeleine-mickeleit/)Jetzt IoT Use Case auf LinkedIn folgen

In this bonus episode, Eric and Kostas preview their upcoming conversation regarding Data Mesh with Paolo Platter, Zhamak Dehghani, and Melissa Logan.

Data Mesh ist eine innovative Herangehensweise an die Organisation von Daten in Unternehmen. Dabei ist jedes Team für die eigenen Daten und Datenprodukte verantwortlich. Wir beleuchten die vier Prinzipien des Data Mesh (Domain Ownership, Data as a Product, Self-Serve Data Platform und Federated Computational Governance). Zum Schluss stellen wir uns die Frage, welche Eigenschaften eine Plattform mitbringen muss, um ein Data Mesh effektiv zu unterstützen, und ob dieser Hype einen Kulturwandel auslösen wird oder Theorie bleibt.  ***Links:*** - inwt Website: https://www.inwt-statistics.de/ - Blog: Data Mesh Principles and Logical Architecture by Zhamak Dehghani https://martinfowler.com/articles/data-mesh-principles.html  - Talk: Data - The land DevOps forgot by Michael Nygard https://www.youtube.com/watch?v=459-H33is6o  - Blog: How to select technology for Data Mesh by Ryan Dawson https://www.thoughtworks.com/insights/blog/data-strategy/how-to-select-technology-data-mesh - White Paper: Simplifying Data Mesh for Self-Service Analytics on an Open Data Lakehouse by Mike Ferguson https://hello.dremio.com/wp-simplifying-data-mesh-on-data-dakehouse-reg.html - White Paper: How to Knit Your Data Mesh on Snowflake https://snowflake.hub.hushly.com/data-mesh-stream/how-to-knit-your-data-mesh-on-snowflake

In this podcast episode, host Ole Olesen-Bagneux is joined by Samia Rahman - one of the rare data mesh pioneers that has worked directly with Zhamak Dehghani and a proven expert in enterprise data strategy and governance - to explore the interconnections between data mesh, data democracy, and domain complexities in healthcare and pharmaceutical industries. The dialogue travels through Samia's diverse experiences, emphasizing the importance of a synergetic approach between technical and domain expertise to unlock the full value of data. Samia shares real-world insights on the challenges and bottlenecks faced in implementing data mesh in highly regulated and centralized industries, and highlights the essential role of domain understanding and respect in bridging the gap between IT and domain specialists. This episode concludes with thoughts on the future trajectory of data mesh, anticipating its increased applicability in specific industries, particularly those involving a spectrum of products and complexities.

Follow: https://stree.ai/podcast | Sub: https://stree.ai/sub | New episodes every Monday! Join us on the Real-Time Analytics Podcast as we delve into the intriguing intersection of data mesh and event streaming with Hubert Dulay, a developer advocate at StarTree and the author of "Streaming Data Mesh." Our host, Tim Berglund, uncovers the journey from Zhamak Dehghani's initial concept to Hubert's vision of implementing it in a streaming context. Understand the essence of treating data as a product, the future of streaming technologies, and the transformative role of data in modern businesses. Hubert's book: https://www.oreilly.com/library/view/streaming-data-mesh/9781098130718/Zhamak Dehghani's Real-Time Analytics Summit keynote: https://youtu.be/Pz3UPpv_JIs

Célia Fischbach est Data Manager chez BlaBlaCar, la plateforme de transports partagés (voiture et bus) leader en Europe avec 20 millions d'utilisateurs en France.On aborde :

Once just a sparkle in a data enthusiast's eye, the data mesh is coming alive! Many data-intensive organizations are realizing the benefits of localized data management, and better managed costs. Perhaps more importantly, there's a deep appreciation growing for the value of data products. Join this episode of DM Radio to learn more! Host @eric_kavanagh will interview data mesh innovator Zhamak Dehghani of NextData, along with Sumit Pal and Doug Kimball of Ontotext. Attendees will learn: * How to enable a meaningful data mesh strategy * Why a knowledge graph is the perfect foundation * Key business drivers for leveraging a data mesh * How to avoid common pitfalls in building a mesh

Zhamak Dehghani is the CEO / Founder of Nextdata and also well known as the creator of Data Mesh. Data Mesh is a set of core principles that enables organizations to become data-driven by changing the way they organize their teams and their data architecture.  Zhamak walks us through her deep expertise working with enterprise data teams and the story of how Data Mesh came to be. Zhamak also discusses the power decentralization and how it allows enterprises to scale insights. Zhamak Dehghani's vision for Nextdata is building a world where AI and ML is powered by equitable and responsible data ownership via decentralization. We also discuss data products as a first class primitive and how data teams can start their Data Mesh journey with practical steps of 'change through movement.' Follow Zhamak Dehghani on Linkedin and Twitter.Follow Nextdata hereCheck out "Data Mesh: Delivering Data-Driven Value at Scale" by Zhamak Dehghani on AmazonWhat's New In Data is a data thought leadership series hosted by John Kutay who leads data and products at Striim. What's New In Data hosts industry practitioners to discuss latest trends, common patterns for real world data patterns, and analytics success stories.

This interview was recorded for the GOTO Book Club.gotopia.tech/bookclubRead the full transcription of the interview hereNeal Ford - Software Consultant at Thoughtworks & Co-Author of "Software Architecture: The Hard Parts"Mark Richards - Hands-On Software Architect, Independent Consultant & Co-Author of "Software Architecture: The Hard Parts"DESCRIPTIONThere are no easy decisions in software architecture. Instead, there are many hard parts--difficult problems or issues with no best practices--that force you to choose among various compromises. With this book, you'll learn how to think critically about the trade-offs involved with distributed architectures.Architecture veterans and practicing consultants Neal Ford, Mark Richards, Pramod Sadalage, and Zhamak Dehghani discuss strategies for choosing an appropriate architecture. By interweaving a story about a fictional group of technology professionals--the Sysops Squad--they examine everything from how to determine service granularity, manage workflows and orchestration, manage and decouple contracts, and manage distributed transactions to how to optimize operational characteristics, such as scalability, elasticity, and performance.By focusing on commonly asked questions, this book provides techniques to help you discover and weigh the trade-offs as you confront the issues you face as an architect.• Analyze trade-offs and effectively document your decisions• Make better decisions regarding service granularity• Understand the complexities of breaking apart monolithic applications• Manage and decouple contracts between services• Handle data in a highly distributed architecture• Learn patterns to manage workflow and transactions when breaking apart applications* Book description: © O'ReillyThe interview is based on Neal's & Mark's co-authored book "Software Architecture: The Hard Parts"RECOMMENDED BOOKSFord, Richards, Sadalage & Dehghani • Software Architecture: The Hard PartsMark Richards & Neal Ford • Fundamentals of Software ArchitectureFord, Parsons, Kua & Sadalage  • Building Evolutionary Architectures 2nd EditionFord, Parsons & Kua • Building Evolutionary ArchitecturesNeal Ford • Functional ThinkingMichael Nygard • Release It! 2nd EditionGene Kim & John Willis • Beyond The Phoenix ProjectTwitterLinkedInFacebookLooking for a unique learning experience?Attend the next GOTO conference near you! Get your ticket: gotopia.techSUBSCRIBE TO OUR YOUTUBE CHANNEL - new videos posted almost daily

Le Data Mesh est la nouvelle tendance des ces dernières années dans le monde de la data. Initiée par Zhamak Dehghani en 2019, cette notion est trop souvent employée en tant que buzzword, c'est-à-dire qu'on en parle sans trop savoir ce que c'est. Bien que reposant sur des fondamentaux accessibles à tous, la notion reste encore trop perméable aux interprétations et donc incompréhensions. Avec Yoann Benoit, CTO de Hymaïa, une société de conseil spécialisé dans la data, nous vous proposons de revenir sur la genèse de ce concept pour ensuite évoluer avec des bases solides sur des possibilités d'implémentation.    Pour en savoir plus sur le Data Mesh : https://www.hymaia.com/qu-est-ce-que/data-mesh

Zhamak Dehghani (CEO & Founder of Next Data) discusses why she started Next Data, and how her company will finally help Data Mesh become a reality. Next Data: https://www.nextdata.com/ Announcement: https://medium.com/@zhamakd/why-we-started-nextdata-dd30b8528fca #datamesh #data

"Was tun wir, wenn Agilität nicht mehr reicht...?" diese Frage bringt Sebastian heute in die Unlock the Future Runde mit. Gerade wenn wir versuchen komplexe Datenprodukte zu entwickeln, steigt die organisatorische Komplexität rasant an. Die moderne Software-Entwicklung hat darauf die Antwort des Domain Driven Design gefunden. Im Bereich Data Science scheint gerade der Data Mesh Ansatz sehr vielversprechend zu sein. Die Erfinderin des dieses Konzeptes ist Zhamak Dehghani. Hier könnt ihr ein Interview von Tim Berglund (confluent) mit ihr dazu finden: https://www.youtube.com/watch?v=CD565M8Eta8 Seid gern dabei, wenn wir in dieser Folge gemeinsam nachdenken und versuchen Antworten zu finden. Du willst mehr zur Data Science Mania erfahren? Dann schau hier rein: http://www.datasciencemania.de

Data Mesh has caught the technology and data industry by storm and is easily one of the hottest topics today. Oftentimes, the harder parts of Data Mesh don't get as much attention or coverage. This was an open session at the Utah Data Engineering Meetup (November 2022) to ask Zhamak about the hard parts of data mesh that have inspired her to start a company to address them head-on.

Today's digital economy is more competitive than ever, and making smart data decisions can be what sets the leaders apart from the rest of the field. With new technologies now democratizing data at an accelerated pace, how can companies ensure that their data strategy is helping them stay ahead of the curve? As our third season comes to a close, Cindi takes a look back at some of our most insightful conversations to lay out six essential rules that every Data Chief should follow to dominate the decade of data.Tune in to learn:Rule 1: Leverage the best-of-breed from the modern data stack (02:30)Rule 2: Empower everyone with true self-service analytics (10:58)Rule 3: Drive actions with operationalized insights (18:44)Rule 4: Build a flexible data foundation (27:26)Rule 5: Utilize third-party data to build a 360-degree view (35:07)Rule 6: Deliver engaging data-driven UX (38:47)MentionsGilead Sciences' Murali Vridhachalam and ZS Associates' Mahmood Majeed on the Modern Data Stack and Data MeshThe Modern Milkman's Chief Strategy Officer, John Hughes on Using Data to Save Our Oceans from Plastic Afterpay's Nitish Mathew on Prioritizing Customer Needs, Balancing Governance and Freedom, and Giving Your Team PurposeServiceNow's Vijay Kotu on the Power of Micro-Decisions and Aligning Data Strategy to Business StrategyThree Must Read Data and Analytics Books with Tim Harford, Zhamak Dehghani, and Brent DykesUnilever's Vandana Khanna and Deeksha Singh on Demystifying Data, Dreaming Bigger, and Inspiring Business Users with The Art of the PossibleOpenTable's Grant Parsamyan on How Data and Analytics is Helping the Restaurant Industry Rebound from COVID-19Get even more insights from data and analytics leaders on The Data Chief. https://www.thoughtspot.com/data-chief Mission.org is a media studio producing content for world-class clients. Learn more at https://mission.org.

“If you want to unlock the value of your data by generating data-driven values, and you want to do it reliably and resiliently at scale, then you need to consider data mesh." Zhamak Dehghani is the author of the “Data Mesh” book. In this episode, we discussed in-depth about the data mesh, a concept she founded in 2018, which has then been becoming an industry trend. We started our conversation by discussing the current challenges working with data, such as the data centralization approach and why the current data tools are still inadequate. Zhamak then described data mesh and why organizations should adopt it to generate data-driven values at scale. Zhamak then explained the 4 principles of data mesh, which include domain ownership, data as a product, the self-serve data platform, and the federated computational governance. Listen out for: Career Journey - [00:06:49] Challenges Working with Data - [00:10:19] Centralization of Data - [00:13:53] Why Current Tools Not Adequate - [00:16:00] Data Mesh & Its Drivers - [00:19:32] Principle of Domain Ownership - [00:25:54] Principle of Data as a Product - [00:35:57] Principle of The Self-Serve Data Platform - [00:40:51] Principle of Federated Computational Governance - [00:46:01] 3 Tech Lead Wisdom - [00:52:23] _____ Zhamak Dehghani's Bio Zhamak Dehghani works as the CEO and founder of a stealth tech startup reimagining the future of data developer experience. She founded the concept of Data Mesh in 2018 and since has been implementing the concept and evangelizing it with the wider industry. She is the author of Architecture the Hard Parts and Data Mesh books. Zhamak serves on multiple tech advisory boards. She has worked as a technologist for over 24 years and has contributed to multiple patents in distributed computing communications. She is an advocate for the decentralization of all things, including architecture, data, and ultimately power. Follow Zhamak: Twitter – @zhamakd LinkedIn – linkedin.com/in/zhamak-dehghani Our Sponsors Mental well-being is a silent pandemic. According to the WHO, depression and anxiety cost the global economy over USD 1 trillion every year. It's time to make a difference! Learn how to enhance your lives through a master class on mental wellness. Visit founderswellbeing.com/masterclass and enter TLJ20 for a 20% discount. The iSAQB® Software Architecture Gathering is the international conference highlight for all those working on solution structures in IT projects: primarily software architects, developers, professionals in quality assurance, and also system analysts. A selection of well-known international experts will share their practical knowledge on the most important topics in state-of-the-art software architecture. The conference takes place online from November 14 to 17, 2022, and we have a 15% discount code for you: TLJ_MP_15. DevTernity 2022 (devternity.com) is the top international software development conference with an emphasis on coding, architecture, and tech leadership skills. The lineup is truly stellar and features many legends of software development like Robert "Uncle Bob" Martin, Kent Beck, Scott Hanselman, Venkat Subramaniam, Kevlin Henney, and many others! The conference takes place online, and we have the 10% discount code for you: AWSM_TLJ. Like this episode? Subscribe on your podcast app. Follow @techleadjournal on LinkedIn, Twitter, and Instagram. Pledge your support by becoming a patron. For episode show notes, visit techleadjournal.dev/episodes/107.

With the data space more complex than ever, many people throw their hands in the air, and use solutions that fix many (but not all) of their problems. Today's guest saw that complexity, and leaned into it. Zhamak Dehghani is the creator of the data mesh. Where others saw chaos, she saw an opportunity. Zhamak dives into decentralization, all things data mesh, how data architecture can empower your workers, and much more.--------“Seeing the real world problems made me curious about the data space—scratching the surface and seeing the discord between the reality of the complex world we're living in with data. The solutions weren't up to the task for dealing with that complexity, so it got me to look further for a solution.” — Zhamak Dehghani--------Time Stamps* (0:00) A brief history of data architecture* (2:45) An intro to the data mesh* (13:08) How analytics impact data engineers* (18:54) The socio-technical approach* (23:26) Getting started in the data mesh* (28:59) Data mesh vs. Data fabric--------SponsorThis podcast is presented by Alation.Hear more radical perspectives on leading data culture at Alation.com/podcast--------LinksConnect with Zhamak on LinkedInCheck out Zhamak's book Data Mesh

Jon Krohn speaks with Zhamak Dehghani, the empathetic technologist who coined the term “data mesh”. They explore what a data mesh is, and how its approach toward secure interconnectivity will help solve a roster of data-led business problems. In this episode you will learn: • The importance of data meshes [3:29] • How standardizing database interfaces helps tech giants like Amazon [6:40] • Current challenges with data meshes [9:33] • How data meshes give users the freedom to work with data [17:09] • The missing piece of the puzzle for data meshes [22:11] • How data meshes connect with the metaverse and Web3 [33:18] • The times when data meshes aren't fit for purpose [42:24] Additional materials: www.superdatascience.com/609

It is once again that time of year when our host, Cindi Howson shares her favorite data and analytics book recommendations. In this special annual episode, we feature three of the industry's top data writers, thinkers, and fellow podcasters. Tim Harford comes to the conversation with his new book, The Data Detective, and big-picture ideas about how traits like curiosity serve data scientists so well. Zhamak Dehghani shares her concept of The Data Mesh, especially as it relates to sharing data across business verticals. Finally, in his book, Effective Data Storytelling, Brent Dykes compels readers to think carefully about the way they craft the message or narrative around the data they're interpreting. Tune in to learn:Making sense of the world through a data lens with Tim Harford (05:18)Tim's favorite of the ten data commandments (07:11)Guard against using data as a control mechanism (15:14)How does curiosity create a healthier relationship with data? (19:57) Data Mesh with Zhamak Dehghani (32:31)Blending centralized and decentralized schools of thought (36:39)Data Mesh isn't for everyone (48:11)How do you begin your data mesh transformation? (53:28)Brent Dykes and the importance of data storytelling (1:02:17)Why do data scientists need to be better communicators? (1:11:13)The Venn diagram of data storytelling (1:15:14)Mentions:The Data Detective: Ten Easy Rules to Make Sense of Statistics by Tim Harford Data Mesh: Delivering Data-Driven Value at Scale by Zhamak Dehghani Effective Data Storytelling: How to Drive Change with Data, Narrative, and Visuals by Brent Dykes Get even more insights from data and analytics leaders like Tim, Zhamak, and Brent on The Data Chief. Mission.org is a media studio producing content for world-class clients. Learn more at mission.org.

We talked about: Zhamak's background What is Data Mesh? Domain ownership Determining what to optimize for with Data Mesh Decentralization Data as a product Self-serve data platforms Data governance Understanding Data Mesh Adopting Data Mesh Resources on implementing Data Mesh Links: Free 30-day code from O'Reilly: https://learning.oreilly.com/get-learning/?code=DATATALKS22 Data Mesh book: https://learning.oreilly.com/library/view/data-mesh/9781492092384/ LinkedIn: https://www.linkedin.com/in/zhamak-dehghani ML Zoomcamp: https://github.com/alexeygrigorev/mlbookcamp-code/tree/master/course-zoomcamp Join DataTalks.Club: https://datatalks.club/slack.html Our events: https://datatalks.club/events.html

Data mesh has the potential to solve a big data problem. The book, Data Mesh: Delivering Data-Driven Value at Scale, authored by Zhamak Dehghani, the creator of data mesh, is a guide for practitioners, architects, technical leaders, and decision makers on their journey from traditional big data architecture to distributed and multidimensional approach to analytical data management. In this episode, Zhamak and Jesse talk about the book and unravel some of the complex and critiqued areas of data mesh, and shed light on how to succeed with data mesh.   This episode is part of a trilogy of conversations with Zhamak and Jesse, as they delve into the different sides of data mesh. Before you dive in, do make sure you've listened to their first and second conversations.     More about our host, Jesse Anderson Read the transcript of this episode Listen to our first conversation with Zhamak from 2021, Data for Everyone, or her second conversation with Jesse, Reflections.  Get Zhamak's book Data Mesh: Delivering Data-Driven Value at Scale Connect with us on social media: Twitter, LinkedIn Find book recommendations and more resources for data professionals at https://dreamteam.soda.io From Soda, the provider of data reliability tools and observability platform to enable data teams to discover, prioritize, and resolve data issues.

Zhamak Dehghani is the founder of data mesh and author of the book, Data Mesh: Delivering Data-Driven Value at Scale. Zhamak returns to the Soda Podcast. In part one, Zhamak and Jesse will delve into data mesh and how that is redefining how we manage data. They talk through the reality, the dream, and the execution, and bring us up to speed on what's been happening since they met in season one. Part two - coming soon - will deep-dive into Zhamak's book. More about our host, Jesse Anderson Read the transcript of this episode Get Zhamak's book Data Mesh: Delivering Data-Driven Value at Scale Connect with us on social media: Twitter, LinkedIn Find book recommendations and more resources for data professionals at https://dreamteam.soda.io From Soda, the provider of data reliability tools and observability platform to enable data teams to discover, prioritize, and resolve data issues.

Zhamak Dehghani is the founder of data mesh and author of Data Mesh: Delivering Data-Driven Value at Scale. We're delighted to welcome Zhamak back to the Soda Podcast to talk about the reality, the dream, and the execution of data mesh, writing a book, and life in general.   More about our host, Jesse Anderson Read the transcript of this episode Connect with us on social media: Twitter, LinkedIn, Facebook Find book recommendations and more resources for data professionals at https://dreamteam.soda.io From Soda, the provider of data reliability tools and observability platform to enable data teams to discover, prioritize, and resolve data issues.

This bonus episode features conversations from season 1 and 2 of the Open||Source||Data podcast. In this episode, you'll hear from Zhamak Dehghani, Director of Emerging Technologies at ThoughtWorks North America; David Thomas, Principal at Deloitte; and Shirshanka Das, Founder of LinkedIn DataHub and Acryl Data.Sam sat down with each guest to discuss data meshes, fabrics, and discovery. You can listen to the full episodes from Zhamak Dehghani, David Thomas, and Shirshanka Das by clicking the links below.-------------------Episode Timestamps:(00:36): Zhamak Dehghani(01:41): David Thomas(02:43): Shirshanka Das-------------------Links:Listen to Zhamak's episodeListen to David's episodeListen to Shirshanka's episode 

Zhamak Dehghani joins the show to chat about all things Data Mesh, and her new O'Reilly book by the same title. This is a rare opportunity to learn all about Data Mesh from Zhamak in this unscripted and candid live chat. Q&A welcome Streamed live on LinkedIn and YouTube #datamesh #dataengineering #data Note - this was initially supposed to be on the Monday Morning Data Chat, but moved to the TGIF, Let's Talk Data Show! (podcast coming soon). Enjoy!

The introduction and socialization of data mesh has caused practitioners, business technology executives and technologists to pause and ask some probing questions about the organization of their data teams, their data strategies, future investments and their current architectural approaches. Some in the technology community have embraced the concept, others have twisted the definition while still others remain oblivious to the momentum building around data mesh. We are in the early days of data mesh adoption. Organizations that have taken the plunge will tell you aligning stakeholders is a non-trivial effort. But one that is necessary to break through the limitations that monolithic data architectures and highly specialized teams have imposed over frustrated business and domain leaders. However, practical data mesh examples often lie in the eyes of the implementer and may not strictly adhere to the principles of data mesh. Part of the problem is the lack of open technologies and standards that can accelerate adoption and reduce friction.This is the topic of today's Breaking Analysis where we investigate some of the key technology and architectural questions around data mesh. To do so, we welcome back the founder of data mesh and Director of Emerging Technologies at ThoughtWorks, Zhamak Dehghani. 

This interview was recorded for the GOTO Book Club.gotopia.tech/bookclubZhamak Dehghani - Author of "Data Mesh" & Director of Emerging Technologies at ThoughtworksSamia Rahman - Director of Data & AI at SeagenDESCRIPTIONHow can modern organizations handle their data in a way that delivers value at scale? Zhamak Dehghani, author of “Data Mesh: Delivering Data-Driven Value at Scale,” covers the key principles of data mesh and how it can help organizations move beyond the data lake to provide meaningful insights. She's joined by Samia Rahman, director of data and AI at Seagen, as they also explore the concept of the earliest explorable data.The interview is based on Zhamak's book "Data Mesh".Read the full transcription of the interview hereRECOMMENDED BOOKSZhamak Dehghani • Data MeshZhamak Dehghani, Neal Ford, Mark Richards & Pramod Sadalage • Software ArchitectureSam Newman • Monolith to MicroservicesSam Newman • Building MicroservicesSandeep Uttamchandani • The Self-Service Data RoadmapPiethein Strengholt • Data Management at ScaleMartin Kleppmann • Designing Data-Intensive ApplicationsTwitterLinkedInFacebookLooking for a unique learning experience?Attend the next GOTO conference near you! Get your ticket at gotopia.techSUBSCRIBE TO OUR YOUTUBE CHANNEL - new videos posted almost daily.Discovery MattersA collection of stories and insights on matters of discovery that advance life...Listen on: Apple Podcasts Spotify Health, Wellness & Performance Catalyst w/ Dr. Brad CooperLooking for a catalyst to optimize your health, wellness & performance? You've found it!!Listen on: Apple Podcasts Spotify

Twitch channel with the full show: https://www.twitch.tv/terminusdbAlso check out Matthijs' coding sessions on twitch - they are a delight, you won't regret.As I don't have any documentation on cardinality, I will just plug this new blog instead:8 Reasons to Version Control your database: https://terminusdb.com/blog/8-reasons-to-version-control-your-database/Data mesh by it's founder Zhamak Dehghani - this is an essential introduction to the ideas: https://martinfowler.com/articles/data-mesh-principles.htmlThis is a good engineering intro to data mesh. I appreciated its straightforward descriptions: https://www.datamesh-architecture.com/Three laws of robotics: https://en.wikipedia.org/wiki/Three_Laws_of_Robotics I, Robot (by Asimov - not the 'adaptation' Will Smith movie): https://en.wikipedia.org/wiki/I,_RobotThe robot vacuum cleaner company is called 'I, Robot': https://www.irobot.ie/roomba  

The data mesh is a thesis that was presented to address the technical and organizational challenges that businesses face in managing their analytical workflows at scale. Zhamak Dehghani introduced the concepts behind this architectural patterns in 2019, and since then it has been gaining popularity with many companies adopting some version of it in their systems. In this episode Zhamak re-joins the show to discuss the real world benefits that have been seen, the lessons that she has learned while working with her clients and the community, and her vision for the future of the data mesh.

“Rabbit, Rabbit!" - a superstition found in Britain and North America wherein a person says or repeats the words "rabbit", "rabbits" and/or "white rabbits" aloud upon waking on the first day of a month, to ensure good luck for the rest of it. Delivered on the first day of the month, this podcast series dives into various Database Performance topics. Enjoy! RR Episode 08 – The Grateful Data, Live and on Tour In this Episode, Quest Senior Consultants, Dave Orlandi and Amit Parikh, share their gratitude for being part of the ever-evolving data landscape. Into the Rabbit Hole (related resources) – Data Mesh: A New Paradigm for Data Management by Dave Vellante referencing the works of author and technologist, Zhamak Dehghani. https://siliconangle.com/2021/10/27/data-mesh-new-paradigm-data-management/ 

A thoughtful conversation examining the paradigm shift and the unlearning required to build a data-driven organization at scale. Hear Zhamak, the founder of data mesh, discuss what data mesh is and what it isn't. This conversation provides insights, failsafe tips, and inspiration to use data to augment and improve business and life.

Zhamak Dehghani is a software engineer, architect, and founder of data mesh. Zhamak founded the concept of data mesh as the paradigm shift needed in how we manage data at scale. Meet Zhamak and learn how data mesh will help organizations achieve data-driven value at scale.

Data mesh is a new way of thinking about how to use data to create organizational value. Leading edge practitioners are beginning to implement data mesh in earnest. Importantly, data mesh is not a single tool or a rigid reference architecture. Rather it's an architectural and organizational model that is designed to address the shortcomings of decades of data challenges and failures. As importantly, it's a new way to think about how to leverage and share data at scale across an organization and ecosystems. Data mesh in our view will become the defining paradigm for the next generation of data excellence. In this Breaking Analysis we welcome the founder and creator of data mesh, author, thought leader, technologist Zhamak Dehghani, who will help us better understand some of core principles of data mesh and the future of decentralized data management. With practical advice for data pros who want to create the next generation of data-driven organizations.

“What is a data mesh?” It's one of the most frequently asked questions on this podcast, and is almost a microcosm for data and analytics as a whole. It may be the next big thing. It may be yet another passing fad in architectural design. We compared and contrasted it with data fabric in Episode 32 and spoke with Zhamak Dehghani, the founder of the data mesh paradigm in Episode 44. We even hosted a special panel discussion dedicated to data mesh. After all these discussions, we are still trying to figure out what it is. So, we're settling this the only way we know how: a good old-fashioned debate. Only one side can win. This episode will feature: The definitive(ish) definition of data mesh A look at why organizations struggle to understand and implement it What is your data “hot take” that everyone else disagrees with?

The data mesh architectural paradigm shift is all about moving analytical data away from a monolithic data warehouse or data lake into a distributed architecture—allowing data to be shared for analytical purposes in real time, right at the point of origin. The idea of data mesh was introduced by Zhamak Dehghani (Director of Emerging Technologies, Thoughtworks) in 2019.  Here, she provides an introduction to data mesh and the fundamental problems that it's trying to solve. Zhamak describes that the complexity and ambition to use data have grown in today's industry. But what is data mesh? For over half a century, we've been trying to democratize data to deliver value and provide better analytic insights. With the ever-growing number of distributed domain data sets, diverse information arrives in increasing volumes and with high velocity. To remove the friction and serve the requirement for data to be consumed by operational needs in various use cases, the best way is to mesh the data. This means connecting data through a peer-to-peer fashion and liberating data for analytics, machine learning, serving up data-intensive applications across the organization, and more. Data mesh tackles the deficiency of the traditional, centralized data lake and data warehouse platform architecture. The data mesh paradigm is founded on four principles: Domain-oriented ownershipData as a productData available everywhere in a self-serve data infrastructureData standardization governanceA decentralized, agnostic data structure enables you to synthesize data and innovate. The starting point is embracing the ideology that data can be anywhere. Source-aligned data should serve as a product available for people across the organization to combine, explore, and drive actionable insights. Zhamak and Tim also discuss the next steps we need to take in order to bring data mesh to life at the industry level.To learn more about the topic, you can visit the all-new Confluent Developer course: Data Mesh 101. Confluent Developer is a single destination with resources to begin your Kafka journey.  EPISODE LINKSZhamak Dehghani: How to Build the Data Mesh FoundationData Mesh 101 CourseSaxo Bank's Best Practices for a Distributed Domain-Driven Architecture Founded on the Data MeshPlacing Apache Kafka at the Heart of a Data Revolution at Saxo BankWatch the video version of this podcastJoin the Confluent CommunityLearn Kafka on Confluent DeveloperLive demo: Event-Driven Microservices with ConfluentUse PODCAST100 to get $100 of Confluent Cloud usage (details)

Zhamak Dehghani, Director of Emerging Technologies at ThoughtWorks joins Dave Vellante for theCUBE on Cloud 2021.

Monolithic applications present challenges for organizations like Saxo Bank, including difficulties when it comes to transitioning to cloud, data efficiency, and performing data management in a regulated environment. Graham Stirling, the head of data platforms at Saxo Bank and also a self-proclaimed recovering architect on the pathway to delivery, shares his experience over the last 2.5 years as Saxo Bank placed Apache Kafka® at the heart of their company—something they call a data revolution. Before adopting Kafka, Saxo Bank encountered scalability problems. They previously relied on a centralized data engineering team, using the database as an integration point and looking to their data warehouse as the center of the analytical universe. However, this needed to evolve. For a better data strategy, Graham turned his attention towards embracing a data mesh architecture: Create a self-serve platform that enables domain teams to publish and consume data assetsFederate ownership of domain data models and centralize oversights to allow a standard language to emerge while ensuring information efficiency Believe in the principle of data as a product to improve business decisions and processes Data mesh was first defined by Zhamak Dehghani in 2019, as a type of data platform architecture paradigm and has now become an integral part of Saxo Bank's approach to data in motion. Using a combination of Kafka GitOps, pipelines, and metadata, Graham intended to free domain teams from having to think about the mechanics, such as connector deployment, language binding, style guide adherence, and data handling of personally identifiable information (PII). To reduce operational complexity, Graham recognized the importance of using Confluent Schema Registry as a serving layer for metadata. Saxo Bank authored schemes with Avro IDL for composability and standardization and later made a switch over to Uber's Buf for strongly typed metadata. A further layer of metadata allows Saxo Bank to define FpML-like coding schemes to specify information classification, reference external standards, and link semantically related concepts. By embarking on the data mesh operating model, Saxo Bank scales data processing in a way that was previously unimaginable, allowing them to generate value sustainably and to be more efficient with data usage. Tune in to this episode to learn more about the following:Data meshTopic/schema as an APIData as a productKafka as a fundamental building block of data strategyEPISODE LINKSZhamak Dehghani Kafka Summit 2021 KeynoteData Mesh 101 CourseData Mesh Principles and Logical ArchitectureSaxo Bank's Best Practices for a Distributed Domain-Driven ArchitectureWatch video version of this podcastJoin the Confluent CommunityLearn Kafka on Confluent DeveloperDemo: Event-Driven Microservices with ConfluentUse PODCAST100 to get $100 of free Confluent Cloud (details)

Data Architecture is evolving and there are many questions with various perspectives. What is the balance between centralization and decentralization? How do you start treating data as a product? How do you incentivize people? What's the role of Data Mesh, Data Fabric, Knowledge Graphs? This special edition of Catalog and Cocktails is the Data Architecture panel from the Knowledge Graph Conference, moderated by Juan Sequeda. Listen and learn from Teresa Tung Chief Technologist of Accenture's Cloud First group , Zhamak Dehghani director of emerging technologies at Thoughtworks and founder of Data Mesh concept, and Jay Yu, Distinguished Architect at Intuit. You can watch the panel here and follow Juan's takeaways.

Modern companies have audacious goals and initiatives to become data-driven, but many still suffer a disconnect. Despite increased investment, expanded commitment and more tools at their disposal, how can companies translate gathering tons of data into becoming a data-driven business? In this episode, Zhamak Dehghani, Director of Emerging Technologies, North America explains how Data Mesh brings people closer to data and insights.

Mesh is everywhere. It’s in our clothes, our fishing nets, our Wifi networks, and now our data architecture. But what is a data mesh? Do you need one? And if so, how do you start? Zhamak Dehghani is the director of Emerging Technologies at Thoughtworks and the leading expert on data mesh. We’ll chat about the emergence of the data mesh as a concept, why the approach works for eliminating architectural silos, and how it's producing more data-driven cultures. Other topics include: Key tools, technologies, and skills for adopting a data mesh What to do if you’re data mesh curious Inspiring architectural designs outside the data space

ThoughtWorks' Director of Next Tech Incubation Zhamak Dehghani joins us for a round on Cocktails and talks about how she founded Data Mesh and the concepts behind it. She expounds on the decentralization and productization of data and discusses the importance of the discovery of new paradigm shifts in the industry.

Barry O’Reilly and this week’s guest, Zhamak Dehghani met 10 years ago when they worked together at ThoughtWorks. Zhamak is currently the Director of Emerging Technologies at ThoughtWorks and the creator of Data Mesh, which Barry describes as “one of the most exciting paradigm shifts in how we manage data at scale.” He and Zhamak discuss why traditional data architecture models are failing and how applying product thinking principles to data management is a way to harvest the data’s full potential. “This show,” Barry remarks, “is for those who are curious to understand how to bring the convergence of product thinking, data management, and distributed systems development together to create platforms and products of the future.” Early Values Zhamak has always believed in distribution of responsibility and decentralization of ownership. She finds that these design principles are more compatible with real life. Colleagues taught her the Unix philosophy early in her career which now forms the basis of her data management approach. “They taught me those wonderful ideas to build systems and programs that do one thing and one thing really well. But most importantly they work together really well,” Zhamak says. “‘Simple is beautiful and beauty is the truth’... Reduce systems to their simple principles; then together can emerge complex behaviors.” She saw an opportunity to bring the UNIX principles to data. Challenging Assumptions It often takes someone new to a system to point out obvious flaws to long-time practitioners. Zhamak says that when she came into the world of big data, she was agnostic to the accepted assumptions, so she felt free to challenge them and conceive a different paradigm. For some reason when it comes to data, people eschew UNIX principles and see it as something to be centralized. Unsurprisingly, a data lake becomes monolithic and departments become siloed. Reimagining the world of data requires a new language, she points out: “The moment you need to imagine something different you need to use a very different language.” Instead of seeing data as an asset - which you want to hoard and get more of - Zhamak advocates that data can be seen as a product which should be used to serve internal and external customers. Barry adds that the idea of the single source of golden data makes companies unable to move as they get bigger. Move to Product Thinking Barry comments that the shift towards product thinking started with Amazon. Their monolithic database was preventing them from scaling. “They realized that they needed to create these smaller, more autonomous units that had the capabilities to build things just like product teams. This is where this notion started to emerge from changing the organizational design... both technically and just how teams would work together,” Barry says. In this new way of working, teams could experiment and own outcomes. They could make small, quick changes and see the effects.   What is Data Mesh? Instead of trying to fit data into a mold, Zhamak feels that its dynamism should be embraced. “Create an architecture and ownership of data that starts with the assumption that data can be useful and shareable and trustworthy right at the point of origin; and then allow for different domains and different aggregations, different projections to get created as a mesh picture,” she posits. She explains how this new view of data impacts ways of working and the type of platform a company would create. The four principles of the Data Mesh philosophy are, “domain ownership of the data; data as a product; self serve data platform to enable autonomous teams; and a federated computational governance to balance the interoperability of a decentralized world with the trust and security built-in.” Read the rest of the show notes on BarryO'Reilly.com Resources Zhamak Dehghani on LinkedIn | Twitter Data Monolith to Data Mesh article Data Mesh Principles article

In this episode we are going to talk about a trending topic in Data Analytics - Data Mesh This movement started a couple of years ago with a blog post from Zhamak Dehghani where she describes in detail what and why to implement a Data Mesh, what to watch for, etc. Felipe and Mike are going to talk about our experience in the subject and what to think about when starting this discussion. Also the differences between what we described before in a Data Lakehouse/Data Lake. Hope you enjoy! Reference: How to Move Beyond a Monolithic Data Lake to a Distributed Data Mesh (martinfowler.com)

Beyond The Data Lake was Director of Emerging Technologies at ThoughtWorks, Zhamak Dehghani's 2017 paper that was a guiding light for Sam Ramji at another point in his career. Listen to how a Data Mesh allows composition of multi-model data across an organization and beyond. See omnystudio.com/listener for privacy information.

Zhamak Dehghani (@zhamakd, Portfolio Tech Director @ThoughtWorks) talks about the concepts behind Data Mesh, the challenges and problems of Data Lakes / Data Warehouses, and how Cloud-native principles can be applied to Data. SHOW: 459SHOW SPONSOR LINKS:strongDM HomepageStart your free 14 day trial today at: strongdm.com/cloudcastDatadog Security Monitoring Homepage - Modern Monitoring and AnalyticsTry Datadog yourself by starting a free, 14-day trial today. Listeners of this podcast will also receive a free Datadog T-shirtCLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotwPodCTL Podcast is Back (Enterprise Kubernetes) - http://podctl.comSHOW NOTES:ThoughtWorks Technology RadarData Mesh (from Technology Radar)How to Move Beyond a Monolithic Data Lake to a Distributed Data MeshTopic 1 - Welcome to the show. We were introduced to you through the O’Reilly events, but you’ve been involved in software development and architecture for quite a while. Tell us a little bit about your background and your focus areas at ThoughtWorks.Topic 2 - About a year ago, you introduced this new concept called “Data Mesh”. Before we get into that, give us a little bit of background on the problems that previous generations of Data Warehouses or Data Lakes created. Topic 3 - Lets begin to walk through how Data Mesh is different from Data Lake. We’re not talking about just dumping all the various data sources into one “pool”, there’s a concept of “domains” within this big pool of data. What are the new concepts of source and consumption?Topic 4 - Explain the concept of how pipelines are tied into Data Mesh and how this allows the creation of new products/features from the Data Mesh.Topic 5 - You talk about the data being truthful, and then you bring an SRE concept of SLO into the truthfulness of the data. Explain how that might work? Topic 6 - Once a Data Mesh is in place, what are the “roles” (or teams) that have specific tasks, and who are the typical consumers of the Data Mesh platform?FEEDBACK?Email: show at thecloudcast dot netTwitter: @thecloudcastnet

When it comes to cloud, you may find that for some applications — ones you intend to maintain over a long period of time — you have to account for the possibility of changing providers in the future. Some vendors promise to make that a seamless transition — but in so doing, you might be losing out on some of the rich functionality that the cloud providers can offer. Our regular podcast host, Zhamak Dehghani talks to ThoughtWorks Australia’s Head of Technology, Scott Shaw about the cost of multi-cloud portability and explores how businesses can calculate the risks and rewards of multi-cloud.

In this podcast, Daniel Bryant sat down with Zhamak Dehghani, principal consultant, member of technical advisory board, and portfolio director at ThoughtWorks. Topics discussed included: the motivations for becoming a data-driven organization; the challenges of adapting legacy data platforms and ETL jobs; and how to design and build the next generation of data platforms using ideas from domain-driven design and product thinking, and modern platform principles such as self-service workflows. Why listen to this podcast: - Becoming a data-driven organization remains one of the top strategic goals of many organizations. Being able to rapidly run experiments and efficiently analyse the resulting data can provide a competitive advantage. - There are several “architecture failure modes” within existing enterprise data platforms. They are centralized and monolithic. The composition of data pipelines are often highly-coupled, meaning that a change to the data format will require a cascade of changes throughout the pipeline. And finally, the ownership of data platforms is often siloed and hyper-specialized. - The next generation of enterprise data platform architecture requires a paradigm shift towards ubiquitous data with a distributed data mesh. Instead of flowing the data from domains into a centrally owned data lake or platform, domains need to host and serve their domain datasets in an easily consumable way. - Domain data teams must apply product thinking to the datasets that they provide; considering their data assets as their products, and the rest of the organization's data scientists, ML and data engineers as their customers. The key to building the data infrastructure as a platform is (a) to not include any domain specific concepts or business logic, keeping it domain agnostic, and (b) make sure the platform hides all the underlying complexity and provides the data infrastructure components in a self-service manner. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/39exTWl You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/39exTWl

For any large, multinational enterprise, there’s a dilemma at the heart of their software architecture. Centralization promises to save costs, providing a standardized template for how to do things and not waste effort reinventing the wheel. But local markets have unique characteristics, whether that’s customer behavior, regulations or the competitive landscape. In this episode, our co-hosts Alexey Boas and Zhamak Dehghani are joined by Sriram Narayan, an advisory consultant for ThoughtWorks India. Together, they explore the price of reuse: the challenges and trade offs for architects that arise from a centralized blueprint for IT.

Traditional, centralized approaches to infrastructure management risk creating organizational friction and bottlenecks for dev teams. By defining a standardized tech stack, you immediately make it harder to satisfy your teams’ diverse needs. A self-serve approach can enable teams to tailor infrastructure to their particular needs. But how can you deliver a self-serve infrastructure without paving the way for chaos and ensure the appropriate guardrails are in place? In this episode, our regular co-hosts Neal Ford and Mike Mason are joined by Evan Bottcher, Head of Technology, ThoughtWorks Australia and Zhamak Dehghani, Technology Principal, ThoughtWorks North America, to explore how to benefit from self-service options.

Everyone knows the adage: move fast and break things. But there are a few fields of endeavour where it is quite so apt as the world of drones. In this episode, our co-hosts Neal Ford and Zhamak Dehghani talk to data scientist, Emily Gorcenski and head of product innovation, Jeremy Abbett, both from ThoughtWorks Germany about their project to build intelligent, autonomous drones. Together they explore the ramifications of writing code in a new, constrained hardware environment, and where faulty code can have huge project ramifications. And yes, there have been a few breakages. 

Over the years, there have been numerous approaches to data management: from data warehouses and data marts to data lakes — collating data in a central place. Today, we see many failure modes when it comes to building big data platforms, with organizations stuck in building data architectures such as data lakes that never deliver business value. In our latest episode, we explore the ideas of data meshes, an alternative approach to serve and surface data organizationally. Our regular co-hosts Mike Mason and Neal Ford talk to Ken Collier, Head of Data Science and Engineering at ThoughtWorks, and Zhamak Dehghani, one of our regular co-hosts and also a Principal Consultant, with a focus on distributed systems architecture.

The technology choices that businesses make are fast becoming key factors in creating great products and great customer experiences. But who should be responsible for those choices: the CIO or the CTO. Our regular co-hosts Zhamak Dehghani and Mike Mason are joined by ThoughtWorks’ very own CTO (and erstwhile podcast host) Rebecca Parsons and principal consultant, Marcelo de Santis, who has previously been both a CIO and CTO at large multinational companies. Together they explore how modern businesses can get the technology leadership they need.

Who wouldn’t want to empower individuals and teams to solve problems as they encounter them? It’s a natural instinct for many developer-led organizations. But as phenomena like Shadow IT or skunkworks have shown, there can be a downside: a lack of coordination over what gets tackled and maintaining solutions over the long term. In this episode, our co-hosts, Rebecca Parsons and Zhamak Dehghani, are joined by Andy Yates, Head of Strategy at ThoughtWorks TechOps, to explore the idea of the Citizen Developer. They look at ways of supporting problem-solving go-getters, while giving them the tools to ensure their efforts are aligned to business needs.

The current trend in data management is to centralize the responsibilities of storing and curating the organization's information to a data engineering team. This organizational pattern is reinforced by the architectural pattern of data lakes as a solution for managing storage and access. In this episode Zhamak Dehghani shares an alternative approach in the form of a data mesh. Rather than connecting all of your data flows to one destination, empower your individual business units to create data products that can be consumed by other teams. This was an interesting exploration of a different way to think about the relationship between how your data is produced, how it is used, and how to build a technical platform that supports the organizational needs of your business.

The shift from legacy to digital infrastructure is central to many companies’ strategy to compete in a fast-changing world. But at the architectural level, such change can be hard — or at least expensive — to effect. Not everyone wants to break things when they move fast. In this episode, our hosts, Zhamak Dehghani and Mike Mason are joined by Ryan Murray, Director of Digital Platforms at ThoughtWorks, to consider how organizations can transform themselves through a deeper understanding of their systems, the information and their business capabilities — and the interrelation between these things.

At ThoughtWorks, we’ve long been wary of the use of multiple cloud providers for the sake of it. We think there’s a significant upside in using the best provider for particular use cases. But in some regulated environments, organizations are forced to adopt a multicloud approach for highly critical workloads. In this episode, our regular co-hosts, Mike Mason and Zhamak Dehghani are joined by Scott Shaw, Head of Technology for ThoughtWorks Australia and James Lewis, Principal Consultant, ThoughtWorks UK. They explore the challenges of delivering an effective multicloud solution and how to assess the criticality and sensitivity of workloads.

Microservices and containers have kick started a revolution in enterprise architectures — and in the developer experience. In this episode, our host Zhamak Dehghani is joined by Sheroy Maker, head of technology at ThoughtWorks Products, look at how these changes have impacted the ideas of continuous delivery. They explore the challenges such as how to maintain the integrity of complex distributed systems and how to manage deployments of disparate technology stacks.   Learn more about ThoughtWorks at thoughtworks.com

Platform thinking promises to give your business more bang for its technology investment buck, by putting your business priorities at the heart of your approach to delivery. But embracing platform thinking has profound implications for your technical architecture landscape. Join our co-hosts Alexey Bôas and Zhamak Dehghani as they hear from Renan Martins and Luiz Ribeiro, both of whom are Lead Consultants based out of ThoughtWorks Brazil. They take an in-depth look at the practical realities of introducing platforms into organizations with legacy infrastructure and well-established development teams.

The concepts of governance can sometimes conjure up images of groups whose only task is to say no to the rest of the organization. But it needn’t be that way. Our co-hosts, Rebecca Parsons and Mike Mason are joined by Jonny LeRoy, Head of Technology for ThoughtWorks North America and Zhamak Dehghani — also one of our regular co-hosts, but here for her expertise in distributed systems architecture. Together they’re exploring new ways of thinking about architectural governance that help the business and IT.

We’re increasingly seeing a trend of organizations exposing events — particularly business domain events — before knowing who the consumers are or what the specific applications are, in the hope that people elsewhere in the organization can discover these events and create value, without us directly orchestrating it. But creating big upfront architectures can make developers nervous — not least about costs. Neal Ford and Zhamak Dehghani are joined by Erik Dörnenburg, Head of Technology at ThoughtWorks Germany, and Evan Bottcher, Technology Lead in ThoughtWorks Australia.   https://www.thoughtworks.com/podcasts

Serverless has become the buzzword du jour. But what does it mean? What are the implications for your enterprise applications when you’re using services where you’re not responsible for the infrastructure that they run on? How do AWS Lambda, Azure Functions and GCP Cloud Functions fit in To explore these issues, our co-hosts Mike Mason and Zhamak Dehghani are joined by Paula Paul, a Tech Principal at ThoughtWorks and Mike Roberts, an external cloud engineering consultant and former ThoughtWorker.

Microservices has emerged as an important architectural choice for the enterprise. But the ecosystem to support microservices is still being built. In this episode, co-hosts Zhamak Dehghani and Mike Mason are joined by James Lewis — author of one of the seminal articles on microservices — and Lakshminarasimhan Sudarshan, both Principal Consultants at Thoughtworks. They explore the state of microservices today, the pitfalls and anti-patterns (like distributed monolith) we see emerging and how microservices is being used successfully.   thoughtworks.com/podcasts

RESTful APIs quickly established themselves as useful architectural style for replatforming legacy systems with web-based ones. But increasingly, developers are running into the boundaries of deploying REST and API purity — confronting real world issues such as the rapid evolution of APIs for frontend layers where connectivity is patchy and where APIs should meet the changing visual representation and behavior of the frontend, where we increasingly see the adoption of alternatives such as GraphQL.   In this episode, co-hosts Mike Mason and Zhamak Dehghani are joined by Brandon Byars, Market Technical Principal at ThoughtWorks North America and James Gregory, Tech Lead at ThoughtWorks Australia to explore the current state of API design and look at the impact of event-based architectures.

Martin Fowler chats about the work he’s done over the last couple of years on the rewrite of the original Refactorings book. He discusses how this thought process has changed and how that’s affected the new edition of the book. In addition to discussing Refactors, Martin and Wes discuss his thoughts on evolutionary architecture, team structures, and how the idea of refactors can be applied in larger architecture contexts. Why listen to this podcast: - Refactoring is the idea of trying to identify the sequence of small steps that allows you to make a big change. That core idea hasn’t changed. - Several new refactorings in the book deal with the idea of transforming data structures into other data structures, Combine Functions into Transform for example. - Several of the refactorings were removed or not added to the book in favor of adding them to a web edition of the book. - A lot of the early refactorings are like cleaning the dirt off the glass of a window. You just need them to be able to see where the hell you are and then you can start looking at the broader ones. - Refactorings can be applied broadly to architecture evolution. Two recent posts How to break a Monolith into Microservices, by Zhamak Dehghani, and How to extract a data-rich service from a monolith by Praful Todkar on MartinFowler.com deal with this specifically. - Evolutionary architecture is a broad principle that architecture is constantly changing. While related to Microservices, it’s not Microservices by another name. You could evolve towards or away from Microservices for example. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2QbdHej You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/2QbdHej

The Internet of Things promises radical change in how we interact with the world around — whether it’s coffee makers that turn on when our alarm clocks go off or using brain control to actuate objects. Join ThoughtWorks Principal consultant, Zhamak Dehghani and Principal Associate, Alexey Boas as they explore the practical implications of building IoT software and hardware with Software Developer, Charlie Gerard and Consultant Software Engineer Desiree Santos.

On this show, we spend a lot of time talking about CI/CD, data engineering, and microservices. These technologies have only been widely talked about for the last 5-10 years. That means that they are easy to adopt for startups that get founded in the last 5-10 years, but not necessarily for older enterprises. Within a The post How to Change an Enterprise’s Software and Culture with Zhamak Dehghani appeared first on Software Engineering Daily.