Podcasts about data observability

Why is Data Observability getting so much attention? Last week at Gartner Data & Analytics Summit in Orlando, I hosted Co-Founders of definity, Roy Daniel and Tom Bar-Yacov on The Ravit Show to discuss why data observability is a top priority for enterprises today. With Gartner's recent report highlighting capability gaps in the market, we explored what's missing, how full-stack data observability fills these gaps, and the biggest shifts enterprise data teams are seeing. They also shared insights on overlooked areas in data management and the real impact of better visibility and control over data pipelines.An important discussion for any team looking to strengthen data reliability—stay tuned for the full conversation!

This episode is sponsored by Netsuite by Oracle, the number one cloud financial system, streamlining accounting, financial management, inventory, HR, and more.   NetSuite is offering a one-of-a-kind flexible financing program. Head to  https://netsuite.com/EYEONAI to know more.   In this episode of Eye on AI, Craig Smith sits down with Barr Moses, Co-Founder & CEO of Monte Carlo, the pioneer of data and AI observability. Together, they explore the hidden force behind every great AI system: reliable, trustworthy data. With AI adoption soaring across industries, companies now face a critical question: Can we trust the data feeding our models? Barr unpacks why data quality is more important than ever, how observability helps detect and resolve data issues, and why clean data—not access to GPT or Claude—is the real competitive moat in AI today.   What You'll Learn in This Episode: Why access to AI models is no longer a competitive advantage How Monte Carlo helps teams monitor complex data estates in real-time The dangers of “data hallucinations” and how to prevent them Real-world examples of data failures and their impact on AI outputs The difference between data observability and explainability Why legacy methods of data review no longer work in an AI-first world Stay Updated: Craig Smith on X:https://x.com/craigss Eye on A.I. on X: https://x.com/EyeOn_AI     (00:00) Intro (01:08) How Monte Carlo Fixed Broken Data   (03:08) What Is Data & AI Observability?   (05:00) Structured vs Unstructured Data Monitoring   (08:48) How Monte Carlo Integrates Across Data Stacks (13:35) Why Clean Data Is the New Competitive Advantage   (16:57) How Monte Carlo Uses AI Internally   (19:20) 4 Failure Points: Data, Systems, Code, Models   (23:08) Can Observability Detect Bias in Data?   (26:15) Why Data Quality Needs a Modern Definition   (29:22) Explosion of Data Tools & Monte Carlo's 50+ Integrations   (33:18) Data Observability vs Explainability   (36:18) Human Evaluation vs Automated Monitoring   (39:23) What Monte Carlo Looks Like for Users   (46:03) How Fast Can You Deploy Monte Carlo?   (51:56) Why Manual Data Checks No Longer Work   (53:26) The Future of AI Depends on Trustworthy Data 

On this episode of Data Driven, we welcome Barr Moses, CEO and co-founder of Monte Carlo, as she delves into the fascinating world of data observability. Join hosts Frank La Vigne and Andy Leonard as they explore how reliable data is crucial for making sound business decisions in today's tech-driven world. Learn why a simple schema change at Unity resulted in a $100 million loss and how Monte Carlo is developing cutting-edge solutions to prevent similar disasters. From discussions on ensuring data integrity to the intriguing potential of AI in anomaly detection, Barr Moses shares insights that might just redefine your understanding of data's role in business. Tune in for a podcast that not only uncovers the nuances of data reliability but also touches on the quirky side of tech, like why, according to Google, you should never use superglue to fix slipping cheese on your pizza.Moments00:00 Monte Carlo: Data Reliability Innovator05:45 "Data & AI Observability Engineering"09:42 Data Industry's Growing Importance12:00 Cereal Supply Chain Data Optimization16:03 Data Observability and Lineage19:29 GenAI Uncertainties and Latency Concerns23:17 "Human Oversight in AI Accuracy"24:12 Data Observability and Human Role28:01 Adapting to Customer Language33:29 Data and Security Management Alignment35:20 Data Reliability and Observability Challenges38:17 Automated Code Analysis Tool Launch42:29 Data-Inspired Childhood44:12 Passionate About Impactful Work48:52 LinkedIn Security Concerns Highlighted53:19 "Data Observability Insights"

Salma Bakouk (CEO of Sifflet) and I discuss the evolving data and AI landscape, the rise of data observability in the age of AI, balancing personal and professional life as a founder, and much more.

Send Everyday AI and Jordan a text messageYour data is your moat. Everyone's got AI now. Find out how reliable data can make your competitive edge happen. Barr Moses, Co-Founder and CEO of Monte Carlo, joins us to discuss.Newsletter: Sign up for our free daily newsletterMore on this Episode: Episode PageJoin the discussion: Ask Jordan and Barr questions on AI and dataUpcoming Episodes: Check out the upcoming Everyday AI Livestream lineupWebsite: YourEverydayAI.comEmail The Show: info@youreverydayai.comConnect with Jordan on LinkedInTopics Covered in This Episode:1. the Importance of Data2. Challenges and Opportunities in Leveraging Data3. Adoption of Data Practices4. Data Use Case Examples5.Generative AI, LLMs, and Data IntegrationTimestamps:00:00 Empower AI proficiency with daily insights.06:02 Data observability ensures reliability and issue resolution.07:15 Understanding data's importance is crucial for businesses.13:07 Personalized AI relies on unique enterprise data.15:20 Large enterprises struggle with data consistency, smaller teams advantage.19:42 Generative AI analyzes sports data for insights.22:56 Personalized financial products using reliable data.23:56 Credit Karma Intune boosts external and internal productivity.28:02 Peak data reached; synthetic data becomes crucial.30:36 Recap available on your everydayai.com.Keywords:Generative AI, Data Usage, Data Accuracy, High-Quality Data, AI Implementation, Brand Reputation, Small Business Data Management, Data Systems, Trusting Data Sources, Everyday AI Podcast, Microsoft Partnership, Barr Moses, Monte Carlo, Data Downtime, Data Issues, Data Products, Data Observability, Data Adoption Forecast, Smaller Team Advantages, Microsoft WorkLab Podcast, Data Quality Monitor Recommendations, AI and Data Integration, Personalized Financial Products, Coding Assistants, AI for Compliance Reporting, Large Language Models, Synthetic Data, Real-World Data, Data Governance, Data Quality Management. Ready for ROI on GenAI? Go to youreverydayai.com/partner

How is IBM transforming Data and AI in partnership with AWS? I hosted Marcela Vairo, VP of Data & AI, Americas, IBM, on The Ravit Show at AWS re: Invent!We covered some exciting topics around Data Quality, Data Observability, and IBM's latest innovations in Data & AI. Marcela shared insights from her recent roundtable discussion at the event and highlighted how IBM's partnership with AWS is creating new opportunities for businesses leveraging cloud-based data solutions.Key highlights from our conversation:-- Takeaways from Marcela's roundtable discussion on the state of Data and AI-- Updates on the IBM + AWS partnership, including RDS for Db2 and expanded collaboration-- New capabilities in the AWS Marketplace, such as Databand for data observability and watsonx.data to enhance data management and AI readiness-- Marcela's vision for what's next in Data and AI for 2025This conversation puts light on how IBM is driving innovation and empowering organizations with cutting-edge Data and AI capabilities.#data #ai #awsreinvent #awsreinvent2024 #reinvent2024 #IBMPartner #ibm #theravitshow

Most early-stage founders I talk to are focused on getting their first customers, hiring their first employees, or maybe, if they're lucky, closing their first round of funding. But what happens after that? For Rohit Choudhary, the answer was building a whole new category. Rohit is the CEO and co-founder of Acceldata, a data observability platform that helps companies manage the complexity of modern data infrastructure. Before starting the company, he spent years inside the problem — working on data engineering challenges at Hortonworks and other enterprise tech firms. Like a lot of technical founders, Rohit didn't start out dreaming of being a CEO — but the problem was too big to ignore. In this episode, we talk about: Why data engineering lacked the right tooling and how that led to Acceldata How his team validated the concept with real-world customer pain points The trade-offs of building in stealth mode vs. in public What he's learned about hiring, scaling, and making the leap from engineer to CEO If you're trying to figure out how to go from technical insight to scalable business, this one's for you. RUNTIME 37:37 EPISODE BREAKDOWN (2:16) “ There are four of us co-founders, and we were all part of the same engineering team at Hortonworks.” (4:33) “ We felt that here was a unique opportunity for us to be able to build something really, really large and big.” (6:16) How Acceldata approached proof-of-concept programs in its early days. (8:23) “ How did you decide which one of you would become the CEO?” (11:31) Rohit's seed-stage recruiting strategy: “ we had to excite them with the long-term vision.” (14:35) “ People like me, we learned how to sell despite coming from an engineering background.” (16:46) Why the co-founders “took a leap of faith” by formalizing their sales process early. (18:46) “ We were familiar with how business is conducted in the U.S.,” which made expansion easier. (21:08) Early challenges they faced after closing a Series A. (23:08) How “a big mistake” from a previous startup still influences Rohit's choices today. (25:30) Wondering if it's time to throw in the towel? Do a self-assessment. (28:31) Three core skills engineers need to acquire if they want to become effective CEOs. (31:39) “ I used to interview almost everyone until we were at about, you know, 170-180.” (33:82) How creating a 10-year strategy informed their day-to-day decision making. (36:27) The one question he'd have to ask the CEO in an interview before he could accept an offer. LINKS Acceldata Rohit Choudhary, co-founder/CEO Ashwin Rajeeva, co-founder/CTO Gaurav Nagar, co-founder/Senior Architect Raghu Mitra Kandikonda, co-founder/Director of Engineering Lightspeed Venture Partners Acceldata Announces $50 Million in Series C Funding to Expand Market Leadership and Product Innovation in Data Observability (press release) SUBSCRIBE LinkedIn Substack Instagram Thanks for listening! – Walter.

Welcome to another insightful episode of Predictable B2B Success! Today, we're diving deep into the ever-evolving world of data observability with Ryan Yackel, a seasoned product strategy leader at IBM. Ryan's expertise helps transform complex data quality issues into streamlined, proactive solutions that drive business success. Join us as Ryan unpacks the critical role of data observability in today's digital age, linking it to broader data governance strategies that resonate at the executive level. He'll share his experiences from open-source conferences in Tel Aviv and New York and discuss the importance of a strong narrative design to differentiate your business in the crowded B2B tech space. Curious about the difference between basic alerting and comprehensive observability? Or how a well-crafted strategic narrative can shift your market positioning? Ryan's insights offer compelling industry knowledge and practical tactics for enhancing data reliability and governance. We'll also delve into how pilot testing and proof-of-concept initiatives can demonstrate real-world value, and the nuances of integrating data observability within IBM's robust tech ecosystem. Whether you're a data engineer, a marketing strategist, or a tech executive, this episode promises to open your eyes to new possibilities in data management. Tune in and discover how to elevate your data strategy to new heights! Some areas we explore in this episode include: Data Observability Campaigns: Awareness efforts and collaborations in the emerging data observability space.Community Engagement: Participation in open-source conferences and tech meetups to discuss technical deployments.Executive-Level Strategy: Aligning data observability with data governance to enhance prioritization.DIY Approach vs. Observability: Comparison between basic alerting/monitoring and comprehensive observability with ML detection.Strategic Narrative and Storytelling: The importance of a strong narrative for effective product communication.Pilot Testing for Proof of Concept: Using pilots to demonstrate the effectiveness of data observability solutions.Data Fabric and Data Mesh: IBM's hybrid architecture and integrating data observability.Data Quality and Observability: The importance of "data quality in motion" and evolving observability tools.Data Acquisition Strategy: Combining top-down and bottom-up approaches for integrating DataBank.IBM Acquisition: The impact of DataBank's acquisition by IBM and cultural integration with AI and quantum computing initiatives.And much, much more...

Send us a textIn this episode of The Digital Executive Podcast, host Brian Thomas welcomes Somesh Saxena, CEO and founder of Pantomath. Transitioning from a high-pressure career in fine dining, including time at Gordon Ramsay's Michelin-starred restaurant, to leading data and analytics at GE Aerospace, Somesh shares his incredible journey of resilience and innovation.Discover how his experiences shaped the creation of Pantomath, a groundbreaking platform that automates data observability and traceability, empowering organizations like Pepsi and Fortune 500 companies to build trust in their data. Somesh delves into the pressing need for data transparency, the cultural shift required for a data-driven organization, and the role of AI in the future of automated data operations.Whether you're a tech enthusiast or a leader navigating digital transformation, this episode offers invaluable insights into the evolving world of data reliability.

Send Everyday AI and Jordan a text messageYour data is your moat. Everyone's got AI now. Find out how reliable data can make your competitive edge happen. Barr Moses, Co-Founder and CEO of Monte Carlo, joins us to discuss. Newsletter: Sign up for our free daily newsletterMore on this Episode: Episode PageJoin the discussion: Ask Jordan and Barr questions on AI and dataUpcoming Episodes: Check out the upcoming Everyday AI Livestream lineupWebsite: YourEverydayAI.comEmail The Show: info@youreverydayai.comConnect with Jordan on LinkedInTopics Covered in This Episode:1. the Importance of Data2. Challenges and Opportunities in Leveraging Data3. Adoption of Data Practices4. Data Use Case Examples5.Generative AI, LLMs, and Data IntegrationTimestamps:00:00 Empower AI proficiency with daily insights.06:02 Data observability ensures reliability and issue resolution.07:15 Understanding data's importance is crucial for businesses.13:07 Personalized AI relies on unique enterprise data.15:20 Large enterprises struggle with data consistency, smaller teams advantage.19:42 Generative AI analyzes sports data for insights.22:56 Personalized financial products using reliable data.23:56 Credit Karma Intune boosts external and internal productivity.28:02 Peak data reached; synthetic data becomes crucial.30:36 Recap available on your everydayai.com.Keywords:Generative AI, Data Usage, Data Accuracy, High-Quality Data, AI Implementation, Brand Reputation, Small Business Data Management, Data Systems, Trusting Data Sources, Everyday AI Podcast, Microsoft Partnership, Barr Moses, Monte Carlo, Data Downtime, Data Issues, Data Products, Data Observability, Data Adoption Forecast, Smaller Team Advantages, Microsoft WorkLab Podcast, Data Quality Monitor Recommendations, AI and Data Integration, Personalized Financial Products, Coding Assistants, AI for Compliance Reporting, Large Language Models, Synthetic Data, Real-World Data, Data Governance, Data Quality Management.

In this episode of the SaaS Revolution Show our host Alex Theuma is joined by Kevin Hu, Co-Founder & CEO at Metaplane, who shares how they navigated Metaplane's journey from pivot to Series A and beyond. "Most of us come from tens of thousands of years of subsistence farmers and we're one of the first generations that has the opportunity to build something that can have a large impact with software and capitals leveraged. And when it comes to the CEO job, one can hope that the problems never stop. When the problems stop, I think that's when growth stops. The best case scenario is when the problems are new problems all the time and it doesn't feel like Groundhog Day." Kevin shares: • The reasons that drove Metaplane to pivot (twice!) • The importance of attracting strong leaders and building a repeatable sales process as a company grows • His approach to decision-making and prioritisation; from the way he collects information to committing to decisions • Why being upfront about expectations and offering upside potential is crucial to recruiting top talent • Metaplane's unique onboarding process where new hires "ship something" on day one • Overcoming market headwinds through a product-led approach and land-and-expand strategy and more! Check out the other ways SaaStock is serving SaaS founders

Exploring data observability's limits in data migration, integrity audits, and the need for specialized tools for reliability. Published at: https://www.eckerson.com/articles/the-shiny-allure-of-data-observability-its-limits-in-data-migration-integrity-audits-and-certification

The Data Product Management In Action podcast, brought to you by Soda and executive producer Scott Hirleman, is a platform for data product management practitioners to share insights and experiences.  In Episode 24 of Data Product Management in Action, our host Nick Zervoudis is joined byTefi Trabuchi, Data Platform Product Manager at SumUp, to discuss her experience transforming a reactive data platform team into a user-focused, strategy-driven powerhouse. Tefi shares how she tackled challenges like burnout, prioritization struggles, and resistance to product practices such as user research and OKRs. She highlights the pivotal role of user interviews in shifting mindsets and the delicate balance between reducing risk, ensuring compliance, and driving innovation. Tefi also emphasizes the value of clear communication and curiosity when working in highly technical domains. This episode offers practical insights for product managers navigating the complexities of data, AI, and machine learning. About our host Nick Zervoudis: Nick is Head of Product at CKDelta, an AI software business within the CKHutchison Holdings group. Nick oversees a portfolio of data products and works with sister companies to uncover new opportunities to innovate using data,analytics, and machine learning.Nick's career has revolved around data and advanced analytics from day one,having worked as an analyst, consultant, product manager, and instructor for startups, SMEs, and enterprises including PepsiCo, Sainsbury's, Lloyds BankingGroup, IKEA, Capgemini Invent, BrainStation, QuantSpark, and Hg Capital. Nick is also the co-host ofLondon's Data Product Management meetup, andspeaks & writes regularly about data & AI product management. Connect with Nick on LinkedIn. About our guest Tefi Trabuchi:Tefi is a Data Platform Product Manager at SumUp, where she focuses on making sure our data tools are not only secure and efficient but also provide a smooth user experience for our internal teams. Before this, she led the development of an in-house Data Observability tool at Glovo, introducing governance rules and SLAs for key datasets. Tefi enjoys working closely with teams to create practical solutions that make accessing and using data easier and more intuitive, so everyone can make more informed decisions faster. Connect with Tefi on LinkedIn. All views and opinions expressed are those of the individuals and do not necessarily reflect their employers or anyone else.  Join the conversation on LinkedIn.  Apply to be a guest or nominate someone that you know.  Do you love what you're listening to? Please rate and review the podcast, and share it with fellow practitioners you know. Your support helps us reach more listeners and continue providing valuable insights!              

I am excited to announce that Somesh Saxena, CEO of Pantomath will be on The Ravit Show! We dive deep into the world of Data Observability and its increasing importance in data-driven landscape and the future! We covered some crucial topics: 1️⃣ Pantomath's journey and why Data Observability is a game-changer 2️⃣ How Data Observability has evolved and what shifts companies need to be aware of 3️⃣ Key components of Data Observability and the challenges businesses face in implementing it 4️⃣ The essential relationship between Data Quality and Observability 5️⃣ How automation and AI are transforming Data Quality at scale Plus, Somesh shared his vision for the future of Data Observability and key advice for leaders just beginning their journey in Data Quality & Observability Don't miss out!

This Week in Startups is brought to you by: Vanta. Compliance and security shouldn't be a deal-breaker for startups to win new business. Vanta makes it easy for companies to get a SOC 2 report fast. TWiST listeners can get $1,000 off for a limited time at https://www.vanta.com/twist Squarespace. Turn your idea into a new website! Go to https://www.Squarespace.com/TWIST for a free trial. When you're ready to launch, use offer code TWIST to save 10% off your first purchase of a website or domain. Kalshi. Kalshi—the largest regulated predictions market—now lets you trade on US elections. Visit https://www.kalshi.com/twist to see live odds, trade, and get $20 when you deposit $100. * Todays show: Alex Wilhelm interviews leaders from Monte Carlo and Mastertech.ai, exploring their roles in data observability and AI applications. Linda Gray shares her journey founding Mastertech.ai and highlighting AI's transformative role in auto shops.(2:05) Monte Carlo's Lior Gavish discusses the importance of data downtime and AI's influence on data monitoring. (35:51) * Timestamps: (0:00) Alex Wilhelm kicks off the show (2:05) Linda Gray's career and Mastertech.ai origin (5:17) Mastertech.ai and the auto repair industry (8:06) Vanta - Get $1000 off your SOC 2 at https://www.vanta.com/twist (9:11) Technological adoption and Mastertech.ai's benefits (16:18) Mastertech.ai's OEM approval process and community data (20:50) Squarespace - Use offer code TWIST to save 10% off your first purchase of a website or domain at https://www.Squarespace.com/TWIST (22:20) Mastertech.ai's AI-driven diagnostics demo (34:34) Kalshi—the largest regulated predictions market—now lets you trade on US elections. Visit https://www.kalshi.com/twist to see live odds, trade, and get $20 when you deposit $100. (35:51) Lior Gavish from Monte Carlo joins Alex (45:15) AI's accelerant effect on Monte Carlo's growth and strategy * Subscribe to the TWiST500 newsletter: https://ticker.thisweekinstartups.com Check out the TWIST500: https://www.twist500.com * Subscribe to This Week in Startups on Apple: https://rb.gy/v19fcp * Monte Carlo: https://www.montecarlodata.com Mastertech.AI: https://www.mastertech.ai * Follow Alex: X: https://x.com/alex LinkedIn: ⁠https://www.linkedin.com/in/alexwilhelm * Follow Lior: X: https://x.com/lgavish LinkedIn: https://www.linkedin.com/in/lgavish * Follow Linda: X: https://x.com/lindach167 LinkedIn: https://www.linkedin.com/in/linda-gray-6433b251 * Thank you to our partners: (8:06) Vanta - Get $1000 off your SOC 2 at https://www.vanta.com/twist (20:50) Squarespace - Use offer code TWIST to save 10% off your first purchase of a website or domain at https://www.Squarespace.com/TWIST (34:34) Kalshi - Sign up to win $100K at https://www.kalshi.com/twist * Great TWIST interviews: Will Guidara, Eoghan McCabe, Steve Huffman, Brian Chesky, Bob Moesta, Aaron Levie, Sophia Amoruso, Reid Hoffman, Frank Slootman, Billy McFarland * Check out Jason's suite of newsletters: https://substack.com/@calacanis * Follow TWiST: Twitter: https://twitter.com/TWiStartups YouTube: https://www.youtube.com/thisweekin Instagram: https://www.instagram.com/thisweekinstartups TikTok: https://www.tiktok.com/@thisweekinstartups Substack: https://twistartups.substack.com * Subscribe to the Founder University Podcast: https://www.youtube.com/@founderuniversity1916

I am excited to announce that Somesh Saxena, CEO of Pantomath will be on The Ravit Show! We dive deep into the world of Data Observability and its increasing importance in data-driven landscape and the future! We covered some crucial topics: 1️⃣ Pantomath's journey and why Data Observability is a game-changer 2️⃣ How Data Observability has evolved and what shifts companies need to be aware of 3️⃣ Key components of Data Observability and the challenges businesses face in implementing it 4️⃣ The essential relationship between Data Quality and Observability 5️⃣ How automation and AI are transforming Data Quality at scale Plus, Somesh shared his vision for the future of Data Observability and key advice for leaders just beginning their journey in Data Quality & Observability Don't miss out! Tune in tomorrow and catch the insights that could help transform your organization's data strategy!

Rohit Choudhary, co-founder and CEO of Acceldata, placed an early bet on data observability, which has proven prescient. In a New Stack Makers podcast episode, Choudhary discussed three key insights that shaped his vision: First, the exponential growth of data in enterprises, further amplified by generative AI and large language models. Second, the rise of a multicloud and multitechnology environment, with a majority of companies adopting hybrid or multiple cloud strategies. Third, a shortage of engineering talent to manage increasingly complex data systems.As data becomes more essential across industries, challenges in data observability have intensified. Choudhary highlights the complexity of tracking where data is produced, used, and its compliance requirements, especially with the surge in unstructured data. He emphasized that data's operational role in business decisions, marketing, and operations heightens the need for better traceability. Moving forward, traceability and the ability to manage the growing volume of alerts will become areas of hyper-focus for enterprises.Learn more from The New Stack about data observability: What Is Data Observability and Why Does It Matter?The Looming Crisis in the Observability MarketThe Growth of Observability Data Is Out of Control!Join our community of newsletter subscribers to stay on top of the news and at the top of your game.  

Andreas und Carsten sprechen über die neuesten Trends aus der Data- und Analytics-Welt und werfen einen kritischen Blick auf die Entwicklungen in London. In dieser Episode diskutieren die beiden die Big Data London und was die Messe für die europäische Data- und Analytics-Szene bedeutet. Mit über 200 Ausstellern und einem Fokus auf generative KI und Datenprodukte war die Veranstaltung ein beeindruckendes Highlight. Carsten gibt Einblicke in neue Trends wie Data Observability und die wiederauflebende Diskussion um Datenkultur. Außerdem schauen sie auf die wachsende Bedeutung von Datenprodukten und fragen sich: Ist der Hype um Data Mesh schon vorbei oder nur eine Begrifflichkeit, die sich langsam ändert? Es gibt auch noch eine neue Rubrik und natüüüürlich die M&A News! Lohnt sich also mal wieder! ⪧ Studien des Monats - Data Mesh and Data Fabric 2024 - Global CPM Trends and Priorities Report 2025 ⪧ Events - The Heart of Data Mesh & Fabric - Würzburg (BARC) - data:unplugged Pre-Event - Hamburg - Testen von DWH- und BI-Systemen - Planning with Power BI (Webinar) - DATA festival #online - BI or DIE Level Up

Key Moments:Why is the data wrong? (6:00)Our products are our data (11:00)The true size of the data quality problem (14:00)Clean your data before you prioritize shiny new tools (26:00)The next frontier: GenAI and unstructured data (31:00)Key Quotes:“The data estate has changed significantly. But the way in which we manage data and data quality specifically has not adapted.” – Barr Moses“I tracked every single change in the data that I made, and could calculate how much money a company saved after a data cleanup. For a mid-size company, the difference was approximately a quarter of a billion dollars. For a large company, it could be several billion dollars. 45% of the data I cleaned had errors.” – Olga Maydanchik“The competitive advantage is really the access to your proprietary data that you have as an enterprise. So you need to make sure that that data is accurate, reliable, and on time. Now, how do you do that? That's something that people are still figuring out.”  – Barr MosesMentions:Information Quality Applied: Best Practices for Improving Business Information, Outcomes and Systems: Book by Larry EnglishThe Rest is History PodcastFreakonomics PodcastThe Matrix Film SeriesThe Play That Goes WrongBio: Barr Moses: Barr Moses is the CEO and Co-Founder of Monte Carlo, the data reliability company. Monte Carlo is the creator of the industry's first end-to-end Data Observability platform. She is also co-author of O'Reilly's Data Quality Fundamentals: Building Reliable Data Pipelines. Previously, she was VP Customer Operations at Gainsight, a management consultant at Bain & Company and served in the Israeli Air Force as a commander of an intelligence data analyst unit.Olga Maydanchik:Olga Maydanchik is a data governance, data quality, and data architecture thought leader and practitioner. She is an expert in design and implementation of enterprise-wide data management programs, who has led data quality efforts at Deutsche Bank, AIG, and at Citi. Hear more from Cindi Howson here. Sponsored by ThoughtSpot.

When Cribl approached Revefi, they were looking for a data observability product. What they got with Revefi was a copilot for data teams that provided observability, quality, performance, usage, and spend insights. Within hours, Revefi had saved them five figures! When we say Revefi pays for itself and likely your other tools, we mean that literally—we have seen this story repeat again and again. I had an insightful session with the CEO and Co-Founder, Revefi on The Ravit Show at the Snowflake Summit, diving deep into what makes Revefi unique: -- Revefi's approach of combining data observability, operations, and FinOps -- The claim of being super quick to production, in under 5 minutes. How does that work? Could I be in production before this interview ends? -- Their new AI-based data engineering assistant, Neo, which has been demoed at their booth Stay tuned for more insights and stories from the Snowflake Summit! #data #ai #snowflakesummit #snowflakeflake2024 #revefi #theravitshow

Day 1 at the Snowflake Summit where I had the pleasure of interviewing Somesh Saxena from Pantomath on The Ravit Show. Here's a rundown of our discussion: - Somesh introduced himself and shared insights about Pantomath - We explored how Data Observability has evolved over the last four years - Discussed the key pillars of Data Observability - Somesh shared some intriguing use cases with our audience - We delved into how AI is making a significant impact in the Data Observability space. Stay tuned for more insights and conversations from the Snowflake Summit! #data #ai #snowflakesummit #snowflakeflake2024 #theravitshow

Join Mona Rakibe on the DATAcated Show to talk about how to get leaders to buy into data reliability & data observability.

Chris Cooney is the Head of Developer Advocacy for Coralogix, a SaaS observability platform that analyzes logs, metrics, and security data in real time.In this episode, we talk about data observability and how it helps enable digital transformation. We discuss why it's important to prioritize observability from the start, how to optimize observability-related costs, the importance of responsible data use, and the impact of AI technologies.Links & mentions:coralogix.comlinkedin.com/chris-cooney

Join this DATAcated Takeover with Ryan Yackel, CMO of IBM Databand and Eric Jones Enterprise Solutions Architect at IBM, to learn about real-world data observability in action. They'll talk about what a day in the life of a data engineer using databand looks like. Data observability is a 'hot topic' but Ryan & Eric are here to tell us what it actually looks like in action! #datacatedtakeover #data #dataobservability --- Support this podcast: https://podcasters.spotify.com/pod/show/datacated/support

It was such a pleasure to chat with Mona Rakibe and Maxim Lukichev, Co-Founders of Telmai, on The Ravit Show at Google Next. We discussed the keynote session at Google Next, Telmai as a leader in the Data Observability space, future of Telmai and much more! #data #ai #googlenext2024 #telmai #theravitshow

In this episode of Enterprising Insights, The Futurum Group's Enterprise Applications Research Director Keith Kirkpatrick discusses Tableau Conference 2024, recent earnings news from major enterprise application vendors. He then closes out the show with the Rant or Rave segment, where he picks one item in the market, and either champions or criticizes it.

What's up everyone, today we have the pleasure of sitting down with Kevin Hu (Hoo), Co-founder and CEO at Metaplane. Summary: Dr. Kevin Hu gives us a masterclass on everything data. Data analysis, data storytelling, data quality, data observability and data anomaly detection. We unpack the power of inquisitive data analysis and a hypothesis-driven approach, emphasizing the importance of balancing data perfection with actually doing the work of activating that data. He highlights data observability and anomaly detection as a key to preempting errors, ensuring data integrity for a seamless user experience. Amid the rise of AI in martech, he champions marketing ops' role in safeguarding data quality, making clear that success hinges on our ability to manage data with precision, creativity, and proactive vigilance. About KevinKevin did his undergrad in Physics at MITHe later collaborated with his biologist sister, assisting in analyzing five years of fish behavior data. This experience inspired him to further his research and earn a master's degree in Data Visualization and Machine LearningHe also completed a PhD in Philosophy at MIT where he led research on automated data visualization and semantic type detection His research was published at several conferences like CHI (pronounced Kai) (human-computer interaction), SIGMOD (database) and KDD (data mining) and featured in the Economist, NYT and WiredIn 2019, Kevin teamed up with former Hubspot and Appcues engineers to launch Metaplane, initially set out to be a product focused on customer success, designed to analyze company data for churn preventionBut after going through Y Combinator, the company pivoted slightly to build data analytics-focused toolsToday Metaplane is a data observability platform powered by ML-based anomaly detection that helps teams prevent and detect data issues — before the CEO pings them about weird revenue numbers.How to Ask the Right Questions in Data AnalysisWhen Kevin shared the profound impact César Hidalgo, his mentor at MIT, had on his journey into the data world, it wasn't just about learning to analyze data; it was about asking the right questions. César put together one of our favorite TED talks ever – Why we should automate politicians with AI agents – this was back in 2018, long before ChatGPT was popular. Hidalgo, recognized not only for AI and ML applications but also developing innovative methods to visualize complex data and making it understandable to a broader audience, was the most important teacher in Kevin's life. He helped Kevin understand that the bottleneck in data analysis wasn't necessarily a lack of coding skills but a gap in understanding what to ask of the data. This revelation came at a pivotal moment as Kevin navigated his path through grad school, influenced by his sister's work in animal behavior and his own struggles with coding tools like R and MATLAB.Under Hidalgo's guidance, Kevin was introduced to a broader perspective on data analysis. This wasn't just about running numbers through a program; it was about diffusing those numbers with context and meaning. Hidalgo's approach to mentorship, characterized by personalized attention and encouragement to delve into complex ideas, like those presented in Steven Pinker's "The Blank Slate," opened up a new world of inquiry for Kevin. It was a world where the questions one asked were as critical as the data one analyzed.This mentorship experience highlights the importance of curiosity and critical thinking in the field of data science. Kevin's reflection on his journey reveals a key insight: mastering coding languages is only one piece of the puzzle. The ability to question, to seek out the stories data tells, and to understand the broader implications of those stories is equally, if not more, important.Kevin's gratitude towards Hidalgo for his investment in students' growth serves as a reminder of the value of mentorship. It's a testament to the idea that the best mentors don't just teach you how to execute tasks; they inspire you to see beyond the immediate horizon. They challenge you to think deeply about your work and its impact on the world.Key takeaway: For marketers delving into data-informed strategies, Kevin's story is a powerful reminder that beyond the technical skills, the ability to ask compelling, insightful questions of your data can dramatically amplify its value. Focus on nurturing a deep, inquisitive approach to understanding consumer behavior and market trends.Bridging Academic Rigor with Startup AgilityDuring his career in academia working alongside Olympian-caliber scientists and researchers, Kevin garnered insights that have since influenced his approach to running a startup. The parallels between academia and startups are striking, with both realms embodying a journey of perseverance and unpredictability. This analogy provides a foundational mindset for entrepreneurs who must navigate the uncertain waters of business development with resilience and adaptability.At the heart of Kevin's philosophy is the adoption of a hypothesis-driven approach. This methodology, borrowed from academic research, emphasizes the importance of formulating hypotheses for various aspects of business operations, particularly in marketing strategies. Identifying the ideal customer profile (ICP), crafting compelling messaging, and selecting the optimal channels are seen not as static decisions but as theories to be rigorously tested and iterated upon. This empirical approach allows for a methodical exploration of what resonates best with the target audience, acknowledging that today's successful strategy may need reevaluation tomorrow.Another vital lesson from academia that Kevin emphasizes is the respect for past endeavors. In a startup ecosystem often obsessed with innovation, there's a tendency to overlook the lessons learned from previous attempts in similar ventures. By acknowledging and building upon the efforts of predecessors, Kevin advocates for a more informed and grounded approach to innovation. This perspective encourages entrepreneurs to consider the historical context of their ideas and strategies, potentially saving time and resources by learning from past mistakes rather than repeating them.Key takeaway: Embracing a hypothesis-driven mindset should be familiar grounds for marketers. Challenge your team to identify and test hypotheses around underexplored or seemingly less significant customer segments. This could involve hypothesizing the effectiveness of personalized content for a niche within your broader audience that has been overlooked, measuring engagement against broader campaigns.Balancing Data Accuracy with Rapid GrowthFor startups grappling with survival, the luxury of perfect data is often out of reach. Kevin points out that data quality should be tailored to the specific needs of the business. For instance, data utilized for quarterly board meetings does not necessitate the same level of freshness as data driving daily customer interactions. This pragmatic approach underscores the importance of defining data quality standards based on the frequency and criticality of business decisions.At the heart of Kevin's argument is the concept that as businesses scale, the stakes of data accuracy and timeliness escalate. He highlights scenarios where real-time data becomes crucial, such as B2B SaaS companies engaging with potential leads or e-commerce platforms optimizing their customer journey. In these cases, even slight inaccuracies or delays can result in missed revenue opportunities or diminished customer trust.This discourse on data quality transcends the binary choice between perfect data and rapid action. Instead, Kevin advoc...

Did you know Gartner Magic Quadrant Research notes have 21 mentions of Data Observability? Check out the discussion with my friend, Ramon Chen, CPO, Acceldata. We discussed about Data Observability, Data Quality, Data Observability + AI and much more! Learn more below! #data #dataobservability #acceldata #gartnerorlando #theravitshow

Summary Working with data is a complicated process, with numerous chances for something to go wrong. Identifying and accounting for those errors is a critical piece of building trust in the organization that your data is accurate and up to date. While there are numerous products available to provide that visibility, they all have different technologies and workflows that they focus on. To bring observability to dbt projects the team at Elementary embedded themselves into the workflow. In this episode Maayan Salom explores the approach that she has taken to bring observability, enhanced testing capabilities, and anomaly detection into every step of the dbt developer experience. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics. Trusted by teams of all sizes, including Comcast and Doordash, Starburst is a data lake analytics platform that delivers the adaptability and flexibility a lakehouse ecosystem promises. And Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data. Want to see Starburst in action? Go to dataengineeringpodcast.com/starburst (https://www.dataengineeringpodcast.com/starburst) and get $500 in credits to try Starburst Galaxy today, the easiest and fastest way to get started using Trino. Dagster offers a new approach to building and running data platforms and data pipelines. It is an open-source, cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability, a declarative programming model, and best-in-class testability. Your team can get up and running in minutes thanks to Dagster Cloud, an enterprise-class hosted solution that offers serverless and hybrid deployments, enhanced security, and on-demand ephemeral test deployments. Go to dataengineeringpodcast.com/dagster (https://www.dataengineeringpodcast.com/dagster) today to get started. Your first 30 days are free! This episode is brought to you by Datafold – a testing automation platform for data engineers that prevents data quality issues from entering every part of your data workflow, from migration to dbt deployment. Datafold has recently launched data replication testing, providing ongoing validation for source-to-target replication. Leverage Datafold's fast cross-database data diffing and Monitoring to test your replication pipelines automatically and continuously. Validate consistency between source and target at any scale, and receive alerts about any discrepancies. Learn more about Datafold by visiting dataengineeringpodcast.com/datafold (https://www.dataengineeringpodcast.com/datafold). Your host is Tobias Macey and today I'm interviewing Maayan Salom about how to incorporate observability into a dbt-oriented workflow and how Elementary can help Interview Introduction How did you get involved in the area of data management? Can you start by outlining what elements of observability are most relevant for dbt projects? What are some of the common ad-hoc/DIY methods that teams develop to acquire those insights? What are the challenges/shortcomings associated with those approaches? Over the past ~3 years there were numerous data observability systems/products created. What are some of the ways that the specifics of dbt workflows are not covered by those generalized tools? What are the insights that can be more easily generated by embedding into the dbt toolchain and development cycle? Can you describe what Elementary is and how it is designed to enhance the development and maintenance work in dbt projects? How is Elementary designed/implemented? How have the scope and goals of the project changed since you started working on it? What are the engineering challenges/frustrations that you have dealt with in the creation and evolution of Elementary? Can you talk us through the setup and workflow for teams adopting Elementary in their dbt projects? How does the incorporation of Elementary change the development habits of the teams who are using it? What are the most interesting, innovative, or unexpected ways that you have seen Elementary used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Elementary? When is Elementary the wrong choice? What do you have planned for the future of Elementary? Contact Info LinkedIn (https://www.linkedin.com/in/maayansa/?originalSubdomain=il) Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. Podcast.__init__ (https://www.pythonpodcast.com) covers the Python language, its community, and the innovative ways it is being used. The Machine Learning Podcast (https://www.themachinelearningpodcast.com) helps you go from idea to production with machine learning. Visit the site (https://www.dataengineeringpodcast.com) to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@dataengineeringpodcast.com (mailto:hosts@dataengineeringpodcast.com)) with your story. Links Elementary (https://www.elementary-data.com/) Data Observability (https://www.montecarlodata.com/blog-what-is-data-observability/) dbt (https://www.getdbt.com/) Datadog (https://www.datadoghq.com/) pre-commit (https://pre-commit.com/) dbt packages (https://docs.getdbt.com/docs/build/packages) SQLMesh (https://sqlmesh.readthedocs.io/en/latest/) Malloy (https://www.malloydata.dev/) SDF (https://www.sdf.com/) The intro and outro music is from The Hug (http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/Love_death_and_a_drunken_monkey/04_-_The_Hug) by The Freak Fandango Orchestra (http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/) / CC BY-SA (http://creativecommons.org/licenses/by-sa/3.0/)

Highlights from this week's conversation include:The evolution of data operations (1:13)Unravel's role in simplifying data operations (2:17)Kunal's journey from fashion to enterprise data management (5:23)The Unravel platform and its components (10:08)Challenges in data operations at scale (16:34)Users of Unravel within an organization (22:32)Calculating ROI on data products (25:55)Understanding the cost of data operations (27:01)Measuring productivity and reliability (30:59)Diversity of technologies in data operations (34:52)Efficiency in cost management (44:15)Implementing observability in AI (47:55)Challenges of AI Adoption (50:17)Final thoughts and takeaways (51:36)The Data Stack Show is a weekly podcast powered by RudderStack, the CDP for developers. Each week we'll talk to data engineers, analysts, and data scientists about their experience around building and maintaining data infrastructure, delivering data and data products, and driving better outcomes across their businesses with data.RudderStack helps businesses make the most out of their customer data while ensuring data privacy and security. To learn more about RudderStack visit rudderstack.com.

In this episode of The Main Scoop™, hosts Greg Lotko and Daniel Newman discuss the importance of observability and generative AI strategies in operations with Cory Minton, Field CTO - Americas at Splunk. It was a great conversation and one you don't want to miss. Like what you've heard? Check out all our past episodes here, and be sure to subscribe so you never miss an episode of The Main Scoop™ series.

In this episode of the Futurum Tech Webcast, host Steven Dickens speaks with IBM's Ryan Yackel, GTM PM and Growth Leader, IBM Databand, about the evolving landscape of data management and AI. They discuss the recent acquisition of Databand by IBM, highlighting Databand's role in data observability within the modern data stack. Yackel explains how data observability is becoming increasingly important due to the challenges faced by data engineering teams and the proliferation of diverse tool stacks. He also delves into how data observability complements data governance, emphasizing its role in improving detection, resolution times, and data SLAs. Their discussion covers: IBM's acquisition of Databand and its integration into IBM's data fabric team That data observability is identified as a critical trend due to the increasing demands on data engineering teams and the complexity of tool stacks How data observability enhances data governance, reliability, and quality within organizational data strategies  The intersection of data management practices with AI deployment, emphasizing the importance of quality and governance in AI strategies To learn more, and to download The Futurum Group's white paper done in partnership with IBM, visit the company's website.

In this episode of TFiR: Let's Talk, Mona Rakibe, Co-Founder and CEO at Telmai, talks about the company and how it is helping companies improve their data quality and investigate anomalies. They go on to talk about the company's journey so far, some of the key capabilities of the platform, and what sets them apart from competitors.

The digital realm's new currency is data, yet its value is often as enigmatic as it is critical. Sanjeev Mohan, Principal of SanjMo and former Gartner analyst, decodes the complexities of data valuation, advocating for a product-oriented view that frames data's utility and impact within an organization. The prerequisites for defining a 'data product'—from maintaining stringent quality and availability standards via Service Level Agreements to managing its lifecycle for enduring relevance— bring into focus the role of the data product manager. This role is vital to ensuring continuous enhancement and overseeing the retirement of data products; a job that guarantees these products remain a driving force for organizational value.In the pursuit of measurable benefits, treating data with the rigor of product management emerges as a beacon for Chief Data Officers, offering concrete metrics through the creation and utilization rates of data products, and providing a clear gauge for the pace and quality of innovation in data work.Three reasons you should listen to this episode:1. Data Product Insights. Grasp the value of data work as Sanjeev Mohan breaks down the essence of data products and their role in shaping business strategies.2. Data Observability. Learn about the critical nature of data observability and quality, and why these factors are non-negotiable in the pursuit of high-caliber data standards.3. Industry Foresight. Gain perspective on the current and future trends of data analytics as seen through the lens of an industry veteran.ResourcesConnect with Sanjeev on LinkedInAnd for a deeper understanding of data products, check out Sanjeev's book, "Data Products for Dummies".Enjoyed this Episode?Be sure to follow us so you never miss an update. You can leave us a review on Apple or Spotify, and share it with your friends and colleagues to help others learn more about the importance of a data-first digital transformation approach.Have questions? You can connect with us on LinkedIn. For more updates, please visit our website.

The Ravit Show interviewed Lior Gavish, CTO and Co-Founder of Monte Carlo, and Ryan Othniel Kearns, the founding data scientist at Monte Carlo. We dived deep into the world of "Data Observability for Generative AI - and Beyond."We also discuss GenAI, AI/ML, and the upcoming IMPACT event.

This episode features an interview with Mona Rakibe, CEO and Co-founder of Telmai, an AI-based data observability platform built for open architecture. Mona is a veteran in the data infrastructure space and has held engineering and product leadership positions that drove product innovation and growth strategies for startups and enterprises. She has served companies like Reltio, EMC, Oracle, and BEA where AI-driven solutions have played a pivotal role.In this episode, Sam sits down with Mona to discuss the application of LLMs, cleaning up data pipelines, and how we should think about data reliability.-------------------“When this push of large language model generative AI came in, the discussions shifted a little bit. People are more keen on, ‘How do I control the noise level in my data, in-stream, so that my model training is proper or is not very expensive, we have better precision?' We had to shift a little bit that, ‘Can we separate this data in-stream for our users?' Like good data, suspicious data, so they train it on little bit pre-processed data and they can optimize their costs. There's a lot that has changed from even people, their education level, but use cases also just within the last three years. Can we, as a tool, let users have some control and what they define as quality data reliability, and then monitor on those metrics was some of the things that we have done. That's how we think of data reliability. Full pipeline from ingestion to consumption, ability to have some human's input in the system.” – Mona Rakibe-------------------Episode Timestamps:(01:04): The journey of Telmai (05:30): How we should think about data reliability, quality, and observability (13:37): What open source data means to Mona(15:34): How Mona guides people on cleaning up their data pipelines (26:08): LLMs in real life(30:37): A question Mona wishes to be asked(33:22): Mona's advice for the audience(36:02): Backstage takeaways with executive producer, Audra Montenegro-------------------Links:LinkedIn - Connect with MonaLearn more about Telmai

I sat with two of my favs, Sanjeev Mohan, Former VP Gartner, and Mona Rakibe, Co-Founder & CEO, Telmai. We discussed key takeaways from Big Data London, Data and analytics, Data Observability, and much more!

You are what you observe, and lately, that's a lot of data! Since the dawn of Kubernetes, observability has skyrocketed as organizations look for ways to optimize uptime for critical data pipelines. The result is a wealth of options for understanding your data ecosystem. But which tools and methods are best for you? This special DM Radio Virtual Summit will delve into the details. Analyst Yves Mulkers will deliver a Keynote on Seeing Is Believing, followed by an Industry Keynote, and an expert panel led by Eric Kavanagh. The event will conclude with a technology deep dive and demos.

In this episode, we're stirring up a robust concoction of data wisdom and craft beer appreciation! Join us as we sit down with the CMO of IBM Databand, Ryan Yackel, to delve deep into the fascinating world of data observability. Ryan will share his exclusive insights on achieving proactive data observability through 5 key steps that are vital in steering your data pipelines to success.But that's not all!As we navigate through these vital steps to attain data observability, we also embark on a sensational beer tasting voyage! Together with Ryan, we will pair each of the 5 key steps with the rich histories, brewing secrets, and unique characteristics of some of the world's most loved beers - Lager, Wheat Beer, Pale Ale, IPA, and Sour Beer.

Today, I dive into the complex and often nebulous world of data observability with a leading expert in the field, Taggart Matthiesen, the Chief Product Officer at LogicMonitor. With an impressive career trajectory that includes pivotal roles at Lyft, Twitter, and Salesforce, Taggart offers an insider's view on the challenges and opportunities of harnessing data for actionable business intelligence. The discussion opens with Taggart sharing his unique perspective on how data has evolved to become the heartbeat of every business. In a world saturated with information, how do enterprises sift through the 'noise' to identify that elusive 1% of data which can drive decision-making and business outcomes? Taggart elucidates on the growing potential in the data observability space, shedding light on why this has become indispensable for businesses of all scales. As AI continues to infiltrate every aspect of our lives, questions around data responsibility become increasingly pertinent. We explore the readiness—or the lack thereof—of using AI to contextualize data. They discuss the ethical and practical implications, offering a balanced view on the technological advancements and the cautionary tales that serve as important guideposts for AI adoption in data observability. The episode then shifts to explore LogicMonitor's AI ops tool, DEXTA, as a case study. Taggart recounts how this tool provided immediate value by pinpointing the cause of a network outage, thereby showcasing the utility of intelligent data observability solutions. What are the design principles that inform such solutions? And what safeguards does LogicMonitor put in place to ensure data integrity and security? If you are a business leader, a product manager in SaaS, or someone simply interested in the frontier of data science and AI. In that case, this episode offers a holistic view on where data observability is headed, why it matters, and how to leverage it responsibly for business success. Prepare for an insightful journey through the world of data, AI, and business innovation.

Unlock the secrets to maximizing the value of your data with an exhilarating episode on data observability vs. data quality! Join us as we sit down with Ryan Yackel, CMO of Databand.ai, and Stephanie Valarezo, Senior Product Manager, IBM Data & AI Data Integration (DataStage), to unravel the crucial distinctions between data observability and data quality. Discover how these twin pillars empower organizations to ensure reliable, accurate, and trustworthy data. We'll get into industry insights and best practices that will revolutionize your data-driven decision-making. Whether you're a data enthusiast, a business leader, or a curious mind, this conversation is your key to unleashing the full potential of your data assets. Tune in now and embark on a journey to master data observability and data quality like never before!

Are you eager to know more about Data Observability? Check out my chat with Raj Joseph, CEO, DQLabs@ In this episode, we discussed Data Observability, Data Quality, Data Teams, and everything Data. We dive into real-world use cases, explore emerging trends, and tackle exciting challenges and opportunities. #data #datascience #dataobservability #datateams #datamodeling #dataanalytics #dataengineering #business #dqlabs #theravitshow

Barr Moses is a consultant turned CEO & Co-Founder of Monte Carlo, a data reliability company. She started her career as a management consultant at Bain & Company and a research assistant at the Statistics Department at Stanford University. Later, she became VP of Customer Operations at customer success company Gainsight, where she built the data and analytics team. She also served in the Israeli Air Force as a commander of an intelligence data analyst unit. Barr graduated from Stanford with a B.Sc. in Mathematical and Computational Science. Today, we'll talk about Barr's career journey, data reliability and observability, and what it means for data teams. If you enjoy the show, subscribe to the channel and leave a 5-star review. Subscribe to Daliana's newsletter on www.dalianaliu.com for more on data science. Barr's LinkedIn: https://www.linkedin.com/in/barrmoses/ Daliana's Twitter: https://twitter.com/DalianaLiu Daliana's LinkedIn: https://www.linkedin.com/in/dalianaliu (00:00:00) Introduction (00:01:24) How did she got into data science (00:08:26) Frameworks for data-driven decisions (00:11:20) Is customer support ticket always bad? (00:15:20) How to quickly find out what is true (00:20:17) Struggles in the data team (00:23:37) Daliana's story about lineage (00:28:00) People stressed about data (00:28:09) Netflix was down because of wrong data (00:30:40) Common issues with data quality (00:33:14) 5 pillars of data observability (00:39:14) How does Monte Carlo help data scientists (00:43:08) Build in-house vs adopt tools (00:45:48) How Daliana fixed a data quality issue (01:02:44) How to measure the impact of the data team (01:09:09) Mistakes she made (01:15:28) Beat the odds

In today's episode of Category Visionaries, we speak with Barr Moses, Co-Founder & CEO of Monte Carlo, a data observability platform that's raised over $230 Million in funding, about why quickly spotting problems in big commercial data can be the difference between a swift resolution or a costly correction later on. By providing data teams with the tools they need to keep up to date with what's going on with their data, Monte Carlo gives them a headstart in resolving, and sometimes even preventing dangerous errors before they can cause major problems. We also speak about Barr's background in the Israeli military, the lessons she learned and brought forward to the world of business, why communication with potential clients was at the heart of the Monte Carlo strategy, how happy Barr is to see the data observability category establishing itself in the marketplace, and why her biggest business inspiration might just be her own Mother. Topics Discussed: Barr's background in the Israeli military, and the lessons in analytics and dealing with challenges that she brought to the world of business How Barr's Mother became her business inspiration from a young age, and what she learned from watching her business journey Why it can be easy to end up making content only for yourself, and how important it is to communicate with your potential clients to know what's really going on Why Monte Carlo's focus is on getting as many customers as possible and making them as happy as they possibly can Why Barr spends time on podcasts, at speaking events, and writing blogs to share the concept behind his new business category The data observability category and why Barr is so thrilled to see it establishing itself

Our guest today is Barr Moses, Co-Founder & CEO of Monte Carlo, the first end-to-end data observability platform. In our conversation, we first talk about how Barr got into the field and the early influence of her parents. Barr shares her previous experiences working with data in the Israeli Army and working on data strategy at Bain. We then dig into Monte Carlo and the new field of DataOps along with data observability and its 5 pillars . Barr explains how and why she founded this company and walks us through the key challenges she faced. If you enjoyed the episode, please leave a 5 star review and subscribe to the AI Stories Youtube channel.To learn more about Monte Carlo: https://www.montecarlodata.com/Follow Barr on LinkedIn: https://www.linkedin.com/in/barrmoses/Follow Neil on LinkedIn: https://www.linkedin.com/in/leiserneil/  ————(00:00) : Intro(01:18) : How Barr got into Data Science(03:09) : Data in the Israeli Army (08:24) : Influence from her parents(11:54) : Data Strategy and consulting at Bain(19:10) : How to quickly become an expert(25:30) : What is Monte Carlo(32:00) : DataOps & 5 pillars of data observability(43:54) : Challenges when building a tech company(49:57) : Mistakes and career advice

Overview Barr Moses, Co-Founder and CEO of Monte Carlo, joins on the podcast. Prior to founding Monte Carlo, Barr was the VP of customer success operations at Gainsight and holds a bachelors of science in mathematics and computer science from Stanford. In today's episode Barr shares her inspiration for founding Monte Carlo, the cost and harms of poor quality data, the five principles of data observability, and her top predictions for data trends in 2023. About Barr Moses Barr Moses is CEO & Co-Founder of Monte Carlo, a data reliability company and creator of the data observability category, backed by Accel, GGV, Redpoint, ICONIQ Growth, Salesforce Ventures, IVP, and other top Silicon Valley investors. Previously, she was VP Customer Operations at customer success company Gainsight, where she helped scale the company 10x in revenue and, among other functions, built the data/analytics team. Prior to that, she was a management consultant at Bain & Company and a research assistant at the Statistics Department at Stanford University. She also served in the Israeli Air Force as a commander of an intelligence data analyst unit. Barr graduated from Stanford with a B.Sc. in Mathematical and Computational Science. Social Handles LinkedIn Barr's Twitter Monte Carlo's Twitter Learn more about our mission and become a member here: https://www.womenindata.org/ --- Support this podcast: https://anchor.fm/women-in-data/support

Barr Moses, CEO of Monte Carlo explains the difference between data quality and data observability, and how to make sure your data is accurate in a world where so many different teams are accessing it.

Barr Moses, CEO of Monte Carlo explains the difference between data quality and data observability, and how to make sure your data is accurate in a world where so many different teams are accessing it.

Acceldata, the company behind a data observability platform used by multinational enterprises including Oracle and Verisk, today announced it has raised $50 million in a Series C round of funding.

Data observability refers to the ability to collect, measure, and analyse data from various sources in order to understand the current state and behaviour of a system.This includes monitoring the system's performance, availability, and errors, as well as identifying patterns and anomalies in the data. By implementing data observability, organisations can gain insights into their systems and make data-driven decisions to improve performance, optimize resources, and reduce costs. Common tools used for data observability include logging, metrics, tracing, and alerting. In this episode of the EM360 Podcast, Analyst Christina Stathopoulos speaks to Lior Gavish, Co-Founder and CTO at Monte Carlo, to discuss:Getting started with data observabilityObservability trends for 2023How to implement and common challenges

Data observability refers to the ability to collect, measure, and analyse data from various sources in order to understand the current state and behaviour of a system.This includes monitoring the system's performance, availability, and errors, as well as identifying patterns and anomalies in the data. By implementing data observability, organisations can gain insights into their systems and make data-driven decisions to improve performance, optimize resources, and reduce costs. Common tools used for data observability include logging, metrics, tracing, and alerting. In this episode of the EM360 Podcast, Analyst Christina Stathopoulos speaks to Lior Gavish, Co-Founder and CTO at Monte Carlo, to discuss:Getting started with data observabilityObservability trends for 2023How to implement and common challenges

Exploring Data observability

Neste episódio falamos com Maayan Salom sobre dbt e Elementary e como essas duas ferramentas tem ajudado times de dados a implementar de forma eficiente e segura pipelines de dados.O dbt se tornou uma das ferramentas mais utilizadas para transformar dados dentro do Data Warehouse por trazer a facilidade de se usar a linguagem SQL para processamento dos dados. Com dbt é possível ter uma visão ampla do que está acontecendo dentro da sua fonte da verdade analítica, além de proporcionar diversas capacidades interessantes para times que desejam escalar de forma rápida e estruturada.O Elementary é um produto open-source cuja responsabilidade é aplicar o conceito de observabilidade dentro dos pipelines de dados construídos no dbt. Essa solução entrega relatórios, detecção de anomalias, validação de desempenho do seu pipeline e pode até entregar alerta no Slack, isso tudo para aprimorar e enriquecer seu processo de ETL.Nesse bate papo você irá entender como o dbt e o Elementary podem reduzir a complexidade durante a criação e observabilidade dos seus pipelines de dados e trazer seu time de dados para um ambiente confiável e monitorado. dbtElementaryMaayan Salom Luan Moreno = https://www.linkedin.com/in/luanmoreno/

The need for data observability, or the ability to understand, diagnose and orchestrate data health across various IT tools, continues to grow as organizations adopt more apps and services.

Maayan Salom is Co-Founder of Elementary Data, the open source data observability platform which allows users to monitor their data warehouse directly from dbt. Their project, also called Elementary, is built for analytics engineers and today has almost 1K GitHub stars and a rapidly growing community of almost 600 users. The company has raised from leading Israel and US-based venture firms as well as a number of high-profile angel investors. In this episode, we discuss having a culture of experimentation, building a community alongside other communities (ie. dbt), using your community for product feedback, the hustle involved in early GTM, learnings from building for a fast-growing community & more!

Data observability provides intelligence about data quality and data pipeline performance, contributing to the disciplines of DataOps and FinOps. Vendors such as DataKitchen, DataOps.live, Informatica, and Unravel offer solutions to help enterprises address these overlapping disciplines. Published at: https://www.eckerson.com/articles/the-blending-disciplines-of-data-observability-dataops-and-finops

Was ist Data Observability und wie kann es im Unternehmenskontext implementiert werden? Monitoring der Daten Infrastruktur.

Andy Petrella is the cofounder of Kensu, a data observability solution that offers companies the opportunity to monitor their data usage. He is the creator of Spark Notebook, an open-source software for data engineers and scientists using Spark and Scala. He is the author of the book Fundamentals of Data Observability. He has built a fantastic career in data over the last 16 years. In this episode, we cover a range of topics including: - data observability - being a founder - being an author - creating open source software - Kensu's unique approach "Data Observability Driven Development" -------- Where to find Prateek Joshi: Newsletter: https://prateekjoshi.substack.com Website: http://prateekj.com LinkedIn: https://www.linkedin.com/in/prateek-joshi-91047b19 

Max is a Software Engineer and Researcher turned entrepreneur with over 17 years of experience building large scale distributed systems, including streaming analytics systems. Max used to work in Fortune 500 companies, early and late stage startups, 3 of which became unicorns (Veeva, Reltio, SignalFx). In 2020 he co-founded Telmai and focused on building SaaS product in Data Observability space.Connect with Behind Company Lines and HireOtter Website Facebook Twitter LinkedIn:Behind Company LinesHireOtter Instagram Buzzsprout

Salma Bakouk est CEO & Cofounder de Sifflet, la plateforme de Data Observability qui permet aux équipes Data d'assurer l'intégrité des données de l'entreprise.On aborde notamment :

Want to know more about the Present and Future of Data Observability? In this episode, Egor Gryaznov, Co-founder and CTO at Bigeye and Sanjeev Mohan, Principal SanjMo & Former Gartner Research VP, talk about Data Observability, Data Reliabilty and much more! #data #datascience #python #machinelearning #analytics #dataobservabilty #ai #bi #artificialintelligence

This episode features an interview with Eric Sammer, CEO of Decodable. Eric has been in the tech industry for over 20 years, holding various roles as an early Cloudera employee. He also was the co-founder and CTO of Rocana, which was acquired by Splunk in 2017. During his time at Splunk, Eric served as the VP and Senior Distinguished Engineer responsible for cloud platform services.In this episode, Sam and Eric discuss the gap between operating infrastructure and the analytical world, stream processing innovations, and why it's important to work with people who are smarter than you.-------------------"The thing about Decodable was just like let's connect systems, let's process the data between them. Apache Flink is the right engine and SQL is the language for programming the engine. It doesn't need to be any more complicated. The trick is getting it right, so that people can think about that part of the data infrastructure, the way they think about the network. They don't question whether the packet makes it to the other side because that infrastructure is so burned in and it scales reasonably well these days. You don't even think about it, especially in the cloud." – Eric Sammer-------------------Episode Timestamps:(01:09): What open source data means to Eric(06:57): What led Eric to Cloudera and Hadoop(12:48): What inspired Eric to create Rocana(20:29): The problem Eric is trying to solve at Flink(29:54): What problems in stream processing we'll have to solve in the next 5 years(36:58): Eric's advice for advancing your career-------------------Links:LinkedIn - Connect with EricTwitter - Follow EricTwitter - Follow DecodableDecodable

Egor Gryaznov is the cofounder and CTO of Bigeye. It's a data observability platform that helps teams measure and improve data quality. Before that, he was at Uber where he scaled the company's first data warehouse, supporting thousands of internal users and mission-critical workloads.In this episode, we cover a range of topics including:- Key takeaways on being a founder- What he's building now at Bigeye- How to do customer discovery- Why do we need data observability- How to measure data quality- What is data reliability- What should early stage ML practitioners spend their time on--------Where to find Prateek Joshi:Newsletter: https://prateekjoshi.substack.comWebsite: http://prateekj.comLinkedIn: https://www.linkedin.com/in/prateek-joshi-91047b19

What's the business impact of data observability? Today we continue the conversation with Salma Bakouk, CEO and co-founder of Sifflet. Episode page https://www.discoveringdata.com/podcast/episode-043 (https://www.discoveringdata.com/podcast/episode-043) For Brands Do you want to showcase your thought leadership with great content and build trust with a global audience of data leaders? We publish conversations with industry leaders to help practitioners create more business outcomes. Explore all the ways to tell your data story here https://www.discoveringdata.com/brands (https://www.discoveringdata.com/brands). For sponsors Want to help educate the next generation of data leaders? As a sponsor, you get to hang out with the very in the industry. Want to see if you are a match? Apply now: https://www.discoveringdata.com/sponsors (https://www.discoveringdata.com/sponsors) For Guests Do you enjoy educating an audience? Do you want to help data leaders build indispensable data products? That's awesome! Great episodes start with a clear transformation. Pitch your idea at https://www.discoveringdata.com/guest (https://www.discoveringdata.com/guest).

A Reflection On Data Observability As It Reaches Broader Adoption

Overname Databand.ai: https://newsroom.ibm.com/2022-07-06-IBM-Aims-to-Capture-Growing-Market-Opportunity-for-Data-Observability-with-Databand-ai-Acquisition Overname Randori: https://newsroom.ibm.com/2022-06-06-IBM-Tackles-Growing-Attack-Surface-Risks-with-Plans-to-Acquire-Randori 148 Exabytes LTO verzonden in 2021: https://www.lto.org/2022/04/lto-tape-capacity-shipments-reach-new-record-in-2021/ Resource projecten: https://research.ibm.com/topics/trustworthy-ai#tools Homomorphic encryption: https://en.wikipedia.org/wiki/Homomorphic_encryption  Data Fabric oplossingen van IBM: https://www.ibm.com/analytics/data-fabric Gebruikte afkorting(en):LTO: Linear Tape OpenTSM: Tivoli Storage Manager Op- en aanmerkingen kunnen gestuurd worden naar: ofjestoptdestekkererin@nl.ibm.com

In this episode of The Stream Life Podcast, Nick Heudecker talks with Lior Gavish, the CTO at Monte Carlo. Monte Carlo is one of the top companies in the data observability market, and we were excited to have clarified some critical differences between observability data and data observability. It's a fun show as always, so pour a cup of coffee and listen in. Links What is Observability? Jobs at Cribl Cribl Secures $150M in Series D Funding and Introduces Cribl Search To Unlock Value of All Observability Data Jobs at Monte Carlo If you want to get every episode of the Stream Life podcast automatically, you can subscribe on Apple Podcasts, Spotify, Pocket Casts, Overcast, RSS, or wherever you get your podcasts.

Bigeye is a data observability platform that helps teams measure, improve, and communicate data quality clearly at any scale. Explore more on their YouTube channel.Bigeye cofounders Kyle Kirwan and Egor Gryaznov met at Uber, where Kyle worked on data and Egor was a staff engineer.Kyle and Egor made a clean break with Uber before founding Bigeye, eager to avoid even the appearance of an Anthony Levandowski-like situation. If you're not familiar with the ex-Google engineer sentenced to prison for stealing trade secrets (and later pardoned by Trump), catch up here.Learn how to save your energy for innovation by choosing boring technology.Connect with Kyle on LinkedIn.Connect with Egor on LinkedIn.Compiler is an original podcast from Red Hat discussing tech topics big, small and strange like, What are tech hiring managers actually looking for? And, do you have to know how to code to get started in open source? Listen to Compiler anywhere you find your podcasts or visit https://link.chtbl.com/compiler?sid=podcast.stack.overflow

IBM today announced that it acquired Databand, a startup developing an observability platform for data and machine learning pipelines. Details of the deal weren't disclosed, but Tel Aviv-based Databand had raised $14.5 million prior to the acquisition. Databand employees will join IBM's data and AI division, with the purchase expected to close on June 27. […]

IBM today announced that it acquired Databand, a startup developing an observability platform for data and machine learning pipelines. Details of the deal weren't disclosed, but Tel Aviv-based Databand had raised $14.5 million prior to the acquisition. Databand employees will join IBM's data and AI division, with the purchase expected to close on June 27. […]

Highlights from this week's conversation include:Kevin's background and career journey (1:54)Metaplane and the problem that is solves (6:47)The silence of data problems (9:53)Data physics work that requires more (13:35)Trusting data when bugs are present (19:12)Building a navigable experience (22:36)Developing anomaly detection (30:06)What Metaplane provides today (35:05)Metaplane's plans for the future (37:45)Comparing Bigquery, Snowflake, and Redshift (40:56)Why data goes bad (48:15)Advice for data trust workers (59:24)The Data Stack Show is a weekly podcast powered by RudderStack, the CDP for developers. Each week we'll talk to data engineers, analysts, and data scientists about their experience around building and maintaining data infrastructure, delivering data and data products, and driving better outcomes across their businesses with data.RudderStack helps businesses make the most out of their customer data while ensuring data privacy and security. To learn more about RudderStack visit rudderstack.com.

This bonus episode features conversations from season 2 of the Open||Source||Data podcast. In this episode, you'll hear from Barr Moses, Co-founder and CEO at Monte Carlo; Einat Orr, Co-founder and CEO at Treeverse; and Shinji Kim, Founder and CEO at Select Star.Sam sat down with each guest to discuss data observability. You can listen to the full episodes from Barr Moses, Einat Orr, and Shinji Kim by clicking the links below.-------------------Episode Timestamps:(00:35): Barr Moses(01:21): Einat Orr(02:07): Shinji Kim-------------------Links:Listen to Barr's episodeListen to Einat's episodeListen to Shinji's episode

Highlights from this week's conversation include:Tristan's background and career journey (2:43)Updating old technology (11:40)Defining “data observability” (18:44)The primary user of a data observability tool (29:56)Handling an incident (33:01)Why multipliers for data observability (37:06)Early symptoms of a data drift (43:12)Tuning in the context of data engineering (50:11)What keeps Tristan working with data (55:12)The Data Stack Show is a weekly podcast powered by RudderStack, the CDP for developers. Each week we'll talk to data engineers, analysts, and data scientists about their experience around building and maintaining data infrastructure, delivering data and data products, and driving better outcomes across their businesses with data.RudderStack helps businesses make the most out of their customer data while ensuring data privacy and security. To learn more about RudderStack visit rudderstack.com.

Eric and Kostas preview their upcoming conversation with Barr Moses of Monte Carlo.

Core hypothesis is all companies are data driven, all have data complexity, and there is not enough talent out there to solve it.  Show Notes04:01 Why Start Acceldata06:02 Core hypothesis for Acceldata08:59 Will Hadoop survive? And a tip of the hat to mainframes11:12 Meaning of Data Observability & managing the data pipeline13:26 The most troublesome data layer20:10 Acceldata value prop24:58 A typical observability engagement29:08 AI is built in30:38 Early client adoption33:26 Preventing future data reliability issues35:10 Acceldata's futures and why now?Find Rohit: https://www.linkedin.com/in/rconline/Find Acceldata : https://www.linkedin.com/company/acceldata/Want to be featured as a guest on Making Data Simple? Reach out to us at [almartintalksdata@gmail.com] and tell us why you should be next. Abstract Making Data Simple Podcast is hosted by Al Martin, WW VP Account Technical Leader IBM Technology Sales, where we explore trending technologies, business innovation, and leadership ... while keeping it simple & fun. 

Core hypothesis is all companies are data driven, all have data complexity, and there is not enough talent out there to solve it.  Show Notes04:01 Why Start Acceldata06:02 Core hypothesis for Acceldata08:59 Will Hadoop survive? And a tip of the hat to mainframes11:12 Meaning of Data Observability & managing the data pipeline13:26 The most troublesome data layer20:10 Acceldata value prop24:58 A typical observability engagement29:08 AI is built in30:38 Early client adoption33:26 Preventing future data reliability issues35:10 Acceldata's futures and why now?Find Rohit: https://www.linkedin.com/in/rconline/Find Acceldata : https://www.linkedin.com/company/acceldata/Want to be featured as a guest on Making Data Simple? Reach out to us at [almartintalksdata@gmail.com] and tell us why you should be next. Abstract Making Data Simple Podcast is hosted by Al Martin, WW VP Account Technical Leader IBM Technology Sales, where we explore trending technologies, business innovation, and leadership ... while keeping it simple & fun. 

Kevin Hu, co-founder and CEO at Metaplane discusses "Data Observability" with host Priyanka Raghavan. The discussion touches upon Data observability roots, components, differences with software observability and tooling.

As a PhD candidate at MIT, Kevin (and friends) published Sherlock, a data type detection engine (a surprisingly bedeviling problem) for data cleaning + data discovery. Now as co-founder and CEO of Metaplane, a data observability startup, Kevin applies these same automated data discovery methods to help data teams keep their data healthy. In this conversation with Tristan & Julia, Kevin wins the coveted award for “most crystal-clear explanations of complex technical concepts through physics analogy.”   For full show notes and to read 6+ years of back issues of the podcast's companion newsletter, head to https://roundup.getdbt.com.

Some of our topic highlights include:The history and background of Monte CarloWhy the planning process is so important for data teamsHow to set (and crush) a data goalThe difference between being obsessed with systems and being obsesses with goalsHow data teams waste timeHow often data downtime and data issues are tied to a lack of knowledge transfer and process within an orgHow we can continue to improve the corporate culture around dataAs always, I'd love to hear your thoughts on the episode over on Twitter @borisjabes.Want to discuss the best practices we covered in this episode? Come hang out in The Operational Analytics Club, where all your favorite data leaders gather. Know someone that you think would be an awesome guest on The Show (hint: you can totally nominate yourself)? Reach out to our content and community team. Resources:Barr Moses on LinkedIn Barr Moses on Twitter Barr Moses on MediumMonte Carlo on Twitter Monte Carlo on LinkedIn Monte Carlo's websiteMusic by the talented Joe Stevens. 

Data observability is a term that has been co-opted by numerous vendors with varying ideas of what it should mean. At Acceldata, they view it as a holistic approach to understanding the computational and logical elements that power your analytical capabilities. In this episode Tristan Spaulding, head of product at Acceldata, explains the multi-dimensional nature of gaining visibility into your running data platform and how they have architected their platform to assist in that endeavor.

Andy Petrella joins us to chat about Data Observability Driven Development (DODD), the modern data space, and much more. Andy is a dear friend, an OG in the world of data, a brilliant engineer, and is one of the keenest people we know. Kensu: https://www.kensu.io/ #dataengineering #dataobservability #datascience --------------------------------- TERNARY DATA We are Matt and Joe, and we're "recovering data scientists". Together, we run a data architecture company called Ternary Data. Ternary Data is not your typical data consultancy. Get no-nonsense, no BS data engineering strategy, coaching, and advice. Trusted by great companies, both huge and small. Subscribe to our newsletter, or check out our services at Ternary Data Site - https://ternarydata.com Please follow our LinkedIn page - https://www.linkedin.com/company/ternary-data/ Subscribe to our YouTube and smash the like button! - https://www.youtube.com/channel/UC3H60XHMp6BrUzR5eUZDyZg Thanks for your support!

Kevin Hu (@kevinzenghu, Co-Founder | CEO at @Metaplane) talks about the concepts behind Data Observability and the unique challenges for Data Engineers.SHOW: 594CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotwCHECK OUT OUR NEW PODCAST - "CLOUDCAST BASICS"SHOW SPONSORS:BMC Wants to Know if your business is on it's A-GameBMC Autonomous Digital EnterpriseUsage.ai (homepage)Start saving up to 57% of your AWS EC2 spend in under 5 minutes with Usage AI. No code change, no downtime, no engineering work required.New Relic (homepage)Services down? New Relic offers full stack visibility with 16 different monitoring products in a single platform.SHOW NOTES:Metaplane (homepage)Topic 1 - Welcome to the show. Let's talk about your background and what led you to start Metaplane.Topic 2 - Let's start by talking about the concept of what is a modern data engineer. What is this person doing, what are they responsible for, and who are their typical “customers” within a business. Topic 3 - Beyond just huge volumes of data and trying to make the data usable (formatting, ETL, storage access, etc.), what sort of problems do data engineers encounter? How much is typically “first-party data” and how much comes from external systems? Topic 4 - Let's talk about Data Observability. First off, what is it?. And second, how is it different from the Observability that we've seen from Datadog or Honeycomb or Observe or many others? Topic 5 - What are the types of Data Observability problems that Metaplane is focused on solving for Data engineers? Are these usually done independently, or in collaboration with the application or business analyst teams?Topic 6 - What are some of the immediate results (improvements) that companies see when adding Data Observability to their environments?FEEDBACK?Email: show at the cloudcast dot netTwitter: @thecloudcastnet

As always, I'd love to hear your thoughts on the episode over on Twitter @borisjabes.Know someone that you think would be an awesome guest on The Show (hint: you can totally nominate yourself)? Reach out to our content and community team. Resources:Bigeye's websiteBigeye on TwitterBigeye on LinkedinBigeye on MediumEgor on LinkedinMusic by the talented Joe Stevens. 

Making the business case for data observability with Rohit Choudhary - CEO of Acceldata Listen to this episode on Anchor FM How do you make the business case for data observability? In this episode of DATAcated on Air, host Kate Strachnyi speaks with Rohit Choudhary, CEO of Acceldata, about data observability, what it means, and why it's important. Listen in to learn more about how data observability is helping businesses reach their goals. You will want to hear this episode if you are interested in... What is data observability? [02:49] Types of companies Acceldata serves [07:24] Starting the conversation [08:24] What does Acceldata do? [10:21] Pitching data observability [16:19] The importance of data observability [21:55] Success with PhonePe [30:55] How does the system handle data drifts? [32:37] The future of data observability [37:03] Resources & People Mentioned Acceldata Zillow PhonePe Sales@acceldata.io Info@acceldata.io Acceldata | LinkedIn Acceldata on Twitter Connect with Rohit Choudhary On LinkedIn Connect with DATAcated http://www.datacated.com/ DATAcated on LinkedIn: https://www.linkedin.com/company/datacated1/ Kate on LinkedIn: https://www.linkedin.com/in/kate-strachnyi-data/ DATAcated on Twitter: https://twitter.com/datacated_ DATAcated on YouTube: https://www.youtube.com/datacated Subscribe to the DATACATED On Air podcast --- Support this podcast: https://anchor.fm/datacated/support

In this episode, Kevin Hu joins the podcast to talk about founding and growing the data observability startup, Metaplane. Listen in to hear about his time in academia at MIT, his experience with Y Combinator, and his current routine as a technical founder. In this episode you will learn: • What is data observability? [4:35] • How to identify data quality issues? [8:56] • Kevin's PhD research on automating data science systems using machine learning [16:18] • Why Kevin launched Metaplane [28:50] • The pros and cons of an academic career relative to the start-up hustle [31:57] • Kevin's experience in Y-Combinator accelerator [39:50] • The software tools he uses daily as a CEO [50:54] • What Kevin looks for in data engineer hires [56:13] Additional materials: www.superdatascience.com/541

Kevin Hu (CEO/Co-founder of Metaplane) joins the Monday Morning Data Chat to discuss the trends in data observability in 2022. Streamed live on LinkedIn and YouTube. #dataengineering #dataobservability #metaplane --------------------------------- TERNARY DATA We are Matt and Joe, and we're "recovering data scientists". Together, we run a data architecture company called Ternary Data. Ternary Data is not your typical data consultancy. Get no-nonsense, no BS data engineering strategy, coaching, and advice. Trusted by great companies, both huge and small. Subscribe to our newsletter, or check out our services at Ternary Data Site - https://ternarydata.com Please follow our LinkedIn page - https://www.linkedin.com/company/ternary-data/ Subscribe to our YouTube and smash the like button! - https://www.youtube.com/channel/UC3H60XHMp6BrUzR5eUZDyZg Thanks for your support!

Data observability is a set of technical and organizational capabilities related to understanding how your data is being processed and used so that you can proactively identify and fix errors in your workflows. In this episode Metaplane founder Kevin Hu shares his working definition of the term and explains the work that he and his team are doing to cut down on the time to adoption for this new set of practices. He discusses the factors that influenced his decision to start with the data warehouse, the potential shortcomings of that approach, and where he plans to go from there. This is a great exploration of what it means to treat your data platform as a living system and apply state of the art engineering to it.

I recently discovered a Belgian company called Soda who focuses on data quality monitoring and testing. Think of it a bit like application or software monitoring which is now ubiquitous. Soda scans for issues caused by human error, firmware upgrades, schema changes, cross-platform integration snafus, bought data, or transformation bugs. The aim is to automate, verify, and validate the flow of data from various sources and encourage collaboration between software/data engineers and downstream business decision-makers. The big idea here is to eliminate so-called ‘silent data issues.' These are the issues that bubble up (hence the name Soda!) further downstream having gone undetected in datasets. In an era when most companies are moving towards being entirely data-driven, ensuring data is of the optimum quality is critical. I invited Soda CEO and founder Maarten Masschelein on the Tech Talks Daily to learn more. Maarten discusses why data quality monitoring is becoming a hot topic and provides examples of where bad data has caused serious problems. I also learn how Soda was listed as a data management Cool Vendor by Gartner.

Do you want to learn more about Data Observability? In this episode Barr Moses, Co-Founder and CEO at Monte Carlo, talks about her journey, data observability, data quality, building data teams, data mesh architecture, and much more! #data #datascience #python #machinelearning #analytics #moderndatastack #ai #bi #artificialintelligence #dataobservability

Imagine for a minute that you're running a profitable business, and that part of your sales strategy is to send the occasional mass email to people who've signed up to be on your mailing list. For a while, this approach leads to a reliable flow of new sales, but then one day, that abruptly stops. What happened? You pour over logs, looking for an explanation, but it turns out that the problem wasn't with your software; it was with your data. Maybe the new intern accidentally added a character to every email address in your dataset, or shuffled the names on your mailing list so that Christina got a message addressed to “John”, or vice-versa. Versions of this story happen surprisingly often, and when they happen, the cost can be significant: lost revenue, disappointed customers, or worse — an irreversible loss of trust. Today, entire products are being built on top of datasets that aren't monitored properly for critical failures — and an increasing number of those products are operating in high-stakes situations. That's why data observability is so important: the ability to track the origin, transformations and characteristics of mission-critical data to detect problems before they lead to downstream harm. And it's also why we'll be talking to Kevin Hu, the co-founder and CEO of Metaplane, one of the world's first data observability startups. Kevin has a deep understanding of data pipelines, and the problems that cap pop up if you they aren't properly monitored. He joined me to talk about data observability, why it matters, and how it might be connected to responsible AI on this episode of the TDS podcast. Intro music: ➞ Artist: Ron Gelinas ➞ Track Title: Daybreak Chill Blend (original mix) ➞ Link to Track: https://youtu.be/d8Y2sKIgFWc 0:00 Chapters: 0:00 Intro 2:00 What is data observability? 8:20 Difference between a dataset's internal and external characteristics 12:20 Why is data so difficult to log? 17:15 Tracing back models 22:00 Algorithmic analyzation of a date 26:30 Data ops in five years 33:20 Relation to cutting-edge AI work 39:25 Software engineering and startup funding 42:05 Problems on a smaller scale 46:40 Future data ops problems to solve 48:45 Wrap-up

"Observability October" continues! This week, we have Josh Benamram (CEO of Databand) on the show to talk about data quality and data observability, and where they fit into your data pipeline. Streamed live on LinkedIn and YouTube #data #dataengineering #machinelearning --------------------------------- TERNARY DATA We are Matt and Joe, and we're "recovering data scientists". Together, we run a data architecture company called Ternary Data. Ternary Data is not your typical data consultancy. Get no-nonsense, no BS data engineering strategy, coaching, and advice. Trusted by great companies, both huge and small. Ternary Data Site - https://ternarydata.com LinkedIn - https://www.linkedin.com/company/ternary-data/ YouTube - https://www.youtube.com/channel/UC3H60XHMp6BrUzR5eUZDyZg

Andy Dang joins the Monday Morning Data Chat to discuss ML and data observability. If you're an ML or data engineer - or want to learn more about observability - you'll learn a lot from Andy! Streamed live on LinkedIn and YouTube #data #dataengineering #machinelearning --------------------------------- TERNARY DATA We are Matt and Joe, and we're "recovering data scientists". Together, we run a data architecture company called Ternary Data. Ternary Data is not your typical data consultancy. Get no-nonsense, no BS data engineering strategy, coaching, and advice. Trusted by great companies, both huge and small. Ternary Data Site - https://ternarydata.com LinkedIn - https://www.linkedin.com/company/ternary-data/ YouTube - https://www.youtube.com/channel/UC3H60XHMp6BrUzR5eUZDyZg

Lior Gavish (CTO, co-founder @ Monte Carlo) joins the Monday Morning Data Chat to discuss data observability, and how you can start doing it in your stack today. Streamed live on LinkedIn and YouTube #data #dataengineering #machinelearning --------------------------------- TERNARY DATA We are Matt and Joe, and we're "recovering data scientists". Together, we run a data architecture company called Ternary Data. Ternary Data is not your typical data consultancy. Get no-nonsense, no BS data engineering strategy, coaching, and advice. Trusted by great companies, both huge and small. Ternary Data Site - https://ternarydata.com LinkedIn - https://www.linkedin.com/company/ternary-data/ YouTube - https://www.youtube.com/channel/UC3H60XHMp6BrUzR5eUZDyZg

Timestamps(01:42) Gleb shared briefly about his upbringing and studying Economics in university in Russia.(04:15) Gleb discussed his move to the US to pursue a Master of Information Systems Management at Carnegie Mellon University.(07:07) Gleb went over his summer internship as a Business Analyst at Autodesk.(08:40) Gleb shared the details of his project architecting data model/ETL pipelines as a PM at Autodesk.(11:34) Gleb unpacked the evolution of his career at Lyft — from an individual data analyst to a PM on data tooling and a high-impact project that he worked on.(16:54) Gleb shared valuable lessons from the experience of leading multiple cross-functional teams of engineers and growing the data organization significantly.(19:48) Gleb mentioned his time as a Product Manager at Phantom Auto, leading the development of a teleoperation product for autonomous vehicles over cellular networks.(25:28) Gleb emphasized the critical factors to consider when choosing a working environment: trusted managers/colleagues, maturity of tools/processes, and the function of data teams within the organization.(29:10) Gleb shared the story behind the founding of Datafold, whose mission is to help companies effectively leverage their data assets while making Data Engineering & Analytics a creative and enjoyable experience.(33:04) Gleb dissected the pain points with regression testing and the benefits of using Data Diff (Datafold's first product) for data engineers.(36:54) Gleb unpacked the data monitoring feature within Datafold's data observability platform.(39:45) Gleb discussed how to choose data warehousing solutions for your use cases (and made the distinction between data warehouse and data lake).(47:03) Gleb gave insights on the need for BI and data observability/quality management tools within the modern analytics stack.(50:40) Gleb emphasized the importance of tooling integration for Datafold's roadmap.(52:07) Gleb has been hosting Data Quality meetups to discuss the under-explored area of data quality.(54:02) Gleb shared his learnings from going through the YC incubator in summer 2020.(55:45) Gleb discussed the hurdles he had to jump through to find early customers of Datafold.(57:47) Gleb emphasized valuable lessons he has learned to attract the right people who are excited about Datafold's mission.(59:17) Gleb shared his advice for founders who are in the process of finding the right investors for their companies.(01:02:11) Closing segment.Gleb's Contact InfoLinkedInDatafold (Twitter and LinkedIn)Data Quality MeetupsMentioned ContentCourseHarvard's CS50: Introduction to Computer ScienceBlog PostsModern Analytics Stack (June 2020)Choosing Data Warehouse for Analytics (June 2020)3 Ways To Be Wrong About Open-Source Data Warehousing Software (June 2020)Buy Not Build (Aug 2020)Datafold Raises a $2.1M Seed Round Led by NEA (Nov 2020)Datafold + dbt: The Perfect Stack for Reliable Data Pipelines (Feb 2021)PeopleMaxime Beauchemin (Founder and CEO at Preset, creator of Apache Superset and Apache Airflow)Tobias Macey (Host of the Data Engineering Podcast)Books“How To Measure Anything” (by Douglas Hubbard)“Lean Analytics” (by Benjamin Yoskovitz and Alistair Croll)NotesMy conversation with Gleb was recorded back in March 2021. Since the podcast was recorded, a lot has happened at Datafold! I'd recommend:Reading Gleb's open-source edition of the modern data stack.Listening to Gleb's appearance on the Data Engineering podcast.Watching the lightning talks and panel discussions from recent Data Quality meetups number 4 and number 5.About the showDatacast features long-form, in-depth conversations with practitioners and researchers in the data community to walk through their professional journeys and unpack the lessons learned along the way. I invite guests coming from a wide range of career paths — from scientists and analysts to founders and investors — to analyze the case for using data in the real world and extract their mental models (“the WHY and the HOW”) behind their pursuits. Hopefully, these conversations can serve as valuable tools for early-stage data professionals as they navigate their own careers in the exciting data universe.Datacast is produced and edited by James Le. Get in touch with feedback or guest suggestions by emailing khanhle.1013@gmail.com.Subscribe by searching for Datacast wherever you get podcasts or click one of the links below:Listen on SpotifyListen on Apple PodcastsListen on Google PodcastsIf you're new, see the podcast homepage for the most recent episodes to listen to, or browse the full guest list.

Apple is facing an antitrust complaint in India for allegedly abusing its App Store's dominance by forcing developers to use its proprietary in-app payments system. The allegations are similar to a case Apple is facing in Europe, where regulators are probing the iPhone maker's imposition of an in-app fee of 30 percent for the distribution of paid digital content, and other restrictions. In our tech conversation, Acceldata's Rohit Choudhary unpacks his data observability platform

Barr Moses discusses with Sam about bringing DevOps into Data Engineering, building a data startup, and letting joy guide your way to creating impact. Learn how being data-driven depends on systems of people and trust. See omnystudio.com/listener for privacy information.

Barr Moses, Co-Founder & CEO of Monte Carlo, joins Hashmap on Tap host, Kelly Kohlleffel, for a chai tea and a discussion on data observability and its immense value. Monte Carlo is an ML-based data observability engine that helps enable greater trust in data by preventing and eliminating data downtime and increasing data reliability. Barr and the Monte Carlo team are helping customers solve this painful issue and she shares her experiences from when she first noticed the need for data observability to where the company is today. Show Notes: Monte Carlo: https://www.montecarlodata.com/ Check out Barr's blog post on data observability Read Barr's article on Forbes Connect with Barr on LinkedIn: https://www.linkedin.com/in/barrmoses/ Start your own conversation around modernizing your data stack: https://www.hashmapinc.com/workshop-dataintegration On tap for today's episode: Chai Tea!

We covered: Barr’s background Market gaps in data reliability Observability in engineering Data downtime Data quality problems and the five pillars of data observability Example: job failing because of a schema change Three pillars of observability (good pipelines and bad data) Observability vs monitoring Finding the root cause Who is accountable for data quality? (the RACI framework) Service level agreements Inferring the SLAs from the historical data Implementing data observability Data downtime maturity curve Monte carlo: data observability solution Open source tools Test-driven development for data Is data observability cloud agnostic? Centralizing data observability Detecting downstream and upstream data usage Getting bad data vs getting unusual data Links: Learn more about Monte Carlo: https://www.montecarlodata.com/ The Data Engineer's Guide to Root Cause Analysis: https://www.montecarlodata.com/the-data-engineers-guide-to-root-cause-analysis/ Why You Need to Set SLAs for Your Data Pipelines: https://www.montecarlodata.com/how-to-make-your-data-pipelines-more-reliable-with-slas/ Data Observability: The Next Frontier of Data Engineering: https://www.montecarlodata.com/data-observability-the-next-frontier-of-data-engineering/ To get in touch with Barr, ping her in the DataTalks.Club group or use barr@montecarlodata.com Join DataTalks.Club: https://datatalks.club/slack.html

This morning Monte Carlo, a startup focused on helping other companies better monitor their data inflows, announced that it has closed a $25 million Series B. The round, which was co-led by GGV and Redpoint, comes mere months after its September Series A that was worth $15 million. Accel led the company's Series A and […]

Data lakes and data warehouses store high volumes of multidimensional data. Data sources for these pieces of infrastructure can become unreliable for a variety of reasons. When data sources break, it can cause downstream problems. One company working to solve the problem of data reliability is Monte Carlo Data. Barr Moses and Lior Gavish are founders of Monte Carlo and join the show to talk about data reliability and the overall landscape of data infrastructure.