Podcasts about azure databricks

In this episode, Frank and Steve discuss the July 2024 FinOps announcements from AWS, Azure, and Google Cloud. They cover topics such as committed use discounts, fully licensed commitments, hibernation for VMs, cost management features, and service discontinuations. The conversation is filled with insights and analysis of the latest developments in the cloud industry.There are new machine types available on GCP, such as the C3 bare metal machine types and the sixth generation Intel-based VMs.Serverless compute options, like Azure Databricks and OpenSearch serverless, provide efficient and cost-effective ways to run workloads without managing infrastructure.Storage enhancements, such as the Convert Azure Premium SSD v2 disk and Amazon FSX for Open ZFS, offer improved performance and cost optimization.Log management tools, like Azure Monitor Logs and AWS Cost Categories, help users collect, analyze, and act on telemetry data from various resources.Cost optimization features, like AWS Focus 1.0 and GCP backup commitment-based use discounts, provide ways to optimize cloud spending. Committed use discounts are available for backup for GKE, allowing users to commit to a consistent amount of usage in exchange for a discounted rate.AWS now offers fully licensed commitments and portable license commitments for purchasing VMware engine commitments, providing customers with more options for using VMware in the cloud.VM hibernation is now generally available in Azure and AWS, allowing users to save compute costs by persisting the VM state in memory and deallocating the VM.Azure has introduced a cost card in the portal, allowing engineers to estimate costs and adjust as needed before deploying VMs.Google Cloud's FinOps Hub now allows users to view their carbon footprint and estimated greenhouse gas emissions for their Google Cloud usage.AWS has announced the discontinuation of several services, including Cloud9, SimpleDB, Forecast, and CodeCommit, indicating a shift in focus and priorities.

In this episode, we're joined by Fred Heller, Senior Director of Data Science and Machine Learning for and Senior Principal & Solution Architect Chris Bennett from Hitachi Solutions to discuss the key highlights and expectations for the upcoming Databricks Data & AI Summit, June 10-13. Host Dave Horstein steers the conversation through the latest partnerships, technological advancements, and innovative solutions that are shaping the future of big data and AI. Get ready for engaging insights and expert analysis on what's new, what's next, and how Hitachi Solutions is actively participating in this transformative event. In this episode you'll hear:·       An introduction to the Summit, it's importance in the data community, and how industry and technology leaders use this platform to share insights. ·       Hitachi Solutions' Role at the Summit, including how Hitachi Solutions is leveraging partnerships with Microsoft and Databricks to deliver cutting-edge solutions.·       Technological Breakdown: Fred provides a digestible explanation of the latest tech buzzwords, including Azure Databricks and Unity Catalog, demonstrating their practical applications in business scenarios.global.hitachi-solutions.com

Podcast SummaryTitle: Microsoft Fabric Conference Wrap Up with Hitachi SolutionsPodcast Air Date: Wednesday, April 17, 2024Featuring: Brad Koontz hosts with Sandeep Pawar and Miles ColeKey Points Discussed:Microsoft Fabric Conference Customer InsightsBrad Koontz introduces Microsoft's first-ever Fabric Conference (Fab Con) held in Las Vegas, with Hitachi Solutions expert presentationsThe conference featured 130 sessions with hands-on experience with Microsoft Fabric capabilities like data warehousing, data movement, AI, real-time analytics, and business intelligence.Hitachi Solutions has been in partnership with Microsoft for nearly two years, offering "Fabric in a Day" training since the previous November.Announcements and IntegrationsMicrosoft announced new fabric capabilities and Hitachi Solutions presented their own sessions at the conference.One significant investment was the integration between Microsoft and Databricks for better interoperability of platforms related to Lakehouse architecture.Mirroring was a new feature showcased, allowing for the replication of databases outside of Fabric, which facilitates data analysis without physical data movement.Hitachi Solutions' Role and OfferingHitachi Solutions is positioned to assist customers with the integration of fabric and their existing investments due to their close relationship and collaboration with Microsoft.Key Takeaways and Call to Action:For Existing Lakehouse Architecture Users:Customers can take advantage of fabric immediately, but nuances with integration exist.Action Item: Hitachi Solutions to provide guidance on interoperability between platforms like Azure Databricks and Microsoft Fabric.For New Users and Educating Customers:Interest in Fabric is high, as demonstrated by conference turnout.Hitachi Solutions offers our "Fabric in a Day" as well as upcoming dates for Microsoft's Fabric Analytics in a Day" events for customersNext Steps:Further engagement with Hitachi Solutions for education on Fabric's capabilities and assistance in implementing new technologies.Related Resources:Jumpstart Your Adoption of Microsoft Fabric with Lumada Empower Data Platform – Hitachi Solutions (hitachi-solutions.com)Fabric in a Day – Hitachi Solutions (hitachi-solutions.com)A Complete Data Estate in 7 Days? We Do It Every Day. – Hitachi Solutions (hitachi-solutions.com)Next Gen Business Intelligence: Microsoft Power BI + Fabric – Hitachi Solutions (hitachi-solutions.com)global.hitachi-solutions.com

In this first episode back from (a horrible) vacation, the trio talk about news from the Fabric blog, the wonderful name change for Microsoft Entra, news from Intune, something called Azure Boost, the general availability of the cross-region load balancer offering and new security and compliance options for Azure Databricks. Hosted on Acast. See acast.com/privacy for more information.

Welcome episode 219 of The Cloud Pod podcast - where the forecast is always cloudy! Today your hosts are Justin and Jonathan, and they discuss all things cloud, including clickstream analytics, databricks, Microsoft Entra, virtual machines, Outlook threats, and some major changes over at the Google Cloud team.  Titles we almost went with this week: TCP is not Entranced with Entra ID The Cave you Fear to Entra, Holds the Treasure you Seek Microsoft should rethink Entra rules for their Email A big thanks to this week's sponsor: Foghorn Consulting, provides top-notch cloud and DevOps engineers to the world's most innovative companies. Initiatives stalled because you have trouble hiring?  Foghorn can be burning down your DevOps and Cloud backlogs as soon as next week.

Data modernization refers to the process of upgrading and transforming data systems, infrastructure, and processes to meet the demands of modern data-driven organizations. It involves the adoption of new technologies and techniques to increase data quality, speed, scalability, and agility. To help organizations navigate this complex process, several data modernization patterns have emerged that provide a framework for modernizing data systems. In this episode, you'll learn: The types of challenges that come with building a data warehouse How customers can embrace modernization The challenges an organization may face going on-premises or to the cloud Some questions we ask: How does modernization data differ from traditional data warehouses? What are some of the major challenges that customers face today? Is the data warehouse dead? Guest bio Entrepreneur and International Business Management Executive Jeeva Akr leads the Cloud Scale Analytics go-to-market for Microsoft, growing the global sales of Azure Cloud Scale Analytics offerings, including Azure Synapse, Azure Databricks, Azure Stream Analytics, Azure Data Factory, Microsoft Purview, and more. He leads a direct team of sales strategists, program owners, go-to-market leaders, and partner development leaders, providing thought leadership and managing sales execution of the entire global business. Resources: Jeeva Akr on LinkedIn Patrick LeBlanc on LinkedIn Discover and follow other Microsoft podcasts at microsoft.com/podcasts Hosted on Acast. See acast.com/privacy for more information.

Get an overview of forming enterprise decision strategies with the help of Data Lake Technologies. We will cover Overview of Data Aggregation Strategies Data Lake Dive into Data Lake Medallion Architecture Understanding Data Lake Storage mechanism like S3 and Azure Gen 2 Storage Orchestrating and Transforming Data context of AWS Glue / Azure Data Factory Overview of Spark and integrations with Azure and AWS , Azure Databricks and AWS Databricks Parquet file system Variations of Data Lake - Datalake as a service Delta Lake --- Send in a voice message: https://podcasters.spotify.com/pod/show/vishnu-vg/message

Jon, Susanne and guest host Jake Switzer are joined by Anthony Brock from the Indiana Pacers. They discussed how customer experience data drives their business, how data and sustainability principles helped shaped the multi-million dollar renovation of the Gainbridge Fieldhouse, and their transition from a traditional data warehouse approach to a data lake house infrastructure, utilizing Azure Data Factory and Azure Databricks. As a bonus, we express our mutual love of Dolly Parton. About Anthony Brock Anthony Brock is the Sr. Director of Customer Insights & Business Analytics at Pacers Sports & Entertainment. His career started in sales, then moved to founding a digital marketing firm. After four years in the startup space, he moved to global manufacturing at Endress+Hauser. He joined PSE nearly two years ago to lead the data/insights team. Since joining in 2021, Anthony's team has restructured the data architecture to drive customer and revenue insights, while reducing operating costs. He is also an Adjunct Professor at Marian University.

The team catches up with the developers of the Databricks Accelerator for Azure Purview to learn when, where, and why you might use it.   Media file: https://azpodcast.blob.core.windows.net/episodes/Episode441.mp3 YouTube: https://youtu.be/W9Dyb6E5eKk Resources: The Databricks to Purview Solution Accelerator Repo: microsoft/Purview-ADB-Lineage-Solution-Accelerator: A connector to ingest Azure Databricks lineage into Microsoft Purview (github.com) Demo Deployment Quickstart: Purview-ADB-Lineage-Solution-Accelerator/deploy-demo.md at release/2.1 · microsoft/Purview-ADB-Lineage-Solution-Accelerator (github.com) YouTube Video overview: Demoing the Azure Databricks lineage solution accelerator in Microsoft Purview - YouTube The OpenLineage Repo: OpenLineage/OpenLineage: An Open Standard for lineage metadata collection (github.com) OpenLineage + Purview Blog: Microsoft Purview Accelerates Lineage Extraction from Azure Databricks | OpenLineage   Other updates: Public preview: 128 vCore option for Azure SQL Database standard-series hardware | Azure updates | Microsoft Azure - 415 GB of memory   Azure Basic Load Balancer will be retired on 30 September 2025—upgrade to Standard Load Balancer | Azure updates | Microsoft Azure    https://azure.microsoft.com/en-us/blog/microsoft-and-int-deploy-ivaap-for-osdu-data-platform-on-microsoft-energy-data-services/   Azure Machine Learning—General availability updates for September 2022 | Azure updates | Microsoft Azure   Azure Machine Learning—Public preview updates for September 2022 | Azure updates | Microsoft Azure   Public preview: Azure Firewall Basic | Azure updates | Microsoft Azure  designed for SMB ; cost effective SKU   https://azure.microsoft.com/en-us/blog/strengthen-your-security-with-policy-analytics-for-azure-firewall/

On The Cloud Pod this week, Amazon announces Amazon Inspector's new support of Windows OS for continual software vulnerability scanning of EC2 workloads, Google has several exciting announcements regarding Chronicle, Azure is announcing pretty much everything under the sun, and Oracle announces OCI Lake in beta. Thank you to our sponsor, Foghorn Consulting, which provides top notch cloud and DevOps engineers to the world's most innovative companies. Initiatives stalled because you're having trouble hiring? Foghorn can be burning down your DevOps and Cloud backlogs as soon as next week. Episode Highlights ⏰  Amazon Inspector now supports Windows operating system (OS) for continual software vulnerability scanning of EC2 workloads. ⏰  Google makes 3 announcements about Chronicle. ⏰  Azure has three–yes, three–new releases this week. ⏰ Oracle announces OCI Lake in beta. Top Quote

On The Cloud Pod this week, the team gets judicial on the Microsoft-Unity partnership. Plus: Amazon acquires iRobot, BigQuery boasts Zero-ETL for Bigtable data, and Serverless SQL for Azure Databricks is in public preview.  A big thanks to this week's sponsor, Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights

### Apero* Les pires réalisations de DALL-E (2 ou version mini ?) -> https://huggingface.co/spaces/dalle-mini/dalle-mini* HOW DALL-E COULD POWER A CREATIVE REVOLUTION -> https://www.theverge.com/23162454/openai-dall-e-image-generation-tool-creative-revolution### Database* Introducing Unistore, Snowflake's New Workload for Transactional and Analytical Data -> https://www.snowflake.com/blog/introducing-unistore/* Snowflake summit 2022 -> https://www.montecarlodata.com/snowflake-summit-2022-keynote-recap-disrupting-data-application-development-in-the-cloud/* PostgreSQL et le principe de "Privacy By Design" -> https://blog.dalibo.com/2022/05/23/privacy-by-design.html### ML* Back from MS Build 2022 : Azure ML -> https://www.youtube.com/watch?v=pxY4i76LMSI* Extension VSCode pour DVC et nouvelles features -> https://marketplace.visualstudio.com/items?itemName=Iterative.dvc### Catalog* Lineage de Azure Databricks dans Microsoft Purview -> https://github.com/microsoft/Purview-ADB-Lineage-Solution-Accelerator### Tooling* La prochaine refonte de l'IHM Intellij IDEA -> ### No Code* Coder Moins Coder Mieux -> https://www.programmez.com/magazine/article/low-code-raise-citizen-developer* Développer avec peu ou sans code, mais développer quand même -> https://www.programmez.com/magazine/article/low-code-raise-citizen-developer* Les dix commandements d'une plateforme no-code mature -> https://blog.octo.com/les-dix-commandements-dune-plateforme-no-code-mature/### Culture* The Billion dollar code (la mini série) -> https://www.netflix.com/title/81074012* The Billion dollar code (le making-of, documentaire) -> https://www.netflix.com/title/81503864SponsorsCette publication est sponsorisée par [Affini-Tech](https://affini-tech.com/) et [CerenIT](https://www.cerenit.fr/).[CerenIT](https://www.cerenit.fr/) vous accompagne pour concevoir, industrialiser ou automatiser vos plateformes mais aussi pour faire parler vos données temporelles. Ecrivez nous à [contact@cerenit.fr](mailto:contact@cerenit.fr) et retrouvez-nous aussi sur [Time Series France](https://www.timeseriesfr.org/).Affini-Tech vous accompagne dans tous vos projets Cloud et Data, pour Imaginer, Expérimenter etExecuter vos services ! ([Affini-Tech](http://affini-tech.com), La plateforme [Datatask](https://datatask.io/)) pour accélérer vos services Data et IAConsulter le [blog d'Affini-Tech](https://affini-tech.com/blog/) et le [blog de Datatask](https://datatask.io/blog/) pour en savoir plus.On recrute ! Venez cruncher de la data avec nous ! Ecrivez nous à [recrutement@affini-tech.com](mailto:recrutement@affini-tech.com)Le générique a été composé et réalisé par Maxence Lecointe.

Jon and Susanne are joined by cloud solution architect Jake Switzer, where they recap and discuss key takeaways from the MIT Sloan Sports Analytics Conference, held March 4 and 5, 2022. Jake Switzer has been using technology to build data-oriented solutions since his time as a student at the University of Alabama. He has held delivery and advisory roles at Microsoft for over nine years, including as a consultant and cloud solution architect. Jake has designed and developed data platform and advanced analytics solutions for an assortment of Microsoft enterprise customers to ensure that their specific business needs were met. Over the last few years, he has focused on advising Microsoft's sports customers how to design and build modern data solutions in Azure. His responsibilities in this role include providing architecture guidance, building proof of concepts, aiding in production deployments, and troubleshooting support issues. He is well-versed in a variety of data engineering technologies and frameworks such as SQL Server, Apache Spark, Azure Data Factory, Azure Databricks, Azure Synapse Analytics, and Power BI. In his free time, he enjoys spending time outdoors hiking and can be found most weekends cooking and sharing a scotch with his wife.

Processing data in real time is a process, as some might say. Angela Chu (Solution Architect, Databricks) and Caio Moreno (Senior Cloud Solution Architect, Microsoft) explain how to integrate Azure, Databricks, and Confluent to build real-time data pipelines that enable you to ingest data, perform analytics, and extract insights from data at hand. They share about where to start within the Apache Kafka® ecosystem and how to maximize the tools and components that it offers using fully managed services like Confluent Cloud for data in motion.EPISODE LINKSConsuming Avro Data from Apache Kafka Topics and Schema Registry with Databricks and Confluent Cloud on Azure Azure Data Lake Storage Gen2 introductionBest practices for using Azure Data Lake Storage Gen2Join the Confluent CommunityLearn more with Kafka tutorials, resources, and guides at Confluent DeveloperLive demo: Kafka streaming in 10 minutes on Confluent CloudUse 60PDCAST to get an additional $60 of free Confluent Cloud usage (details)

I asked 5 questions in 20 minutes from Dinesh Priyankara (MVP) on Azure Databricks Watch it on YouTube -> https://www.youtube.com/watch?v=c47_3C_wafQ&t=441s

In this special episode, Todd Dube & Julia Barnhart share their real world experiences with Azure Databricks working on a Fortune 500 website. This episode was recorded live at the Azure Data Fest in Reston, VA on Oct 11, 2019. You can watch the entire live stream here: http://franksworld.com/2019/10/11/azure-data-fest-reston-live-stream/

Frank and Andy mix things up a bit and talk about running R in SQL Azure, becoming Anti-Fragile, Appalachia, and how they got blocked by a big time blogger. Links (http://thedatadrivenbook.com) Sponsor: Audible.com (http://thedatadrivenbook.com) – Get a free audio book when you sign up for a free trial! Notable Quotes Andy and Frank agree The Expanse (https://www.imdb.com/title/tt3230854/) is well-written. ([02:00]) Frank’s super-secret conference… wasn’t. ([04:00]) You should definitely check out Franks World (http://www.franksworld.com/) ([04:30]) Keep up with Azure Data Fest (https://twitter.com/azuredatafest) on Twitter ([05:00]) AI Super-Powers (https://www.amazon.com/dp/B0795DNWCF/) ([05:20]) Frank and Andy “learned a lot” when we tried to land a “big fish”… ([05:40]) … and were blocked on Twitter ([06:15]) (It’s all Andy’s fault. Frank’s Twitter block was collateral damage.) ([06:30]) Frank is a Microsoft AI Ambassador ([07:15]) Check out the show with Ronald Schmelzer and Kathleen Walch on AI, Enterprises, and Startups (http://datadriven.tv/ronald-schmelzer-kathleen-walch-ai-enterprises-startups/) ([08:00]) Shoutout to Milena Rodban (http://datadriven.tv/milena-rodban-geopolitical-risk-cybersecurity-tennis/) and her show on Geopolitical Risk, Cybersecurity, and Tennis ([08:30]) Milena’s LinkedIn article ([09:15]) DIVE DIVE DIVE ([10:00]) “R-uh” ([10:45]) Kent’s show (http://datadriven.tv/kent-bradshaw-microsoft-data-science-professional-certification/) ([11:00]) Frank has another certification: AI ([11:15]) “No brakes on the F train…” ([12:00]) Frank has 36 certifications in the past 2.5 years ([12:30]) COBOL mentioned… ([13:00]) Regarding “SELECT *…” ([14:15]) More information about Azure Data Explorer (https://docs.microsoft.com/en-us/azure/data-explorer/data-explorer-overview) ([15:30]) On dataframes… ([17:30]) Setting up R in Azure (https://docs.microsoft.com/en-us/azure/sql-database/sql-database-connect-query-r) ([18:00]) Frank writes the Artificially Intelligent (https://social.msdn.microsoft.com/Search/en-US/magazine?query=Artificially%20Intelligent&pgArea=header&Refinement=118&emptyWatermark=true&ac=4) column at MSDN magazine ([20:00]) Learn more about Azure Databricks (https://azure.microsoft.com/en-us/services/databricks/) ([23:30]) Graeme Malcolm (https://www.linkedin.com/in/graemesplace/) is an awesome presenter! ([26:00]) Frank totaled his car in December 2018 ([26:30]) More information on Honda Adaptive Cruise Control (https://owners.honda.com/vehicles/information/2019/Accord-Sedan/features/Adaptive-Cruise-Control) ([28:00]) Frank’s role – as a driver – has changed. ([31:12]) Book Recommendation: Anti-Fragile (https://smile.amazon.com/dp/B0083DJWGO) ([35:30]) Frank’s brush with “Ponch” ([36:50]) Interesting article about combination of tolerances (http://adcats.et.byu.edu/Publication/87-5/WAM2.html) ([38:50]) Andy shares thoughts on the economics of self-driving trucks ([43:00]) Frank shares thoughts on the shifting role of a driver in self-driving trucks ([45:30]) “Learn how to code” is not particularly helpful ([47:00]) AFAF == “Anti-Fragile As Frank” ([47:30]) Upcoming show with Anders Schneiderman, who has not (yet) blocked us on Twitter ([50:00]) “Disruption is now the norm.” ([51:30]) Mr. T predicts pain (https://www.youtube.com/watch?v=lSPNQ82Sq4E) . ([53:30]) Frank’s *DataPoint* Be Playful With Your Data, but Judicious With Your Time (http://datadriven.tv/datapoint-playful-data-judicious-time/) ([54:30]) “Potpourri episode” ([55:55]) Book reference: @nntaleb...

Matei Zaharia is a co-founder and Chief Technologist at Databricks, an Assistant Professor of Computer Science at Stanford and the inventor of Apache Spark. Microsoft has partnered with Databricks to bring you Azure Databricks, a Spark-based analytics platform optimized for Azure offering simple setup, streamlined workflows and ease of collaboration between data scientists, engineers and business analysts. Let’s see what Matei has to say about Spark, ML and interesting AI applications he’s encountered lately. Databricks, https://databricks.com/ Azure Databricks, https://azure.microsoft.com/en-us/services/databricks/  

Gaurav Malhotra joins Lara Rubbelke to discuss how you can operationalize Jars and Python scripts running on Azure Databricks as an activity step in a Data Factory pipeline.Jump To: [01:55] Demo Start For more information:Transform data by running a Jar activity in Azure Databricks docsTransform data by running a Python activity in Azure Databricks docsAzure Databricks overviewAzure Data Factory overviewCreate a free account (Azure)Follow @sqlgal Follow @AzureFriday

Gaurav Malhotra joins Lara Rubbelke to discuss how you can operationalize Jars and Python scripts running on Azure Databricks as an activity step in a Data Factory pipeline.Jump To: [01:55] Demo Start For more information:Transform data by running a Jar activity in Azure Databricks docsTransform data by running a Python activity in Azure Databricks docsAzure Databricks overviewAzure Data Factory overviewCreate a free account (Azure)Follow @sqlgal Follow @AzureFriday

Today's business managers depend heavily on reliable data integration systems that run complex ETL/ELT workflows (extract, transform/load and load/transform data). Gaurav Malhotra joins Scott Hanselman to discuss how you can iteratively build, debug, deploy, and monitor your data integration workflows (including analytics workloads in Azure Databricks) using Azure Data Factory pipelines.   For more information:Ingest, prepare, and transform using Azure Databricks and Data Factory (blog)Run a Databricks notebook with the Databricks Notebook Activity in Azure Data Factory (docs)Create a free account (Azure)Follow @SHanselman Follow @AzureFriday Follow @gauravmalhot12

Today's business managers depend heavily on reliable data integration systems that run complex ETL/ELT workflows (extract, transform/load and load/transform data). Gaurav Malhotra joins Scott Hanselman to discuss how you can iteratively build, debug, deploy, and monitor your data integration workflows (including analytics workloads in Azure Databricks) using Azure Data Factory pipelines.   For more information:Ingest, prepare, and transform using Azure Databricks and Data Factory (blog)Run a Databricks notebook with the Databricks Notebook Activity in Azure Data Factory (docs)Create a free account (Azure)Follow @SHanselman Follow @AzureFriday Follow @gauravmalhot12

I sat down with Ali Ghodsi, CEO and found of Databricks, and John Chirapurath, GM for Data Platform Marketing at Microsoft related to the recent announcement of Azure Databricks. When I heard about the announcement, my first thoughts were two-fold.  First, the possibility of optimized integrations with existing Azure services.  This would be a big benefit to heavy Azure users who also want to use Spark.  Second, the benefits of active directory to control Databricks access for large enterprise. Hear Ali and JG's thoughts and comments on what makes Azure Databricks a novel offering.