Podcasts about ml ops

In this episode, Carlos Peralta returns to The Tech Trek to dive deep into data culture in the wearable tech space, sharing how WHOOP turns petabytes of real-time biometric data into personalized, actionable insights. We explore the technical complexities behind data ingestion, transformation, and delivery, and how the mission-driven nature of WHOOP influences both their engineering decisions and company culture.

As businesses increasingly harness the power of artificial intelligence, the road from innovation to reliable production is fraught with challenges. In this episode of the Predictable B2B Success podcast, host Vinay Koshy welcomes Brad Micklea, the visionary CEO of Jozu, to unravel the intricacies of AI integration. Jozu, an MLOps collaboration platform, bridges the gap between AI, ML, and app development, promising innovation and predictable revenue growth. While many firms acknowledge AI's potential to offer a competitive edge, only 20% have scaled their AI initiatives effectively. What causes this disparity? Data silos, talent gaps, and alignment challenges are just the tip of the iceberg. Brad shares his unique journey from founding CodeNB to scaling AI solutions that drive substantial business outcomes. With over 25 years in developing software tools, he delves into the importance of aligning AI projects with strategic goals and the benefits of decentralizing AI expertise. Prepare to be enlightened by Brad's insights on AI's role in revolutionizing production, risk mitigation, and organizational culture shifts. Tune in to discover how you can navigate the complexities of AI projects and confidently revolutionize your business strategy. Some areas we explore in this episode include: The challenges of integrating AI into business production processes.Brad Micklea's career journey and the inception of Jozu.The transformation of Jozu from its original focus to its current mission in ML Ops.Personal background and strengths of Brad Micklea, highlighting the importance of diverse perspectives.The risks and complexities involved in transitioning AI prototypes to production systems.The debate on decentralizing AI expertise versus centralizing it within organizations.The cultural shifts required for successful AI adoption and innovation within companies.The significance of open source tools in AI development and the misconceptions surrounding them.Strategies for aligning AI development with broader business objectives and regulatory compliance.Lessons learned from Brad Micklea's previous startup experience and their application to Jozu.And much, much more...

Sophia Rowland, Senior Product Manager at SAS, discusses her journey from data science to product management at SAS, focusing on the integration of AI and analytics. She explains the concepts of Model Ops and ML Ops, the challenges organizations face in operationalizing machine learning models, and the critical role of analytics in this process. Key Takeaways:  Dependency management errors that occur when IT and data science teams work in silos The connection between algorithms and psychology, using data and software to tap into motivation How to discern hype from meaningful advancements in emerging technologies The influence of user behavior on AI adoption Ways to stay updated in the rapidly evolving field of AI Guest Bio: Sophia Rowland is a Senior Product Manager focusing on ModelOps and ML Ops at SAS. In her previous role as a data scientist, Sophia worked with dozens of organizations to solve a variety of problems using analytics. As an active speaker and writer, Sophia has spoken at events like the AI Summit, All Things Open, SAS Explore, and SAS Innovate; she has also written dozens of articles and blog posts. As a lifelong North Carolinian, Sophia holds degrees from both UNC-Chapel Hill and Duke, including bachelor's degrees in computer science and psychology, and a Master of Science in Quantitative Management: Business Analytics from the Fuqua School of Business. Outside of work, Sophia enjoys reading an eclectic assortment of books, hiking throughout North Carolina, and trying to stay upright while ice skating. ---------------------------------------------------------------------------------------- About this Show: The Brave Technologist is here to shed light on the opportunities and challenges of emerging tech. To make it digestible, less scary, and more approachable for all! Join us as we embark on a mission to demystify artificial intelligence, challenge the status quo, and empower everyday people to embrace the digital revolution. Whether you're a tech enthusiast, a curious mind, or an industry professional, this podcast invites you to join the conversation and explore the future of AI together. The Brave Technologist Podcast is hosted by Luke Mulks, VP Business Operations at Brave Software—makers of the privacy-respecting Brave browser and Search engine, and now powering AI everywhere with the Brave Search API. Music by: Ari Dvorin Produced by: Sam Laliberte

Re-Platforming Your Tech Stack // MLOps Podcast #281 with Michelle Marie Conway, Lead Data Scientist at Lloyds Banking Group and Andrew Baker, Data Science Delivery Lead at Lloyds Banking Group. // Abstract Lloyds Banking Group is on a mission to embrace the power of cloud and unlock the opportunities that it provides. Andrew and his team have been on a journey over the last 12 months to take their portfolio of circa 10 Machine Learning models in production and migrate them from an on-prem solution to a cloud-based environment. During the podcast, Andrew shares his reflections as well as some dos (and don'ts!) of managing the migration of an established portfolio. // Bio Michelle Marie Conway Michelle is a Lead Data Scientist in the high-performance data science team at Lloyds Banking Group. With deep expertise in managing production-level Python code and machine learning models, she has worked alongside fellow senior manager Andrew to drive the bank's transition to the Google Cloud Platform. Together, they have played a pivotal role in modernising the ML portfolio in collaboration with a remarkable ML Ops team. Originally from Ireland and now based in London, Michelle blends her technical expertise with a love for the arts. Andrew Baker Andrew graduated from the University of Birmingham with a first-class honours degree in Mathematics and Music with a Year in Computer Science and joined Lloyds Banking Group on their Retail graduate scheme in 2015. Since 2021 Andrew has worked in the world of data, firstly in shaping the Retail data strategy and most recently as a Data Science Delivery Lead, growing and managing a team of Data Scientists and Machine Learning Engineers. He has built a high-performing team responsible for building and maintaining ML models in production for the Consumer Lending division of the bank. Andrew is motivated by the role that data science and ML can play in transforming the business and its processes, and is focused on balancing the power of ML with the need for simplicity and explainability that enables business users to engage with the opportunities that exist in this space and the demands of a highly regulated environment. // MLOps Swag/Merch https://shop.mlops.community/ // Related Links Website: https://www.michelleconway.co.uk/ https://www.linkedin.com/pulse/artificial-intelligence-just-when-data-science-answer-andrew-baker-hfdge/ https://www.linkedin.com/pulse/artificial-intelligence-conundrum-generative-ai-andrew-baker-qla7e/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Michelle on LinkedIn: https://www.linkedin.com/in/michelle--conway/ Connect with Andrew on LinkedIn: https://www.linkedin.com/in/andrew-baker-90952289

Geoff Buteau, Director of Booz Allen Hamilton AI Practice and Saeed Uri, SVP for the Sustainable Development Impact Lab at Chemonics International join Mike Shanley to discuss AI in Federal & USAID Markets. Specifically, this episode covers:  - Status of AI in USAID market - Status and applications in Federal market - AI beyond GenAI - Ideas for USAID AI applications   RESOURCES The Helix, BAH Center for Innovation Geoff Buteau LinkedIn Saeed Uri LinkedIn   BIOGRAPHY Mr. Buteau is an Artificial Intelligence technical delivery consultant and project manager with more than 19 years of experience in the defense and public sector. Geoff leads the Booz Allen Hamilton's ML Ops delivery portfolio across defense, civil, and law enforcement agencies, which includes product management, the management of technical build and integration teams, AI requirements development, and AI business, technical, and ethical risk analysis for US federal agencies. Geoff's work in Booz Allen's AI practice also includes investment initiatives in AI strategy, responsible AI, and emerging technology scouting. Geoff holds a BS in Journalism and Public Relations from Ithaca College and a Master of Int'l Affairs with a focus in development economics and management analytics from the Columbia University School of International and Public Affairs. ---- Saeed Uri is Chemonics' senior vice president for impact. He has more than fifteen years of experience managing development projects, including more than ten years in complex, high-speed, and challenging positions in fragile or transitional environments such as Iraq, Syria, Palestine, Sudan, and Libya. While leading Chemonics' efforts on adaptive programming in dynamic environments, Saeed also spearheaded the adoption of innovative approaches and technologies to achieve greater impact. In Syria, Saeed led Chemonics' partnership with the Syria Civil Defense (also known as the White Helmets) to provide emergency response services to millions of civilians. Most recently, Saeed led programming to strengthen community resilience against climate and other sources of instability by working with local partners to increase community involvement in addressing issues. Saeed also has expertise in supporting early recovery and durable returns, and countering disinformation. He holds an M.A. in international peace and conflict resolution and speaks Arabic fluently.  LEARN MORE Thank you for tuning into this episode of the Aid Market Podcast. You can learn more about working with USAID by visiting our homepage: Konektid International and AidKonekt. To connect with our team directly, message the host Mike Shanley on LinkedIn.

Send us a textWillkommen zurück aus der Sommerpause! Nach einer kurzen Auszeit sind wir wieder da und freuen uns, mit spannenden Themen rund um Künstliche Intelligenz durchzustarten. Zunächst ein kleiner Hinweis: Unsere Co-Moderatorin Nina ist kürzlich Mutter geworden – herzlichen Glückwunsch an dieser Stelle! Falls ihr im Hintergrund gelegentlich Babygeräusche hört, bitten wir um Verständnis.KI ist ein Treiber für neue Geschäftsideen. Allein in Deutschland wurden 2022 über 300 KI-Start-ups gezählt, was einen Anstieg von 9 % im Vergleich zum Vorjahr bedeutet. Die globale Finanzierung für KI-Start-ups erreichte 2023 fast 50 Milliarden US-Dollar, was zeigt, dass das Interesse an KI-Lösungen weiterhin stark wächst. Das Geschäftspotenzial für KI-Projekte wird für das Jahr 2025 sehr optimistisch eingeschätzt. Der globale KI-Markt wird voraussichtlich bis 2025 ein Volumen von über 1 Billion US-Dollar erreichen. Wichtige Treiber dieses Wachstums sind die steigende Nachfrage nach KI-Lösungen in verschiedenen Branchen wie dem Gesundheitswesen, den Finanzen, der Automobilindustrie und dem Einzelhandel. Doch Schätzungen zufolge scheitern bis zu 85 % der KI- und maschinellen Lernprojekte. Warum ist das so?Zu Gast ist heute Prof. Dr. Rene Brunner, Dozent an der Macromedia Hochschule, CEO von Datamics, Autor und Experte für ML-Ops und Daten.Fundstücke der Woche:Entwickler sind von KI nicht begeistert, ihre Manager schonKI Ampel in Essen verärgert AutofahrerOPITZ CONSULTING■■■ Digitale Service ManufakturDisclaimer: This post contains affiliate links. If you make a purchase, I may receive a commission at no extra cost to you.Nina & Frank laden sich Gäste ein und sprechen mit ihnen über aktuelle Entwicklungen im Umfeld der Künstlichen Intelligenz.

Thank you for 1m downloads of the podcast and 2m readers of the Substack!

Our guest is Magnus Malmström. He is the CTO of Sandvik. He explains to us what his customers demand from him in the area of AI, why his day-to-day work is dominated by AI and how he enriches his CAM systems. And he reveals the new AI based products by Sandvik. Thanks for listening. We welcome suggestions for topics, criticism and a few stars on Apple, Spotify and Co. We thank our partner **SIEMENS** https://www.siemens.de/de/ [More about Sandvik's AI for manufacturing:](https://bit.ly/3XHPfXH) Our guest is [Magnus Malmström](https://www.linkedin.com/in/magnusmalmstrom/) #machinelearning #ai #aimodel #industrialautomation #manufacturing #automation #genai #datascience #mlops #llm #IndustrialAI #artificialintelligence #CAM

This episode is sponsored by Oracle. AI is revolutionizing industries, but needs power without breaking the bank. Enter Oracle Cloud Infrastructure (OCI): the one-stop platform for all your AI needs, with 4-8x the bandwidth of other clouds. Train AI models faster and at half the cost. Be ahead like Uber and Cohere. If you want to do more and spend less like Uber, 8x8, and Databricks Mosaic - take a free test drive of OCI at https://oracle.com/eyeonai In this episode of the Eye on AI podcast, join us as we sit down with Lukas Biewald, CEO & co-founder of Weights & Biases, the AI developer platform with tools for training models, fine-tuning models, and leveraging foundation models.  Lukas takes us through his journey, from his early days at Stanford and his work in natural language processing, to the founding of CrowdFlower and its evolution into a major player in data annotation. He shares the insights that led him to start Weights and Biases, aiming to provide comprehensive tools for the entire machine learning workflow. Lukas discusses the importance of high-quality data annotation, the shift in AI applications, and the role of reinforcement learning with human feedback (RLHF) in refining large models. Discover how Weights and Biases helps ML practitioners with data lineage and compliance, ensuring that models are trained on the right data and adhere to regulatory standards. Lukas also highlights the significance of tracking and visualizing experiments, retaining intellectual property, and evolving the company's products to meet industry needs. Tune in to gain valuable insights into the world of ML Ops, data annotation, and the critical tools that support machine learning practitioners in deploying reliable models. Don't forget to like, subscribe, and hit the notification bell for more on groundbreaking AI technologies. Stay Updated: Craig Smith Twitter: https://twitter.com/craigss Eye on A.I. Twitter: https://twitter.com/EyeOn_AI (00:00) Preview and Intro (01:39) Lukas's Background and Career  (04:09) Founding CrowdFlower and Early Machine Learning  (06:59) Current Trends in Machine Learning (08:46) Reinforcement Learning with Human Feedback (RLHF)  (12:43) Weights and Biases: Origin and Mission   (16:44) Visualizations and Compliance in AI  (22:43) US vs. EU AI Regulations (25:20) Importance of Experiment Tracking in ML  (28:47) Evolving Products to Meet Industry Needs  (30:38) Prompt Engineering in Modern AI  (33:34) Challenges in Monitoring AI Models  (37:25) Monitoring Functions of Weights and Biases (39:33) Future of Weights and Biases

This episode focuses on “Machine Learning Operations (MLOps)," an Independent Research and Development (IRAD) project by a team of GTRI researchers that was presented in 2023 during GTRI's IRAD Extravaganza. A noteworthy part of the IRAD Extravaganza is the IRAD of the Year Ceremony, which awards particularly outstanding projects. For each annual IRAD Extravaganza, projects are nominated for "IRAD of the Year" awards. Finalists for the IRAD of the Year were judged in two categories: Large Investment Projects, with multiyear funding greater than $50,000. Small Investment Projects, which have one-year funding of $50,000 or less.  The “Machine Learning Operations (MLOps)" project won in the Large Investment Projects category.  Research team members Maia Gatlin and Austin Ruth are the guests in this podcast episode. Gatlin and Ruth are both Research Engineers in GTRI's Electronic Systems (ELSYS) Laboratory. This IRAD focuses on the development of Infrastructure as Code (IaC) to create a deployable platform of various tools for Machine Learning Operations (MLOps). The team has successfully deployed and tested the infrastructure to showcase the benefits of the platform through various use­  cases. The primary goal is to show that the infrastructure in place can not only support inference and training of machine learning models but also can incorporate active learning and continuous delivery of models to specified repositories. With the IaC, the platform is also deployable to edge and fog machines to perform tasks at the supported resource level.

An Introduction to ML Ops  Building data science products requires many things we've discussed on this podcast before: insight, customer empathy, strategic thinking, flexibility, and a whole lot of determination. But it requires one more thing we haven't talked about nearly as much: a stable, performant, and easy-to-use foundation. Setting up that foundation is the chief goal of the field of machine learning operations, aka ML Ops. This month on the Klaviyo Data Science Podcast, we give a brief but thorough introduction to the field of ML Ops. You'll hear about: How ML Ops is different from the similar fields of data science and DevOps What skills a successful ML Ops developer should have, and what an ML Ops developer's day-to-day looks like Why concepts like “velocity” and “stability” have their own special nuances in the world of ML Ops For the full show notes, including who's who, see the ⁠⁠⁠⁠⁠⁠Medium writeup⁠⁠⁠⁠⁠⁠.

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/ Amritha Arun Babu Mysore has been an expert in the field of consumer electronics, software products, and online marketplaces for the past 15 years. She has experience developing supply chains from the ground up, delivering AI-based products to millions of users, and advocating for ethical AI across Amazon, Wayfair, Salesforce, and NetApp. Abhik Choudhury is a Senior Analytics Managing Consultant and Data Scientist with 11 years of experience in designing and implementing scalable data solutions for organizations across various industries. Huge thank you to  @latticeflow  for sponsoring this episode. LatticeFlow - https://latticeflow.ai/ MLOps podcast #221 with Amritha Arun Babu Mysore, ML Product Leader at Klaviyo and Abhik Choudhury, Managing Consultant Analytics at IBM, MLOps - Design Thinking to Build ML Infra for ML and LLM Use Cases. // Abstract As machine learning (ML) and large language models (LLMs) continue permeating industries, robust ML infrastructure and operations (ML Ops) are crucial to deploying these AI systems successfully. This podcast discusses best practices for building reusable, scalable, and governable ML Ops architectures tailored to ML and LLM use cases. // Bio Amritha Arun Babu Mysore Amritha is an accomplished technology leader with over 12 years of experience spearheading product innovation and strategic initiatives at both large enterprises and rapid-growth startups. Leveraging her background in engineering, supply chain, and business, Amritha has led high-performing teams to deliver transformative solutions solving complex challenges. She has driven product road mapping, requirements analysis, system design, and launch execution for advanced platforms in domains like machine learning, logistics, and e-commerce. Abhik Choudhury Abhik is a Senior Analytics Managing Consultant and Data Scientist with 11 years of experience in designing and implementing scalable data solutions for organizations across various industries. Throughout his career, Abhik developed a strong understanding of AI/ML, Cloud computing, database management systems, data modeling, ETL processes, and Big Data Technologies. Abhik's expertise lies in leading cross-functional teams and collaborating with stakeholders at all levels to drive data-driven decision-making in longitudinal pharmacy and medical claims and wholesale drug distribution areas. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links AI Quality in Person Conference in collaboration with Kolena: https://www.aiqualityconference.com/ LatticeFlow website: https://latticeflow.ai/ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Abhik on LinkedIn: https://www.linkedin.com/in/abhik-choudhury-35450058 Connect with Amritha on LinkedIn: https://www.linkedin.com/in/amritha-arun-babu-a2273729/

Dans cet épisode, nous explorons l'univers de la métallurgie et plus précisément l'usinage de barres d'inox. Les méthodes manuelles de contrôle de qualité des outils d'usinage nécessitent de nombreuses étapes manuelles, ce qui peut être fastidieux et prendre du temps. Pour pallier ce problème, UGITECH a mis en place un système basé sur l'apprentissage machine pour simplifier la vie des opérateurs et accélérer le processus. Nous découvrons ensemble les modèles d'apprentissage machine qui ont été sélectionnés, leur entraînement et leur orchestration. Nous évoquons également ML Ops et le pipeline de déploiement des nouveaux modèles en production.

Dans cet épisode, nous explorons l'univers de la métallurgie et plus précisément l'usinage de barres d'inox. Les méthodes manuelles de contrôle de qualité des outils d'usinage nécessitent de nombreuses étapes manuelles, ce qui peut être fastidieux et prendre du temps. Pour pallier ce problème, UGITECH a mis en place un système basé sur l'apprentissage machine pour simplifier la vie des opérateurs et accélérer le processus. Nous découvrons ensemble les modèles d'apprentissage machine qui ont été sélectionnés, leur entraînement et leur orchestration. Nous évoquons également ML Ops et le pipeline de déploiement des nouveaux modèles en production.

Welcome to another exciting episode with Brian from quantlabs.net. Recorded on the 13th of March, noontime, this engaging and enlightening talk revolves around machine learning and the best practices in engineering with machine learning. Although Brian admits to not being an expert, he invites listeners, even those who may not find the subject generally useful, to engage with him as he explores this intriguing world. GET SOME FREE TRADING TECH BOOK PDFS HTTP://QUANTLABS.NET/BOOKS Join our Discord for quant trading and programming news https://discord.gg/k29hRUXdk2 Don't forget to subscribe to my Substack for more trading tips and strategies! Let's keep learning and growing together. https://quantlabs.substack.com/ The podcast delves into an article originally discovered on Reddit, within the toting subreddit. Another piece of content that sparks discussion is an article from Medium.com penned by Luis Bermondes which gives an overview of ML Ops (Machine Learning Operations). Of particular interest is a diagram depicting the ML op stack and the direction it operates in. Brian undertakes a comprehensive walkthrough of the ML Ops stack, pointing out key areas such as the Data Collection, Experimentation, Evaluation, and Maintenance. He additionally highlights the right-hand side of the diagram, ascending from Infrastructure layer, Component layer, Pipeline layer, to Run layer. This episode invites listeners to join the conversation about machine learning and artificial intelligence by sharing their insights and comments through various platforms. Brian encourages feedback and insights via his discord community, email, his website, or social media. Everyone is urged to share their thoughts whether they consider themselves 'novices' or experts in the field, contributing to this fascinating exploration.   medium.com/machinevision/overview-of-mlops-a07053fc2a80 reddit.com/r/coding/comments/1bd4w76/what_are_best_practices_for_machine_learning/

We talked about: Nemanja's background When Nemanja first work as a data person Typical problems that ML Ops folks solve in the financial sector What Nemanja currently does as an ML Engineer The obstacle of implementing new things in financial sector companies Going through the hurdles of DevOps Working with an on-premises cluster “ML Ops on a Shoestring” (You don't need fancy stuff to start w/ ML Ops) Tactical solutions Platform work and code work Programming and soft skills needed to be an ML Engineer The challenges of transitioning from and electrical engineering and sales to ML Ops The ML Ops tech stack for beginners Working on projects to determine which skills you need Links: LinkedIn: https://www.linkedin.com/in/radojkovic/ Free Data Engineering course: https://github.com/DataTalksClub/data-engineering-zoomcamp Join DataTalks.Club: https://datatalks.club/slack.html Our events: https://datatalks.club/events.html

We discuss: AI Tool, HuggingChat, AI startup valuations, Open Source Machine Learning, Transformers Library, LLM Repository, Model Library, Git, Github, Model Hub, Llama 2, Launching AI models, business moat, deploying AI models, AI in the cloud, AI model leaderboards, Image classification, language models, sentiment analysis, ai music, nocode AI, low-code AI, model space, AI community, AI ecosystem, ML Ops, Emergent capabilities, generalist models, specialist models, GPT-3.5, GPT-4, ChatGPT, AI Exploration Links: - Competitors showing increased enquiries following the leadership Farce at Open AI with Sam Altman https://www.cnbc.com/2023/11/28/openai-competitors-hugging-face-and-cohere-report-increased-inquiries.html API Token issues https://www.theregister.com/2023/12/04/exposedhuggingfaceapitokens/ Interesting Hacker News post on hugging face (with ex-HF workers weighing in on strategy): https://news.ycombinator.com/item?id=37248895 Reddit post: Does Hugging Face do too many things?: https://www.reddit.com/r/MachineLearning/comments/160ts9g/d_is_it_me_or_huggingface_do_too_many_things/ transformers library: https://github.com/huggingface/transformers Model hub: https://huggingface.co/docs/hub/models-the-hub BLOOM model: https://huggingface.co/bigscience/bloom BigScience group: https://bigscience.huggingface.co/ In this episode of Using AI, we delve into the capabilities and offerings of Hugging Face, the leading AI repository often likened to GitHub for AI enthusiasts. Hosting over 250,000 datasets and 500,000 AI models, Hugging Face has revolutionised the AI world with its open-source initiatives. Watch Using AI on YouTube (and see our daft AI-generated background images): https://www.youtube.com/channel/UCHsQu4IipA7Ri2AqKcQZ1Yw --- Send in a voice message: https://podcasters.spotify.com/pod/show/using-ai/message

Dans cet épisode, Katia, Arnaud et Emmanuel discutent les nouvelles de cette fin 2023. Le gatherer dans les stream Java, les exceptions, JavaScript dans la JVM, recherche vectorielle, coût du cloud, Gemini, Llama et autres animaux fantastiques et pleins d'outils sympathiques pour fêter la fin de l'année. Enregistré le 15 décembre 2023 Téléchargement de l'épisode LesCastCodeurs-Episode-304.mp3 News Aide Les Cast Codeurs et remplis un petit formulaire pour nous guider l'année prochaine https://lescastcodeurs.com/sondage Langages Avec JEP 461, arrivée dans en preview dans Java 22 de la notion de “gatherer” pour les streams https://groovy.apache.org/blog/groovy-gatherers dans cet article de Paul King, de l'équipe Groovy, il montre et contraste ce que l'on pouvait faire en Groovy depuis des années, comme des sliding windows, par exemple explique l'approche des gatherers avec ses opérations intermédiaires gatherer sont des operations intermediaires custom qui prennent un etat et le prochain element pour decided quoi faire, et meme changer le stream d'elements suivants (en publier) (via la fonction integrate certains peuvent permettre de combiner les resultats intermediaires (pour paralleliser) Examples : fenetres de taille fixe, fenettres glissantes Joe Duffy, qui est CEO de Pulumi, mais qui avait travaillé chez Microsoft sur le project Midori (un futur OS repensé) parle du design des exceptions, des erreurs, des codes de retour https://joeduffyblog.com/2016/02/07/the-error-model/ Il compare les codes d'erreurs, les exceptions, checked et non-checked il separe les bugs des erreurs attendues (bugs doivent arreter le process) il raconte l'histoire des unchecked exception et leurs problemes et des checked exceptopns et poourquoi les developeurs java les detestent (selon lui) long article maisn interessant dans ses retours mais lon je ne suis pas allé au bout :smile: Après la disparition de Nashorn dans le JDK, on peut se tourner vers le projet Javet https://www.caoccao.com/Javet/index.html Javet permet d'intégrer JavaScript avec le moteur V8 Mais aussi carrément Node.js c'est super comme capacité car on a les deux mielleurs moteurs, par contre le support hors x86 est plus limité (genre arm sous windows c'est non) Librairies Une partie de l'équipe Spring se fait lourder après le rachat effectif de Broadcom https://x.com/odrotbohm/status/1729231722498425092?s=20 peu d'info en vrai à part ce tweet mais l'acquisition Broadcome n'a pas l'air de se faire dans le monde des bisounours Marc Wrobel annonce la sortie de JBanking 4.2.0 https://www.marcwrobel.fr/sortie-de-jbanking-4-2-0 support de Java 21 possibilité de générer aléatoirement des BIC amélioration de la génération d'IBAN jbanking est une bibliotheque pour manipuler des structures typiques des banques comme les IBAN les BIC, les monnaies, les SEPA etc. Hibernate Search 7 est sorti https://in.relation.to/2023/12/05/hibernate-search-7-0-0-Final/ Support ElasticSearch 8.10-11 et openSearch 2.10-11 Rebasé sur Lucerne 9.8 support sur Amazon OpenSearch Serverless (experimental) attention sous ensemble de fonctionnalités sur Serverless, c'est un API first search cluster vendu a la lambda En lien aussi sur la version 7.1 alpha1 Hibernate ORM 6.4 est sorti https://in.relation.to/2023/11/23/orm-640-final/ support pour SoftDelete (colonne marquant la suppression) support pour les operations vectorielles (support postgreSQL initialement) les fonctions vectorielles sont particulièrement utilisées par l'IA/ML événement spécifiques JFR Intégration de citrus et Quarkus pour les tests d'intégrations de pleins de protocoles et formats de message https://quarkus.io/blog/testing-quarkus-with-citrus/ permet de tester les entrees / sorties attendues de systèmes de messages (HTTP, Kafka, serveur mail etc) top pour tester les application Event Driven pas de rapport mais Quarkus 3.7 ciblera Java 17 (~8% des gens utilisaient Java 11 dans les builds qui ont activé les notifications) Hibernate Search 7.1 (dev 7.1.0.Alpha1) avec dernière version de Lucene (9.8), Infinispan rajoute le support pour la recherche vectorielle. https://hibernate.org/search/releases/7.1/ https://infinispan.org/blog/2023/12/13/infinispan-vector-search Hibernate Search permet maintenant la recherche vectorielle La dernière version est intégrée en Infinispan 15 (dev) qui sortira La recherche vectoriolle et stockage de vecteurs, permettent convertir Infinispan en Embedding Store (langchain) Cloud Comment choisir sa region cloud https://blog.scottlogic.com/2023/11/23/conscientious-cloud-pick-your-cloud-region-deliberately.html pas si simple le coût la securité légale de vos données la consommation carbone de la région choisie (la France est top, la Pologne moins) la latence vs où sont vos clients les services supportés Web Vers une standardisation des Webhooks ? https://www.standardwebhooks.com/ Des gens de Zapier, Twilio, Ngrok, Kong, Supabase et autres, se rejoignent pour essayer de standardiser l'approche des Webhooks La spec est open source (Apache) sur Github https://github.com/standard-webhooks/standard-webhooks/blob/main/spec/standard-webhooks.md Les objectifs sont la sécurité, la reliabilité, l'interopérabilité, la simplicité et la compatibilité (ascendante / descendante) sans la spec, chaque webhook est different dans son comportement et donc les clients doivent s'adapter dans la sematique et les erreurs etc la (meta-) structure de la payload, la taille, la securisation via signature (e.g. hmac), les erreurs (via erreurs HTTP), etc Data et Intelligence Artificielle Google annonce Gemini, son nouveau Large Language Model https://blog.google/technology/ai/google-gemini-ai/#sundar-note modèle multimodal qui peut prendre du texte, en entrée, mais aussi des images, du son, des vidéos d'après les benchmarks, il est largement aussi bon que GPT4 plusieurs tailles de modèles disponible : Nano pour être intégré aux mobiles, Pro qui va être utilisé dans la majeure partie des cas, et Ultra pour les besoins de réflexion les plus avancés Android va rajouter aussi des librairies AICore pour utiliser Gemini Nano dans les téléphones Pixel https://android-developers.googleblog.com/2023/12/a-new-foundation-for-ai-on-android.html Gemini Pro va être disponible dans Bard (en anglais et dans 170 pays, mais l'Europe va devoir attendre un petit peu pour que ce soit dispo) Gemini Ultra devrait aussi rejoindre Bard, dans une version étendue https://blog.google/products/bard/google-bard-try-gemini-ai/ Gemini va être intégré progressivement dans plein de produits Google DeepMind parlant de Gemini https://deepmind.google/technologies/gemini/#introduction Un rapport de 60 pages sur Gemini https://storage.googleapis.com/deepmind-media/gemini/gemini_1_report.pdf Gemini a permis aussi de pouvoir développer une nouvelle version du modèle AlphaCode qui excelle dans les compétitions de coding https://storage.googleapis.com/deepmind-media/AlphaCode2/AlphaCode2_Tech_Report.pdf Liste de petites vidéos sur YouTube avec des interviews et démonstrations des capacités de Gemini https://www.youtube.com/playlist?list=PL590L5WQmH8cSyqzo1PwQVUrZYgLcGZcG malheureusement certaines des annonces sont un peu fausse ce qui a amené un discrédit (non du) sur Gemini par exemple la video “aspirationelle” était vendue comme du réel mais ce n'est pas le cas. et ultra n'est pas disponible encore ausso la comparaison de ChatGPT sur la page (initialement au moins) comparait des choux et des carottes, meme si le papier de recherche était correct Avec la sortie de Gemini, Guillaume a écrit sur comment appeler Gemini en Java https://glaforge.dev/posts/2023/12/13/get-started-with-gemini-in-java/ Gemini est multimodèle, donc on peut passer aussi bien du texte que des images, ou même de la vidéo Il y a un SDK en Java pour interagir avec l'API de Gemini Facebook, Purple Llama https://ai.meta.com/blog/purple-llama-open-trust-safety-generative-ai/ Opensource https://ai.meta.com/llama/ dans l'optique des modeles GenAI ouverts, Facebook fournit des outils pour faire des IA responsables (mais pas coupables :wink: ) notament des benchmarks pour evaluler la sureté et un classifier de sureté, par exemple pour ne pas generer du code malicieux (ou le rendre plus dur) llama purple sera un projet parapluie D'ailleurs Meta IBM, Red Hat et pleins d'autres ont annoncé l'AI Alliance pour une AI ouverte et collaborative entre académique et industriels. Sont notammenrt absent Google, OpenAI (pas ouvert) et Microsoft Juste une annouce pour l'instant mais on va voir ce que ces acteurs de l'AI Alliance feront de concret il y a aussi un guide d'utilisateur l'usage IA responsable (pas lu) Apple aussi se met aux librairies de Machine Learning https://ml-explore.github.io/mlx/build/html/index.html MLX est une librairie Python qui s'inspire fortement de NumPy, PyTorch, Jax et ArrayFire Surtout, c'est développé spécifiquement pour les Macs, pour tirer au maximum parti des processeurs Apple Silicon Dans un des repos Github, on trouve également des exemples qui font tourner nativement sur macOS les modèles de Llama, de Mistral et d'auters https://github.com/ml-explore/mlx-examples non seulement les Apple Silicon amis aussi la memoire unifiee CPU/GPU qui est une des raisons clés de la rapidité des macs Faire tourner Java dans un notebook Jupyter https://www.javaadvent.com/2023/12/jupyter-notebooks-and-java.html Max Andersen explore l'utilisation de Java dans les notebooks Jupyter, au lieu du classique Python il y a des kernels java selon vos besoins mais il faut les installer dans la distro jupyter qu'on utilise et c'est la que jbang installable via pip vient a la rescousse il installe automatiquement ces kernels en quelques lignes Outillage Sfeir liste des jeux orientés développeurs https://www.sfeir.dev/tendances/notre-selection-de-jeux-de-programmation/ parfait pour Noël mais c'est pour ceux qui veulent continuer a challenger leur cerveau après le boulot jeu de logique, jeu de puzzle avec le code comme forme, jeu autour du machine learning, jeu de programmation assembleur Les calendriers de l'Avent sont populaires pour les développeurs ! En particulier avec Advent of Code https://adventofcode.com/ Mais il y a aussi l'Advent of Java https://www.javaadvent.com/ Ou un calendrier pour apprendre les bases de SVG https://svg-tutorial.com/ Le calendrier HTML “hell” https://www.htmhell.dev/adventcalendar/ qui parle d'accessibilité, de web components, de balises meta, de toutes les choses qu'on peut très bien faire en HTML/CSS sans avoir besoin de JavaScript Pour les développeurs TypeScript, il y a aussi un calendrier de l'Avent pour vous ! https://typehero.dev/aot-2023 Un super thread de Clara Dealberto sur le thème de la “dataviz” (data visualization) https://twitter.com/claradealberto/status/1729447130228457514 Beaucoup d'outil librement accessibles sont mentionnés pour faire toutes sortes de visualisations (ex. treemap, dendros, sankey…) mais aussi pour la cartographie Quelques ressources de site qui conseillent sur l'utilisation du bon type de visualisation en fonction du problème et des données que l'on a notemment celui du financial time qui tiens dans une page de PDF Bref c'est cool mais c'est long a lire Une petite liste d'outils sympas - jc pour convertir la sortie de commandes unix en JSON https://github.com/kellyjonbrazil/jc - AltTab pour macOS pour avoir le même comportement de basculement de fenêtre que sous Windows https://alt-tab-macos.netlify.app/ - gron pour rendre le JSON grep-able, en transformant chaque valeur en ligne ressemblant à du JSONPath https://github.com/tomnomnom/gron - Marker, en Python, pour transformer des PDF en beau Markdown https://github.com/VikParuchuri/marker - n8n un outil de workflow open source https://n8n.io/ gron en fait montre des lignes avec des assignments genre jsonpath = value et tu peux ungroner apres pour revenir a du json Marker utilise du machine learning mais il halklucine moins que nougat (nous voilà rassuré) Docker acquiert Testcontainers https://techcrunch.com/2023/12/11/docker-acquires-atomicjar-a-testing-startup-that-raised-25m-in-january/ Annonce par AtomicJar https://www.atomicjar.com/2023/12/atomicjar-is-now-part-of-docker/ Annonce par Docker https://www.docker.com/blog/docker-whale-comes-atomicjar-maker-of-testcontainers/ Architecture Comment implémenter la reconnaissance de chanson, comme Shazam https://www.cameronmacleod.com/blog/how-does-shazam-work il faut d'abord passer en mode fréquence avec des transformées de Fourrier pour obtenir des spectrogrammes puis créer une sorte d'empreinte qui rassemble des pics de fréquences notables à divers endroits de la chanson d'associer ces pics pour retrouver un enchainement de tels pics de fréquence dans le temps l'auteur a partagé son implémentation sur Github https://github.com/notexactlyawe/abracadabra/blob/e0eb59a944d7c9999ff8a4bc53f5cfdeb07b39aa/abracadabra/recognise.py#L80 Il y avait également une très bonne présentation sur ce thème par Moustapha Agack à DevFest Toulouse https://www.youtube.com/watch?v=2i4nstFJRXU les pics associés sont des hash qui peut etre comparés et le plus de hash veut dire que les chansons sont plus similaires Méthodologies Un mémo de chez ThoughtWorks à propos du coding assisté par IA https://martinfowler.com/articles/exploring-gen-ai.html#memo-08 Avec toute une liste de questions à se poser dans l'utilisation d'un outil tel que Copilot Il faut bien réaliser que malheureusement, une IA n'a pas raison à 100% dans ses réponses, et même plutôt que la moitié du temps, donc il faut bien mettre à jour ses attentes par rapport à cela, car ce n'est pas magique La conclusion est intéressante aussi, en suggérant que grosso modo dans 40 à 60% des situations, tu peux arriver à 40 à 80% de la solution. Est-ce que c'est à partir de ce niveau là qu'on peut vraiment gagner du temps et faire confiance à l'IA ? Ne perdez pas trop de temps non plus à essayer de convaincre l'IA de faire ce que vous voulez qu'elle fasse. Si vous n'y arrivez pas, c'est sans doute parce que l'IA n'y arrivera même pas elle même ! Donc au-delà de 10 minutes, allez lire la doc, chercher sur Google, etc. notamment, faire genrer les tests par l'IA dans al foulée augmente les risques surtout si on n'est pas capable de bien relire le code si on introduit un choix de pattern genre flexbox en CSS, si c'est sur une question de sécuriter, vérifier (ceinture et bretelle) est-ce le framework de la semaine dernière? L'info ne sera pas dans le LLM (sans RAG) Quelles capacités sont nécessaires pour déployer un projet AI/ML https://blog.scottlogic.com/2023/11/22/capabilities-to-deploy-ai-in-your-organisation.html C'est le MLOps et il y a quelques modèles end to end Google, IBM mais vu la diversité des organisations, c'est difficile a embrasser ces versions completes ML Ops est une métier, data science est un metier, donc intégrer ces competences sachez gérer votre catalogue de données Construire un process pour tester vos modèles et continuellement La notion de culture de la recherche et sa gestion (comme un portefeuille financier, accepter d'arrêter des experience etc) la culture de la recherche est peu présente en engineering qui est de construire des choses qui foncitonnent c'est un monde pre LLM Vous connaissez les 10 dark patterns de l'UX ? Pour vous inciter à cliquer ici ou là, pour vous faire rester sur le site, et plus encore https://dodonut.com/blog/10-dark-patterns-in-ux-design/ Parmi les dark patterns couverts Confirmshaming Fake Urgency and the Fear of Missing Out Nagging Sneaking Disguised Ads Intentional Misdirection The Roach Motel Pattern Preselection Friend Spam Negative Option Billing or Forced Continuity L'article conclut avec quelques pistes sur comment éviter ces dark patterns en regardant les bons patterns de la concurrence, en testant les interactions UX, et en applicant beaucoup de bon sens ! les dark patterns ne sont pas des accidents, ils s'appuient sur la psychologie et sont mis en place specifiquement Comment choisir de belles couleurs pour la visualisation de données ? https://blog.datawrapper.de/beautifulcolors/ Plutôt que de penser en RGB, il vaut mieux se positionner dans le mode Hue Saturation Brightness Plein d'exemples montrant comment améliorer certains choix de couleurs Mieux vaut éviter des couleurs trop pures ou des couleurs trop brillantes et saturées Avoir un bon contraste Penser aussi aux daltoniens ! j'ai personnellement eu toujours du mal avec saturationm vs brightness faire que les cloueirs en noir et blanc soient separees evant de le remettre (en changeant la brightness de chaque couleur) ca aide les daltoniens eviter les couleurs aux 4 coins amis plutot des couleurs complementaires (proches) rouge orange et jaune (non saturé) et variations de bleu sont pas mal les couleurs saturées sont aggressives et stressent les gens Pourquoi vous devriez devenir Engineering Manager? https://charity.wtf/2023/12/15/why-should-you-or-anyone-become-an-engineering-manager/ L'article parle de l'évolution de la perception de l'engineering management qui n'est plus désormais le choix de carrière par défaut pour les ingénieurs ambitieux. Il met en évidence les défis auxquels les engineering managers sont confrontés, y compris les attentes croissantes en matière d'empathie, de soutien et de compétences techniques, ainsi que l'impact de la pandémie de COVID-19 sur l'attrait des postes de management. L'importance des bons engineering mnanagers est soulignée, car ils sont considérés comme des multiplicateurs de force pour les équipes, contribuant de manière significative à la productivité, à la qualité et au succès global dans les environnements organisationnels complexes. L'article fournit des raisons pour lesquelles quelqu'un pourrait envisager de devenir Engineering Manager, y compris acquérir une meilleure compréhension de la façon dont les entreprises fonctionnent, contribuer au mentorat et influencer les changements positifs dans la dynamique des équipes et les pratiques de l'industrie. Une perspective est présentée, suggérant que devenir Engineering manager peut conduire à la croissance personnelle et à l'amélioration des compétences de vie, telles que l'autorégulation, la conscience de soi, la compréhension des autres, l'établissement de limites, la sensibilité à la dynamique du pouvoir et la maîtrise des conversations difficiles. L'article encourage à considérer la gestion comme une occasion de développer et de porter ces compétences pour la vie. Sécurité LogoFAIL une faille du bootloader de beaucoup de machines https://arstechnica.com/security/2023/12/just-about-every-windows-and-linux-device-vulnerable-to-new-logofail-firmware-attack/ en gros en changeant les eimages qu'on voit au boot permet d'executer du code arbitraire au tout debuit de la securisation du UEFI (le boot le plus utilisé) donc c'est game over parce que ca demarre avant l'OS c'est pas une exploitation a distance, il faut etre sur la machine avec des droits assez elevés deja mais ca peut etre la fin de la chaine d'attaque et comme d'hab un interpreteur d'image est la cause de ces vulnerabilités Conférences L'IA au secours de conférences tech: rajoute des profile tech femme comme speaker au programme pour passer le test diversité online via des profiles fake. https://twitter.com/GergelyOrosz/status/1728177708608450705 https://www.theregister.com/2023/11/28/devternity_conference_fake_speakers/ https://www.developpez.com/actu/351260/La-conference-DevTernity-sur-la-technologie-s-e[…]s-avoir-cree-de-fausses-oratrices-generees-automatiquement/ j'avais lu le tweet du createur de cette conf qui expliquait que c'etait des comptes de tests et que pris dans le rush ils avaient oublié de les enlever mais en fait les comptes de tests ont des profils “Actifs” sur le reseaux sociaux apparemment donc c'était savamment orchestré Au final beaucoup de speakers et des sponsors se desengagent La liste des conférences provenant de Developers Conferences Agenda/List par Aurélie Vache et contributeurs : 31 janvier 2024-3 février 2024 : SnowCamp - Grenoble (France) 1 février 2024 : AgiLeMans - Le Mans (France) 6 février 2024 : DevFest Paris - Paris (France) 8-9 février 2024 : Touraine Tech - Tours (France) 15-16 février 2024 : Scala.IO - Nantes (France) 6-7 mars 2024 : FlowCon 2024 - Paris (France) 14-15 mars 2024 : pgDayParis - Paris (France) 19 mars 2024 : AppDeveloperCon - Paris (France) 19 mars 2024 : ArgoCon - Paris (France) 19 mars 2024 : BackstageCon - Paris (France) 19 mars 2024 : Cilium + eBPF Day - Paris (France) 19 mars 2024 : Cloud Native AI Day Europe - Paris (France) 19 mars 2024 : Cloud Native Wasm Day Europe - Paris (France) 19 mars 2024 : Data on Kubernetes Day - Paris (France) 19 mars 2024 : Istio Day Europe - Paris (France) 19 mars 2024 : Kubeflow Summit Europe - Paris (France) 19 mars 2024 : Kubernetes on Edge Day Europe - Paris (France) 19 mars 2024 : Multi-Tenancy Con - Paris (France) 19 mars 2024 : Observabiity Day Europe - Paris (France) 19 mars 2024 : OpenTofu Day Europe - Paris (France) 19 mars 2024 : Platform Engineering Day - Paris (France) 19 mars 2024 : ThanosCon Europe - Paris (France) 19-21 mars 2024 : IT & Cybersecurity Meetings - Paris (France) 19-22 mars 2024 : KubeCon + CloudNativeCon Europe 2024 - Paris (France) 26-28 mars 2024 : Forum INCYBER Europe - Lille (France) 28-29 mars 2024 : SymfonyLive Paris 2024 - Paris (France) 4-6 avril 2024 : Toulouse Hacking Convention - Toulouse (France) 17-19 avril 2024 : Devoxx France - Paris (France) 18-20 avril 2024 : Devoxx Greece - Athens (Greece) 25-26 avril 2024 : MiXiT - Lyon (France) 25-26 avril 2024 : Android Makers - Paris (France) 8-10 mai 2024 : Devoxx UK - London (UK) 16-17 mai 2024 : Newcrafts Paris - Paris (France) 24 mai 2024 : AFUP Day Nancy - Nancy (France) 24 mai 2024 : AFUP Day Poitiers - Poitiers (France) 24 mai 2024 : AFUP Day Lille - Lille (France) 24 mai 2024 : AFUP Day Lyon - Lyon (France) 2 juin 2024 : PolyCloud - Montpellier (France) 6-7 juin 2024 : DevFest Lille - Lille (France) 6-7 juin 2024 : Alpes Craft - Grenoble (France) 27-28 juin 2024 : Agi Lille - Lille (France) 4-5 juillet 2024 : Sunny Tech - Montpellier (France) 19-20 septembre 2024 : API Platform Conference - Lille (France) & Online 7-11 octobre 2024 : Devoxx Belgium - Antwerp (Belgium) 10-11 octobre 2024 : Volcamp - Clermont-Ferrand (France) 10-11 octobre 2024 : Forum PHP - Marne-la-Vallée (France) 17-18 octobre 2024 : DevFest Nantes - Nantes (France) Nous contacter Pour réagir à cet épisode, venez discuter sur le groupe Google https://groups.google.com/group/lescastcodeurs Contactez-nous via twitter https://twitter.com/lescastcodeurs Faire un crowdcast ou une crowdquestion Soutenez Les Cast Codeurs sur Patreon https://www.patreon.com/LesCastCodeurs Tous les épisodes et toutes les infos sur https://lescastcodeurs.com/

Is your business missing out on the AI revolution?Artificial intelligence has exploded onto the business scene, leaving many leaders wondering: How can we leverage AI to boost efficiency, revenue, and results at our company?

MLOps podcast #192 with Chris Van Pelt, CISO and co-founder of Weights & Biases, Enterprises Using MLOps, the Changing LLM Landscape, MLOps Pipelines sponsored by  @WeightsBiases . // Abstract Chris, provides insights into his machine learning (ML) journey, emphasizing the significance of ML evaluation processes and the evolving landscape of MLOps. The conversation covers effective evaluation metrics, demo-driven development nuances, and the complexities of ML Ops pipelines. Chris reflects on his experience with Crowdflower, detailing its transition to Weights and Biases and stressing the early integration of security measures. The discussion extends to the transformative impact of ML on the tech industry, challenges in detecting subtle bugs, and the potential of open-source models and multimodal capabilities. // Bio Chris Van Pelt is a co-founder of Weights & Biases, a developer MLOps platform. In 2009, Chris founded Figure Eight/CrowdFlower. Over the past 12 years, Chris has dedicated his career optimizing ML workflows and teaching ML practitioners, making machine learning more accessible to all. Chris has worked as a studio artist, computer scientist, and web engineer. He studied both art and computer science at Hope College. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links Website: https://wandb.ai/site ⁠ --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Chris on LinkedIn: https://www.linkedin.com/in/chrisvanpelt/ Timestamps: [00:00] Chris' preferred coffee [00:33] Takeaways[03:50] Huge shout out to Weights & Biases for sponsoring this episode! [04:15] Please like, share, and subscribe to our MLOps channels! [04:25] CrowdFlower [07:02] Difference of CrowdFlower and Trajectory [09:13] Transition from CrowdFlower to Weights & Biases [13:05] Excel spreadsheets being passed around via email [15:45] Evolution of Weights & Biases [19:24] CISO role [22:23] Advise for easy wins [25:32] Transition into LLMs [27:36] Prompt injection risks on data [29:42] LLMs for New Personas [34:42] Iterative Value Evaluation Process [36:36] Iterating on New Release [39:31] Evaluation survey [43:21] Landscape of LLMs and its evolution [45:40] Conan O'Brien [46:48] Wrap up

Welcome to another captivating episode of the Women in Data® Podcast! Today Karen is joined by two exceptional experts in the field: Michelle Conway, Lead Data Scientist at Lloyds Banking Group, and Sarah Schlobohm, Head of AI and Particle Physicist. Together they discuss the world of Machine Learning Ops.  A central theme of the discussion revolves around the lifecycle of MLOps and the concept of Machine Learning as a "solution to a problem”. Michelle, Sarah and Karen delve into this idea, stressing the importance of identifying the right opportunities for Machine Learning to truly be effective. They discuss how businesses can avoid falling into the trap of implementing ML without a clear understanding of appropriate applications.  They also highlight how Machine Learning doesn't operate in isolation but requires seamless integration with other data departments, such as data governance. This comprehensive approach ensures that ML initiatives are well-aligned with the overall data strategy of the organisation, leading to more successful outcomes.  This episode is a must-listen for anybody looking to learn more about ML Ops and the potential applications of this exciting field. 

Introduktion till MLOps - Machine Learning Operations. Hur du produktionssätter maskininlärningsmodeller inom AI. Kim Berg, lösningsarkitekt på Sogeti, berättar mer om hur man kan göra maskininlärning till en operationell del av verksamheten istället för en isolerad experimentell process. Om avsnittet: Målgrupp: Företagsledare, Dataanalytiker, IT-chefer, AI-entusiaster Lär dig: MLOps, AI-implementering, maskininlärning, DevOps, företagsinnovation, framtidens teknologi, AI-strategi, datahantering, kontinuerlig förbättring Utmaningar och Fördelar med ML Ops Ett av de stora hindren för framgångsrik AI-implementering är bristen på en effektiv process för att hantera och förvalta de många modeller som utvecklas. Kim förklarar hur MLOps hanterar detta genom att integrera maskininlärning med DevOps-metodiken. Detta resulterar i bättre processkontroll, bättre modellprestanda och kontinuerlig förbättring över tiden. Framtiden för företag med MLOps Få inblickar om hur MLOps kommer att forma framtidens företag. Med en ökande mängd data och en växande medvetenhet om AI: s potential blir det allt viktigare för företag att implementera AI på ett strukturerat sätt. ML Ops ger företag möjlighet att skala upp och hantera sina AI-initiativ på ett sätt som tidigare inte var möjligt. Detta är en nyckelkomponent för att frigöra AI: s fulla potential i företag. För att lyckas är det viktigt att bygga starka team med olika kompetenser, samarbeta effektivt och fokusera på att produktionssätta modeller istället för att fastna i experimentella faser. Med MLOps kan företag skapa en solid grund för framtidens AI-drivna innovationer och förbättringar. Jonas Jaani, Kim Berg (24:39) Videoversion av avsnittet: https://youtu.be/FGa8Xxtl1mw https://youtu.be/FGa8Xxtl1mw Länkar / mer information: Om Kim Berg:Kim Berg är en passionerad lösingsarkitekt med fokus på Azure. Med över ett decennium i IT-branschen har han utvecklat en djup kompetens och erfarenhet. Under de senaste sex åren har han specialiserat sig inom Microsoft Azure Kim brinner för att skapa innovativa och nyskapande lösningar. Hans engagemang sträcker sig bortom tekniken, då han delar sin värdefulla kunskap genom mentorskap och inspirerande föreläsningar. AI, Machine Learning, Kubernetes och IoT-lösningar. Kim jobbar integrerad Sogetis nationella Center of Excellence (CoE), där han konstant strävar efter att lyfta fram de senaste innovationerna och bästa praxis inom teknologi. Ett särskilt ansvarsområde för Kim är att leda lösningsacceleratorn SARAH (Scalable Adaptive Robust AI Hub). Översiktsbilder på MLOps Alla avsnitt av digitaliseringens podcast Effekten Prenumerera: Apple Podcasts Google Podcasts Spotify: https://open.spotify.com/show/5Z49zvPOisoSwhwojtUoCm Vill du att vi tar upp ett specifikt ämne eller intervjuar en person inom digitalisering? Maila oss på info(a)effekten(punkt)se

In this episode of the AI Today podcast hosts Kathleen Walch and Ron Schmelzer define the terms DevOps, Machine Learning Operations (ML Ops), explain how these terms relate to AI and why it's important to know about them. Show Notes: FREE Intro to CPMAI mini course CPMAI Training and Certification AI Glossary AI Glossary Series – Data Drift, Model Drift, Model Retraining Glossary Series: (Artificial) Neural Networks, Node (Neuron), Layer Continue reading AI Today Podcast: AI Glossary Series – DevOps, Machine Learning Operations (ML Ops) at AI & Data Today.

Larysa placed the start of her journey in the country she was born in, the Soviet Union, with her discovery of those books filled with 0s and 1s that she could not understand. Fast forward a few years, and she is hooked by Maths and Computer Science and doesn't want to do anything else. After her Master's degree in Odessa, she moved to Germany and studied again. From then on, we talked about how she pursued a Ph.D., pivoted on the problem she was tackling, and discovered a new field ML-Ops. We discussed the state of ML-Ops, how it is placed in the "Data" and ML problem chain, and how one should go at it.Here are the links from the showhttps://www.twitter.com/visengerhttps://ml-ops.org/https://ml-ops.org/content/phase-zerohttps://ml-ops.org/content/end-to-end-ml-workflowhttps://github.com/visenger/awesome-mlopshttps://women-in-data-ai.tech/https://mlops.community/https://www.linkedin.com/in/larysavisenger/https://www.goodreads.com/book/show/530415.The_Art_of_Doing_Science_and_EngineeringCreditsCover Legends by HoliznaCC0 is licensed CC0 1.0 Universal License.Your host is Timothée (Tim) Bourguignon; more about him at timbourguignon.fr.Gift the podcast a rating on one of the significant platforms https://devjourney.info/subscribePracticing Connection: Working together to help families and communities thrive.Jessica Beckendorf and Bob Bertsch host this exploration of personal and collective...Listen on: Apple Podcasts SpotifySupport the show1. Your host is Timothée (Tim) Bourguignon; more about him at timbourguignon.fr.2. Gift the podcast a rating on the platform of your choice.3. Become a supporter of the show on Patreon or on Buzzsprout (our hoster).

In this bonus episode, Eric and Kostas preview their upcoming conversation with Simba Khadder of Featureform.

Weights & Biases is a developer-focused MLOps platform last valued at over $1 billion. Their platform helps developers streamline their ML workflow from end to end. Weights & Biases currently has over 700 customers using their product to manage their ML models.  In this episode, Sandhya Hegde chats with Lukas Biewald, the CEO and co-founder of Weights & Biases about the company's path to product-market fit. Join us as we discuss: (2:09) Lukas's inspiration for starting the company (8:02) Early design partners for Weights & Biases (13:32) Early use cases for the Weights & Biases platform (16:13) The founders' early strategies for finding users and getting feedback (22:47) Finding product-market fit by focusing on a specific user persona (ML practitioner) (29:14) Growth tactics to increase awareness and expand their user base (34:13) Positive surprises in user feedback, and how it informed the company's product roadmap (40:14) Lukas's approach to leadership and building a strong, connected culture (43:16) Advice for aspiring founders building AI-native products Sandhya Hegde is a General Partner at Unusual Ventures, leading investments in modern SaaS companies with a focus on AI. Previously an early executive at Amplitude, Sandhya is a product-led growth (PLG) coach and mentor. She can be reached at sandhya@unusual.vc and  Twitter - https://twitter.com/sandhya LinkedIn - https://www.linkedin.com/in/sandhyahegde/ Lukas Biewald is the CEO and the co-founder of Weights & Biases. Previously, Biewald was the founder and CEO of Figure Eight Inc. (formerly CrowdFlower). Unusual Ventures is a seed-stage venture capital firm designed from the ground up to give a distinct advantage to founders building the next generation of software companies. Unusual has invested in category-defining companies like Webflow, Arctic Wolf Networks, Carta, Robinhood, and Harness. Learn more about us at https://www.unusual.vc/. Further reading from the Startup Field Guide: Define your Ideal Customer Profile: https://www.field-guide.unusual.vc/field-guide-enterprise/ideal-customer-profile-and-personas Introduction to product-led growth:https://www.field-guide.unusual.vc/field-guide-enterprise/introduction-to-plg Introduction to design partners:https://www.field-guide.unusual.vc/field-guide-enterprise/what-are-design-partners

Episode 26 FinOps X Session Preview - Thiago Gil - Financial Challenges of ML Ops Thiago Gil previews his session discussing the financial challenges and ethical implications inherent in ML Ops/AI.Register for FinOps X now: x.finops.orgCheck out the other breakout sessions at X: https://x.finops.org/agenda/Thiago Bittencourt Gil | LinkedInFinOps X - June 27-30, 2023 San Diego, California

Dr Lou Kratz, Principle Research Engineer of Bazaarvoice, shares how his organization generates value from data, cutting time to value by 6x and leveraging AWS to save big on cloud costs.Topics include:Making smart decisions automatically from data with machine learningBazaarvoice business modelUnderstanding value is the most important part of any machine learning projectLeveraging AI/ML for content moderationLeveraging AWS SagemakerBeing cost effective with serverless optionsCutting time to value by 6x with savings of 82%Investing in ML OpsFinal recommendations for data and ML Ops

It's good to recognize what AI is and what it is not. In this episode, George Brooks is joined by Michelle Frost to de-bunk myths & misconceptions about what AI can mean for processes in the workplace. Learn about the necessary steps to train a model, from collecting and preparing data to testing and evaluating the model. Plus, the importance of ML Ops and the impact of using third-party libraries and packages.

Today we're joined by Vinesh Sukumar, a senior director and head of AI/ML product management at Qualcomm Technologies. In our conversation with Vinesh, we explore how mobile and automotive devices have different requirements for AI models and how their AI stack helps developers create complex models on both platforms. We also discuss the growing interest in text-based input and the shift towards transformers, generative content, and recommendation engines. Additionally, we explore the challenges and opportunities for ML Ops investments on the edge, including the use of synthetic data and evolving models based on user data. Finally, we delve into the latest advancements in large language models, including Prometheus-style models and GPT-4. The complete show notes for this episode can be found at twimlai.com/go/623.

We're so glad to launch our first podcast episode with Logan Kilpatrick! This also happens to be his first public interview since joining OpenAI as their first Developer Advocate. Thanks Logan!Recorded in-person at the beautiful StudioPod studios in San Francisco. Full transcript is below the fold.Timestamps* 00:29: Logan's path to OpenAI* 07:06: On ChatGPT and GPT3 API* 16:16: On Prompt Engineering* 20:30: Usecases and LLM-Native Products* 25:38: Risks and benefits of building on OpenAI* 35:22: OpenAI Codex* 42:40: Apple's Neural Engine* 44:21: Lightning RoundShow notes* Sam Altman's interview with Connie Loizos* OpenAI Cookbook* OpenAI's new Embedding Model* Cohere on Word and Sentence Embeddings* (referenced) What is AGI-hard?Lightning Rounds* Favorite AI Product: https://www.synthesia.io/* Favorite AI Community: MLOps * One year prediction: Personalized AI, https://civitai.com/* Takeaway: AI Revolution is here!Transcript[00:00:00] Alessio Fanelli: Hey everyone. Welcome to the Latent Space podcast. This is Alessio, partner and CTO in residence at Decibel Partners. I'm joined by my cohost, swyx writer editor of L Space Diaries. Hey.[00:00:20] swyx: Hey . Our guest today is Logan Kilpatrick. What I'm gonna try to do is I'm gonna try to introduce you based on what people know about you, and then you can fill in the blanks.[00:00:28] Introducing Logan[00:00:28] swyx: So you are the first. Developer advocate at OpenAI, which is a humongous achievement. Congrats. You're also the lead developer community advocate of the Julia language. I'm interested in a little bit of that and apparently as I've did a bit of research on you, you got into Julia through NASA where you interned and worked on stuff that's gonna land on the moon apparently.[00:00:50] And you are also working on computer vision at Apple. And had to sit at path, the eye as you fell down the machine learning rabbit hole. What should people know about you that's kind of not on your LinkedIn that like sort of ties together your interest[00:01:02] Logan Kilpatrick: in story? It's a good question. I think so one of the things that is on my LinkedIn that wasn't mentioned that's super near and dear to my heart and what I spend a lot of time in sort of wraps a lot of my open source machine learning developer advocacy experience together is supporting NumFOCUS.[00:01:17] And NumFOCUS is the nonprofit that helps enable a bunch of the open source scientific projects like Julia, Jupyter, Pandas, NumPy, all of those open source projects are. Facilitated legal and fiscally through NumFOCUS. So it's a very critical, important part of the ecosystem and something that I, I spend a bunch of my now more limited free time helping support.[00:01:37] So yeah, something that's, It's on my LinkedIn, but it's, it's something that's important to me. Well,[00:01:42] swyx: it's not as well known of a name, so maybe people kind of skip over it cuz they were like, I don't know what[00:01:45] Logan Kilpatrick: to do with this. Yeah. It's super interesting to see that too. Just one point of context for that is we tried at one point to get a Wikipedia page for non focus and it's, it's providing, again, the infrastructure for, it's like a hundred plus open source scientific projects and they're like, it's not notable enough.[00:01:59] I'm like, well, you know, there's something like 30 plus million developers around the world who use all these open source tools. It's like the foundation. All open source like science that happens. Every breakthrough in science is they discovered the black hole, the first picture of the black hole, all that stuff using numb focus tools, the Mars Rovers, NumFOCUS tools, and it's interesting to see like the disconnect between the nonprofit that supports those projects and the actual success of the projects themselves.[00:02:26] swyx: Well, we'll, we'll get a bunch of people focused on NumFOCUS and we'll get it on Wikipedia. That that is our goal. . That is the goal. , that is our shot. Is this something that you do often, which is you? You seem to always do a lot of community stuff. When you get into something, you're also, I don't know where this, where you find time for this.[00:02:42] You're also a conference chair for DjangoCon, which was last year as well. Do you fall down the rabbit hole of a language and then you look for community opportunities? Is that how you get into.[00:02:51] Logan Kilpatrick: Yeah, so the context for Django stuff was I'd actually been teaching and still am through Harvard's division of continuing education as a teaching fellow for a Django class, and had spent like two and a half years actually teaching students every semester, had a program in Django and realized that like it was kind of the one ecosystem or technical tool that I was using regularly that I wasn't actually contributing to that community.[00:03:13] So, I think sometime in 2021 like applied to be on the board of directors of the Django Events Foundation, north America, who helps run DjangoCon and was fortunate enough to join a support to be the chair of DjangoCon us and then just actually rolled off the board because of all the, all the craziness and have a lot less free time now.[00:03:32] And actually at PATH ai. Sort of core product was also using, was using Django, so it also had a lot of connections to work, so it was a little bit easier to justify that time versus now open ai. We're not doing any Django stuff unfortunately, so, or[00:03:44] swyx: Julia, I mean, should we talk about this? Like, are you defecting from Julia?[00:03:48] What's going on? ,[00:03:50] Logan Kilpatrick: it's actually felt a little bit strange recently because I, for the longest time, and, and happy to talk about this in the context of Apple as well, the Julie ecosystem was my outlet to do a lot of the developer advocacy, developer relations community work that I wanted to do. because again, at Apple I was just like training machine learning models.[00:04:07] Before that, doing software engineering at Apple, and even at Path ai, we didn't really have a developer product, so it wasn't, I was doing like advocacy work, but it wasn't like developer relations in the traditional sense. So now that I'm so deeply doing developer relations work at Open OpenAI, it's really difficult to.[00:04:26] Continue to have the energy after I just spent nine hours doing developer relations stuff to like go and after work do a bunch more developer relations stuff. So I'll be interested to see for myself like how I'm able to continue to do that work and I. The challenge is that it's, it's such critical, important work to happen.[00:04:43] Like I think the Julie ecosystem is so important. I think the language is super important. It's gonna continue to grow in, in popularity, and it's helping scientists and engineers solve problems they wouldn't otherwise be able to. So it's, yeah, the burden is on me to continue to do that work, even though I don't have a lot of time now.[00:04:58] And I[00:04:58] Alessio Fanelli: think when it comes to communities, the machine learning technical community, I think in the last six to nine months has exploded. You know, you're the first developer advocate at open ai, so I don't think anybody has a frame of reference on what that means. What is that? ? So , what do you, how did, how the[00:05:13] swyx: job, yeah.[00:05:13] How do you define the job? Yeah, let's talk about that. Your role.[00:05:16] Logan Kilpatrick: Yeah, it's a good question and I think there's a lot of those questions that actually still exist at OpenAI today. Like I think a lot of traditional developed by advocacy, at least like what you see on Twitter, which I think is what a lot of people's perception of developer advocacy and developer relations is, is like, Just putting out external content, going to events, speaking at conferences.[00:05:35] And I think OpenAI is very unique in the sense that, at least at the present moment, we have so much inbound interest that there's, there is no desire for us to like do that type of developer advocacy work. So it's like more from a developer experience point of view actually. Like how can we enable developers to be successful?[00:05:53] And that at the present moment is like building a strong foundation of documentation and things like that. And we had a bunch of amazing folks internally who were. Who were doing some of this work, but it really wasn't their full-time job. Like they were focused on other things and just helping out here and there.[00:06:05] And for me, my full-time job right now is how can we improve the documentation so that people can build the next generation of, of products and services on top of our api. And it's. Yeah. There's so much work that has to happen, but it's, it's, it's been a ton of fun so far. I find[00:06:20] swyx: being in developer relations myself, like, it's kind of like a fill in the blanks type of thing.[00:06:24] Like you go to where you, you're needed the most open. AI has no problem getting attention. It is more that people are not familiar with the APIs and, and the best practices around programming for large language models, which is a thing that did not exist three years ago, two years ago, maybe one year ago.[00:06:40] I don't know. When she launched your api, I think you launched Dall-E. As an API or I, I don't[00:06:45] Logan Kilpatrick: know. I dunno. The history, I think Dall-E was, was second. I think it was some of the, like GPT3 launched and then GPT3 launched and the API I think like two years ago or something like that. And then Dali was, I think a little more than a year ago.[00:06:58] And then now all the, the Chachi Beast ChatGPT stuff has, has blown it all outta the water. Which you have[00:07:04] swyx: a a wait list for. Should we get into that?[00:07:06] Logan Kilpatrick: Yeah. .[00:07:07] ChatGPT[00:07:07] Alessio Fanelli: Yeah. We would love to hear more about that. We were looking at some of the numbers you went. Zero to like a million users in five days and everybody, I, I think there's like dozens of ChatGPT API wrappers on GitHub that are unofficial and clearly people want the product.[00:07:21] Like how do you think about that and how developers can interact with it.[00:07:24] Logan Kilpatrick: It. It's absolutely, I think one of the most exciting things that I can possibly imagine to think about, like how much excitement there was around ChatGPT and now getting to hopefully at some point soon, put that in the hands of developers and see what they're able to unlock.[00:07:38] Like I, I think ChatGPT has been a tremendous success, hands down without a question, but I'm actually more excited to see what developers do with the API and like being able to build those chat first experiences. And it's really fascinating to see. Five years ago or 10 years ago, there was like, you know, all this like chatbot sort of mm-hmm.[00:07:57] explosion. And then that all basically went away recently, and the hype went to other places. And I think now we're going to be closer to that sort of chat layer and all these different AI chat products and services. And it'll be super interesting to see if that sticks or not. I, I'm not. , like I think people have a lot of excitement for ChatGPT right now, but it's not clear to me that that that's like the, the UI or the ux, even though people really like it in the moment, whether that will stand the test of time, I, I just don't know.[00:08:23] And I think we'll have to do a podcast in five years. Right. And check in and see whether or not people are still really enjoying that sort of conversational experience. I think it does make sense though cause like that's how we all interact and it's kind of weird that you wouldn't do that with AI products.[00:08:37] So we. and I think like[00:08:40] Alessio Fanelli: the conversational interface has made a lot of people, first, the AI to hallucinate, you know, kind of come up with things that are not true and really find all the edge cases. I think we're on the optimism camp, you know, like we see the potential. I think a lot of people like to be negative.[00:08:56] In your role, kind of, how do you think about evangelizing that and kind of the patience that sometimes it takes for these models to become.[00:09:03] Logan Kilpatrick: Yeah, I think what, what I've done is just continue to scream from the, the mountains that like ChatGPT has, current form is definitely a research preview. The model that underlies ChatGPT GPT 3.5 is not a research preview.[00:09:15] I think there's things that folks can do to definitely reduce the amount of hall hallucinations and hopefully that's something that over time I, I, again have full confidence that it'll, it'll solve. Yeah, there's a bunch of like interesting engineering challenges. you have to solve in order to like really fix that problem.[00:09:33] And I think again, people are, are very fixated on the fact that like in, you know, a few percentage points of the conversations, things don't sound really good. Mm-hmm. , I'm really more excited to see, like, again when the APIs and the Han developers like what are the interesting solutions that people come up with, I think there's a lot that can be explored and obviously, OpenAI can explore all them because we have this like one product that's using the api.[00:09:56] And once you get 10,000, a hundred thousand developers building on top of that, like, we'll see what are the different ways that people handle this. And I imagine there's a lot of low-hanging fruit solutions that'll significantly improve the, the amount of halluc hallucinations that are showing up. Talk about[00:10:11] swyx: building on top of your APIs.[00:10:13] Chat GPTs API is not out yet, but let's assume it is. Should I be, let's say I'm, I'm building. A choice between GP 3.5 and chat GPT APIs. As far as I understand, they are kind of comparable. What should people know about deciding between either of them? Like it's not clear to me what the difference is.[00:10:33] Logan Kilpatrick: It's a great question.[00:10:35] I don't know if there's any, if we've made any like public statements about like what the difference will be. I think, I think the point is that the interface for the Chachi B API will be like conversational first, and that's not the case now. If you look at text da Vinci oh oh three, like you, you just put in any sort of prompt.[00:10:52] It's not really built from the ground up to like keep the context of a conversation and things like that. And so it's really. Put in some sort of prompt, get a response. It's not always designed to be in that sort of conversational manner, so it's not tuned in that way. I think that's the biggest difference.[00:11:05] I think, again, the point that Sam made in a, a strictly the strictly VC talk mm-hmm. , which was incredible and I, I think that that talk got me excited and my, which, which part? The whole thing. And I think, I haven't been at open AI that long, so like I didn't have like a s I obviously knew who Sam was and had seen a bunch of stuff, but like obviously before, a lot of the present craziness with Elon Musk, like I used to think Elon Musk seemed like a really great guy and he was solving all these really important problems before all the stuff that happened.[00:11:33] That's a hot topic. Yeah. The stuff that happened now, yeah, now it's much more questionable and I regret having a Tesla, but I, I think Sam is actually. Similar in the sense that like he's solving and thinking about a lot of the same problems that, that Elon, that Elon is still today. But my take is that he seems like a much more aligned version of Elon.[00:11:52] Like he's, he's truly like, I, I really think he cares deeply about people and I think he cares about like solving the problems that people have and wants to enable people. And you can see this in the way that he's talked about how we deploy models at OpenAI. And I think you almost see Tesla in like the completely opposite end of the spectrum, where they're like, whoa, we.[00:12:11] Put these 5,000 pound machines out there. Yeah. And maybe they'll run somebody over, maybe they won't. But like it's all in the interest of like advancement and innovation. I think that's really on the opposite end of the spectrum of, of what open AI is doing, I think under Sam's leadership. So it's, it's interesting to see that, and I think Sam said[00:12:30] Alessio Fanelli: that people could have built Chen g p t with what you offered like six, nine months ago.[00:12:35] I[00:12:35] swyx: don't understand. Can we talk about this? Do you know what, you know what we're talking about, right? I do know what you're talking about. da Vinci oh three was not in the a p six months before ChatGPT. What was he talking about? Yeah.[00:12:45] Logan Kilpatrick: I think it's a little bit of a stretch, but I do think that it's, I, I think the underlying principle is that.[00:12:52] The way that it, it comes back to prompt engineering. The way that you could have engineered, like the, the prompts that you were put again to oh oh three or oh oh two. You would be able to basically get that sort of conversational interface and you can do that now. And, and I, you know, I've seen tutorials.[00:13:05] We have tutorials out. Yep. No, we, I mean, we, nineties, we have tutorials in the cookbook right now in on GitHub. We're like, you can do this same sort of thing. And you just, it's, it's all about how you, how you ask for responses and the way you format data and things like that. It. The, the models are currently only limited by what people are willing to ask them to do.[00:13:24] Like I really do think that, yeah, that you can do a lot of these things and you don't need the chat CBT API to, to build that conversational layer. That is actually where I[00:13:33] swyx: feel a little bit dumb because I feel like I don't, I'm not smart enough to think of new things to ask the models. I have to see an example and go, oh, you can do that.[00:13:43] All right, I'm gonna do that for now. You know, and, and that's why I think the, the cookbook is so important cuz it's kind of like a compendium of things we know about the model that you can ask it to do. I totally[00:13:52] Logan Kilpatrick: agree and I think huge shout out to the, the two folks who I work super closely with now on the cookbook, Ted and Boris, who have done a lot of that work and, and putting that out there and it's, yeah, you see number one trending repo on, on GitHub and it was super, like when my first couple of weeks at Open ai, super unknown, like really, we were only sort of directing our customers to that repo.[00:14:13] Not because we were trying to hide it or anything, but just because. It was just the way that we were doing things and then all of a sudden it got picked up on GitHub trending and a bunch of tweets went viral, showing the repo. So now I think people are actually being able to leverage the tools that are in there.[00:14:26] And, and Ted's written a bunch of amazing tutorials, Boris, as well. So I think it's awesome that more people are seeing those. And from my perspective, it's how can we take those, make them more accessible, give them more visibility, put them into the documentation, and I don't think that that connection right now doesn't exist, which I'm, I'm hopeful we'll be able to bridge those two things.[00:14:44] swyx: Cookbook is kind of a different set of documentation than API docs, and I think there's, you know, sort of existing literature about how you document these things and guide developers the right way. What, what I, what I really like about the cookbook is that it actually cites academic research. So it's like a nice way to not read the paper, but just read the conclusions of the paper ,[00:15:03] Logan Kilpatrick: and, and I think that's, that's a shout out to Ted and Boris cuz I, I think they're, they're really smart in that way and they've done a great job of finding the balance and understanding like who's actually using these different tools.[00:15:13] So, . Yeah.[00:15:15] swyx: You give other people credit, but you should take credit for yourself. So I read your last week you launched some kind of documentation about rate limiting. Yeah. And one of my favorite things about reading that doc was seeing examples of, you know, you were, you're telling people to do exponential back off and, and retry, but you gave code examples with three popular libraries.[00:15:32] You didn't have to do that. You could have just told people, just figure it out. Right. But you like, I assume that was you. It wasn't.[00:15:38] Logan Kilpatrick: So I think that's the, that's, I mean, I'm, I'm helping sort of. I think there's a lot of great stuff that people have done in open ai, but it was, we have the challenge of like, how can we make that accessible, get it into the documentation and still have that high bar for what goes into the doc.[00:15:51] So my role as of recently has been like helping support the team, building that documentation first culture, and supporting like the other folks who actually are, who wrote that information. The information was actually already in. Help center but it out. Yeah, it wasn't in the docs and like wasn't really focused on, on developers in that sense.[00:16:10] So yeah. I can't take the, the credit for the rate limit stuff either. , no, this[00:16:13] swyx: is all, it's part of the A team, that team effort[00:16:16] On Prompt Engineering[00:16:16] Alessio Fanelli: I was reading on Twitter, I think somebody was saying in the future will be kind of like in the hair potter word. People have like the spell book, they pull it out, they do all the stuff in chat.[00:16:24] GP z. When you talk with customers, like are they excited about doing prompt engineering and kind of getting a starting point or do they, do they wish there was like a better interface? ?[00:16:34] Logan Kilpatrick: Yeah, that's a good question. I think prompt engineering is so much more of an art than a science right now. Like I think there are like really.[00:16:42] Systematic things that you can do and like different like approaches and designs that you can take, but really it's a lot of like, you kind of just have to try it and figure it out. And I actually think that this remains to be one of the challenges with large language models in general, and not just head open ai, but for everyone doing it is that it's really actually difficult to understand what are the capabilities of the model and how do I get it to do the things that I wanted to do.[00:17:05] And I think that's probably where a lot of folks need to do like academic research and companies need to invest in understanding the capabilities of these models and the limitations because it's really difficult to articulate the capabilities of a model without those types of things. So I'm hopeful that, and we're shipping hopefully some new updated prompt engineering stuff.[00:17:24] Cause I think the stuff we have on the website is old, and I think the cookbook actually has a little bit more up-to-date stuff. And so hopefully we'll ship some new prompt engineering stuff in the, in the short term. I think dispel some of the myths and rumors, but like I, it's gonna continue to be like a, a little bit of a pseudoscience, I would imagine.[00:17:41] And I also think that the whole prompt engineering being like a job in the future meme, I think is, I think it's slightly overblown. Like I think at, you see this now actually with like, there's tools that are showing up and I forgot what the, I just saw went on Twitter. The[00:17:57] swyx: next guest that we are having on this podcast, Lang.[00:17:59] Yeah. Yeah.[00:18:00] Logan Kilpatrick: Lang Chain and Harrison on, yeah, there's a bunch of repos too that like categorize and like collect all the best prompts that you can put into chat. For example, and like, that's like the people who are, I saw the advertisement for someone to be like a prompt engineer and it was like a $350,000 a year.[00:18:17] Mm-hmm. . Yeah, that was, that was philanthropic. Yeah, so it, it's just unclear to me like how, how sustainable stuff like that is. Cuz like, once you figure out the interesting prompts and like right now it's kind of like the, the Wild West, but like in a year you'll be able to sort of categorize all those and then people will be able to find all the good ones that are relevant for what they want to do.[00:18:35] And I think this goes back to like, having the examples is super important and I'm, I'm with you as well. Like every time I use Dall-E the little. While it's rendering the image, it gives you like a suggestion of like how you should ask for the art to be generated. Like do it in like a cyberpunk format. Do it in a pixel art format.[00:18:53] Et cetera, et cetera, and like, I really need that. I'm like, I would never come up with asking for those things had it not prompted me to like ask it that way. And now I always ask for pixel art stuff or cyberpunk stuff and it looks so cool. That's what I, I think,[00:19:06] swyx: is the innovation of ChatGPT as a format.[00:19:09] It reduces. The need for getting everything into your prompt in the first try. Mm-hmm. , it takes it from zero shot to a few shot. If, if, if that, if prompting as, as, as shots can be concerned.[00:19:21] Logan Kilpatrick: Yeah. , I think that's a great perspective and, and again, this goes back to the ux UI piece of it really being sort of the differentiating layer from some of the other stuff that was already out there.[00:19:31] Because you could kind of like do this before with oh oh three or something like that if you just made the right interface and like built some sort of like prompt retry interface. But I don't think people were really, were really doing that. And I actually think that you really need that right now. And this is the, again, going back to the difference between like how you can use generative models versus like large scale.[00:19:53] Computer vision systems for self-driving cars, like the, the answer doesn't actually need to be right all the time. That's the beauty of, of large language models. It can be wrong 50% of the time and like it doesn't really cost you anything to like regenerate a new response. And there's no like, critical safety issue with that, so you don't need those.[00:20:09] I, I keep seeing these tweets about like, you need those like 99.99% reliability and like the three nines or whatever it is. Mm-hmm. , but like you really don't need that because the cost of regenerating the prop is again, almost, almost. I think you tweeted a[00:20:23] Alessio Fanelli: couple weeks ago that the average person doesn't yet fully grasp how GBT is gonna impact human life in the next four, five years.[00:20:30] Usecases and LLM-Native Products[00:20:30] Alessio Fanelli: I think you had an example in education. Yeah. Maybe touch on some of these. Example of non-tech related use cases that are enabling, enabled by C G B[00:20:38] T.[00:20:39] Logan Kilpatrick: I'm so excited and, and there's a bunch of other like random threads that come to my mind now. I saw a thread and, and our VP of product was, Peter, was, was involved in that thread as well, talking about like how the use of systems like ChatGPT will unlock like pretty almost low to zero cost access to like mental health services.[00:20:59] You know, you can imagine like the same use case for education, like really personalized tutors and like, it's so crazy to think about, but. The technology is not actually , like it's, it's truly like an engineering problem at this point of like somebody using one of these APIs to like build something like that and then hopefully the models get a little bit better and make it, make it better as well.[00:21:20] But like it, I have no doubt in my mind that three years from now that technology will exist for every single student in the world to like have that personalized education experience, have a pr, have a chat based experience where like they'll be able. Ask questions and then the curriculum will just evolve and be constructed for them in a way that keeps, I think the cool part is in a way that keeps them engaged, like it doesn't have to be sort of like the same delivery of curriculum that you've always seen, and this now supplements.[00:21:49] The sort of traditional education experience in the sense of, you know, you don't need teachers to do all of this work. They can really sort of do the thing that they're amazing at and not spend time like grading assignments and all that type of stuff. Like, I really do think that all those could be part of the, the system.[00:22:04] And same thing, I don't know if you all saw the the do not pay, uh, lawyer situation, say, I just saw that Twitter thread, I think yesterday around they were going to use ChatGPT in the courtroom and basically I think it was. California Bar or the Bar Institute said that they were gonna send this guy to prison if he brought, if he put AirPods in and started reading what ChatGPT was saying to him.[00:22:26] Yeah.[00:22:26] swyx: To give people the context, I think, like Josh Browder, the CEO of Do Not Pay, was like, we will pay you money to put this AirPod into your ear and only say what we tell you to say fr from the large language model. And of course the judge was gonna throw that out. I mean, I, I don't see how. You could allow that in your court,[00:22:42] Logan Kilpatrick: Yeah, but I, I really do think that, like, the, the reality is, is that like, again, it's the same situation where the legal spaces even more so than education and, and mental health services, is like not an accessible space. Like every, especially with how like overly legalized the United States is, it's impossible to get representation from a lawyer, especially if you're low income or some of those things.[00:23:04] So I'm, I'm optimistic. Those types of services will exist in the future. And you'll be able to like actually have a, a quality defense representative or just like some sort of legal counsel. Yeah. Like just answer these questions, what should I do in this situation? Yeah. And I like, I have like some legal training and I still have those same questions.[00:23:22] Like I don't know what I would do in that situation. I would have to go and get a lawyer and figure that out. And it's, . It's tough. So I'm excited about that as well. Yeah.[00:23:29] Alessio Fanelli: And when you think about all these vertical use cases, do you see the existing products implementing language models in what they have?[00:23:35] Or do you think we're just gonna see L L M native products kind of come to market and build brand[00:23:40] Logan Kilpatrick: new experiences? I think there'll be a lot of people who build the L l M first experience, and I think that. At least in the short term, those are the folks who will have the advantage. I do think that like the medium to long term is again, thinking about like what is your moat for and like again, and everyone has access to, you know, ChatGPT and to the different models that we have available.[00:24:05] So how can you build a differentiated business? And I think a lot of it actually will come down to, and this is just the true and the machine learning world in general, but having. Unique access to data. So I think if you're some company that has some really, really great data about the legal space or about the education space, you can use that and be better than your competition by fine tuning these models or building your own specific LLMs.[00:24:28] So it'll, it'll be interesting to see how that plays out, but I do think that. from a product experience, it's gonna be better in the short term for people who build the, the generative AI first experience versus people who are sort of bolting it onto their mm-hmm. existing product, which is why, like, again, the, the Google situation, like they can't just put in like the prompt into like right below the search bar.[00:24:50] Like, it just, it would be a weird experience and, and they have to sort of defend that experience that they have. So it, it'll be interesting to see what happens. Yeah. Perplexity[00:24:58] swyx: is, is kind of doing that. So you're saying perplexity will go Google ?[00:25:04] Logan Kilpatrick: I, I think that perplexity has a, has a chance in the short term to actually get more people to try the product because it's, it's something different I think, whether they can, I haven't actually used, so I can't comment on like that experience, but like I think the long term is like, How can they continue to differentiate?[00:25:21] And, and that's really the focus for like, if you're somebody building on these models, like you have to be, your first thought should be, how do I build a differentiated business? And if you can't come up with 10 reasons that you can build a differentiated business, you're probably not gonna succeed in, in building something that that stands the test of time.[00:25:37] Yeah.[00:25:37] Risks and benefits of building on OpenAI[00:25:37] swyx: I think what's. As a potential founder or something myself, like what's scary about that is I would be building on top of open ai. I would be sending all my stuff to you for fine tuning and embedding and what have you. By the way, fine tuning, embedding is their, is there a third one? Those are the main two that I know of.[00:25:55] Okay. And yeah, that's the risk. I would be a open AI API reseller.[00:26:00] Logan Kilpatrick: Yeah. And, and again, this, this comes back down to like having a clear sense of like how what you're building is different. Like the people who are just open AI API resellers, like, you're not gonna, you're not gonna have a successful business doing that because everybody has access to the Yeah.[00:26:15] Jasper's pretty great. Yeah, Jasper's pretty great because I, I think they've done a, they've, they've been smart about how they've positioned the product and I was actually a, a Jasper customer before I joined OpenAI and was using it to do a bunch of stuff. because the interface was simple because they had all the sort of customized, like if you want for like a response for this sort of thing, they'd, they'd pre-done that prompt engineering work for us.[00:26:39] I mean, you could really just like put in some exactly what you wanted and then it would make that Amazon product description or whatever it is. So I think like that. The interface is the, the differentiator for, for Jasper. And again, whether that send test time, hopefully, cuz I know they've raised a bunch of money and have a bunch of employees, so I'm, I'm optimistic for them.[00:26:58] I think that there's enough room as well for a lot of these companies to succeed. Like it's not gonna, the space is gonna get so big so quickly that like, Jasper will be able to have a super successful business. And I think they are. I just saw some, some tweets from the CEO the other day that I, I think they're doing, I think they're doing well.[00:27:13] Alessio Fanelli: So I'm the founder of A L L M native. I log into open ai, there's 6 million things that I can do. I'm on the playground. There's a lot of different models. How should people think about exploring the surface area? You know, where should they start? Kind of like hugging the go deeper into certain areas.[00:27:30] Logan Kilpatrick: I think six months ago, I think it would've been a much different conversation because people hadn't experienced ChatGPT before.[00:27:38] Now that people have experienced ChatGPT, I think there's a lot more. Technical things that you should start looking into and, and thinking about like the differentiators that you can bring. I still think that the playground that we have today is incredible cause it does sort of similar to what Jasper does, which is like we have these very focused like, you know, put in a topic and we'll generate you a summary, but in the context of like explaining something to a second grader.[00:28:03] So I think all of those things like give a sense, but we only have like 30 on the website or something like that. So really doing a lot of exploration around. What is out there? What are the different prompts that you can use? What are the different things that you can build on? And I'm super bullish on embeddings, like embed everything and that's how you can build cool stuff.[00:28:20] And I keep seeing all these Boris who, who I talked about before, who did a bunch of the cookbook stuff, tweeted the other day that his like back of the hand, back of the napkin math, was that 50 million bucks you can embed the whole internet. I'm like, Some companies gonna spend the 50 million and embed the whole internet and like, we're gonna find out what that product looks like.[00:28:40] But like, there's so many cool things that you could do if you did have the whole internet embedded. Yeah, and I, I mean, I wouldn't be surprised if Google did that cuz 50 million is a drop in the bucket and they already have the whole internet, so why not embed it?[00:28:52] swyx: Can can I ask a follow up question on that?[00:28:54] Cuz I am just learning about embeddings myself. What makes open eyes embeddings different from other embeddings? If, if there's like, It's okay if you don't have the, the numbers at hand, but I'm just like, why should I use open AI emitting versus others? I[00:29:06] Logan Kilpatrick: don't understand. Yeah, that's a really good question.[00:29:08] So I'm still ramping up on my understanding of embeddings as well. So the two things that come to my mind, one, going back to the 50 million to embed the whole internet example, it's actually just super cheap. I, I don't know the comparisons of like other prices, but at least from what I've seen people talking about on Twitter, like the embeddings that that we have in the API is just like significantly cheaper than a lot of other c.[00:29:30] Embeddings. Also the accuracy of some of the benchmarks that are like, Sort of academic benchmarks to use in embeddings. I know at least I was just looking back through the blog post from when we announced the new text embedding model, which is what Powers embeddings and it's, yeah, the, on those metrics, our API is just better.[00:29:50] So those are the those. I'll go read it up. Yeah, those are the two things. It's a good. It's a good blog post to read. I think the most recent one that came out, but, and also the original one from when we first announced the Embeddings api, I think also was a, it had, that one has a little bit more like context around if you're trying to wrap your head around embeddings, how they work.[00:30:06] That one has the context, the new one just has like the fancy new stuff and the metrics and all that kind of stuff.[00:30:11] swyx: I would shout a hugging face for having really good content around what these things like foundational concepts are. Because I was familiar with, so, you know, in Python you have like text tove, my first embedding as as a, as someone getting into nlp.[00:30:24] But then developing the concept of sentence embeddings is, is as opposed to words I think is, is super important. But yeah, it's an interesting form of lock in as a business because yes, I'm gonna embed all my source data, but then every inference needs an embedding as. . And I think that is a risk to some people, because I've seen some builders should try and build on open ai, call that out as, as a cost, as as like, you know, it starts to add a cost to every single query that you, that you[00:30:48] Logan Kilpatrick: make.[00:30:49] Yeah. It'll be interesting to see how it all plays out, but like, my hope is that that cost isn't the barrier for people to build because it's, it's really not like the cost for doing the incremental like prompts and having them embedded is, is. Cent less than cents, but[00:31:06] swyx: cost I, I mean money and also latency.[00:31:08] Yeah. Which is you're calling the different api. Yeah. Anyway, we don't have to get into that.[00:31:13] Alessio Fanelli: No, but I think embeds are a good example. You had, I think, 17 versions of your first generation, what api? Yeah. And then you released the second generation. It's much cheaper, much better. I think like the word on the street is like when GPT4 comes out, everything else is like trash that came out before it.[00:31:29] It's got[00:31:30] Logan Kilpatrick: 100 trillion billion. Exactly. Parameters you don't understand. I think Sam has already confirmed that those are, those are not true . The graphics are not real. Whatever you're seeing on Twitter about GPT4, you're, I think the direct quote was, you're begging to be disappointed by continuing to, to put that hype out.[00:31:47] So[00:31:48] Alessio Fanelli: if you're a developer building on these, What's kind of the upgrade path? You know, I've been building on Model X, now this new model comes out. What should I do to be ready to move on?[00:31:58] Logan Kilpatrick: Yeah. I think all of these types of models folks have to think about, like there will be trade offs and they'll also be.[00:32:05] Breaking changes like any other sort of software improvement, like things like the, the prompts that you were previously expecting might not be the prompts that you're seeing now. And you can actually, you, you see this in the case of the embeddings example that you just gave when we released Tex embeddings, ADA oh oh two, ada, ada, whichever it is oh oh two, and it's sort of replaced the previous.[00:32:26] 16 first generation models, people went through this exact experience where like, okay, I need to test out this new thing, see how it works in my environment. And I think that the really fascinating thing is that there aren't, like the tools around doing this type of comparison don't exist yet today. Like if you're some company that's building on lms, you sort of just have to figure it out yourself of like, is this better in my use case?[00:32:49] Is this not better? In my use case, it's, it's really difficult to tell because the like, Possibilities using generative models are endless. So I think folks really need to focus on, again, that goes back to how to build a differentiated business. And I think it's understanding like what is the way that people are using your product and how can you sort of automate that in as much way and codify that in a way that makes it clear when these different models come up, whether it's open AI or other companies.[00:33:15] Like what is the actual difference between these and which is better for my use case because the academic be. It'll be saturated and people won't be able to use them as a point of comparison in the future. So it'll be important to think about. For your specific use case, how does it differentiate?[00:33:30] swyx: I was thinking about the value of frameworks or like Lang Chain and Dust and what have you out there.[00:33:36] I feel like there is some value to building those frameworks on top of Open Eyes, APIs. It kind of is building what's missing, essentially what, what you guys don't have. But it's kind of important in the software engineering sense, like you have this. Unpredictable, highly volatile thing, and you kind of need to build a stable foundation on top of it to make it more predictable, to build real software on top of it.[00:33:59] That's a super interesting kind of engineering problem. .[00:34:03] Logan Kilpatrick: Yeah, it, it is interesting. It's also the, the added layer of this is that the large language models. Are inherently not deterministic. So I just, we just shipped a small documentation update today, which, which calls this out. And you think about APIs as like a traditional developer experience.[00:34:20] I send some response. If the response is the same, I should get the same thing back every time. Unless like the data's updating and like a, from like a time perspective. But that's not the, that's not the case with the large language models, even with temperature zero. Mm-hmm. even with temperature zero. Yep.[00:34:34] And that's, Counterintuitive part, and I think someone was trying to explain to me that it has to do with like Nvidia. Yeah. Floating points. Yes. GPU stuff. and like apparently the GPUs are just inherently non-deterministic. So like, yes, there's nothing we can do unless this high Torch[00:34:48] swyx: relies on this as well.[00:34:49] If you want to. Fix this. You're gonna have to tear it all down. ,[00:34:53] Logan Kilpatrick: maybe Nvidia, we'll fix it. I, I don't know, but I, I think it's a, it's a very like, unintuitive thing and I don't think that developers like really get that until it happens to you. And then you're sort of scratching your head and you're like, why is this happening?[00:35:05] And then you have to look it up and then you see all the NVIDIA stuff. Or hopefully our documentation makes it more clear now. But hopefully people, I also think that's, it's kinda the cool part as well. I don't know, it's like, You're not gonna get the same stuff even if you try to.[00:35:17] swyx: It's a little spark of originality in there.[00:35:19] Yeah, yeah, yeah, yeah. The random seed .[00:35:22] OpenAI Codex[00:35:22] swyx: Should we ask about[00:35:23] Logan Kilpatrick: Codex?[00:35:23] Alessio Fanelli: Yeah. I mean, I love Codex. I use it every day. I think like one thing, sometimes the code is like it, it's kinda like the ChatGPT hallucination. Like one time I asked it to write up. A Twitter function, they will pull the bayou of this thing and it wrote the whole thing and then the endpoint didn't exist once I went to the Twitter, Twitter docs, and I think like one, I, I think there was one research that said a lot of people using Co Palace, sometimes they just auto complete code that is wrong and then they commit it and it's a, it's a big[00:35:51] Logan Kilpatrick: thing.[00:35:51] swyx: Do you secure code as well? Yeah, yeah, yeah, yeah. I saw that study.[00:35:54] Logan Kilpatrick: How do[00:35:54] Alessio Fanelli: you kind of see. Use case evolving. You know, you think, like, you obviously have a very strong partnership with, with Microsoft. Like do you think Codex and VS code will just keep improving there? Do you think there's kind of like a. A whole better layer on top of it, which is from the scale AI hackathon where the, the project that one was basically telling the l l m, you're not the back end of a product[00:36:16] And they didn't even have to write the code and it's like, it just understood. Yeah. How do you see the engineer, I, I think Sean, you said copilot is everybody gets their own junior engineer to like write some of the code and then you fix it For me, a lot of it is the junior engineer gets a senior engineer to actually help them write better code.[00:36:32] How do you see that tension working between the model and the. It'll[00:36:36] Logan Kilpatrick: be really interesting to see if there's other, if there's other interfaces to this. And I think I've actually seen a lot of people asking, like, it'd be really great if I had ChatGPT and VS code because in, in some sense, like it can, it's just a better, it's a better interface in a lot of ways to like the, the auto complete version cuz you can reprompt and do, and I know Via, I know co-pilot actually has that, where you can like click and then give it, it'll like pop up like 10 suggested.[00:36:59] Different options instead of brushes. Yeah, copilot labs, yeah. Instead of the one that it's providing. And I really like that interface, but again, this goes back to. I, I do inherently think it'll get better. I think it'll be able to do a lot, a lot more of the stuff as the models get bigger, as they have longer context as they, there's a lot of really cool things that will end up coming out and yeah, I don't think it's actually very far away from being like, much, much better.[00:37:24] It'll go from the junior engineer to like the, the principal engineer probably pretty quickly. Like I, I don't think the gap is, is really that large between where things are right now. I think like getting it to the point. 60% of the stuff really well to get it to do like 90% of the stuff really well is like that's within reach in the next, in the next couple of years.[00:37:45] So I'll be really excited to see, and hopefully again, this goes back to like engineers and developers and people who aren't thinking about how to integrate. These tools, whether it's ChatGPT or co-pilot or something else into their workflows to be more efficient. Those are the people who I think will end up getting disrupted by these tools.[00:38:02] So figuring out how to make yourself more valuable than you are today using these tools, I think will be super important for people. Yeah.[00:38:09] Alessio Fanelli: Actually use ChatGPT to debug, like a react hook the other day. And then I posted in our disc and I was like, Hey guys, like look, look at this thing. It really helped me solve this.[00:38:18] And they. That's like the ugliest code I've ever seen. It's like, why are you doing that now? It's like, I don't know. I'm just trying to get[00:38:24] Logan Kilpatrick: this thing to work and I don't know, react. So I'm like, that's the perfect, exactly, that's the perfect solution. I, I did this the other day where I was looking at React code and like I have very briefly seen React and run it like one time and I was like, explain how this is working.[00:38:38] So, and like change it in this way that I want to, and like it was able to do that flawlessly and then I just popped it in. It worked exactly like I. I'll give a[00:38:45] swyx: little bit more context cause I was, I was the guy giving you feedback on your code and I think this is a illustrative of how large language models can sort of be more confident than they should be because you asked it a question which is very specific on how to improve your code or fix your code.[00:39:00] Whereas a real engineer would've said, we've looked at your code and go, why are you doing it at at all? Right? So there's a sort of sycophantic property of martial language. Accepts the basis of your question, whereas a real human might question your question. Mm-hmm. , and it was just not able to do that. I mean, I, I don't see how he could do that.[00:39:17] Logan Kilpatrick: Yeah. It's, it's interesting. I, I saw another example of this the other day as well with some chatty b t prompt and I, I agree. It'll be interesting to see if, and again, I think not to, not to go back to Sam's, to Sam's talk again, but like, he, he talked real about this, and I think this makes a ton of sense, which is like you should be able to have, and this isn't something that that exists right now, but you should be able to have the model.[00:39:39] Tuned in the way that you wanna interact with. Like if you want a model that sort of questions what you're asking it to do, like you should be able to have that. And I actually don't think that that's as far away as like some of the other stuff. Um, It, it's a very possible engineering problem to like have the, to tune the models in that way and, and ask clarifying questions, which is even something that it doesn't do right now.[00:39:59] It'll either give you the response or it won't give you the response, but it'll never say like, Hey, what do you mean by this? Which is super interesting cuz that's like we spend as humans, like 50% of our conversational time being like, what do you mean by that? Like, can you explain more? Can you say it in a different way?[00:40:14] And it's, it's fascinating that the model doesn't do that right now. It's, it's interesting.[00:40:20] swyx: I have written a piece on sort of what AGI hard might be, which is the term that is being thrown around as like a layer of boundary for what is, what requires an A real AGI to do and what, where you might sort of asymptotically approach.[00:40:33] So, What people talk about is essentially a theory of mind, developing a con conception of who I'm talking to and persisting that across sessions, which essentially ChatGPT or you know, any, any interface that you build on top of GPT3 right now would not be able to do. Right? Like, you're not persisting you, you are persisting that history, but you don't, you're not building up a conception of what you know and what.[00:40:54] I should fill in the blanks for you or where I should question you. And I think that's like the hard thing to understand, which is what will it take to get there? Because I think that to me is the, going back to your education thing, that is the biggest barrier, which is I, the language model doesn't have a memory or understanding of what I know.[00:41:11] and like, it's, it's too much to tell them what I don't know. Mm-hmm. , there's more that I don't know than I, than I do know . I think the cool[00:41:16] Logan Kilpatrick: part will be when, when you're able to, like, imagine you could upload all of the, the stuff that you've ever done, all the texts, the work that you've ever done before, and.[00:41:27] The model can start to understand, hey, what are the, what are the conceptual gaps that this person has based on what you've said, based on what you've done? I think that would be really interesting. Like if you can, like I have good notes on my phone and I can still go back to see all of the calculus classes that I took and I could put in all my calculus notebooks and all the assignments and stuff that I did in, in undergrad and grad school, and.[00:41:50] basically be like, Hey, here are the gaps in your understanding of calculus. Go and do this right now. And I think that that's in the education space. That's exactly what will end up happening. You'll be able to put in all this, all the work that you've done. It can understand those ask and then come up with custom made questions and prompts and be like, Hey, how, you know, explain this concept to me and if it.[00:42:09] If you can't do that, then it can sort of put that into your curriculum. I think like Khan Academy as an example, already does some of this, like personalized learning. You like take assessments at the beginning of every Khan Academy model module, and it'll basically only have you watch the videos and do the assignments for the things that like you didn't test well into.[00:42:27] So that's, it's, it's sort of close to already being there in some sense, but it doesn't have the, the language model interface on top of it before we[00:42:34] swyx: get into our lightning round, which is like, Quick response questions. Was there any other topics that you think you wanted to cover? We didn't touch on, whisper.[00:42:40] We didn't touch on Apple. Anything you wanted to[00:42:42] Logan Kilpatrick: talk?[00:42:43] Apple's Neural Engine[00:42:43] Logan Kilpatrick: Yeah, I think the question around Apple stuff and, and the neural engine, I think will be really interesting to see how it all plays out. I think, I don't know if you wanna like ask just to give the context around the neural engine Apple question. Well, well, the[00:42:54] swyx: only thing I know it's because I've seen Apple keynotes.[00:42:57] Everyone has, you know, I, I have a m M one MacBook Cure. They have some kind of neuro chip. , but like, I don't see it in my day-to-day life, so when is this gonna affect me, essentially? And you worked at Apple, so I I was just gonna throw the question over to you, like, what should we[00:43:11] Logan Kilpatrick: expect out of this? Yeah.[00:43:12] The, the problem that I've seen so far with the neural engine and all the, the Mac, and it's also in the phones as well, is that the actual like, API to sort of talk to the neural engine isn't something that's like a common you like, I'm pretty sure it's either not exposed at all, like it only like Apple basically decides in the software layer Yeah.[00:43:34] When, when it should kick in and when it should be used, which I think doesn't really like help developers and it doesn't, that's why no one is using it. I saw a bunch of, and of course I don't have any good insight on this, but I saw a bunch of rumors that we're talking about, like a lot of. Main use cases for the neural engine stuff.[00:43:50] It's, it's basically just in like phantom mode. Now, I'm sure it's doing some processing, but like the main use cases will be a lot of the ar vr stuff that ends up coming out and like when it gets much heavier processing on like. Graphic stuff and doing all that computation, that's where it'll be. It'll be super important.[00:44:06] And they've basically been able to trial this for the last, like six years and have it part of everything and make sure that they can do it cheaply in a cost effective way. And so it'll be cool to see when that I'm, I hope it comes out. That'll be awesome.[00:44:17] swyx: Classic Apple, right? They, they're not gonna be first, but when they do it, they'll make a lot of noise about it.[00:44:21] Yeah. . It'll be[00:44:22] Logan Kilpatrick: awesome. Sure.[00:44:22] Lightning Round[00:44:22] Logan Kilpatrick: So, so are we going to light. Let's[00:44:24] Alessio Fanelli: do it. All right. Favorite AI products not[00:44:28] Logan Kilpatrick: open AI. Build . I think synthesis. Is synthesis.io is the, yeah, you can basically put in like a text prompt and they have like a human avatar that will like speak and you can basically make content in like educational videos.[00:44:44] And I think that's so cool because maybe as people who are making content, like it's, it's super hard to like record video. It just takes a long time. Like you have to edit all the stuff, make sure you sound right, and then when you edit yourself talking it's super weird cuz your mouth is there and things.[00:44:57] So having that and just being able to ChatGPT A script. Put it in. Hopefully I saw another demo of like somebody generating like slides automatically using some open AI stuff. Like I think that type of stuff. Chat, BCG, ,[00:45:10] swyx: a fantastic name, best name of all time .[00:45:14] Logan Kilpatrick: I think that'll be cool. So I'm super excited,[00:45:16] swyx: but Okay.[00:45:16] Well, so just a follow up question on, on that, because we're both in that sort of Devrel business, would you put AI Logan on your video, on your videos and a hundred[00:45:23] Logan Kilpatrick: percent, explain that . A hundred percent. I would, because again, if it reduces the time for me, like. I am already busy doing a bunch of other stuff,[00:45:31] And if I could, if I could take, like, I think the real use case is like I've made, and this is in the sense of like creators wanting to be on every platform. If I could take, you know, the blog posts that I wrote and then have AI break it up into a bunch of things, have ai Logan. Make a TikTok, make a YouTube video.[00:45:48] I cannot wait for that. That's gonna be so nice. And I think there's probably companies who are already thinking about doing that. I'm just[00:45:53] swyx: worried cuz like people have this uncanny valley reaction to like, oh, you didn't tell me what I just watched was a AI generated thing. I hate you. Now you know there, there's a little bit of ethics there and I'm at the disclaimer,[00:46:04] Logan Kilpatrick: at the top.[00:46:04] Navigating. Yeah. I also think people will, people will build brands where like their whole thing is like AI content. I really do think there are AI influencers out there. Like[00:46:12] swyx: there are entire Instagram, like million plus follower accounts who don't exist.[00:46:16] Logan Kilpatrick: I, I've seen that with the, the woman who's a Twitch streamer who like has some, like, she's using like some, I don't know, that technology from like movies where you're like wearing like a mask and it like changes your facial appearance and all that stuff.[00:46:27] So I think there's, there's people who find their niche plus it'll become more common. So, cool. My[00:46:32] swyx: question would be, favorite AI people in communities that you wanna shout up?[00:46:37] Logan Kilpatrick: I think there's a bunch of people in the ML ops community where like that seemed to have been like the most exciting. There was a lot of innovation, a lot of cool things happening in the ML op space, and then all the generative AI stuff happened and then all the ML Ops two people got overlooked.[00:46:51] They're like, what's going on here? So hopefully I still think that ML ops and things like that are gonna be super important for like getting machine learning to be where it needs to be for us to. AGI and all that stuff. So a year from[00:47:05] Alessio Fanelli: now, what will people be the most[00:47:06] Logan Kilpatrick: surprised by? N. I think the AI is gonna get very, very personalized very quickly, and I don't think that people have that feeling yet with chat, BT, but I, I think that that's gonna, that's gonna happen and they'll be surprised in like the, the amount of surface areas in which AI is present.[00:47:23] Like right now it's like, it's really exciting cuz Chat BT is like the one place that you can sort of get that cool experience. But I think that, The people at Facebook aren't dumb. The people at Google aren't dumb. Like they're gonna have, they're gonna have those experiences in a lot of different places and I think that'll be super fascinating to see.[00:47:40] swyx: This is for the builders out there. What's an AI thing you would pay for if someone built it with their personal[00:47:45] Logan Kilpatrick: work? I think more stuff around like transfer learning for, like making transfer, learning easier. Like I think that's truly the way to. Build really cool things is transfer learning, fine tuning, and I, I don't think that there's enough.[00:48:04] Jeremy Howard who created Fasted AI talks a lot about this. I mean, it's something that really resonates with me and, and for context, like at Apple, all the machine learning stuff that we did was transfer learning because it was so powerful. And I think people have this perception that they need to.[00:48:18] Build things from scratch and that's not the case. And I think especially as large language models become more accessible, people need to build layers and products on top of this to make transfer learning more accessible to more people. So hopefully somebody builds something like that and we can all train our own models.[00:48:33] I think that's how you get like that personalized AI experiences you put in your stuff. Make transfer learning easy. Everyone wins. Just just to vector in[00:48:40] swyx: a little bit on this. So in the stable diffusion community, there's a lot of practice of like, I'll fine tune a custom dis of stable diffusion and share it.[00:48:48] And then there also, there's also this concept of, well, first it was textual inversion and then dream booth where you essentially train a concept that you can sort of add on. Is that what you're thinking about when you talk about transfer learning or is that something[00:48:59] Logan Kilpatrick: completely. I feel like I'm not as in tune with the generative like image model community as I probably should be.[00:49:07] I, I think that that makes a lot of sense. I think there'll be like whole ecosystems and marketplaces that are sort of built around exactly what you just said, where you can sort of fine tune some of these models in like very specific ways and you can use other people's fine tunes. That'll be interesting to see.[00:49:21] But, c.ai is,[00:49:23] swyx: what's it called? C C I V I Ts. Yeah. It's where people share their stable diffusion checkpoints in concepts and yeah, it's[00:49:30] Logan Kilpatrick: pretty nice. Do you buy them or is it just like free? Like open. Open source? It's, yeah. Cool. Even better.[00:49:34] swyx: I think people might want to sell them. There's a, there's a prompt marketplace.[00:49:38] Prompt base, yeah. Yeah. People hate it. Yeah. They're like, this should be free. It's just text. Come on, .[00:49:45] Alessio Fanelli: Hey, it's knowledge. All right. Last question. If there's one thing you want everyone to take away about ai, what would.[00:49:51] Logan Kilpatrick: I think the AI revolution is gonna, you know, it's been this like story that people have been talking about for the longest time, and I don't think that it's happened.[00:50:01] It was really like, oh, AI's gonna take your job, AI's gonna take your job, et cetera, et cetera. And I think people have sort of like laughed that off for a really long time, which was fair because it wasn't happening. And I think now, Things are going to accelerate very, very quickly. And if you don't have your eyes wide open about what's happening, like there's a good chance that something that you might get left behind.[00:50:21] So I'm, I'm really thinking deeply these days about like how that is going to impact a lot of people. And I, I'm hopeful that the more widespread this technology becomes, the more mainstream this technology becomes, the more people will benefit from it and hopefully not be affected in that, in that negative way.[00:50:35] So use these tools, put them into your workflow, and, and hopefully that will, and that will acceler. Well,[00:50:41] swyx: we're super happy that you're at OpenAI getting this message out there, and I'm sure we'll see a l

In this episode of Fibonacci, the Red Olive Data Podcast, we interview James Gardiner, head of data technology for Admiral Group. James has 25 years of experience working with data and has been at Admiral for the past seven years, recently managing its move to the cloud.We chat about the importance of applying an engineering mindset to data projects, data governance, the art of the possible and more in a wide-ranging chat.Here are the topics we discuss with their timecodes:How James got into data engineering (1m)The importance of emphasising an engineering mentality to data teams (6m 9s)Giving people the data to make good decisions, including live streams and integrating it with sales processes (7m 17s)Understanding customers to give them the best price, while balancing risk (9m)Scoping a project and getting both the right people and architecture in place (10m 35s)The importance of “the why” of a project (12m 42s)Security is one of the cloud's biggest risks: how to manage that with good governance (14m)Training a young team to respond well when things go wrong (17m)Applying machine learning to monitoring (19m)Mesh architecture and ML Ops (20m 18s)Communicating well with your team and re-using code (24m 30s)Using data governance as an enabler, rather than a blocker (27m 50s)Learning from Red Olive (31m)The important skills for people to learn, and data automation (33m 18s)

Es ist leichter geworden, Machine Learning einzusetzen. Viele Tutorials und Anleitungen und auch die passende Software gibt es umsonst oder für wenig Geld. Sind Entwickler:innen erfahren, können sie sich schnell in die Materie einarbeiten und Ergebnisse produzieren. Klingt zu gut, um wahr zu sein? Ist es auch. Viele dieser Ansätze schaffen es nicht über den Protoyp-Status hinaus. Für die reine Softwareentwicklung hat sich DevOPs etabliert, also das Zusammendenken von Entwicklung und Betrieb komplexer IT-Systeme. Doch wie hilft ML Ops beim Machine Learning?Link zum Fundstück der Woche.Heute ein Gast aus den eigenen Reihen: Jeffrey Remien Softwarearchitekt bei OPITZ CONSULTING und Machine Learning Enthusiast.OPITZ CONSULTING ■■■ Digitale Service Manufaktur

On the show this week, Carter Morgan and Anu Srivastava talk about AI and ML data analytics with Dataiku VP of Platform Strategy, Jed Dougherty, and Head of Product Marketing, Dan Darnell. Dataiku is an AI platform targeted for business team collaboration. The low and no code environments make it easy for developers and not so tech savvy employees to work together on analytics projects. It strives for everyday AI, making these normally highly technical data processes more accessible. Our guests detail the tools Dataiku provides customers, including ML Ops features for efficient models. Dataiku's managed offering allows businesses to concentrate on the model while Dataiku takes care of things like the deployment processes behind the scenes. We hear about the partnership between Dataiku and Google Cloud and Dataiku's integration with AlloyDB. Through a real example, our guests run us through the use of these two tools together. Jed talks about why Google Cloud works so well with Dataiku, especially for businesses looking for cutting edge technology. Jed Dougherty Jed is the VP of Platform Strategy at Dataiku. In this role he acts as a strategic technical advisor to Dataiku customers and prospects. He also works tightly with Engineering and Product stakeholders in order to ensure that all technical platform requests are properly followed, scoped and implemented. Dan Darnell Dan has over 20 years of experience in the analytics industry at established software companies, hyper-growth technology companies, and small technology start-ups. As the Head of Product Marketing at Dataiku, he owns positioning, evangelism, and content creation for product offerings and education on products for customers and partners. Cool things of the week Google Cloud supercharges NLP with large language models blog Practicing the principle of least privilege with Cloud Build and Artifact Registry blog Interview Dataiku site Dataiku YouTube videos BigQuery site Kubernetes site GKE site AlloyDB for PostgreSQL site Accelerate AI Adoption: 3 Steps to Deploy Dataiku for Google Cloud Platform blog Implementing Dataiku with BigQuery docs GCP Podcast Episode 238: ASML with Arnaud Hubaux podcast GCP Podcast Episode 229: Lucidworks with Radu Miclaus podcast What's something cool you're working on? Anu is working on interesting speech use cases and Google's Speech to Text. Join in with this tutorial! Carter is working on getting organized and working on something super cool! Hosts Carter Morgan and Anu Srivastava

In this episode: Lauren Hawker Zafer is joined by Rasmus Hauch the CTO of 2021.AI Who Can Benefit From This Conversation? This conversation is for all listeners who are interested in understanding what boundaries need to be shifted in a trustworthy AI context in both the private and public sector. You will learn what tools and datasets are necessary when building models for production in machine learning projects and where maintenance and retraining is of essence. In addition, you will learn about how 2021.AI help with the ML OPs cycle in corporations and what processes need to be implemented to ensure that an organisation can confirm that models in production and new models being built are ethical. Who is Rasmus? Rasmus is the Chief Technology Officer at 2021.AI, an experienced technical leader and self-proclaimed geek with business drive. He is passionate about achieving business objectives and creating a better world using new technologies like AI, ML and IoT. He is skilled in relationship management and takes pride in people-development and company building. He has worked as a Program Manager, Project Manager, Product and Services Sales, Technical Sales, Product Manager, Product Owner, CTO, People Manager, DPO, Chief/Lead Architect, Senior Consultant and Software Engineer for various international Financial, Insurance, Healthcare, Pharma, Life Science, Industrial, Supply Chain, Energy and Telecom customers. REDEFINING AI is powered by The Squirro Academy - learn.squirro.com. Try our free courses on AI, ML, NLP and Cognitive Search at the Squirro Academy and find out more about Squirro here.

Moses Guttman from Clear ML joins us to share insights about how organizations leveraging machine learning keep their programs on track.  While many parallels exist between the software development life cycle (SWLC) and the machine learning development life cycle, successful deployments of ML in production have demonstrated that a unique set of tools is required.  Moses and I discuss the emergence of ML Ops, success stories, and how modern teams leverage tools like Clear ML's open source solution to maximize the value of ML in the organization.  

If you enjoyed this video, here are additional resources to look at:Coursera + Duke Specialization: Building Cloud Computing Solutions at Scale Specialization: https://www.coursera.org/specializations/building-cloud-computing-solutions-at-scalePython, Bash, and SQL Essentials for Data Engineering Specialization: https://www.coursera.org/specializations/python-bash-sql-data-engineering-dukeAWS Certified Solutions Architect - Professional (SAP-C01) Cert Prep: 1 Design for Organizational Complexity:https://www.linkedin.com/learning/aws-certified-solutions-architect-professional-sap-c01-cert-prep-1-design-for-organizational-complexity/design-for-organizational-complexity?autoplay=trueO'Reilly Book: Practical MLOps: https://www.amazon.com/Practical-MLOps-Operationalizing-Machine-Learning/dp/1098103017O'Reilly Book: Python for DevOps: https://www.amazon.com/gp/product/B082P97LDW/O'Reilly Book: Developing on AWS with C#: A Comprehensive Guide on Using C# to Build Solutions on the AWS Platformhttps://www.amazon.com/Developing-AWS-Comprehensive-Solutions-Platform/dp/1492095877Pragmatic AI: An Introduction to Cloud-based Machine Learning: https://www.amazon.com/gp/product/B07FB8F8QP/Pragmatic AI Labs Book: Python Command-Line Tools: https://www.amazon.com/gp/product/B0855FSFYZPragmatic AI Labs Book: Cloud Computing for Data Analysis: https://www.amazon.com/gp/product/B0992BN7W8Pragmatic AI Book: Minimal Python: https://www.amazon.com/gp/product/B0855NSRR7Pragmatic AI Book: Testing in Python: https://www.amazon.com/gp/product/B0855NSRR7Subscribe to Pragmatic AI Labs YouTube Channel: https://www.youtube.com/channel/UCNDfiL0D1LUeKWAkRE1xO5QSubscribe to 52 Weeks of AWS Podcast: https://52-weeks-of-cloud.simplecast.comView content on noahgift.com: https://noahgift.com/View content on Pragmatic AI Labs Website: https://paiml.com/[00:00.000 --> 00:02.260] Hey, three, two, one, there we go, we're live.[00:02.260 --> 00:07.260] All right, so welcome Simon to Enterprise ML Ops interviews.[00:09.760 --> 00:13.480] The goal of these interviews is to get people exposed[00:13.480 --> 00:17.680] to real professionals who are doing work in ML Ops.[00:17.680 --> 00:20.360] It's such a cutting edge field[00:20.360 --> 00:22.760] that I think a lot of people are very curious about.[00:22.760 --> 00:23.600] What is it?[00:23.600 --> 00:24.960] You know, how do you do it?[00:24.960 --> 00:27.760] And very honored to have Simon here.[00:27.760 --> 00:29.200] And do you wanna introduce yourself[00:29.200 --> 00:31.520] and maybe talk a little bit about your background?[00:31.520 --> 00:32.360] Sure.[00:32.360 --> 00:33.960] Yeah, thanks again for inviting me.[00:34.960 --> 00:38.160] My name is Simon Stebelena or Simon.[00:38.160 --> 00:40.440] I am originally from Austria,[00:40.440 --> 00:43.120] but currently working in the Netherlands and Amsterdam[00:43.120 --> 00:46.080] at Transaction Monitoring Netherlands.[00:46.080 --> 00:48.780] Here I am the lead ML Ops engineer.[00:49.840 --> 00:51.680] What are we doing at TML actually?[00:51.680 --> 00:55.560] We are a data processing company actually.[00:55.560 --> 00:59.320] We are owned by the five large banks of Netherlands.[00:59.320 --> 01:02.080] And our purpose is kind of what the name says.[01:02.080 --> 01:05.920] We are basically lifting specifically anti money laundering.[01:05.920 --> 01:08.040] So anti money laundering models that run[01:08.040 --> 01:11.440] on a personalized transactions of businesses[01:11.440 --> 01:13.240] we get from these five banks[01:13.240 --> 01:15.760] to detect unusual patterns on that transaction graph[01:15.760 --> 01:19.000] that might indicate money laundering.[01:19.000 --> 01:20.520] That's a natural what we do.[01:20.520 --> 01:21.800] So as you can imagine,[01:21.800 --> 01:24.160] we are really focused on building models[01:24.160 --> 01:27.280] and obviously ML Ops is a big component there[01:27.280 --> 01:29.920] because that is really the core of what you do.[01:29.920 --> 01:32.680] You wanna do it efficiently and effectively as well.[01:32.680 --> 01:34.760] In my role as lead ML Ops engineer,[01:34.760 --> 01:36.880] I'm on the one hand the lead engineer[01:36.880 --> 01:38.680] of the actual ML Ops platform team.[01:38.680 --> 01:40.200] So this is actually a centralized team[01:40.200 --> 01:42.680] that builds out lots of the infrastructure[01:42.680 --> 01:47.320] that's needed to do modeling effectively and efficiently.[01:47.320 --> 01:50.360] But also I am the craft lead[01:50.360 --> 01:52.640] for the machine learning engineering craft.[01:52.640 --> 01:55.120] These are actually in our case, the machine learning engineers,[01:55.120 --> 01:58.360] the people working within the model development teams[01:58.360 --> 01:59.360] and cross functional teams[01:59.360 --> 02:01.680] actually building these models.[02:01.680 --> 02:03.640] That's what I'm currently doing[02:03.640 --> 02:05.760] during the evenings and weekends.[02:05.760 --> 02:09.400] I'm also lecturer at the University of Applied Sciences, Vienna.[02:09.400 --> 02:12.080] And there I'm teaching data mining[02:12.080 --> 02:15.160] and data warehousing to master students, essentially.[02:16.240 --> 02:19.080] Before TMNL, I was at bold.com,[02:19.080 --> 02:21.960] which is the largest eCommerce retailer in the Netherlands.[02:21.960 --> 02:25.040] So I always tend to see the Amazon of the Netherlands[02:25.040 --> 02:27.560] or been a lux actually.[02:27.560 --> 02:30.920] It is still the biggest eCommerce retailer in the Netherlands[02:30.920 --> 02:32.960] even before Amazon actually.[02:32.960 --> 02:36.160] And there I was an expert machine learning engineer.[02:36.160 --> 02:39.240] So doing somewhat comparable stuff,[02:39.240 --> 02:42.440] a bit more still focused on the actual modeling part.[02:42.440 --> 02:44.800] Now it's really more on the infrastructure end.[02:45.760 --> 02:46.760] And well, before that,[02:46.760 --> 02:49.360] I spent some time in consulting, leading a data science team.[02:49.360 --> 02:50.880] That's actually where I kind of come from.[02:50.880 --> 02:53.360] I really come from originally the data science end.[02:54.640 --> 02:57.840] And there I kind of started drifting towards ML Ops[02:57.840 --> 02:59.200] because we started building out[02:59.200 --> 03:01.640] a deployment and serving platform[03:01.640 --> 03:04.440] that would as consulting company would make it easier[03:04.440 --> 03:07.920] for us to deploy models for our clients[03:07.920 --> 03:10.840] to serve these models, to also monitor these models.[03:10.840 --> 03:12.800] And that kind of then made me drift further and further[03:12.800 --> 03:15.520] down the engineering lane all the way to ML Ops.[03:17.000 --> 03:19.600] Great, yeah, that's a great background.[03:19.600 --> 03:23.200] I'm kind of curious in terms of the data science[03:23.200 --> 03:25.240] to ML Ops journey,[03:25.240 --> 03:27.720] that I think would be a great discussion[03:27.720 --> 03:29.080] to dig into a little bit.[03:30.280 --> 03:34.320] My background is originally more on the software engineering[03:34.320 --> 03:36.920] side and when I was in the Bay Area,[03:36.920 --> 03:41.160] I did individual contributor and then ran companies[03:41.160 --> 03:44.240] at one point and ran multiple teams.[03:44.240 --> 03:49.240] And then as the data science field exploded,[03:49.240 --> 03:52.880] I hired multiple data science teams and worked with them.[03:52.880 --> 03:55.800] But what was interesting is that I found that[03:56.840 --> 03:59.520] I think the original approach of data science[03:59.520 --> 04:02.520] from my perspective was lacking[04:02.520 --> 04:07.240] in that there wasn't really like deliverables.[04:07.240 --> 04:10.520] And I think when you look at a software engineering team,[04:10.520 --> 04:12.240] it's very clear there's deliverables.[04:12.240 --> 04:14.800] Like you have a mobile app and it has to get better[04:14.800 --> 04:15.880] each week, right?[04:15.880 --> 04:18.200] Where else, what are you doing?[04:18.200 --> 04:20.880] And so I would love to hear your story[04:20.880 --> 04:25.120] about how you went from doing kind of more pure data science[04:25.120 --> 04:27.960] to now it sounds like ML Ops.[04:27.960 --> 04:30.240] Yeah, yeah, actually.[04:30.240 --> 04:33.800] So back then in consulting one of the,[04:33.800 --> 04:36.200] which was still at least back then in Austria,[04:36.200 --> 04:39.280] data science and everything around it was still kind of[04:39.280 --> 04:43.720] in this infancy back then 2016 and so on.[04:43.720 --> 04:46.560] It was still really, really new to many organizations,[04:46.560 --> 04:47.400] at least in Austria.[04:47.400 --> 04:50.120] There might be some years behind in the US and stuff.[04:50.120 --> 04:52.040] But back then it was still relatively fresh.[04:52.040 --> 04:55.240] So in consulting, what we very often struggled with was[04:55.240 --> 04:58.520] on the modeling end, problems could be solved,[04:58.520 --> 05:02.040] but actually then easy deployment,[05:02.040 --> 05:05.600] keeping these models in production at client side.[05:05.600 --> 05:08.880] That was always a bit more of the challenge.[05:08.880 --> 05:12.400] And so naturally kind of I started thinking[05:12.400 --> 05:16.200] and focusing more on the actual bigger problem that I saw,[05:16.200 --> 05:19.440] which was not so much building the models,[05:19.440 --> 05:23.080] but it was really more, how can we streamline things?[05:23.080 --> 05:24.800] How can we keep things operating?[05:24.800 --> 05:27.960] How can we make that move easier from a prototype,[05:27.960 --> 05:30.680] from a PUC to a productionized model?[05:30.680 --> 05:33.160] Also how can we keep it there and maintain it there?[05:33.160 --> 05:35.480] So personally I was really more,[05:35.480 --> 05:37.680] I saw that this problem was coming up[05:38.960 --> 05:40.320] and that really fascinated me.[05:40.320 --> 05:44.120] So I started jumping more on that exciting problem.[05:44.120 --> 05:45.080] That's how it went for me.[05:45.080 --> 05:47.000] And back then we then also recognized it[05:47.000 --> 05:51.560] as a potential product in our case.[05:51.560 --> 05:54.120] So we started building out that deployment[05:54.120 --> 05:56.960] and serving and monitoring platform, actually.[05:56.960 --> 05:59.520] And that then really for me, naturally,[05:59.520 --> 06:01.840] I fell into that rabbit hole[06:01.840 --> 06:04.280] and I also never wanted to get out of it again.[06:05.680 --> 06:09.400] So the system that you built initially,[06:09.400 --> 06:10.840] what was your stack?[06:10.840 --> 06:13.760] What were some of the things you were using?[06:13.760 --> 06:17.000] Yeah, so essentially we had,[06:17.000 --> 06:19.560] when we talk about the stack on the backend,[06:19.560 --> 06:20.560] there was a lot of,[06:20.560 --> 06:23.000] so the full backend was written in Java.[06:23.000 --> 06:25.560] We were using more from a user perspective,[06:25.560 --> 06:28.040] the contract that we kind of had,[06:28.040 --> 06:32.560] our goal was to build a drag and drop platform for models.[06:32.560 --> 06:35.760] So basically the contract was you package your model[06:35.760 --> 06:37.960] as an MLflow model,[06:37.960 --> 06:41.520] and then you basically drag and drop it into a web UI.[06:41.520 --> 06:43.640] It's gonna be wrapped in containers.[06:43.640 --> 06:45.040] It's gonna be deployed.[06:45.040 --> 06:45.880] It's gonna be,[06:45.880 --> 06:49.680] there will be a monitoring layer in front of it[06:49.680 --> 06:52.760] based on whatever the dataset is you trained it on.[06:52.760 --> 06:55.920] You would automatically calculate different metrics,[06:55.920 --> 06:57.360] different distributional metrics[06:57.360 --> 06:59.240] around your variables that you are using.[06:59.240 --> 07:02.080] And so we were layering this approach[07:02.080 --> 07:06.840] to, so that eventually every incoming request would be,[07:06.840 --> 07:08.160] you would have a nice dashboard.[07:08.160 --> 07:10.040] You could monitor all that stuff.[07:10.040 --> 07:12.600] So stackwise it was actually MLflow.[07:12.600 --> 07:15.480] Specifically MLflow models a lot.[07:15.480 --> 07:17.920] Then it was Java in the backend, Python.[07:17.920 --> 07:19.760] There was a lot of Python,[07:19.760 --> 07:22.040] especially PySpark component as well.[07:23.000 --> 07:25.880] There was a, it's been quite a while actually,[07:25.880 --> 07:29.160] there was a quite some part written in Scala.[07:29.160 --> 07:32.280] Also, because there was a component of this platform[07:32.280 --> 07:34.800] was also a bit of an auto ML approach,[07:34.800 --> 07:36.480] but that died then over time.[07:36.480 --> 07:40.120] And that was also based on PySpark[07:40.120 --> 07:43.280] and vanilla Spark written in Scala.[07:43.280 --> 07:45.560] So we could facilitate the auto ML part.[07:45.560 --> 07:48.600] And then later on we actually added that deployment,[07:48.600 --> 07:51.480] the easy deployment and serving part.[07:51.480 --> 07:55.280] So that was kind of, yeah, a lot of custom build stuff.[07:55.280 --> 07:56.120] Back then, right?[07:56.120 --> 07:59.720] There wasn't that much MLOps tooling out there yet.[07:59.720 --> 08:02.920] So you need to build a lot of that stuff custom.[08:02.920 --> 08:05.280] So it was largely custom built.[08:05.280 --> 08:09.280] Yeah, the MLflow concept is an interesting concept[08:09.280 --> 08:13.880] because they provide this package structure[08:13.880 --> 08:17.520] that at least you have some idea of,[08:17.520 --> 08:19.920] what is gonna be sent into the model[08:19.920 --> 08:22.680] and like there's a format for the model.[08:22.680 --> 08:24.720] And I think that part of MLflow[08:24.720 --> 08:27.520] seems to be a pretty good idea,[08:27.520 --> 08:30.080] which is you're creating a standard where,[08:30.080 --> 08:32.360] you know, if in the case of,[08:32.360 --> 08:34.720] if you're using scikit learn or something,[08:34.720 --> 08:37.960] you don't necessarily want to just throw[08:37.960 --> 08:40.560] like a pickled model somewhere and just say,[08:40.560 --> 08:42.720] okay, you know, let's go.[08:42.720 --> 08:44.760] Yeah, that was also our thinking back then.[08:44.760 --> 08:48.040] So we thought a lot about what would be a,[08:48.040 --> 08:51.720] what would be, what could become the standard actually[08:51.720 --> 08:53.920] for how you package models.[08:53.920 --> 08:56.200] And back then MLflow was one of the little tools[08:56.200 --> 08:58.160] that was already there, already existent.[08:58.160 --> 09:00.360] And of course there was data bricks behind it.[09:00.360 --> 09:02.680] So we also made a bet on that back then and said,[09:02.680 --> 09:04.920] all right, let's follow that packaging standard[09:04.920 --> 09:08.680] and make it the contract how you would as a data scientist,[09:08.680 --> 09:10.800] then how you would need to package it up[09:10.800 --> 09:13.640] and submit it to the platform.[09:13.640 --> 09:16.800] Yeah, it's interesting because the,[09:16.800 --> 09:19.560] one of the, this reminds me of one of the issues[09:19.560 --> 09:21.800] that's happening right now with cloud computing,[09:21.800 --> 09:26.800] where in the cloud AWS has dominated for a long time[09:29.480 --> 09:34.480] and they have 40% market share, I think globally.[09:34.480 --> 09:38.960] And Azure's now gaining and they have some pretty good traction[09:38.960 --> 09:43.120] and then GCP's been down for a bit, you know,[09:43.120 --> 09:45.760] in that maybe the 10% range or something like that.[09:45.760 --> 09:47.760] But what's interesting is that it seems like[09:47.760 --> 09:51.480] in the case of all of the cloud providers,[09:51.480 --> 09:54.360] they haven't necessarily been leading the way[09:54.360 --> 09:57.840] on things like packaging models, right?[09:57.840 --> 10:01.480] Or, you know, they have their own proprietary systems[10:01.480 --> 10:06.480] which have been developed and are continuing to be developed[10:06.640 --> 10:08.920] like Vertex AI in the case of Google,[10:09.760 --> 10:13.160] the SageMaker in the case of Amazon.[10:13.160 --> 10:16.480] But what's interesting is, let's just take SageMaker,[10:16.480 --> 10:20.920] for example, there isn't really like this, you know,[10:20.920 --> 10:25.480] industry wide standard of model packaging[10:25.480 --> 10:28.680] that SageMaker uses, they have their own proprietary stuff[10:28.680 --> 10:31.040] that kind of builds in and Vertex AI[10:31.040 --> 10:32.440] has their own proprietary stuff.[10:32.440 --> 10:34.920] So, you know, I think it is interesting[10:34.920 --> 10:36.960] to see what's gonna happen[10:36.960 --> 10:41.120] because I think your original hypothesis which is,[10:41.120 --> 10:44.960] let's pick, you know, this looks like it's got some traction[10:44.960 --> 10:48.760] and it wasn't necessarily tied directly to a cloud provider[10:48.760 --> 10:51.600] because Databricks can work on anything.[10:51.600 --> 10:53.680] It seems like that in particular,[10:53.680 --> 10:56.800] that's one of the more sticky problems right now[10:56.800 --> 11:01.800] with MLopsis is, you know, who's the leader?[11:02.280 --> 11:05.440] Like, who's developing the right, you know,[11:05.440 --> 11:08.880] kind of a standard for tooling.[11:08.880 --> 11:12.320] And I don't know, maybe that leads into kind of you talking[11:12.320 --> 11:13.760] a little bit about what you're doing currently.[11:13.760 --> 11:15.600] Like, do you have any thoughts about the, you know,[11:15.600 --> 11:18.720] current tooling and what you're doing at your current company[11:18.720 --> 11:20.920] and what's going on with that?[11:20.920 --> 11:21.760] Absolutely.[11:21.760 --> 11:24.200] So at my current organization,[11:24.200 --> 11:26.040] Transaction Monitor Netherlands,[11:26.040 --> 11:27.480] we are fully on AWS.[11:27.480 --> 11:32.000] So we're really almost cloud native AWS.[11:32.000 --> 11:34.840] And so that also means everything we do on the modeling side[11:34.840 --> 11:36.600] really evolves around SageMaker.[11:37.680 --> 11:40.840] So for us, specifically for us as MLops team,[11:40.840 --> 11:44.680] we are building the platform around SageMaker capabilities.[11:45.680 --> 11:48.360] And on that end, at least company internal,[11:48.360 --> 11:52.880] we have a contract how you must actually deploy models.[11:52.880 --> 11:56.200] There is only one way, what we call the golden path,[11:56.200 --> 11:59.800] in that case, this is the streamlined highly automated path[11:59.800 --> 12:01.360] that is supported by the platform.[12:01.360 --> 12:04.360] This is the only way how you can actually deploy models.[12:04.360 --> 12:09.360] And in our case, that is actually a SageMaker pipeline object.[12:09.640 --> 12:12.680] So in our company, we're doing large scale batch processing.[12:12.680 --> 12:15.040] So we're actually not doing anything real time at present.[12:15.040 --> 12:17.040] We are doing post transaction monitoring.[12:17.040 --> 12:20.960] So that means you need to submit essentially DAX, right?[12:20.960 --> 12:23.400] This is what we use for training.[12:23.400 --> 12:25.680] This is what we also deploy eventually.[12:25.680 --> 12:27.720] And this is our internal contract.[12:27.720 --> 12:32.200] You need to provision a SageMaker in your model repository.[12:32.200 --> 12:34.640] You got to have one place,[12:34.640 --> 12:37.840] and there must be a function with a specific name[12:37.840 --> 12:41.440] and that function must return a SageMaker pipeline object.[12:41.440 --> 12:44.920] So this is our internal contract actually.[12:44.920 --> 12:46.600] Yeah, that's interesting.[12:46.600 --> 12:51.200] I mean, and I could see like for, I know many people[12:51.200 --> 12:53.880] that are using SageMaker in production,[12:53.880 --> 12:58.680] and it does seem like where it has some advantages[12:58.680 --> 13:02.360] is that AWS generally does a pretty good job[13:02.360 --> 13:04.240] at building solutions.[13:04.240 --> 13:06.920] And if you just look at the history of services,[13:06.920 --> 13:09.080] the odds are pretty high[13:09.080 --> 13:12.880] that they'll keep getting better, keep improving things.[13:12.880 --> 13:17.080] And it seems like what I'm hearing from people,[13:17.080 --> 13:19.080] and it sounds like maybe with your organization as well,[13:19.080 --> 13:24.080] is that potentially the SDK for SageMaker[13:24.440 --> 13:29.120] is really the win versus some of the UX tools they have[13:29.120 --> 13:32.680] and the interface for Canvas and Studio.[13:32.680 --> 13:36.080] Is that what's happening?[13:36.080 --> 13:38.720] Yeah, so I think, right,[13:38.720 --> 13:41.440] what we try to do is we always try to think about our users.[13:41.440 --> 13:44.880] So how do our users, who are our users?[13:44.880 --> 13:47.000] What capabilities and skills do they have?[13:47.000 --> 13:50.080] And what freedom should they have[13:50.080 --> 13:52.640] and what abilities should they have to develop models?[13:52.640 --> 13:55.440] In our case, we don't really have use cases[13:55.440 --> 13:58.640] for stuff like Canvas because our users[13:58.640 --> 14:02.680] are fairly mature teams that know how to do their,[14:02.680 --> 14:04.320] on the one hand, the data science stuff, of course,[14:04.320 --> 14:06.400] but also the engineering stuff.[14:06.400 --> 14:08.160] So in our case, things like Canvas[14:08.160 --> 14:10.320] do not really play so much role[14:10.320 --> 14:12.960] because obviously due to the high abstraction layer[14:12.960 --> 14:15.640] of more like graphical user interfaces,[14:15.640 --> 14:17.360] drag and drop tooling,[14:17.360 --> 14:20.360] you are also limited in what you can do,[14:20.360 --> 14:22.480] or what you can do easily.[14:22.480 --> 14:26.320] So in our case, really, it is the strength of the flexibility[14:26.320 --> 14:28.320] that the SageMaker SDK gives you.[14:28.320 --> 14:33.040] And in general, the SDK around most AWS services.[14:34.080 --> 14:36.760] But also it comes with challenges, of course.[14:37.720 --> 14:38.960] You give a lot of freedom,[14:38.960 --> 14:43.400] but also you're creating a certain ask,[14:43.400 --> 14:47.320] certain requirements for your model development teams,[14:47.320 --> 14:49.600] which is also why we've also been working[14:49.600 --> 14:52.600] about abstracting further away from the SDK.[14:52.600 --> 14:54.600] So our objective is actually[14:54.600 --> 14:58.760] that you should not be forced to interact with the raw SDK[14:58.760 --> 15:00.600] when you use SageMaker anymore,[15:00.600 --> 15:03.520] but you have a thin layer of abstraction[15:03.520 --> 15:05.480] on top of what you are doing.[15:05.480 --> 15:07.480] That's actually something we are moving towards[15:07.480 --> 15:09.320] more and more as well.[15:09.320 --> 15:11.120] Because yeah, it gives you the flexibility,[15:11.120 --> 15:12.960] but also flexibility comes at a cost,[15:12.960 --> 15:15.080] comes often at the cost of speeds,[15:15.080 --> 15:18.560] specifically when it comes to the 90% default stuff[15:18.560 --> 15:20.720] that you want to do, yeah.[15:20.720 --> 15:24.160] And one of the things that I have as a complaint[15:24.160 --> 15:29.160] against SageMaker is that it only uses virtual machines,[15:30.000 --> 15:35.000] and it does seem like a strange strategy in some sense.[15:35.000 --> 15:40.000] Like for example, I guess if you're doing batch only,[15:40.000 --> 15:42.000] it doesn't matter as much,[15:42.000 --> 15:45.000] which I think is a good strategy actually[15:45.000 --> 15:50.000] to get your batch based predictions very, very strong.[15:50.000 --> 15:53.000] And in that case, maybe the virtual machines[15:53.000 --> 15:56.000] make a little bit less of a complaint.[15:56.000 --> 16:00.000] But in the case of the endpoints with SageMaker,[16:00.000 --> 16:02.000] the fact that you have to spend up[16:02.000 --> 16:04.000] these really expensive virtual machines[16:04.000 --> 16:08.000] and let them run 24 seven to do online prediction,[16:08.000 --> 16:11.000] is that something that your organization evaluated[16:11.000 --> 16:13.000] and decided not to use?[16:13.000 --> 16:15.000] Or like, what are your thoughts behind that?[16:15.000 --> 16:19.000] Yeah, in our case, doing real time[16:19.000 --> 16:22.000] or near real time inference is currently not really relevant[16:22.000 --> 16:25.000] for the simple reason that when you think a bit more[16:25.000 --> 16:28.000] about the money laundering or anti money laundering space,[16:28.000 --> 16:31.000] typically when, right,[16:31.000 --> 16:34.000] all every individual bank must do anti money laundering[16:34.000 --> 16:37.000] and they have armies of people doing that.[16:37.000 --> 16:39.000] But on the other hand,[16:39.000 --> 16:43.000] the time it actually takes from one of their systems,[16:43.000 --> 16:46.000] one of their AML systems actually detecting something[16:46.000 --> 16:49.000] that's unusual that then goes into a review process[16:49.000 --> 16:54.000] until it eventually hits the governmental institution[16:54.000 --> 16:56.000] that then takes care of the cases that have been[16:56.000 --> 16:58.000] at least twice validated that they are indeed,[16:58.000 --> 17:01.000] they look very unusual.[17:01.000 --> 17:04.000] So this takes a while, this can take quite some time,[17:04.000 --> 17:06.000] which is also why it doesn't really matter[17:06.000 --> 17:09.000] whether you ship your prediction within a second[17:09.000 --> 17:13.000] or whether it takes you a week or two weeks.[17:13.000 --> 17:15.000] It doesn't really matter, hence for us,[17:15.000 --> 17:19.000] that problem so far thinking about real time inference[17:19.000 --> 17:21.000] has not been there.[17:21.000 --> 17:25.000] But yeah, indeed, for other use cases,[17:25.000 --> 17:27.000] for also private projects,[17:27.000 --> 17:29.000] we've also been considering SageMaker Endpoints[17:29.000 --> 17:31.000] for a while, but exactly what you said,[17:31.000 --> 17:33.000] the fact that you need to have a very beefy machine[17:33.000 --> 17:35.000] running all the time,[17:35.000 --> 17:39.000] specifically when you have heavy GPU loads, right,[17:39.000 --> 17:43.000] and you're actually paying for that machine running 2047,[17:43.000 --> 17:46.000] although you do have quite fluctuating load.[17:46.000 --> 17:49.000] Yeah, then that definitely becomes quite a consideration[17:49.000 --> 17:51.000] of what you go for.[17:51.000 --> 17:58.000] Yeah, and I actually have been talking to AWS about that,[17:58.000 --> 18:02.000] because one of the issues that I have is that[18:02.000 --> 18:07.000] the AWS platform really pushes serverless,[18:07.000 --> 18:10.000] and then my question for AWS is,[18:10.000 --> 18:13.000] so why aren't you using it?[18:13.000 --> 18:16.000] I mean, if you're pushing serverless for everything,[18:16.000 --> 18:19.000] why is SageMaker nothing serverless?[18:19.000 --> 18:21.000] And so maybe they're going to do that, I don't know.[18:21.000 --> 18:23.000] I don't have any inside information,[18:23.000 --> 18:29.000] but it is interesting to hear you had some similar concerns.[18:29.000 --> 18:32.000] I know that there's two questions here.[18:32.000 --> 18:37.000] One is someone asked about what do you do for data versioning,[18:37.000 --> 18:41.000] and a second one is how do you do event based MLOps?[18:41.000 --> 18:43.000] So maybe kind of following up.[18:43.000 --> 18:46.000] Yeah, what do we do for data versioning?[18:46.000 --> 18:51.000] On the one hand, we're running a data lakehouse,[18:51.000 --> 18:54.000] where after data we get from the financial institutions,[18:54.000 --> 18:57.000] from the banks that runs through massive data pipeline,[18:57.000 --> 19:01.000] also on AWS, we're using glue and step functions actually for that,[19:01.000 --> 19:03.000] and then eventually it ends up modeled to some extent,[19:03.000 --> 19:06.000] sanitized, quality checked in our data lakehouse,[19:06.000 --> 19:10.000] and there we're actually using hoodie on top of S3.[19:10.000 --> 19:13.000] And this is also what we use for versioning,[19:13.000 --> 19:16.000] which we use for time travel and all these things.[19:16.000 --> 19:19.000] So that is hoodie on top of S3,[19:19.000 --> 19:21.000] when then pipelines,[19:21.000 --> 19:24.000] so actually our model pipelines plug in there[19:24.000 --> 19:27.000] and spit out predictions, alerts,[19:27.000 --> 19:29.000] what we call alerts eventually.[19:29.000 --> 19:33.000] That is something that we version based on unique IDs.[19:33.000 --> 19:36.000] So processing IDs, we track pretty much everything,[19:36.000 --> 19:39.000] every line of code that touched,[19:39.000 --> 19:43.000] is related to a specific row in our data.[19:43.000 --> 19:46.000] So we can exactly track back for every single row[19:46.000 --> 19:48.000] in our predictions and in our alerts,[19:48.000 --> 19:50.000] what pipeline ran on it,[19:50.000 --> 19:52.000] which jobs were in that pipeline,[19:52.000 --> 19:56.000] which code exactly was running in each job,[19:56.000 --> 19:58.000] which intermediate results were produced.[19:58.000 --> 20:01.000] So we're basically adding lineage information[20:01.000 --> 20:03.000] to everything we output along that line,[20:03.000 --> 20:05.000] so we can track everything back[20:05.000 --> 20:09.000] using a few tools we've built.[20:09.000 --> 20:12.000] So the tool you mentioned,[20:12.000 --> 20:13.000] I'm not familiar with it.[20:13.000 --> 20:14.000] What is it called again?[20:14.000 --> 20:15.000] It's called hoodie?[20:15.000 --> 20:16.000] Hoodie.[20:16.000 --> 20:17.000] Hoodie.[20:17.000 --> 20:18.000] Oh, what is it?[20:18.000 --> 20:19.000] Maybe you can describe it.[20:19.000 --> 20:22.000] Yeah, hoodie is essentially,[20:22.000 --> 20:29.000] it's quite similar to other tools such as[20:29.000 --> 20:31.000] Databricks, how is it called?[20:31.000 --> 20:32.000] Databricks?[20:32.000 --> 20:33.000] Delta Lake maybe?[20:33.000 --> 20:34.000] Yes, exactly.[20:34.000 --> 20:35.000] Exactly.[20:35.000 --> 20:38.000] It's basically, it's equivalent to Delta Lake,[20:38.000 --> 20:40.000] just back then when we looked into[20:40.000 --> 20:42.000] what are we going to use.[20:42.000 --> 20:44.000] Delta Lake was not open sourced yet.[20:44.000 --> 20:46.000] Databricks open sourced a while ago.[20:46.000 --> 20:47.000] We went for Hoodie.[20:47.000 --> 20:50.000] It essentially, it is a layer on top of,[20:50.000 --> 20:53.000] in our case, S3 that allows you[20:53.000 --> 20:58.000] to more easily keep track of what you,[20:58.000 --> 21:03.000] of the actions you are performing on your data.[21:03.000 --> 21:08.000] So it's essentially very similar to Delta Lake,[21:08.000 --> 21:13.000] just already before an open sourced solution.[21:13.000 --> 21:15.000] Yeah, that's, I didn't know anything about that.[21:15.000 --> 21:16.000] So now I do.[21:16.000 --> 21:19.000] So thanks for letting me know.[21:19.000 --> 21:21.000] I'll have to look into that.[21:21.000 --> 21:27.000] The other, I guess, interesting stack related question is,[21:27.000 --> 21:29.000] what are your thoughts about,[21:29.000 --> 21:32.000] I think there's two areas that I think[21:32.000 --> 21:34.000] are interesting and that are emerging.[21:34.000 --> 21:36.000] Oh, actually there's, there's multiple.[21:36.000 --> 21:37.000] Maybe I'll just bring them all up.[21:37.000 --> 21:39.000] So we'll do one by one.[21:39.000 --> 21:42.000] So these are some emerging areas that I'm, that I'm seeing.[21:42.000 --> 21:49.000] So one is the concept of event driven, you know,[21:49.000 --> 21:54.000] architecture versus, versus maybe like a static architecture.[21:54.000 --> 21:57.000] And so I think obviously you're using step functions.[21:57.000 --> 22:00.000] So you're a fan of, of event driven architecture.[22:00.000 --> 22:04.000] Maybe we start, we'll start with that one is what are your,[22:04.000 --> 22:08.000] what are your thoughts on going more event driven in your organization?[22:08.000 --> 22:09.000] Yeah.[22:09.000 --> 22:13.000] In, in, in our case, essentially everything works event driven.[22:13.000 --> 22:14.000] Right.[22:14.000 --> 22:19.000] So since we on AWS, we're using event bridge or cloud watch events.[22:19.000 --> 22:21.000] I think now it's called everywhere.[22:21.000 --> 22:22.000] Right.[22:22.000 --> 22:24.000] This is how we trigger pretty much everything in our stack.[22:24.000 --> 22:27.000] This is how we trigger our data pipelines when data comes in.[22:27.000 --> 22:32.000] This is how we trigger different, different lambdas that parse our[22:32.000 --> 22:35.000] certain information from your log, store them in different databases.[22:35.000 --> 22:40.000] This is how we also, how we, at some point in the back in the past,[22:40.000 --> 22:44.000] how we also triggered new deployments when new models were approved in[22:44.000 --> 22:46.000] your model registry.[22:46.000 --> 22:50.000] So basically everything we've been doing is, is fully event driven.[22:50.000 --> 22:51.000] Yeah.[22:51.000 --> 22:56.000] So, so I think this is a key thing you bring up here is that I've,[22:56.000 --> 23:00.000] I've talked to many people who don't use AWS, who are, you know,[23:00.000 --> 23:03.000] all alternatively experts at technology.[23:03.000 --> 23:06.000] And one of the things that I've heard some people say is like, oh,[23:06.000 --> 23:13.000] well, AWS is in as fast as X or Y, like Lambda is in as fast as X or Y or,[23:13.000 --> 23:17.000] you know, Kubernetes or, but, but the point you bring up is exactly the[23:17.000 --> 23:24.000] way I think about AWS is that the true advantage of AWS platform is the,[23:24.000 --> 23:29.000] is the tight integration with the services and you can design event[23:29.000 --> 23:31.000] driven workflows.[23:31.000 --> 23:33.000] Would you say that's, that's absolutely.[23:33.000 --> 23:34.000] Yeah.[23:34.000 --> 23:35.000] Yeah.[23:35.000 --> 23:39.000] I think designing event driven workflows on AWS is incredibly easy to do.[23:39.000 --> 23:40.000] Yeah.[23:40.000 --> 23:43.000] And it also comes incredibly natural and that's extremely powerful.[23:43.000 --> 23:44.000] Right.[23:44.000 --> 23:49.000] And simply by, by having an easy way how to trigger lambdas event driven,[23:49.000 --> 23:52.000] you can pretty much, right, pretty much do everything and glue[23:52.000 --> 23:54.000] everything together that you want.[23:54.000 --> 23:56.000] I think that gives you a tremendous flexibility.[23:56.000 --> 23:57.000] Yeah.[23:57.000 --> 24:00.000] So, so I think there's two things that come to mind now.[24:00.000 --> 24:07.000] One is that, that if you are developing an ML ops platform that you[24:07.000 --> 24:09.000] can't ignore Lambda.[24:09.000 --> 24:12.000] So I, because I've had some people tell me, oh, well, we can do this and[24:12.000 --> 24:13.000] this and this better.[24:13.000 --> 24:17.000] It's like, yeah, but if you're going to be on AWS, you have to understand[24:17.000 --> 24:18.000] why people use Lambda.[24:18.000 --> 24:19.000] It isn't speed.[24:19.000 --> 24:24.000] It's, it's the ease of, ease of developing very rich solutions.[24:24.000 --> 24:25.000] Right.[24:25.000 --> 24:26.000] Absolutely.[24:26.000 --> 24:28.000] And then the glue between, between what you are building eventually.[24:28.000 --> 24:33.000] And you can even almost your, the thoughts in your mind turn into Lambda.[24:33.000 --> 24:36.000] You know, like you can be thinking and building code so quickly.[24:36.000 --> 24:37.000] Absolutely.[24:37.000 --> 24:41.000] Everything turns into which event do I need to listen to and then I trigger[24:41.000 --> 24:43.000] a Lambda and that Lambda does this and that.[24:43.000 --> 24:44.000] Yeah.[24:44.000 --> 24:48.000] And the other part about Lambda that's pretty, pretty awesome is that it[24:48.000 --> 24:52.000] hooks into services that have infinite scale.[24:52.000 --> 24:56.000] Like so SQS, like you can't break SQS.[24:56.000 --> 24:59.000] Like there's nothing you can do to ever take SQS down.[24:59.000 --> 25:02.000] It handles unlimited requests in and unlimited requests out.[25:02.000 --> 25:04.000] How many systems are like that?[25:04.000 --> 25:05.000] Yeah.[25:05.000 --> 25:06.000] Yeah, absolutely.[25:06.000 --> 25:07.000] Yeah.[25:07.000 --> 25:12.000] So then this kind of a followup would be that, that maybe data scientists[25:12.000 --> 25:17.000] should learn Lambda and step functions in order to, to get to[25:17.000 --> 25:18.000] MLOps.[25:18.000 --> 25:21.000] I think that's a yes.[25:21.000 --> 25:25.000] If you want to, if you want to put the foot into MLOps and you are on AWS,[25:25.000 --> 25:31.000] then I think there is no way around learning these fundamentals.[25:31.000 --> 25:32.000] Right.[25:32.000 --> 25:35.000] There's no way around learning things like what is a Lambda?[25:35.000 --> 25:39.000] How do I, how do I create a Lambda via Terraform or whatever tool you're[25:39.000 --> 25:40.000] using there?[25:40.000 --> 25:42.000] And how do I hook it up to an event?[25:42.000 --> 25:47.000] And how do I, how do I use the AWS SDK to interact with different[25:47.000 --> 25:48.000] services?[25:48.000 --> 25:49.000] So, right.[25:49.000 --> 25:53.000] I think if you want to take a step into MLOps from, from coming more from[25:53.000 --> 25:57.000] the data science and it's extremely important to familiarize yourself[25:57.000 --> 26:01.000] with how do you, at least the fundamentals, how do you architect[26:01.000 --> 26:03.000] basic solutions on AWS?[26:03.000 --> 26:05.000] How do you glue services together?[26:05.000 --> 26:07.000] How do you make them speak to each other?[26:07.000 --> 26:09.000] So yeah, I think that's quite fundamental.[26:09.000 --> 26:14.000] Ideally, ideally, I think that's what the platform should take away from you[26:14.000 --> 26:16.000] as a, as a pure data scientist.[26:16.000 --> 26:19.000] You don't, should not necessarily have to deal with that stuff.[26:19.000 --> 26:23.000] But if you're interested in, if you want to make that move more towards MLOps,[26:23.000 --> 26:27.000] I think learning about infrastructure and specifically in the context of AWS[26:27.000 --> 26:31.000] about the services and how to use them is really fundamental.[26:31.000 --> 26:32.000] Yeah, it's good.[26:32.000 --> 26:33.000] Because this is automation eventually.[26:33.000 --> 26:37.000] And if you want to automate, if you want to automate your complex processes,[26:37.000 --> 26:39.000] then you need to learn that stuff.[26:39.000 --> 26:41.000] How else are you going to do it?[26:41.000 --> 26:42.000] Yeah, I agree.[26:42.000 --> 26:46.000] I mean, that's really what, what, what Lambda step functions are is their[26:46.000 --> 26:47.000] automation tools.[26:47.000 --> 26:49.000] So that's probably the better way to describe it.[26:49.000 --> 26:52.000] That's a very good point you bring up.[26:52.000 --> 26:57.000] Another technology that I think is an emerging technology is the[26:57.000 --> 26:58.000] managed file system.[26:58.000 --> 27:05.000] And the reason why I think it's interesting is that, so I 20 plus years[27:05.000 --> 27:11.000] ago, I was using file systems in the university setting when I was at[27:11.000 --> 27:14.000] Caltech and then also in film, film industry.[27:14.000 --> 27:22.000] So film has been using managed file servers with parallel processing[27:22.000 --> 27:24.000] farms for a long time.[27:24.000 --> 27:27.000] I don't know how many people know this, but in the film industry,[27:27.000 --> 27:32.000] the, the, the architecture, even from like 2000 was there's a very[27:32.000 --> 27:38.000] expensive file server and then there's let's say 40,000 machines or 40,000[27:38.000 --> 27:39.000] cores.[27:39.000 --> 27:40.000] And that's, that's it.[27:40.000 --> 27:41.000] That's the architecture.[27:41.000 --> 27:46.000] And now what's interesting is I see with data science and machine learning[27:46.000 --> 27:52.000] operations that like that, that could potentially happen in the future is[27:52.000 --> 27:57.000] actually a managed NFS mount point with maybe Kubernetes or something like[27:57.000 --> 27:58.000] that.[27:58.000 --> 28:01.000] Do you see any of that on the horizon?[28:01.000 --> 28:04.000] Oh, that's a good question.[28:04.000 --> 28:08.000] I think for our, for our, what we're currently doing, that's probably a[28:08.000 --> 28:10.000] bit further away.[28:10.000 --> 28:15.000] But in principle, I could very well imagine that in our use case, not,[28:15.000 --> 28:17.000] not quite.[28:17.000 --> 28:20.000] But in principle, definitely.[28:20.000 --> 28:26.000] And then maybe a third, a third emerging thing I'm seeing is what's going[28:26.000 --> 28:29.000] on with open AI and hugging face.[28:29.000 --> 28:34.000] And that has the potential, but maybe to change the game a little bit,[28:34.000 --> 28:38.000] especially with hugging face, I think, although both of them, I mean,[28:38.000 --> 28:43.000] there is that, you know, in the case of pre trained models, here's a[28:43.000 --> 28:48.000] perfect example is that an organization may have, you know, maybe they're[28:48.000 --> 28:53.000] using AWS even for this, they're transcribing videos and they're going[28:53.000 --> 28:56.000] to do something with them, maybe they're going to detect, I don't know,[28:56.000 --> 29:02.000] like, you know, if you recorded customers in your, I'm just brainstorm,[29:02.000 --> 29:05.000] I'm not seeing your company did this, but I'm just creating a hypothetical[29:05.000 --> 29:09.000] situation that they recorded, you know, customer talking and then they,[29:09.000 --> 29:12.000] they transcribe it to text and then run some kind of a, you know,[29:12.000 --> 29:15.000] criminal detection feature or something like that.[29:15.000 --> 29:19.000] Like they could build their own models or they could download the thing[29:19.000 --> 29:23.000] that was released two days ago or a day ago from open AI that transcribes[29:23.000 --> 29:29.000] things, you know, and then, and then turn that transcribe text into[29:29.000 --> 29:34.000] hugging face, some other model that summarizes it and then you could[29:34.000 --> 29:38.000] feed that into a system. So it's, what is, what is your, what are your[29:38.000 --> 29:42.000] thoughts around some of these pre trained models and is your, are you[29:42.000 --> 29:48.000] thinking of in terms of your stack, trying to look into doing fine tuning?[29:48.000 --> 29:53.000] Yeah, so I think pre trained models and especially the way that hugging face,[29:53.000 --> 29:57.000] I think really revolutionized the space in terms of really kind of[29:57.000 --> 30:02.000] platformizing the entire business around or the entire market around[30:02.000 --> 30:07.000] pre trained models. I think that is really quite incredible and I think[30:07.000 --> 30:10.000] really for the ecosystem a changing way how to do things.[30:10.000 --> 30:16.000] And I believe that looking at the, the costs of training large models[30:16.000 --> 30:19.000] and looking at the fact that many organizations are not able to do it[30:19.000 --> 30:23.000] for, because of massive costs or because of lack of data.[30:23.000 --> 30:29.000] I think this is a, this is a clear, makes it very clear how important[30:29.000 --> 30:33.000] such platforms are, how important sharing of pre trained models actually is.[30:33.000 --> 30:37.000] I believe it's a, we are only at the, quite at the beginning actually of that.[30:37.000 --> 30:42.000] And I think we're going to see that nowadays you see it mostly when it[30:42.000 --> 30:47.000] comes to fairly generalized data format, images, potentially videos, text,[30:47.000 --> 30:52.000] speech, these things. But I believe that we're going to see more marketplace[30:52.000 --> 30:57.000] approaches when it comes to pre trained models in a lot more industries[30:57.000 --> 31:01.000] and in a lot more, in a lot more use cases where data is to some degree[31:01.000 --> 31:05.000] standardized. Also when you think about, when you think about banking,[31:05.000 --> 31:10.000] for example, right? When you think about transactions to some extent,[31:10.000 --> 31:14.000] transaction, transaction data always looks the same, kind of at least at[31:14.000 --> 31:17.000] every bank. Of course you might need to do some mapping here and there,[31:17.000 --> 31:22.000] but also there is a lot of power in it. But because simply also thinking[31:22.000 --> 31:28.000] about sharing data is always a difficult thing, especially in Europe.[31:28.000 --> 31:32.000] Sharing data between organizations is incredibly difficult legally.[31:32.000 --> 31:36.000] It's difficult. Sharing models is a different thing, right?[31:36.000 --> 31:40.000] Basically, similar to the concept of federated learning. Sharing models[31:40.000 --> 31:44.000] is significantly easier legally than actually sharing data.[31:44.000 --> 31:48.000] And then applying these models, fine tuning them and so on.[31:48.000 --> 31:52.000] Yeah, I mean, I could just imagine. I really don't know much about[31:52.000 --> 31:56.000] banking transactions, but I would imagine there could be several[31:56.000 --> 32:01.000] kinds of transactions that are very normal. And then there's some[32:01.000 --> 32:06.000] transactions, like if you're making every single second,[32:06.000 --> 32:11.000] you're transferring a lot of money. And it happens just[32:11.000 --> 32:14.000] very quickly. It's like, wait, why are you doing this? Why are you transferring money[32:14.000 --> 32:20.000] constantly? What's going on? Or the huge sum of money only[32:20.000 --> 32:24.000] involves three different points in the network. Over and over again,[32:24.000 --> 32:29.000] just these three points are constantly... And so once you've developed[32:29.000 --> 32:33.000] a model that is anomaly detection, then[32:33.000 --> 32:37.000] yeah, why would you need to develop another one? I mean, somebody already did it.[32:37.000 --> 32:41.000] Exactly. Yes, absolutely, absolutely. And that's[32:41.000 --> 32:45.000] definitely... That's encoded knowledge, encoded information in terms of the model,[32:45.000 --> 32:49.000] which is not personally... Well, abstracts away from[32:49.000 --> 32:53.000] but personally identifiable data. And that's really the power. That is something[32:53.000 --> 32:57.000] that, yeah, as I've said before, you can share significantly easier and you can[32:57.000 --> 33:03.000] apply to your use cases. The kind of related to this in[33:03.000 --> 33:09.000] terms of upcoming technologies is, I think, dealing more with graphs.[33:09.000 --> 33:13.000] And so is that something from a stackwise that your[33:13.000 --> 33:19.000] company's investigated resource can do? Yeah, so when you think about[33:19.000 --> 33:23.000] transactions, bank transactions, right? And bank customers.[33:23.000 --> 33:27.000] So in our case, again, it's a... We only have pseudonymized[33:27.000 --> 33:31.000] transaction data, so actually we cannot see anything, right? We cannot see names, we cannot see[33:31.000 --> 33:35.000] iPads or whatever. We really can't see much. But[33:35.000 --> 33:39.000] you can look at transactions moving between[33:39.000 --> 33:43.000] different entities, between different accounts. You can look at that[33:43.000 --> 33:47.000] as a network, as a graph. And that's also what we very frequently do.[33:47.000 --> 33:51.000] You have your nodes in your network, these are your accounts[33:51.000 --> 33:55.000] or your presence, even. And the actual edges between them,[33:55.000 --> 33:59.000] that's what your transactions are. So you have this[33:59.000 --> 34:03.000] massive graph, actually, that also we as TMNL, as Transaction Montenegro,[34:03.000 --> 34:07.000] are sitting on. We're actually sitting on a massive transaction graph.[34:07.000 --> 34:11.000] So yeah, absolutely. For us, doing analysis on top of[34:11.000 --> 34:15.000] that graph, building models on top of that graph is a quite important[34:15.000 --> 34:19.000] thing. And like I taught a class[34:19.000 --> 34:23.000] a few years ago at Berkeley where we had to[34:23.000 --> 34:27.000] cover graph databases a little bit. And I[34:27.000 --> 34:31.000] really didn't know that much about graph databases, although I did use one actually[34:31.000 --> 34:35.000] at one company I was at. But one of the things I learned in teaching that[34:35.000 --> 34:39.000] class was about the descriptive statistics[34:39.000 --> 34:43.000] of a graph network. And it[34:43.000 --> 34:47.000] is actually pretty interesting, because I think most of the time everyone talks about[34:47.000 --> 34:51.000] median and max min and standard deviation and everything.[34:51.000 --> 34:55.000] But then with a graph, there's things like centrality[34:55.000 --> 34:59.000] and I forget all the terms off the top of my head, but you can see[34:59.000 --> 35:03.000] if there's a node in the network that's[35:03.000 --> 35:07.000] everybody's interacting with. Absolutely. You can identify communities[35:07.000 --> 35:11.000] of people moving around a lot of money all the time. For example,[35:11.000 --> 35:15.000] you can detect different metric features eventually[35:15.000 --> 35:19.000] doing computations on your graph and then plugging in some model.[35:19.000 --> 35:23.000] Often it's feature engineering. You're computing between the centrality scores[35:23.000 --> 35:27.000] across your graph or your different entities. And then[35:27.000 --> 35:31.000] you're building your features actually. And then you're plugging in some[35:31.000 --> 35:35.000] model in the end. If you do classic machine learning, so to say[35:35.000 --> 35:39.000] if you do graph deep learning, of course that's a bit different.[35:39.000 --> 35:43.000] So basically that could for people that are analyzing[35:43.000 --> 35:47.000] essentially networks of people or networks, then[35:47.000 --> 35:51.000] basically a graph database would be step one is[35:51.000 --> 35:55.000] generate the features which could be centrality.[35:55.000 --> 35:59.000] There's a score and then you then go and train[35:59.000 --> 36:03.000] the model based on that descriptive statistic.[36:03.000 --> 36:07.000] Exactly. So one way how you could think about it is[36:07.000 --> 36:11.000] whether we need a graph database or not, that always depends on your specific use case[36:11.000 --> 36:15.000] and what database. We're actually also running[36:15.000 --> 36:19.000] that using Spark. You have graph frames, you have[36:19.000 --> 36:23.000] graph X actually. So really stuff in Spark built for[36:23.000 --> 36:27.000] doing analysis on graphs.[36:27.000 --> 36:31.000] And then what you usually do is exactly what you said. You are trying[36:31.000 --> 36:35.000] to build features based on that graph.[36:35.000 --> 36:39.000] Based on the attributes of the nodes and the attributes on the edges and so on.[36:39.000 --> 36:43.000] And so I guess in terms of graph databases right[36:43.000 --> 36:47.000] now, it sounds like maybe the three[36:47.000 --> 36:51.000] main players maybe are there's Neo4j which[36:51.000 --> 36:55.000] has been around for a long time. There's I guess Spark[36:55.000 --> 36:59.000] and then there's also, I forgot what the one is called for AWS[36:59.000 --> 37:03.000] is it? Neptune, that's Neptune.[37:03.000 --> 37:07.000] Have you played with all three of those and did you[37:07.000 --> 37:11.000] like Neptune? Neptune was something we, Spark of course we actually currently[37:11.000 --> 37:15.000] using for exactly that. Also because it allows us to do[37:15.000 --> 37:19.000] to keep our stack fairly homogeneous. We did[37:19.000 --> 37:23.000] also PUC in Neptune a while ago already[37:23.000 --> 37:27.000] and well Neptune you definitely have essentially two ways[37:27.000 --> 37:31.000] how to query Neptune either using Gremlin or SparkQL.[37:31.000 --> 37:35.000] So that means the people, your data science[37:35.000 --> 37:39.000] need to get familiar with that which then is already one bit of a hurdle[37:39.000 --> 37:43.000] because usually data scientists are not familiar with either.[37:43.000 --> 37:47.000] But also what we found with Neptune[37:47.000 --> 37:51.000] is also that it's not necessarily built for[37:51.000 --> 37:55.000] as an analytics graph database. It's not necessarily made for[37:55.000 --> 37:59.000] that. And that then become, then it's sometimes, at least[37:59.000 --> 38:03.000] for us, it has become quite complicated to handle different performance considerations[38:03.000 --> 38:07.000] when you actually do fairly complex queries across that graph.[38:07.000 --> 38:11.000] Yeah, so you're bringing up like a point which[38:11.000 --> 38:15.000] happens a lot in my experience with[38:15.000 --> 38:19.000] technology is that sometimes[38:19.000 --> 38:23.000] the purity of the solution becomes the problem[38:23.000 --> 38:27.000] where even though Spark isn't necessarily[38:27.000 --> 38:31.000] designed to be a graph database system, the fact is[38:31.000 --> 38:35.000] people in your company are already using it. So[38:35.000 --> 38:39.000] if you just turn on that feature now you can use it and it's not like[38:39.000 --> 38:43.000] this huge technical undertaking and retraining effort.[38:43.000 --> 38:47.000] So even if it's not as good, if it works, then that's probably[38:47.000 --> 38:51.000] the solution your company will use versus I agree with you like a lot of times[38:51.000 --> 38:55.000] even if a solution like Neo4j is a pretty good example of[38:55.000 --> 38:59.000] it's an interesting product but[38:59.000 --> 39:03.000] you already have all these other products like do you really want to introduce yet[39:03.000 --> 39:07.000] another product into your stack. Yeah, because eventually[39:07.000 --> 39:11.000] it all comes with an overhead of course introducing it. That is one thing[39:11.000 --> 39:15.000] it requires someone to maintain it even if it's a[39:15.000 --> 39:19.000] managed service. Somebody needs to actually own it and look after it[39:19.000 --> 39:23.000] and then as you said you need to retrain people to also use it effectively.[39:23.000 --> 39:27.000] So it comes at significant cost and that is really[39:27.000 --> 39:31.000] something that I believe should be quite critically[39:31.000 --> 39:35.000] assessed. What is really the game you have? How far can you go with[39:35.000 --> 39:39.000] your current tooling and then eventually make[39:39.000 --> 39:43.000] that decision. At least personally I'm really[39:43.000 --> 39:47.000] not a fan of thinking tooling first[39:47.000 --> 39:51.000] but personally I really believe in looking at your organization, looking at the people[39:51.000 --> 39:55.000] what skills are there, looking at how effective[39:55.000 --> 39:59.000] are these people actually performing certain activities and processes[39:59.000 --> 40:03.000] and then carefully thinking about what really makes sense[40:03.000 --> 40:07.000] because it's one thing but people need to[40:07.000 --> 40:11.000] adopt and use the tooling and eventually it should really speed them up and improve[40:11.000 --> 40:15.000] how they develop. Yeah, I think it's very[40:15.000 --> 40:19.000] that's great advice that it's hard to understand how good of advice it is[40:19.000 --> 40:23.000] because it takes experience getting burned[40:23.000 --> 40:27.000] creating new technology. I've[40:27.000 --> 40:31.000] had experiences before where[40:31.000 --> 40:35.000] one of the mistakes I've made was putting too many different technologies in an organization[40:35.000 --> 40:39.000] and the problem is once you get enough complexity[40:39.000 --> 40:43.000] it can really explode and then[40:43.000 --> 40:47.000] this is the part that really gets scary is that[40:47.000 --> 40:51.000] let's take Spark for example. How hard is it to hire somebody that knows Spark? Pretty easy[40:51.000 --> 40:55.000] how hard is it going to be to hire somebody that knows[40:55.000 --> 40:59.000] Spark and then hire another person that knows the gremlin query[40:59.000 --> 41:03.000] language for Neptune, then hire another person that knows Kubernetes[41:03.000 --> 41:07.000] then tire another, after a while if you have so many different kinds of tools[41:07.000 --> 41:11.000] you have to hire so many different kinds of people that all[41:11.000 --> 41:15.000] productivity goes to a stop. So it's the hiring as well[41:15.000 --> 41:19.000] Absolutely, I mean it's virtually impossible[41:19.000 --> 41:23.000] to find someone who is really well versed with gremlin for example[41:23.000 --> 41:27.000] it's incredibly hard and I think tech hiring is hard[41:27.000 --> 41:31.000] by itself already[41:31.000 --> 41:35.000] so you really need to think about what can I hire for as well[41:35.000 --> 41:39.000] what expertise can I realistically build up?[41:39.000 --> 41:43.000] So that's why I think AWS[41:43.000 --> 41:47.000] even with some of the limitations about the ML platform[41:47.000 --> 41:51.000] the advantages of using AWS is that[41:51.000 --> 41:55.000] you have a huge audience of people to hire from and then the same thing like[41:55.000 --> 41:59.000] Spark, there's a lot of things I don't like about Spark but a lot of people[41:59.000 --> 42:03.000] use Spark and so if you use AWS and you use Spark[42:03.000 --> 42:07.000] let's say those two which you are then you're going to have a much easier time[42:07.000 --> 42:11.000] hiring people, you're going to have a much easier time training people[42:11.000 --> 42:15.000] there's tons of documentation about it so I think a lot of people[42:15.000 --> 42:19.000] are very wise that you're thinking that way but a lot of people don't think about that[42:19.000 --> 42:23.000] they're like oh I've got to use the latest, greatest stuff and this and this and this[42:23.000 --> 42:27.000] and then their company starts to get into trouble because they can't hire[42:27.000 --> 42:31.000] people, they can't maintain systems and then productivity starts to[42:31.000 --> 42:35.000] to degrees. Also something[42:35.000 --> 42:39.000] not to ignore is the cognitive load you put on a team[42:39.000 --> 42:43.000] that needs to manage a broad range of very different[42:43.000 --> 42:47.000] tools or services. It also puts incredible[42:47.000 --> 42:51.000] cognitive load on that team and you suddenly also need an incredible breadth[42:51.000 --> 42:55.000] of expertise in that team and that means you're also going[42:55.000 --> 42:59.000] to create single points of failures if you don't really[42:59.000 --> 43:03.000] scale up your team.[43:03.000 --> 43:07.000] It's something to really, I think when you go for[43:07.000 --> 43:11.000] new tooling you should really look at it from a holistic perspective[43:11.000 --> 43:15.000] not only about this is the latest and greatest.[43:15.000 --> 43:19.000] In terms of Europe versus[43:19.000 --> 43:23.000] US, have you spent much time in the US at all?[43:23.000 --> 43:27.000] Not at all actually, flying to the US Monday but no, not at all.[43:27.000 --> 43:31.000] That also would be kind of an interesting[43:31.000 --> 43:35.000] comparison in that the culture of the United States[43:35.000 --> 43:39.000] is really this culture of[43:39.000 --> 43:43.000] I would say more like survival of the fittest or you work[43:43.000 --> 43:47.000] seven days a week and you're constantly like you don't go on vacation[43:47.000 --> 43:51.000] and you're proud of it and I think it's not[43:51.000 --> 43:55.000] a good culture. I'm not saying that's a good thing, I think it's a bad[43:55.000 --> 43:59.000] thing and that a lot of times the critique people have[43:59.000 --> 44:03.000] about Europe is like oh will people take vacation all the time and all this[44:03.000 --> 44:07.000] and as someone who has spent time in both I would say[44:07.000 --> 44:11.000] yes that's a better approach. A better approach is that people[44:11.000 --> 44:15.000] should feel relaxed because when[44:15.000 --> 44:19.000] especially the kind of work you do in MLOPs[44:19.000 --> 44:23.000] is that you need people to feel comfortable and happy[44:23.000 --> 44:27.000] and more the question[44:27.000 --> 44:31.000] what I was going to is that[44:31.000 --> 44:35.000] I wonder if there is a more productive culture[44:35.000 --> 44:39.000] for MLOPs in Europe[44:39.000 --> 44:43.000] versus the US in terms of maintaining[44:43.000 --> 44:47.000] systems and building software where the US[44:47.000 --> 44:51.000] what it's really been good at I guess is kind of coming up with new[44:51.000 --> 44:55.000] ideas and there's lots of new services that get generated but[44:55.000 --> 44:59.000] the quality and longevity[44:59.000 --> 45:03.000] is not necessarily the same where I could see[45:03.000 --> 45:07.000] in the stuff we just talked about which is if you're trying to build a team[45:07.000 --> 45:11.000] where there's low turnover[45:11.000 --> 45:15.000] you have very high quality output[45:15.000 --> 45:19.000] it seems like that maybe organizations[45:19.000 --> 45:23.000] could learn from the European approach to building[45:23.000 --> 45:27.000] and maintaining systems for MLOPs.[45:27.000 --> 45:31.000] I think there's definitely some truth in it especially when you look at the median[45:31.000 --> 45:35.000] tenure of a tech person in an organization[45:35.000 --> 45:39.000] I think that is actually still significantly lower in the US[45:39.000 --> 45:43.000] I'm not sure I think in the Bay Area somewhere around one year or two months or something like that[45:43.000 --> 45:47.000] compared to Europe I believe[45:47.000 --> 45:51.000] still fairly low. Here of course in tech people also like to switch companies more often[45:51.000 --> 45:55.000] but I would say average is still more around[45:55.000 --> 45:59.000] two years something around that staying with the same company[45:59.000 --> 46:03.000] also in tech which I think is a bit longer[46:03.000 --> 46:07.000] than you would typically have it in the US.[46:07.000 --> 46:11.000] I think from my perspective where I've also built up most of the[46:11.000 --> 46:15.000] current team I think it's[46:15.000 --> 46:19.000] super important to hire good people[46:19.000 --> 46:23.000] and people that fit to the team fit to the company culture wise[46:23.000 --> 46:27.000] but also give them[46:27.000 --> 46:31.000] let them not be in a sprint all the time[46:31.000 --> 46:35.000] it's about having a sustainable way of working in my opinion[46:35.000 --> 46:39.000] and that sustainable way means you should definitely take your vacation[46:39.000 --> 46:43.000] and I think usually in Europe we have quite generous[46:43.000 --> 46:47.000] even by law vacation I mean in Netherlands by law you get 20 days a year[46:47.000 --> 46:51.000] but most companies give you 25 many IT companies[46:51.000 --> 46:55.000] 30 per year so that's quite nice[46:55.000 --> 46:59.000] but I do take that so culture wise it's really everyone[46:59.000 --> 47:03.000] likes to take vacations whether that's sea level or whether that's an engineer on a team[47:03.000 --> 47:07.000] and that's in many companies that's also really encouraged[47:07.000 --> 47:11.000] to have a healthy work life balance[47:11.000 --> 47:15.000] and of course it's not only about vacations also but growth opportunities[47:15.000 --> 47:19.000] letting people explore develop themselves[47:19.000 --> 47:23.000] and not always pushing on max performance[47:23.000 --> 47:27.000] so really at least I always see like a partnership[47:27.000 --> 47:31.000] the organization wants to get something from an[47:31.000 --> 47:35.000] employee but the employee should also be encouraged and developed[47:35.000 --> 47:39.000] in that organization a

W najnowszym odcinku gościem Łukasza Kobylińskiego był Jakub Czakon - CMO w neptune.ai STRESZCZENIE ODCINKA: 1. Przedstawienie Gościa 2. Czym zajmuje się neptune.ai? 3. Czym się wyróżniacie na tle konkurencji? 4. Czym różni się MLOps od DevOps? 5. Jak częste jest w firmach zajmujących się AI w Polsce stosowanie ML Ops? 6. Jak to wygląda na świecie? Czy można tu obserwować jakieś trendy? 7. Jesteście startupem - czy trudno było pozyskać finansowanie na rozwój? 8. Jakie są różnice w środowisku startupowym w Europie, USA i Izraelu? 9. Jak buduje się produkt dla programistów? Czy jest to trudniejsze, niż w przypadku produktów dla szerokiego grona użytkowników? 10. Jak radzić sobie z niezadowolonymi użytkownikami - wytykającymi błędy lub oczekującymi wciąż nowych funkcji? 11. Jakie kanały komunikacji najlepiej sprawdzają się w kontekście programistów? Na co warto szczególnie zwracać uwagę, formułując przekazy dla programistów?

A collection of highlights powered by mathematics, statistics, and a little bit of R magic: Mapping wind data with R, calculating the expected statistic in football, and how the vetiver package fits in an ML-Ops production flow using Docker and Plumber. Episode Links This week's curator: Jon Calder (@jonmcalder (https://twitter.com/jonmcalder)) Mapping wind data with R (https://milospopovic.net/mapping-wind-data-in-r/) Calculating and comparing expected points from different expected goals sources (soccer) (https://tonyelhabr.rbind.io/post/epl-xpts-simulation-1/) Use Docker to deploy a model for #TidyTuesday LEGO sets (https://juliasilge.com/blog/lego-sets/) Entire issue available at rweekly.org/2022-W37 (https://rweekly.org/2022-W37.html) Supplement Resources Julia Silge's YouTube Channel: https://www.youtube.com/c/JuliaSilge

The approach: in contrast to Auto ML, the approach is more resource efficient. Tangent Works has promised a new modeling approach designed specifically for time series. In the end, only one model is created for the use case, the Belgians assure, and at high speed. For many applications, it's just seconds, they say. The secret ingredient is called Information Geometry. This provides the speed and makes automation possible. The podcast is growing and we want to keep growing. That's why our German-language podcast is now available in English. We are happy about new listeners. We thank our new partner [Siemens](https://new.siemens.com/global/en/products/automation/topic-areas/artificial-intelligence-in-industry.html) [More AI in the industry? (mostly German) ](https://kipodcast.de/podcast-archiv) [Or our book AI in Industry: (German) ](https://www.hanser-fachbuch.de/buch/KI+in+industry/9783446463455) [Contact our guest ](https://www.linkedin.com/in/philipwauters/)

Today, we are joined by Piotr Niedźwiedź, Founder and CEO of Neptune.ai. Piotr discusses common MLOps activities by data science teams and how they can take advantage of Neptune.ai for better experiment tracking and efficiency. Listen for more!

Today I had the pleasure of interviewing the incredible  @Miki Bazeley - The MLOps Engineer for the second time!  Watch the first interview here: https://www.youtube.com/watch?v=Ii2Qo5pwWho&ab_channel=Ken%27sNearestNeighborsPodcast . We dive into everything ML ops in this episode. What is it, what is the future of it, and how can you break into it. We also touch on cultural differences and their impacts on the workplace. 

In this podcast4 in season3, we will be having a detailed conversation with a start-up founder from the ML Ops platform Katonic AI as an example. The conversation focuses mainly around how enterprises benefit while leveraging the ML Ops platforms in the context of infusing intelligence into their business processes or solving the business needs. We talk about the various stages involved in an AI project flow and the current need of ML Ops on the lines of dev Ops! This is a good addition to the recently written article on the same topic by me at the medium blogs and this podcast also serves as a transition from my earlier podcast #23. --- Send in a voice message: https://anchor.fm/raghu-banda/message

Online wagering is one of the most sophisticated and complex fields for data and analytics. This week on the Data Futurology podcast, Mia O'Dell, the GM of Data Science at Sportsbet, kicks thing off by explaining how the company brings together three separate data teams, across three lines of business, to achieve meaningful and collaborative data outcomes. Sportsbet is also growing its data practice and looking to nearly double its team sizes by the end of the year. O'Dell – who was also responsible for scaling the data practice in a previous organisation – also shares some insights about how to approach data scaling. There's no “one size fits all” approach, she says. Success depends on being able to work with the teams to come up with a strong and compelling vision. Finally, O'Dell also shares her concept of “machine learning offense” and “machine learning defence” as a way to help articulate the value of ML Ops at a time where non-data executives within enterprises are still struggling to understand the breakdown and operation of ML Ops teams. It's also important to understand where and when ML Ops becomes important to a business, O'Dell adds, saying that a lot of organisations make the mistake of going all-out when they're just at the start of the journey, where the value of ML Ops will be marginal and difficult to articulate. “If your first machine learning model is something that's extremely critical to the success of the business, of course you want to over invest in its reliance,” she says. “But for something that isn't necessarily core to the business, ML Ops can result in putting far too much effort on the defensive side, and not enough yet on the offensive side.” Tune in for in-depth insights into this, and more, with Mia O'Dell. Enjoy the show! Thank you to you our sponsor, Talent Insights Group! Join us for one of our upcoming events: https://www.datafuturology.com/events Join our Slack Community: https://hubs.li/Q01gKNBn0 Read the full podcast episode summary here. --- Send in a voice message: https://anchor.fm/datafuturology/message

In episode number seven, we meet Jacopo Tagliabue and discuss behavioral testing for recommender systems and experiences from ecommerce. Before Jacopo became the director of artificial intelligence at Coveo, he had founded tooso, which was later acquired by Coveo. Jacopo holds a PhD in cognitive intelligence and made many contributions to conferences like SIGIR, WWW, or RecSys. In addition, he serves as adjunct professor at NYU.In this episode we introduce behavioral testing for recommender systems and the corresponding framework RecList that was created by Jacopo and his co-authors. Behavioral testing goes beyond pure retrieval accuracy metrics and tries to uncover unintended behavior of recommender models. RecList is an adaption of CheckList that applies behavioral testing to NLP and which was proposed by Microsoft some time ago. RecList comes with an open-source framework with ready set datasets for different recommender use-cases like similar, sequence-based and complementary item recommendations. Furthermore, it offers some sample tests to make it easier for newcomers to get started with behavioral testing. We also briefly touch on the upcoming CIKM data challenge that is going to focus on the evaluation of recommender systems.In the end of this episode Jacopo also shares his insights from years of building and using diverse ML Ops tools and talk about what he refers to as the "post-modern stack".Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Links from the Episode: Jacopo Tagliabue on LinkedIn GitHub: RecList CIKM RecEval Analyticup 2022 (sign up!) GitHub: You Don't Need a Bigger Boat - end-to-end (Metaflow-based) implementation of an intent prediction (and session recommendation) flow Coveo SIGIR eCOM 2021 Data Challenge Dataset Blogposts: The Post-Modern Stack - Joining the modern data stack with the modern ML stack TensorFlow Recommenders TorchRec NVIDIA Merlin Recommenders (by Microsoft) recbole Papers: Chia et al. (2022): Beyond NDCG: behavioral testing of recommender systems with RecList Ribeiro et al. (2020): Beyond Accuracy: Behavioral Testing of NLP models with CheckList Bianchi et al. (2020): Fantastic Embeddings and How to Align Them: Zero-Shot Inference in a Multi-Shop Scenario General Links: Follow me on Twitter: https://twitter.com/LivesInAnalogia Send me your comments, questions and suggestions to marcel@recsperts.com Podcast Website: https://www.recsperts.com/

Machine Learning Operations (MLOps or ML Ops) is a set of practices that aims to deploy and maintain machine learning models in production reliably and efficiently, as defined in various publications.  In this podcast we take on the topic of  MLOPs. What is it and is it like DevOps for AI?  Turns out it's broader than you might think including everything monitoring to governance and explainability.  Adewumni shares why it's both necessary and exciting. For her 30 second recommendation, Ade shared the Cloudera Fast Forward Labs blog which can be found here. She also mentioned a report by the Algorithmic Justice League on bug bounties for algorithmic harms which can be found here. You can follow Ade on Twitter  @Adewunmi  and  @FastForwardLabs . You can also find her on Medium medium.com/@adeadewunmi and LinkedIn here. You can follow me on Twitter @MaribelLopez and on LinkedIn here. 

This week, we invited Kelvin Tham, an MLOps Data Program Manager at ViSenze - AI for Visual Commerce. Kelvin has a wide range experience across ML Ops, data analytics, and business process improvement. He is currently working on design, development and shipping of ML Ops model management. In this episode, he shared about how is it like to be working at an AI startup company and his war stories of wearing multiple hats at one go. He also talked about the differences between being a program manager and a developer, pros and cons of each role, and shed a light on what to look out for when you are exploring your future career options. Have a listen. You can connect with Kelvin here: https://www.linkedin.com/in/kelvinthamkh/

MLOps Coffee Sessions #91 with Joseph Haaga, The Shipyard: Lessons Learned While Building an ML Platform / Automating Adherence. // Abstract Joseph Haaga and the Interos team walk us through their design decisions in building an internal data platform. Joseph talks about why their use case wasn't a fit for off the self solutions, what their internal tool snitch does, and how they use git as a model registry. Shipyard blogpost series: https://medium.com/interos-engineering. // Bio Joseph leads the ML Platform team at Interos, the operational resilience company. He was introduced to ML Ops while working as a Senior Data Engineer and has spent the past year building a platform for experimentation and serving. He lives in Washington, DC, with his dog Cheese. // MLOps Jobs board https://mlops.pallet.xyz/jobs // Related Links Website: https://joehaaga.xyz Medium: https://medium.com/interos-engineering Shipyard blogpost series: https://medium.com/interos-engineering --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Vishnu on LinkedIn: https://www.linkedin.com/in/vrachakonda/ Connect with Joseph on LinkedIn: https://www.linkedin.com/in/joseph-haaga/ Timestamps: [00:00] Introduction to Joseph Haaga [02:07] Please subscribe, follow, like, rate, review our Spotify and Youtube channels [02:31] New! Best of Slack Weekly Newsletter [03:03] Interos [04:33] Global supply chain [05:45] Machine Learning use cases of Interos [06:17] Forecasting and optimization of routes [07:14] Build, buy, open-source decision making [10:06] Experiences with Kubeflow [11:05] Creating standards and rules when creating the platform [13:29] Snitches [14:10] Inter-team discussions when processes fall apart [16:56] Examples of the development process on the feedback of ML engineers and data scientists [20:35] Preserving flexibility when introducing new models and formats [21:37] Organizational structure of Interos [23:40] Surface area for product [24:46] Use of Git Ops to manage boarding pass [28:04] Cultural emphasis [30:02] Naming conventions [32:28] Benefit of a clean slate [33:16] One-size-fits-all choice [37:34] Wrap up

Today I had the pleasure of interviewing Demetrios Brinkmann. Demetrios is one of the main organizers of the MLOps Community and currently resides in a small town outside Frankfurt, Germany. He is an avid traveler who taught English as a second language to see the world and learn about new cultures. Brinkmann fell into the Machine Learning Operations world, and since, has interviewed the leading names around MLOps, Data Science, and Machine Learning. Since diving into the nitty-gritty of ML Operations he felt a strong calling to explore the ethical issues surrounding AI/ML. He loves how communities tick and their inner workings so much so it has become one of his passions. When he is not conducting interviews you can find him making stone sackings with his daughter in the woods or playing the ukulele by the campfire. In this episode we learn about how Demetrios was able to grow an incredible community and some of the growing pains he faced when scaling, we also learn about how he avoids burnout and his love for ice baths. 

On today's AI Today podcast hosts Kathleen Walch and Ron Schmelzer are going to do their annual AI Market Forecast & Trends. They will spend some time reflecting on what they are seeing in the market and where they forecast the AI markets will go in 2022. They will talk about the Data Labeling, Data Engineering & Preparation, ML Platforms, ML Ops, and RPA markets. Continue reading AI Today Podcast: The State of AI (and AI Today) heading into 2022 at Cognilytica.

About TimTim's tech career spans over 20 years through various sectors. Tim's initial journey into tech started as a US Marine. Later, he left government contracting for the private sector, working both in large corporate environments and in small startups. While working in the private sector, he honed his skills in systems administration and operations for large Unix-based datastores. Today, Tim leverages his years in operations, DevOps, and Site Reliability Engineering to advise and consult with clients in his current role. Tim is also a father of five children, as well as a competitive Brazilian Jiu-Jitsu practitioner. Currently, he is the reigning American National and 3-time Pan American Brazilian Jiu-Jitsu champion in his division.TranscriptCorey: Hello, and welcome to Screaming in the Cloud with your host, Chief cloud economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored in part by our friends at Vultr. Spelled V-U-L-T-R because they're all about helping save money, including on things like, you know, vowels. So, what they do is they are a cloud provider that provides surprisingly high performance cloud compute at a price that—while sure they claim its better than AWS pricing—and when they say that they mean it is less money. Sure, I don't dispute that but what I find interesting is that it's predictable. They tell you in advance on a monthly basis what it's going to going to cost. They have a bunch of advanced networking features. They have nineteen global locations and scale things elastically. Not to be confused with openly, because apparently elastic and open can mean the same thing sometimes. They have had over a million users. Deployments take less that sixty seconds across twelve pre-selected operating systems. Or, if you're one of those nutters like me, you can bring your own ISO and install basically any operating system you want. Starting with pricing as low as $2.50 a month for Vultr cloud compute they have plans for developers and businesses of all sizes, except maybe Amazon, who stubbornly insists on having something to scale all on their own. Try Vultr today for free by visiting: vultr.com/screaming, and you'll receive a $100 in credit. Thats v-u-l-t-r.com slash screaming.Corey: This episode is sponsored in part by something new. Cloud Academy is a training platform built on two primary goals. Having the highest quality content in tech and cloud skills, and building a good community the is rich and full of IT and engineering professionals. You wouldn't think those things go together, but sometimes they do. Its both useful for individuals and large enterprises, but here's what makes it new. I don't use that term lightly. Cloud Academy invites you to showcase just how good your AWS skills are. For the next four weeks you'll have a chance to prove yourself. Compete in four unique lab challenges, where they'll be awarding more than $2000 in cash and prizes. I'm not kidding, first place is a thousand bucks. Pre-register for the first challenge now, one that I picked out myself on Amazon SNS image resizing, by visiting cloudacademy.com/corey. C-O-R-E-Y. That's cloudacademy.com/corey. We're gonna have some fun with this one!Corey: Welcome to Screaming in the Cloud. I am Cloud Economist Corey Quinn joined by Principal Cloud Economist here at The Duckbill Group Tim Banks. Tim, how are you?Tim: I'm doing great, Corey. How about yourself?Corey: I am tickled pink that we are able to record this not for the usual reasons you would expect, but because of the glorious pun in calling this our Banksgiving episode. I have a hard and fast rule of, I don't play pun games or make jokes about people's names because that can be an incredibly offensive thing. “And oh, you're making jokes about my name? I've never heard that one before.” It's not that I can't do it—I play games with language all the time—but it makes people feel crappy. So, when you suggested this out of the blue, it was yes, we're doing it. But I want to be clear, I did not inflict this on you. This is your own choice; arguably a poor one. We're going to find out.Tim: 1000% my idea.Corey: So, this is your show. It's a holiday week. So, what do you want to do with our Banksgiving episode?Tim: I want to give thanks for the folks who don't normally get acknowledged through the year. Like you know, we do a lot of thanking the rock stars, we do a lot of thanking the big names, right, we also do a lot of, you know, some snarky jabs at some folks. Deservingly—not folks, but groups and stuff like that; some folks deserve it, and we won't be giving them thanks—but some orgs and some groups and stuff like that. And I do think with that all said, we should acknowledge and thank the folks that we normally don't get to, folks who've done some great contributions this year, folks who have helped us, helped the industry, and help services that go unsung, I think a great one that you brought up, it's not the engineers, right? It's the people that make sure we get paid. Because I don't work for charity. And I don't know about you, Corey. I haven't seen the books yet, but I'm pretty sure none of us here do and so how do we get paid? Like I don't know.Corey: Oh, sure you have. We had a show on a somewhat simplified P&L during the all hands meeting because, you know, transparency matters. But you're right, those are numbers there and none of that is what we could have charged but didn't because we decided to do more volunteer work for AWS. If we were going to go down that path, we would just be Community Heroes and be done with it.Tim: That's true. But you know, it's like, I do my thing and then, you know, I get a paycheck every now and then. And so, as far as I know, I think most of that happens because of Dan.Corey: Dan is a perfect example. He's been a guest on this show, I don't know it has as aired at the time that this goes out because I don't have to think about that, which is kind of the point. Dan's our CFO and makes sure that a lot of the financial trains keep running on time. But let's also be clear, the fact that I can make predictions about what the business is going to be doing by a metric other than how much cash is in the bank account at this very moment really freed up some opportunity for us. It turned into adult supervision for folks who, when I started this place and then Mike joined, and it was very much not an area that either one of us was super familiar with. Which is odd given what we do here, but we learned quickly.The understanding not just how these things work—which we had an academic understanding of—but why it mattered and how that applies to real life. Finance is one of those great organizations that doesn't get a lot of attention or respect outside of finance itself. Because it's, “Oh, well they just control the money. How hard could it be?” Really, really hard.Tim: It really is. And when we dig into some of these things and some of the math that goes and some of what the concerns are that, you know, a lot of engineers don't really have a good grasp on, and it's eye opening to understand some of the concerns. At least some of the concerns at least from an engineering aspect. And I really don't give much consideration day to day about the things that go on behind the scenes to make sure that I get paid.But you look at this throughout the industry, like, how many of the folks that we work with, how many folks out there doing this great work for the industry, do they know who their payroll person is? Do they know who their accountant team is? Do they know who their CFO or the other people out there that are doing the work and making sure the lights stay on, that people get paid and all the other things that happen, right? You know, people take that for granted. And it's a huge work and those people really don't get the appreciation that I think they deserve. And I think it's about time we did that.Corey: It's often surprising to me how many people that I encounter, once they learn that there are 12 employees here, automatically assume that it's you, me, and maybe occasionally Mike doing all the work, and the other nine people just sort of sit here and clap when I tell a funny joke, and… well, yes, that is, of course, a job duty, but that's not the entire purpose of why people are here.Natalie in marketing is a great example. “Well, Corey, I thought you did the marketing. You go and post on Twitter and that's where business comes from.” Well, kind of. But let's be clear, when I do that, and people go to the website to figure out what the hell I'm talking about.Well, that website has words on it. I didn't put those words on that site. It directs people to contact us forms, and there are automations behind that that make sure they go to the proper place because back before I started this place and I was independent, people would email me asking for help with their bill and I would just never respond to them. It's the baseline adult supervision level of competence that I keep aspiring to. We have a sales team that does fantastic work.And that often is one of those things that'll get engineering hackles up, but they're not out there cold-calling people to bug them about AWS bills. It's when someone reaches out saying we have a problem with our AWS spend, can you help us? The answer is invariably, “Let's talk about that.” It's a consultative discussion about why do you care about the bill, what does success look like, how do you know this will be a success, et cetera, et cetera, et cetera, that make sure that we're aimed at the right part of the problem. That's incredibly challenging work and I am grateful beyond words, I don't have to be involved with the day-in, day-out of any of those things.Tim: I think even beyond just that handling, like, the contracts and the NDAs, and the various assets that have to be exchanged just to get us virtually on site, I've [unintelligible 00:06:46] a couple of these things, I'm glad it's not my job. It is, for me, overwhelmingly difficult for me to really get a grasp and all that kind of stuff. And I am grateful that we do have a staff that does that. You've heard me, you see me, you know, kind of like, sales need to do better, and a lot of times I do but I do want to make sure we are appreciating them for the work that they do to make sure that we have work to do. Their contribution cannot be underestimated.Corey: And I think that's something that we could all be a little more thankful for in the industry. And I see this on Twitter sometimes, and it's probably my least favorite genre of tweet, where someone will wind up screenshotting some naive recruiter outreach to them, and just start basically putting the poor person on blast. I assure you, I occasionally get notices like that. The most recent example of that was, I got an email to my work email address from an associate account exec at AWS asking what projects I have going on, how my work in the cloud is going, and I can talk to them about if I want to help with cost optimization of my AWS spend and the rest. And at first, it's one of those, I could ruin this person's entire month, but I don't want to be that person.And I did a little LinkedIn stalking and it turns out, this looks like this person's first job that they've been in for three months. And I've worked in jobs like that very early in my career; it is a numbers game. When you're trying to reach out to 1000 people a month or whatnot, you aren't sitting there googling what every one of them is, does, et cetera. It's something that I've learned, that is annoying, sure. But I'm in an incredibly privileged position here and dunking on someone who's doing what they are told by an existing sales apparatus and crapping on them is not fair.That is not the same thing as these passive-aggressive [shit-tier 00:08:38] drip campaigns of, “I feel like I'm starting to stalk you.” Then don't send the message, jackhole. It's about empathy and not crapping on people who are trying to find their own path in this ridiculous industry.Tim: I think you brought up recruiters, and, you know, we here at The Duckbill Group are currently recruiting for a senior cloud economist and we don't actually have a recruiter on staff. So, we're going through various ways to find this work and it has really made me appreciate the work that recruiters in the past that I've worked with have done. Some of the ones out there are doing really fantastic work, especially sourcing good candidates, vetting good candidates, making sure that the job descriptions are inclusive, making sure that the whole recruitment process is as smooth as it can be. And it can't always be. Having to deal with all the spinning plates of getting interviews with folks who have production workloads, it is pretty impressive to me to see how a lot of these folks get—pull it off and it just seems so smooth. Again, like having to actually wade through some of this stuff, it's given me a true appreciation for the work that good recruiters do.Corey: We don't have automated systems that disqualify folks based on keyword matches—I've never been a fan of that—but we do get applicants that are completely unsuitable. We've had a few come in that are actual economists who clearly did not read the job description; they're spraying their resume everywhere. And the answer is you smile, you decline it and you move on. That is the price you pay of attempting to hire people. You don't put them on blast, you don't go and yell at an entire ecosystem of people because looking for jobs sucks. It's hard work.Back when I was in my employee days, I worked harder finding new jobs than I often did in the jobs themselves. This may be related to why I get fired as much, but I had to be good at finding new work. I am, for better or worse, in a situation where I don't have to do that anymore because once again, we have people here who do the various moving parts. Plus, let's be clear here, if I'm out there interviewing at other companies for jobs, I feel like that sends a message to you and the rest of the team that isn't terrific.Tim: We might bring that up. [laugh].Corey: “Why are you interviewing for a job over there?” It's like, “Because they have free doughnuts in the office. Later, jackholes.” It—I don't think that is necessarily the culture we're building here.Tim: No, no, it's not. Specially—you know, we're more of a cinnamon roll culture anyways.Corey: No. In my case, it's one of those, “Corey, why are you interviewing for a job at AWS?” And the answer is, “Oh, it's going to be an amazing shitpost. Just wait and watch.”Tim: [laugh]. Now, speaking of AWS, I have to absolutely shout out to Emily Freeman over there who has done some fantastic work this year. It's great when you see a person get matched up with the right environment with the right team in the right role, and Emily has just been hitting out of the park ever since he got there, so I'm super, super happy to see her there.Corey: Every time I get to collaborate with her on something, I come away from the experience even more impressed. It's one of those phenomenal collaborations. I just—I love working with her. She's human, she's empathetic, she gets it. She remains, as of this recording, the only person who has ever given a talk that I have heard on ML Ops, and come away with a better impression of that space and thinking maybe it's not complete nonsense.And that is not just because it's Emily, so I—because—I'm predisposed to believe her, though I am, it's because of how she frames it, how she views these things, and let's be clear, the content that she says. And that in turn makes me question my preconceptions on this, and that is why she has that I will listen and pay attention when she speaks. So yeah, if Emily's going to try and make a point, there's always going to be something behind it. Her authenticity is unimpeachable.Tim: Absolutely. I do take my hat's off to everyone who's been doing DevRel and evangelism and those type of roles during pandemics. And we just, you know, as the past few months, I've started back to in-person events. But the folks who've been out there finding new way to do those jobs, finding a way to [crosstalk 00:12:50]—Corey: Oh, staff at re:Invent next week. Oh, my God.Tim: Yeah. Those folks, I don't know how they're being rewarded for their work, but I can assure you, they probably need to be [unintelligible 00:12:57] better than they are. So, if you are staff at re:Invent, and you see Corey and I, next week when we're there—if you're listening to this in time—we would love to shake your hand, elbow bump you, whatever it is you're comfortable with, and laud you for the work you're doing. Because it is not easy work under the best of circumstances, and we are certainly not under the best of circumstances.Corey: I also want to call out specific thanks to a group that might take some people aback. But that group is AWS marketing, which given how much grief I give them seems like an odd thing for me to say, but let's be clear, I don't have any giant companies whose ability to continue as a going concern is dependent upon my keeping systems up and running. AWS does. They have to market and tell stories to everyone because that is generally who their customers are: they round to everyone. And an awful lot of those companies have unofficial mottos of, “That's not funny.” I'm amazed that they can say anything at all, given how incredibly varied their customer base is, I could get away with saying whatever I want solely because I just don't care. They have to care.Tim: They do. And it's not only that they have to care, they're in a difficult situation. It's like, you know, they—every company that sizes is, you know, they are image conscious, and they have things that say what like, “Look, this is the deal. This is the scenario. This is how it went down, but you can still maintain your faith and confidence in us.” And people do when AWS services, they have problems, if anything comes out like that, it does make the news and the reason it doesn't make the news is because it is so rare. And when they can remind us of that in a very effective way, like, I appreciate that. You know, people say if anything happens to S3, everybody knows because everyone depends on it and that's for good reason.Corey: And let's not forget that I run The Duckbill Group. You know, the company we work for. I have the Last Week in AWS newsletter and blog. I have my aggressive shitposting Twitter feed. I host the AWS Morning Brief podcast, and I host this Screaming in the Cloud. And it's challenging for me to figure out how to message all of those things because when people ask what you do, they don't want to hear a litany that goes on for 25 seconds, they want a sentence.I feel like I've spread in too many directions and I want to narrow that down. And where do I drive people to and that was a bit of a marketing challenge that Natalie in our marketing department really cut through super well. Now, pretend I work in AWS. The way that I check this based upon a public list of parameters they stub into Systems Manager Parameter Store, there are right now 291 services that they offer. That is well beyond any one person's ability to keep in their head. I can talk incredibly convincingly now about AWS services that don't exist and people who work in AWS on messaging, marketing, engineering, et cetera, will not call me out on it because who can provably say that ‘AWS Strangle Pony' isn't a real service.Tim: I do want to call out the DevOps—shout out I should say, the DevOps term community for AWS Infinidash because that was just so well done, and AWS took that with just the right amount of tongue in cheek, and a wink and a nod and let us have our fun. And that was a good time. It was a great exercise in improv.Corey: That was Joe Nash out of Twilio who just absolutely nailed it with his tweet, “I am convinced that a small and dedicated group of Twitter devs could tweet hot takes about a completely made up AWS product—I don't know AWS Infinidash or something—and it would appear as a requirement on job specs within a week.” And he was right.Tim: [laugh]. Speaking of Twitter, I want to shout out Twitter as a company or whoever does a product management over there for Twitter Spaces. I remember when Twitter Spaces first came out, everyone was dubious of its effect, of it's impact. They were calling it, you know, a Periscope clone or whatever it was, and there was a lot of sneering and snarking at it. But Twitter Spaces has become very, very effective in having good conversations in the group and the community of folks that have just open questions, and then to speak to folks that they probably wouldn't only get to speak to about this questions and get answers, and have really helpful, uplifting and difficult conversations that you wouldn't otherwise really have a medium for. And I'm super, super happy that whoever that product manager was, hats off to you, my friend.Corey: One group you're never going to hear me say a negative word about is AWS support. Also, their training and certification group. I know that are technically different orgs, but it often doesn't feel that way. Their job is basically impossible. They have to teach people—even on the support side, you're still teaching people—how to use all of these different varied services in different ways, and you have to do it in the face of what can only really be described as abuse from a number of folks on Twitter.When someone is having trouble with an AWS service, they can turn into shitheads, I've got to be honest with you. And berating the poor schmuck who has to handle the AWS support Twitter feed, or answer your insulting ticket or whatnot, they are not empowered to actually fix the underlying problem with a service. They are effectively a traffic router to get the message to someone who can, in a format that is understood internally. And I want to be very clear that if you insult people who are in customer service roles and blame them for it, you're just being a jerk.Tim: No, it really is because I'm pretty sure a significant amount of your listeners and people initially started off working in tech support, or customer service, or help desk or something like that, and you really do become the dumping ground for the customers' frustrations because you are the only person they get to talk to. And you have to not only take that, but you have to try and do the emotional labor behind soothing them as well as fixing the actual problem. And it's really, really difficult. I feel like the people who have that in their background are some of the best consultants, some of the best DevRel folks, and the best at talking to people because they're used to being able to get some technical details out of folks who may not be very technical, who may be under emotional distress, and certainly in high stress situations. So yeah, AWS support, really anybody who has support, especially paid support—phone or chat otherwise—hats off again. That is a service that is thankless, it is a service that is almost always underpaid, and is almost always under appreciated.Corey: This episode is sponsored by our friends at Oracle HeatWave is a new high-performance accelerator for the Oracle MySQL Database Service. Although I insist on calling it “my squirrel.” While MySQL has long been the worlds most popular open source database, shifting from transacting to analytics required way too much overhead and, ya know, work. With HeatWave you can run your OLTP and OLAP, don't ask me to ever say those acronyms again, workloads directly from your MySQL database and eliminate the time consuming data movement and integration work, while also performing 1100X faster than Amazon Aurora, and 2.5X faster than Amazon Redshift, at a third of the cost. My thanks again to Oracle Cloud for sponsoring this ridiculous nonsense.Corey: I'll take another team that's similar to that respect: Commerce Platform. That is the team that runs all of AWS billing. And you would be surprised that I'm thanking them, but no, it's not the cynical approach of, “Thanks for making it so complicated so I could have a business.” No, I would love it if it were so simple that I had to go find something else to do because the problem was that easy for customers to solve. That is the ideal and I hope, sincerely, that we can get there.But everything that happens in AWS has to be metered and understood as far as who has done what, and charge people appropriately for it. It is also generally invisible; people don't understand anything approaching the scale of that, and what makes it worst of all, is that if suddenly what they were doing broke and customers weren't built for their usage, not a single one of them would complain about it because, “All right, I'll take it.” It's a thankless job that is incredibly key and central to making the cloud work at all, but it's a hard job.Tim: It really is. And is a lot of black magic and voodoo to really try and understand how this thing works. There's no simple way to explain it. I imagine if they were going to give you the index overview of how it works with a 10,000 feet, that alone would be, like, a 300 page document. It is a gigantic moving beast.And it is one of those things where scale will show all the flaws. And no one has scale I think like AWS does. So, the folks that have to work and maintain that are just really, again, they're under appreciated for all that they do. I also think that—you know, you talk about the same thing in other orgs, as we talked about the folks that handle the billing and stuff like that, but you mentioned AWS, and I was thinking the other day how it's really awesome that I've got my AWS driver. I have the same, like, group of three or four folks that do all my deliveries for AWS.And they have been inundated over this past year-and-a-half with more and more and more stuff. And yet, I've still managed—my stuff is always put down nicely on my doorstep. It's never thrown, it's not damaged. I'm not saying it's never been damaged, but it's not damaged, like, maybe FedEx I've [laugh] had or some other delivery services where it's just, kind of, carelessly done. They still maintain efficiency, they maintain professionalism [unintelligible 00:21:45] talking to folks.What they've had to do at their scale and at that the amount of stuff they've had to do for deliveries over this past year-and-a-half has just been incredible. So, I want to extend it also to, like, the folks who are working in the distribution centers. Like, a lot of us here talk about AWS as if that's Amazon, but in essence, it is those folks that are working those more thankless and invisible jobs in the warehouses and fulfillment centers, under really bad conditions sometimes, who's still plug away at it. I'm glad that Amazon is at least saying they're making efforts to improve the conditions there and improve the pay there, things like that, but those folks have enabled a lot of us to work during this pandemic with a lot of conveniences that they themselves would never be able to enjoy.Corey: Yeah. It's bad for society, but I'm glad it exists, obviously. The thing is, I would love it if things showed up a little more slowly if it meant that people could be treated humanely along the process. That said, I don't have any conception of what it takes to run a company with 1.2 million people.I have learned that as you start managing groups and managing managers of groups, it's counterintuitive, but so much of what you do is no longer you doing the actual work. It is solely through influence and delegation. You own all of the responsibility but no direct put-finger-on-problem capability of contributing to the fix. It takes time at that scale, which is why I think one of the dumbest series of questions from, again, another group that deserves a fair bit of credit which is journalists because this stuff is hard, but a naive question I hear a lot is, “Well, okay. It's been 100 days. What has Adam Selipsky slash Andy Jassy changed completely about the company?”It's, yeah, it's a $1.6 trillion company. They are not going to suddenly grab the steering wheel and yank. It's going to take years for shifts that they do to start manifesting in serious ways that are externally visible. That is how big companies work. You don't want to see a complete change in direction from large blue chip companies that run things. Like, again, everyone's production infrastructure. You want it to be predictable, you want it to be boring, and you want shifts to be gradual course corrections, not vast swings.Tim: I mean, Amazon is a company with a population of a medium to medium-large sized city and a market cap of the GDP of several countries. So, it is not a plucky startup; it is not this small little tech company. It is a vast enterprise that's distributed all over the world with a lot of folks doing a lot of different jobs. You cannot, as you said, steer that ship quickly.Corey: I grew up in Maine and Amazon has roughly the same number employees as live in Maine. It is hard to contextualize how all of that works. There are people who work there that even now don't always know who Andy Jassy is. Okay, fine, but I'm not talking about don't know him on site or whatever. I'm saying they do not recognize the name. That's a very big company.Tim: “Andy who?”Corey: Exactly. “Oh, is that the guy that Corey makes fun of all the time?” Like, there we go. That's what I tend to live for.Tim: I thought that was Werner.Corey: It's sort of every one, though I want to be clear, I make it a very key point. I do not make fun of people personally because it—even if they're crap, which I do not believe to be the case in any of the names we've mentioned so far, they have friends and family who love and care about them. You don't want someone to go on the internet and Google their parent's name or something, and then just see people crapping all over. That's got to hurt. Let people be people. And, on some level, when you become the CEO of a company of that scale, you're stepping out of reality and into the pages of legend slash history, at some point. 200 years from now, people will read about you in history books, that's a wild concept.Tim: It is I think you mentioned something important that we would be remiss—especially Duckbill Group—to mention is that we're very thankful for our families, partners, et cetera, for putting up with us, pets, everybody. As part of our jobs, we invite strangers from the internet into our homes virtually to see behind us what is going on, and for those of us that have kids, that involves a lot of patience on their part, a lot of patients on our partners' parts, and other folks that are doing those kind of nurturing roles. You know, our pets who want to play with us are sitting there and not able to. It has not been easy for all of us, even though we're a remote company, but to work under these conditions that we have been over the past year-and-a-half. And I think that goes for a lot of the folks in industry where now all of a sudden, you've been occupying a room in the house or space in the house for some 18-plus months, where before you're always at work or something like that. And that's been a hell of an adjustment. And so we talk about that for us folks that are here pontificating on podcasts, or banging out code, but the adjustments and the things our families have had to go through and do to tolerate us being there cannot be overstated how important that is.Corey: Anyone else that's on your list of people to thank? And this is the problem because you're always going to forget people. I mean, the podcast production crew: the folks that turn our ramblings into a podcast, the editing, the transcription, all of it; the folks that HumblePod are just amazing. The fact that I don't have to worry about any of this stuff as if by magic, means that you're sort of insulated from it. But it's amazing to watch that happen.Tim: You know, honestly, I super want to thank just all the folks that take the time to interact with us. We do this job and Corey shitposts, and I shitpost and we talk, but we really do this and rely on the folks that do take the time to DM us, or tweet us, or mention us in the thread, or reach out in any way to ask us questions, or have a discussion with us on something we said, those folks encourage us, they keep us accountable, and they give us opportunities to learn to be better. And so I'm grateful for that. It would be—this role, this job, the thing we do where we're viewable and seen by the public would be a lot less pleasant if it wasn't for y'all. So, it's too many to name, but I do appreciate you.Corey: Well, thank you, I do my best. I find this stuff to be so boring if you couldn't have fun with it. And so many people can't have fun with it, so it feels like I found a cheat code for making enterprise software solutions interesting. Which even saying that out loud sounds like I'm shitposting. But here we are.Tim: Here we are. And of course, my thanks to you, Corey, for reaching out to me one day and saying, “Hey, what are you doing? Would you want to come interview with us at The Duckbill Group?”Corey: And it was great because, like, “Well, I did leave AWS within the last 18 months, so there might be a non-compete issue.” Like, “Oh, please, I hope so. Oh, please, oh, please, oh, please. I would love to pick that fight publicly.” But sadly, no one is quite foolish enough to take me up on it.Don't worry. That's enough of a sappy episode, I think. I am convinced that our next encounter on this podcast will be our usual aggressive self. But every once in a while it's nice to break the act and express honest and heartfelt appreciation. I'm really looking forward to next week with all of the various announcements that are coming out.I know people have worked extremely hard on them, and I want them to know that despite the fact that I will be making fun of everything that they have done, there's a tremendous amount of respect that goes into it. The fact that I can make fun of the stuff that you've done without any fear that I'm punching down somehow because, you know it is at least above a baseline level of good speaks volumes. There are providers I absolutely do not have that confidence towards them.Tim: [laugh]. Yeah, AWS, as the enterprise level service provider is an easy target for a lot of stuff. The people that work there are not. They do great work. They've got amazing people in all kinds of roles there. And they're often unseen for the stuff they do. So yeah, for all the folks who have contributed to what we're going to partake in at re:Invent—and it's a lot and I understand from having worked there, the pressure that's put on you for this—I'm super stoked about it and I'm grateful.Corey: Same here. If I didn't like this company, I would not have devoted years to making fun of it. Because that requires a diagnosis, not a newsletter, podcast, or shitposting Twitter feed. Tim, thank you so much for, I guess, giving me the impetus and, of course, the amazing name of the show to wind up just saying thank you, which I think is something that we could all stand to do just a little bit more of.Tim: My pleasure, Corey. I'm glad we could run with this. I'm, as always, happy to be on Screaming in the Cloud with you. I think now I get a vest and a sleeve. Is that how that works now?Corey: Exactly. Once you get on five episodes, then you end up getting the dinner jacket, just, like, hosting SNL. Same story. More on that to come in the new year. Thanks, Tim. I appreciate it.Tim: Thank you, Corey.Corey: Tim Banks, principal cloud economist here at The Duckbill Group. I am, of course, Corey Quinn, and thank you for listening.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

We’re learning all about Vertex AI this week as Carter Morgan and Jay Jenkins host guest Erwin Huizenga. He helps us understand what is meant by Asia Pacific and how Machine Learning is growing there. APAC’s Machine Learning scene is exciting for its enterprise companies leveraging ML for innovative projects at scale. The ML journey of many of these customers revealed challenges with things like efficiency that Vertex AI was built to solve. The Vertex AI platform boasts tools that help with everything from the beginning stages of data collection to analysis, validation, transformation, model training, evaluation, serving the model, and metadata tracking. Erwin offers detailed examples of this pipeline process and describes how Feature Store helps clients manage their projects. Using Vertex AI not only simplifies the initial development process but streamlines the iteration process as the model is adjusted over time. Pipelines offers automation options that help with this, Erwin explains. ML Operations are also built into Vertex AI to ensure everything is done in compliance with industry standards, even at scale. Using customer recommendations as an example, Erwin walks us through how Vertex AI can employ embedding to enhance customer experiences through ML. By using Vertex AI in combination with other Google offerings like AutoML, companies can effectively build working ML projects without data science experience. We talk about the Vertex AI user interface and the other tools and APIS that are available there. Erwin tells us how Digits Financial uses Vertex AI and Pipeline to bring models to production in days rather than months, and how others can get started with Vertex AI, too. Erwin Huizenga Erwin Huizenga is a Data Scientist at Google specializing in TensorFLow, Python, and ML. Cool things of the week Announcing Spot Pods for GKE Autopilot—save on fault tolerant workloads blog Indosat Ooredoo and Google Launch Strategic Partnership to Accelerate Digitalization Across SMBs and Enterprises in Indonesia site Indosat Ooredoo dan Google Luncurkan Kemitraan Strategis untuk Percepatan Digitalisasi UMKM dan Perusahaan di Indonesia site Interview Vertex AI site Google Cloud in Asia Pacific blog Introduction to Vertex AI docs What Is a Machine Learning Pipeline? site TensorFlow site PyTorch site Vertex AI Feature Store docs AutoML site BigQuery ML site Vertex AI Matching Engine docs ScaNN site Announcing ScaNN: Efficient Vector Similarity Search blog Vertex AI Workbench site Vertex Pipeline Case Study: Digits Financial site Intro to Vertex Pipelines Codelab site Vertex AI: Training and serving a custom model Codelab site Vertex AI Workbench: Build an image classification model with transfer learning and the notebook executor Codelab site APAC Best of Next 2021 site TFX: A TensorFlow-Based Production-Scale Machine Learning Platform site Rules of Machine Learning site Google Cloud Skills Boost: Build and Deploy Machine Learning Solutions on Vertex AI site Monitoring feature attributions: How Google saved one of the largest ML services in trouble blog What’s something cool you’re working on? Jay is working on APAC Best of Next and will be doing a session on sustainability! Carter is working on transitioning the GCP Podcast to a video format!

Bu bölümde Amerikada bir ML-ops girişimi fal.ai (features & labels) kurucuları Burkay Gür ve Görkem Yurtseven konuk ettik. Bol bol ML-Ops konuştuğumuz bu sohbette feature storage'dan model serving'e ve ml-ops trendlerine kadar bir çok konuyu ele aldık. İletişim icin Burkay'a ve Gorkem'e email adreslerinden ulasabilirisiniz: burkay@fal.ai gorkem@fal.ai

The panel jumps in to attempt to break your mental build regarding testing your ML Ops. They advocate for good testing practices around your code and systems and discuss how you can reliable test the various parts of your applications including your Machine Learning models. Panel Ben WilsonCharles Max WoodFrancois Bertrand Sponsors Dev Influencers AcceleratorLevel Up | Devchat.tv Picks Ben- The Boston Housing DatasetCharles- Coaching | Top End DevsCharles- The 360 Degree LeaderCharles- The Laws of Wealth: Psychology and the Secret to Investing Success Francois- Dune Contact Ben: DatabricksGitHub | BenWilson2/ML-EngineeringGitHub | databrickslabs/automl-toolkitLinkedIn: Benjamin Wilson Contact Charles: Devchat.tvDevChat.tv | FacebookTwitter: DevChat.tv ( @devchattv ) Contact Francois: Francois BertrandGitHub | fbdesignpro/sweetviz

Over last few years, organizations have been busy experimenting with ML models for specific use cases and working with data scientists to optimize model accuracy/ performance. Now, they want to deploy these models at scale and realizing that there are lot many new challenges ahead of them! In this episode, Infosys AI experts Amit Gaonkar and Kaushal Desai tell us why organizations need to think of ML Ops in strategic way and not just as a toolkit to automate deployment of machine learning models. Listen to know how an enterprise MLOps layer built on good architecture principles, enables organizations to build future proof, scalable & responsible enterprise AI with adaptable governance mechanisms. Hosted by Abhiram Mahajani, Sales Director, AI and Automation Services, UK and Europe, Infosys

Tushit is carrying over 11 years of experience in project management , Sales and proposal engineering and in data science. He is Currently associated with HCL as a data science consultant and handling a product ML Ops and NLP.

Ils ont fait x4 en effectif en moins de 2 ans et ont recruté plus de 60 Data Scientists.  Preligens est une scale-up proposant des solutions logicielles dans le domaine de la défense et a connu une croissance fulgurante ces deux dernières années. Aujourd'hui, je reçois Marie-Caroline Corbineau, Data Scientist au sein de l'équipe R&D de Preligens pour revenir sur les effets de cet afflux de nouveaux contributeurs sur leur capacité à développer et livrer des algorithmes à l'état de l'art répondant aux besoins de leurs clients. Marie-Caroline nous raconte son parcours et son arrivée à Preligens avant de faire un focus sur l'infrastructure ML qu'ils utilisent en interne. C'est cette AI Factory qui leur permet de réduire les délais dans leur itérations produits et de déployer plus vite. Ressources: - The Sequence of AI knowledge  - newsletter distilant les avancées en deep learning et proposant une perspective business sur les valorisations et levées de fonds. https://thesequence.substack.com/   - How we built an AI Factory - Preligens - Un medium racontant l'intiative interne entreprise à Peligens pour développer leur framework de ML Ops https://medium.com/earthcube-stories/how-we-built-an-ai-factory-part-1-2fb34c4cc648 

נושא ה-MLOPS הוא ה-תחום הצומח בעולם הפיתוח. חברות רבות מחפשות כיצד להשתמש בהתלהבות הגדולה ובפיתוחי הAI החדשים. אך כדי להצליח להטמיע מערכות MLOPS בארגון, ישנם לא מעט דברים שכדאי להכיר.בפרק הזה בסדרת above the cloud עם שותפי הענן של google cloud אירחנו את גיא שפיר, CTO בחברת WIDEOPS שיסביר לנו על הקונספט של ML Ops, איך הוא מתבטא בחברות בשלבים שונים, נביא גם כמה סיפורים מעניינים של שימושים בקונספט ונראה את הכלים שגוגל מעמידה לרשות חברות כדי לעזור להטמיע מערכות MLOPS.האזנה נעימה,חן

On this episode of the AI Show, we're talking about MLOps. Seth welcomes Microsoft Data Scientist, Spyros Marketos, ML Engineer, Davide Fornelli and Data Engineer, Samarendra Panda. Together they make up an AI Taskforce and they'll give us a high-level intro into MLOps and share some of the surprises and lessons they've learned along the way!Jump to:[00:17] AI Show Intro[00:34] Welcome and Introductions[01:41] Use cases from the AI Taskforce[02:47] Commonalities across projects[03:50] Common challenges - from the Data Engineer perspective[06:47] Common challenges - from the ML Engineer perspective[08:46] Common challenges from the Data Science perspective[10:48] What does success in MLOps look like?[12:30] Surprising challenges working with customers and how to avoid them[19:27] Review - what is ML Ops[19:45] MLOps in Delivery mission[21:57] MLOps principles[27:52] Tips from the pros Learn more:Machine Learning for Data Scientists https://aka.ms/AIShow/MLforDataScientistsPakt: Principles of Data Science https://aka.ms/AIShow/DataSciencePacktZero to Hero Machine Learning on Azure https://aka.ms/ZerotoHero/MLonAzureZero to Hero Azure AI https://aka.ms/ZerotoHero/AzureAICreate a Free account (Azure) https://aka.ms/aishow-seth-azurefreeFollow Seth https://twitter.com/sethjuarezFollow Spyros https://www.linkedin.com/in/smarketos/Follow Davide https://www.linkedin.com/in/davidefornelli/Follow Sam https://www.linkedin.com/in/samarendra-panda/Don't miss new episodes, subscribe to the AI Show https://aka.ms/AIShowsubscribeAI Show Playlist https://aka.ms/AIShowPlaylistJoin us every other Friday, for an AI Show livestream on Learn TV and YouTube https://aka.ms/LearnTV - https://aka.ms/AIShowLive

In part 2 of our interview with Abhi Seth, he tells us that part of his role is to really drive scale for analytics across the ten businesses within TE. We learn how the adoption of analytics is enabled throughout the organization and a big component of that is driving, understanding and building capability within the Centre of Excellence (COE). He says the first 90 days in the COE is about building capability and being able to have experts in data science, cloud, dev ops, ML Ops, data visualization, user experience, storytelling, and data engineering. Abhi says his focus is now on building a small COE team within each of the businesses and moving to a “hub & spoke” model as the analytical maturity of the organisation improves. Abhi goes on to tell us about how he creates and enables “seed teams'' and how it's important to ensure the problems you're solving are creating value for the company and are tied to a strategy. He also says you should have a committed executive sponsor. Throughout the episode we discuss the results of our poll questions: Does your organization have a data science or analytics Center of Excellence? How is your organization's cloud migration going? Does your organization centrally manage the delivery of analytics across the enterprise? Does your organization measure the success of the analytics function? Does your organization develop their analytical talent? Enjoy the show! Read the full episode summary here: Ep 163 Thank you to our sponsor, Talent Insights Group. --- Send in a voice message: https://anchor.fm/datafuturology/message

Today's guest is Shawn Ramirez, Head of Data Science at Shelf Engine in Seattle. Shawn is an accomplished data science leader and SME in causal inference, experimentation, Machine Learning, statistics, optimization and game theory. She drives AI product development building, testing, and leveraging statistical and cutting edge machine learning at scale on high impact problems. Shawn's passion is working on complex questions about behavior, users and customers. Founded in 2015, Shelf Engine uses machine learning to help grocery stores dial in their orders to minimize waste and maximize profits. Shawn joined the company in late 2020 and leads a high-performing team in data science, machine learning, research, ML Ops, causal inference and experimentation. Shawn and her team are working on forecasting and price optimization to solve the $160B food waste problem, lower prices and feed America. In today's episode, Shawn tells us about: Shelf Engine's work within grocery forecasting, Problems they are solving within food waste and hunger, How they are applying Machine Learning to solve these problems, What she looks for when hiring into the team, Advice on how to become a leader within Data Science and Why she loves working at Shelf Engine

Data Ops is about working with everyone who deals with Data to deploy data related projects together. It is not just one person’s job. Christopher Bergh, CEO of Data Kitchen has embarked on Data Ops journey much earlier than the industry was asking for it. Nowadays, everybody including Gartner is talking about Ops, Data Ops, Dev Ops, ML Ops, X-ops etc. But Ops should not be a single person’s job. It should be 10% of every team member’s job to think about Operations. Just like Deming prescribed in a manufacturing process, it should be part of the system and framework.

Data Ops is about working with everyone who deals with Data to deploy data related projects together. It is not just one person's job. Christopher Bergh, CEO of Data Kitchen has embarked on Data Ops journey much earlier than the industry was asking for it. Nowadays, everybody including Gartner is talking about Ops, Data Ops, Dev Ops, ML Ops, X-ops etc. But Ops should not be a single person's job. It should be 10% of every team member's job to think about Operations. Just like Deming prescribed in a manufacturing process, it should be part of the system and framework.

In this new world of machine learning and AI, where data basically writes code and algorithms, MLOps has developed into something broad and very important. So much so that a huge community has developed around this space.David Aponte and Demetrios Brinkmann are our guests on this episode of the Georgian Impact Podcast. David is a Software Engineer at Microsoft with a focus on MLOps. Demetrios is the Community Coordinator for the MLOps Community and also works in the ethical AI space. Together they will break down MLOps and why it is so important.You'll Hear About:The MLOps community and how it has grown.How MLOps compares with other fields and what sets it apart.The challenges of integrating ML with product management.How the MLOps community works to address accuracy and bias.The need to diversify the community with not only technical side of things but also the ethical.How the cloud has influenced the rate of development.

“Only 22 percent of companies using machine learning have successfully deployed a model”.(Deeplearning.ai ) Why such a big gap between machine learning model development and production? What are the big challenges and how are they being solved? Adrià Salvador, lead of the Data Science Productivisation team at Glovo, is here with us today to share his tricks and tips in the field of ML Ops. From his honest revelation of personal experiences, we see the challenges leading a team to productionize data science directly into the company's operation. Other than real case references, Adrià also introduced concepts of best data science practices, machine learning model production pipeline in Glovo, and open source tool packs for us to add to our next learning list. Don't miss the opportunity and listen in! Speaker Bio: Adrià Salvador Palau (Barcelona, 1990) holds a BSc and MSc in Physics. He also holds a PhD in Engineering from the University of Cambridge. In his PhD, Adrià developed distributed machine learning architectures to predict failures in large fleets of industrial machines. Adrià's research focused both on the technological and economical challenges of implementing these technologies in industrial scenarios. He joined Glovo two years ago to work as a Data Scientist. Since then, he has been promoted to lead the Data Science Productivisation team at Glovo. His team has the responsibility of speeding up productivisation of machine learning models in glovo and helping determining MLOPS best practices within the company. Resources: MLOPS by google MLOPS in Towards Data Science Deep Learning by Goodfellow Umap Paperswithcode Poetry

In today's episode we are joined by Jenn Gamble (PhD). Jenn is the Data Science Practice Lead at Very and holds a PhD in electrical engineering.  We talk about building machine learning products and the different practices it involves. We dive into the data science development lifecycle, agile development practices, interdisciplinary team work and practices such as ML Ops, test driven development and pair programming. 

Learn more about the ML Ops Community: https://mlops.community/ (https://mlops.community/) Every Thursday I send out the most useful things I've learned, curated specifically for the busy machine learning engineer. Sign up here: https://cyou.ai/newsletter (https://cyou.ai/newsletter) Follow Charlie on Twitter: https://twitter.com/CharlieYouAI (https://twitter.com/CharlieYouAI) Subscribe to ML Engineered: https://mlengineered.com/listen (https://mlengineered.com/listen) Comments? Questions? Submit them here: http://bit.ly/mle-survey (http://bit.ly/mle-survey) Take the Giving What We Can Pledge: https://www.givingwhatwecan.org/ (https://www.givingwhatwecan.org/) Timestamps: 02:45 Intro 04:10 How I got into data science and machine learning 08:25 My experience working as an ML engineer and starting the podcast 12:15 Project management methods for machine learning 20:50 ML job roles are trending towards more specialization 26:15 ML tools enable collaboration between roles and encode best practices 34:00 Data privacy, security, and provenance as first class considerations 39:30 The future of managed ML platforms and cloud providers 49:05 What I've learned about building a career in ML engineering 54:10 Dealing with information overload Links: https://www.mlengineered.com/episode/josh-tobin (Josh Tobin: Research at OpenAI, Full Stack Deep Learning, ML in Production) https://towardsdatascience.com/the-third-wave-data-scientist-1421df7433c9 (The Third Wave Data Scientist) https://www.youtube.com/watch?v=GvAyV8m8ICI (Practical ML Ops // Noah Gift // MLOps Coffee Sessions) https://cyou.ai/podcast/pavle-jeremic/ (Building a Post-Scarcity Future using Machine Learning with Pavle Jeremic (Aether Bio)) https://www.youtube.com/watch?v=Fu87cHHfOE4 (SRE for ML Infra // Todd Underwood // MLOps Coffee Sessions) https://www.youtube.com/watch?v=ShBod1yXUeg (Luigi Patruno on the ML Ops Community podcast) https://www.mlengineered.com/episode/luigi-patruno (Luigi Patruno: ML in Production, Adding Business Value with Data Science, "Code 2.0")

https://www.engati.com/ Engati is the world's leading no-code, multi-lingual chatbot platform. https://open.spotify.com/show/3G0uQwPnQib22emRi9VhUg Blog link: https://engati.com/blog | Subscribe now. Check out CX Community page - https://www.engati.com/cx-community And CX Content page - https://www.engati.com/cx-content Jan van der Vegt talks about ML Ops and explains to us about how the Cubonacci platform help in automating the machine learning lifecycle. Follow us on Facebook: http://s.engati.com/157 LinkedIn: http://s.engati.com/158 Twitter: http://s.engati.com/156 Instagram: https://www.instagram.com/getengati/  #EngatiCX #digital #data #ai #MLOps #Cubonacci

Josh Tobin holds a CS PhD from UC Berkeley, which he completed in four years while also working at OpenAI as a research scientist. His focus was on robotic perception and control, and contributed to the famous Rubik's cube robot hand video. He co-organizes the phenomenal Full Stack Deep Learning course and is now working on a new stealth startup. Learn more about Josh: http://josh-tobin.com/ (http://josh-tobin.com/) https://twitter.com/josh_tobin_ (https://twitter.com/josh_tobin_) Want to level-up your skills in machine learning and software engineering? Join the ML Engineered Newsletter: https://mlengineered.ck.page/943aa3fd46 (https://mlengineered.ck.page/943aa3fd46) Comments? Questions? Submit them here: https://charlie266.typeform.com/to/DA2j9Md9 (https://charlie266.typeform.com/to/DA2j9Md9) Follow Charlie on Twitter: https://twitter.com/CharlieYouAI (https://twitter.com/CharlieYouAI) Take the Giving What We Can Pledge: https://www.givingwhatwecan.org/ (https://www.givingwhatwecan.org/) Subscribe to ML Engineered: https://mlengineered.com/listen (https://mlengineered.com/listen) Timestamps: 01:32 Follow Charlie on Twitter (http://twitter.com/charlieyouai (twitter.com/charlieyouai)) 02:43 How Josh got started in CS and ML 11:05 Why Josh worked on ML for robotics 15:03 ML for Robotics research at OpenAI 28:20 Josh's research process 34:56 Why putting ML into production is so difficult 44:46 What Josh thinks the ML Ops landscape will look like 49:49 Common mistakes that production ML teams and companies make 53:11 How ML systems will be built in the future 59:37 The most valuable skills that ML engineers should develop 01:03:50 Rapid Fire Questions Links https://course.fullstackdeeplearning.com/ (Full Stack Deep Learning) https://arxiv.org/abs/1703.06907 (Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World) https://arxiv.org/abs/1710.06425 (Domain Randomization and Generative Models for Robotic Grasping) https://deepmind.com/blog/article/neural-scene-representation-and-rendering (DeepMind Generative Query Network (GQN) paper) https://arxiv.org/abs/1911.04554 (Geometry Aware Neural Rendering) https://www2.eecs.berkeley.edu/Pubs/TechRpts/2019/EECS-2019-104.pdf (Josh's PhD Thesis) https://www.youtube.com/watch?v=x4O8pojMF0w (OpenAI Rubik's Cube Robot Hand video) https://www.wandb.com/podcast/josh-tobin (Weights and Biases interview with Josh) https://www.oreilly.com/library/view/designing-data-intensive-applications/9781491903063/ (Building Data Intensive Applications) http://creativeselection.io/ (Creative Selection)

In this episode of The IoT Unicorn Podcast, Rene Haas, President Intellectual Property Group at Arm, discusses the development of edge devices and the 5G wave. Download Transcript Here 00:00 PETE BERNARD: Rene, thanks again for joining us here on the IoT Unicorn. I was trying to remember the last time actually we saw each other face-to-face. That's something that we do these days. I think it was Barcelona 2019 or something. It was a while ago. But again, thanks for joining us today. 00:23 RENE HAAS: You are welcome. I wasn't sure if it was CES of 2020, but... 00:28 PETE BERNARD: It could be. 00:28 RENE HAAS: Gosh, you might be right. Barcelona, 2019. My gosh, over 18 months ago. 00:32 PETE BERNARD: Yeah, that was a long time ago. Well, CES 2020 was our last... It was kind of the last hurrah for events, although going to Vegas always has its potential infection rates of all sorts of things going on there, but... Not in that case, but... Cool, yeah, no, it's good to see you again, and we've known each other for a little while and worked on some interesting projects, so it was great to have you on the show, and obviously very timely with the DevSummit coming up and some recent news that we'll talk about as well. But maybe you can give us and the listeners a little background on your journey to where you're at as President of Arm IP. 02:07 RENE HAAS: So my role at Arm is I run the IP products group. Our acronym is IPG, Intellectual Property Products Group, and that's the sales marketing development of all of our products, GPUs, CPUs, NPUs for the markets that we serve, the client market, infrastructure market, automotive autonomous and IoT. I am in the Bay Area now, but I've had a fun journey at Arm. I have spent seven years at Arm, but only a few years in the Bay Area. I was in Shanghai, China for two years, and I was in the UK for three, living in London, commuting to Cambridge. And I just came back to the Bay Area at the beginning of 2020, and... 02:50 PETE BERNARD: Are you an original California person or what's your... Where is your home base? 02:54 RENE HAAS: I'm originally from Upstate New York. Yeah, I'm, originally from Upstate New York. 02:58 PETE BERNARD: Wow, cool. 02:58 RENE HAAS: My dad was a Xerox guy, so I was a son of a Xerox guy working in... He was working in Rochester, New York, which is where I grew up. And then I came out to California in the mid-1990s, and I've been here ever since. 03:12 PETE BERNARD: I'm a New Jersey person myself, so that's something we have in common, the Tri-state area. Although Rochester is pretty far up state there. 03:21 RENE HAAS: Serious snow country. 03:22 PETE BERNARD: Serious, yes. Good, good. Excellent. So you've been at Arm for a while then, and you also spent a little bit of time at Nvidia. 03:31 RENE HAAS: I did, I did. I'm gonna pre-fetch probably your next set of questions, but before I spent... 03:37 PETE BERNARD: No pun intended. 03:38 RENE HAAS: Seven years at Arm I was with Nvidia for seven years doing a number of different roles there, but primarily in the notebook graphics space, GPUs, as well as Arm-based CPUs that went into all different types of laptops including the very first Surface that was running Windows 8 on Arm. 04:00 PETE BERNARD: Yes, those were the days. I had one of those. A lot of us up in Redmond had one of those. [chuckle] Unfortunately, not a lot of the other people had them. That was the problem. [chuckle] But, so cool. So now sort of full circle, just to touch upon that topic, Nvidia and Arm. For you, it's kind of break out the old badge, I guess... 04:20 RENE HAAS: Yeah. It's something that came live last Monday. Obviously, the rumors had been out for a number of weeks, so some people were surprised, but some people were not so surprised when it finally was announced to everyone actually last Sunday. It was supposed to be on Monday, and then we pulled it forward to Sunday. We're actually very excited about it at Arm, we think it's a really, really amazing opportunity. Nvidia is an amazing company, has done some fantastic things over the years obviously. And Arm efforts around client and data center, autonomous and such. When we think about what's going on in the next wave of computing where everything is gonna be touching something that is around artificial intelligence, I think the opportunities for the two companies to be a combined entity in this new area of computing, the opportunities are somewhat limitless. 05:17 RENE HAAS: So we're quite excited. Me, on a personal level, sometimes when these M&A things [05:21] ____ talking to the company on either side, there's a lot of questions of, "Do I know these folks? And can we really understand what their language is?" But for me, having spent equal amount of time in both places, I feel very fortunate to be in a position to be where we are on this, and it should be very exciting. And someone over there even pinged me not long after the announcement and said, "Hey, your email address is still available." So it's interesting how things circle back. 05:55 PETE BERNARD: Yeah, yeah, I wonder if you get credited those seven years at Nvidia as part of your Arm tenure. So how that works I'm not sure. 06:00 RENE HAAS: You know what, that's a really good question. I haven't... 06:03 PETE BERNARD: You might get a double hit on that one. 06:06 RENE HAAS: Yeah. In fact [06:08] ____ Pete, that was not on the FAQ. That's a good one. I'm gonna go check on that. 06:14 PETE BERNARD: Well, one of the things that's happened over the past number of years, what's been super exciting working with Arm is kind of the proliferation of where Arm is, the Arm silicon showing up. And you mentioned the early experiments, early efforts I should say, on Windows on Arm, but we had kind of a relaunch or a re-emergence of that tech a couple years ago, and I know I had the pleasure of working with you guys on that. So Windows on Arm, Windows on Snapdragon and all that stuff, it seems to be kind of a resurgence now on that as well. So what are your thoughts there? 06:50 RENE HAAS: Oh gosh. And as I mentioned, the history with working with Nvidia and Arm and Microsoft for me goes way back. And having worked on the original Surface product, that was basically what we called [07:06] ____ back in the day. And if I just think back to the value proposition we were hoping to get from those systems, it was really around extended battery life, always on, always connected, things like that. But you go back those years, there was no connectivity story, so those were just obviously purely WiFi devices. And the app story was really, really incomplete. I remember meeting with analysts early on and one of the biggest questions that I got asked when we were going to press reviews was, "Will it run iTunes?" And the answer to that question at the time was, "No." And that was a bit of a killer, if you just think about how people were getting access to music back and when these products came out. Fast-forward to now, the landscape is so different when you just think about, A, how many of our applications exist in the Cloud? B, the devices that have been introduced by third-party OEMs and as well as Microsoft. You have these amazing connectivity type of solutions that are brought forward by Snapdragon, so there's a great story in terms of connectivity. There's a great story in terms of app compatibilities on Windows 10 with everything running across. So we... 08:19 PETE BERNARD: Including iTunes, by the way. So iTunes now runs on that. 08:23 RENE HAAS: ITunes runs. And I bet you if I went through and asked that analyst and told them that iTunes ran successfully on these Windows devices, he would not care. But yeah, the experience is great. We use a lot of them inside of Arm. In fact, when I was living in the UK, I used to use it all the time on the train because the WiFi was actually spotty on the train and the cellular worked pretty good, and it was a great device to use. And not the least of which, I would literally leave my power supply back in the flat during the day. I wouldn't bring it with me, wouldn't need it. And so the devices have really, really advanced, and then there's just more great things to come. 09:04 PETE BERNARD: Yeah, fantastic. I use the Galaxy Book as my main PC and yeah, it's a game changer. When you don't have to worry about power and connectivity, all of a sudden, it's like a behavioral change in how you use a PC, so it's pretty cool stuff. And then I guess the other big thing where you're making a lot of headway with partners is in the Cloud and sort of bringing a lot of low-power. A lot of times, people think of low-power as battery life, but it's not just battery life, it's just low-power, a greener, more smarter consumption of power, overall, especially in a big data center. 09:42 RENE HAAS: Yeah, no, that's exactly right. Arm has been working on products for the data center for actually a long time. Even from back in the time when I was at Nvidia, Arm was working with early partners around SSEs for the data center and such. Like everything else, over 10 years a lot of things have changed. Confluence of a lot of work being done on the engineering side to get great products. We've gone from 32-bit to 64-bit. The performance has increased. Geometries have also gone in such a way that you've gone from 10 to seven to five nanometre type of technologies now, so you can get some really, really powerful type of processing. And then just again, like any technology trend, you need a confluence of a number of things to take place. 10:32 RENE HAAS: 10 years ago, we were thinking largely about the enterprise; we weren't thinking as much about the Cloud. And what has happened with everything moving towards the Cloud, to your point, it's put such a premium on data efficiency, on power. These Cloud data centers typically have a very, very fixed power budget and a very fixed area where they put the compute capacity. So efficiency really, really matters, it's really, really important. And we continue to innovate in this area. We've introduced some new products. Our Neoverse V1, which has scalar vector processing for HPC and high-end computing. Our N2 platform, which is 40% more efficient than our N1 platforms. And we've seen some of the large hyperscalers including AWS who have announced products based upon our N1 with their Graviton2 processor. And they've talked very publicly about a 40% power advantage at the same performance level versus the competition. So yeah, it's very real and people might think, "Oh, my gosh, it's happened overnight." And you've been in this industry a long time, you know it doesn't. 11:46 PETE BERNARD: That's right. 11:47 RENE HAAS: It's a long, long effort by a lot of partners and a lot of people inside of Arm. But yeah, now I think confluence of a lot of things in the marketplace, it's really starting to take off. 11:56 PETE BERNARD: Yeah, it's true. For a lot of things, it's a matter of the right time and the right tech and the right need for it to all come together. Actually, interesting anecdote, just to circle back to the PC discussion. We were first working on the Windows on Snapdragon PCs, we had a big beta test inside of Microsoft and we handed them out to all of our engineering managers and stuff. And we started to get bug reports that the battery meter was not working right because it was just always full. And it turned out the battery meter was working fine, it's just people weren't used to the fact that this thing would last for whatever, 20 hours. And so it was an interesting discussion with folks that that's actually how it's supposed to work. 12:38 RENE HAAS: Which is game changer, like you said. 12:41 PETE BERNARD: Yeah, yeah. So let's get to IoT. This is called the IoT Unicorn, so we might as well dig into that. Probably the real fascinating things happening on the edge, the far edge, the near edge. The definition of the edge depends on where you're standing, I guess. But Arm at the edge and things that are happening out there, what do you see as disruptions that we should be expecting beyond the incremental things getting faster and less power, but what's the view there? One of the interesting things for our listeners that aren't aware is an IP license is like pretty far up the food chain. So you get probably one of the best long-term views of what's happening in the business over the next, whatever, five years. But be curious on the IoT and edge side, where do you see things heading? 13:30 RENE HAAS: Yeah, no, it's a great question. And that area is evolving fast. Even over the last number of years, we've seen a real acceleration of activity, innovation in that space. And particularly around the area of that these edge devices are increasingly becoming small computers in of themselves. When IoT kicked off with VIGOR inside of Arm, we were talking to companies about this. It included a small microcontroller with potentially a sensor and a Bluetooth connector that could send the data back somewhere. Now you're talking about a heavy degree of compute power, you're talking about machine learning at the edge. Increasingly, we have partners who are looking to not only use our micro-controllers that have extensions for machine learning, but even tiny MPUs, tiny ML doing some level of inference at the edge. 14:24 RENE HAAS: And with that, you have a much different requirement for security because now these devices are small computers, they're dealing with a tremendous amount of data, the data needs to be protected, you need to ensure that you have an architecture that will keep the data secure. So we've done a lot of work with our partners around an innovation that we call the platform security architecture, which does a number of things. We've done a lot of work over the years around Root of Trust and things at that nature. With this platform security architecture, we actually allow for third parties to certify the devices that will essentially assure a level of data encryption and security going up the line. And with that, I think it just all feeds onto itself relative to... These are small computers, these small computers are doing more and more compute intensive tasks, they're sending more and more data through the Cloud, you then have 5g that is also adding more bandwidth and more compute capability. So what that basically means is you just start pushing from the data center to the edge, the amount of compute capacity is going up exponentially. 15:41 RENE HAAS: And I think over the next number of years, these edge devices are gonna become even more powerful and more sophisticated in terms of their capability. And you'll have a very interesting trade-off between the applications that run with that edge device at the node next to it, things that are cloud-native where the app can be running in a number of different spots. I think also you're gonna see huge innovation. And that's gonna mean certain things like autonomous entities. Not necessarily cars. Obviously cars are the most popular areas that get a lot of attention, but drones and robotics and things that can run at a much more sophisticated way, factory floor robotics, all kinds of things around managing warehousing, things of that nature. All of this is gonna become much more intelligent and much more sophisticated. 16:27 RENE HAAS: And then, back to the Nvidia/Arm potential about the edge of AI, these devices will learn, they'll get smarter. And as they get smarter, that again builds on having the compute capabilities. I know it sounds a bit of a cliche, and I've been around the industry probably to see at least a number of these waves of computing, but we're definitely into another very large one. And 5g, because of the additional bandwidth, is gonna be able to enable a lot of that. 16:55 PETE BERNARD: Yeah. I think I had this discussion with Rob Tiffany from Ericsson on the last episode or two episodes ago, but we were talking about the confluence of 5g, AI and IoT, sort of three, these... It's like peanut butter, chocolate and whatever the third thing is. But I haven't... The metaphor breaks down after that. It's like you get these ultra low latency, high performance networks combined with AI, which you could either do at the edge or the cloud or somewhere in between, with the concept of Internet of Things, which is just things connected to the Cloud and sending intelligent data back and forth and actuating in real-time. And then all of a sudden, you've got some really potential transformative scenarios there, right? 17:34 RENE HAAS: Yeah. 17:36 PETE BERNARD: And so I think... So it's sort of like... And I've had Qualcomm on the show before and other folks, and we talk about IoT being a team sport, that that statement of 5g, AI and IoT is an interesting example 'cause you need lots of different companies to come to the table to work together on behalf of a customer problem, 'cause it all starts with a customer having a problem that they need solved. And, yeah, I agree with you. You mentioned also about the fact that we're bringing AI horsepower into MCU devices or really tiny edge devices that previously were controlling a light switch are now going to be smart, and be able to learn and execute AI models. And I think that's fascinating. 18:22 RENE HAAS: Yeah. And you still have to get into... And by the way, I like that peanut butter and chocolate analogy, which are two of my favorite ingredients on [18:28] ____. You just need a third, but... 18:29 PETE BERNARD: [chuckle] Peanut butter, chocolate and more chocolate, I don't know if that's fair or not. 18:32 RENE HAAS: But similar to... One of the stories I like to talk about is a bit of what these new waves of technology enable. When we went from 3g to 4g, and I know you and I both were around for that, people were not talking about the fact that 3g to 4g was going to enable a brand new ride sharing capability, and it was gonna be able to enable people to rent their homes for vacations and such. Yet Airbnb, and Uber, and location bearing apps and things you can do on a smartphone all came through with that. I think the same thing is true for 5g and IoT. It's a little hard to completely imagine all of the possibilities that can happen. There's a lot of smart people and, as you said it, it takes a village of a combination of chip people and OEMs and software and makers to come up with a lot of ideas to advance this. But it will be there because there's such a profound shift of compute power that's gonna exist in these edge devices that is going to allow for a lot of really, really interesting potential. So it's gonna be really exciting to see. 19:37 PETE BERNARD: Let me kind of cut into one blurb here around AI Toolchain, because I believe one of the things we've done with Arm and I think should be announced for DevSummit, if not, we'll edit it out, but we've come to some agreement with you, I believe, to integrate your AI Toolchain into Azure. 19:56 RENE HAAS: Yeah. 19:56 PETE BERNARD: One of the things is around... ML Ops is a kind of a hot term, but how do you leverage a hyperscaler cloud to develop and train models and then manage those models across the edge to the cloud securely on updating these edge devices with new AI capabilities or models or trainings and tunings? And so your Toolchain's kind of at the core of a lot of that for a lot of Silicon partners, so the ability to sort of integrate that Toolchain into Azure for our customers should be a big deal, right? 20:26 RENE HAAS: Oh, it's a really, really huge opportunity. We're actually quite excited about it. We do a lot of work on the Toolchain with Compute Libraries and frameworks and different things to allow folks to develop solutions for ML at the edge, and I think we probably have as many people in our ML group doing hardware MPUs and also are doing the software libraries and frameworks. So it's really, really large. And you're reaching a brand new set of developers, if you will, and think about a Raspberry Pi or an Arduino-like platform for people who are developing things for the edge. If you can now allow those to integrate, upscale into the Azure cloud framework, because all of this tiny data becomes big data in the cloud, and then ultimately it can get serviced in such a way that end users can benefit. It's actually a really exciting thing and we've been partners with Microsoft for such a long time in a broad set of areas. I'm very excited to be involved here as well. 21:27 PETE BERNARD: Yeah, that'd be great. Hey, so DevSummit. We're on the eve of DevSummit or the day one of DevSummit. I'm not sure what the publication timeline is here, but it's a big deal. It's very exciting. Obviously this year kind of highly virtualized, but still exciting. Do you have any kind of words of wisdom if you're an attendee for DevSummit? What are some of the things you wanna look for or try to get out of? And maybe first time visitors or whoever, how do people really grok the scene? 21:57 RENE HAAS: It's a big change on a couple fronts. Obviously, first off, it's virtual. It's not live. So that's for starters. So go to your favorite search engine and search for DevSummit and you get all the details about registering and such, but we have moved it to a virtual event. For those of who are saying, "Okay, it's virtual, I get it, but I've never heard of DevSummit. Tell me what DevSummit is," DevSummit is the re-branded name of a show we used to call TechCon. And so, TechCon was the show we had every fall. And it used to be in Santa Clara for many years and we moved it to San Jose the last couple of years. So, what's new is old, what's old is new. It's the TechCon show that we're now targeting really more towards... Broadly towards developers, although I would say we think 60% of the folks who have registered are self-proclaimed or self-identified software types, versus about 40% hardware types. 22:54 RENE HAAS: We've got about 4000, 5000 people already pre-registered. We think we'll have a bit more when the time comes. It will be very broad, as Arm typically is in nature. We'll be talking about things like cloud native, chip design, autonomous vehicles. It will run the gamut of all the areas that we're involved in, relative to what it takes to integrate Arm IP and an SoC and what do you need to know about hardware libraries and partners in that space, versus everything around open source software and popular development tools and operating environments that we just talked about on the software space. There will be a lot of emphasis around autonomous, which is a pretty hot area. A lot of areas also around cloud native. You'll see the typical key notes from Simon, myself and some of the other leaders inside of Arm. I would also encourage folks to tune in because there will be some special surprise guests. I won't... 23:56 PETE BERNARD: I can imagine. 23:57 RENE HAAS: Give that away at this point of time, but it should be a very, very interesting and fun event. We have our annual Arm partner meeting every August. I think you've been to it. It's not a public event, it's an NDA event. But I bring that up just in the context of... We've had one rodeo with doing this thing virtually. So I'd like to think we've got some good practice in terms of things that... The dos and don't-'s in terms of doing something from a virtual standpoint. But yeah, it should be very, very good. We're looking forward to it. 24:26 PETE BERNARD: Cool. Yeah, it's interesting, Microsoft's done a number of events now virtual and I don't think we published the data but my understanding is the engagement we get because it's virtualized, we get so much broader engagement, we get so many more people quote, unquote, "attending" and engaged in the content than you would if it was only a... You had to get on a plane and go somewhere. So I think one of the nice by-products, if there is a nice by-product out of all this craziness, is we are all building more muscle about how to enable people to be more engaged regardless of where they are. And especially when you talk about developers, developers everywhere in the world and there should be. And now to be able to enable them to plug in and get educated and learn some new things, that's a fantastic by-product. 25:13 RENE HAAS: Yeah, yeah. No, you're completely right. We'd love to do these events live versus virtual, but when I think about the size of the developer community that exists... Arm is a fairly broad platform, as you know, and it would be really hard to figure out events that could bring all the potential developers who work on Arm... And it's all over the place. There are apps developers, there are kernel developers, there are people who do open source software, it's a broad, broad community. So we're actually kind of excited to do this thing virtually. It'll be a bit of a lab test to see how that works in terms of reaching the development community in a virtual way, but we're looking forward to it. 25:53 PETE BERNARD: Cool, awesome. Well, lots of stuff going on at Arm these days. And so it was great again to connect, Rene. I think hopefully we'll keep in touch here as things transform into Nvidia landscape. Maybe you'll get those extra years on your seniority. [chuckle] But that would be great. 26:14 RENE HAAS: I should get some credit somehow for that. I am going to talk to Jensen about that next time I have our consultation with him. 26:21 PETE BERNARD: Yeah. Cool. Well, good. Any last closing thoughts? It sounds like we've really covered [26:27] ____ here today. 26:29 RENE HAAS: [26:29] ____ I appreciate it and [26:29] ____ as I mentioned, I was listening to some of the podcasts you had done prior and I really enjoyed them and I'm very, very honored on behalf of Arm to join you and be part of what you're building here. It's really cool. 26:43 PETE BERNARD: Sounds good. Alright, Rene. Well, take care and I'm sure our paths will cross again. 26:48 RENE HAAS: Alright, great. Thanks. 26:50 PETE BERNARD: Alright, take care. Thanks.

IBM Research recently introduced their perspective on a machine learning paradigm called Federated Learning in which multiple parties can all participate in training a single model with a shared goal. You can use data that is distributed between competitors, or even data distributed in one company across multiple geographies. They can participate in this so securely without sharing their raw data, and consequently get models that are much more generalizable than they would otherwise be able to achieve on their own.Links related to this episode: Nathalie Baracaldo, IBM Research - AI Security & Privacy Private federated learning - Learn together without sharing data Federated Learning Part 2

The #AI Eye: IBM (NYSE: $IBM) Readies Businesses for AI by Bringing Enhancements to Cloud Pak, HPE (NYSE: $HPE) Announces ML Ops Solution

The #AI Eye: IBM (NYSE: $IBM) Readies Businesses for AI by Bringing Enhancements to Cloud Pak, HPE (NYSE: $HPE) Announces ML Ops Solution