Podcasts about GraphQL

Swimming against the current sometimes leads to unexpected treasures. In this fascinating conversation, Adam Fortuna reveals how migrating Hardcover—a social network for readers with 30,000 users—from Next.js back to Ruby on Rails delivered surprising performance improvements and development simplicity.The journey begins with Adam explaining how Hardcover originated as a response to Goodreads shutting down their API. As a longtime Rails developer who initially chose Next.js for its server-side rendering capabilities, Adam found himself drawn back to Rails once modern tools made it viable to combine Rails' backend strengths with React's frontend interactivity. The migration wasn't a complete rewrite—they preserved their React components while replacing GraphQL with ActiveRecord—and unexpectedly saw significant improvements in page load speeds and SEO rankings.At the heart of this technical evolution is Inertia.js, which Adam describes as "the missing piece for Rails for a long time." This elegant solution allows direct connections between Rails controllers and React components without duplicating routes, creating a seamless developer experience. We dive into the challenges they faced, particularly with generating Open Graph images and handling API abuse, and how they solved these problems with pragmatic hybrid approaches.The conversation takes an exciting turn as Adam discusses their work on book recommendation engines, combining collaborative filtering with content analysis to help readers discover their next favorite book. As someone currently enjoying the Dungeon Crawler Carl series (described as "RPG mixed with Hitchhiker's Guide"), Adam's passion for both books and elegant technical solutions shines throughout.Listen in as we explore how going against conventional wisdom sometimes leads to better outcomes, and discover why Hardcover is now being open-sourced to invite community collaboration. Whether you're interested in Rails, JavaScript frameworks, or book recommendations, this episode offers valuable insights into making technical decisions based on real-world results rather than following trends.Linkshttps://hardcover.app/blog/part-1-how-we-fell-out-of-love-with-next-js-and-back-in-love-with-ruby-on-rails-inertia-jshttps://adamfortuna.com/https://bsky.app/profile/adamfortuna.comSend us some love.HoneybadgerHoneybadger is an application health monitoring tool built by developers for developers.JudoscaleAutoscaling that actually works. Take control of your cloud hosting.Disclaimer: This post contains affiliate links. If you make a purchase, I may receive a commission at no extra cost to you.Support the show

Episode SummaryJeremy Snyder is the co-founder and CEO of FireTail, a company that enables organizations to adopt AI safely without sacrificing speed or innovation. In this conversation, Jeremy shares his deep expertise in API and AI security, highlighting the second wave of cloud adoption and his pivotal experiences at AWS during key moments in its growth from startup onwards.Show NotesIn this episode of The Secure Developer, host Danny Allan sits down with Jeremy Snyder, the Co-founder and CEO of FireTail, to unravel the complexities of API security and explore its critical intersection with the burgeoning field of Artificial Intelligence. Jeremy brings a wealth of experience, tracing his journey from early days in computational linguistics and IT infrastructure, through a pivotal period at AWS during its startup phase, to eventually co-founding FireTail to address the escalating challenges in API security driven by modern, decoupled software architectures.The conversation dives deep into the common pitfalls and crucial best practices for securing APIs. Jeremy clearly distinguishes between authentication (verifying identity) and authorization (defining permissions), emphasizing that failures in authorization are a leading cause of API-related data breaches. He sheds light on vulnerabilities like Broken Object-Level Authorization (BOLA), explaining how seemingly innocuous practices like using sequential integer IDs can expose entire datasets if server-side checks are missed. The discussion also touches on the discoverability of backend APIs and the persistent challenges surrounding multi-factor authentication, including the human element in security weaknesses like SIM swapping.Looking at current trends, Jeremy shares insights from FireTail's ongoing research, including their annual "State of API Security" report, which has uncovered novel attack vectors such as attempts to deploy malware via API calls. A significant portion of the discussion focuses on the new frontier of AI security, where APIs serve as the primary conduit for interaction—and potential exploitation. Jeremy details how AI systems and LLM integrations introduce new risks, citing a real-world example of how a vulnerability in an AI's web crawler API could be leveraged for DDoS attacks. He speculates on the future evolution of APIs, suggesting that technologies like GraphQL might become more prevalent to accommodate the non-deterministic and data-hungry nature of AI agents. Despite the evolving threats, Jeremy concludes with an optimistic view, noting that the gap between business adoption of new technologies and security teams' responses is encouragingly shrinking, leading to more proactive and integrated security practices.LinksFireTailRapid7Snyk - The Developer Security Company Follow UsOur WebsiteOur LinkedIn

React Core team member Dan Abramov joins us to explore "JSX over the wire" and the evolving architecture of React Server Components. We dive into the shift from traditional REST APIs to screen-specific data shaping, the concept of Backend for Frontend (BFF), and why centering UI around the user experience—not server/client boundaries—matters more than ever. Links https://danabra.mov https://github.com/gaearon https://bsky.app/profile/danabra.mov https://overreacted.io https://www.youtube.com/@danabramov Resources JSX Over The Wire: https://overreacted.io/jsx-over-the-wire/ Impossible Components: https://overreacted.io/impossible-components/ What Does "use client" Do?: https://overreacted.io/what-does-use-client-do/ Our Journey With Caching: https://nextjs.org/blog/our-journey-with-caching https://parceljs.org https://nextjs.org/docs/app We want to hear from you! How did you find us? Did you see us on Twitter? In a newsletter? Or maybe we were recommended by a friend? Let us know by sending an email to our producer, Emily, at emily.kochanekketner@logrocket.com (mailto:emily.kochanekketner@logrocket.com), or tweet at us at PodRocketPod (https://twitter.com/PodRocketpod). Follow us. Get free stickers. Follow us on Apple Podcasts, fill out this form (https://podrocket.logrocket.com/get-podrocket-stickers), and we'll send you free PodRocket stickers! What does LogRocket do? LogRocket provides AI-first session replay and analytics that surfaces the UX and technical issues impacting user experiences. Start understand where your users are struggling by trying it for free at [LogRocket.com]. Try LogRocket for free today.(https://logrocket.com/signup/?pdr) Special Guest: Dan Abramov.

Conosciamo Patrick Arminio, Founding Engineer presso FastAPI Labs, creatore di Strawberry e presidente di Python Italia. Iniziamo con una panoramica di GraphQL e REST API. Chiudiamo poi con la nuovissima FastAPI Labs, avventura imprenditoriale di Patrick Arminio e Sebastián Ramírez (tiangolo).

Join Pascal and Sabrina on the latest Meta Tech Podcast episode as they discuss the evolution and future of GraphQL. From client-side consistency to innovative APIs, learn how GraphQL is making developers' lives easier and enhancing user experiences. Discover surprising insights into the challenges of building a mobile GraphQL platform and how it's transforming product development at Meta.  Got feedback? Send it to us on Threads (https://threads.net/@metatechpod), Instagram (https://instagram.com/metatechpod) and don't forget to follow our host Pascal (https://mastodon.social/@passy, https://threads.net/@passy_). Fancy working with us? Check out https://www.metacareers.com/. Links GraphQL: https://graphql.org/  Relay: https://relay.dev/  Sabrina at GraphQL Conf 2024: https://www.youtube.com/watch?v=PGBC-0E-kco  Timestamps Intro 0:06 Introduction Sabrina 1:42 Sabrina's team 2:47 What's GraphQL? 3:18 Relay and Mobile GraphQL Clients 4:01 GraphQL Consistency Engine 4:54 Pando Mobile GraphQL Client 7:16 Interfacing with Pando 8:03 Code generation 9:14 Inventing new features 10:43 The hidden complexity behind pagination 11:52 Working inside the GraphQL spec 16:00 Complexity tradeoffs 18:30 State of GraphQL at Meta 21:16 Measuring success 24:58 Optimistic Mutations 27:31 Collaboration model 31:42 Preventing early adoption 34:43 The challenge of migrating FBApp 37:10 What's next for mobile GraphQL? 40:22 Outro 41:54  

El próximo 21 de marzo, a las 19:30h, el Centro Cívico de Canido se convertirá en el punto de encuentro ideal para todos aquellos desarrolladores y marketers interesados en llevar sus sitios web al siguiente nivel. El taller "Revolucionando WordPress: Arquitectura Headless con Next.js, GraphQL y Vercel" promete ser una experiencia transformadora, donde los participantes podrán descubrir cómo modernizar sus sitios web basados en WordPress mediante una arquitectura headless. Néstor López, Platform Engineer, será el ponente encargado de guiar a los asistentes en este proceso de transformación, mostrando cómo hacer que WordPress sea más rápido, seguro y escalable mediante herramientas de vanguardia como Next.js, GraphQL y Vercel. El taller está dirigido tanto a desarrolladores como a profesionales del marketing digital, quienes podrán aprender cómo optimizar la estructura de sus webs para una mejor experiencia de usuario y un rendimiento superior. Como es habitual en estos eventos, al finalizar la parte técnica, los asistentes podrán disfrutar del esperado #MomentoNetworking, donde podrán relajarse, compartir experiencias y conocer a otros profesionales del sector mientras disfrutan de pinchos y cervezas, cortesía de los patrocinadores Raiola Networks.

Prisma started as a GraphQL backend and pivoted into one of the most widely used ORMs in the world. Now, they've launched Prisma Postgres, and CEO Søren Bramer Schmidt is here to break down the journey, the challenges, and the massive technical innovations behind it—including bare-metal servers, Firecracker microVMs, and unikernels. If you care about databases, performance, or scaling, this one's for you.Want to learn more Postgres? Check out my Postgres course: https://masteringpostgres.com.Follow Søren:Twitter: https://twitter.com/sorenbsGitHub: https://github.com/prisma/prismaPrisma Postgres: https://www.prisma.io/postgresFollow Aaron:Twitter:  https://twitter.com/aarondfrancis LinkedIn: https://www.linkedin.com/in/aarondfrancisWebsite: https://aaronfrancis.com - find articles, podcasts, courses, and more.Chapters:00:00 - Introduction01:15 - The Origins of Prisma: From GraphQL to ORM02:55 - Why Firebase & Parse Inspired Prisma04:04 - The Pivot: From GraphQL to Prisma ORM06:00 - Why They Abandoned Backend-as-a-Service08:07 - The Open Source Business Model Debate10:15 - The Challenges of Monetizing an ORM12:42 - Building Prisma Accelerate & Pulse14:55 - How Prisma Accelerate Optimizes Database Access17:00 - Real-Time Database Updates with Prisma Pulse20:03 - How Prisma Pulse Handles Change Data Capture (CDC)23:15 - Users Wanted a Hosted Database (Even When Prisma Didn't)25:40 - Why Prisma Finally Launched Prisma Postgres27:32 - Unikernels, Firecracker MicroVMs & Running Millions of Databases31:10 - Bare Metal Servers vs. AWS: The Controversial Choice34:40 - How Prisma Routes Queries for Low Latency38:02 - Scaling, Cost Efficiency & Performance Benefits42:10 - The Prisma Postgres Roadmap & Future Features45:30 - Why Prisma is Competing with AWS & The Big Cloud Players48:05 - Final Thoughts & Where to Learn More

Max Stoiber is Co-Founder & CEO ofStellate, the GraphQL edge platformrecently acquired by Shopify.In this episode, we discuss:The Stellate journey from idea to initial traction to acquisitionThe market size (and limitations) for GraphQL, APIs, and DevToolsHow he ran a top-notch acquisition process for StellateWhy startups fail

Karina Nguyen leads research at OpenAI, where she's been pivotal in developing groundbreaking products like Canvas, Tasks, and the o1 language model. Before OpenAI, Karina was at Anthropic, where she led post-training and evaluation work for Claude 3 models, created a document upload feature with 100,000 context windows, and contributed to numerous other innovations. With experience as an engineer at the New York Times and as a designer at Dropbox and Square, Karina has a rare firsthand perspective on the cutting edge of AI and large language models. In our conversation, we discuss:• How OpenAI builds product• What people misunderstand about AI model training• Differences between how OpenAI and Anthropic operate• The role of synthetic data in model development• How to build trust between users and AI models• Why she moved from engineering to research• Much more—Brought to you by:• Enterpret—Transform customer feedback into product growth• Vanta—Automate compliance. Simplify security• Loom—The easiest screen recorder you'll ever use—Find the transcript at: https://www.lennysnewsletter.com/p/why-soft-skills-are-the-future-of-work-karina-nguyen—Where to find Karina Nguyen:• X: https://x.com/karinanguyen_• LinkedIn: https://www.linkedin.com/in/karinanguyen28• Website: https://karinanguyen.com/—Where to find Lenny:• Newsletter: https://www.lennysnewsletter.com• X: https://twitter.com/lennysan• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/—In this episode, we cover:(00:00) Introduction to Karina Nguyen(04:42) Challenges in model training(08:21) Synthetic data and its importance(12:38) Creating Canvas(18:33) Day-to-day operations at OpenAI(20:28) Writing evaluations(23:22) Prototyping and product development(26:57) Building Canvas and Tasks(33:34) Understanding the job of a researcher(35:36) The future of AI and its impact on work and education(42:15) Soft skills in the age of AI(47:50) AI's role in creativity and strategy development(53:34) Comparing Anthropic and OpenAI(57:11) Innovations and future visions(01:07:13) The potential of AI agents(01:11:36) Final thoughts and career advice—Referenced:• What's in your stack: The state of tech tools in 2025: https://www.lennysnewsletter.com/p/whats-in-your-stack-the-state-of• Anthropic: https://www.anthropic.com/• OpenAI: https://openai.com/• What is synthetic data—and how can it help you competitively?: https://mitsloan.mit.edu/ideas-made-to-matter/what-synthetic-data-and-how-can-it-help-you-competitively• GPQA: https://datatunnel.io/glossary/gpqa/• Canvas: https://openai.com/index/introducing-canvas/• Barret Zoph on LinkedIn: https://www.linkedin.com/in/barret-zoph-65990543/• Mira Murati on LinkedIn: https://www.linkedin.com/in/mira-murati-4b39a066/• JSON Schema: https://json-schema.org/• Anthropic—100K Context Windows: https://www.anthropic.com/news/100k-context-windows• Claude 3 Haiku: https://www.anthropic.com/news/claude-3-haiku• A.I. Chatbots Defeated Doctors at Diagnosing Illness: https://www.nytimes.com/2024/11/17/health/chatgpt-ai-doctors-diagnosis.html• Cursor: https://www.cursor.com/• How AI will impact product management: https://www.lennysnewsletter.com/p/how-ai-will-impact-product-management• Lee Byron on LinkedIn: https://www.linkedin.com/in/lee-byron/• GraphQL: https://graphql.org/• Claude in Slack: https://www.anthropic.com/claude-in-slack• Sam Altman on X: https://x.com/sama• Jakub Pachocki on LinkedIn: https://www.linkedin.com/in/jakub-pachocki/• Lennybot: https://www.lennybot.com/• ElevenLabs: https://elevenlabs.io/• Westworld on Prime Video: https://www.amazon.com/Westworld-Season-1/dp/B01N05UD06• A conversation with OpenAI's CPO Kevin Weil, Anthropic's CPO Mike Krieger, and Sarah Guo: https://www.youtube.com/watch?v=IxkvVZua28k• Tuple: https://tuple.app/• How Shopify builds a high-intensity culture | Farhan Thawar (VP and Head of Eng): https://www.lennysnewsletter.com/p/how-shopify-builds-a-high-intensity-culture-farhan-thawar—Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.—Lenny may be an investor in the companies discussed. Get full access to Lenny's Newsletter at www.lennysnewsletter.com/subscribe

Today we are talking about GraphQL, Drupal Decoupled, and What to do with them with guest Jesus Manuel Olivas. We'll also cover CORS UI as our module of the week. For show notes visit: https://www.talkingDrupal.com/486 Topics What is GraphQL How do you use GraphQL with Drupal Would you use GraphQL without a headless theme Do you need additional server requirements What are some of your favorite GraphQL modules What caused the change from v3 to v4 What is meant by Drupal Decoupled What are the best use cases How do you handle caching and performance How do you handle roles and permissions Do you think AI has made decoupled more interesting Resources GraphQL GraphQL Compose GraphQL Compose Preview GraphQL Compose Webform GraphQL Compose Fragments Swagger UI Custom Field Drupal Decoupled Guests Jesus Manuel Olivas - drupal-decoupled.octahedroid.com jmolivas Hosts Nic Laflin - nLighteneddevelopment.com nicxvan John Picozzi - epam.com johnpicozzi Scott Weston - scott-weston MOTW Correspondent Martin Anderson-Clutz - mandclu.com mandclu Brief description: Have you ever wanted to control your site's Cross-Origin Resource Sharing (aka CORS) configuration, directly within the Drupal admin UI? There's a module for that. Module name/project name: CORS UI Brief history How old: created in Sep 2016 by Sam Becker (sam152), a prolific module maintainer in his own right, though the most recent release is by Matt Glaman, who has been on this show and will need no introduction for many of our listeners Versions available: 8.x-1.2 which supports Drupal 9, 10, and 11 Maintainership Actively maintained Security coverage Number of open issues: 2 open issues, 1 of which is a bug, and also has a patch available Usage stats: 274 sites according to drupal.org Module features and usage By default cross-origin requests to Drupal applications will be denied. Since Drupal 8.2 you can add a section to your site's services.yml file to enable responses, and specify what headers, methods, and origins should be supported This module provides a form within Drupal to control these values. This could be helpful if, for example, these values need to change on a frequent basis, or for less technical users who are experimenting with a headless architecture. I should note that the bug mentioned earlier throws a fatal error in PHP 8, but is a simple fix. So if you want to try out this module, make sure you apply the patch.

Hoje o papo é sobre GraphQL no mobile. Neste episódio, conversamos sobre o histórico do GraphQL, desde os problemas que ele veio para resolver, até ecossistema, o que é (e o que não é) responsabilidade do GraphQL, vantagens e desvantagens do uso de GraphQL versus REST, e muito mais. Vem ver quem participou desse papo: André David, o host que já é o tradicional co-host Vinny Neves, Líder de Front-End na Alura Yago Oliveira, Coordenador de Conteúdo Técnico na Alura William Bezerra, Instrutor na Alura e Engenheiro Sênior no QuintoAndar

Søren Bramer Schmidt, co-founder and CEO of Prisma, joins us to discuss the journey of building one of the largest developer communities in DevTools. Søren shares how Prisma's deliberate strategies have shaped its growth, feature prioritization, and the launch of new products like Prisma Postgres. We also explore the challenges of managing a vast user base and how Prisma is adapting to shifts in application development.We discuss:How intentional partnerships with educators and influencers fueled Prisma's early growth.Strategies to engage the GraphQL community and gain visibility on platforms like Hacker News.Managing a large developer community while balancing innovation with stability.The evolution from Graphcool to Prisma ORM, including lessons from early pivots.Launching Prisma Postgres and how community feedback influenced its development.Implementing a simple, usage-based pricing model and reducing infrastructure costs through self-hosting.This episode is brought to you by WorkOS. If you're thinking about selling to enterprise customers, WorkOS can help you add enterprise features like Single Sign On and audit logs. https://workos.com/Links:Prisma (https://www.prisma.io/)Prisma Postgres (https://www.prisma.io/postgres)Feldera (https://feldera.com/)

In this episode of Compressed FM, Dustin Goodman shares insights from his journey from IC to engineering manager at companies like ClickUp and This Dot. The conversation explores the nuances of technical leadership, team dynamics, and the importance of understanding personal values in management. The discussion then shifts to a deep dive into React Server Components, examining their implementation challenges and potential impact on the framework ecosystem. SponsorsWix Studio combines the best of both worlds—intuitive design tools for clients and full-stack flexibility for developers. Customize every detail with your own code and take control of your projects.Chapter Marks00:00:00 - Intro00:00:42 - Sponsor: Wix Studio00:01:33 - Engineering Management Journey00:05:11 - Managing Different Experience Levels00:07:14 - Technical Skills in Management00:09:27 - Should Managers Code?00:12:19 - Managing Up vs Managing Down00:17:27 - Team Values Discussion00:20:11 - Strengths and Management Styles00:26:07 - React Server Components Introduction00:29:27 - RSC Implementation Challenges00:34:34 - GraphQL and Server Components00:39:13 - Future of React Frameworks00:43:10 - Vite 6 Discussion00:47:52 - React Community Evolution00:51:21 - Picks and PlugsAmy Dutton:Pick: Browse AI (web scraping tool with AI capabilities)Plug: Advent of CSS and Advent of JavaScript (24 coding challenges in December)Dustin Goodman:Pick: Cursor (AI-powered code editor)Plug: "Engineering Management for the Rest of Us" by Sarah DrasnerBrad Garropy:Pick: Helldivers 2 (video game)Plug: Raycast extension for Stripe (automatically fills checkouts with test cards)01:00:14 - Show Wrap-upLinksBooks Mentioned:"The Manager's Path" by Camille Fournier"Engineering Management for the Rest of Us" by Sarah DrasnerTools & Software:Wix StudioBrowse AICursor (code editor)RaycastRaycast Stripe extensionVite 6Next.jsSocial/Community:BlueSky (Brad and Amy)Bytes NewsletterConnectTech conferencePeople Referenced:Ryan BurgessGergely OroszTracy LeeDan AbramovTanner LindsleyJohn LindquistDavid KhourshidAssessment Tools:Clifton StrengthsFinderAPIs/Documentation:Stripe test cards documentationReact Server Components documentationVite documentationProjects:Advent of CSS (adventofcss.com)Advent of JavaScript (adventofjs.com) 

In this special episode of Watson Weekly, Rick Watson is joined by Kelly Goetsch, a Commercetools Advisor and industry thought leader. Kelly shares his unique insights into the evolving landscape of e-commerce, focusing on the intersection of technology and healthcare. Together, they explore key topics like consumer behavior trends, the growing role of composable commerce, and the untapped opportunities in health tech. From tackling HIPAA compliance to redefining retail experiences, this episode dives deep into the transformative potential of technology across industries. Don't miss this engaging discussion packed with expertise and forward-thinking strategies.About Kelly - Kelly Goetsch is a commercetools Advisor. Until January 2025, Kelly was the company's Chief Strategy Officer, and prior to that, he served as the Chief Product Officer at commercetools for nearly six years. Goetsch is an industry thought-leader who champions the MACH (Microservices, API, Cloud-Native, and Headless) movement, and co-founded the MACH Alliance, a group of 100+ independent, future-thinking tech companies dedicated to advocating for open, best-of-breed technology ecosystems. Prior to commercetools, Goetsch held senior-level product development and go-to-market responsibilities at Oracle and held the role of Senior Architect ATG (acquired by Oracle), where he was instrumental to 31 large-scale ATG implementationsHe is the author of four books - GraphQL for Modern Commerce (O'Reilly, 2020), APIs for Modern Commerce (O'Reilly, 2017), Microservices for Modern Commerce (O'Reilly, 2016) and E-Commerce in the Cloud (O'Reilly, 2014). He holds three patents, including one key to distributed computing.

Applications for the 2025 AI Engineer Summit are up, and you can save the date for AIE Singapore in April and AIE World's Fair 2025 in June.Happy new year, and thanks for 100 great episodes! Please let us know what you want to see/hear for the next 100!Full YouTube Episode with Slides/ChartsLike and subscribe and hit that bell to get notifs!Timestamps* 00:00 Welcome to the 100th Episode!* 00:19 Reflecting on the Journey* 00:47 AI Engineering: The Rise and Impact* 03:15 Latent Space Live and AI Conferences* 09:44 The Competitive AI Landscape* 21:45 Synthetic Data and Future Trends* 35:53 Creative Writing with AI* 36:12 Legal and Ethical Issues in AI* 38:18 The Data War: GPU Poor vs. GPU Rich* 39:12 The Rise of GPU Ultra Rich* 40:47 Emerging Trends in AI Models* 45:31 The Multi-Modality War* 01:05:31 The Future of AI Benchmarks* 01:13:17 Pionote and Frontier Models* 01:13:47 Niche Models and Base Models* 01:14:30 State Space Models and RWKB* 01:15:48 Inference Race and Price Wars* 01:22:16 Major AI Themes of the Year* 01:22:48 AI Rewind: January to March* 01:26:42 AI Rewind: April to June* 01:33:12 AI Rewind: July to September* 01:34:59 AI Rewind: October to December* 01:39:53 Year-End Reflections and PredictionsTranscript[00:00:00] Welcome to the 100th Episode![00:00:00] Alessio: Hey everyone, welcome to the Latent Space Podcast. This is Alessio, partner and CTO at Decibel Partners, and I'm joined by my co host Swyx for the 100th time today.[00:00:12] swyx: Yay, um, and we're so glad that, yeah, you know, everyone has, uh, followed us in this journey. How do you feel about it? 100 episodes.[00:00:19] Alessio: Yeah, I know.[00:00:19] Reflecting on the Journey[00:00:19] Alessio: Almost two years that we've been doing this. We've had four different studios. Uh, we've had a lot of changes. You know, we used to do this lightning round. When we first started that we didn't like, and we tried to change the question. The answer[00:00:32] swyx: was cursor and perplexity.[00:00:34] Alessio: Yeah, I love mid journey. It's like, do you really not like anything else?[00:00:38] Alessio: Like what's, what's the unique thing? And I think, yeah, we, we've also had a lot more research driven content. You know, we had like 3DAO, we had, you know. Jeremy Howard, we had more folks like that.[00:00:47] AI Engineering: The Rise and Impact[00:00:47] Alessio: I think we want to do more of that too in the new year, like having, uh, some of the Gemini folks, both on the research and the applied side.[00:00:54] Alessio: Yeah, but it's been a ton of fun. I think we both started, I wouldn't say as a joke, we were kind of like, Oh, we [00:01:00] should do a podcast. And I think we kind of caught the right wave, obviously. And I think your rise of the AI engineer posts just kind of get people. Sombra to congregate, and then the AI engineer summit.[00:01:11] Alessio: And that's why when I look at our growth chart, it's kind of like a proxy for like the AI engineering industry as a whole, which is almost like, like, even if we don't do that much, we keep growing just because there's so many more AI engineers. So did you expect that growth or did you expect that would take longer for like the AI engineer thing to kind of like become, you know, everybody talks about it today.[00:01:32] swyx: So, the sign of that, that we have won is that Gartner puts it at the top of the hype curve right now. So Gartner has called the peak in AI engineering. I did not expect, um, to what level. I knew that I was correct when I called it because I did like two months of work going into that. But I didn't know, You know, how quickly it could happen, and obviously there's a chance that I could be wrong.[00:01:52] swyx: But I think, like, most people have come around to that concept. Hacker News hates it, which is a good sign. But there's enough people that have defined it, you know, GitHub, when [00:02:00] they launched GitHub Models, which is the Hugging Face clone, they put AI engineers in the banner, like, above the fold, like, in big So I think it's like kind of arrived as a meaningful and useful definition.[00:02:12] swyx: I think people are trying to figure out where the boundaries are. I think that was a lot of the quote unquote drama that happens behind the scenes at the World's Fair in June. Because I think there's a lot of doubt or questions about where ML engineering stops and AI engineering starts. That's a useful debate to be had.[00:02:29] swyx: In some sense, I actually anticipated that as well. So I intentionally did not. Put a firm definition there because most of the successful definitions are necessarily underspecified and it's actually useful to have different perspectives and you don't have to specify everything from the outset.[00:02:45] Alessio: Yeah, I was at um, AWS reInvent and the line to get into like the AI engineering talk, so to speak, which is, you know, applied AI and whatnot was like, there are like hundreds of people just in line to go in.[00:02:56] Alessio: I think that's kind of what enabled me. People, right? Which is what [00:03:00] you kind of talked about. It's like, Hey, look, you don't actually need a PhD, just, yeah, just use the model. And then maybe we'll talk about some of the blind spots that you get as an engineer with the earlier posts that we also had on on the sub stack.[00:03:11] Alessio: But yeah, it's been a heck of a heck of a two years.[00:03:14] swyx: Yeah.[00:03:15] Latent Space Live and AI Conferences[00:03:15] swyx: You know, I was, I was trying to view the conference as like, so NeurIPS is I think like 16, 17, 000 people. And the Latent Space Live event that we held there was 950 signups. I think. The AI world, the ML world is still very much research heavy. And that's as it should be because ML is very much in a research phase.[00:03:34] swyx: But as we move this entire field into production, I think that ratio inverts into becoming more engineering heavy. So at least I think engineering should be on the same level, even if it's never as prestigious, like it'll always be low status because at the end of the day, you're manipulating APIs or whatever.[00:03:51] swyx: But Yeah, wrapping GPTs, but there's going to be an increasing stack and an art to doing these, these things well. And I, you know, I [00:04:00] think that's what we're focusing on for the podcast, the conference and basically everything I do seems to make sense. And I think we'll, we'll talk about the trends here that apply.[00:04:09] swyx: It's, it's just very strange. So, like, there's a mix of, like, keeping on top of research while not being a researcher and then putting that research into production. So, like, people always ask me, like, why are you covering Neuralibs? Like, this is a ML research conference and I'm like, well, yeah, I mean, we're not going to, to like, understand everything Or reproduce every single paper, but the stuff that is being found here is going to make it through into production at some point, you hope.[00:04:32] swyx: And then actually like when I talk to the researchers, they actually get very excited because they're like, oh, you guys are actually caring about how this goes into production and that's what they really really want. The measure of success is previously just peer review, right? Getting 7s and 8s on their um, Academic review conferences and stuff like citations is one metric, but money is a better metric.[00:04:51] Alessio: Money is a better metric. Yeah, and there were about 2200 people on the live stream or something like that. Yeah, yeah. Hundred on the live stream. So [00:05:00] I try my best to moderate, but it was a lot spicier in person with Jonathan and, and Dylan. Yeah, that it was in the chat on YouTube.[00:05:06] swyx: I would say that I actually also created.[00:05:09] swyx: Layen Space Live in order to address flaws that are perceived in academic conferences. This is not NeurIPS specific, it's ICML, NeurIPS. Basically, it's very sort of oriented towards the PhD student, uh, market, job market, right? Like literally all, basically everyone's there to advertise their research and skills and get jobs.[00:05:28] swyx: And then obviously all the, the companies go there to hire them. And I think that's great for the individual researchers, but for people going there to get info is not great because you have to read between the lines, bring a ton of context in order to understand every single paper. So what is missing is effectively what I ended up doing, which is domain by domain, go through and recap the best of the year.[00:05:48] swyx: Survey the field. And there are, like NeurIPS had a, uh, I think ICML had a like a position paper track, NeurIPS added a benchmarks, uh, datasets track. These are ways in which to address that [00:06:00] issue. Uh, there's always workshops as well. Every, every conference has, you know, a last day of workshops and stuff that provide more of an overview.[00:06:06] swyx: But they're not specifically prompted to do so. And I think really, uh, Organizing a conference is just about getting good speakers and giving them the correct prompts. And then they will just go and do that thing and they do a very good job of it. So I think Sarah did a fantastic job with the startups prompt.[00:06:21] swyx: I can't list everybody, but we did best of 2024 in startups, vision, open models. Post transformers, synthetic data, small models, and agents. And then the last one was the, uh, and then we also did a quick one on reasoning with Nathan Lambert. And then the last one, obviously, was the debate that people were very hyped about.[00:06:39] swyx: It was very awkward. And I'm really, really thankful for John Franco, basically, who stepped up to challenge Dylan. Because Dylan was like, yeah, I'll do it. But He was pro scaling. And I think everyone who is like in AI is pro scaling, right? So you need somebody who's ready to publicly say, no, we've hit a wall.[00:06:57] swyx: So that means you're saying Sam Altman's wrong. [00:07:00] You're saying, um, you know, everyone else is wrong. It helps that this was the day before Ilya went on, went up on stage and then said pre training has hit a wall. And data has hit a wall. So actually Jonathan ended up winning, and then Ilya supported that statement, and then Noam Brown on the last day further supported that statement as well.[00:07:17] swyx: So it's kind of interesting that I think the consensus kind of going in was that we're not done scaling, like you should believe in a better lesson. And then, four straight days in a row, you had Sepp Hochreiter, who is the creator of the LSTM, along with everyone's favorite OG in AI, which is Juergen Schmidhuber.[00:07:34] swyx: He said that, um, we're pre trading inside a wall, or like, we've run into a different kind of wall. And then we have, you know John Frankel, Ilya, and then Noam Brown are all saying variations of the same thing, that we have hit some kind of wall in the status quo of what pre trained, scaling large pre trained models has looked like, and we need a new thing.[00:07:54] swyx: And obviously the new thing for people is some make, either people are calling it inference time compute or test time [00:08:00] compute. I think the collective terminology has been inference time, and I think that makes sense because test time, calling it test, meaning, has a very pre trained bias, meaning that the only reason for running inference at all is to test your model.[00:08:11] swyx: That is not true. Right. Yeah. So, so, I quite agree that. OpenAI seems to have adopted, or the community seems to have adopted this terminology of ITC instead of TTC. And that, that makes a lot of sense because like now we care about inference, even right down to compute optimality. Like I actually interviewed this author who recovered or reviewed the Chinchilla paper.[00:08:31] swyx: Chinchilla paper is compute optimal training, but what is not stated in there is it's pre trained compute optimal training. And once you start caring about inference, compute optimal training, you have a different scaling law. And in a way that we did not know last year.[00:08:45] Alessio: I wonder, because John is, he's also on the side of attention is all you need.[00:08:49] Alessio: Like he had the bet with Sasha. So I'm curious, like he doesn't believe in scaling, but he thinks the transformer, I wonder if he's still. So, so,[00:08:56] swyx: so he, obviously everything is nuanced and you know, I told him to play a character [00:09:00] for this debate, right? So he actually does. Yeah. He still, he still believes that we can scale more.[00:09:04] swyx: Uh, he just assumed the character to be very game for, for playing this debate. So even more kudos to him that he assumed a position that he didn't believe in and still won the debate.[00:09:16] Alessio: Get rekt, Dylan. Um, do you just want to quickly run through some of these things? Like, uh, Sarah's presentation, just the highlights.[00:09:24] swyx: Yeah, we can't go through everyone's slides, but I pulled out some things as a factor of, like, stuff that we were going to talk about. And we'll[00:09:30] Alessio: publish[00:09:31] swyx: the rest. Yeah, we'll publish on this feed the best of 2024 in those domains. And hopefully people can benefit from the work that our speakers have done.[00:09:39] swyx: But I think it's, uh, these are just good slides. And I've been, I've been looking for a sort of end of year recaps from, from people.[00:09:44] The Competitive AI Landscape[00:09:44] swyx: The field has progressed a lot. You know, I think the max ELO in 2023 on LMSys used to be 1200 for LMSys ELOs. And now everyone is at least at, uh, 1275 in their ELOs, and this is across Gemini, Chadjibuti, [00:10:00] Grok, O1.[00:10:01] swyx: ai, which with their E Large model, and Enthopic, of course. It's a very, very competitive race. There are multiple Frontier labs all racing, but there is a clear tier zero Frontier. And then there's like a tier one. It's like, I wish I had everything else. Tier zero is extremely competitive. It's effectively now three horse race between Gemini, uh, Anthropic and OpenAI.[00:10:21] swyx: I would say that people are still holding out a candle for XAI. XAI, I think, for some reason, because their API was very slow to roll out, is not included in these metrics. So it's actually quite hard to put on there. As someone who also does charts, XAI is continually snubbed because they don't work well with the benchmarking people.[00:10:42] swyx: Yeah, yeah, yeah. It's a little trivia for why XAI always gets ignored. The other thing is market share. So these are slides from Sarah. We have it up on the screen. It has gone from very heavily open AI. So we have some numbers and estimates. These are from RAMP. Estimates of open AI market share in [00:11:00] December 2023.[00:11:01] swyx: And this is basically, what is it, GPT being 95 percent of production traffic. And I think if you correlate that with stuff that we asked. Harrison Chase on the LangChain episode, it was true. And then CLAUD 3 launched mid middle of this year. I think CLAUD 3 launched in March, CLAUD 3. 5 Sonnet was in June ish.[00:11:23] swyx: And you can start seeing the market share shift towards opening, uh, towards that topic, uh, very, very aggressively. The more recent one is Gemini. So if I scroll down a little bit, this is an even more recent dataset. So RAM's dataset ends in September 2 2. 2024. Gemini has basically launched a price war at the low end, uh, with Gemini Flash, uh, being basically free for personal use.[00:11:44] swyx: Like, I think people don't understand the free tier. It's something like a billion tokens per day. Unless you're trying to abuse it, you cannot really exhaust your free tier on Gemini. They're really trying to get you to use it. They know they're in like third place, um, fourth place, depending how you, how you count.[00:11:58] swyx: And so they're going after [00:12:00] the Lower tier first, and then, you know, maybe the upper tier later, but yeah, Gemini Flash, according to OpenRouter, is now 50 percent of their OpenRouter requests. Obviously, these are the small requests. These are small, cheap requests that are mathematically going to be more.[00:12:15] swyx: The smart ones obviously are still going to OpenAI. But, you know, it's a very, very big shift in the market. Like basically 2023, 2022, To going into 2024 opening has gone from nine five market share to Yeah. Reasonably somewhere between 50 to 75 market share.[00:12:29] Alessio: Yeah. I'm really curious how ramped does the attribution to the model?[00:12:32] Alessio: If it's API, because I think it's all credit card spin. . Well, but it's all, the credit card doesn't say maybe. Maybe the, maybe when they do expenses, they upload the PDF, but yeah, the, the German I think makes sense. I think that was one of my main 2024 takeaways that like. The best small model companies are the large labs, which is not something I would have thought that the open source kind of like long tail would be like the small model.[00:12:53] swyx: Yeah, different sizes of small models we're talking about here, right? Like so small model here for Gemini is AB, [00:13:00] right? Uh, mini. We don't know what the small model size is, but yeah, it's probably in the double digits or maybe single digits, but probably double digits. The open source community has kind of focused on the one to three B size.[00:13:11] swyx: Mm-hmm . Yeah. Maybe[00:13:12] swyx: zero, maybe 0.5 B uh, that's moon dream and that is small for you then, then that's great. It makes sense that we, we have a range for small now, which is like, may, maybe one to five B. Yeah. I'll even put that at, at, at the high end. And so this includes Gemma from Gemini as well. But also includes the Apple Foundation models, which I think Apple Foundation is 3B.[00:13:32] Alessio: Yeah. No, that's great. I mean, I think in the start small just meant cheap. I think today small is actually a more nuanced discussion, you know, that people weren't really having before.[00:13:43] swyx: Yeah, we can keep going. This is a slide that I smiley disagree with Sarah. She's pointing to the scale SEAL leaderboard. I think the Researchers that I talked with at NeurIPS were kind of positive on this because basically you need private test [00:14:00] sets to prevent contamination.[00:14:02] swyx: And Scale is one of maybe three or four people this year that has really made an effort in doing a credible private test set leaderboard. Llama405B does well compared to Gemini and GPT 40. And I think that's good. I would say that. You know, it's good to have an open model that is that big, that does well on those metrics.[00:14:23] swyx: But anyone putting 405B in production will tell you, if you scroll down a little bit to the artificial analysis numbers, that it is very slow and very expensive to infer. Um, it doesn't even fit on like one node. of, uh, of H100s. Cerebras will be happy to tell you they can serve 4 or 5B on their super large chips.[00:14:42] swyx: But, um, you know, if you need to do anything custom to it, you're still kind of constrained. So, is 4 or 5B really that relevant? Like, I think most people are basically saying that they only use 4 or 5B as a teacher model to distill down to something. Even Meta is doing it. So with Lama 3. [00:15:00] 3 launched, they only launched the 70B because they use 4 or 5B to distill the 70B.[00:15:03] swyx: So I don't know if like open source is keeping up. I think they're the, the open source industrial complex is very invested in telling you that the, if the gap is narrowing, I kind of disagree. I think that the gap is widening with O1. I think there are very, very smart people trying to narrow that gap and they should.[00:15:22] swyx: I really wish them success, but you cannot use a chart that is nearing 100 in your saturation chart. And look, the distance between open source and closed source is narrowing. Of course it's going to narrow because you're near 100. This is stupid. But in metrics that matter, is open source narrowing?[00:15:38] swyx: Probably not for O1 for a while. And it's really up to the open source guys to figure out if they can match O1 or not.[00:15:46] Alessio: I think inference time compute is bad for open source just because, you know, Doc can donate the flops at training time, but he cannot donate the flops at inference time. So it's really hard to like actually keep up on that axis.[00:15:59] Alessio: Big, big business [00:16:00] model shift. So I don't know what that means for the GPU clouds. I don't know what that means for the hyperscalers, but obviously the big labs have a lot of advantage. Because, like, it's not a static artifact that you're putting the compute in. You're kind of doing that still, but then you're putting a lot of computed inference too.[00:16:17] swyx: Yeah, yeah, yeah. Um, I mean, Llama4 will be reasoning oriented. We talked with Thomas Shalom. Um, kudos for getting that episode together. That was really nice. Good, well timed. Actually, I connected with the AI meta guy, uh, at NeurIPS, and, um, yeah, we're going to coordinate something for Llama4. Yeah, yeah,[00:16:32] Alessio: and our friend, yeah.[00:16:33] Alessio: Clara Shi just joined to lead the business agent side. So I'm sure we'll have her on in the new year.[00:16:39] swyx: Yeah. So, um, my comment on, on the business model shift, this is super interesting. Apparently it is wide knowledge that OpenAI wanted more than 6. 6 billion dollars for their fundraise. They wanted to raise, you know, higher, and they did not.[00:16:51] swyx: And what that means is basically like, it's very convenient that we're not getting GPT 5, which would have been a larger pre train. We should have a lot of upfront money. And [00:17:00] instead we're, we're converting fixed costs into variable costs, right. And passing it on effectively to the customer. And it's so much easier to take margin there because you can directly attribute it to like, Oh, you're using this more.[00:17:12] swyx: Therefore you, you pay more of the cost and I'll just slap a margin in there. So like that lets you control your growth margin and like tie your. Your spend, or your sort of inference spend, accordingly. And it's just really interesting to, that this change in the sort of inference paradigm has arrived exactly at the same time that the funding environment for pre training is effectively drying up, kind of.[00:17:36] swyx: I feel like maybe the VCs are very in tune with research anyway, so like, they would have noticed this, but, um, it's just interesting.[00:17:43] Alessio: Yeah, and I was looking back at our yearly recap of last year. Yeah. And the big thing was like the mixed trial price fights, you know, and I think now it's almost like there's nowhere to go, like, you know, Gemini Flash is like basically giving it away for free.[00:17:55] Alessio: So I think this is a good way for the labs to generate more revenue and pass down [00:18:00] some of the compute to the customer. I think they're going to[00:18:02] swyx: keep going. I think that 2, will come.[00:18:05] Alessio: Yeah, I know. Totally. I mean, next year, the first thing I'm doing is signing up for Devin. Signing up for the pro chat GBT.[00:18:12] Alessio: Just to try. I just want to see what does it look like to spend a thousand dollars a month on AI?[00:18:17] swyx: Yes. Yes. I think if your, if your, your job is a, at least AI content creator or VC or, you know, someone who, whose job it is to stay on, stay on top of things, you should already be spending like a thousand dollars a month on, on stuff.[00:18:28] swyx: And then obviously easy to spend, hard to use. You have to actually use. The good thing is that actually Google lets you do a lot of stuff for free now. So like deep research. That they just launched. Uses a ton of inference and it's, it's free while it's in preview.[00:18:45] Alessio: Yeah. They need to put that in Lindy.[00:18:47] Alessio: I've been using Lindy lately. I've been a built a bunch of things once we had flow because I liked the new thing. It's pretty good. I even did a phone call assistant. Um, yeah, they just launched Lindy voice. Yeah, I think once [00:19:00] they get advanced voice mode like capability today, still like speech to text, you can kind of tell.[00:19:06] Alessio: Um, but it's good for like reservations and things like that. So I have a meeting prepper thing. And so[00:19:13] swyx: it's good. Okay. I feel like we've, we've covered a lot of stuff. Uh, I, yeah, I, you know, I think We will go over the individual, uh, talks in a separate episode. Uh, I don't want to take too much time with, uh, this stuff, but that suffice to say that there is a lot of progress in each field.[00:19:28] swyx: Uh, we covered vision. Basically this is all like the audience voting for what they wanted. And then I just invited the best people I could find in each audience, especially agents. Um, Graham, who I talked to at ICML in Vienna, he is currently still number one. It's very hard to stay on top of SweetBench.[00:19:45] swyx: OpenHand is currently still number one. switchbench full, which is the hardest one. He had very good thoughts on agents, which I, which I'll highlight for people. Everyone is saying 2025 is the year of agents, just like they said last year. And, uh, but he had [00:20:00] thoughts on like eight parts of what are the frontier problems to solve in agents.[00:20:03] swyx: And so I'll highlight that talk as well.[00:20:05] Alessio: Yeah. The number six, which is the Hacken agents learn more about the environment, has been a Super interesting to us as well, just to think through, because, yeah, how do you put an agent in an enterprise where most things in an enterprise have never been public, you know, a lot of the tooling, like the code bases and things like that.[00:20:23] Alessio: So, yeah, there's not indexing and reg. Well, yeah, but it's more like. You can't really rag things that are not documented. But people know them based on how they've been doing it. You know, so I think there's almost this like, you know, Oh, institutional knowledge. Yeah, the boring word is kind of like a business process extraction.[00:20:38] Alessio: Yeah yeah, I see. It's like, how do you actually understand how these things are done? I see. Um, and I think today the, the problem is that, Yeah, the agents are, that most people are building are good at following instruction, but are not as good as like extracting them from you. Um, so I think that will be a big unlock just to touch quickly on the Jeff Dean thing.[00:20:55] Alessio: I thought it was pretty, I mean, we'll link it in the, in the things, but. I think the main [00:21:00] focus was like, how do you use ML to optimize the systems instead of just focusing on ML to do something else? Yeah, I think speculative decoding, we had, you know, Eugene from RWKB on the podcast before, like he's doing a lot of that with Fetterless AI.[00:21:12] swyx: Everyone is. I would say it's the norm. I'm a little bit uncomfortable with how much it costs, because it does use more of the GPU per call. But because everyone is so keen on fast inference, then yeah, makes sense.[00:21:24] Alessio: Exactly. Um, yeah, but we'll link that. Obviously Jeff is great.[00:21:30] swyx: Jeff is, Jeff's talk was more, it wasn't focused on Gemini.[00:21:33] swyx: I think people got the wrong impression from my tweet. It's more about how Google approaches ML and uses ML to design systems and then systems feedback into ML. And I think this ties in with Lubna's talk.[00:21:45] Synthetic Data and Future Trends[00:21:45] swyx: on synthetic data where it's basically the story of bootstrapping of humans and AI in AI research or AI in production.[00:21:53] swyx: So her talk was on synthetic data, where like how much synthetic data has grown in 2024 in the pre training side, the post training side, [00:22:00] and the eval side. And I think Jeff then also extended it basically to chips, uh, to chip design. So he'd spend a lot of time talking about alpha chip. And most of us in the audience are like, we're not working on hardware, man.[00:22:11] swyx: Like you guys are great. TPU is great. Okay. We'll buy TPUs.[00:22:14] Alessio: And then there was the earlier talk. Yeah. But, and then we have, uh, I don't know if we're calling them essays. What are we calling these? But[00:22:23] swyx: for me, it's just like bonus for late in space supporters, because I feel like they haven't been getting anything.[00:22:29] swyx: And then I wanted a more high frequency way to write stuff. Like that one I wrote in an afternoon. I think basically we now have an answer to what Ilya saw. It's one year since. The blip. And we know what he saw in 2014. We know what he saw in 2024. We think we know what he sees in 2024. He gave some hints and then we have vague indications of what he saw in 2023.[00:22:54] swyx: So that was the Oh, and then 2016 as well, because of this lawsuit with Elon, OpenAI [00:23:00] is publishing emails from Sam's, like, his personal text messages to Siobhan, Zelis, or whatever. So, like, we have emails from Ilya saying, this is what we're seeing in OpenAI, and this is why we need to scale up GPUs. And I think it's very prescient in 2016 to write that.[00:23:16] swyx: And so, like, it is exactly, like, basically his insights. It's him and Greg, basically just kind of driving the scaling up of OpenAI, while they're still playing Dota. They're like, no, like, we see the path here.[00:23:30] Alessio: Yeah, and it's funny, yeah, they even mention, you know, we can only train on 1v1 Dota. We need to train on 5v5, and that takes too many GPUs.[00:23:37] Alessio: Yeah,[00:23:37] swyx: and at least for me, I can speak for myself, like, I didn't see the path from Dota to where we are today. I think even, maybe if you ask them, like, they wouldn't necessarily draw a straight line. Yeah,[00:23:47] Alessio: no, definitely. But I think like that was like the whole idea of almost like the RL and we talked about this with Nathan on his podcast.[00:23:55] Alessio: It's like with RL, you can get very good at specific things, but then you can't really like generalize as much. And I [00:24:00] think the language models are like the opposite, which is like, you're going to throw all this data at them and scale them up, but then you really need to drive them home on a specific task later on.[00:24:08] Alessio: And we'll talk about the open AI reinforcement, fine tuning, um, announcement too, and all of that. But yeah, I think like scale is all you need. That's kind of what Elia will be remembered for. And I think just maybe to clarify on like the pre training is over thing that people love to tweet. I think the point of the talk was like everybody, we're scaling these chips, we're scaling the compute, but like the second ingredient which is data is not scaling at the same rate.[00:24:35] Alessio: So it's not necessarily pre training is over. It's kind of like What got us here won't get us there. In his email, he predicted like 10x growth every two years or something like that. And I think maybe now it's like, you know, you can 10x the chips again, but[00:24:49] swyx: I think it's 10x per year. Was it? I don't know.[00:24:52] Alessio: Exactly. And Moore's law is like 2x. So it's like, you know, much faster than that. And yeah, I like the fossil fuel of AI [00:25:00] analogy. It's kind of like, you know, the little background tokens thing. So the OpenAI reinforcement fine tuning is basically like, instead of fine tuning on data, you fine tune on a reward model.[00:25:09] Alessio: So it's basically like, instead of being data driven, it's like task driven. And I think people have tasks to do, they don't really have a lot of data. So I'm curious to see how that changes, how many people fine tune, because I think this is what people run into. It's like, Oh, you can fine tune llama. And it's like, okay, where do I get the data?[00:25:27] Alessio: To fine tune it on, you know, so it's great that we're moving the thing. And then I really like he had this chart where like, you know, the brain mass and the body mass thing is basically like mammals that scaled linearly by brain and body size, and then humans kind of like broke off the slope. So it's almost like maybe the mammal slope is like the pre training slope.[00:25:46] Alessio: And then the post training slope is like the, the human one.[00:25:49] swyx: Yeah. I wonder what the. I mean, we'll know in 10 years, but I wonder what the y axis is for, for Ilya's SSI. We'll try to get them on.[00:25:57] Alessio: Ilya, if you're listening, you're [00:26:00] welcome here. Yeah, and then he had, you know, what comes next, like agent, synthetic data, inference, compute, I thought all of that was like that.[00:26:05] Alessio: I don't[00:26:05] swyx: think he was dropping any alpha there. Yeah, yeah, yeah.[00:26:07] Alessio: Yeah. Any other new reps? Highlights?[00:26:10] swyx: I think that there was comparatively a lot more work. Oh, by the way, I need to plug that, uh, my friend Yi made this, like, little nice paper. Yeah, that was really[00:26:20] swyx: nice.[00:26:20] swyx: Uh, of, uh, of, like, all the, he's, she called it must read papers of 2024.[00:26:26] swyx: So I laid out some of these at NeurIPS, and it was just gone. Like, everyone just picked it up. Because people are dying for, like, little guidance and visualizations And so, uh, I thought it was really super nice that we got there.[00:26:38] Alessio: Should we do a late in space book for each year? Uh, I thought about it. For each year we should.[00:26:42] Alessio: Coffee table book. Yeah. Yeah. Okay. Put it in the will. Hi, Will. By the way, we haven't introduced you. He's our new, you know, general organist, Jamie. You need to[00:26:52] swyx: pull up more things. One thing I saw that, uh, Okay, one fun one, and then one [00:27:00] more general one. So the fun one is this paper on agent collusion. This is a paper on steganography.[00:27:06] swyx: This is secret collusion among AI agents, multi agent deception via steganography. I tried to go to NeurIPS in order to find these kinds of papers because the real reason Like NeurIPS this year has a lottery system. A lot of people actually even go and don't buy tickets because they just go and attend the side events.[00:27:22] swyx: And then also the people who go and end up crowding around the most popular papers, which you already know and already read them before you showed up to NeurIPS. So the only reason you go there is to talk to the paper authors, but there's like something like 10, 000 other. All these papers out there that, you know, are just people's work that they, that they did on the air and they failed to get attention for one reason or another.[00:27:42] swyx: And this was one of them. Uh, it was like all the way at the back. And this is a deep mind paper that actually focuses on collusion between AI agents, uh, by hiding messages in the text that they generate. Uh, so that's what steganography is. So a very simple example would be the first letter of every word.[00:27:57] swyx: If you Pick that out, you know, and the code sends a [00:28:00] different message than that. But something I've always emphasized is to LLMs, we read left to right. LLMs can read up, down, sideways, you know, in random character order. And it's the same to them as it is to us. So if we were ever to get You know, self motivated, underlined LLMs that we're trying to collaborate to take over the planet.[00:28:19] swyx: This would be how they do it. They spread messages among us in the messages that we generate. And he developed a scaling law for that. So he marked, I'm showing it on screen right now, the emergence of this phenomenon. Basically, for example, for Cypher encoding, GPT 2, Lama 2, mixed trial, GPT 3. 5, zero capabilities, and sudden 4.[00:28:40] swyx: And this is the kind of Jason Wei type emergence properties that people kind of look for. I think what made this paper stand out as well, so he developed the benchmark for steganography collusion, and he also focused on shelling point collusion, which is very low coordination. For agreeing on a decoding encoding format, you kind of need to have some [00:29:00] agreement on that.[00:29:00] swyx: But, but shelling point means like very, very low or almost no coordination. So for example, if I, if I ask someone, if the only message I give you is meet me in New York and you're not aware. Or when you would probably meet me at Grand Central Station. That is the Grand Central Station is a shelling point.[00:29:16] swyx: And it's probably somewhere, somewhere during the day. That is the shelling point of New York is Grand Central. To that extent, shelling points for steganography are things like the, the, the common decoding methods that we talked about. It will be interesting at some point in the future when we are worried about alignment.[00:29:30] swyx: It is not interesting today, but it's interesting that DeepMind is already thinking about this.[00:29:36] Alessio: I think that's like one of the hardest things about NeurIPS. It's like the long tail. I[00:29:41] swyx: found a pricing guy. I'm going to feature him on the podcast. Basically, this guy from NVIDIA worked out the optimal pricing for language models.[00:29:51] swyx: It's basically an econometrics paper at NeurIPS, where everyone else is talking about GPUs. And the guy with the GPUs is[00:29:57] Alessio: talking[00:29:57] swyx: about economics instead. [00:30:00] That was the sort of fun one. So the focus I saw is that model papers at NeurIPS are kind of dead. No one really presents models anymore. It's just data sets.[00:30:12] swyx: This is all the grad students are working on. So like there was a data sets track and then I was looking around like, I was like, you don't need a data sets track because every paper is a data sets paper. And so data sets and benchmarks, they're kind of flip sides of the same thing. So Yeah. Cool. Yeah, if you're a grad student, you're a GPU boy, you kind of work on that.[00:30:30] swyx: And then the, the sort of big model that people walk around and pick the ones that they like, and then they use it in their models. And that's, that's kind of how it develops. I, I feel like, um, like, like you didn't last year, you had people like Hao Tian who worked on Lava, which is take Lama and add Vision.[00:30:47] swyx: And then obviously actually I hired him and he added Vision to Grok. Now he's the Vision Grok guy. This year, I don't think there was any of those.[00:30:55] Alessio: What were the most popular, like, orals? Last year it was like the [00:31:00] Mixed Monarch, I think, was like the most attended. Yeah, uh, I need to look it up. Yeah, I mean, if nothing comes to mind, that's also kind of like an answer in a way.[00:31:10] Alessio: But I think last year there was a lot of interest in, like, furthering models and, like, different architectures and all of that.[00:31:16] swyx: I will say that I felt the orals, oral picks this year were not very good. Either that or maybe it's just a So that's the highlight of how I have changed in terms of how I view papers.[00:31:29] swyx: So like, in my estimation, two of the best papers in this year for datasets or data comp and refined web or fine web. These are two actually industrially used papers, not highlighted for a while. I think DCLM got the spotlight, FineWeb didn't even get the spotlight. So like, it's just that the picks were different.[00:31:48] swyx: But one thing that does get a lot of play that a lot of people are debating is the role that's scheduled. This is the schedule free optimizer paper from Meta from Aaron DeFazio. And this [00:32:00] year in the ML community, there's been a lot of chat about shampoo, soap, all the bathroom amenities for optimizing your learning rates.[00:32:08] swyx: And, uh, most people at the big labs are. Who I asked about this, um, say that it's cute, but it's not something that matters. I don't know, but it's something that was discussed and very, very popular. 4Wars[00:32:19] Alessio: of AI recap maybe, just quickly. Um, where do you want to start? Data?[00:32:26] swyx: So to remind people, this is the 4Wars piece that we did as one of our earlier recaps of this year.[00:32:31] swyx: And the belligerents are on the left, journalists, writers, artists, anyone who owns IP basically, New York Times, Stack Overflow, Reddit, Getty, Sarah Silverman, George RR Martin. Yeah, and I think this year we can add Scarlett Johansson to that side of the fence. So anyone suing, open the eye, basically. I actually wanted to get a snapshot of all the lawsuits.[00:32:52] swyx: I'm sure some lawyer can do it. That's the data quality war. On the right hand side, we have the synthetic data people, and I think we talked about Lumna's talk, you know, [00:33:00] really showing how much synthetic data has come along this year. I think there was a bit of a fight between scale. ai and the synthetic data community, because scale.[00:33:09] swyx: ai published a paper saying that synthetic data doesn't work. Surprise, surprise, scale. ai is the leading vendor of non synthetic data. Only[00:33:17] Alessio: cage free annotated data is useful.[00:33:21] swyx: So I think there's some debate going on there, but I don't think it's much debate anymore that at least synthetic data, for the reasons that are blessed in Luna's talk, Makes sense.[00:33:32] swyx: I don't know if you have any perspectives there.[00:33:34] Alessio: I think, again, going back to the reinforcement fine tuning, I think that will change a little bit how people think about it. I think today people mostly use synthetic data, yeah, for distillation and kind of like fine tuning a smaller model from like a larger model.[00:33:46] Alessio: I'm not super aware of how the frontier labs use it outside of like the rephrase, the web thing that Apple also did. But yeah, I think it'll be. Useful. I think like whether or not that gets us the big [00:34:00] next step, I think that's maybe like TBD, you know, I think people love talking about data because it's like a GPU poor, you know, I think, uh, synthetic data is like something that people can do, you know, so they feel more opinionated about it compared to, yeah, the optimizers stuff, which is like,[00:34:17] swyx: they don't[00:34:17] Alessio: really work[00:34:18] swyx: on.[00:34:18] swyx: I think that there is an angle to the reasoning synthetic data. So this year, we covered in the paper club, the star series of papers. So that's star, Q star, V star. It basically helps you to synthesize reasoning steps, or at least distill reasoning steps from a verifier. And if you look at the OpenAI RFT, API that they released, or that they announced, basically they're asking you to submit graders, or they choose from a preset list of graders.[00:34:49] swyx: Basically It feels like a way to create valid synthetic data for them to fine tune their reasoning paths on. Um, so I think that is another angle where it starts to make sense. And [00:35:00] so like, it's very funny that basically all the data quality wars between Let's say the music industry or like the newspaper publishing industry or the textbooks industry on the big labs.[00:35:11] swyx: It's all of the pre training era. And then like the new era, like the reasoning era, like nobody has any problem with all the reasoning, especially because it's all like sort of math and science oriented with, with very reasonable graders. I think the more interesting next step is how does it generalize beyond STEM?[00:35:27] swyx: We've been using O1 for And I would say like for summarization and creative writing and instruction following, I think it's underrated. I started using O1 in our intro songs before we killed the intro songs, but it's very good at writing lyrics. You know, I can actually say like, I think one of the O1 pro demos.[00:35:46] swyx: All of these things that Noam was showing was that, you know, you can write an entire paragraph or three paragraphs without using the letter A, right?[00:35:53] Creative Writing with AI[00:35:53] swyx: So like, like literally just anything instead of token, like not even token level, character level manipulation and [00:36:00] counting and instruction following. It's, uh, it's very, very strong.[00:36:02] swyx: And so no surprises when I ask it to rhyme, uh, and to, to create song lyrics, it's going to do that very much better than in previous models. So I think it's underrated for creative writing.[00:36:11] Alessio: Yeah.[00:36:12] Legal and Ethical Issues in AI[00:36:12] Alessio: What do you think is the rationale that they're going to have in court when they don't show you the thinking traces of O1, but then they want us to, like, they're getting sued for using other publishers data, you know, but then on their end, they're like, well, you shouldn't be using my data to then train your model.[00:36:29] Alessio: So I'm curious to see how that kind of comes. Yeah, I mean, OPA has[00:36:32] swyx: many ways to publish, to punish people without bringing, taking them to court. Already banned ByteDance for distilling their, their info. And so anyone caught distilling the chain of thought will be just disallowed to continue on, on, on the API.[00:36:44] swyx: And it's fine. It's no big deal. Like, I don't even think that's an issue at all, just because the chain of thoughts are pretty well hidden. Like you have to work very, very hard to, to get it to leak. And then even when it leaks the chain of thought, you don't know if it's, if it's [00:37:00] The bigger concern is actually that there's not that much IP hiding behind it, that Cosign, which we talked about, we talked to him on Dev Day, can just fine tune 4.[00:37:13] swyx: 0 to beat 0. 1 Cloud SONET so far is beating O1 on coding tasks without, at least O1 preview, without being a reasoning model, same for Gemini Pro or Gemini 2. 0. So like, how much is reasoning important? How much of a moat is there in this, like, All of these are proprietary sort of training data that they've presumably accomplished.[00:37:34] swyx: Because even DeepSeek was able to do it. And they had, you know, two months notice to do this, to do R1. So, it's actually unclear how much moat there is. Obviously, you know, if you talk to the Strawberry team, they'll be like, yeah, I mean, we spent the last two years doing this. So, we don't know. And it's going to be Interesting because there'll be a lot of noise from people who say they have inference time compute and actually don't because they just have fancy chain of thought.[00:38:00][00:38:00] swyx: And then there's other people who actually do have very good chain of thought. And you will not see them on the same level as OpenAI because OpenAI has invested a lot in building up the mythology of their team. Um, which makes sense. Like the real answer is somewhere in between.[00:38:13] Alessio: Yeah, I think that's kind of like the main data war story developing.[00:38:18] The Data War: GPU Poor vs. GPU Rich[00:38:18] Alessio: GPU poor versus GPU rich. Yeah. Where do you think we are? I think there was, again, going back to like the small model thing, there was like a time in which the GPU poor were kind of like the rebel faction working on like these models that were like open and small and cheap. And I think today people don't really care as much about GPUs anymore.[00:38:37] Alessio: You also see it in the price of the GPUs. Like, you know, that market is kind of like plummeted because there's people don't want to be, they want to be GPU free. They don't even want to be poor. They just want to be, you know, completely without them. Yeah. How do you think about this war? You[00:38:52] swyx: can tell me about this, but like, I feel like the, the appetite for GPU rich startups, like the, you know, the, the funding plan is we will raise 60 million and [00:39:00] we'll give 50 of that to NVIDIA.[00:39:01] swyx: That is gone, right? Like, no one's, no one's pitching that. This was literally the plan, the exact plan of like, I can name like four or five startups, you know, this time last year. So yeah, GPU rich startups gone.[00:39:12] The Rise of GPU Ultra Rich[00:39:12] swyx: But I think like, The GPU ultra rich, the GPU ultra high net worth is still going. So, um, now we're, you know, we had Leopold's essay on the trillion dollar cluster.[00:39:23] swyx: We're not quite there yet. We have multiple labs, um, you know, XAI very famously, you know, Jensen Huang praising them for being. Best boy number one in spinning up 100, 000 GPU cluster in like 12 days or something. So likewise at Meta, likewise at OpenAI, likewise at the other labs as well. So like the GPU ultra rich are going to keep doing that because I think partially it's an article of faith now that you just need it.[00:39:46] swyx: Like you don't even know what it's going to, what you're going to use it for. You just, you just need it. And it makes sense that if, especially if we're going into. More researchy territory than we are. So let's say 2020 to 2023 was [00:40:00] let's scale big models territory because we had GPT 3 in 2020 and we were like, okay, we'll go from 1.[00:40:05] swyx: 75b to 1. 8b, 1. 8t. And that was GPT 3 to GPT 4. Okay, that's done. As far as everyone is concerned, Opus 3. 5 is not coming out, GPT 4. 5 is not coming out, and Gemini 2, we don't have Pro, whatever. We've hit that wall. Maybe I'll call it the 2 trillion perimeter wall. We're not going to 10 trillion. No one thinks it's a good idea, at least from training costs, from the amount of data, or at least the inference.[00:40:36] swyx: Would you pay 10x the price of GPT Probably not. Like, like you want something else that, that is at least more useful. So it makes sense that people are pivoting in terms of their inference paradigm.[00:40:47] Emerging Trends in AI Models[00:40:47] swyx: And so when it's more researchy, then you actually need more just general purpose compute to mess around with, uh, at the exact same time that production deployments of the old, the previous paradigm is still ramping up,[00:40:58] swyx: um,[00:40:58] swyx: uh, pretty aggressively.[00:40:59] swyx: So [00:41:00] it makes sense that the GPU rich are growing. We have now interviewed both together and fireworks and replicates. Uh, we haven't done any scale yet. But I think Amazon, maybe kind of a sleeper one, Amazon, in a sense of like they, at reInvent, I wasn't expecting them to do so well, but they are now a foundation model lab.[00:41:18] swyx: It's kind of interesting. Um, I think, uh, you know, David went over there and started just creating models.[00:41:25] Alessio: Yeah, I mean, that's the power of prepaid contracts. I think like a lot of AWS customers, you know, they do this big reserve instance contracts and now they got to use their money. That's why so many startups.[00:41:37] Alessio: Get bought through the AWS marketplace so they can kind of bundle them together and prefer pricing.[00:41:42] swyx: Okay, so maybe GPU super rich doing very well, GPU middle class dead, and then GPU[00:41:48] Alessio: poor. I mean, my thing is like, everybody should just be GPU rich. There shouldn't really be, even the GPU poorest, it's like, does it really make sense to be GPU poor?[00:41:57] Alessio: Like, if you're GPU poor, you should just use the [00:42:00] cloud. Yes, you know, and I think there might be a future once we kind of like figure out what the size and shape of these models is where like the tiny box and these things come to fruition where like you can be GPU poor at home. But I think today is like, why are you working so hard to like get these models to run on like very small clusters where it's like, It's so cheap to run them.[00:42:21] Alessio: Yeah, yeah,[00:42:22] swyx: yeah. I think mostly people think it's cool. People think it's a stepping stone to scaling up. So they aspire to be GPU rich one day and they're working on new methods. Like news research, like probably the most deep tech thing they've done this year is Distro or whatever the new name is.[00:42:38] swyx: There's a lot of interest in heterogeneous computing, distributed computing. I tend generally to de emphasize that historically, but it may be coming to a time where it is starting to be relevant. I don't know. You know, SF compute launched their compute marketplace this year, and like, who's really using that?[00:42:53] swyx: Like, it's a bunch of small clusters, disparate types of compute, and if you can make that [00:43:00] useful, then that will be very beneficial to the broader community, but maybe still not the source of frontier models. It's just going to be a second tier of compute that is unlocked for people, and that's fine. But yeah, I mean, I think this year, I would say a lot more on device, We are, I now have Apple intelligence on my phone.[00:43:19] swyx: Doesn't do anything apart from summarize my notifications. But still, not bad. Like, it's multi modal.[00:43:25] Alessio: Yeah, the notification summaries are so and so in my experience.[00:43:29] swyx: Yeah, but they add, they add juice to life. And then, um, Chrome Nano, uh, Gemini Nano is coming out in Chrome. Uh, they're still feature flagged, but you can, you can try it now if you, if you use the, uh, the alpha.[00:43:40] swyx: And so, like, I, I think, like, you know, We're getting the sort of GPU poor version of a lot of these things coming out, and I think it's like quite useful. Like Windows as well, rolling out RWKB in sort of every Windows department is super cool. And I think the last thing that I never put in this GPU poor war, that I think I should now, [00:44:00] is the number of startups that are GPU poor but still scaling very well, as sort of wrappers on top of either a foundation model lab, or GPU Cloud.[00:44:10] swyx: GPU Cloud, it would be Suno. Suno, Ramp has rated as one of the top ranked, fastest growing startups of the year. Um, I think the last public number is like zero to 20 million this year in ARR and Suno runs on Moto. So Suno itself is not GPU rich, but they're just doing the training on, on Moto, uh, who we've also talked to on, on the podcast.[00:44:31] swyx: The other one would be Bolt, straight cloud wrapper. And, and, um, Again, another, now they've announced 20 million ARR, which is another step up from our 8 million that we put on the title. So yeah, I mean, it's crazy that all these GPU pores are finding a way while the GPU riches are also finding a way. And then the only failures, I kind of call this the GPU smiling curve, where the edges do well, because you're either close to the machines, and you're like [00:45:00] number one on the machines, or you're like close to the customers, and you're number one on the customer side.[00:45:03] swyx: And the people who are in the middle. Inflection, um, character, didn't do that great. I think character did the best of all of them. Like, you have a note in here that we apparently said that character's price tag was[00:45:15] Alessio: 1B.[00:45:15] swyx: Did I say that?[00:45:16] Alessio: Yeah. You said Google should just buy them for 1B. I thought it was a crazy number.[00:45:20] Alessio: Then they paid 2. 7 billion. I mean, for like,[00:45:22] swyx: yeah.[00:45:22] Alessio: What do you pay for node? Like, I don't know what the game world was like. Maybe the starting price was 1B. I mean, whatever it was, it worked out for everybody involved.[00:45:31] The Multi-Modality War[00:45:31] Alessio: Multimodality war. And this one, we never had text to video in the first version, which now is the hottest.[00:45:37] swyx: Yeah, I would say it's a subset of image, but yes.[00:45:40] Alessio: Yeah, well, but I think at the time it wasn't really something people were doing, and now we had VO2 just came out yesterday. Uh, Sora was released last month, last week. I've not tried Sora, because the day that I tried, it wasn't, yeah. I[00:45:54] swyx: think it's generally available now, you can go to Sora.[00:45:56] swyx: com and try it. Yeah, they had[00:45:58] Alessio: the outage. Which I [00:46:00] think also played a part into it. Small things. Yeah. What's the other model that you posted today that was on Replicate? Video or OneLive?[00:46:08] swyx: Yeah. Very, very nondescript name, but it is from Minimax, which I think is a Chinese lab. The Chinese labs do surprisingly well at the video models.[00:46:20] swyx: I'm not sure it's actually Chinese. I don't know. Hold me up to that. Yep. China. It's good. Yeah, the Chinese love video. What can I say? They have a lot of training data for video. Or a more relaxed regulatory environment.[00:46:37] Alessio: Uh, well, sure, in some way. Yeah, I don't think there's much else there. I think like, you know, on the image side, I think it's still open.[00:46:45] Alessio: Yeah, I mean,[00:46:46] swyx: 11labs is now a unicorn. So basically, what is multi modality war? Multi modality war is, do you specialize in a single modality, right? Or do you have GodModel that does all the modalities? So this is [00:47:00] definitely still going, in a sense of 11 labs, you know, now Unicorn, PicoLabs doing well, they launched Pico 2.[00:47:06] swyx: 0 recently, HeyGen, I think has reached 100 million ARR, Assembly, I don't know, but they have billboards all over the place, so I assume they're doing very, very well. So these are all specialist models, specialist models and specialist startups. And then there's the big labs who are doing the sort of all in one play.[00:47:24] swyx: And then here I would highlight Gemini 2 for having native image output. Have you seen the demos? Um, yeah, it's, it's hard to keep up. Literally they launched this last week and a shout out to Paige Bailey, who came to the Latent Space event to demo on the day of launch. And she wasn't prepared. She was just like, I'm just going to show you.[00:47:43] swyx: So they have voice. They have, you know, obviously image input, and then they obviously can code gen and all that. But the new one that OpenAI and Meta both have but they haven't launched yet is image output. So you can literally, um, I think their demo video was that you put in an image of a [00:48:00] car, and you ask for minor modifications to that car.[00:48:02] swyx: They can generate you that modification exactly as you asked. So there's no need for the stable diffusion or comfy UI workflow of like mask here and then like infill there in paint there and all that, all that stuff. This is small model nonsense. Big model people are like, huh, we got you in as everything in the transformer.[00:48:21] swyx: This is the multimodality war, which is, do you, do you bet on the God model or do you string together a whole bunch of, uh, Small models like a, like a chump. Yeah,[00:48:29] Alessio: I don't know, man. Yeah, that would be interesting. I mean, obviously I use Midjourney for all of our thumbnails. Um, they've been doing a ton on the product, I would say.[00:48:38] Alessio: They launched a new Midjourney editor thing. They've been doing a ton. Because I think, yeah, the motto is kind of like, Maybe, you know, people say black forest, the black forest models are better than mid journey on a pixel by pixel basis. But I think when you put it, put it together, have you tried[00:48:53] swyx: the same problems on black forest?[00:48:55] Alessio: Yes. But the problem is just like, you know, on black forest, it generates one image. And then it's like, you got to [00:49:00] regenerate. You don't have all these like UI things. Like what I do, no, but it's like time issue, you know, it's like a mid[00:49:06] swyx: journey. Call the API four times.[00:49:08] Alessio: No, but then there's no like variate.[00:49:10] Alessio: Like the good thing about mid journey is like, you just go in there and you're cooking. There's a lot of stuff that just makes it really easy. And I think people underestimate that. Like, it's not really a skill issue, because I'm paying mid journey, so it's a Black Forest skill issue, because I'm not paying them, you know?[00:49:24] Alessio: Yeah,[00:49:25] swyx: so, okay, so, uh, this is a UX thing, right? Like, you, you, you understand that, at least, we think that Black Forest should be able to do all that stuff. I will also shout out, ReCraft has come out, uh, on top of the image arena that, uh, artificial analysis has done, has apparently, uh, Flux's place. Is this still true?[00:49:41] swyx: So, Artificial Analysis is now a company. I highlighted them I think in one of the early AI Newses of the year. And they have launched a whole bunch of arenas. So, they're trying to take on LM Arena, Anastasios and crew. And they have an image arena. Oh yeah, Recraft v3 is now beating Flux 1. 1. Which is very surprising [00:50:00] because Flux And Black Forest Labs are the old stable diffusion crew who left stability after, um, the management issues.[00:50:06] swyx: So Recurve has come from nowhere to be the top image model. Uh, very, very strange. I would also highlight that Grok has now launched Aurora, which is, it's very interesting dynamics between Grok and Black Forest Labs because Grok's images were originally launched, uh, in partnership with Black Forest Labs as a, as a thin wrapper.[00:50:24] swyx: And then Grok was like, no, we'll make our own. And so they've made their own. I don't know, there are no APIs or benchmarks about it. They just announced it. So yeah, that's the multi modality war. I would say that so far, the small model, the dedicated model people are winning, because they are just focused on their tasks.[00:50:42] swyx: But the big model, People are always catching up. And the moment I saw the Gemini 2 demo of image editing, where I can put in an image and just request it and it does, that's how AI should work. Not like a whole bunch of complicated steps. So it really is something. And I think one frontier that we haven't [00:51:00] seen this year, like obviously video has done very well, and it will continue to grow.[00:51:03] swyx: You know, we only have Sora Turbo today, but at some point we'll get full Sora. Oh, at least the Hollywood Labs will get Fulsora. We haven't seen video to audio, or video synced to audio. And so the researchers that I talked to are already starting to talk about that as the next frontier. But there's still maybe like five more years of video left to actually be Soda.[00:51:23] swyx: I would say that Gemini's approach Compared to OpenAI, Gemini seems, or DeepMind's approach to video seems a lot more fully fledged than OpenAI. Because if you look at the ICML recap that I published that so far nobody has listened to, um, that people have listened to it. It's just a different, definitely different audience.[00:51:43] swyx: It's only seven hours long. Why are people not listening? It's like everything in Uh, so, so DeepMind has, is working on Genie. They also launched Genie 2 and VideoPoet. So, like, they have maybe four years advantage on world modeling that OpenAI does not have. Because OpenAI basically only started [00:52:00] Diffusion Transformers last year, you know, when they hired, uh, Bill Peebles.[00:52:03] swyx: So, DeepMind has, has a bit of advantage here, I would say, in, in, in showing, like, the reason that VO2, while one, They cherry pick their videos. So obviously it looks better than Sora, but the reason I would believe that VO2, uh, when it's fully launched will do very well is because they have all this background work in video that they've done for years.[00:52:22] swyx: Like, like last year's NeurIPS, I already was interviewing some of their video people. I forget their model name, but for, for people who are dedicated fans, they can go to NeurIPS 2023 and see, see that paper.[00:52:32] Alessio: And then last but not least, the LLMOS. We renamed it to Ragops, formerly known as[00:52:39] swyx: Ragops War. I put the latest chart on the Braintrust episode.[00:52:43] swyx: I think I'm going to separate these essays from the episode notes. So the reason I used to do that, by the way, is because I wanted to show up on Hacker News. I wanted the podcast to show up on Hacker News. So I always put an essay inside of there because Hacker News people like to read and not listen.[00:52:58] Alessio: So episode essays,[00:52:59] swyx: I remember [00:53:00] purchasing them separately. You say Lanchain Llama Index is still growing.[00:53:03] Alessio: Yeah, so I looked at the PyPy stats, you know. I don't care about stars. On PyPy you see Do you want to share your screen? Yes. I prefer to look at actual downloads, not at stars on GitHub. So if you look at, you know, Lanchain still growing.[00:53:20] Alessio: These are the last six months. Llama Index still growing. What I've basically seen is like things that, One, obviously these things have A commercial product. So there's like people buying this and sticking with it versus kind of hopping in between things versus, you know, for example, crew AI, not really growing as much.[00:53:38] Alessio: The stars are growing. If you look on GitHub, like the stars are growing, but kind of like the usage is kind of like flat. In the last six months, have they done some[00:53:4

I continue my conversation with Sophia Willows, the Head of Engineering at Rye, an a16z-backed developer tools startup making APIs for online commerce. At Rye, Sophia shapes the technical direction of the company and is responsible for building a high-performance engineering culture.In this part Sophia discusses the challenges of creating consistent and user-friendly APIs. She emphasizes the importance of involving developers early in the design process, using clear and consistent documentation, and leveraging tools like GraphQL to enforce structure and consistency.The conversation shifts to the impact of AI on the future of software development. While AI can automate certain tasks, it's unlikely to replace the need for human creativity and problem-solving skills. Sophia encourages developers to focus on higher-level thinking and domain expertise, which are areas where AI is less likely to make significant inroads.Finally, Sophia addresses the issue of burnout and the importance of finding a sustainable work-life balance. She suggests that developers should identify what energizes them and focus on those activities, whether they're technical or non-technical. By understanding their own motivations and setting realistic expectations, developers can thrive in the ever-evolving field of software engineering.Sophia Willows is the Head of Engineering at Rye, an a16z-backed developer tools startup making APIs for online commerce. At Rye, Sophia shapes the technical direction of the company and is responsible building a high-performance engineering culture.She previously worked in EdTech as Engineering Manager for Crimson Education's AI team. Designing and implementing their generative AI strategy, Sophia lead the development of solutions that enhanced the company's educational offerings and personalized student learning experiences. Just before she moved to Rye, Crimson secured a $1B market valuation.Sophia is a recognized figure in the tech community, frequently judging global hackathons and contributing to industry discussions through speaking engagements and blogging.Handles:* https://sophiabits.com/blog* https://www.linkedin.com/in/sophia-willows

In today's jam-packed episode, they dive deep into the world of API design, logging best practices, and effective configuration management. Our esteemed guests, Michael Dawson, James Snell, Matteo Collina, and Natalia Venditto, bring their extensive expertise to the table, discussing the nuances between GraphQL and REST/Open API, the merits of API First vs. Code First approaches, and the impacts of global states in Node.js applications.You'll hear insights on how to maintain effective API contracts, avoid common pitfalls in software development, and implement robust error handling and logging mechanisms. Additionally, the episode covers practical advice on optimizing large-scale ecosystems with tools like Pino and managing dependencies thoughtfully to avoid technical debt.They also touch on the personal side of development, with James Snell emphasizing the importance of well-being by taking regular breaks. Charles Max Wood shares his recent experience at a board game convention and recommends the TV show "Reacher" for some downtime entertainment.So, sit back and enjoy this enlightening conversation that spans across technical deep dives and light-hearted discussions, offering valuable takeaways for developers at all levels.SocialsLinkedIn: James SnellLinkedIn: Michael DawsonLinkedIn: Matteo CollinaLinkedIn: Natalia VendittoPicksCharles - Gnome Hollow | Board GameCharles - Reacher (TV Series 2022Michael - MakerWorld: Download Free 3D Printing Models Become a supporter of this podcast: https://www.spreaker.com/podcast/javascript-jabber--6102064/support.

In this conversation, Ben, a Director of Product at Shopify, shares his journey to the company, discusses the latest updates in Shopify Editions Winter '25, and highlights improvements in the CLI and GraphQL API. Ben emphasizes the importance of community engagement and the potential for future developments in the Shopify ecosystem. Ongoing improvements and future vision for Liquid and its integration with developer tools like VS Code are highlighted. Ben also emphasizes the importance of enhancing the developer experience, streamlining workflows, and leveraging community feedback. The discussion also touches on the role of AI in development, the significance of open-source collaboration, and the need for a cohesive and intuitive coding environment. *Episode Takeaways* - The Winter Edition focuses on refining existing features rather than introducing new ones, "The Boring Edition" - CLI improvements aim to enhance the developer experience significantly. - GraphQL API enhancements allow for better theme management and integration. - Community feedback is crucial for product development at Shopify. - Ben's journey to Shopify involved building his own company first. - The CLI has been rebuilt to improve functionality and ease of use. - GraphQL is now fully integrated for managing themes and other resources. - Ben encourages developers to build apps using Shopify's public APIs. - The future of Shopify includes more extensibility and community-driven tools. The goal is to enhance the developer experience with tools like VS Code. - Streamlining Liquid development is crucial for efficiency. - Future improvements will focus on making Liquid more expressive and simpler. - Community engagement is vital for the evolution of Liquid. - AI tools like Copilot can significantly impact development workflows. - The integration of various tools can create a seamless experience for developers. - Liquid's evolution aims to maintain simplicity while adding functionality. - Building in public fosters transparency and collaboration. - The developer experience (DX) is directly tied to user experience (UX). - Hot reloading and better asset management are key future features. *Timestamps* 00:00 Ben's Journey to Shopify and Product Role 08:21 Winter Editions Overview and New Features 25:40 Embracing GraphQL for Enhanced API Management 45:12 Building a Strong Foundation for Future Development 50:47 Aligning Developer and Business Goals 56:34 Community Engagement and Open Source Development 01:09:17 Philosophical Insights on Development and Collaboration *Find Ben Online* Twitter(X): https://x.com/benjaminsehl LinkedIn: https://www.linkedin.com/in/benjaminsehl/ *Resources* Shopify Editions Winter '25: https://www.shopify.com/editions/winter2025 KOTN: https://kotn.com/ Sanity.io Groq: https://www.sanity.io/docs/groq Liquid RFCs: https://github.com/Shopify/liquid/discussions/categories/requests-for-suggestions Jeffrey Guenther Shopkeeper: https://github.com/TheBeyondGroup/shopkeeper Vite Plugin for Shopify Dev: https://github.com/barrel/shopify-vite *Picks of the Week* Ben: - Dami Dina AI Generator for Liquid sections https://x.com/DamiDina/status/1861755659353542741 - Teenage Engineering CM-15 https://teenage.engineering/store/cm-15 Karl: Fresca https://www.coca-cola.com/us/en/brands/fresca-sparkling-soda Taylor: Dev Duck https://shopify.supply/products/rubber-duck Signup for Liquid Weekly Don't miss out on expert insights and tips—subscribe to Liquid Weekly for more content like this. https://liquidweekly.com/

Joël and Stephanie go back to fundamentals as they pick apart some recent conversations they've been having around the office. Together they discuss the advantages of GraphQL over a REST API, how they utilise JSONB over a regular column or table, and the use-cases for and against a frontend framework like React. But what's the theme that ties all these conversations together? — The article mentioned in this episode was Why I'm over GraphQL (https://bessey.dev/blog/2024/05/24/why-im-over-graphql/) Your hosts for this episode have been thoughtbot's own Stephanie Minn and Joël Quenneville (https://www.linkedin.com/in/joel-quenneville-96b18b58/). If you would like to support the show, head over to our GitHub page (https://github.com/sponsors/thoughtbot), or check out our website (https://bikeshed.thoughtbot.com). Got a question or comment about the show? Why not write to our hosts: hosts@bikeshed.fm This has been a thoughtbot (https://thoughtbot.com/) podcast. Stay up to date by following us on social media - LinkedIn (https://www.linkedin.com/company/150727/) - Mastodon (https://thoughtbot.social/@thoughtbot) - Instagram (https://www.instagram.com/thoughtbot/) © 2024 thoughtbot, inc.

Austin Story, Senior Engineering Director at Doximity, joins Robby to explore the intricacies of building maintainable systems, fostering team accountability, and enabling faster iteration without sacrificing quality. Austin shares how his team approached migrating from a monolithic GraphQL architecture to a federated model, why simplicity matters for long-term success, and how guiding principles like YAGNI influence his decision-making.Doximity is a leading digital platform for medical professionals, and their technology blog offers deep dives into the systems and tools that power their innovative solutions.Key Topics Discussed[00:00:41] What is maintainable software? Austin highlights key traits, including testability, simplicity, and ease of removal.[00:02:09] Designing for removability: Why it's important and how it enables iterative progress.[00:03:05] YAGNI (You Aren't Gonna Need It): How this principle shapes Austin's approach to feature development.[00:04:13] Migrating to GraphQL Federation: Benefits of breaking up a monolithic GraphQL server and the challenges faced during the transition.[00:05:56] GraphQL vs. REST: How GraphQL aids developer productivity while maintaining backward compatibility.[00:10:53] Collaboration between data and application teams: Using tools like Kafka to bridge gaps and improve workflow.[00:17:00] Upgrading Ruby on Rails applications: Balancing autonomy with central guidance for seamless updates.[00:27:55] Fostering ownership on teams: The cultural practices that empower engineers to take initiative and drive results.[00:34:29] Prioritizing work effectively: How Austin's team uses quarterly planning and measurable "goalposts" to align efforts with impact.[00:40:00] Avoiding bike-shedding: Keeping meetings and reviews focused on meaningful progress.Key TakeawaysSimplicity Wins: Maintainable software is easier to adapt, remove, and iterate on when it's kept simple.Iterate and Refine: Use principles like YAGNI to avoid over-engineering and ensure systems are built to evolve.Collaboration Drives Success: Bridging communication between specialized teams can unlock untapped potential.Focus on Outcomes: Define clear goals and track measurable results to ensure projects align with business needs.Resources MentionedYAGNI (You Aren't Gonna Need It)GraphQL Federation OverviewDoximity Technology BlogThe Mom Test by Rob FitzpatrickAustin Story on LinkedInAustin Story's WebsiteStay ConnectedFollow Austin:LinkedInWebsiteThanks to Our Sponsor!Turn hours of debugging into just minutes! AppSignal is a performance monitoring and error-tracking tool designed for Ruby, Elixir, Python, Node.js, Javascript, and other frameworks.It offers six powerful features with one simple interface, providing developers with real-time insights into the performance and health of web applications.Keep your coding cool and error-free, one line at a time! Use the code maintainable to get a 10% discount for your first year. Check them out! Subscribe to Maintainable on:Apple PodcastsSpotifyOr search "Maintainable" wherever you stream your podcasts.Keep up to date with the Maintainable Podcast by joining the newsletter.

In this episode, Bill Kennedy interviews Tanmai Gopal, co-founder and CEO of Hasura, discussing the evolution of San Francisco post-pandemic, the innovative approach of Hasura, and the importance of data security and access. Tanmai shares insights from his academic journey, including his experiences with internships and his master's degree in computer vision, culminating in a fascinating project involving drones. In this conversation, Tanmai Gopal discusses his journey from academia to entrepreneurship, focusing on his experiences in building a consulting business and transitioning to product development. He shares insights on the evolution of GraphQL, the challenges of navigating business decisions, and the future of data access in the context of AI and emerging technologies. The discussion highlights the importance of understanding data modeling and the need for innovative solutions in the software industry.00:00 Introduction03:15 What is Tanmai Doing Today05:45 Understanding Hasura's Approach to APIs14:40 Pre-Hosted Solutions in Hasura22:26 First Memories of a Computer35:40 Favorite Classes During University49:25 From Consulting to Product1:01:35 Extending GraphQL 1:10:30 Competitors of Hasura1:18:40 Data Privacy1:22:10 Contact InfoConnect with Tanmai: Linkedin: https://www.linkedin.com/in/tanmaig/X: https://x.com/tanmaigo?lang=enMentioned in today's episode:Hasura: https://hasura.io/GraphQL: https://graphql.org/Want more from Ardan Labs? You can learn Go, Kubernetes, Docker & more through our video training, live events, or through our blog!Online Courses : https://ardanlabs.com/education/ Live Events : https://www.ardanlabs.com/live-training-events/ Blog : https://www.ardanlabs.com/blog Github : https://github.com/ardanlabs

An airhacks.fm conversation with Phillip Krueger (@phillipkruger) about: early programming experiences with Visual Basic and Java, transition from actuarial science to computer science, first job at a bank working with Java Swing and RMI over CORBA, experience with J2EE and XML technologies, working with XML and XSLT, development of open-source Swing components, work on dotMobi sites for mobile phones in Africa, creation of API extensions for Java EE and MicroProfile, involvement in the MicroProfile GraphQL specification, joining Red Hat and working on quarkus, development of SmallRye GraphQL, improvements to OpenAPI support in Quarkus, work on Quarkus Dev UI, discussion about the evolution of Java application servers and frameworks, comparison of REST and GraphQL, thoughts on Java development culture in South Africa Phillip Krueger on twitter: @phillipkruger

In this episode of the Liquid Weekly Podcast, hosts Karl Meisterheim and Taylor Page welcome Kirill Platonov, a Shopify developer specializing in Ruby on Rails. The conversation explores Kirill's journey into development, his experiences building Shopify apps, and the evolution of the Rails ecosystem. They discuss the challenges and advantages of using Rails with Shopify, the impact of open-source contributions, and the transition to GraphQL. Kirill shares insights on the future of Rails development and the importance of community support in the tech space. Timestamps 00:00 Guest Introduction and Background 02:17 Transitioning to Ruby and Rails 05:12 Building Shopify Apps and Early Experiences 08:03 Challenges with Shopify's Ecosystem 11:00 Developing with Hotwire and AppBridge 14:15 Open Source Contributions and Community Impact 17:10 Working with Shopify's Development Team 20:19 Current Projects and Future Plans 23:21 Reflections on the App Store Landscape 26:11 The Future of Rails in Shopify Development 32:11 Exploring the Full Stack with Rails 37:35 Simplifying App Development with Rails 40:29 Getting Started with Ruby on Rails 43:38 Transitioning to GraphQL 50:30 Updates in the Developer Community 56:22 Personal Updates and Picks of the Week Find Kirill Online Website: https://kirillplatonov.com/ Github: https://github.com/sponsors/kirillplatonov Twitter(X): https://x.com/kirplatonov LinkedIn: https://www.linkedin.com/in/kirplatonov/ Wife's shop: https://bleakandsleek.shop/ Kirill's Apps and Repos Platmart: Bulk Price Editor: https://apps.shopify.com/fast-bulk-price-editor Platmart: Color Swatches: https://apps.shopify.com/fast-product-colors Platmart Size Charts: https://apps.shopify.com/platmart-size-charts Shopify Hotwire Sample: https://github.com/kirillplatonov/shopify-hotwire-sample Polaris View Components: https://github.com/baoagency/polaris_view_components Shopify GraphQL Gem: https://github.com/kirillplatonov/shopify_graphql Resources Shopify App Bridge: https://shopify.dev/docs/api/app-bridge Dev Changelog New .dev community forum: https://community.shopify.dev/ Built for Shopify update to grace period for programmatically assessed criteria: https://shopify.dev/changelog/built-for-shopify-update-to-grace-period-for-programmatically-assessed-criteria Storefront API Cart now supports removing Gift Cards: https://shopify.dev/changelog/storefront-api-cart-now-supports-removing-gift-cards Breaking Changes to CAPTCHA protection on Storefront forms: https://shopify.dev/changelog/breaking-changes-to-captcha-protection-on-storefront-forms New validation against duplicate handles in productCreate, productUpdate, and productSet mutation inputs: https://shopify.dev/changelog/new-validation-against-duplicate-handles-in-productcreate-productupdate-and-productset-mutation-inputs Picks of the Week Kirill: Cursor AI https://www.cursor.com/ Karl: The Mysterious Cities of Gold https://en.wikipedia.org/wiki/The_Mysterious_Cities_of_Gold Taylor: Duolingo https://www.duolingo.com/ Signup for Liquid Weekly Don't miss out on expert insights and tips—subscribe to Liquid Weekly for more content like this. https://liquidweekly.com/

Erik Hedberg besöker återigen Kompilator och hjälper Bartek att reda ut vad GraphQL är _egentligen_.Hostingen av Kompilator sponsras av Dekalfabriken

In this episode, Chris Nowicki shares his path from aerospace to web development and the unique challenges of transitioning into tech. Chris and James discuss how Chris got involved in the open-source project "Deals for Devs," including the tech stack, managing contributions, and handling obstacles. This episode offers a first-hand look at the value of community in development and tips for new devs on getting started in open source.SponsorPostman is an API platform for building and using APIs. Postman simplifies each step of the API lifecycle and streamlines collaboration so you can create better APIs—faster.Show Notes00:00 - Intro01:08 - Chris Nowicki's Journey into Tech02:12 - Bootcamp Experience and Structure05:07 - Breaking into Tech Through Community Involvement08:38 - Deals for Devs: The Project Origin11:10 - Sponsor Message: Postman12:06 - Tech Stack Overview for Deals for Devs13:22 - Tech Stack: Resend, React Email, Tailwind, and Xata17:00 - Prisma Integration with Xata20:00 - Challenges in Managing Community Projects23:54 - Planning and Issue Management for Deals for Devs28:00 - Feature Flags and Release Management37:15 - Subscription System Workflow45:45 - Creating a Dynamic Email Subscription System51:58 - Managing Admin and Approval for Deals52:26 - ClosingLinksOpenSaucedRedwoodJSDeals for Devs ProjectPostmanReact EmailVercelXataResendFrontend MentorLaunchDarklyGrid Iron SurvivorDev.to article on CRON jobs

Join Dan Vega as he explores Spring for GraphQL with special guests Brian Clozel and Rossen Stoyanchev from the Spring team. In this deep-dive episode, the experts discuss the evolution of Spring for GraphQL, its relationship with GraphQL Java, and how it compares to Netflix's DGS framework. Learn about GraphQL Federation, handling N+1 problems with batch loading, and when to choose GraphQL over REST. The conversation covers practical insights on error handling, security considerations, and the future roadmap of Spring for GraphQL.Show Notes:* Origins of Spring for GraphQL and collaboration with GraphQL Java* Use cases for choosing GraphQL in enterprise applications* Federation support and microservices architecture* Batch loading and handling N+1 problems* Error handling in GraphQL vs REST* Spring for GraphQL and Netflix DGS framework integration* Future roadmap with Spring Framework 7* Tips for getting started with Spring for GraphQLJoin the live stream to ask questions or catch the replay on your preferred podcast platform.

Amy Williams from Skellig Automation joins hosts Phil Seboa and Ed Fuentes to dive into the world of industrial IoT and automation in life sciences. Learn about Amy's journey into life sciences, challenges in the pharmaceutical industry, and the potential of digital tools like the Unified Namespace and Industry 4.0. Discover how new technologies are improving data management and making healthcare more accessible. Tune in for an inspiring conversation on the future of life sciences and the digital revolution. 00:00 Introduction and Welcome 00:30 Introduction of Amy Williams 01:45 Amy's Background in Life Sciences and Automation 05:32 Paper vs. Digital Workflow in Pharmaceuticals 08:40 Insights on Graph Databases 10:50 GraphQL and Database Automation 13:00 The Future Landscape of Tech in Life Sciences 15:12 The Role of Biosimilars in Healthcare 17:35 Addressing US Drug Shortages 20:05 Skellig's Initiatives in Supply Chain Digitalization 22:16 Challenges in Technology Adoption 24:40 Amy's Influences and Family Background 27:00 From College to Automation Engineering 30:15 Transitioning to Life Sciences 33:03 Personal Health Journey and its Impact 36:22 Making Healthcare More Accessible 38:40 Inefficiencies in Manufacturing and Costs to Patients 42:10 Advocacy for Industry Change 44:00 Upcoming News on Industry Innovations 46:30 Advice for Embracing Industry 4.0 49:00 Conclusion and Final Thoughts Connect with Amy on LinkedIn: https://www.linkedin.com/in/amy-williams-a8974b114/ Connect with Phil on LinkedIn: https://www.linkedin.com/in/phil-seboa/ Connect with Ed on LinkedIn: https://www.linkedin.com/in/ed-fuentes-2046121a/ About Industry Sage Media: Industry Sage Media is your backstage pass to industry experts and the conversations that are shaping the future of the manufacturing industry. Learn more at: http://www.industrysagemedia.com

Scott and Wes talk with Søren Bramer Schmidt, Founder and CEO of Prisma, about database best practices, including the latest developments in serverless, local-first, and typed SQL solutions. Show Notes 00:00 Welcome to Syntax! 02:55 Søren's thoughts on GraphQL 03:53 Brought to you by Sentry.io 06:57 Common database mistakes 08:52 Prisma's stability and user experience 10:42 Typed SQL and advanced querying Announcing TypedSQL: Make your raw SQL queries type-safe with Prisma ORM Prisma Optimize 20:47 Serverless challenges and solutions Prisma Accelerate 27:11 Cloudflare's potential to dethrone AWS 29:13 Prisma and local-first development Prisma & Expo: A Better Path to Local-First Apps | App.js Conf 2024 35:30 Making local-first development mainstream 40:10 Challenges with async 42:43 Søren's thoughts on Drizzle 43:41 Søren's favorite database 47:21 The read your writes problem 48:58 Prisma hosted Postgres 51:44 Sick Picks & Shameless Plugs Sick Picks Søren: Cursor Shameless Plugs Søren: 1: Prisma Optimize 2: Prisma Postgres (coming soon) Hit us up on Socials! Syntax: X Instagram Tiktok LinkedIn Threads Wes: X Instagram Tiktok LinkedIn Threads Scott: X Instagram Tiktok LinkedIn Threads Randy: X Instagram YouTube Threads

GraphQL is an open-source query language for APIs and a runtime for executing those queries. It was developed by Facebook to address the problem of over-fetching or under-fetching data, which is a common issue with traditional REST APIs. Matt Bessey is a Principal Engineer and Software Architect. Earlier this year Matt wrote a blog post The post The End of GraphQL with Matt Bessey appeared first on Software Engineering Daily.

GraphQL is an open-source query language for APIs and a runtime for executing those queries. It was developed by Facebook to address the problem of over-fetching or under-fetching data, which is a common issue with traditional REST APIs. Matt Bessey is a Principal Engineer and Software Architect. Earlier this year Matt wrote a blog post The post The End of GraphQL with Matt Bessey appeared first on Software Engineering Daily.

Kent C. Dodds, web dev educator, discusses the evolution of web architectures, the potential of React Server Components, and the latest advancements in React 19, offering insights perfect for developers eager to stay ahead. Links https://kentcdodds.com https://x.com/kentcdodds https://github.com/kentcdodds https://www.youtube.com/c/KentCDodds-vids https://www.linkedin.com/in/kentcdodds https://www.epicreact.dev https://www.testingjavascript.com https://www.epicweb.dev We want to hear from you! How did you find us? Did you see us on Twitter? In a newsletter? Or maybe we were recommended by a friend? Let us know by sending an email to our producer, Emily, at emily.kochanekketner@logrocket.com (mailto:emily.kochanekketner@logrocket.com), or tweet at us at PodRocketPod (https://twitter.com/PodRocketpod). Follow us. Get free stickers. Follow us on Apple Podcasts, fill out this form (https://podrocket.logrocket.com/get-podrocket-stickers), and we'll send you free PodRocket stickers! What does LogRocket do? LogRocket provides AI-first session replay and analytics that surfaces the UX and technical issues impacting user experiences. Start understand where your users are struggling by trying it for free at [LogRocket.com]. Try LogRocket for free today.(https://logrocket.com/signup/?pdr) Special Guest: Kent C. Dodds.

In this episode of Breaking Changes, Postman Head of Product-Observability Jean Yang sits down with Nick Schrock, the co-creator of GraphQL, to dive into the fascinating journey behind GraphQL's development. They discuss how GraphQL transitioned from an internal system at Facebook to a widely adopted technology—as well as how Nick's newest venture, Dagster Labs, is revolutionizing data orchestration with asset-oriented pipelines. This conversation dives deep into the realm of data engineering and its transformative potential for businesses. Nick and Jean also share insights into the intersection of AI and software engineering, and Nick offers his perspective on responsible AI development.   For more on Nick Schrock, check out the following: LinkedIn: https://www.linkedin.com/in/schrockn/ Twitter: https://twitter.com/schrockn GraphQL Website: https://graphql.org/   Follow Jean on Twitter/X @jeanqasaur. And remember, never miss an episode by subscribing to the Breaking Changes Podcast on your favorite streaming platform or Postman's YouTube Channel—just hit that bell for notifications. #BreakingChanges #data #postman #grahpql #TechLeadership #ai #podcast

 In today's episode, Charles, Steve, and AJ, are joined by back-end engineer and team lead at Homebound, Stephen Haberman. We delve into the fascinating world of SQL c and its revolutionary approach to managing SQL queries with dedicated SQL files, delivering benefits such as reduced typing errors and pre-deployment checks. Stephen also walks us through the advantages and limitations of ORMs versus query builders like Prisma and Drizzle, sharing insights into Joyce ORM's unique philosophy and simplified CRUD operations.They explore the intricacies of Domain Driven Design (DDD), its emphasis on ubiquitous language, and how it shapes business logic and storage management. AJ contributes by discussing the potential of SQL c and Slonik for dynamic query building. Additionally, they discuss Steven's innovative work with GraphFileWorker and GrafAST, highlighting the performance improvements in GraphQL backends. Whether you're intrigued by the technicalities of ORMs, the evolution of database tools, or just love a good anecdote, this episode packed with technical insights and lively discussions is one you won't want to miss. Join them on this journey into the world of database management and development!SocialsLinkedIn: Stephen HabermanPicks AJ - TypeScript to JSDocAJ - MySQL to TypeScriptAJ - sqlcAJ - Slonik (Node + Postgres)AJ - SwiftUI EssentialsAJ - Introduction to SwiftUI AJ - Trump, but not saying dumb thingsCharles - Biblios | Board GameCharles - FreeStyle Libre 3 System | Continuous Glucose MonitoringStephen - Grafast | GrafastBecome a supporter of this podcast: https://www.spreaker.com/podcast/javascript-jabber--6102064/support.

Tom Occhino, Chief Product Officer at Vercel and former Engineering Director at Facebook, joins Sam to talk about the pivotal moments in React's history. He talks about how React popularized the ideas of declarative rendering and unidirectional data flow, how GraphQL furthered React's goal of co-locating all the concerns of a particular piece of UI, the problems that GraphQL led to at Facebook and how Relay solved them, and how Suspense, Server Components, and PPR are the generalized spiritual successors to the stack used at Facebook.Timestamps:0:00 - Intro2:53 - Declarative rendering as React's legacy8:12 - How GraphQL enabled complex components to be fully self-contained20:12 - How React's goal has always been to co-locate all the concerns of a particular piece of UI22:58 - The problem with co-locating GraphQL with components, and how Relay solved it26:28 - How RSC is the generalized spiritual successor to BigPipe and GraphQL34:46 - What PPR is, and how it and Suspense fit into this story55:55 - The general paradigm shift of getting static code to the device as soon as possibleLinks:Tom Occhino with Ben DunphyReact: The DocumentaryReact Roundtable with Andrew Clark and Sebastian MarkbågeTom Occhino on Twitter

Guest Brian Douglas Panelist Richard Littauer Show Notes In this episode of Sustain, host Richard Littauer talks with Brian “bdougie” Douglas, founder and CEO of Open Sauced. They discuss the multifaceted aspects of sustaining open source projects, Brian's journey in developer advocacy, and the unique goals of Open Sauced. Brian shares insights from his experiences at GitHub and Netlify, elaborates on concepts like lottery factor and the significance of unique issue authors, and tackles the challenges of maintaining open source sustainability. He also explores the balance of addressing enterprise needs while supporting smaller, less visible projects and emphasizes the importance of education and community engagement in open source. Press download now! [00:01:54] Brian discusses his background at GitHub and Netlify, his role in promoting GraphQL, GitHub Actions, Codespaces, and the inception of Open Sauced. [00:03:08] We hear about the features of Open Sauced's dashboard which enhances GitHub insights, OSSF scorecards, and workspace customizations for managing multiple projects. [00:04:31] Open Sauced's business model is currently founded by VC money and aims to serve large organizations with significant open source dependencies, and Brian talks about the team size and funding history. [00:06:08] Brian elaborates on Open Sauced's long-term sustainability plan, focusing on enterprise-level solutions for open source project observability and contributions. [00:09:31] There's a discussion on how Open Sauced interacts with open source communities and the importance of real-world testing and contributions to open source projects. [00:11:06] Richard highlights the FOSS Funders initiative, encouraging companies to support open source projects financially and through active participation. [00:12:44] Brian shares insights on effective metrics for evaluating open source projects, emphasizing the importance of engaging with unique issue authors rather than focusing solely on superficial metrics like pull requests, and discusses his approach to starting meaningful conversations in the open source community. [00:16:08] Brian explains why he renamed “Lottery Factor” to “Contributor Absence Factor,” and discusses the Pgvector project to illustrate the importance of understanding the “Contributor Absence Factor” and the sustainability concerns when a project relies heavily on a few contributors. [00:18:20] We learn more about how Open Sauced sources its data, including their use of GitHub's events feed and their development of the “Pizza Oven” tool to generate insights from Git repositories. [00:20:21] Richard and Brian discuss the challenges of maintaining an open source ethos when dealing with large companies' internal projects, avoiding becoming merely service providers for large corporate entities. [00:24:14] Brian discusses the long-term implications of open source projects that receive substantial funding or become integrated into larger corporate frameworks. [00:27:27] Richard brings up the difficulty many open source projects face in accessing significant funding and Brian shares his vision for supporting less prominent open source projects drawing analogies from his personal experiences. [00:32:42] Richard questions the “up the chain” analogy, comparing it to a pyramid scheme or academia's tenure track. Brian acknowledges the need to support contributors at all levels, not just those at the top, and he introduces the concept of a S Bomb to provide transparency about project dependencies. [00:39:36] Find out where you can follow Brian on the web. Spotlight [00:40:17] Richard's spotlight is Mr. Carreras, an awesome music teacher. [00:40:59] Brian's spotlight is Dawn Foster at the CHAOSS Project and the CHAOSS Practitioner Guides. Links SustainOSS (https://sustainoss.org/) podcast@sustainoss.org (email) (mailto:podcast@sustainoss.org) richard@theuserismymom.com (email) (mailto:richard@theuserismymom.com) SustainOSS Discourse (https://discourse.sustainoss.org/) SustainOSS Mastodon (https://mastodon.social/tags/sustainoss) Open Collective-SustainOSS (Contribute) (https://opencollective.com/sustainoss) Richard Littauer Socials (https://www.burntfen.com/2023-05-30/socials) Brian Douglas- Open Sauced (https://app.opensauced.pizza/u/bdougie) Brian Douglas Website (https://b.dougie.dev/) Brian Douglas GitHub (https://github.com/bdougie) Brian Douglas X/Twitter (https://github.com/bdougie) The Secret Sauce Open Sauced Podcast (https://podcasts.apple.com/us/podcast/the-secret-sauce/id1644263270) The Secret Sauce Podcast: ‘The Future of Cloud Native and AI with Brendan Burns' (https://podcasts.apple.com/fr/podcast/the-future-of-cloud-native-and-ai-with-brendan-burns/id1644263270?i=1000658092259) Open Sauced (https://opensauced.pizza/) Renaming Bus Factor #632 (CHAOSS community) (https://github.com/chaoss/community/issues/632#issuecomment-2152929617) FOSS Funders (https://fossfunders.com/) Andrew Kane GitHub (https://github.com/ankane) Chad Whitacre Website (https://chadwhitacre.com/) Fair Source (https://fair.io/) CHAOSS (https://chaoss.community/) Your Copilot for Git History (Open Sauced) (https://opensauced.pizza/docs/features/star-search/) Open Sauced GitHub (https://github.com/open-sauced/pizza) InnerSource Commons (https://innersourcecommons.org/) Sustain Podcast-Episode 148: Ali Nehzat of thanks.dev and OSS Funding (https://podcast.sustainoss.org/148) Learning in Public with Kelsey Hightower (Curiefense) (https://www.curiefense.io/blog/learning-in-public-with-kelsey-hightower/) Welcome to Wrexham (https://en.wikipedia.org/wiki/Welcome_to_Wrexham) Sustain Podcast-Episode 159: Dawn Foster & Andrew Nesbitt at State of Open Con 2023 (https://podcast.sustainoss.org/guests/foster) Dr. Dawn Foster Mastodon (https://hachyderm.io/@geekygirldawn) About the CHAOSS Practitioner Guides (https://chaoss.community/about-chaoss-practitioner-guides/) Credits Produced by Richard Littauer (https://www.burntfen.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr Peachtree Sound (https://www.peachtreesound.com/) Special Guest: Brian Douglas.

Erik Hanchett, senior developer advocate at AWS Amplify, explores the world of Fullstack TypeScript. He discusses the significance of end-to-end type safety, the tools to achieve it, and delves into the benefits and functionalities of AWS Amplify. Links https://www.programwitherik.com https://x.com/erikch https://www.youtube.com/c/programwitherik https://www.linkedin.com/in/erikhanchett https://trpc.io https://orval.dev https://docs.amplify.aws We want to hear from you! How did you find us? Did you see us on Twitter? In a newsletter? Or maybe we were recommended by a friend? Let us know by sending an email to our producer, Emily, at emily.kochanekketner@logrocket.com (mailto:emily.kochanekketner@logrocket.com), or tweet at us at PodRocketPod (https://twitter.com/PodRocketpod). Follow us. Get free stickers. Follow us on Apple Podcasts, fill out this form (https://podrocket.logrocket.com/get-podrocket-stickers), and we'll send you free PodRocket stickers! What does LogRocket do? LogRocket provides AI-first session replay and analytics that surfaces the UX and technical issues impacting user experiences. Start understand where your users are struggling by trying it for free at [LogRocket.com]. Try LogRocket for free today.(https://logrocket.com/signup/?pdr) Special Guest: Erik Hanchett.

How do you integrate GraphQL into your Python web development? How about quickly building graph-based APIs inside Django's battery-included framework? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder's Weekly articles and projects.

In episode 800 of Syntax, Scott and Wes sit down with John Resig, the creator of jQuery, to discuss the current state of React and TypeScript. They dive into the evolution of frontend frameworks, the challenges of server-side rendering, and the tech stack at Khan Academy. Show Notes 00:00 Welcome to Syntax! 00:59 Brought to you by Sentry.io. 01:32 What is jQuery? 05:31 Did you anticipate the success jQuery had? 07:16 allow-discrete, @starting-style. Install Nothing: App UIs With Native Browser APIs - Scott Tolinski. 07:54 Building the community around jQuery. 11:16 jQuery plugins. 13:00 Did you ever make money from jQuery? 16:13 What is your role at Khan Academy. 17:58 What is the tech stack at Khan Academy? 21:56 Why do you want to change your CSS and JS framework? 24:03 TypeScript vs Flow. 25:25 GraphQL federation. 28:08 What was your frontend framework journey? 30:23 Is there any part of React you wish would improve? 32:37 Reservations using React Router. 33:14 Khan Academy web platform vs native platform. 35:21 What do you use for state management? 38:48 What's harder than it should be on the web today? Kilian's Question On X. Polypane.app. 42:46 Opinions on JavaScript Sprinkles. 44:04 What's with the $ sign in jQuery? 45:29 The challenges of having your name in such a widely used software. 51:06 Challenges with server-side rendering in React. 52:42 Sick Picks & Shameless Plugs. 54:48 What are the performance issues associated with internationalization? 56:57 Back to Sick Picks & Shameless Plugs. Sick Picks John: Biome, Remix, Lingui. Shameless Plugs John: Khan Academy. Hit us up on Socials! Syntax: X Instagram Tiktok LinkedIn Threads Wes: X Instagram Tiktok LinkedIn Threads Scott: X Instagram Tiktok LinkedIn Threads Randy: X Instagram YouTube Threads

David Flanagan, founder of Rawdoke Academy, discusses why WebAssembly (WASM) could be the future of serverless technology and explores the evolution, benefits, and potential of WASM in transforming server-side applications across various environments. Links https://davidflanagan.com https://github.com/davidflanagan https://twitter.com/__DavidFlanagan https://www.linkedin.com/in/rawkode https://rawkode.academy https://youtube.com/@RawkodeAcademy https://www.hopp.bio/rawkode We want to hear from you! How did you find us? Did you see us on Twitter? In a newsletter? Or maybe we were recommended by a friend? Let us know by sending an email to our producer, Emily, at emily.kochanekketner@logrocket.com (mailto:emily.kochanekketner@logrocket.com), or tweet at us at PodRocketPod (https://twitter.com/PodRocketpod). Follow us. Get free stickers. Follow us on Apple Podcasts, fill out this form (https://podrocket.logrocket.com/get-podrocket-stickers), and we'll send you free PodRocket stickers! What does LogRocket do? LogRocket provides AI-first session replay and analytics that surfaces the UX and technical issues impacting user experiences. Start understand where your users are struggling by trying it for free at [LogRocket.com]. Try LogRocket for free today.(https://logrocket.com/signup/?pdr) Special Guest: David Flanagan.

In this episode of Maintainable, Robby Russell sits down with Tanmai Gopal, the CEO and co-founder of Hasura. Tanmai shares his insights into the characteristics of well-maintained software and the importance of a codebase that no team member fears. He emphasizes the need for accessibility and understandability in code, making it easier for developers to work with and iterate upon.Tanmai dives deep into the metaphor of technical debt, urging teams to prioritize product outcomes over best practices. He highlights the value of addressing technical debt contextually and in a way that aligns with product goals.A significant portion of the discussion revolves around the concept of the "super graph" in GraphQL. Tanmai explains how a unified API, created through federated GraphQL, can streamline API integration and reduce latency. He compares GraphQL with RESTful APIs, showcasing the advantages of a graph-based approach for handling complex data relationships.Tanmai also introduces Hasura's platform, which introspects databases, code, and APIs to create a comprehensive super graph. This platform simplifies API management, making it easier for developers to maintain and evolve their applications.Listeners will also learn about Hasura's upcoming user conference and the new features they plan to unveil. Tanmai shares his top science fiction book recommendations and where to follow his thoughts on software engineering online.Key Takeaways:The importance of a fearless codebase for well-maintained software.Strategies to improve code accessibility and understandability.The metaphor of technical debt and its contextual importance.The concept and benefits of a super graph in GraphQL.How Hasura simplifies API management through introspection.Upcoming Hasura user conference and new features.Resources Mentioned:HasuraTanmai Gopal on LinkedInN.K. Jemisin's Broken Earth TrilogyNaomi Novik's UprootedMartha Wells' Murderbot DiariesThanks to Our Sponsor!Turn hours of debugging into just minutes! AppSignal is a performance monitoring and error tracking tool designed for Ruby, Elixir, Python, Node.js, Javascript, and soon, other frameworks.It offers six powerful features with one simple interface, providing developers with real-time insights into the performance and health of web applications.Keep your coding cool and error-free, one line at a time! Check them out! Subscribe to Maintainable on:Apple PodcastsSpotifyOr search "Maintainable" wherever you stream your podcasts.Keep up to date with the Maintainable Podcast by joining the newsletter.

Dive into the world of GraphQL APIs on AWS this week! We'll explore the recently launched feature in AppSync: asynchronous Lambda functions for GraphQL resolvers. But first, we'll break down the advantages of GraphQL over REST APIs and the limitations of synchronous calls in GraphQL. Then, we'll uncover the power of async Lambdas: stream data directly to your client for a more responsive experience and unlock innovative use cases, like generative AI-powered chatbots built with Lambdas. Curious how this can transform your applications? Tune in to learn more! With Derek Bingham, Developer Advocate, AWS https://www.linkedin.com/in/derekwbingham/ - Derek's blog about AppSync async Lambda resolvers https://community.aws/content/2hlqAp86YvckSS2DrVvZ1qdArqF/async-lambda-and-appsync?lang=en - AWS AppSync https://docs.aws.amazon.com/appsync/latest/devguide/what-is-appsync.html - AWS Lambda https://docs.aws.amazon.com/lambda/latest/dg/welcome.html - Streaming a response from a Lambda function https://docs.aws.amazon.com/lambda/latest/dg/configuration-response-streaming.html - AWS AppSync sample code https://github.com/aws-samples/aws-appsync-resolver-samples - Michael (App Sync Developer Advocate) YouTube channel https://www.youtube.com/@focusotter/videos

Naoki Hiroshima さんをゲストに迎えて、米最高裁、大統領選、叡王戦、Apple, ローカライゼーションなどについて話しました。 Show Notes Supreme Court overrules Chevron, kneecapping federal regulators Could Democrats Replace Biden as their nominee? Election 2024: Trump proposes green cards for foreign grads of US colleges McCain Counters Obama 'Arab' Question Trump's Second Term: Last Week Tonight with John Oliver There's a 16-Year Old, 7'3" Chinese Women's Basketball Player Caitlin Clark ハイキュー!! 叡王戦 KADOKAWA広報: 当社へのランサムウェア攻撃による情報漏洩に関して Apple is first company charged with violating EU's DMA rules Apple Intelligence features probably won't launch in the EU in 2024 “a SQL” or “an SQL”? Why, after 6 years, I'm over GraphQL

Summary Data lakehouse architectures have been gaining significant adoption. To accelerate adoption in the enterprise Microsoft has created the Fabric platform, based on their OneLake architecture. In this episode Dipti Borkar shares her experiences working on the product team at Fabric and explains the various use cases for the Fabric service. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst is an end-to-end data lakehouse platform built on Trino, the query engine Apache Iceberg was designed for, with complete support for all table formats including Apache Iceberg, Hive, and Delta Lake. Trusted by teams of all sizes, including Comcast and Doordash. Want to see Starburst in action? Go to dataengineeringpodcast.com/starburst (https://www.dataengineeringpodcast.com/starburst) and get $500 in credits to try Starburst Galaxy today, the easiest and fastest way to get started using Trino. Your host is Tobias Macey and today I'm interviewing Dipti Borkar about her work on Microsoft Fabric and performing analytics on data withou Interview Introduction How did you get involved in the area of data management? Can you describe what Microsoft Fabric is and the story behind it? Data lakes in various forms have been gaining significant popularity as a unified interface to an organization's analytics. What are the motivating factors that you see for that trend? Microsoft has been investing heavily in open source in recent years, and the Fabric platform relies on several open components. What are the benefits of layering on top of existing technologies rather than building a fully custom solution? What are the elements of Fabric that were engineered specifically for the service? What are the most interesting/complicated integration challenges? How has your prior experience with Ahana and Presto informed your current work at Microsoft? AI plays a substantial role in the product. What are the benefits of embedding Copilot into the data engine? What are the challenges in terms of safety and reliability? What are the most interesting, innovative, or unexpected ways that you have seen the Fabric platform used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on data lakes generally, and Fabric specifically? When is Fabric the wrong choice? What do you have planned for the future of data lake analytics? Contact Info LinkedIn (https://www.linkedin.com/in/diptiborkar/) Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. Podcast.__init__ (https://www.pythonpodcast.com) covers the Python language, its community, and the innovative ways it is being used. The Machine Learning Podcast (https://www.themachinelearningpodcast.com) helps you go from idea to production with machine learning. Visit the site (https://www.dataengineeringpodcast.com) to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@dataengineeringpodcast.com (mailto:hosts@dataengineeringpodcast.com) with your story. Links Microsoft Fabric (https://www.microsoft.com/microsoft-fabric) Ahana episode (https://www.dataengineeringpodcast.com/ahana-presto-cloud-data-lake-episode-217) DB2 Distributed (https://www.ibm.com/docs/en/db2/11.5?topic=managers-designing-distributed-databases) Spark (https://spark.apache.org/) Presto (https://prestodb.io/) Azure Data (https://azure.microsoft.com/en-us/products#analytics) MAD Landscape (https://mattturck.com/mad2024/) Podcast Episode (https://www.dataengineeringpodcast.com/mad-landscape-2023-data-infrastructure-episode-369) ML Podcast Episode (https://www.themachinelearningpodcast.com/mad-landscape-2023-ml-ai-episode-21) Tableau (https://www.tableau.com/) dbt (https://www.getdbt.com/) Medallion Architecture (https://dataengineering.wiki/Concepts/Medallion+Architecture) Microsoft Onelake (https://learn.microsoft.com/fabric/onelake/onelake-overview) ORC (https://orc.apache.org/) Parquet (https://parquet.incubator.apache.org) Avro (https://avro.apache.org/) Delta Lake (https://delta.io/) Iceberg (https://iceberg.apache.org/) Podcast Episode (https://www.dataengineeringpodcast.com/iceberg-with-ryan-blue-episode-52/) Hudi (https://hudi.apache.org/) Podcast Episode (https://www.dataengineeringpodcast.com/hudi-streaming-data-lake-episode-209) Hadoop (https://hadoop.apache.org/) PowerBI (https://www.microsoft.com/power-platform/products/power-bi) Podcast Episode (https://www.dataengineeringpodcast.com/power-bi-business-intelligence-episode-154) Velox (https://velox-lib.io/) Gluten (https://gluten.apache.org/) Apache XTable (https://xtable.apache.org/) GraphQL (https://graphql.org/) Formula 1 (https://www.formula1.com/) McLaren (https://www.mclaren.com/) The intro and outro music is from The Hug (http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/Love_death_and_a_drunken_monkey/04_-_The_Hug) by The Freak Fandango Orchestra (http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/) / CC BY-SA (http://creativecommons.org/licenses/by-sa/3.0/)

Editor's note: One of the top reasons we have hundreds of companies and thousands of AI Engineers joining the World's Fair next week is, apart from discussing technology and being present for the big launches planned, to hire and be hired! Listeners loved our previous Elicit episode and were so glad to welcome 2 more members of Elicit back for a guest post (and bonus podcast) on how they think through hiring. Don't miss their AI engineer job description, and template which you can use to create your own hiring plan! How to Hire AI EngineersJames Brady, Head of Engineering @ Elicit (ex Spring, Square, Trigger.io, IBM)Adam Wiggins, Internal Journalist @ Elicit (Cofounder Ink & Switch and Heroku)If you're leading a team that uses AI in your product in some way, you probably need to hire AI engineers. As defined in this article, that's someone with conventional engineering skills in addition to knowledge of language models and prompt engineering, without being a full-fledged Machine Learning expert.But how do you hire someone with this skillset? At Elicit we've been applying machine learning to reasoning tools since 2018, and our technical team is a mix of ML experts and what we can now call AI engineers. This article will cover our process from job description through interviewing. (You can also flip the perspectives here and use it just as easily for how to get hired as an AI engineer!)My own journeyBefore getting into the brass tacks, I want to share my journey to becoming an AI engineer.Up until a few years ago, I was happily working my job as an engineering manager of a big team at a late-stage startup. Like many, I was tracking the rapid increase in AI capabilities stemming from the deep learning revolution, but it was the release of GPT-3 in 2020 which was the watershed moment. At the time, we were all blown away by how the model could string together coherent sentences on demand. (Oh how far we've come since then!)I'd been a professional software engineer for nearly 15 years—enough to have experienced one or two technology cycles—but I could see this was something categorically new. I found this simultaneously exciting and somewhat disconcerting. I knew I wanted to dive into this world, but it seemed like the only path was going back to school for a master's degree in Machine Learning. I started talking with my boss about options for taking a sabbatical or doing a part-time distance learning degree.In 2021, I instead decided to launch a startup focused on productizing new research ideas on ML interpretability. It was through that process that I reached out to Andreas—a leading ML researcher and founder of Elicit—to see if he would be an advisor. Over the next few months, I learned more about Elicit: that they were trying to apply these fascinating technologies to the real-world problems of science, and with a business model that aligned it with safety goals. I realized that I was way more excited about Elicit than I was about my own startup ideas, and wrote about my motivations at the time.Three years later, it's clear this was a seismic shift in my career on the scale of when I chose to leave my comfy engineering job at IBM to go through the Y Combinator program back in 2008. Working with this new breed of technology has been more intellectually stimulating, challenging, and rewarding than I could have imagined.Deep ML expertise not requiredIt's important to note that AI engineers are not ML experts, nor is that their best contribution to a tech team.In our article Living documents as an AI UX pattern, we wrote:It's easy to think that AI advancements are all about training and applying new models, and certainly this is a huge part of our work in the ML team at Elicit. But those of us working in the UX part of the team believe that we have a big contribution to make in how AI is applied to end-user problems.We think of LLMs as a new medium to work with, one that we've barely begun to grasp the contours of. New computing mediums like GUIs in the 1980s, web/cloud in the 90s and 2000s, and multitouch smartphones in the 2000s/2010s opened a whole new era of engineering and design practices. So too will LLMs open new frontiers for our work in the coming decade.To compare to the early era of mobile development: great iOS developers didn't require a detailed understanding of the physics of capacitive touchscreens. But they did need to know the capabilities and limitations of a multi-touch screen, the constrained CPU and storage available, the context in which the user is using it (very different from a webpage or desktop computer), etc.In the same way, an AI engineer needs to work with LLMs as a medium that is fundamentally different from other compute mediums. That means an interest in the ML side of things, whether through their own self-study, tinkering with prompts and model fine-tuning, or following along in #llm-paper-club. But this understanding is so that they can work with the medium effectively versus, say, spending their days training new models.Language models as a chaotic mediumSo if we're not expecting deep ML expertise from AI engineers, what are we expecting? This brings us to what makes LLMs different.We'll assume already that our ideal candidate is already inspired by, and full of ideas about, all the new capabilities AI can bring to software products. But the flip side is all the things that make this new medium difficult to work with. LLM calls are annoying due to high latency (measured in tens of seconds sometimes, rather than milliseconds), extreme variance on latency, high error rates even under normal operation. Not to mention getting extremely different answers to the same prompt provided to the same model on two subsequent calls!The net effect is that an AI engineer, even working at the application development level, needs to have a skillset comparable to distributed systems engineering. Handling errors, retries, asynchronous calls, streaming responses, parallelizing and recombining model calls, the halting problem, and fallbacks are just some of the day-in-the-life of an AI engineer. Chaos engineering gets new life in the era of AI.Skills and qualities in candidatesLet's put together what we don't need (deep ML expertise) with what we do (work with capabilities and limitations of the medium). Thus we start to see what Elicit looks for in AI engineers:* Conventional software engineering skills. Especially back-end engineering on complex, data-intensive applications.* Professional, real-world experience with applications at scale.* Deep, hands-on experience across a few back-end web frameworks.* Light devops and an understanding of infrastructure best practices.* Queues, message buses, event-driven and serverless architectures, … there's no single “correct” approach, but having a deep toolbox to draw from is very important.* A genuine curiosity and enthusiasm for the capabilities of language models.* One or more serious projects (side projects are fine) of using them in interesting ways on a unique domain.* …ideally with some level of factored cognition, e.g. breaking the problem down into chunks, making thoughtful decisions about which things to push to the language model and which stay within the realm of conventional heuristics and compute capabilities.* Personal studying with resources like Elicit's ML reading list. Part of the role is collaborating with the ML engineers and researchers on our team. To do so, the candidate needs to “speak their language” somewhat, just as a mobile engineer needs some familiarity with backends in order to collaborate effectively on API creation with backend engineers.* An understanding of the challenges that come along with working with large models (high latency, variance, etc.) leading to a defensive, fault-first mindset.* Careful and principled handling of error cases, asynchronous code (and ability to reason about and debug it), streaming data, caching, logging and analytics for understanding behavior in production.* This is a similar mindset that one can develop working on conventional apps which are complex, data-intensive, or large-scale apps. The difference is that an AI engineer will need this mindset even when working on relatively small scales!On net, a great AI engineer will combine two seemingly contrasting perspectives: knowledge of, and a sense of wonder for, the capabilities of modern ML models; but also the understanding that this is a difficult and imperfect foundation, and the willingness to build resilient and performant systems on top of it.Here's the resulting AI engineer job description for Elicit. And here's a template that you can borrow from for writing your own JD.Hiring processOnce you know what you're looking for in an AI engineer, the process is not too different from other technical roles. Here's how we do it, broken down into two stages: sourcing and interviewing.SourcingWe're primarily looking for people with (1) a familiarity with and interest in ML, and (2) proven experience building complex systems using web technologies. The former is important for culture fit and as an indication that the candidate will be able to do some light prompt engineering as part of their role. The latter is important because language model APIs are built on top of web standards and—as noted above—aren't always the easiest tools to work with.Only a handful of people have built complex ML-first apps, but fortunately the two qualities listed above are relatively independent. Perhaps they've proven (2) through their professional experience and have some side projects which demonstrate (1).Talking of side projects, evidence of creative and original prototypes is a huge plus as we're evaluating candidates. We've barely scratched the surface of what's possible to build with LLMs—even the current generation of models—so candidates who have been willing to dive into crazy “I wonder if it's possible to…” ideas have a huge advantage.InterviewingThe hard skills we spend most of our time evaluating during our interview process are in the “building complex systems using web technologies” side of things. We will be checking that the candidate is familiar with asynchronous programming, defensive coding, distributed systems concepts and tools, and display an ability to think about scaling and performance. They needn't have 10+ years of experience doing this stuff: even junior candidates can display an aptitude and thirst for learning which gives us confidence they'll be successful tackling the difficult technical challenges we'll put in front of them.One anti-pattern—something which makes my heart sink when I hear it from candidates—is that they have no familiarity with ML, but claim that they're excited to learn about it. The amount of free and easily-accessible resources available is incredible, so a motivated candidate should have already dived into self-study.Putting all that together, here's the interview process that we follow for AI engineer candidates:* 30-minute introductory conversation. Non-technical, explaining the interview process, answering questions, understanding the candidate's career path and goals.* 60-minute technical interview. This is a coding exercise, where we play product manager and the candidate is making changes to a little web app. Here are some examples of topics we might hit upon through that exercise:* Update API endpoints to include extra metadata. Think about appropriate data types. Stub out frontend code to accept the new data.* Convert a synchronous REST API to an asynchronous streaming endpoint.* Cancellation of asynchronous work when a user closes their tab.* Choose an appropriate data structure to represent the pending, active, and completed ML work which is required to service a user request.* 60–90 minute non-technical interview. Walk through the candidate's professional experience, identifying high and low points, getting a grasp of what kinds of challenges and environments they thrive in.* On-site interviews. Half a day in our office in Oakland, meeting as much of the team as possible: more technical and non-technical conversations.The frontier is wide openAlthough Elicit is perhaps further along than other companies on AI engineering, we also acknowledge that this is a brand-new field whose shape and qualities are only just now starting to form. We're looking forward to hearing how other companies do this and being part of the conversation as the role evolves.We're excited for the AI Engineer World's Fair as another next step for this emerging subfield. And of course, check out the Elicit careers page if you're interested in joining our team.Podcast versionTimestamps* [00:00:24] Intros* [00:05:25] Defining the Hiring Process* [00:08:42] Defensive AI Engineering as a chaotic medium* [00:10:26] Tech Choices for Defensive AI Engineering* [00:14:04] How do you Interview for Defensive AI Engineering* [00:19:25] Does Model Shadowing Work?* [00:22:29] Is it too early to standardize Tech stacks?* [00:32:02] Capabilities: Offensive AI Engineering* [00:37:24] AI Engineering Required Knowledge* [00:40:13] ML First Mindset* [00:45:13] AI Engineers and Creativity* [00:47:51] Inside of Me There Are Two Wolves* [00:49:58] Sourcing AI Engineers* [00:58:45] Parting ThoughtsTranscript[00:00:00] swyx: Okay, so welcome to the Latent Space Podcast. This is another remote episode that we're recording. This is the first one that we're doing around a guest post. And I'm very honored to have two of the authors of the post with me, James and Adam from Elicit. Welcome, James. Welcome, Adam.[00:00:22] James Brady: Thank you. Great to be here.[00:00:23] Hey there.[00:00:24] Intros[00:00:24] swyx: Okay, so I think I will do this kind of in order. I think James, you're, you're sort of the primary author. So James, you are head of engineering at Elicit. You also, We're VP Eng at Teespring and Spring as well. And you also , you have a long history in sort of engineering. How did you, , find your way into something like Elicit where, , it's, you, you are basically traditional sort of VP Eng, VP technology type person moving into a more of an AI role.[00:00:53] James Brady: Yeah, that's right. It definitely was something of a Sideways move if not a left turn. So the story there was I'd been doing, as you said, VP technology, CTO type stuff for around about 15 years or so, and Notice that there was this crazy explosion of capability and interesting stuff happening within AI and ML and language models, that kind of thing.[00:01:16] I guess this was in 2019 or so, and decided that I needed to get involved. , this is a kind of generational shift. And Spent maybe a year or so trying to get up to speed on the state of the art, reading papers, reading books, practicing things, that kind of stuff. Was going to found a startup actually in in the space of interpretability and transparency, and through that met Andreas, who has obviously been on the, on the podcast before asked him to be an advisor for my startup, and he countered with, maybe you'd like to come and run the engineering team at Elicit, which it turns out was a much better idea.[00:01:48] And yeah, I kind of quickly changed in that direction. So I think some of the stuff that we're going to be talking about today is how actually a lot of the work when you're building applications with AI and ML looks and smells and feels much more like conventional software engineering with a few key differences rather than really deep ML stuff.[00:02:07] And I think that's one of the reasons why I was able to transfer skills over from one place to the other.[00:02:12] swyx: Yeah, I[00:02:12] James Brady: definitely[00:02:12] swyx: agree with that. I, I do often say that I think AI engineering is about 90 percent software engineering with like the, the 10 percent of like really strong really differentiated AI engineering.[00:02:22] And that might, that obviously that number might change over time. I want to also welcome Adam onto my podcast because you welcomed me onto your podcast two years ago.[00:02:31] Adam Wiggins: Yeah, that was a wonderful episode.[00:02:32] swyx: That was, that was a fun episode. You famously founded Heroku. You just wrapped up a few years working on Muse.[00:02:38] And now you've described yourself as a journalist, internal journalist working on Elicit.[00:02:43] Adam Wiggins: Yeah, well I'm kind of a little bit in a wandering phase here and trying to take this time in between ventures to see what's out there in the world and some of my wandering took me to the Elicit team. And found that they were some of the folks who were doing the most interesting, really deep work in terms of taking the capabilities of language models and applying them to what I feel like are really important problems.[00:03:08] So in this case, science and literature search and, and, and that sort of thing. It fits into my general interest in tools and productivity software. I, I think of it as a tool for thought in many ways, but a tool for science, obviously, if we can accelerate that discovery of new medicines and things like that, that's, that's just so powerful.[00:03:24] But to me, it's a. It's kind of also an opportunity to learn at the feet of some real masters in this space, people who have been working on it since it was, before it was cool, if you want to put it that way. So for me, the last couple of months have been this crash course, and why I sometimes describe myself as an internal journalist is I'm helping to write some, some posts, including Supporting James in this article here we're doing for latent space where I'm just bringing my writing skill and that sort of thing to bear on their very deep domain expertise around language models and applying them to the real world and kind of surface that in a way that's I don't know, accessible, legible, that, that sort of thing.[00:04:03] And so, and the great benefit to me is I get to learn this stuff in a way that I don't think I would, or I haven't, just kind of tinkering with my own side projects.[00:04:12] swyx: I forgot to mention that you also run Ink and Switch, which is one of the leading research labs, in my mind, of the tools for thought productivity space, , whatever people mentioned there, or maybe future of programming even, a little bit of that.[00:04:24] As well. I think you guys definitely started the local first wave. I think there was just the first conference that you guys held. I don't know if you were personally involved.[00:04:31] Adam Wiggins: Yeah, I was one of the co organizers along with a few other folks for, yeah, called Local First Conf here in Berlin.[00:04:36] Huge success from my, my point of view. Local first, obviously, a whole other topic we can talk about on another day. I think there actually is a lot more what would you call it , handshake emoji between kind of language models and the local first data model. And that was part of the topic of the conference here, but yeah, topic for another day.[00:04:55] swyx: Not necessarily. I mean , I, I selected as one of my keynotes, Justine Tunney, working at LlamaFall in Mozilla, because I think there's a lot of people interested in that stuff. But we can, we can focus on the headline topic. And just to not bury the lead, which is we're talking about hire, how to hire AI engineers, this is something that I've been looking for a credible source on for months.[00:05:14] People keep asking me for my opinions. I don't feel qualified to give an opinion and it's not like I have. So that's kind of defined hiring process that I'm super happy with, even though I've worked with a number of AI engineers.[00:05:25] Defining the Hiring Process[00:05:25] swyx: I'll just leave it open to you, James. How was your process of defining your hiring, hiring roles?[00:05:31] James Brady: Yeah. So I think the first thing to say is that we've effectively been hiring for this kind of a role since before you, before you coined the term and tried to kind of build this understanding of what it was.[00:05:42] So, which is not a bad thing. Like it's, it was a, it was a good thing. A concept, a concept that was coming to the fore and effectively needed a name, which is which is what you did. So the reason I mentioned that is I think it was something that we kind of backed into, if you will. We didn't sit down and come up with a brand new role from, from scratch of this is a completely novel set of responsibilities and skills that this person would need.[00:06:06] However, it is a A kind of particular blend of different skills and attitudes and and curiosities interests, which I think makes sense to kind of bundle together. So in the, in the post, the three things that we say are most important for a highly effective AI engineer are first of all, conventional software engineering skills, which is Kind of a given, but definitely worth mentioning.[00:06:30] The second thing is a curiosity and enthusiasm for machine learning and maybe in particular language models. That's certainly true in our case. And then the third thing is to do with basically a fault first mindset, being able to build systems that can handle things going wrong in, in, in some sense.[00:06:49] And yeah, the I think the kind of middle point, the curiosity about ML and language models is probably fairly self evident. They're going to be working with, and prompting, and dealing with the responses from these models, so that's clearly relevant. The last point, though, maybe takes the most explaining.[00:07:07] To do with this fault first mindset and the ability to, to build resilient systems. The reason that is, is so important is because compared to normal APIs, where normal, think of something like a Stripe API or a search API or something like this. The latency when you're working with language models is, is wild, like you can get 10x variation.[00:07:32] I mean, I was looking at the stats before, actually, before, before the podcast. We do often, normally, in fact, see a 10x variation in the P90 latency over the course of, Half an hour, an hour when we're prompting these models, which is way higher than if you're working with a, more kind of conventional conventionally backed API.[00:07:49] And the responses that you get, the actual content and the responses are naturally unpredictable as well. They come back with different formats. Maybe you're expecting JSON. It's not quite JSON. You have to handle this stuff. And also the, the semantics of the messages are unpredictable too, which is, which is a good thing.[00:08:08] Like this is one of the things that you're looking for from these language models, but it all adds up to needing to. Build a resilient, reliable, solid feeling system on top of this fundamentally, well, certainly currently fundamentally shaky foundation. The models do not behave in the way that you would like them to.[00:08:28] And yeah, the ability to structure the code around them such that it does give the user this warm, reassuring, Snappy, solid feeling is is really what we're driving for there.[00:08:42] Defensive AI Engineering as a chaotic medium[00:08:42] Adam Wiggins: What really struck me as we, we dug in on the content for this article was that third point there. The, the language models is this kind of chaotic medium, this, this dragon, this wild horse you're, you're, you're riding and trying to guide in the direction that is going to be useful and reliable to users, because I think.[00:08:58] So much of software engineering is about making things not only high performance and snappy, but really just making it stable, reliable, predictable, which is literally the opposite of what you get from from the language models. And yet, yeah, the output is so useful, and indeed, some of their Creativity, if you want to call it that, which is, is precisely their value.[00:09:19] And so you need to work with this medium. And I guess the nuanced or the thing that came out of Elissa's experience that I thought was so interesting is quite a lot of working with that is things that come from distributed systems engineering. But you have really the AI engineers as we're defining them or, or labeling them on the illicit team is people who are really application developers.[00:09:39] You're building things for end users. You're thinking about, okay, I need to populate this interface with some response to user input. That's useful to the tasks they're trying to do, but you have this. This is the thing, this medium that you're working with that in some ways you need to apply some of this chaos engineering, distributed systems engineering, which typically those people with those engineering skills are not kind of the application level developers with the product mindset or whatever, they're more deep in the guts of a, of a system.[00:10:07] And so it's, those, those skills and, and knowledge do exist throughout the engineering discipline, but sort of putting them together into one person that is That feels like sort of a unique thing and working with the folks on the Elicit team who have that skills I'm quite struck by that unique that unique blend.[00:10:23] I haven't really seen that before in my 30 year career in technology.[00:10:26] Tech Choices for Defensive AI Engineering[00:10:26] swyx: Yeah, that's a Fascinating I like the reference to chaos engineering. I have some appreciation, I think when you had me on your podcast, I was still working at Temporal and that was like a nice Framework, if you live within Temporal's boundaries, you can pretend that all those faults don't exist, and you can, you can code in a sort of very fault tolerant way.[00:10:47] What is, what is you guys solutions around this, actually? Like, I think you're, you're emphasizing having the mindset, but maybe naming some technologies would help? Not saying that you have to adopt these technologies, but they're just, they're just quick vectors into what you're talking about when you're, when you're talking about distributed systems.[00:11:03] Like, that's such a big, chunky word, , like are we talking, are Kubernetes or, and I suspect we're not, , like we're, we're talking something else now.[00:11:10] James Brady: Yeah, that's right. It's more at the application level rather than at the infrastructure level, at least, at least the way that it works for us.[00:11:17] So there's nothing kind of radically novel here. It is more a careful application of existing concepts. So the kinds of tools that we reach for to handle these kind of slightly chaotic objects that Adam was just talking about, are retries and fallbacks and timeouts and careful error handling. And, yeah, the standard stuff, really.[00:11:39] There's also a great degree of dependence. We rely heavily on parallelization because, , these language models are not innately very snappy, and , there's just a lot of I. O. going back and forth. So All these things I'm talking about when I was in my earlier stages of a career, these are kind of the things that are the difficult parts that most senior software engineers will be better at.[00:12:01] It is careful error handling, and concurrency, and fallbacks, and distributed systems, and, , eventual consistency, and all this kind of stuff and As Adam was saying, the kind of person that is deep in the guts of some kind of distributed systems, a really high, high scale backend kind of a problem would probably naturally have these kinds of skills.[00:12:21] But you'll find them on, on day one, if you're building a, , an ML powered app, even if it's not got massive scale. I think one one thing that I would mention that we do do yeah, maybe, maybe two related things, actually. The first is we're big fans of strong typing. We share the types all the way from the Backend Python code all the way to the to the front end in TypeScript and find that is I mean We'd probably do this anyway But it really helps one reason around the shapes of the data which can going to be going back and forth and that's really important When you can't rely upon You you're going to have to coerce the data that you get back from the ML if you want if you want for it to be structured basically speaking and The second thing which is related is we use checked exceptions inside our Python code base, which means that we can use the type system to make sure we are handling, properly handling, all of the, the various things that could be going wrong, all the different exceptions that could be getting raised.[00:13:16] So, checked exceptions are not, not really particularly popular. Actually there's not many people that are big fans of them. For our particular use case, to really make sure that we've not just forgotten to handle, , This particular type of error we have found them useful to to, to force us to think about all the different edge cases that can come up.[00:13:32] swyx: Fascinating. How just a quick note of technology. How do you share types from Python to TypeScript? Do you, do you use GraphQL? Do you use something[00:13:39] James Brady: else? We don't, we don't use GraphQL. Yeah. So we've got the We've got the types defined in Python, that's the source of truth. And we go from the OpenAPI spec, and there's a, there's a tool that you work and use to generate types dynamically, like TypeScript types from those OpenAPI definitions.[00:13:57] swyx: Okay, excellent. Okay, cool. Sorry, sorry for diving into that rabbit hole a little bit. I always like to spell out technologies for people to dig their teeth into.[00:14:04] How do you Interview for Defensive AI Engineering[00:14:04] swyx: One thing I'll, one thing I'll mention quickly is that a lot of the stuff that you mentioned is typically not part of the normal interview loop.[00:14:10] It's actually really hard to interview for because this is the stuff that you polish out in, as you go into production, the coding interviews are typically about the happy path. How do we do that? How do we, how do we design, how do you look for a defensive fault first mindset?[00:14:24] Because you can defensive code all day long and not add functionality. to your to your application.[00:14:29] James Brady: Yeah, it's a great question and I think that's exactly true. Normally the interview is about the happy path and then there's maybe a box checking exercise at the end of the candidate says of course in reality I would handle the edge cases or something like this and that unfortunately isn't isn't quite good enough when when the happy path is is very very narrow and yeah there's lots of weirdness on either side so basically speaking, it's just a case of, of foregrounding those kind of concerns through the interview process.[00:14:58] It's, there's, there's no magic to it. We, we talk about this in the, in the po in the post that we're gonna be putting up on, on Laton space. The, there's two main technical exercises that we do through our interview process for this role. The first is more coding focus, and the second is more system designy.[00:15:16] Yeah. White whiteboarding a potential solution. And in, without giving too much away in the coding exercise. You do need to think about edge cases. You do need to think about errors. The exercise consists of adding features and fixing bugs inside the code base. And in both of those two cases, it does demand, because of the way that we set the application up and the interview up, it does demand that you think about something other than the happy path.[00:15:41] But your thinking is the right prompt of how do we get the candidate thinking outside of the, the kind of normal Sweet spot, smooth smooth, smoothly paved path. In terms of the system design interview, that's a little easier to prompt this kind of fault first mindset because it's very easy in that situation just to say, let's imagine that, , this node dies, how does the app still work?[00:16:03] Let's imagine that this network is, is going super slow. Let's imagine that, I don't know, like you, you run out of, you run out of capacity in, in, in this database that you've sketched out here, how do you handle that, that, that sort of stuff. So. It's, in both cases, they're not firmly anchored to and built specifically around language models and ways language models can go wrong, but we do exercise the same muscles of thinking defensively and yeah, foregrounding the edge cases, basically.[00:16:32] Adam Wiggins: James, earlier there you mentioned retries. And this is something that I think I've seen some interesting debates internally about things regarding, first of all, retries are, can be costly, right? In general, this medium, in addition to having this incredibly high variance and response rate, and, , being non deterministic, is actually quite expensive.[00:16:50] And so, in many cases, doing a retry when you get a fail does make sense, but actually that has an impact on cost. And so there is Some sense to which, at least I've seen the AI engineers on our team, worry about that. They worry about, okay, how do we give the best user experience, but balance that against what the infrastructure is going to, , is going to cost our company, which I think is again, an interesting mix of, yeah, again, it's a little bit the distributed system mindset, but it's also a product perspective and you're thinking about the end user experience, but also the.[00:17:22] The bottom line for the business, you're bringing together a lot of a lot of qualities there. And there's also the fallback case, which is kind of, kind of a related or adjacent one. I think there was also a discussion on that internally where, I think it maybe was search, there was something recently where there was one of the frontline search providers was having some, yeah, slowness and outages, and essentially then we had a fallback, but essentially that gave people for a while, especially new users that come in that don't the difference, they're getting a They're getting worse results for their search.[00:17:52] And so then you have this debate about, okay, there's sort of what is correct to do from an engineering perspective, but then there's also what actually is the best result for the user. Is giving them a kind of a worse answer to their search result better, or is it better to kind of give them an error and be like, yeah, sorry, it's not working right at the moment, try again.[00:18:12] Later, both are obviously non optimal, but but this is the kind of thing I think that that you run into or, or the kind of thing we need to grapple with a lot more than you would other kinds of, of mediums.[00:18:24] James Brady: Yeah, that's a really good example. I think it brings to the fore the two different things that you could be optimizing for of uptime and response at all costs on one end of the spectrum and then effectively fragility, but kind of, if you get a response, it's the best response we can come up with at the other end of the spectrum.[00:18:43] And where you want to land there kind of depends on, well, it certainly depends on the app, obviously depends on the user. I think it depends on the, feature within the app as well. So in the search case that you, that you mentioned there, in retrospect, we probably didn't want to have the fallback. And we've actually just recently on Monday, changed that to Show an error message rather than giving people a kind of degraded experience in other situations We could use for example a large language model from a large language model from provider B rather than provider A and Get something which is within the A few percentage points performance, and that's just a really different situation.[00:19:21] So yeah, like any interesting question, the answer is, it depends.[00:19:25] Does Model Shadowing Work?[00:19:25] swyx: I do hear a lot of people suggesting I, let's call this model shadowing as a defensive technique, which is, if OpenAI happens to be down, which, , happens more often than people think then you fall back to anthropic or something.[00:19:38] How realistic is that, right? Like you, don't you have to develop completely different prompts for different models and won't the, won't the performance of your application suffer from whatever reason, right? Like it may be caused differently or it's not maintained in the same way. I, I think that people raise this idea of fallbacks to models, but I don't think it's, I don't, I don't see it practiced very much.[00:20:02] James Brady: Yeah, it is, you, you definitely need to have a different prompt if you want to stay within a few percentage points degradation Like I, like I said before, and that certainly comes at a cost, like fallbacks and backups and things like this It's really easy for them to go stale and kind of flake out on you because they're off the beaten track And In our particular case inside of Elicit, we do have fallbacks for a number of kind of crucial functions where it's going to be very obvious if something has gone wrong, but we don't have fallbacks in all cases.[00:20:40] It really depends on a task to task basis throughout the app. So I can't give you a kind of a, a single kind of simple rule of thumb for, in this case, do this. And in the other, do that. But yeah, we've it's a little bit easier now that the APIs between the anthropic models and opening are more similar than they used to be.[00:20:59] So we don't have two totally separate code paths with different protocols, like wire protocols to, to speak, which makes things easier, but you're right. You do need to have different prompts if you want to, have similar performance across the providers.[00:21:12] Adam Wiggins: I'll also note, just observing again as a relative newcomer here, I was surprised, impressed, not sure what the word is for it, at the blend of different backends that the team is using.[00:21:24] And so there's many The product presents as kind of one single interface, but there's actually several dozen kind of main paths. There's like, for example, the search versus a data extraction of a certain type, versus chat with papers, versus And each one of these, , the team has worked very hard to pick the right Model for the job and craft the prompt there, but also is constantly testing new ones.[00:21:48] So a new one comes out from either, from the big providers or in some cases, Our own models that are , running on, on essentially our own infrastructure. And sometimes that's more about cost or performance, but the point is kind of switching very fluidly between them and, and very quickly because this field is moving so fast and there's new ones to choose from all the time is like part of the day to day, I would say.[00:22:11] So it isn't more of a like, there's a main one, it's been kind of the same for a year, there's a fallback, but it's got cobwebs on it. It's more like which model and which prompt is changing weekly. And so I think it's quite, quite reasonable to to, to, to have a fallback that you can expect might work.[00:22:29] Is it too early to standardize Tech stacks?[00:22:29] swyx: I'm curious because you guys have had experience working at both, , Elicit, which is a smaller operation and, and larger companies. A lot of companies are looking at this with a certain amount of trepidation as, as, , it's very chaotic. When you have, when you have , one engineering team that, that, knows everyone else's names and like, , they, they, they, they meet constantly in Slack and knows what's going on.[00:22:50] It's easier to, to sync on technology choices. When you have a hundred teams, all shipping AI products and all making their own independent tech choices. It can be, it can be very hard to control. One solution I'm hearing from like the sales forces of the worlds and Walmarts of the world is that they are creating their own AI gateway, right?[00:23:05] Internal AI gateway. This is the one model hub that controls all the things and has our standards. Is that a feasible thing? Is that something that you would want? Is that something you have and you're working towards? What are your thoughts on this stuff? Like, Centralization of control or like an AI platform internally.[00:23:22] James Brady: Certainly for larger organizations and organizations that are doing things which maybe are running into HIPAA compliance or other, um, legislative tools like that. It could make a lot of sense. Yeah. I think for the TLDR for something like Elicit is we are small enough, as you indicated, and need to have full control over all the levers available and switch between different models and different prompts and whatnot, as Adam was just saying, that that kind of thing wouldn't work for us.[00:23:52] But yeah, I've spoken with and, um, advised a couple of companies that are trying to sell into that kind of a space or at a larger stage, and it does seem to make a lot of sense for them. So, for example, if you're trying to sell If you're looking to sell to a large enterprise and they cannot have any data leaving the EU, then you need to be really careful about someone just accidentally putting in, , the sort of US East 1 GPT 4 endpoints or something like this.[00:24:22] I'd be interested in understanding better what the specific problem is that they're looking to solve with that, whether it is to do with data security or centralization of billing, or if they have a kind of Suite of prompts or something like this that people can choose from so they don't need to reinvent the wheel again and again I wouldn't be able to say without understanding the problems and their proposed solutions , which kind of situations that be better or worse fit for but yeah for illicit where really the The secret sauce, if there is a secret sauce, is which models we're using, how we're using them, how we're combining them, how we're thinking about the user problem, how we're thinking about all these pieces coming together.[00:25:02] You really need to have all of the affordances available to you to be able to experiment with things and iterate rapidly. And generally speaking, whenever you put these kind of layers of abstraction and control and generalization in there, that, that gets in the way. So, so for us, it would not work.[00:25:19] Adam Wiggins: Do you feel like there's always a tendency to want to reach for standardization and abstractions pretty early in a new technology cycle?[00:25:26] There's something comforting there, or you feel like you can see them, or whatever. I feel like there's some of that discussion around lang chain right now. But yeah, this is not only so early, but also moving so fast. , I think it's . I think it's tough to, to ask for that. That's, that's not the, that's not the space we're in, but the, yeah, the larger an organization, the more that's your, your default is to, to, to want to reach for that.[00:25:48] It, it, it's a sort of comfort.[00:25:51] swyx: Yeah, I find it interesting that you would say that , being a founder of Heroku where , you were one of the first platforms as a service that more or less standardized what, , that sort of early developer experience should have looked like.[00:26:04] And I think basically people are feeling the differences between calling various model lab APIs and having an actual AI platform where. , all, all their development needs are thought of for them. , it's, it's very much, and, and I, I defined this in my AI engineer post as well.[00:26:19] Like the model labs just see their job ending at serving models and that's about it. But actually the responsibility of the AI engineer has to fill in a lot of the gaps beyond that. So.[00:26:31] Adam Wiggins: Yeah, that's true. I think, , a huge part of the exercise with Heroku, which It was largely inspired by Rails, which itself was one of the first frameworks to standardize the SQL database.[00:26:42] And people had been building apps like that for many, many years. I had built many apps. I had made my own templates based on that. I think others had done it. And Rails came along at the right moment. We had been doing it long enough that you see the patterns and then you can say look let's let's extract those into a framework that's going to make it not only easier to build for the experts but for people who are relatively new the best practices are encoded into you.[00:27:07] That framework, , Model View Controller, to take one example. But then, yeah, once you see that, and once you experience the power of a framework, and again, it's so comforting, and you can develop faster, and it's easier to onboard new people to it because you have these standards. And this consistency, then folks want that for something new that's evolving.[00:27:29] Now here I'm thinking maybe if you fast forward a little to, for example, when React came on the on the scene, , a decade ago or whatever. And then, okay, we need to do state management. What's that? And then there's, , there's a new library every six months. Okay, this is the one, this is the gold standard.[00:27:42] And then, , six months later, that's deprecated. Because of course, it's evolving, you need to figure it out, like the tacit knowledge and the experience of putting it in practice and seeing what those real What those real needs are are, are critical, and so it's, it is really about finding the right time to say yes, we can generalize, we can make standards and abstractions, whether it's for a company, whether it's for, , a library, an open source library, for a whole class of apps and it, it's very much a, much more of a A judgment call slash just a sense of taste or , experience to be able to say, Yeah, we're at the right point.[00:28:16] We can standardize this. But it's at least my, my very, again, and I'm so new to that, this world compared to you both, but my, my sense is, yeah, still the wild west. That's what makes it so exciting and feels kind of too early for too much. too much in the way of standardized abstractions. Not that it's not interesting to try, but , you can't necessarily get there in the same way Rails did until you've got that decade of experience of whatever building different classes of apps in that, with that technology.[00:28:45] James Brady: Yeah, it's, it's interesting to think about what is going to stay more static and what is expected to change over the coming five years, let's say. Which seems like when I think about it through an ML lens, it's an incredibly long time. And if you just said five years, it doesn't seem, doesn't seem that long.[00:29:01] I think that, that kind of talks to part of the problem here is that things that are moving are moving incredibly quickly. I would expect, this is my, my hot take rather than some kind of official carefully thought out position, but my hot take would be something like the You can, you'll be able to get to good quality apps without doing really careful prompt engineering.[00:29:21] I don't think that prompt engineering is going to be a kind of durable differential skill that people will, will hold. I do think that, The way that you set up the ML problem to kind of ask the right questions, if you see what I mean, rather than the specific phrasing of exactly how you're doing chain of thought or few shot or something in the prompt I think the way that you set it up is, is probably going to be remain to be trickier for longer.[00:29:47] And I think some of the operational challenges that we've been talking about of wild variations in, in, in latency, And handling the, I mean, one way to think about these models is the first lesson that you learn when, when you're an engineer, software engineer, is that you need to sanitize user input, right?[00:30:05] It was, I think it was the top OWASP security threat for a while. Like you, you have to sanitize and validate user input. And we got used to that. And it kind of feels like this is the, The shell around the app and then everything else inside you're kind of in control of and you can grasp and you can debug, etc.[00:30:22] And what we've effectively done is, through some kind of weird rearguard action, we've now got these slightly chaotic things. I think of them more as complex adaptive systems, which , related but a bit different. Definitely have some of the same dynamics. We've, we've injected these into the foundations of the, of the app and you kind of now need to think with this defined defensive mindset downwards as well as upwards if you, if you see what I mean.[00:30:46] So I think it would gonna, it's, I think it will take a while for us to truly wrap our heads around that. And also these kinds of problems where you have to handle things being unreliable and slow sometimes and whatever else, even if it doesn't happen very often, there isn't some kind of industry wide accepted way of handling that at massive scale.[00:31:10] There are definitely patterns and anti patterns and tools and whatnot, but it's not like this is a solved problem. So I would expect that it's not going to go down easily as a, as a solvable problem at the ML scale either.[00:31:23] swyx: Yeah, excellent. I would describe in, in the terminology of the stuff that I've written in the past, I describe this inversion of architecture as sort of LLM at the core versus LLM or code at the core.[00:31:34] We're very used to code at the core. Actually, we can scale that very well. When we build LLM core apps, we have to realize that the, the central part of our app that's orchestrating things is actually prompt, prone to, , prompt injections and non determinism and all that, all that good stuff.[00:31:48] I, I did want to move the conversation a little bit from the sort of defensive side of things to the more offensive or, , the fun side of things, capabilities side of things, because that is the other part. of the job description that we kind of skimmed over. So I'll, I'll repeat what you said earlier.[00:32:02] Capabilities: Offensive AI Engineering[00:32:02] swyx: It's, you want people to have a genuine curiosity and enthusiasm for the capabilities of language models. We just, we're recording this the day after Anthropic just dropped Cloud 3. 5. And I was wondering, , maybe this is a good, good exercise is how do people have Curiosity and enthusiasm for capabilities language models when for example the research paper for cloud 3.[00:32:22] 5 is four pages[00:32:23] James Brady: Maybe that's not a bad thing actually in this particular case So yeah If you really want to know exactly how the sausage was made That hasn't been possible for a few years now in fact for for these new models but from our perspective as when we're building illicit What we primarily care about is what can these models do?[00:32:41] How do they perform on the tasks that we already have set up and the evaluations we have in mind? And then on a slightly more expansive note, what kinds of new capabilities do they seem to have? Can we elicit, no pun intended, from the models? For example, well, there's, there's very obvious ones like multimodality , there wasn't that and then there was that, or it could be something a bit more subtle, like it seems to be getting better at reasoning, or it seems to be getting better at metacognition, or Or it seems to be getting better at marking its own work and giving calibrated confidence estimates, things like this.[00:33:19] So yeah, there's, there's plenty to be excited about there. It's just that yeah, there's rightly or wrongly been this, this, this shift over the last few years to not give all the details. So no, but from application development perspective we, every time there's a new model release, there's a flow of activity in our Slack, and we try to figure out what's going on.[00:33:38] What it can do, what it can't do, run our evaluation frameworks, and yeah, it's always an exciting, happy day.[00:33:44] Adam Wiggins: Yeah, from my perspective, what I'm seeing from the folks on the team is, first of all, just awareness of the new stuff that's coming out, so that's, , an enthusiasm for the space and following along, and then being able to very quickly, partially that's having Slack to do this, but be able to quickly map that to, okay, What does this do for our specific case?[00:34:07] And that, the simple version of that is, let's run the evaluation framework, which Lissa has quite a comprehensive one. I'm actually working on an article on that right now, which I'm very excited about, because it's a very interesting world of things. But basically, you can just try, not just, but try the new model in the evaluations framework.[00:34:27] Run it. It has a whole slew of benchmarks, which includes not just Accuracy and confidence, but also things like performance, cost, and so on. And all of these things may trade off against each other. Maybe it's actually, it's very slightly worse, but it's way faster and way cheaper, so actually this might be a net win, for example.[00:34:46] Or, it's way more accurate. But that comes at its slower and higher cost, and so now you need to think about those trade offs. And so to me, coming back to the qualities of an AI engineer, especially when you're trying to hire for them, It's this, it's, it is very much an application developer in the sense of a product mindset of What are our users or our customers trying to do?[00:35:08] What problem do they need solved? Or what what does our product solve for them? And how does the capabilities of a particular model potentially solve that better for them than what exists today? And by the way, what exists today is becoming an increasingly gigantic cornucopia of things, right? And so, You say, okay, this new model has these capabilities, therefore, , the simple version of that is plug it into our existing evaluations and just look at that and see if it, it seems like it's better for a straight out swap out, but when you talk about, for example, you have multimodal capabilities, and then you say, okay, wait a minute, actually, maybe there's a new feature or a whole new There's a whole bunch of ways we could be using it, not just a simple model swap out, but actually a different thing we could do that we couldn't do before that would have been too slow, or too inaccurate, or something like that, that now we do have the capability to do.[00:35:58] I think of that as being a great thing. I don't even know if I want to call it a skill, maybe it's even like an attitude or a perspective, which is a desire to both be excited about the new technology, , the new models and things as they come along, but also holding in the mind, what does our product do?[00:36:16] Who is our user? And how can we connect the capabilities of this technology to how we're helping people in whatever it is our product does?[00:36:25] James Brady: Yeah, I'm just looking at one of our internal Slack channels where we talk about things like new new model releases and that kind of thing And it is notable looking through these the kind of things that people are excited about and not It's, I don't know the context, the context window is much larger, or it's, look at how many parameters it has, or something like this.[00:36:44] It's always framed in terms of maybe this could be applied to that kind of part of Elicit, or maybe this would open up this new possibility for Elicit. And, as Adam was saying, yeah, I don't think it's really a I don't think it's a novel or separate skill, it's the kind of attitude I would like to have all engineers to have at a company our stage, actually.[00:37:05] And maybe more generally, even, which is not just kind of getting nerd sniped by some kind of technology number, fancy metric or something, but how is this actually going to be applicable to the thing Which matters in the end. How is this going to help users? How is this going to help move things forward strategically?[00:37:23] That kind of, that kind of thing.[00:37:24] AI Engineering Required Knowledge[00:37:24] swyx: Yeah, applying what , I think, is, is, is the key here. Getting hands on as well. I would, I would recommend a few resources for people listening along. The first is Elicit's ML reading list, which I, I found so delightful after talking with Andreas about it.[00:37:38] It looks like that's part of your onboarding. We've actually set up an asynchronous paper club instead of my discord for people following on that reading list. I love that you separate things out into tier one and two and three, and that gives people a factored cognition way of Looking into the, the, the corpus, right?[00:37:55] Like yes, the, the corpus of things to know is growing and the water is slowly rising as far as what a bar for a competent AI engineer is. But I think, , having some structured thought as to what are the big ones that everyone must know I think is, is, is key. It's something I, I haven't really defined for people and I'm, I'm glad that this is actually has something out there that people can refer to.[00:38:15] Yeah, I wouldn't necessarily like make it required for like the job. Interview maybe, but , it'd be interesting to see like, what would be a red flag. If some AI engineer would not know, I don't know what, , I don't know where we would stoop to, to call something required knowledge, , or you're not part of the cool kids club.[00:38:33] But there increasingly is something like that, right? Like, not knowing what context is, is a black mark, in my opinion, right?[00:38:40] I think it, I think it does connect back to what we were saying before of this genuine Curiosity about and that. Well, maybe it's, maybe it's actually that combined with something else, which is really important, which is a self starting bias towards action, kind of a mindset, which again, everybody needs.[00:38:56] Exactly. Yeah. Everyone needs that. So if you put those two together, or if I'm truly curious about this and I'm going to kind of figure out how to make things happen, then you end up with people. Reading, reading lists, reading papers, doing side projects, this kind of, this kind of thing. So it isn't something that we explicitly included.[00:39:14] We don't have a, we don't have an ML focused interview for the AI engineer role at all, actually. It doesn't really seem helpful. The skills which we are checking for, as I mentioned before, this kind of fault first mindset. And conventional software engineering kind of thing. It's, it's 0. 1 and 0.[00:39:32] 3 on the list that, that we talked about. In terms of checking for ML curiosity and there are, how familiar they are with these concepts. That's more through talking interviews and culture fit types of things. We want for them to have a take on what Elisa is doing. doing, certainly as they progress through the interview process.[00:39:50] They don't need to be completely up to date on everything we've ever done on day zero. Although, , that's always nice when it happens. But for them to really engage with it, ask interesting questions, and be kind of bought into our view on how we want ML to proceed. I think that is really important, and that would reveal that they have this kind of this interest, this ML curiosity.[00:40:13] ML First Mindset[00:40:13] swyx: There's a second aspect to that. I don't know if now's the right time to talk about it, which is, I do think that an ML first approach to building software is something of a different mindset. I could, I could describe that a bit now if that, if that seems good, but yeah, I'm a team. Okay. So yeah, I think when I joined Elicit, this was the biggest adjustment that I had to make personally.[00:40:37] So as I said before, I'd been, Effectively building conventional software stuff for 15 years or so, something like this, well, for longer actually, but professionally for like 15 years. And had a lot of pattern matching built into my brain and kind of muscle memory for if you see this kind of problem, then you do that kind of a thing.[00:40:56] And I had to unlearn quite a lot of that when joining Elicit because we truly are ML first and try to use ML to the fullest. And some of the things that that means is, This relinquishing of control almost, at some point you are calling into this fairly opaque black box thing and hoping it does the right thing and dealing with the stuff that it sends back to you.[00:41:17] And that's very different if you're interacting with, again, APIs and databases, that kind of a, that kind of a thing. You can't just keep on debugging. At some point you hit this, this obscure wall. And I think the second, the second part to this is the pattern I was used to is that. The external parts of the app are where most of the messiness is, not necessarily in terms of code, but in terms of degrees of freedom, almost.[00:41:44] If the user can and will do anything at any point, and they'll put all sorts of wonky stuff inside of text inputs, and they'll click buttons you didn't expect them to click, and all this kind of thing. But then by the time you're down into your SQL queries, for example, as long as you've done your input validation, things are pretty pretty well defined.[00:42:01] And that, as we said before, is not really the case. When you're working with language models, there is this kind of intrinsic uncertainty when you get down to the, to the kernel, down to the core. Even, even beyond that, there's all that stuff is somewhat defensive and these are things to be wary of to some degree.[00:42:18] Though the flip side of that, the really kind of positive part of taking an ML first mindset when you're building applications is that you, If you, once you get comfortable taking your hands off the wheel at a certain point and relinquishing control, letting go then really kind of unexpected powerful things can happen if you lean on the, if you lean on the capabilities of the model without trying to overly constrain and slice and dice problems with to the point where you're not really wringing out the most capability from the model that you, that you might.[00:42:47] So, I was trying to think of examples of this earlier, and one that came to mind was we were working really early when just after I joined Elicit, we were working on something where we wanted to generate text and include citations embedded within it. So it'd have a claim, and then a, , square brackets, one, in superscript, something, something like this.[00:43:07] And. Every fiber in my, in my, in my being was screaming that we should have some way of kind of forcing this to happen or Structured output such that we could guarantee that this citation was always going to be present later on that the kind of the indication of a footnote would actually match up with the footnote itself and Kind of went into this symbolic.[00:43:28] I need full control kind of kind of mindset and it was notable that Andreas Who's our CEO, again, has been on the podcast, was was the opposite. He was just kind of, give it a couple of examples and it'll probably be fine. And then we can kind of figure out with a regular expression at the end. And it really did not sit well with me, to be honest.[00:43:46] I was like, but it could say anything. I could say, it could literally say anything. And I don't know about just using a regex to sort of handle this. This is a potent feature of the app. But , this is that was my first kind of, , The starkest introduction to this ML first mindset, I suppose, which Andreas has been cultivating for much longer than me, much longer than most, of yeah, there might be some surprises of stuff you get back from the model, but you can also It's about finding the sweet spot, I suppose, where you don't want to give a completely open ended prompt to the model and expect it to do exactly the right thing.[00:44:25] You can ask it too much and it gets confused and starts repeating itself or goes around in loops or just goes off in a random direction or something like this. But you can also over constrain the model. And not really make the most of the, of the capabilities. And I think that is a mindset adjustment that most people who are coming into AI engineering afresh would need to make of yeah, giving up control and expecting that there's going to be a little bit of kind of extra pain and defensive stuff on the tail end, but the benefits that you get as a, as a result are really striking.[00:44:58] The ML first mindset, I think, is something that I struggle with as well, because the errors, when they do happen, are bad. , they will hallucinate, and your systems will not catch it sometimes if you don't have large enough of a sample set.[00:45:13] AI Engineers and Creativity[00:45:13] swyx: I'll leave it open to you, Adam. What else do you think about when you think about curiosity and exploring capabilities?[00:45:22] Do people are there reliable ways to get people to push themselves? for joining us on Capabilities, because I think a lot of times we have this implicit overconfidence, maybe, of we think we know what it is, what a thing is, when actually we don't, and we need to keep a more open mind, and I think you do a particularly good job of Always having an open mind, and I want to get that out of more engineers that I talk to, but I, I, I, I struggle sometimes.[00:45:45] Adam Wiggins: I suppose being an engineer is, at its heart, this sort of contradiction of, on one hand, yeah,

A popular open source iOS authenticator app goes rogue under new ownership, Andreas Kling steps back from SerenityOS & forks Ladybird, Vhyrro takes a thought-provoking try at a “static effect system”, Matt Bessey is over GraphQL & Marc-Andre Giroux still likes GraphQL sometimes (in the right context).

Welcome to episode 500 of the TestGuild Automation Podcast! Today, we're diving deep into contract testing with our expert speakers, Marie Cruz and Lewis Prescott. Listen in to discover the challenges and innovative solutions for introducing contract testing for public and third-party APIs, where control is often limited. Marie and Lewis share their insights on the provider-driven and bidirectional contract testing approaches, emphasizing the importance of human communication between teams alongside automated tests. We also take a sneak peek into their book, "Contract Testing in Action," packed with practical guidance and now available with a special 40% discount until August 24th. Whether you're dealing with web, mobile, GraphQL, or event-driven services, this episode covers implementing contract testing across different types, integrating it into your CI pipeline, and the strategic shift from traditional integration tests to contract tests for early and reliable feedback. Join us as we uncover the intricacies of contract testing, tools like Pact and PactFlow, and the best practices for making it part of your development workflow. Take advantage of valuable insights and real-world examples from two of the industry's leading experts, and learn how to elevate your testing strategy to ensure seamless, bug-free software releases.

On this repeat, we talk to Max Stoiber, Co-founder of Stellate, about his experience working with GraphQL, how devs can understand and manage their data layer, and how to use his open-source project, Fuse, to simplify working in the data layer. Links https://mxstbr.com https://github.com/mxstbr/mxstbr.com https://twitter.com/mxstbr https://linkedin.com/in/mxstbr https://instagram.com/mxstbr https://mxstbr.blog https://www.youtube.com/@MaxStoiber We want to hear from you! How did you find us? Did you see us on Twitter? In a newsletter? Or maybe we were recommended by a friend? Let us know by sending an email to our producer, Emily, at emily.kochanekketner@logrocket.com (mailto:emily.kochanekketner@logrocket.com), or tweet at us at PodRocketPod (https://twitter.com/PodRocketpod). Follow us. Get free stickers. Follow us on Apple Podcasts, fill out this form (https://podrocket.logrocket.com/get-podrocket-stickers), and we'll send you free PodRocket stickers! What does LogRocket do? LogRocket combines frontend monitoring, product analytics, and session replay to help software teams deliver the ideal product experience. Try LogRocket for free today. (https://logrocket.com/signup/?pdr) Special Guest: Max Stoiber.

What if instead of sending multiple queries out to APIs and getting disparate data back, you could just send a single query and receive a single answer. That's exactly what GraphQL does for you. Rick Donato joins the show today to teach us about GraphQL and how it can help us on the path to... Read more »

What if instead of sending multiple queries out to APIs and getting disparate data back, you could just send a single query and receive a single answer. That's exactly what GraphQL does for you. Rick Donato joins the show today to teach us about GraphQL and how it can help us on the path to... Read more »

Shachar Binyamin, CEO and co-founder of Inigo, joins host Priyanka Raghavan to discuss GraphQL security. They begin with a look at the state of adoption of GraphQL and why it's so popular. From there, they consider why GraphQL security is important as they take a deep dive into a range of known security issues that have been exploited in GraphQL, including authentication, authorization, and denial of service attacks with references from the OWASP Top 10 API Security Risks. They discuss some mitigation strategies and methodologies for solving GraphQL security problems, and the show ends with discussion of Inigo and Shachar's top three recommendations for building safe GraphQL applications. Brought to you by IEEE Software and IEEE Computer Society.