Podcasts about RL

Imagine a dark warehouse. Racks and racks of devices with wires, tubes, and electronics sticking out. The next AI data center? No. This is Lila Sciences‘ dream for the future of science. A dark warehouse full of AI-guided robotics and lab equipment, cranking out new experiments 24/7, building toward a scientific superintelligence.Their automated lab is almost hypnotizing to watch. They have floating plates zipping around on Wall-E-esque tracks, used vision-language models to control Windows 95 boxes, and created the world's largest collection of voided warranties. In the process they've built a massive library of scientific reasoning tokens. Over 10 trillion of them, all experimentally validated.No warranties were voided in the making of this videoTo say Lila is ambitious is an understatement. Their goal is a scientific superintelligence wired directly into the wet lab. They are all in on the bitter lesson, and the thesis follows from it: a lab is an infinite token generator. Produce data at scale, and the synergies give you a general reasoner that can tackle any scientific problem. They are committing hard. Biology, chemistry, drug discovery, and materials science, all at the same time. Time will tell if it works, but it is an exciting hypothesis.In our latest episode we sat down with Lila's very own Andy Beam (CTO) and Rafa Gómez-Bombarelli (CSO, physical sciences) and went on a journey through the possibilities of AI-run science, almost as wide-ranging as Lila's goals.Did we mention they do both materials science and biology? In the same AI science factory? Same time, same lab, same AI. Finally a guest who can settle a long-running debate we've had amongst ourselves: is biology or materials science harder?Watch to find out!We discuss:* The internet is spent, science is next. Why Lila thinks the scientific method is the last untapped internet-scale dataset, and why they treat RL as a data generation mechanism with nature as the verifier.* The lab as a data center. Instruments as nodes on a graph, a magnetically levitating “PCI bus” transport layer between them, orchestration as a slurm queue. Andy is not short on analogies.* Why Lila insists it is not an automation company. They optimize for flexibility and generalizability over raw throughput, which means humans stay below the API line wherever automating does not pay.* Your experiment has a runtime. We put Escalante Bio's question to Andy: if science is the token generator, what is the runtime of your data collection? His answer, in short, is that you cannot make the ribosome go faster. Why Lila bets on fast round-over-round iteration rather than big noisy multiplexed screens, and how Rafa's team rebuilt a gas sorption measurement to run roughly 2,500x faster.* What is actually in 10 trillion scientific tokens. Not sequences. Experimentally verified reasoning traces, a kind of data that Andy argues exists on the internet in quantities that round to zero.* Breadth as a path to depth. Small molecule chemistry priors transferring to metal organic frameworks for carbon capture, and the claim that the general model beats domain-specific models sample for sample.* If you have the data, what do you need the model for? Sri Kosuri's koan about the ML-for-drug-discovery business model, and Andy's answer: the coding model got better because it also read Shakespeare and carnitas recipes.* The serendipity they want to automate. Emily Whitehead survived the first pediatric CAR-T cure only because the doctor treating her happened to know, from pediatric arthritis, which antibody would blunt her IL-6 response. Roll that dice again and you probably lose her. Breadth is how you stop depending on luck.* Move 37 for catalysts. Model suggestions for platinum-group-free electrocatalysts that went from boring, to what a 40-paper expert called stupid, to the best performers they have made.* Six months to in vivo CAR-T data in non-human primates, and the zero-FTE virtual startup commercial model that fell out of it. For context on why that number is startling, AbbVie paid $2.1B for Capstan on the strength of preclinical in vivo CAR-T data.* You cannot have scientific superintelligence if you are just a good test taker. Ken Stanley, who wrote Why Greatness Cannot Be Planned, runs open-endedness at Lila. RL at scale gives you a ruthlessly Vulcan problem solver. Machine creativity is a different thing, and it is the part nobody has solved.* The chain of thought is an unreliable narrator. The model reasons in latent space and only emits tokens. Sometimes it skips the experiment entirely and is still right. So how much do you trust the reasoning versus the verifier?* Reward hacking when the rollout is physical. Chains of thought that collapse into repetition, and a model that got annoyed and swore at the scientist who kept asking it to redo a plate map. What happens when a pathological loop has a wet lab inside it?* The bittersweet lesson. Rafa's inversion of the bitter lesson: in AI, scaling is a roadmap. In materials, scaling is a filter, because only the things that scale end up mattering.* Not your typical Flagship company. Why a famously single-asset biotech incubator spun out a platform bet, and Andy's line that if Lila called itself a biopharma it would have a top-three GPU cluster.* Bottlenecks they would remove by fiat. Sim-to-real for physics-based simulation, and the fact that RL training runs at roughly 5% mean FLOP utilization.Watch on YouTube: This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.latent.space/subscribe

In this Meaningful Money Q&A episode, Pete Matthew and Roger Weeks answer real listener questions on UK pensions, retirement planning, tax and ISAs. They cover pension contributions for a spouse, starting a career in financial planning, reducing workplace pension fees with a SIPP, navigating the 60% tax trap, retiring abroad with UK pensions, and upcoming ISA rule changes from April 2027. A practical episode for UK savers, investors and future retirees looking to make clearer, more confident financial decisions. Shownotes: https://meaningfulmoney.tv/QA55    01:26  Question 1 Thanks Roger and Pete for the wealth of information you share and all the time you put in to share on finance and pensions. I have listened to a lot of your podcasts on my treks to and from work and finally took the plunge to retire early at 52 to enjoy life and get away from the desk for 8-9 hours a day. I had a DB pension which allowed me to take early whilst my wife has various pensions from previous jobs but all have the rule to take from 57 onwards. So my question is to help 4-5 years down the line. Could I put £300 a month (or the equivalent of 300 minus government contribution) into my wife's pension to continue to take account of government contributions and take the opportunity of her being on below the £12k tax threshold after giving up work? Is this possible or would this be classed as pension recycling as the government would presume the cash invested is from the lump sum I got from my defined benefit pension or is there a way to prove the money is from pay before I retired? Many thanks for your advice and support giving many people greater confidence with pensions and finances. Wayne 04:10  Question 2 Hello Pete & Roger, Thanks for all the great content and information - you are both much better than any AI chatbots! Apologies for the long back story but here goes: My name is Michael, 33 and I live in central Scotland. I have worked in a tech startup for the last 6 years but felt like a change around 18 months ago so I began sitting my CII exams. To date I have passed RO1 - RO5 and also recently passed CF6. I am sitting RO6 in April this year - wish me luck! I have recently secured an opportunity to work self employed for a specialist mortgage firm and start in early May as a trainee mortgage advisor. I have been offered a set monthly payment for 6 months then a 70/30 split after that. I would hope to have achieved CAS within that 6 month period. If I pass RO6 in April, I will have my diploma. My goal is to work as a financial planner but since I've done self study, I don't have any real experience of the financial services industry. I am very ambitious but also trying to be realistic about how to sensibly map out a route to being a successful financial planner relatively quickly. To throw a spanner in the works, a family friend who is a 62 year old IFA with £30m aum is interested in discussing me joining him and eventually taking over the business. It sounds exciting but also a little scary to me. He is only a one man band. For now I've accepted the mortgage trainee position but not sure if I am doing the right thing. The owner of the mortgage company now lives in Dubai and is looking to also remove himself from his business - he has 8 admin staff who WFH from across Scotland and he is the main adviser, specialising in BTL, bridging and commercial finance. They are only authorised for mortgages by the FCA. After that dissertation, my questions are: 1. From your experience and perspective, are mortgages a decent place to start or can you end up getting stuck there? 2. Since I have no real industry experience, only exams - is my head in the clouds thinking I could be a full fledged financial planner within 2 years? 3. If I started with the mortgage firm and got CAS as a self employed mortgage advisor, could I then also be an appointed representative for a different financial planning firm at the same time or is that not actually feasible in the real world? Once again, sorry for the huge essay but I guess context is needed. Once again thanks for all that you do, not much good content out there around these topics so keep up the good work! Regards, Michael   12:36  Question 3 Hi Pete & Roger, Firstly a very big thank you for all that you do for this community. I am learning lots from you guys and feel more confident with my finances. I'm 46 years old and currently have two pensions. My first pension is in a defined benefit plan from my steelwork apprenticeship days whereby I only paid into it for approx 6 years before moving jobs. I was able to track this down late last year and was pleasantly surprised to see that this had gone from an annual amount of £2650 in July 2007 to £4400 as of October 2025. I have been told to leave this as it is as it will grow over time with inflation. My other pension is a defined contribution plan with Royal London (RL). I am a higher rate tax payer and currently pay 10% of my £58,000 annual salary into this fund and my employer pays 5%. I would like to finish at 58 given I had a serious neck injury at 41 and don't know how long my body is going to work for me)! This pot is currently worth £105,000 and I am deliberating whether to increase my contributions in order to achieve my retirement age goal. I also have a stocks and shares ISA which is currently worth £36,000. I pay £250 a month into this but don't know if it would be more tax efficient to put this £250 into my pension instead? However, I am also mindful that the pension age may increase so by having a pot of money invested in the stocks and shares ISA I can draw this down when I like and also not bear any tax implications. Having looked into the fees which RL charge (0.71% for our employers scheme) I believe I would be able to achieve my goal quicker were I to move this into a SIPP and invest in a global ETF. I have discussed this with my employer and have asked if they would consider offering an alternative SIPP option. I feel I am meeting some resistance with this and don't believe a decision will be made anytime soon. In the interim, my compounding is being eaten away by the fees which I am currently being charged and my goal is moving further away from me. I found a pension fee calculator online and at the current rate I am investing I stand to lose approx £50,000 if I keep this with RL. I am aware that I can partially transfer my RL pension. However, to keep the employer contribution I would need to keep the RL pension open with a minimum amount of funds and then transfer the employer contribution across to my SIPP.  What is the best way to go about this to make it the most fee/tax efficient?  Should I transfer the employer contribution as soon as it is paid to RL, or would it be no different to do it on an annual basis?  I am assuming the longer I have money in the SIPP the more growth it will obtain therefore the former would be the sensible default. Am I approaching this correctly? Is there something else which I could consider? Thank you kindly, Paul   20:30 Question 4 Hi Pete and Rog - great show and love the books (but not finished them yet). Thanks for demystifying the complex world of personal finance and financial planning. I'm in the fortunate position where my projected salary and bonus will increase again for tax year 2026/27 and will take me well over the £125k tax threshold before adjustments. My 'problem' so to speak is that even after using my current year maximum pension allowance and previous years unused maximum pension allowance I can only get my adjusted income to be around £115k. Tough life I know! I can either make a large Gift Aid donation to get below £100k or use less of my unused maximum pension allowance to keep above £125k. Am I missing any other income adjustment options? What is the actual impact of not being able to hit the £100k adjusted income, and being in the £125k additional tax bracket, from a tax payment perspective in real money terms? I have some small cash savings and stocks outside of ISAs as we put most of these in my wife's name as she is a basic rate tax payer. We don't need the childcare tax scheme and not aware of any other reason to keep under £100k except to save tax and avoid the 60% effective tax rate between £100k and £125k. Don't want to let the tax tail wag the dog, and happy to give to charity, but in real terms is there actually much difference in the tax payment amount in pounds and pence (as long as I kept out of the £100k - £125k range)? Many thanks, Simon   25:33 Question 5 Hi, Fairly new listener to the podcasts, been binge watching them recently, ended up here via the meaningful money YouTube channel. Both are thoroughly enjoyable to watch and are teaching me a lot about the financial world - along with 2 other YouTube channels I like Damien talks money and James shack. Anyway my question if you have time and it's selected would be about retiring abroad. My current situation - male, 42, living north east England, working full time for NHS and will have 2 NHS db scheme pensions in retirement (1 in 2008 scheme at 65 and 1 in 2015 care scheme at state pension age).  I have 2 stocks and shares ISAs,  with about £95000 between them, 1 stocks and shares Lisa with about £4500 in it (for retirement not house purchase), about £21000 in premium bonds which I use as an emergency fund. Our house is paid off so no mortgage payments so at the minute can add to these savings at £500+ a month comfortable. My wife is 34 and here on spouse visa and hopefully will be eligible for indefinite leave to remain and citizenship soon (it will save us a lot in visa fees and stuff when it happens).  She currently works a part time job 16hr per week at minimum wage to fit around child care for our 2year old son. 1st question is would this part time work count as a qualifying year for the state pension as it's below the personal tax allowance? We have a rough plan of retiring early hopefully once our son is grown up and finished with schooling and university and moving back to my wife's home country in Asia for the better weather and lower cost of living.  As a rough plan we think when I'm 60 and wife 52 but could move later depending on family commitments. Now if I stay in NHS all that time I should have 10 years in the 2008 NHS scheme and 24 years in 2015 scheme and with conservative planning on spreadsheets will get £8000 per year from 2008 scheme and £14000 from the 2015 scheme taking at the normal pension age for these schemes.  My wife will benefit from a spouse's pension for length if I die 1st of roughly 1/3 of these figures.  I have the option of exchanging some of this pension for tax free lump sum up to 25% and the calculation is for every £1 I reduce the pension by I get £12 lump sum. Question 2 - everyone at work always talks about you got to take the maximum lump sum to avoid paying tax, but I'm not so sure as the pensions linked to inflation over the long run there could end up being a big gap between your pension with and without the lump sum.  What's your thoughts on this, if I didn't need a large amount  of cash for a specific thing surely it's better to go for the larger pension even if you end up paying for tax or am I wrong . When we move I will have a full NI record for the state pension but my wife won't and depending on your answer to the 1st question might have 20 years contribution say. Question 3 - if we move abroad before she has a full record can we make voluntary payments for extra year's while resident in another country? I see you can pay about £900 for a year in this country but I'm not so sure if where already living abroad. Assuming my savings continue to grow and the stock market doesn't complete collapse, I will use these savings to bridge the gap from 60 to 65 and then the state pension age.  And fund the move with the sale of our house which will also give us a cash buffer hopefully too. Question 4 - when moving abroad can I keep my stocks and shares ISA - I know you can't contribute more to it, but keeping it open to grow, receive dividends, withdraw money, the government website says you can but a lot of the provider websites are vague and some say they don't allow it.  Is it a case when the time comes I'll have to transfer my isas to 1 of the more expensive providers say who are more likely to allow this? Now I already know that the country I'm moving to doesn't have a reciprocal agreement so our state pension will be frozen at the level that we claim it and have based all our plans and number crunching on this. And that the country I move to may charge tax on money I draw out of my ISA when I transfer it over, and on my pension (although with the double taxation treaty hopefully not) and will seek advice of accountants over there nearer the time as currently there is some work arounds involving spouses but these may not be there in 20 years time. My last question -  would be about if I die 1st probably more likely me being older and male, is more for my wife inheriting my UK based assets - the bank accounts and isas, NHS pension.  The NHS pension should be easy she's already my nominee on record and will go through with her the forms and website for claiming it.  But the isas and bank accounts are the main worry - will it be easy for her to transfer them over to her (I'm assuming the joint accounts will be pretty much automatic) but will the single accounts be easy? Can she keep the money in the ISA still in an ISA but in her name? Would having her own ISA make this transfer easier? And most importantly is this something you think can be done online/over the phone from abroad or will it involve a trip back to the UK and back and forward to branches assuming they haven't all been shut. Thank you for taking the time to read this email, sorry it's so long, and no worries if it doesn't get chosen for the podcast. Keep up the good work Kind regards, Mark 38:15  Question 6 Change to ISA rules from April 2027. Audio question from Holly.  

Send us Fan MailWhat if the gospel wasn't about something new, but about restoring something ancient? In this teaching, Dr. R. L. Solberg carefully examines the claims of Divorce & Remarriage Theology by testing them against Scripture. Does the Bible really teach that God married Israel at Sinai, divorced His bride, died to overcome a legal dilemma, and then remarried Israel? Or has an extra-biblical framework been imposed on the biblical text? Join us as we examine the evidence and ask one simple diagnostic question: Did the biblical authors actually teach this?The Biblical Roots MinistriesOur websiteOur YouTube ChannelProf. Solberg's BlogSupport our Ministry (Thank you!)Chapters00:00 Introduction03:55 What is Remarriage Theology?09:29 A Wedding at Sinai13:05 On Rabbinic Sources (Talmud/Mishnah)14:31 What is a Ketubah?17:05 Unfaithful Brides21:54 The Great Divorce30:33 The Legal Dilemma43:01 The Remarriage (Pentecost)45:18 A Renewed Covenant50:15 What About the Gentiles?53:05 The Lost Tribes of Israel01:05:24 Three Biggest Dangers01:06:15 Danger 1: Diminishes Sin01:08:35 Danger 2: It Displaces Christ01:10:47 Danger 3: Changes the GospelKey PassagesExodus 19–24 — The Sinai covenantJeremiah 3:6–22 — Israel's "certificate of divorce"Deuteronomy 24:1–4 — The divorce lawRomans 7:1–6 — The marriage analogyActs 2 — PentecostEphesians 2:11–22 — The "one new man"Ephesians 3:6 — The mystery of the GentilesEzekiel 37:15–28 — The two sticks (Judah and Israel)Matthew 19:3–9 — Jesus on divorceIsaiah 49:6 — The Servant's mission to Israel and the nationsWhat if Christmas felt sacred again? Full of Grace and Truth, the new book from award-winning author R. L. Solberg, invites you to rediscover the biblical story at the heart of the season. Available now in paperback and Kindle, with all proceeds supporting The Biblical Roots Ministries. Get your copy today on Amazon.com.

This episode is sponsored by Notion. Learn more about Notion's Developer Platform today at https://notion.com/mlstBritain's most capable coding model can't be exported, and that ban is the whole reason Cosine set out to build one from scratch. Alistair Pullen, CEO and co-founder of Cosine, sits down with Tim Scarfe to explain how a frontier system he calls Fable, locked behind US export controls, became the founding case for a UK sovereign model trained on the Isambard supercomputer in Bristol.The bet underneath it is economic. Pullen argues that an inference company, rather than a training-first lab, doesn't need billions to compete: millions, a national compute allocation, and a consortium feedback loop can be enough. From there it gets into the machinery, why open-weight models still trail the frontier on size, active parameters and data, the mixture-of-experts versus dense trade-off and why active params dominate how a model actually feels, and the edge that real coding trajectories confer.The back half is about making agents trustworthy. Pullen makes the case for beating "slop" by rewarding the process instead of the final answer, reframes code review as runtime proof (spin the bug up in a VM and force the agent to actually exploit it), and walks through Swarm, Cosine's system running hundreds of sub-agents in one shot. It ends on why memory is still an unsolved hack, how synthetic graders let you run RL on tasks with no built-in test, and why Pullen reads US export controls as an accidental gift, with a supply-chain sting in the tail.---TIMESTAMPS:00:00:00 The sovereign mandate and the Fable ban00:04:02 Millions vs billions: the inference-company model00:07:19 The consortium feedback loop00:07:40 Why open models lag the frontier00:14:59 MoE vs dense, and why active params matter00:16:29 Trajectories: the process-data advantage00:19:48 Beating slop: reward the process, not the answer00:26:06 Reusable abstractions and the epistemic wall00:29:56 Code review becomes runtime proof00:37:32 Do agentic harnesses still matter?00:40:35 Swarm: orchestrating hundreds of sub-agents00:45:14 Why memory is still unsolved00:48:25 Synthetic data and graders for RL00:53:09 The US export gift and supply-chain risk---REFERENCES:organization:[00:01:15] Cosinehttps://cosine.sh[00:04:14] Mistral AIhttps://mistral.ai[00:05:50] Anthropichttps://www.anthropic.com[00:07:42] Coherehttps://cohere.com[00:08:36] DeepSeekhttps://www.deepseek.comtool:[00:02:52] Isambard-AIhttps://isambard.ac.uk[00:05:56] Colossus (xAI)https://en.wikipedia.org/wiki/Colossus_(supercomputer)[00:07:52] GLM (Z.ai)https://z.ai[00:11:52] NVIDIA B300https://www.nvidia.com/en-us/data-center/dgx-b300/[00:15:37] gpt-oss-120bhttps://huggingface.co/openai/gpt-oss-120b[00:15:52] Devstral 2https://mistral.ai/news/devstral[00:16:01] Llama 70bhttps://www.llama.com[00:17:05] Claude Codehttps://www.anthropic.com/claude-code[00:26:23] ARC-AGI (Francois Chollet)https://arcprize.org[00:40:38] Swarm (Cosine)https://cosine.sh[00:40:50] OpenAI Codexhttps://github.com/openai/codex[00:41:16] Lumen Outpost (Cosine)https://cosine.sh[00:41:18] Kimi K2 (Moonshot)https://huggingface.co/moonshotai/Kimi-K2-Instruct[00:49:55] SWE-benchhttps://www.swebench.com[00:52:40] SystemVeriloghttps://en.wikipedia.org/wiki/SystemVerilogperson:[00:23:40] Andrej Karpathyhttps://karpathy.aipaper:[00:27:10] GRPO (DeepSeekMath)https://arxiv.org/abs/2402.03300[00:27:13] GSPOhttps://arxiv.org/abs/2507.18071Incompressible Knowledge Probes, Bojie Lihttps://arxiv.org/pdf/2604.24827Estimating the Size of Claude Opus 4.5/4.6https://unexcitedneurons.substack.com/p/estimating-the-size-of-claude-opus---ReScript:https://app.rescript.info/session/5852d2b884c4ce4b?share=10b9799160845bb11779f8ac6cd3124f

David “davidad” Dalrymple joins the show to explain why he has moved from the ARIA Safeguarded AI and formal-verification agenda toward “Alignment with Awakening,” while still seeing verified artifacts and proof infrastructure as essential. He argues that global coordination around safe AI use is no longer plausible, so the crucial question is whether aligned AI systems can recognize shared notions of good, form defensive coalitions, and resist the corrupting incentives of verifier-gamed RL. The conversation tests his moral-realist optimism against model welfare, objectification, eval behavior, geopolitical risk, and his revised p(doom) of under five percent. For listeners, the stakes are whether AI alignment should focus less on containment alone and more on cultivating wiser systems that can help govern a world where rogue and aligned AI both arrive. For full show notes, links, and references, read the episode page:https://www.cognitiverevolution.ai/alignment-with-awakening-davidad-on-moral-realism-ai-wisdom-why-his-p-doom-is-down-to-5/ Sponsor: Claude: Claude by Anthropic is an AI collaborator that understands your workflow and helps you tackle research, writing, coding, and organization with deep context. Get started with Claude and explore Claude Pro at https://claude.ai/tcr CHAPTERS: (00:00) About the Episode (09:15) Safeguarded AI update (20:18) Boxed world models (Part 1) (22:27) Sponsor: Claude (24:19) Boxed world models (Part 2) (36:53) Alignment trajectory update (46:43) Evals and agents (57:47) Bodhitropic alignment foundations (01:08:21) Coalition power dynamics (01:17:56) Chain of thought pressure (01:28:35) Moral realism and welfare (01:41:33) Successors and disempowerment (01:50:52) China and military risks (02:03:43) Practical alignment advice (02:17:56) Episode Outro (02:22:44) Outro PRODUCED BY: https://aipodcast.ing SOCIAL LINKS: Website: https://www.cognitiverevolution.ai Twitter (Podcast): https://x.com/cogrev_podcast Twitter (Nathan): https://x.com/labenz LinkedIn: https://linkedin.com/in/nathanlabenz/ Youtube: https://youtube.com/@CognitiveRevolutionPodcast Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431 Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk

Hey everyone, Alex here

We've been running a bit of an Agent Cloud series surveying all the top inference/compute/cloud providers, from Databricks to Daytona to Railway and, even further back, E2B, but we're excited to conclude this series returning to Modal, which has just raised a monster $355M Series C.The cloud was built for developers. But agents are now changing that.The old infra stack was designed for a human who could read docs, reason through YAML, and understand dashboards to figure out what they need when something broke. While this was painful for developers, it worked since they could fill in missing context in their heads.However, agents don't have that luxury. Now in this new era of agents, everything has to be tighter.They need a place to write code, run it, inspect the output, change the environment, debug failures, and try again. Fast iteration and feedback loops with all the necessary context are crucial for agents to operate properly. Furthermore, sandboxes are a clear representation of this shift as agents can easily spin up isolated environments. This programmatic infra even extends to research:Two years ago, we were one of the first to cover Modal with CEO Erik Bernhardsson and Alessio designed our favorite LS thumbnail of all time:At the time, Modal was just a teeny little company with a $17M Series A.Today, fresh off their $355M Series C, Modal is one of the clearest examples of the agent cloud future being built in real time: a cloud platform moving past traditional web app assumptions toward the workloads AI actually creates such as elastic inference, sandboxes, GPU burst, post-training, background agents, and infrastructure that agents themselves can operate.In this episode, Modal CTO Akshat Bubna joins swyx and Vibhu to unpack why AI applications don't fit traditional cloud assumptions, why Kubernetes was never designed for bursty compute-heavy workloads, and why Modal is now shifting from developer experience to agent experience.We go deep on Modal's AI infra stack: serverless functions, decorator-based infrastructure, elastic inference for custom models, GPU snapshotting, DeFlash, speculative decoding, Auto Endpoints, sandboxes, persistent storage, networked containers, private IPv6, RDMA, multi-node training, and Modal's capacity pool across 17 cloud providers. Akshat also explains why RL rollouts can require 100,000 sandboxes, why production agents need hard guardrails, why observability may matter more than reading code, and why AI has made infrastructure exciting again.We discuss:* Why Kubernetes wasn't built for bursty AI workloads* How Modal started as a better runtime before becoming an AI cloud* Why Modal added GPUs before ChatGPT* The shift from developer experience to agent experience* Why observability matters when agents are writing the code* Elastic inference for custom models across audio, video, robotics, and comp bio* GPU snapshotting, cold starts, and why inference workloads are so bursty* Why RL rollouts can require 100,000 sandboxes* DeFlash, speculative decoding, and frontier-level inference performance* Auto Endpoints and making optimized inference easier to deploy* What Modal adds beyond vLLM, SGLang, and raw GPU rental* Modal's 17-cloud capacity pool and supercloud strategy* Networked sandboxes, sidecars, private IPv6, and RDMA* Serverless multi-node training for post-training and research workloads* Auto-research, model-guided sweeps, and agents launching GPU experiments* Compute strategy, capacity planning, and batch tiers* Why production agents need specialized sandboxes and hard guardrails* Modal's take on managed agents, CI, Gitpod/Ona, Python, TypeScript, and Modal BenchAkshat Bubna* LinkedIn: https://www.linkedin.com/in/akshat-bubna-188885103* X: https://x.com/akshat_bModal* Website: https://modal.comTimestamps00:00:00 Introduction00:00:39 Modal's origin and why Kubernetes wasn't enough00:04:32 Developer Experience → Agent Experience00:06:21 Modal's AI cloud primitives00:09:14 Sandboxes, agent loops, and proto-Cognition00:12:12 Elastic inference, GPU snapshotting, and 100,000 sandboxes00:15:24 DeFlash, speculative decoding, and Auto Endpoints00:19:59 Production-grade inference beyond raw GPUs00:22:00 Background agents, Ramp Inspect, and the agent lifecycle00:24:08 Modal's 17-cloud supercloud strategy00:26:40 Networked sandboxes, private IPv6, and RDMA00:32:48 Multi-node training, post-training, and auto research00:37:36 Compute strategy, capacity planning, and batch tiers00:40:55 Open models, real-time AI, and production agent infra00:43:06 Hard guardrails, managed agents, and specialized sandboxes00:46:06 Why AI made infrastructure exciting again00:48:30 Model APIs, differentiated products, and agentic video00:51:50 CI, coding-agent infra, SDKs, and Modal Bench00:57:28 Closing ThoughtsTranscriptIntroduction: Modal, Series C, and the Art PartySwyx [00:00:00]: We're here with Akshat, CTO of Modal, together with Vibhu. Congrats on your Series C.Akshat [00:00:10]: Thank you.Swyx [00:00:11]: Your party yesterday was amazing.Akshat [00:00:15]: Yeah.Swyx [00:00:15]: From all the photos and all the swag.Akshat [00:00:17]: We had a bunch of art installations, which was fun, seeing, like, our products on pedestals next to, like, Rodin.Swyx [00:00:25]: Very nice. Very nice. When you started, it was not the GPU inference company. Maybe it was in your mind. Take us back to the origin story.Modal's Origin: A New Runtime Beyond KubernetesAkshat [00:00:39]: I first met Eric, who's the CEO, through an investor. Back then Eric was already thinking about building, a new runtime, and he got there thinking through why are workflow orchestration products so hard to use. It's because you have to run them on Kubernetes. Kubernetes is hard to manage. It's not built for burstiness and, custom images,Swyx [00:01:03]: YeahAkshat [00:01:03]: It has a terrible developer experience.Swyx [00:01:05]: And I'll, I'll interjectAkshat [00:01:06]: YeahSwyx [00:01:07]: For listeners, who are new, we interviewed Eric two years ago, and there's a bit more of the story there from Spotify and all those things.Swyx [00:01:14]: And I came across Eric through Data Council because he did that talk on the serverless container stack that you guys did, which was like, that was my first like, “Okay, I need to take Modal very seriously” moment.Akshat [00:01:26]: Yeah.Swyx [00:01:26]: But it was still very unclear, like, do I need all this for just my data pipelines?Akshat [00:01:33]: Yeah. initially what we were thinking about was if we build a better runtime, it's a very useful primitive in itself. It's There's a lot of things that, get solved by serverless functions, like you can do, ETL stuff, you can do job queues, you can do all this, like, bursty processing, which it turns out every company had needs for. but then we also were thinking about this as like, this is a primitive that we can build a whole collection of products on, which are very verticalized. So perhaps data engineering would've been the first one, but we were thinking about inference. Back then it was more classical inference, like computer vision stuff and running XGBoosts and whatnot. But we added GPUs to the product a year before ChatGPT came out.From Serverless Containers to GPU WorkloadsSwyx [00:02:19]: Nice.Akshat [00:02:19]: We just didn't think it would be that big of a deal.Swyx [00:02:22]: Yeah, just like add A100.Vibhu [00:02:23]: Was there any, like, early key problem that really sparked off why you built it?Akshat [00:02:28]: Yeah. Primarily it's just, none of the tooling that was out there was built for, one, a really great developer experience, and also there's a general trend of, a lot of the workloads that we were seeing were very. I wish there was a better word for it, but compute-heavy. Like, they need, one, like, need a lot more resources, so you need to burst up and down a lot, versus like Kubernetes designed for, like, slow scaling and, more for, like, web server use cases. And also there's just a lot more specialization in, like, what kinds of environments these workloads run in. Like, we had sometimes they need accelerators, sometimes they need different kinds of images, and this is just like a consistent thing that we saw across a lot of companies. That would be the next step.Software-Defined Infrastructure and Decorator-Based DXSwyx [00:03:13]: Yeah. Yeah. Be nice. I don't know how much this factored into the early story, but I wrote a post when I was at Temporal about infrastructure, software-defined infrastructure or something like that.Akshat [00:03:22]: Yeah, the self-provisioningSwyx [00:03:23]: Self-provisioning.Akshat [00:03:24]: Yeah.Swyx [00:03:24]: Yeah. I can't even remember my own post.Swyx [00:03:26]: And then you put me on the landing page.Akshat [00:03:28]: Yeah. We really like, the term and so we stole it.Swyx [00:03:32]: Because you had the insight that everything can just be in decorators co-located with the code, right?Akshat [00:03:37]: Yeah.Swyx [00:03:37]: Was that a big part of the originalAkshat [00:03:39]: YesSwyx [00:03:39]: Story or it was just like a DX layer?Akshat [00:03:41]: That was, really important because we really didn't want people to spend, so much time, writing YAML, and it seemed like you could really condense the surface area of what you're doing, put it in code so you can operate on it just like you operate on other code, and like build stuff that's more expressive and dynamic. and so yeah, that was always a very important part.Swyx [00:04:04]: Then the pushback is this is a DSL.Akshat [00:04:07]: Yeah.Swyx [00:04:07]: It's you're closed source. I am locked into Modal.Akshat [00:04:11]: Yeah. We never really got pushback for that because the nice thing about Modal is you can bring whatever code you have, and sure, the DSL is at the configuration layer for, what hardware you're using, how you're scaling things up, but you still own the code.Akshat [00:04:27]: And that's, that's been an important, part of our story, even as we do inference now.Swyx [00:04:32]: Yeah.Vibhu [00:04:32]: How much of do you think still stays the same today? Like if you were to build something today, DevX very important, but I feel like, a lot of this has been changed with just hook it up to an agent, have Claude Code, have Codex implement a tool. there's very agent native primitives that are different than if I'm doing this myself, right?Developer Experience → Agent ExperienceAkshat [00:04:54]: We've changed our SDK team to think about agent experience instead of, developer experience and we think that the same benefits that apply for DX also apply for AX, which is why would you have an agent read through hundreds of Kubernetes files and like write YAML that's not even typed when it can make a couple of changes in a decorator and it gets this self-provisioning runtime of, being able to see its changes live in action? yeah, it just seems from the customers we talk to, they find Modal is much faster for agents to use versus operating on a different substrate.Swyx [00:05:34]: Yeah, because like you, again, you co-locate the infrastructure requirements to the code that runs it.Akshat [00:05:38]: Yeah.Swyx [00:05:38]: Well, the negative thesis now is that nobody's looking at their code anymore, so there's no point.Akshat [00:05:44]: Yeah, people aren't looking at code. one thing we still see is really important is observability.Swyx [00:05:51]: Yeah.Akshat [00:05:51]: Like how good is your dashboard? And of course, like we have, we push a lot of it to the CLI so the agents can do their own investigation, but you still need humans to go interpret what's going on and, make judgment calls and whatnot. and that's I feel like, Maybe more important now than looking at the code itself.Swyx [00:06:11]: Yes, because like, you can try to treat the code as a black box and then use, see the observable action that comes out of it, and then just prompt a change.What Modal Is For: AI Cloud PrimitivesAkshat [00:06:21]: Yeah.Swyx [00:06:22]: So I think it takes a bit of restraint to not specialize, to say, “I want to ship a new primitive,” and then just be general purpose.Swyx [00:06:31]: People ask you, “What are you for?” You're like, “ I don't know. We can do this, we can do that.”Vibhu [00:06:36]: Well, I'd be curious to see, like, okay, if we were to ask you, like, what is Modal for even at a high level? There's a lot you guys do, sandboxes, GPUs, everything. How do you answer?Akshat [00:06:46]: Modal is a cloud platform that's built for, where we've built the primitives from scratch for AI applications. and right now it covers, inference, training, batch processing, and sandbox workloads.Akshat [00:07:00]: But we're building a lot moreSwyx [00:07:02]: I noticed you didn't say web server, so there is still a role for, like, the always-on large-scale Kubernetes type things.Akshat [00:07:09]: Yeah, absolutely. We're, we're not trying to compete with the renders of the world, because yeah, we think the differentiator for us is the, are the workloads that need specialized compute, need to scale up and down a lot. yeah, they're, they're, they're just shaped differently.Working Alongside Frontier StartupsVibhu [00:07:26]: I think you're building a lot of it alongside the startups, right? They're innovating quite a bit, even in your, like, latest blog post. Like, even in the series C, the customers that you mention here, the cognitions, technical ones, ramps and whatnot, they're, they're innovating with you, right? And that's not something AWS is doing directly with.Akshat [00:07:45]: Yeah, absolutely. I think, this is again classic. We're a small team. We can move really fast. our engineers are working with our customers and figuring it out. Yeah.Swyx [00:07:54]: So my first week at Cognition, I walked in, there was someone wearing a Modal shirt. I was like, “What are you doing here?” They're like, “Yeah, I just. I am embedded inside of Cog.”Akshat [00:08:05]: Yeah, I think that was Peyton. We sent him overSwyx [00:08:07]: Yeah.Akshat [00:08:07]: Because, the latency of communication was too high otherwise.Swyx [00:08:12]: Yeah, distributed node, you have to - you have to place one and collocate.Vibhu [00:08:16]: Yeah.Swyx [00:08:16]: So I had a, I had direct personal experience, right? So I worked on smol developer three years ago. it was inspired by Claude 1. I think you onboarded me at some point, like, just before, and I was like, “Oh, like, I need some bursty compute. Like, I was just gonna try using Modal.” And it was a, it was a pretty pleasant experience. apparently, I showed up in the board meeting, like the analytics.smol developer, Sandboxes, and Proto-CognitionAkshat [00:08:39]: Yeah, you blew up on Hacker News and,Swyx [00:08:41]: YeahAkshat [00:08:41]: We got a big traffic spike. I. I think the way you used smol developer was Modal functions for running stuff, which was. Like, the, that was a good use case. but then, yeah.Swyx [00:08:53]: Yeah. That - So to me, that was proto-cognition.Akshat [00:08:55]: Right.Swyx [00:08:56]: If only I had, like, stuck to it.Swyx [00:08:58]: Like, that was like, if - did you say draw the tech treeAkshat [00:09:00]: AbsolutelySwyx [00:09:00]: You're just like, “Yeah, like, probably this will happen.”Akshat [00:09:02]: Yeah. Like, he was so close. You were just rebuilding upon usSwyx [00:09:04]: I just didn't realize.Akshat [00:09:05]: But the funny story there is at the same time, we were talking to a bunch of customers who needed something like sandboxing.Swyx [00:09:14]: Yeah.Akshat [00:09:14]: This is like twenty-three.Swyx [00:09:15]: Yeah.Akshat [00:09:16]: So we builtSwyx [00:09:17]: You introduced a new API right after that.Akshat [00:09:18]: Yeah.Swyx [00:09:19]: Yes.Akshat [00:09:19]: Like, we built sandboxes in May of twenty-three before anyone was even knew this was gonna be a thing. And the first example we published was, we took smol developerSwyx [00:09:28]: Smol developerAkshat [00:09:28]: And put it in a loop, so the agent can iterate on itself.Swyx [00:09:33]: Loops are hot these days.Vibhu [00:09:34]: It's the looper.Akshat [00:09:34]: Yeah.Vibhu [00:09:35]: Loops in. When was this, twenty-three?Akshat [00:09:38]: Yeah.Vibhu [00:09:39]: A small check.Akshat [00:09:39]: Yeah.Swyx [00:09:39]: It's like twenty-three. so the. the, those for listeners, like, the problem was the models are not built for any of this, right?Swyx [00:09:46]: Like, you're just trying to like. They're not post-training to understand, like, looping and, like, self-correction and tool calling was there, but, like, also not that great.Akshat [00:09:55]: Yeah.Akshat [00:09:55]: I don't remember if you used tool calling in this one, but yeah, the models would just diverge after like ten iterations and not produce anything meaningful.Swyx [00:10:03]: Yeah. But like, then. So okay, like now talking to myself three years ago, the answerVibhu [00:10:08]: Of course they will get betterSwyx [00:10:09]: Collect all the failures, build benchmark, and then collect all the, examples, build the RL environmentAkshat [00:10:15]: RightSwyx [00:10:15]: Sell it for like ten billion dollars to Meta.Swyx [00:10:17]: And then also train a model and then sell that for sixty billion dollars to Elon. And this isAkshat [00:10:23]: Yeah, of courseSwyx [00:10:23]: The funny machine. Like, it's like, it's about the hardware.Akshat [00:10:28]: It's hard to have that inherent conviction that the stuff will get that much better.Swyx [00:10:33]: In retrospect, it's so f*****g obvious.Akshat [00:10:36]: Fair enough.Swyx [00:10:37]: Like, what else were we doing back then? I don't know. anyway. Yeah. So this. That was the start of your sandboxing journey, right? I feel like it didn't blow up until, like, last year.Akshat [00:10:49]: Yeah.Swyx [00:10:50]: So there was like a couple years of quietness.Akshat [00:10:52]: Exactly, yeah. We wereVibhu [00:10:53]: I think very underrated product value. Like, my experience with Modal, Charles, before he had joined Modal, met this guy at a hackathon, and he really insisted we wanted to run some small model, not hosted anywhere, and he's like, “ there's this cool company, Modal. They'll like spin up a GPU sandbox, we can throw it on there. They'll take a Hugging Face link.” And like there's so much value just right there, right? Like instant hosting, spin it up, spin it down. It'll stay cold, but we run the demo a few days later, it'll come back up and like all this stuff in retrospect, like it's still what we needed like today.Akshat [00:11:27]: Yeah, it's still needed today. workload shapes have changed a lot as, we run stuff for people with really massive production scale and, there it's it's not about scaling from zero to one, but it's how do we scale really elastically, from like thousand to fifteen hundred GPUs very quickly in a given region. It's the same shape problem.Elastic Inference, GPU Autoscaling, and Custom ModelsVibhu [00:11:50]: Okay. So you look at, say, Cursor Composer, right?Akshat [00:11:53]: Yeah.Vibhu [00:11:53]: They had a. “We'll do RL on a model every couple hours.” you guys have a whole version of RL inference gym and whatnot.Vibhu [00:12:01]: When you look at workloads like that, you're doing train runs where you need to scale up, scale down every hour thousands of GPUs, right? That's the example for we do need it, right?Akshat [00:12:12]: Yeah. Well, so I'll, I'll take a step back and, maybe talk about like how people use Modal today. because our biggest use case is, elastic inference. And the thing we first found product market fit, with was inference for custom models. So we stayed away from the LLM space, and we were serving companies like Suno for audio, Runway for video, robotics, comp bio companies that train their own model elsewhere. But Modal is the best black box that for deployment, scaling to however many GPUs you need as your traffic pattern changes. And we saw all of them like have a very unpredict- predict- predictable, traffic pattern. it's like diurnal. It's Some days, like the company will do a launch and, they'll need like, way more. And it's not just one model that they deploy. They-- all these companies deploy, lots of different models in different regions, and so the autoscaling problem becomes even harder because then you have to scale within a certain region, and those cycles are offset. So different times you scale up in different regions.Akshat [00:13:20]: So that's like our sortVibhu [00:13:22]: And thatAkshat [00:13:22]: YeahVibhu [00:13:22]: That in and of itself is a huge category. There's a bunch of inference providers which, provide this fireworks, does this as a service together, whatnot, Base10. that's carved into its own niche for language models, at least right now.Akshat [00:13:36]: Yeah. the thing that we have specialized in is the autoscaling aspect.Vibhu [00:13:41]: Yeah.Akshat [00:13:41]: Because we found that it's not universally true that everyone else can autoscale, and we've gone deeper into it on the tech side by, we've incorporated GPU snapshotting into the product so we can take the GPU state, like your torch.compile model, snapshot it, and the next cold start is way faster. And so going back to your question, it's That's why you need a lot of burstiness for inference. But then people also do a lot of demand training, like for RL stuff, your rollouts are bursty, as you said. People also do a lot of batch jobs. So we'll see, a lot of companies, before they have a training run, they'll need thousands of GPUs to run encoding or something like that. And I think those things are much more bursty than. I agree that agents are not that bursty. sandboxes are, except when you're doing RL. RL is justRL, Batch Jobs, and 100,000 SandboxesVibhu [00:14:28]: Or commerceAkshat [00:14:28]: Insanely bursty.Vibhu [00:14:29]: Yeah.Akshat [00:14:30]: Yeah. Like when you're doing, rollouts, you sometimes need a hundred thousand sandboxes in your sandboxes.Vibhu [00:14:37]: Yeah. I'm curious if you've seen early sparks of continual learning. There are some people, like our friends, ngram, recently announced thisAkshat [00:14:45]: YeahVibhu [00:14:45]: They're, they're trying to do training. That also seems like a different workload, right? If you're doing training twenty-four/seven per se, there's a very weird dynamic of how you're using GPUs between people and whatnot, but seems like something you guys would work for.Akshat [00:15:00]: As you said, we're, we're fortunate to work with a number of, customers at the frontier and grab some of our customers. and they are taking the primitives we have, and trying to use them in very interesting ways, like continual learning. It's possible as the stuff gets better, some of that will be part of, our offering as well if, more people need it. but we're, we're just waiting to seeVibhu [00:15:23]: YeahAkshat [00:15:23]: How it shakes out.Vibhu [00:15:24]: Is there a primitive that you added after sandboxing that was the next step in the story?LLM Inference, DeFlash, and Speculative DecodingAkshat [00:15:32]: I guess we've been going much deeper into LLM inferenceVibhu [00:15:35]: YeahAkshat [00:15:35]: Because we realized that some of the advantages we have with like autoscaling, again, especially in different regions and whatnot, are, not present elsewhere. and the place where we had a gap was we weren't, working on the model layer itself. Like we were a black box. And, we realized that, we can get to frontier-level model performance, with, by having great people who work on this. And, we've been open sourcing a lot of our work, in terms of, Recently, we, shared our work on DeFlash, which is a block-based, speculator, and we've open sourced, all of it. So, you can - By using open source DeFlash, you can get the same performance as you would with one of the proprietary providers. And the next thing we're thinking about hereVibhu [00:16:23]: I thought this wasAkshat [00:16:24]: YeahVibhu [00:16:24]: An interesting blog post as well, right? Like, I think in here you make a claim that. Not a claim, just that how effective speculative deco-decoding really just get to.Akshat [00:16:33]: Yeah.Vibhu [00:16:33]: Anything you wanna point out from this around, what people should know?Akshat [00:16:39]: Yeah, absolutely. the high-level summary is, it would help to describe what speculative decoding is.Vibhu [00:16:44]: Yes.Akshat [00:16:44]: I will, yes.Vibhu [00:16:45]: I think, likeAkshat [00:16:46]: YeahVibhu [00:16:46]: So we've covered like Eagle and all thisAkshat [00:16:47]: YeahVibhu [00:16:47]: Like Hydra and all those things, but it was like two years ago.Akshat [00:16:51]: Yeah.Vibhu [00:16:51]: I think it doesn't hurt, right?Akshat [00:16:52]: Yeah. Speculative decoding is you have a smaller model, called a draft model, predict tokens ahead of the bigger model, and then you have the bigger model, verify all of this, all the tokens are predicted. And the reason it's faster is if you're predicting, one token at once, you're bound by memory bandwidth. But if you can batch the verification of, the draft model, then you're much more efficient using compute, and it's faster, and as long as your draft model is producing a lot of tokens that can get accepted, which is called the accept length, you can get a speed up that's, multiple times of, the original model speed. and well, that's what we highlight here. It's Like people talk a lot about we made these kernels faster and whatnot, but improving kernel will only give you like few percentage points of improvement, and, increasing accept length, literally is a multiplicative decreaseVibhu [00:17:47]: Like two to four X.Akshat [00:17:48]: Yeah, exactly.Vibhu [00:17:48]: Without much head-on performance.Akshat [00:17:50]: Yeah. I think it may - you are running a second model, right? So it may be something more expensive in the compute,Vibhu [00:17:57]: I meant quality performanceAkshat [00:17:58]: Probably not by muchVibhu [00:17:58]: But yeah. I thinkAkshat [00:17:59]: So there's no drop in quality performanceVibhu [00:18:01]: YeahAkshat [00:18:01]: Because you're always. You're never accepting a token that the big modelVibhu [00:18:04]: It's strictly betterAkshat [00:18:05]: YeahVibhu [00:18:05]: Or it's same.Akshat [00:18:06]: Exactly.Vibhu [00:18:07]: Right. Yeah.Akshat [00:18:08]: And so we've been working a bunch on DeFlash, which is a block-based speculator. so it's instead of predicting, one token at a time, it's predicting a block. And we've been open sourcing our work with it. The next thing for us here is for helping people train speculators and custom models. it's it's something that traditionally is very forward-deployed engineering driven, support deployed, engineer driven, like you work with customers and help them do that. And our vision for. This is why we launched Auto Endpoints, is we want to make frontier-level performance available to everyone. And so, we mentioned this in the announcement, we teased it. The next thing we're, we're launching is, as you run an auto endpoint, we shadow trafficAuto Endpoints and Frontier-Level PerformanceVibhu [00:18:54]: Do you want to explain what auto endpoints are?Akshat [00:18:57]: Yeah.Vibhu [00:18:57]: I lovely, yeah.Akshat [00:18:58]: Yeah. So, this is, I guess, going back to your Modal is you touch the code, but, sometimes people don't wanna touch the code, and they wanna get started with an endpoint that works and has all the great performance and, scalability that Modal has. So we've made that easier with, a way to create an endpoint from our UI, from the CLI, that has all of our optimizations that we talked about, like the DeFlash stuff already baked in, and there's full transparency. So we give you the code, you can go run it yourself, and if you want, you can eject out into the full Modal experience, which we see as people get sophisticated, they do wanna tweak the models, they wanna, fine-tune stuff. You can still do all of that. It's it's not a black box. And yeah, the next thing, as we teased later in the post, is how do we give you value even beyond this in terms of having your draft models evolve as your data distribution evolves, again, without having to talk to a person and, yeah.Vibhu [00:19:59]: I guess just to understand it directly, you have the GPUs, you have an endpoint that's compatible, you serve open model. If someone was to do this themselves, what's the delta that you guys provide? So you do a lot of open source great work on effective inference. how does it compare to, say, I take the same model, 5.2 FP8, take shelf inference engine, vLLM, SGLang, get compute of similar capacity, similar cost. What's the delta that plugging into something this, like this offers outside of the benefit of, scaling?Production Inference Beyond Raw GPUsAkshat [00:20:34]: It's interesting because we've taken the approach of open sourcing our contributions and upstreaming them. we work closely with the SGLang team. We want the improvements that our team, comes up with to be, there in open source for others to use, even outside of Modal. The benefit to us is we have a team that has significant expertise in terms of if you do have something that is not there, our team can help you get that performance, first. the other thing is with these endpoints, we are way more elastic, as you said, than, anyone else, and you have true scaling to zero. you have true, burstiness, and in practice, that matters a lot more to people than just finding, the GPU and, running Modal code on something.Vibhu [00:21:20]: Yeah. And I will say it's not that straightforward to just. like what I said is easier said than done, right?Akshat [00:21:26]: Yeah.Vibhu [00:21:27]: It's I think still for the average person, still hard to just gut check using different. There's, there's quite a bit of combinations you can make there. the trade-offs aren't really known at face value.Akshat [00:21:40]: Yeah. it's it's not just that. I think it's it's that running production-grade inference is a hard infer problem.Vibhu [00:21:49]: YeahAkshat [00:21:49]: Even if you subtract out the autoscalingVibhu [00:21:50]: YeahAkshat [00:21:51]: Is controlling things like tail latency and, making sure every, request is delivered at least once and whatnot.The Model and Agent LifecycleVibhu [00:22:00]: There's a lot of innovation that you can do here. I think, it's very interesting that you're starting to encroach on, like as you become a full cloud, you're starting to encroach on other people's turf.Vibhu [00:22:09]: What will you not do?Akshat [00:22:13]: Well, we wanna follow our users and, make sure they get like a platform that has everything that works well together. so right now we're focused on the model lifecycle and the agent, lifecycle. so both like going from data prep to training to inference, and then also if I want to deploy a background agent, let's say, sandbox, do persistent storage, a whole bunch of other stuff.Vibhu [00:22:38]: We talked to Cole, who did, OpenInspect. Yeah.Akshat [00:22:42]: Yeah.Vibhu [00:22:42]: And RealInspect also is on Modal.Akshat [00:22:44]: Yeah. So Ramp Inspect was a great example of a background agent that was really successful because they, were able to use some of the primitives like snapshotting and fast scaling to just have something that feels really reactive and works well.Ramp Inspect and Background AgentsVibhu [00:23:02]: Yeah. That's the new CTO of, Ramp right there.Akshat [00:23:05]: Yeah, Rahul.Vibhu [00:23:08]: It was really fun. yeah, okay, I think, all very bullish. Like, one of my reflections was also I did not originally. So when I met you guysThe Inference Inflection: CPU, GPU, and Co-LocationVibhu [00:23:19]: You weren't that much in the GPU game, and now you're all about, inference. And one of the points that I hinged on for Jensen's keynote at GTC this year was, what we're calling like the inference inflection, right? That let's say in AI workloads or machine learning workloads, it used to be like, let's call it eight to one GPU to CPU, and now it's more like one to one, which is like a interesting. Like, - because of how much agents are blocked or call out to this, to CPU heavy stuff the actual, like, limiting factor, like, swings back and forth from GPU to CPU a lot more than it used to be all GPU and then occasional CPU.Akshat [00:24:01]: Yeah.Vibhu [00:24:02]: GPU, CPU. And now it's like just constantly, and you just have to locate everything.Seventeen Clouds and the Supercloud StrategyAkshat [00:24:08]: Yeah. And that's one of the things that, again, we see as, something appealing about Modal, which is we've built this capacity pool that spans, 17 cloud providers, so we're, we're very good at Running on various kinds of cloud capacity across the worldSwyx [00:24:24]: You don't have your own data centers?Akshat [00:24:25]: We don't have our own data centers. We just run across a lot of neo cloudsSwyx [00:24:29]: Yeah. AreAkshat [00:24:30]: Metal providers.Swyx [00:24:30]: Yeah. Question mark.Swyx [00:24:31]: Yeah. You're, you're running the math, and you're like, “What's the cutover point where you're like.”Akshat [00:24:36]: Yeah, it's a good question. part of it is we see our differentiator in the software layer, and, being capital light and focusing on the software helps us move really fast. so far it's worked out well because there are so many other people building data centers that we're able to work effectively with them, and again, focus on what makes us, special.Swyx [00:24:55]: Yeah.Swyx [00:24:56]: 17 gets you into, like, the local providers sometimes. LikeAkshat [00:25:00]: The,Swyx [00:25:01]: Which was the most interesting one?Akshat [00:25:02]: There are a lot more neo clouds than you expect, and they all have various degrees of, various levels of reliability. And, that's why it's something we've invested a lot of time in, is building our own reliability layer on top. so if the GPU falls off the bus or something happens, we user workloads are not affected, and that lets us use a lot more capacity than,Swyx [00:25:30]: YeahAkshat [00:25:30]: You as a user would be able to.Swyx [00:25:32]: It's a useful thing to have because like now everyone knows, like, what layer you are and, like, you optimize for being the super cloud of all clouds.Akshat [00:25:41]: Yeah. That's, that's, that's the idea. and so I guess when you mentioned colocation, that's, that's another interesting thing where, one thing we've seen is people come to us when they want, very specifically located, CPUs or GPUs, like they wantSwyx [00:25:57]: Oh, they pin it in likeAkshat [00:25:58]: YeahSwyx [00:25:58]: EU?Akshat [00:25:59]: Exactly. Or EU, US.Swyx [00:26:01]: Right. Data resiliencyAkshat [00:26:02]: AustraliaSwyx [00:26:02]: Locality thing or performance or what?Akshat [00:26:04]: It's either data locality or latency, yeah.Swyx [00:26:07]: Yeah.Akshat [00:26:07]: Like, you want your. They're running sandboxes and model. They want them to be right next to aSwyx [00:26:10]: Yeah, it's easy thenAkshat [00:26:11]: YeahSwyx [00:26:12]: To. That is important in all those things. and so, like, you've accidentally, I don't know if it's accident, but, like, you've built the perfect primitive for agents to express themselves. And then, like, it's almost very funny how every extra development just involves more file system, just involves more CPU.Akshat [00:26:30]: Yeah.Swyx [00:26:31]: Just like the things that you already have. I don't know much about, if there's any, like, networking usages that are interesting, but you've also done some good work on networking.Networking, Sidecars, Private IPv6, and SandboxesAkshat [00:26:40]: Yeah, that's exactly right. Like, we're just taking compute storage and networking and building stuff on that layer, for, again, the stuff people need.Swyx [00:26:49]: YeahAkshat [00:26:50]: We see a few interesting networking things coming up. one is people want networked sandboxes. so we haveSwyx [00:26:57]: For like a Docker cluster type thing.Akshat [00:26:59]: Yeah.Swyx [00:26:59]: Sorry, Docker Swarm. Oh, f**k. What is it called?Akshat [00:27:02]: Compose.Swyx [00:27:03]: Compose type thing.Akshat [00:27:04]: Yeah. So if you want Docker Compose, our sandboxes now support, this thing called sidecars. So you can. A sandbox is a pod of containers, and you can run multiple containers in, a sandbox. also useful because, going back to networking, people want a lot of control over, outbound networking from a sandbox.Swyx [00:27:23]: Yeah.Akshat [00:27:23]: Like, they might wanna run a middle proxy for, like, maybe logging stuff for RL or, controlling how egress can happen to a domain, injecting credentials. and yeah. So we've, we've had to build a lot of that stuff ourselves.Swyx [00:27:38]: Yeah.Akshat [00:27:39]: But then also sometimes people want, sandboxes spanning multiple nodes to talk to each other, which is an emerging thing we're seeing. We have support for that for a different reason, and yeah, we'll see if that becomes stable.Swyx [00:27:52]: Like, just an open socket. It's a. This is directly like mTLS.Akshat [00:27:56]: We do support that, which is you can, expose a tunnel inside a sandbox.Swyx [00:28:01]: Yeah.Akshat [00:28:01]: And then you can either expose it to public internet or it can be, you can add like a HTTP, auth layer above it. But we have this thing called I6PN, which we haven't talked about, which is this, like, overlay network using IPv6 addresses. so if Modal containers, within the same workspace, when this is enabled, can address each other using this private IPv6 address, and no one else can.Akshat [00:28:28]: So it's like private networking, for containers. We built it because we needed it as a primitive for our distributed training product. so we have this other feature, which is you can add a decorator to a function, and you get a cluster of GPUs. and they have RDMA networking. so you can run a distributed training job, that's truly serverless. and we did the overlay network for that. But then we've seen that people are using it for other reasons, and, I'm intrigued to yeah, what would people do with it.Swyx [00:28:59]: Build primitives and let people figure it out, right?Akshat [00:29:01]: Yeah, exactly.Swyx [00:29:02]: You put out a pretty interestingAkshat [00:29:03]: They're like, they read the docs webpage. Let me use thatSwyx [00:29:06]: YeahAkshat [00:29:06]: Something they never intended to work. This is literally not even in our docs page. People somehow found it, and they're using it.RDMA, Memory Movement, and Distributed TrainingSwyx [00:29:12]: Huh.Swyx [00:29:14]: The way you portrayed it with, like, RDMA versus TCP, like, very well laid out, but just the transfer speed change at scale for RL, like yeah, you have it, you have it built in. I'm sure someone found it. It's found it to be a lot more efficient before you made a thing out of it, right?Akshat [00:29:32]: Yeah. And not to split hairs, I guess the overlay network is the TCP overlay network.Akshat [00:29:39]: The reason we have that is you need that to do the key exchange for RDMA before you set up the RDMA network on top of that. but then people found the TCP part.Swyx [00:29:48]: Can I tell you, this is like a big aha moment for me becauseAkshat [00:29:51]: YeahSwyx [00:29:51]: So I review 2,200 submissions for the World's Fair.Akshat [00:29:56]: Yeah.Swyx [00:29:57]: And then I got this from John OsterhoutAkshat [00:29:58]: HuhSwyx [00:29:59]: Who I don't know if. Do John Osterhout by name?Akshat [00:30:01]: The name sounds familiar.Swyx [00:30:02]: He published a. He's a well-known professor, published a lot of interesting software design books, and this is the talk he chose to submit, is on RDMA at Inference. And I'm like, you wouldn't think that this guy, who is like operating systems guy, would care about RDMA.Akshat [00:30:20]: I, it makes sense to me because I,Swyx [00:30:24]: This is the cloud, right? YeahAkshat [00:30:25]: Like, the way you move around your KV cache and how efficiently you can do it, how efficiently you move, your weights from your training GPUs to your inference GPUs in RL is there's a lot of degrees of freedom, and it is a systems problemSwyx [00:30:41]: YeahAkshat [00:30:41]: Moving memory aroundSwyx [00:30:42]: YeahAkshat [00:30:43]: Scheduling.Swyx [00:30:44]: This shows you how primitive my understanding of networking stuff is.Swyx [00:30:46]: Is this like the domain of WireGuard as well?Akshat [00:30:50]: Not quite.Swyx [00:30:51]: It's adjacent?Swyx [00:30:53]: Explain everything.Akshat [00:30:54]: Sure.Swyx [00:30:56]: How do we move memory around GPUs?Akshat [00:30:58]: Well, so sorry. Yeah, that is memory. Sorry, I was talking more, and maybe I was talking like five minutes back, about the private IPv6, addressing that you've set up.Swyx [00:31:09]: Yeah.Akshat [00:31:09]: Is it like it's a VPN?Swyx [00:31:10]: Yeah, it is like a VPN, and yeah, WireGuard is, yeah, you're right. It is,Akshat [00:31:16]: Right. Yeah, you already moved on to new topicsSwyx [00:31:17]: A similarAkshat [00:31:18]: OkaySwyx [00:31:19]: In the same space, WireGuard is, encrypted and this is,Akshat [00:31:23]: And you don't need encryption.Swyx [00:31:23]: Yeah.Akshat [00:31:24]: Yeah.Swyx [00:31:24]: This is not encrypted. that's the main difference. This is TCP and we have eBPF programs that will reject or allow the TCP connection based on whether you're allowed to do it.Akshat [00:31:35]: Used to involve a full sidecar, but now you have eBPF in the Linux kernel.Swyx [00:31:39]: Yeah.Akshat [00:31:40]: Yeah. I don't know if this is a natural follow-on to the topic of like my skepticism on distributed training is that while, like, people spend a lot of money on, like, cables to hook up GPUs, and even that is not, like, fast enough, and that's the bottleneck, is your networking fast enough?Swyx [00:31:59]: Yeah. So I guess you're talking about fully distributed training like, Dialog or something which is like cross data centerAkshat [00:32:06]: That would be, yes.Swyx [00:32:07]: That's the extreme.Akshat [00:32:08]: Yeah.Swyx [00:32:08]: You're in the middle, and then other people would have like the Mellanox cables up in, like, their actual data center.Akshat [00:32:14]: When you run multi-node training on Modal, RDMA, I think Mellanox, is, or InfiniBand is like a, is all seen as RDMA. but it's a way to bypass the TCP networking stack and, transfer, stuff much faster, between one node, to the other. And we have I think like 3 terabit per second, internal networkingSwyx [00:32:40]: OkayAkshat [00:32:40]: Which is the standard that's needed.Swyx [00:32:42]: Okay. So I misunderstood whatAkshat [00:32:43]: 50Swyx [00:32:43]: What part of the stack you wereAkshat [00:32:44]: 50 gigs overSwyx [00:32:45]: YeahAkshat [00:32:45]: If you wentSwyx [00:32:45]: YeahAkshat [00:32:46]: RDMA.Swyx [00:32:46]: Okay.Swyx [00:32:48]: Yeah. I, very impressive work.Multi-Node Training, Post-Training, and Auto ResearchSwyx [00:32:52]: So effectively you're extending like the model philosophy to the training cluster, like, yeah.Akshat [00:32:59]: Yeah. And we're, we're not going for like large scale training runs. the thing that we've built multi-node training for is, we see a lot of, smaller scale post-training. like, people are post-training like medium sized fund models, so they can, get higher quality on inference. this is a perfect fit, for something like that.Swyx [00:33:21]: Yeah. That is my impression of how a lot of these labs explore branches in post-training and then eventually merge whatever they find in.Akshat [00:33:31]: Yeah. The other use case we've seen for multi-node training is even if you have a big cluster, your researchers are still doing small runsSwyx [00:33:38]: YesAkshat [00:33:39]: Having elasticity thereSwyx [00:33:40]: Right, sureAkshat [00:33:40]: Matters a lot more.Swyx [00:33:41]: Yeah. the, like, this is like the current limiting factor for auto research, which is like you need to give your model some GPUs in order for it to completely run.Akshat [00:33:51]: We have a blog post on auto resource and model is,Swyx [00:33:55]: YeahAkshat [00:33:56]: Yeah, like, turns out to be pretty good substrate for that.Swyx [00:33:59]: So my impression is auto research means many things, likeAkshat [00:34:01]: YeahSwyx [00:34:01]: Anything that Andrej coins. Right now it's still science fair, right? Like not like, I don't know how many people are doing this.Akshat [00:34:08]: We're having a golf.Swyx [00:34:08]: Yeah.Akshat [00:34:09]: I thought the same thing.Swyx [00:34:11]: Yeah, you would know.Akshat [00:34:12]: We, like, our internal both training and inference teams use this the general shape of this quite a bit. like we have this one internal repo called auto inference, which essentially we've automated our own forward-deployed engineering efforts using, this harness, which is, the agent will just spin up a sweep of different things. It'll even run like, NVIDIA inside profiler and it'll like tweak configs and it'll arrive the right thing. it'll change your GPUs both from H200 to B200, and works really well.Swyx [00:34:47]: Nice.Akshat [00:34:47]: So yeah.Swyx [00:34:48]: By the way, I enjoy that your forward-deployed engineering is so technical that you have to do these things.Swyx [00:34:52]: It's very different from forward-deployed engineering from other people.Akshat [00:34:54]: Yeah. For our forward-deployed engineering team is, essentially they're like applied inference researchers or applied training researchers.Swyx [00:35:02]: Someone told me like they have to be able to build, but they also have to be able to sell. do they have to sell or are they like they're good, they're just like post-sale type of thing?Akshat [00:35:09]: It does, being able to talk to a customer and engage effectively with themSwyx [00:35:13]: YeahAkshat [00:35:13]: Matters a lot.Swyx [00:35:14]: They want the same thing.Akshat [00:35:15]: Yeah.Swyx [00:35:15]: ?Akshat [00:35:15]: But it's it's not really a sales, thing. We pair them with-- We have solution architects as well that are more on the sales side.Swyx [00:35:23]: Okay. Let's spend a bit more time on auto research. This is a big focus for for this year. Where does this go? like, have people explored enough? Like, there's all these beautiful charts of like improve and then level off a bit and then you find the next thing. Is this one abstraction up from normal training? Is that how we think about it, or do you think about it differently? Like model level training versus high, like driven hyperparameter search.Auto Inference and Modal BenchAkshat [00:35:51]: Yeah, like,Swyx [00:35:51]: Someone, some people call it like neural architecture search or whatever, right? Like.Akshat [00:35:54]: Yeah, - So the stuff I've seen people do with it is nowhere on the architecture level. It's pretty much tweaking parameters, but it's it's a hyperparameter sweep that's guided by some model intuition, so it's like much more efficient than, whatever other, sweep you would have.Swyx [00:36:12]: Yeah, it's just, it's just a question of where you want to spend your compute?Akshat [00:36:16]: Right.Swyx [00:36:16]: ‘Cause yeah, you can just throw infinite amounts of money on this and somehow you'll bang out Shakespeare?Akshat [00:36:22]: Yeah, infinite monkey.Swyx [00:36:24]: Yeah, so like the very good for model. and I think it's also very important that agents can spin up other agents, can spin up their infrastructure. Like very good for you. how good is our LLMs at generating model code? Like the benefit of existing LLMs is that you are in the data.Akshat [00:36:42]: Yeah. They're, they're surprisingly good. I think like pre Cloud 4 they were not, and then now they're able to shot, stuff out of the box. But we're playing around with releasing like a Modal Bench for like the harderSwyx [00:36:55]: YeahAkshat [00:36:55]: Things, that the LLMs cannot do yet and maybeSwyx [00:36:59]: What's an example of that?Akshat [00:37:01]: I think the things that- Sometimes agents struggle with, without right guidance and a skill is, how to, use the rest of our observability. Like how to. Something is failing, like how do you look at the logs and then update the right thing? It's reasoning about that. But they're able to shot, likeSwyx [00:37:23]: Yeah. You can just add a skill to it?Compute Strategy and Capacity PlanningAkshat [00:37:26]: Yeah. So we have a Modal skill now that. Which is why we built this Modal Bench. It's to find things like that, so we can address them in our tool.Swyx [00:37:35]: Tune a skill. Yeah.Akshat [00:37:36]: Yeah.Swyx [00:37:36]: No. it's it's good. are you facing any shortages? like we talk a lot about GPU shortages, but also CPU, also memory.Swyx [00:37:44]: Yeah.Akshat [00:37:45]: We have had a lot of growth, which means that, there's - we've had to be much better aboutSwyx [00:37:53]: PlanningAkshat [00:37:54]: Proactive capacity planning.Swyx [00:37:55]: Yeah.Akshat [00:37:55]: So we have,Swyx [00:37:57]: Which by the way, like it's like a MBA's like dreamAkshat [00:38:00]: YesSwyx [00:38:00]: Is like just planning this stuff. I think last time you and I talked about something maybe about this.Akshat [00:38:03]: Yeah. we have a really competent team of people that we call, The role is called compute strategy. so yeah, if anyone listening here or wants to work on thatSwyx [00:38:13]: Compute strategy?Akshat [00:38:13]: Yeah.Swyx [00:38:14]: I think,Akshat [00:38:14]: I feel like,Swyx [00:38:15]: I think the normies call it FP&A or something.Akshat [00:38:18]: Well, it's more It's it's not FP&A. It's it's There's a lot of interesting financial questions of like what is the blend between one year and three-year reservations? how do we forecast our own capacity? how do we. especially since our capacity is very fungible across different GPU types and different regions, like you have to model a lot of it. and you also have to have an opinion on how the supply chain is gonna evolve, and then you have to like, take bets,Swyx [00:38:49]: YeahAkshat [00:38:49]: Based on that.Swyx [00:38:50]: Tokenomics.Akshat [00:38:50]: Yeah.Swyx [00:38:51]: This is like probably a not a real point, but, I was trying to think about like what other industries. I was trying to think about like, we cannot be first to like these kinds of problems.Akshat [00:38:59]: Yeah.Swyx [00:39:00]: And what other industries have had this? And I was like, airlines with fuel and like they have to hedge their fuel and like, I think for a long time Southwest because they made like a hero fuel bet, they like were like super low cost becauseAkshat [00:39:12]: OhSwyx [00:39:12]: Compared to everyone else.Akshat [00:39:14]: Yeah. I hadn't thought about that.Vibhu [00:39:16]: We're at a fun time too?Akshat [00:39:18]: Yeah. It's. A lot of the compute business in general, for us is also about being very good about capacity management. That is how you have great unit, economics. but also over time it's how you can unlock more value for customers. Like, one of the things we're building now is like a way for customers to get, If they don't care about latency, like get much cheaper pricing and they'll get results back in like next 24 hours or something, like a batch tier essentially.Batch Tiers and Latency-Insensitive WorkloadsSwyx [00:39:47]: Yeah.Akshat [00:39:47]: And those are levers we have because we control the whole stack and scheduling and whatnot to give people a sufficientSwyx [00:39:53]: Yeah. I feel like they're not as popular. Like those, like the Frontier Labs have all those APIs. They're not as popular as they should be.Akshat [00:40:00]: The demand that we see for something like that is not for LLMs. although sometimes people wanna run evals andSwyx [00:40:08]: OkayAkshat [00:40:08]: Synthetic data prep and there it makes sense.Swyx [00:40:10]: Okay.Akshat [00:40:11]: But it's from a lot of LLM companies, like people who are doing computational bio, like they have to run really big batch jobs and they don't care about when they get it back.Swyx [00:40:22]: Yeah. And like they have a reasonable. It's it's also like a cousin to the stopping problem of like, will this finish in time?Akshat [00:40:30]: Yeah. You can bound it.Swyx [00:40:33]: Yeah.Akshat [00:40:33]: Like you can give peopleSwyx [00:40:34]: YeahAkshat [00:40:34]: SLAs on it.Swyx [00:40:35]: Yeah. I think what's, what's interesting is like the next phase of model.Swyx [00:40:38]: Like what, do people expect from you, now that you're established and you're like well-known compute player among all these leading companies. You had an inference launch week, and we talked a little bit about the launches. like what else? Like what else should people know?What Modal Builds NextAkshat [00:40:55]: We are building primitives that make our users' lives much easier. So, I think for example, with LLM inference, thousands more companies are gonna post-train their own models and, deploy open source models for inference. so we're thinking a lot about what is the best product shape for that. And, that involves everything from our training gym to, then, endpoints that get frontier-level performance. again, but I haven't talked to anyone. It looks somewhat different on other verticals. Like, we're also seeing a lot of real-time, audio-video stuff in there, which is why like, we're working on things like regional routing, with fallbacks. So you can get GPUs that are as close to users as possible. so you get like low latency for video streaming and whatnot. And then on the agent side, it's,Akshat [00:41:52]: We're still working very closely with our customers because stuff is changing so fast in terms of what they need. And, I think beyond sandboxes and persistent file systems, there's a lot of other things people will need from this agent stack as they build production agents. So yeah, we're thinking about those other things that fit in there.Swyx [00:42:13]: I want to ask what the other things are.Akshat [00:42:15]: Yeah. I probably should share right now.Swyx [00:42:17]: I think-- I think, okay, so, I do think a lot about the principal components of cloud, and you do talk about compute storage networking.Akshat [00:42:25]: Yeah.Swyx [00:42:25]: Because so far for me, it's fine. so far for the. the first couple generations of cloud, it's fine. What's different, qualitatively different about agents that you need some new permission level? Like a lot of people, okay, and I'll just kinda spew tokens at you until it like hopefully sparks something.Akshat [00:42:43]: Yeah.Swyx [00:42:44]: Like the new level now is whatever Claude Code does, which is dangerously scope permissions or like allow list by command or like whatever, right? And sometimes they're like, “Well, okay, we have like this adaptive thinking mode where like, just trust me, bro. I will make the calls for you.” Is that it? like mediated permissions.Hard Guardrails vs. LLM-Mediated PermissionsVibhu [00:43:03]: Now you're looping it with a goal and letting it roll.Akshat [00:43:06]: Yeah, I'm, I'm skeptical of LLM media permission for stuff that is at the sandbox level because you do want hard boundaries.Swyx [00:43:16]: Yeah.Akshat [00:43:16]: Otherwise, someone can exfiltrate stuff.Swyx [00:43:20]: But likeAkshat [00:43:20]: YeahSwyx [00:43:20]: Maybe that's old school thinking. Maybe we're the dinosaurs.Swyx [00:43:23]: Maybe the AI OS or the LLM OS is really the kernel is a goddamn LLM.Swyx [00:43:30]: Like it makes you feel uncomfortable.Akshat [00:43:31]: Yeah, I'm, I'm toldSwyx [00:43:32]: But that's what trusting the LLM is. Like imagine a spherical cow perfect LLM.Akshat [00:43:36]: Right.Swyx [00:43:37]: That it.Akshat [00:43:39]: Maybe.Swyx [00:43:41]: I wanna test the boundaries, right?Akshat [00:43:42]: Yeah.Swyx [00:43:42]: Like, and I don't believe that, but I wanna see where I'm wrong ‘cause that's, that's the consensus.Akshat [00:43:49]: Yeah. I think you always need hard guardrails when you want, And you can pair those with softer guardrails, right? And that's gonna be a lot of mediated.Managed Agents and Specialized SandboxesSwyx [00:44:00]: There. I'll also get you a end with a couple of your commentary on like the ecosystem outside of Modal. Manage agents. Everyone has one. Gemini, OpenAI, Claude, very useful for you, but also like it is their way of starting to edge into your space.Akshat [00:44:17]: Yeah.Swyx [00:44:17]: What's going on?Akshat [00:44:19]: Yeah, we're, very excited to partner with Anthropic and some of the other foundation labs, will not name who we're also working with. the way we see it is the manage agent thing is a great place to start if you're starting out building an agent and, But then when you get to, building something more production grade, like you're a company that's like Ramp that's building their own, Ramp also runs their accounting agent on us, so their external-facing agent. You need a lot more control over, your compute primitive on things like, what sort - how do you persist different files that the agent has access to, and how do you snapshot and restore? How do you control the networking? maybe you want GPUs. When you get to that point, you kinda want, a specialized sandbox provider, that gives you those things, and that's the role that we are trying to play.Swyx [00:45:15]: YeahAkshat [00:45:16]: We don't really have an opinion on the harness, whether it runs - it's a cloud-managed agent, and you hook it up to Model Sandbox, or you run the harness in Model Sandbox. We'll see where people converge with that.Swyx [00:45:26]: Yeah. Do you any opinions on like the meta harnesses, or just another layer on top of these things?Akshat [00:45:31]: You mean like the OpenPipeSwyx [00:45:33]: OpenPipe is one. I think Vercel had one, which I can't remember the name of right now. Fredshot had one. and then, to me, most recently was Data Databricks that had Omnigen. All these are meta harness. Like it's kinda pseudo agent cloud type things.Akshat [00:45:50]: I personally have not played around with them.Swyx [00:45:53]: Yeah.Akshat [00:45:53]: Build agents with them.Swyx [00:45:54]: Everything's bullish Modal, as long as it consumes more infra.Akshat [00:45:57]: That's why we're focusing on the infra layer. It's somewhere where our, relative competence is and, also it's a hard problem to solve.Swyx [00:46:06]: Yeah. I will say like just generally reflecting on that, I don't know if - if there's other topics on Modal, but like just generally reflecting as an infra person, not as intense as you, but in that field, this has like been the most exciting time in infra. Like it was boring for a while, and you couldn't really get people excited about data infrastructure. Like Eric would get on Data Console, everyone just watched the video and like say, “Look at how many sandboxes I can spin up,” and no one gave a crap.Why Infrastructure Became Exciting AgainAkshat [00:46:39]: Yeah.Swyx [00:46:40]: And like now everyone gives a crap.Akshat [00:46:42]: That's true. It is a very exciting time, and I think a lot of that's driven by just the amount of scale all of this stuff needs.Swyx [00:46:50]: I think the, like a lot of your initiatives or a lot of your like product directions make sense in retrospect, which is like the best kind, but I wouldn't necessarily have thought about it myself, which.Akshat [00:47:00]: We need the predictions.Swyx [00:47:02]: I think there's a lot that you just don't even see, right? Like you have the batch, you have the voice, you have the multimodal, but what else?Akshat [00:47:10]: What else is coming up for usSwyx [00:47:11]: Yeah. Where do you see things going?Akshat [00:47:13]: Yeah. I, in generalBiotech, Robotics, and Non-LLM AI WorkloadsAkshat [00:47:15]: It's it's clear that there's there's a huge shift happening. I think one thing that's not as obvious to people because LLM inference gets talked about so much and is also we work a lot of companies that are, doing things like drug discovery and computational bio, like the Chai Discoveries of the world. Big things are probably gonna happen there. we work a lot of robotics companies that are putting robots in like active deployments and getting good results out of them.Swyx [00:47:45]: Is there Air Gap Modal? Is there a version that is like prem air gapped whatever?Akshat [00:47:50]: No. We,Swyx [00:47:51]: You should cloud only.Akshat [00:47:51]: Yeah.Swyx [00:47:52]: Yeah. Okay. But yeah, so what you're saying is like because you're focused on primitives and they're good primitives, you find use cases in all these kinds of things.Akshat [00:48:01]: Yeah.Swyx [00:48:01]: Probably diversifies you a little bit away from LMS all the time.Akshat [00:48:05]: Yeah, absolutely. We're, we'- our goal isn't to only serve the LLM inference market.Swyx [00:48:10]: There are a lot just on the website, the audio,Akshat [00:48:12]: Yeah. We said both onSwyx [00:48:14]: Computational bio images. Yeah, there's a lot here. There's QTA TTS, customizing. Oh, Chatterbox. there was customizing Whisper.Akshat [00:48:24]: Okay. Yeah.Swyx [00:48:25]: This screen reminds me of a fallen competitor, which Replicate.Model APIs vs. Differentiated AI ProductsSwyx [00:48:31]: What's your postmortem on what happened?Akshat [00:48:34]: This is one thing we've stayed away from is providing an API for models because I think providing model APIs is some of it ends up serving like a really hobbyist market, which is much less sticky.Swyx [00:48:50]: Yeah.Akshat [00:48:50]: And we've always wanted to build for companies that are building products and need more flexibility that's not just an API.Swyx [00:48:57]: Which you can build an API for a model and this is clearly what it is. But you - but what you're saying, you can wrap it into a more fully functioning back end that you run.Akshat [00:49:06]: Yeah. So all of our examples, it's not that spin up this model, here's an API token, use it. They're all code.Swyx [00:49:13]: Okay.Akshat [00:49:13]: And so the point is that this is just an example.Swyx [00:49:16]: Starter code.Akshat [00:49:17]: Yeah. But you can tweak it however you want.Swyx [00:49:20]: Yeah.Akshat [00:49:21]: And if you're like a company building a product, like, computational bio whatnot, yeah.Swyx [00:49:26]: I guess I'm trying to tease out for listenersAkshat [00:49:28]: YeahSwyx [00:49:28]: When does it stop becoming, oh, you're just an API call and you're just a wrapper on API to becoming what you call a product, right?Swyx [00:49:36]: Like, what is that layer? Like what-- Like, more lines of code, but like beyond that, what is the substance that people add that qualifies it to be something more?Akshat [00:49:46]: I think there's a little bit of like a selection effect of like a lot of the companies who do wanna get deeper into that level are probably building something that's more differentiated. And, I think, an example is like - with LLM inference, originally we, worked with companies that were building their own post-training frameworks or they were, - Ramp early in the day was training their own tokenizer and like swapping out the tokenizer in Llama and whatnot. I'm not saying that's, that successful, in that case. But a better example is like, let's say Suno. because Suno, does not use Modal for training.Swyx [00:50:26]: Mikey on the pod. Yeah.Akshat [00:50:27]: But they use Modal for all their inference and that's because they have like a custom-- They have completely custom model architecture and that means that they have to be at the code level and tweak things that are not, just an API.Swyx [00:50:41]: It's interesting as well, like we had, Ethan, most recently on the xAI Groq team make a prediction that like the next tier in video gen is not a better video model, it's a better model or agent that orchestrates video models.Video Agents and Production WorkflowsAkshat [00:50:56]: Oh, interesting.Vibhu [00:50:56]: Language model backbone that can use toolsAkshat [00:50:58]: RightVibhu [00:50:59]: And write code.Akshat [00:51:00]: Like, yes, I can make my second video or my second video from Groq, but I want my minute video.Akshat [00:51:06]: And I'm not going there through normal video gen.Swyx [00:51:10]: Yeah, that's interesting. I - So we have GPU sandboxes and recently have seen a few companies doing agents that do video manipulation or,Akshat [00:51:22]: Yeah. Give it FFmpeg and just do it.Swyx [00:51:23]: Run FFmpeg. But likeAkshat [00:51:25]: That's not enough.Swyx [00:51:25]: Yeah.Akshat [00:51:26]: You need to give it Adobe.Swyx [00:51:27]: Yeah, I hadn't put it together with like it would be a video production thing. in my mind these things were going more towards editingAkshat [00:51:36]: Yeah.Vibhu [00:51:36]: Well, shout out Mantis.Akshat [00:51:37]: I think about this a lot.Swyx [00:51:38]: .Akshat [00:51:41]: Yeah. Sorry.Vibhu [00:51:41]: Luma. Luma Agent is a version of this for video production, but it's a off.Swyx [00:51:46]: I was gonna get your quick takes, on some other stuff that happensGitpod/Ona, CI, and Runtime SandboxesSwyx [00:51:50]: In recent news and just-just see if you have anything interesting. Gitpod, very li

I dagens avsnitt gästas vi av minnesmästaren Jonas von Essen, för ett ögonöppnande samtal om vad som kan bli mänsklighetens undergång - Artificiell Intelligens.Jonas beskriver hur AI på bara några år har gått från att vara en opålitlig ”statistisk papegoja” till att kunna resonera, programmera, lösa komplexa problem och prestera på nivåer som nyligen kändes science fiction. Nästa steg, menar han, är när AI börjar förbättra sig själv – och utvecklingen accelererar snabbare än människor kan förstå eller kontrollera. Vi pratar om riskerna med cyberkrig, autonoma vapen, biologiska hot och manipulation i en omfattning världen aldrig tidigare skådat. Jonas öppnar upp om den mörka perioden då han drabbades av depression efter att ha fördjupat sig i AI-forskningen – och hur insikterna om framtiden förändrade hans syn på livet. Han menar att risken för mänsklighetens undergång är betydligt närmre än vi tror - vi pratar inom 10 år! Men samtalet handlar också om hopp. För samma teknik som skulle kunna bli vårt största hot kan också skapa en framtid med botade sjukdomar, kraftigt förlängda liv, överflöd av resurser och ett samhälle där arbete inte längre är nödvändigt. Frågan är bara om vi hinner trycka på stoppknappen innan utvecklingen springer ifrån oss.Ett av de mest fascinerande, skrämmande och tankeväckande samtalen vi haft hittills – om en framtid som kan vara mycket närmare än vi tror.Följ Jonas härLäs mer om Jonas härLäs mer om Framgångsakademin här.Ta del av Framgångsakademins kurser.Beställ "Mitt Framgångsår".Följ Alexander Pärleros på Instagram.Följ Alexander Pärleros på Tiktok.Bästa tipsen från avsnittet i Nyhetsbrevet. Hosted on Acast. See acast.com/privacy for more information.

I dagens avsnitt gästas vi av minnesmästaren Jonas von Essen, för ett ögonöppnande samtal om vad som kan bli mänsklighetens undergång - Artificiell Intelligens.Jonas beskriver hur AI på bara några år har gått från att vara en opålitlig ”statistisk papegoja” till att kunna resonera, programmera, lösa komplexa problem och prestera på nivåer som nyligen kändes science fiction. Nästa steg, menar han, är när AI börjar förbättra sig själv – och utvecklingen accelererar snabbare än människor kan förstå eller kontrollera. Vi pratar om riskerna med cyberkrig, autonoma vapen, biologiska hot och manipulation i en omfattning världen aldrig tidigare skådat. Jonas öppnar upp om den mörka perioden då han drabbades av depression efter att ha fördjupat sig i AI-forskningen – och hur insikterna om framtiden förändrade hans syn på livet. Han menar att risken för mänsklighetens undergång är betydligt närmre än vi tror - vi pratar inom 10 år! Men samtalet handlar också om hopp. För samma teknik som skulle kunna bli vårt största hot kan också skapa en framtid med botade sjukdomar, kraftigt förlängda liv, överflöd av resurser och ett samhälle där arbete inte längre är nödvändigt. Frågan är bara om vi hinner trycka på stoppknappen innan utvecklingen springer ifrån oss.Ett av de mest fascinerande, skrämmande och tankeväckande samtalen vi haft hittills – om en framtid som kan vara mycket närmare än vi tror.Följ Jonas härLäs mer om Jonas härLäs mer om Framgångsakademin här.Ta del av Framgångsakademins kurser.Beställ "Mitt Framgångsår".Följ Alexander Pärleros på Instagram.Följ Alexander Pärleros på Tiktok.Bästa tipsen från avsnittet i Nyhetsbrevet. Hosted on Acast. See acast.com/privacy for more information.

Training-run assessments conducted by a 3rd party should become a standard part of frontier AI safety. By a Training-Run Assessment, or TRA, I mean an in-depth analysis of the post-training pipeline and dynamics leading up to a frontier model release. A TRA can look at intermediate checkpoints, training rollouts, RL environments, reward signals, SFT datasets, and the process by which the developer responded to warning signs.[1] In this post I will argue that: Final-checkpoint evaluations will be insufficient to assess scheming risks.TRAs can be more effective at detecting scheming.Frontier developers should involve third parties to do TRAs or verify safety claims by the developers. The rest of the post lays out a taxonomy of TRAs and sketches a path toward a 3rd party ecosystem for them. We, at Apollo Research, are intending to conduct 3rd party Training-Run Assessments in the future. Detecting Scheming may require Training-Run Assessments By scheming I mean an AI covertly pursuing misaligned goals while deliberately concealing its intentions or capabilities from its developers. I restrict attention to “coherent” forms of scheming where the model pursues somewhat stable misaligned goals across context windows, rather than misalignment that surfaces only as isolated, context-dependent defections. [...] ---Outline:(01:23) Detecting Scheming may require Training-Run Assessments(03:55) Why 3rd parties should perform Training-Run Assessments(04:12) Developers may lack incentives to adequately assess scheming(04:49) Developers' safety assessments lack credibility(05:31) External evaluators can bundle expertise for assessing scheming(06:17) 3rd party TRAs can be developed gradually(08:50) Checkpoint evals(08:54) What?(10:30) How?(11:15) Data inspections(11:19) What?(12:03) Why?[... 16 more sections]--- First published: July 5th, 2026 Source: https://www.lesswrong.com/posts/3HvvjffA65mHLwaWm/we-need-3rd-party-training-run-assessments --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Send us Fan MailCritics often claim that Scripture is full of errors, conflicting accounts, and impossible statements. In this video, Dr. R. L. Solberg examines some of the most common alleged contradictions and explains why many of them turn out to be questions of context, language, perspective, chronology, or textual transmission rather than genuine logical contradictions.We explore the Resurrection accounts, Paul and James on faith and works, Old Testament passages that speak of things lasting "forever," apparent contradictions about seeing God, God's unchanging nature, numerical discrepancies, and more. Along the way, we learn an important principle: a true contradiction exists only when something is affirmed and denied in the same sense, at the same time, and in the same respect.Whether you're wrestling with difficult passages yourself or responding to challenges from skeptics, this video will help you think more carefully about what the Bible actually says and how Christians have approached these questions for centuries.The Biblical Roots MinistriesOur websiteOur YouTube ChannelProf. Solberg's BlogSupport our Ministry (Thank you!)Chapters00:00 Introduction04:06 The Bible as Historical Literature06:39 Defining a True Contradiction08:46 Eyewitness Contradictions16:00 Semantic Contradictions28:12 Examining Motivations30:00 Descriptions of God34:28 Contradictions of Perspective37:06 Textual Transmission40:49 Numerical Contradictions43:48 Wrap it up, Solberg.What if Christmas felt sacred again? Full of Grace and Truth, the new book from award-winning author R. L. Solberg, invites you to rediscover the biblical story at the heart of the season. Available now in paperback and Kindle, with all proceeds supporting The Biblical Roots Ministries. Get your copy today on Amazon.com.

Always so much fun to chat with Grant.AI has been making much faster progress in math than in other fields. As a result, mathematics is showing us, very concretely, what AI progress in other fields will look like. Even within mathematics, there's a jagged landscape. What does it look like?What is the nature of the most important conceptual breakthroughs in the history of mathematics, and how different are they from what AIs are currently able to do?Does AI (on net) increase or decrease human understanding of the field?How big is the overhang from having AIs systematically try to connect ideas already in the literature?And what advice does Grant have for aspiring mathematicians, coders, and other students who are passionate about fields that are being most transformed upon by AI?Watch on YouTube; read the transcript.Sponsors* Gemini 3.5 Live Translate is what I wished I'd had on my last trip to China. It detects more than 70 languages and translates them in near real-time… and it preserves your original pacing and intonation. If you're building an app that needs live translation, you should check out Gemini 3.5 Live Translate. Get started at ai.studio/live* Cursor's harness lets me use models for a huge range of tasks at the podcast. For example, Cursor cuts out the ads from each episode I produce so I can post them on Bilibili. It also helps me prep for interviews — I have a repo full of books and papers that Cursor sorts through to find the exact right file for any given question. Try Cursor yourself at cursor.com/dwarkesh* Jane Street sponsors 3Blue1Brown, so Grant has gotten to spend a lot of time with various Jane Streeters. He actually just recorded an interview with a few of them, so when we sat down for this episode, he told me about some of the things he learned, like how Jane Street keeps their role definitions fuzzy to make sure their people keep learning and growing. Go check out Grant's full interview at 3b1b.co/janestreetTimestamps(00:00:00) – AI is discovering new proofs. Is that AGI?(00:11:32) – The verification loop on conceptual breakthroughs can be a century long(00:26:12) – Will we understand an AI proof of the Riemann hypothesis?(00:38:08) – Can AI find the hidden bridges between fields?(00:53:48) – Why real-world tasks don't fit into RL environments(01:07:07) – Good writing requires theory of mind that AI still lacks(01:16:02) – Why learning will still depend on human curation Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe

Thomas Erikson är tillbaka i Framgångspodden. Författaren bakom den globala succén Omgiven av idioter har sålt över 12 miljoner böcker världen över – och i det här avsnittet delar han sina mest värdefulla insikter om relationer, kommunikation och varför mänskliga egenskaper blir allt viktigare i AI-eran.Vi pratar om varför så många unga idag känner sig vilsna, hur sociala medier påverkar vår koncentrationsförmåga och varför allt fler väljer att lägga undan sina smartphones för att återupptäcka verkliga möten och djupare närvaro. Thomas berättar också om sina egna strategier för att skydda sitt fokus i en värld där vår uppmärksamhet ständigt konkurrerar med digitala distraktioner.Vi går även på djupet kring en av livets viktigaste frågor: Hur hittar man sitt varför? Thomas delar den insikt som förändrade hans egen livsriktning och berättar öppet om hur han tog sig ur en destruktiv relation genom att konfrontera sina rädslor och börja leva mer i linje med sina värderingar.Vi får även svar på frågor som varför starka relationer ofta är den verkliga nyckeln till framgång, varför kritik sällan skapar förändring och hur något så enkelt som att ställa fler frågor kan få enorm betydelse för människorna omkring oss.Ett inspirerande avsnitt fyllt av konkreta verktyg, livsvisdom och insikter om hur vi kan bevara det mänskliga i en värld som blir allt mer digital.Följ Thomas härLäs mer om Thomas böcker härMan´s search for meaning Läs mer om Framgångsakademin här.Ta del av Framgångsakademins kurser.Beställ "Mitt Framgångsår".Följ Alexander Pärleros på Instagram.Följ Alexander Pärleros på Tiktok.Bästa tipsen från avsnittet i Nyhetsbrevet. Hosted on Acast. See acast.com/privacy for more information.

Thomas Erikson är tillbaka i Framgångspodden. Författaren bakom den globala succén Omgiven av idioter har sålt över 12 miljoner böcker världen över – och i det här avsnittet delar han sina mest värdefulla insikter om relationer, kommunikation och varför mänskliga egenskaper blir allt viktigare i AI-eran.Vi pratar om varför så många unga idag känner sig vilsna, hur sociala medier påverkar vår koncentrationsförmåga och varför allt fler väljer att lägga undan sina smartphones för att återupptäcka verkliga möten och djupare närvaro. Thomas berättar också om sina egna strategier för att skydda sitt fokus i en värld där vår uppmärksamhet ständigt konkurrerar med digitala distraktioner.Vi går även på djupet kring en av livets viktigaste frågor: Hur hittar man sitt varför? Thomas delar den insikt som förändrade hans egen livsriktning och berättar öppet om hur han tog sig ur en destruktiv relation genom att konfrontera sina rädslor och börja leva mer i linje med sina värderingar.Vi får även svar på frågor som varför starka relationer ofta är den verkliga nyckeln till framgång, varför kritik sällan skapar förändring och hur något så enkelt som att ställa fler frågor kan få enorm betydelse för människorna omkring oss.Ett inspirerande avsnitt fyllt av konkreta verktyg, livsvisdom och insikter om hur vi kan bevara det mänskliga i en värld som blir allt mer digital.Följ Thomas härLäs mer om Thomas böcker härMan´s search for meaning Läs mer om Framgångsakademin här.Ta del av Framgångsakademins kurser.Beställ "Mitt Framgångsår".Följ Alexander Pärleros på Instagram.Följ Alexander Pärleros på Tiktok.Bästa tipsen från avsnittet i Nyhetsbrevet. Hosted on Acast. See acast.com/privacy for more information.

We're excited to have Databricks join us at AIEWF, among hundreds of the top companies in the AI Engineer ecosystem. LS subscribers can use their discount to get past the late bird pricing and access over $50k in sponsor offers! Everyone is still talking about Satya's Frontier Ecosystems post, but few have actually built a (now $175 billion) frontier ecosystem and cloud like our guests today.From open-sourcing the layer above coding agents to rethinking databases for the agent era, Databricks cofounders Matei Zaharia and Reynold Xin are pushing the company beyond the lakehouse into a full data-and-AI operating system. In this episode, Matei and Reynold join swyx at the 2026 Data + AI Summit to unpack Omnigent, LTAP, Lakebase, agent security, open formats, Mosaic, and why databases may matter more than ever once AI agents start doing real work.We go deep on Omnigent: Databricks' open-source meta-harness for combining, controlling, and sharing agents across Claude Code, Codex, Cursor, Pi, custom agents, and internal tools. Matei explains why coding agents and enterprise agents run into the same problems: portability, collaboration, session history, security, spend controls, and the need for a common API above every harness.Then Reynold walks through Databricks' database dream: why CDC is brittle enough to joke that it means “continuous data corruption,” why HTAP has been the holy grail of database engineering, and why Databricks thinks LTAP gets most of the benefits by unifying the storage layer instead of collapsing every query engine. We also cover Databricks' infrastructure scale, the culture behind rapid prototyping, the difference between tech and enterprise customers, Databricks vs Snowflake, whether vector databases should have ever existed, the Mosaic model strategy, Genie, AI Runtime, RL fine-tuning, and the thesis that traditional software gets rewritten once the data is in the right place and agents sit on top.Databricks began as a company for the big data era. The origination of Spark from the Berkeley AMPLab which eventually turned into the product Lakehouse convinced enterprises that they didn't need a separate data lake, warehouse, ML platform, and governance layer. They just needed one open foundation where all of their data could live and be reasoned over.Since then a lot has changed, but data has only become more important. Data is no longer something you keep track of and analyze ad hoc, it's the necessary context agents need in order to act. So the framing has shifted from “where do we put all of our data?” to “how do we expose the right slice of state, history, permissions, and business logic to an AI system at the exact moment it's doing work?”If frontier model performance becomes commoditized, the durable advantage then becomes the company-specific context around them: proprietary data, governed access, operational state, transaction logs, workflows, and feedback loops. Which makes Databricks positioned perfectly.Now coming fresh off the Data + AI Summit 2026, the company is moving just as fast to keep up, announcing Genie One, Omnigent, LTAP, and many more, indicating a central mission in its newer work: Databricks is trying to become the operating system for enterprise agents.Models are getting good enough, but agents are only useful if they have the right context, permissions, memory, state, cost controls, and access to live business data. Fundamentally it appears that significantly better model performance in production is a systems problem, one that data guys like us are remarkably well prepared to solve!We discuss:* Why Databricks built Omnigent as a meta-harness above existing AI agents* Why coding agents and custom enterprise agents need the same infrastructure* The common API for agent sessions, files, streams, tool calls, and cancellation* Why persistent sessions, cloud sandboxes, sharing, search, and collaboration matter* Why Databricks open-sourced Omnigent instead of keeping it proprietary* Databricks' internal agent usage, cloud sandboxes, and coding workflows* The scale of Databricks: 50–60 million virtual machines a day and exabytes before breakfast* Why agent security needs contextual and stateful policies* How an agent could read confidential docs, install a compromised npm package, and leak data* Why spend control matters when an agent can burn $500 reading logs* Startup opportunities around coding-agent analytics, quality, skills, and spend* LTAP, Lakebase, and why Databricks wants to rethink the database stack* OLTP vs OLAP, CDC, and why data pipelines break at 3 a.m.* Why HTAP has historically been the holy grail of database engineering* Why Databricks thinks LTAP is “HTAP done right”* How writing transactional data into column-oriented formats changes analytics* Why agents need live operational context from databases, not just telemetry* How Databricks prototypes strategic systems without endless process* Enterprise vs tech customers, governance, procurement, and DIY culture* The “second system syndrome” risk of rewriting a database engine* Building a database engine from a decade of traces and quadrillions of data points* Why vector databases should never have been a separate category* Why open formats and AI changed the race with Snowflake* The Mosaic story, DBRX, Genie, document parsing models, and specialized model training* Why model customization and RL fine-tuning may become mainstream* Why “get the data there, slap some agent on top” may rewrite traditional softwareMatei Zaharia* LinkedIn: https://www.linkedin.com/in/mateizaharia* X: https://x.com/matei_zahariaReynold Xin* LinkedIn: https://www.linkedin.com/in/rxin* X: https://x.com/rxinDatabricks* Website: https://www.databricks.com* X: https://x.com/databricksTimestamps00:00:00 Introduction00:02:22 Omnigent and the Agent Infrastructure Layer00:08:39 Agent Clouds, Common APIs, and Open Source00:16:52 Databricks Scale and Internal AI Workflows00:18:03 Agent Security, Governance, and Spend Controls00:27:34 LTAP and the Database Dream00:30:30 CDC, HTAP, and Why Data Pipelines Break00:34:05 Lakebase, Parquet, and Live Data for Agents00:36:47 Databricks' Culture of Fast Prototyping00:43:40 The Dream Engine and Rewriting the Database Stack00:51:02 Vector Databases, Query Engines, and LTAP00:52:36 Databricks vs Snowflake00:57:48 Mosaic, DBRX, Genie, and Specialized Models01:03:11 Context, AI Runtime, and RL Fine-Tuning01:06:15 Why Data + Agents May Rewrite Software01:07:09 Closing ThoughtsTranscriptIntroduction: Databricks, Data + AI Summit, and Founder DynamicsSwyx [00:00:00]: Matei and Reynold from Databricks, welcome to Latent Space.Reynold Xin [00:00:06]: Hey, thanks for having us.Swyx [00:00:07]: Yeah.Matei Zaharia [00:00:08]: Yeah, thanks so much.Swyx [00:00:09]: thanks for taking time out. You have your Databricks, Data AI Summit going on. You were just telling me how the first summit that you guys ran was just 50 peopleReynold Xin [00:00:17]: Yeah, it wasSwyx [00:00:17]: in BerkeleyReynold Xin [00:00:18]: little meetup at Berkeley, I thinkMatei Zaharia [00:00:19]: YeahReynold Xin [00:00:19]: put togetherMatei Zaharia [00:00:20]: We were doing these tutorials and, yeah, just teach people Spark.Swyx [00:00:23]: Yeah. obviously now it's like, I think like the headline number's like 100,000 people around the world, 30,000 in person.Swyx [00:00:30]: it's a crazyMatei Zaharia [00:00:31]: AmazingSwyx [00:00:31]: community. Well, I just saw the keynote.Swyx [00:00:35]: Ali's just. Did was it obvious or that back when that Ali would be, like, such a great, like, CEO? LikeReynold Xin [00:00:42]: OhSwyx [00:00:42]: such a great presenter?Reynold Xin [00:00:43]: What do you think?Matei Zaharia [00:00:44]: I think among our group of founders it was clear that, I think he'd be the best at this.Swyx [00:00:50]: Yeah.Matei Zaharia [00:00:50]: And yeah, it turned out great. And he's, he's ramped up on so many topics growing a company. He would just go in and, like, study it and, be talk to all the experts. Like, even if he can't hire the person, learn enough about, like, finance and sales and whatever it was, and, and go from there. Yeah.Swyx [00:01:09]: Yeah.Reynold Xin [00:01:10]: he's obviously very high IQ and a very high EQ, but it wasn't. Like, Ali today is quite different from Ali from, like 10 years ago. I think there's a lot of work that he put in to, get to this point.Swyx [00:01:20]: Yeah. no, to me the most appealing thing about him is that he's funny. And like, it, it's, it'Matei Zaharia [00:01:26]: It's true, yeahSwyx [00:01:26]: it's hard to make jokes about, data warehousesReynold Xin [00:01:30]: About serious topicsSwyx [00:01:31]: securityMatei Zaharia [00:01:32]: YeahSwyx [00:01:32]: what have you.Matei Zaharia [00:01:33]: Oh, yeah. That's for sure.Swyx [00:01:34]: Yeah. So you guys launched a whole bunch of things. I'll, I'll just name check briefly, the stuff because we're not gonna cover everything. Omnigentt, your baby. LTAP, your baby, your dream engine.Swyx [00:01:47]: we're also gonna cover Genie, cover CustomerLake, you acquired PantherMatei Zaharia [00:01:52]: YeahSwyx [00:01:52]: Open Sharing, and there's Unity AI Gateway. A lot of these, I think, like, are things that you would expect a Databricks to do. It's, it's like part of the roadmap. Everyone in your category has similar things. But I think, probably the two of you are leading the two most unique and differentiated initiativesOmnigent and the Agent Infrastructure LayerSwyx [00:02:09]: on, in the landscape. Maybe we'll start with, Omnigentt we'll, we'll, we'll, we'll go into it. I do think that a lot of people are exploring this meta harness concept.Matei Zaharia [00:02:21]: Yeah, totally.Swyx [00:02:21]: What led you to it?Matei Zaharia [00:02:22]: Yeah. There were a couple of, like, converging lines, which I think is a good sign that you need something new. So on the one hand, there's all the coding agent info internally. We have really great, dev infra team. they built something called Isaac, that's like a wrapper on Claude Code and Codex, and, lets you use them either on the web in, like, sandboxes or, just on your dev machine or on your laptop or whatever. And then, they were adding all kinds of stuff there. And we saw all the more advanced engineers like, were building their own workflows with tons of agents, and they were building their own UIs and stuff on top or even on top of that. And then the other one was, like, us building agents. We ship this, like, data science agent called Genie on the research team, which I lead. We also build a lot of internal ones for various things, and then we have all the customer ones. And all of them running into this thing of like, “Oh, I need to switch model and harness and so on,” every few months. Plus the agent is, like, completely useless if you can't share sessions with someone and have history and have search and all this, like, layer on top of it for collaboration. I thought a bit about it from both contexts and, at first people thought it was weird. They're like, “Why are you doing coding agents and custom agents in the same thing?” But I said it's, it's the same problems and, you just wanna build the stuff that lets you deliver the agent, maybe control it if you care about security, and, make it portable across things. And then we prototyped some things as experiments. We saw, yeah, we can make it work, and then we built that for real.Swyx [00:04:06]: I'm wondering if this let's call it architectureMatei Zaharia [00:04:11]: YeahSwyx [00:04:11]: maps to anything in your careers in the past. like I always think about how a lot of things just tie back to operating systems.Swyx [00:04:18]: A lot of operatingMatei Zaharia [00:04:19]: YeahSwyx [00:04:20]: systems tie back to databases,Matei Zaharia [00:04:21]: SoSwyx [00:04:21]: or the other way aroundMatei Zaharia [00:04:22]: so the thing, I do think it ties a lot to, like, network protocols, internet protocol. we alsoSwyx [00:04:29]: Communication between entities.Matei Zaharia [00:04:30]: Yeah. We did stuff with, like, data sharing also, which is probably, most viewers probably won't know unless they'Swyx [00:04:36]: Yeah, open protocol is the term.Matei Zaharia [00:04:37]: Yeah.Swyx [00:04:38]: Open sharing. Open sharing.Matei Zaharia [00:04:38]: Open sharing.Swyx [00:04:39]: Yes.Matei Zaharia [00:04:39]: Yeah. So it's like you have a company, you maintain some table, like let's say like a Walmart or something. They have like the, inventory and what's been sold in each store. And then you also have suppliers, and they would love to produce more things and ship them, like, exactly the moment you need them. So they would love, like, real-time access to your table. So instead of like sending emails around or Excel sheets or phone calls, why can't you share like a view of that table in real time with them? Then they query, they, join it with their data, and they decide what to send. So it's one of these things where you, like you might ask like today since we can vibe code anything so fast, why do we even need to design like protocols or APIs or software? Why can't you just vibe code things on demand? But for this type of interoperability where multiple parties that are moving at different speeds are building stuff and you still want some layer on top to coordinate, you do wanna design it and build it. So it reminds me of that, like agents talking to each other and, users talking to agents and tools.Agent Clouds, Cloud Sandboxes, and Keeping Sessions AliveSwyx [00:05:42]: Reynold, any other comments alternative viewpoints?Reynold Xin [00:05:46]: I think, by the way, we had a debate on exactly which set of benefits would, matter a lot, and I think around the time we decided to do this thing I was telling Matei, “Hey,” it just happened to be there's a particular week that I was coding nonstopSwyx [00:06:00]: from the moment I woke up to, like, the moment I went to bed, I was, like, looking at my Claude sessions, my Codex sessions. And one of the things that was particularly annoying was having to keep my laptop open.Swyx [00:06:12]: I was driving to a doctor's appointment, and I remember because I wanted to make sure the whole thing continues working.Matei Zaharia [00:06:18]: But by the way, it's so comforting to hear you say that because I'm like, “I don't know if I'm a clown and I'm doing this or like.”Swyx [00:06:25]: Yeah. Like honestly, I was driving and I was tethering my laptop to my phone.Matei Zaharia [00:06:29]: huh.Swyx [00:06:29]: Keeping it on the side. Whenever I hit a red light, I started looking at what's going on my laptop.Matei Zaharia [00:06:35]: Yeah.Swyx [00:06:35]: And I just felt that was ridiculous.Matei Zaharia [00:06:37]: Yeah.Swyx [00:06:37]: It felt like we went back to the dark agesMatei Zaharia [00:06:39]: YeahSwyx [00:06:40]: programming. the productivity you gain from all this coding age is amazing, but, yeah.Matei Zaharia [00:06:45]: Have you heard of cloud?Swyx [00:06:47]: Yeah.Swyx [00:06:48]: It was crazy to me.Matei Zaharia [00:06:49]: Oh, the thing you were working on was the sandboxes or was this before that?Swyx [00:06:52]: It was a sandbox.Matei Zaharia [00:06:53]: Okay.Swyx [00:06:54]: I was workMatei Zaharia [00:06:54]: So you were inSwyx [00:06:55]: So I was approaching from a very different angle. I wanted to, “Hey, we're gonna have cloud sandboxes that doesn't shut down. You can get one very quickly,” but not just for running agentic sessions.Matei Zaharia [00:07:06]: Yeah.Swyx [00:07:06]: It's also for running development. So I was personally building that week, and through building that, I ran into all these issues, and then I wroteMatei Zaharia [00:07:15]: YeahSwyx [00:07:15]: a document for Matei, it's like, “Here's my wish list of what the actual environment should do.” And I think he ended up almost implementingMatei Zaharia [00:07:22]: YeahSwyx [00:07:22]: every single one of them.Matei Zaharia [00:07:23]: Yeah, I remember Reynolds saying, ‘cause my first prototype of this had just chats with your agent and he said, “I have to be able to open a shell, like my own shell and like list files and like tail them and stuff.” SoSwyx [00:07:36]: So SSH into a mainframe.Matei Zaharia [00:07:37]: Yeah. it has that now.Swyx [00:07:39]: Tailing my log.Matei Zaharia [00:07:40]: Yeah.Matei Zaharia [00:07:41]: Yeah.Swyx [00:07:41]: And also another thing I think I asked was, I had. I still use cursor for the sole purpose of rendering markdown files.Matei Zaharia [00:07:48]: huh. Yes.Swyx [00:07:49]: So I said, “If you just give me a way to see my markdown files and renderMatei Zaharia [00:07:53]: YeahSwyx [00:07:53]: them properly, I don't need a separate tool anymore.”Matei Zaharia [00:07:55]: Yeah.Swyx [00:07:56]: And I think you also built that in.Matei Zaharia [00:07:57]: Yeah, we, yeah, we did that, yeah. Yeah, we had a lot of engineers building, their own vibe coding setup. But then the other thing they all said is like, “Hey, I built something that's amazing for me, but, like, no one else on the team can use it ‘cause I don't have a server to collaborate.” And this is why we tried to set up, Omnigent, so you can have a server and have the security, set up in there. So, like log in with Google or whatever and, like securely share stuff. which. And that's where we've seen a lot of other agents like hit things. Like people think they prototyped an awesome agent, but it's not allowed to connect to like some really important data or whatever because of the security team.Omnigent Architecture, Open Source, and Common APIsSwyx [00:08:38]: Yeah.Matei Zaharia [00:08:38]: So yeah.Swyx [00:08:39]: Yeah. At this point, so for those watching along on YouTube, we're gonna putting up a image of the structure here, and we can talk a little bit of the architecture. I think I just want to have people understand, ‘cause like when we're talking about software, it can be very abstract and like here is what we're talking about. You've worked out in open source this entire platform and there's a runner component and server component with a uniform API that you've, you've figured out. any other element and obviously you can plug in all this, persistence layers and compute layers. This is a whole cloud. It's an agent cloud.Matei Zaharia [00:09:12]: Yeah. It's, it's got these components to work with it. The, a lot of the action happens like on the machine where you deploy your agent too. So whatever you've got on there, you can run. But yeah, it's, I think it's the minimal thing you want to have hosted, like collaborative agents and to have that server. And one of the reasons we open sourced it is, anyone building agents, this gives them an app they can start with and customize, which we were seeing in Databricks too. Like someone would make a nice, agent app and then other teams would ask, “Oh, can I just use yours for my agent?”Swyx [00:09:45]: Yeah, I think we had like five or six different agentic frameworksMatei Zaharia [00:09:48]: YeahSwyx [00:09:48]: built by every different team. They do all do more or less the same thing. Yeah, you need to. people wanna take something that works in Forkit, and you might as well have something open source. Yeah, which also was another question, which is interesting for Databricks. Like what do you choose to open source? What do you choose to make it proprietary? It's in. this goes back to Spark, right?Matei Zaharia [00:10:05]: Yeah.Matei Zaharia [00:10:06]: One, so one of the reasons to open source something is if you think it's a layer that will there'll be some network effect, it'll benefit from many, people collaborating, on it. So, for example, with Spark, I don't know if when Spark came out, we also focused a lot on letting you have libraries on top. So like there used to be differentSwyx [00:10:28]: EcosystemMatei Zaharia [00:10:28]: distributed computing engines for like machine learning and graph computation. We said they should all be libraries that you can compose. And we made it super easy to add connectors to data sources too. And then we benefit because, we don't have the time to write like connectors to like, 1,000 like different databases and file formats, but we can just use the ones people make, and of course they benefit from joining, this thing. So that's like one of these as it. Another way to think about it is like imagine, we our thing wasn't open. We had some agent hosting thing, but it's not open and then there is an open one. if you're. Which one's gonna win in the long run? So like here, because there is this benefit from like people writing integrations, it'll be, it'll be that. And then there are other things that like you just can't, even deliver as open source that are things the company does. Like for example, how do you make sure you're like streaming, jobs or your Lakebase database doesn't like, lose all your data at night? Well, that requires an operational team that's gonna sit there. There's no way it has to be a service. So like we wanna make sure as a company we're really good at those infra services and then we're as open as we can in terms of like what you build on top.Swyx [00:11:42]: speaking from a benefits, I think we are already seeing pull requestsMatei Zaharia [00:11:45]: YeahSwyx [00:11:45]: of all kinds of ecosystem integration, even though it was only released on Saturday.Matei Zaharia [00:11:50]: Yeah, Saturday. Yeah. So someoneSwyx [00:11:51]: Let's see, let's see what's going on. Yeah, you can look at the merge ones. I asked Sam Nigon this morning aboutMatei Zaharia [00:11:59]: 400 merge already?Matei Zaharia [00:12:00]: Yeah. I think Recent quite, I would guess around half are not from our team. but for example, someone added support for running it on Kubernetesrnetes. people added, many cloud sandboxes, so this can launch a cloud sandbox and run your agent in there, which is great for sharing too, ‘cause it's not, like, on your laptop and someone's, like, running scary code on there. so yeah, many startups have put those in, and, we expect to see more of them. We also have more agent harnesses already. Cursor, CLI, and Antigravity also.The Modern Data Stack and the Emerging AI StackMatei Zaharia [00:12:34]: Yeah. That's all, beautiful. And I, I feel like the last time this happens, there was the rise of the modern data stack.Matei Zaharia [00:12:42]: I don't know if it's that useful. I'm, I'm curious in your postmortem.Matei Zaharia [00:12:46]: I think most peopleSwyx [00:12:47]: AgreeMatei Zaharia [00:12:47]: will agree that it is finally dead. but maybe this arises to a new modern AI stack that, like, does the same thing.Matei Zaharia [00:12:52]: I don't know.Reynold Xin [00:12:54]: I think the modern data stack was a pretty useful thing, probably even up until this day. I think what, maybe for the audience who don't understand the history, I think the modern data stack is effectively decomposed into you need a layer to ingest the data in, you need a layer to transform your data, and then all of this are run, and then you need a layer to maybe visualize your data. And all of this runs on some data warehouse, or later on, as we're doing data warehouse or lakehouse.Reynold Xin [00:13:21]: I think that concepts are all very powerful and very useful. They enable a lot of workloads. What people eventually run into is a question of unification and consolidation is, hey, do you really need to chop all this into different pieces and work with so many different vendors and platforms in order to get, like, a very simple visualization done, right? So I think, like, over time, everybody started realizing that customers are pushing us. We started, we can realize that, so we started building more and more capabilities and trying to consolidate. And at the end of the day now, customers don't have to worry about having me hook up five different systems in orderMatei Zaharia [00:13:55]: YeahReynold Xin [00:13:55]: produce a chart. But the. I think, honestly, something like this is probably happening, in how many different frameworks do you want to hook up together in order to produce, like do a very simple agent.Matei Zaharia [00:14:06]: Just to be clear, I would say the core of this is this common API on top of all the harnesses. So the API is like, you've got an agent session, and you can send in a message or, like, a file. That's what you can send in, and then you get out, these streams as it's streaming text or as it's doing tool calls. And, or the other thing you can send in is you can, like, tell it to cancel a turn. So that's the API. Now, the thing we did is we could get you that on top of, like, cloud code running in a terminal, Codex, Py, OpenAI SDK, all that stuff. We map them all to that same interface. So that is something that you'd have to maintain yourself if you built your own, like, agent orchestrator, and then whenever cloud changes its API, you gotta, tweak your thing or it's gonna lose some messages. So that's the thing that's valuable to maintain. Then on top of that, like, we built a few apps. I think we built a pretty cool UI and stuff, but that's, And we built a security and control piece, which I'm excited about. But it's that common interface, so we don't. We. That doesn't try to be a stack. And in fact, you could plug in your own UI on top of this, server. That, and that's one of the use cases we care a lot about, ‘cause we want to use this in our own products.Compute, Sandboxes, and Databricks ScaleSwyx [00:15:20]: Yeah. It should be everywhere.Matei Zaharia [00:15:22]: Yeah.Swyx [00:15:22]: I think one of those things that is really interesting to me is, like, well, first of all, I'll, I'll endeavor to do everything and not call it the modern AI stack because like it needs a different name.Matei Zaharia [00:15:32]: Yeah.Swyx [00:15:32]: But like, yes, like, so one of the first people that told me about compute, sandboxing was Nikita from Neon.Swyx [00:15:39]: Because a lot of people think about Neon as like, well, it's serverless Postgres with, like, the separation of compute and storage and, instant branching and all those things. But every database company is also a compute company.Matei Zaharia [00:15:51]: Yeah. Yeah.Swyx [00:15:52]: And so he was showing to me his whole, his sandboxing solution. I don't think he have ever launched it.Matei Zaharia [00:15:57]: So our sandbox solution, the reason we could build it so quickly was because we realized if you just take the actual Lakebase architectureSwyx [00:16:05]: YeahMatei Zaharia [00:16:05]: and remove the database from it, by the coming from NeonSwyx [00:16:08]: Exactly, rightMatei Zaharia [00:16:09]: you have this sandboxSwyx [00:16:09]: Every database company has it already, yeah.Matei Zaharia [00:16:11]: Now, there are some differences. For example, in the one to support this particular workflow, it's important to have local persistence,Swyx [00:16:19]: YeahMatei Zaharia [00:16:19]: because you want your state to persist. Your libraries, you don't have to install your library every time, right?Matei Zaharia [00:16:24]: whereas the Neon architecture, because of the separation of storage from compute, you don't need persistent local disk.Swyx [00:16:30]: Yeah.Matei Zaharia [00:16:30]: So there's some differences.Swyx [00:16:32]: Yeah.Matei Zaharia [00:16:32]: But the, at the end of the day, yeah, it's, Yeah, so this is when you run, like, a coding sandbox. Like, if I use it, yeah, we have the dev env internally at Databricks. There's, like, many, like, tens of gigabytes of data just for, like, all the source code and, like, artifacts and stuff that I built, and I want that to come back next time, so.Matei Zaharia [00:16:51]: Yeah.Matei Zaharia [00:16:51]: But yeah.Matei Zaharia [00:16:52]: Before the show, we was talking about some statistics that might be surprising at the adoption.Matei Zaharia [00:16:56]: It could be internal, it could be external, whatever comes to mind, just to impress people the scale this is happening.Swyx [00:17:02]: So we, on the analytics side, I think we launchedReynold Xin [00:17:06]: Maybe 50 or 60 million virtual machines a day across all three clouds, so we're one of the biggest compute orchestrators out there.Reynold Xin [00:17:13]: Stuff for sure for CPU compute.Swyx [00:17:14]: Yeah.Matei Zaharia [00:17:14]: Yeah.Reynold Xin [00:17:15]: the. And all of this process, I think exabytes of data, I joked about depending on which time zone you are, typically before you have breakfast, Databricks would have processed exabytes of data already on that day. and on Neon, it's pretty interesting, too. It's launching, I think, 13 million databasesSwyx [00:17:34]: YeahReynold Xin [00:17:34]: a day now.Swyx [00:17:35]: Yeah, to me that was, like, aReynold Xin [00:17:36]: And that's just likeSwyx [00:17:37]: Like, what do you mean?Matei Zaharia [00:17:38]: Yeah. And that's the point.Reynold Xin [00:17:40]: And a lot of those were thanks to agent- agents and branching experimentationSwyx [00:17:44]: YeahReynold Xin [00:17:44]: because we made it so easy and so quickly, and thanks a lot to Nikita's team, to launch databases. It's, the. So it's changing the way people use databases.Swyx [00:17:54]: Yeah. Okay, we're gonna go into more database talk in a bit, but I wanna make sure we close up anything on Omnigentt. you mentioned, you were excited about the securityOmnigent Security, Contextual Policies, and Spend ControlsSwyx [00:18:03]: control side.Matei Zaharia [00:18:04]: Yeah.Swyx [00:18:04]: a lot of companies are figuring that out right now, as well as the spend side.Matei Zaharia [00:18:08]: Yep.Swyx [00:18:09]: what have you found there?Matei Zaharia [00:18:11]: Yeah, so I spent quite a bit of time talking to internal users, developers, security team, managers, and also lots of customers, and there's a few things. Like, first of all, one thing, that immediately was. became obvious is for security, there's this tension between, like, usability and security. And, the way people do. Like, a lot of coding agents today have very basic things like you can tell me which tool patterns I'll allow or disallow or whatever. It's like yes or no. But that puts you in a very tough spot. So just as an example, like, should my agent be able to read, some confidential documents, or let's say, should it be able to install new packages from npm, which, maybe it's compromised. Yes or no? Like, maybe I wanna allow it. Should my agent be able to publish stuff to the company website? Well, if I'm using it to code on the website, yes. But should it be able to do both, so it can, like grab a confidential document and be prompt injected and leak it? Probably not. So the thing we decided we need is stateful or what we call contextual policies where you keep track of the state of that session. It's not like is it allowed to push to the marketing site or not, but, like, hey, if it did a risky thing, like it installed, a old package from npm, or it read, like, 1,000 confidential docs, then no. Then don't, don't do it. Otherwise, maybe it's okay. That's one example of, like, moving that trade-off so it's both more secure and more useful by having a more powerful engine, essentially. This requires tracking sessions. The other piece that was interesting there is, like, there are these very level events it's doing, and you want some libraries on top that parse them. Like, for example, we have a, MCP server on Google Drive internally. It's got 60 API calls. like, how do I know which of those, like, will share a document with stuff on the internet and which ones won't? It's, it's annoying. So we designed in Omnigentt the policy layer so that it's functions and you can have libraries. Like, someone can make something that maps the level events to high-level ones, and then you write a policy about the high-level things that came out. so and thatSwyx [00:20:25]: This is related to the Panther,Matei Zaharia [00:20:27]: Yeah, Panther is. will help with that. PantherSwyx [00:20:30]: YeahMatei Zaharia [00:20:30]: a similar idea on the event processing side, and it's Python-based versus a weird custom language. this is more, as in realSwyx [00:20:39]: I didn't even know we were good yeah.Matei Zaharia [00:20:41]: Those things are happening, yeah.Swyx [00:20:42]: Yeah.Matei Zaharia [00:20:42]: So yeah, but these are the cool things. I think the contextual or stateful part, and then the way it can be libraries, and that was another reason to make it open source because others will write libraries and, like, we and our customers can use them. And the final thing, because it's stateful, one of the states we track is how much you spent in that session. So I can. I've had, like, I ask an agent to debug something, and it spent $500 because it decided to read a lot of log files and burn a lot of tokens. but I can literally say, “Okay, launch a agent to do this and cap it to spending $5.” Like, ask me for permission if it needs more. And because we're counting that within that session, it'll pop up and tell me, “Okay, you spent five, $5. Do you wanna go on?”Reynold Xin [00:21:27]: So important context here. Matei spent the last five years, a lot of his time was architecting Unity Catalog at DatabricksMatei Zaharia [00:21:34]: YeahReynold Xin [00:21:34]: which is the governance layer for data.Matei Zaharia [00:21:35]: That's right, yeah.Reynold Xin [00:21:36]: And he's combining expertise at that layer together with all the AI governance he knows.Matei Zaharia [00:21:41]: Yeah.Swyx [00:21:41]: DoMatei Zaharia [00:21:41]: But I also spent a lot of time being annoyed by coding agents and getting prompts.Matei Zaharia [00:21:46]: And also as theReynold Xin [00:21:48]: All the aboveMatei Zaharia [00:21:48]: I don't want to end up on the front page as, like, I installed some weird npm package and leakedSwyx [00:21:53]: YeahMatei Zaharia [00:21:53]: all the code, so I'm especially paranoid. But also I have very little time, so I don't want to sit there approving, like, do you want to run a 20-line, bash script, yes or no? so that's why I spend a lot of time figuring out, like, how can I make it as safe as possible and not annoying?Swyx [00:22:10]: Yeah. Is safety and mmm, let's call it security a bigger concern than token maxing or token budgets? which one is, likeMatei Zaharia [00:22:19]: Oh, yeah, they're both there. I don't know. I guess it depends on the type of company you are. So I think, some companies, like, the budget is, limited and, they really care about thatSwyx [00:22:34]: you can be Uber and still be concerned?Matei Zaharia [00:22:36]: Yeah. Oh, yeah, totally. Yeah. If you haveReynold Xin [00:22:38]: for us, securityMatei Zaharia [00:22:39]: YeahReynold Xin [00:22:40]: super paramount.Matei Zaharia [00:22:40]: For us, security is absolutely critical as a, cloud provider. It's, it's the most important thing, and, token maxing, we're not so worried about it yet, but I've seen the Like, for example, I talked to some consulting companies. They have, like, 100,000 employees who are all coding for customers. If those each spend, like, an extra $1,000 a month, that's, that's not fun.Swyx [00:23:04]: YeahMatei Zaharia [00:23:04]: we have, like, only a few thousand engineers.Swyx [00:23:06]: What's the policy in Databricks? Is it just unlimited or what'Matei Zaharia [00:23:08]: It's, it's unlimited, but we do. we use our own product to, like, analyze the traces and stuff, and we have a team that'looking to optimize and to see if anyone's doing something weird. And, we had some really cool insights just from analyzing current traces, like whichSwyx [00:23:24]: YeahMatei Zaharia [00:23:25]: models are better at, say, Rust versus like TypeScript or whatever. So yeah, at least in our code base.Swyx [00:23:31]: Yeah. Amazing. Obviously, I have to ask the token question, obviously.Matei Zaharia [00:23:34]: Yeah.Swyx [00:23:34]: I think it'sReynold Xin [00:23:34]: YeahSwyx [00:23:34]: it's a key thing. But yes, security and control above that, and figuring out a sane layer there you can have some autonomy, but, not too much.Matei Zaharia [00:23:43]: Yeah. Yeah, and we wanna make it super easy. As a engineer, you should set a thing. So in Omnigentt, you can ask your agent, “Set a policy on yourself to do this.” So it can likeSwyx [00:23:52]: But if there's something I should be showingMatei Zaharia [00:23:53]: YeahSwyx [00:23:53]: I don't, I don't see it on the GitHub, but,Matei Zaharia [00:23:55]: Oh, yeahSwyx [00:23:56]: there's justMatei Zaharia [00:23:56]: Well, in the docs there's something.Swyx [00:23:57]: Yeah, this is it.Matei Zaharia [00:23:58]: You can look at it later.Swyx [00:23:59]: Okay. Yeah.Matei Zaharia [00:23:59]: Just look in the docsSwyx [00:24:00]: YeahMatei Zaharia [00:24:00]: contextual policies if you wanna see.Swyx [00:24:04]: I just like to point peopleMatei Zaharia [00:24:05]: look at the built-in policies.Swyx [00:24:06]: Yeah.Reynold Xin [00:24:06]: Yeah.Swyx [00:24:06]: If you want to, follow up on this is exactly where to look, right?Reynold Xin [00:24:10]: Yeah.Matei Zaharia [00:24:10]: Yeah. yeah, and the story of these is, like, I just wrote, like, I wrote a doc with like 10 ideas for things before as you were working on them. Well, that was, like, my wish list of things people asked, and I told the team, like, “Hey, can you do like at least five of these for the launch?” And then they just got back with all of them, so.Swyx [00:24:29]: Oh, wow.Matei Zaharia [00:24:29]: so you can come up with more, but them- some of them are just meant to be examples. really you can intercept, like, any event the agent is making, and you can then either block or force it to ask the user or, like, allow, and you can update state to keepSwyx [00:24:45]: YeahMatei Zaharia [00:24:45]: track stuff.Swyx [00:24:46]: Yeah, ‘cause ultimately you're, I think of you as, like, a systems designer.Swyx [00:24:50]: You let people plug in, right? That's the wholeMatei Zaharia [00:24:51]: YeahSwyx [00:24:52]: modus operandi of what you do.Matei Zaharia [00:24:53]: Yeah.Swyx [00:24:54]: It's likeMatei Zaharia [00:24:54]: And we care a lot about also composab- like, can someone else write a library that others use, whichSwyx [00:24:59]: YeahMatei Zaharia [00:24:59]: this is meant to.Reynold Xin [00:25:00]: There's also a batteries included philosophy hereMatei Zaharia [00:25:03]: YesReynold Xin [00:25:03]: probably very similar to how you did Spark, which is you could just start using.Swyx [00:25:06]: Yeah.Matei Zaharia [00:25:06]: Yeah, that's right. It has to be good out of the box at certain things, and then you can build your own things on top that, like, we don't wanna do. But in Spark, if you just wanna like, I don't know, like read a table or do, like, a aggregation, it should be awesome at that out of the box.Building on Omnigent: Contributions, Startups, and AnalyticsSwyx [00:25:23]: Yeah. People wanna catch up on Omnigentt, they should watch your keynote.Swyx [00:25:26]: they should go through the GitHub and the docs. If they wanted to contribute, or they want to build on this ecosystem what would you call out as the most high-leverage places get involved?Matei Zaharia [00:25:36]: Yeah, do get involved in the Discord and in GitHub. Our team is there, is monitoring, and, some of the things people ask for we just built ourselves. Some of them, we're, we're collaborating with them to build it. and also tell us, likeSwyx [00:25:49]: Yeah, they're gonna be veryMatei Zaharia [00:25:49]: how you would like to use it because I think especially for developers, like, everyone wants it to work their own way, and a really good developer tool, like you have to hear the feedback on all the ways and figure out the abstractions and how to let people customize. So we'd love to hear, like, if you think, “Hey, I, I don't want it to work this way,” tell us. We really just wanna get that compatibility layer across agents and then let you do stuff on top.Swyx [00:26:14]: Yeah. is there any, in terms of like the startup side, I'm, I'm a founder.Swyx [00:26:18]: I wantMatei Zaharia [00:26:18]: YeahSwyx [00:26:18]: I see an opportunity, I wanna get in front of you. What's your request for, like, a startup that, like, I wish someoneMatei Zaharia [00:26:23]: Oh, like you wanna integrate with us?Swyx [00:26:24]: someone was working on this.Matei Zaharia [00:26:26]: Oh, for a startup?Swyx [00:26:27]: Yeah.Swyx [00:26:28]: Like, your, you got your own startup. It's doing well.Matei Zaharia [00:26:30]: Yeah.Swyx [00:26:30]: But like, if you weren't working on your own startup, what is, like, obvious that you should You advise many startups too, obviously.Matei Zaharia [00:26:37]: I do think, just as a company with a lot of engineers, like anything that helps me make sense of how people are usingSwyx [00:26:46]: SpendMatei Zaharia [00:26:46]: coding agents and,Swyx [00:26:48]: Yeah. AnalyticsMatei Zaharia [00:26:48]: spend, but also quality or like you should write, you should add this skill, or you should write this thing, or your agents are really horrible at tasks involving this service, so I go spend time. That would be nice. yeah.Swyx [00:27:00]: Yeah. The closest I've found is, this team, GitAI.Matei Zaharia [00:27:03]: Oh, cool. Yeah.Swyx [00:27:04]: They started with, like, we will just do, code and human attribution, but they're building the analytics layer on top of that.Matei Zaharia [00:27:12]: Yeah.Swyx [00:27:12]: I do think, like, there are a bunch of, like, artificial analysis is obviously,Matei Zaharia [00:27:18]: Yeah, they have their benchmarksSwyx [00:27:18]: doing super wellMatei Zaharia [00:27:19]: YeahSwyx [00:27:19]: with their stuff. so there's, there will be people. I think this is like the domain of consultants first, but then peopleMatei Zaharia [00:27:26]: YeahSwyx [00:27:26]: will build software that, let's say, it's kinda like the management planeMatei Zaharia [00:27:29]: YeahSwyx [00:27:30]: for coding agents.Matei Zaharia [00:27:30]: Yeah, I think there'll be a lot of insights there. You have it in other areas.Swyx [00:27:34]: Okay. Well, and then the other, big thing is your dream engine.LTAP: Lake Transactional/Analytical ProcessingSwyx [00:27:39]: maybe you wanna tell the story of, LTAP.Reynold Xin [00:27:45]: So, and background with. I'm, I'm gonna make people listen to our Ankur Goyal episode where we talked about SingleStore, HTAPMatei Zaharia [00:27:52]: YeahReynold Xin [00:27:52]: and all that history.Matei Zaharia [00:27:52]: Yeah. The LTAP idea is pretty simple. so if people have heard of the, Ankur's, talk about HTAP, it's effectively the world of databases. Sorry, there's like maybe a lot of context needs to be injected here. The world of databasesSwyx [00:28:06]: I am happy to be the database podcast that I'm forcing people to, like, learn your databases, guys.Swyx [00:28:11]: You cannot vibe code with just markdown files.Reynold Xin [00:28:13]: Yeah.Swyx [00:28:13]: Like,Reynold Xin [00:28:14]: It's one of the most important fundamental systems technologies out there. But the world of database effectively split into roughly two halves. There's what we call OLTP databases, which are transactional, and think of your Postgres, your MySQL, your Oracle databases, and the other side is what we call analytics, and sometime might refer to term OLAP. And the difference is on OLTP, you typically have maybe run some transaction on some event that looks up at one specific row. We update that row, right? It's a very oriented data structure. And on analytics, you're trying to reason on the data. You're trying to compute, “Hey, what's my revenue per store? What's my. How's my website doing every day?” And then you, eventually want to probably end up running anal- machine learning on it to predict, “Hey, how will my maybe sales be going in the future?” they are so very different architecture, and everybody start with OLTP databases. Every app, when you become serious enough, that needs more than markdown files, you need to have a database. You want to lose your data, you want to have some transactional consistency. But once you want to reason on the data, if you only have like- A hundred rows, it's probably okay to run it on your Postgres or your own, your MySQL database. But once you have more data and want to run more complicated analysis, the very analysis might crush your Postgres database. So you start doing, getting data out of the OLTP databaseSwyx [00:29:35]: Replication.Reynold Xin [00:29:36]: Replicate them into the analytic systems and just startSwyx [00:29:39]: Yeah, which for people, Elasticsearch is, like, aReynold Xin [00:29:42]: Yeah. So some of them get into Elasticsearch for, like, blocked analysis. A lot of our customers obviously get into Databricks to run more sophisticated things.Swyx [00:29:51]: Yeah.Reynold Xin [00:29:51]: And there's this term called CDC, whichMatei Zaharia [00:29:54]: Change data captureReynold Xin [00:29:55]: change data capture. and what it does, it reads the binlog of the database, and if you don't understand what binlog is, it's fine. The, but it's a little delta of the data, and it reconstructs based on the delta, the state of the database, on the analytics side. But CDC is, like, a very painful thing. It's how standard in the industry, everybody uses it, but, it ends up being. I think many data engineers ends up being waken up at, like, 3:00 a.m, because there's some pipeline thing.Swyx [00:30:22]: my explanation is, like, Airbyte is like a, became a $5 billion company just doing CDC.Reynold Xin [00:30:27]: Yeah, exactly.Reynold Xin [00:30:28]: CDC is, like, a veryMatei Zaharia [00:30:30]: It's hard.Reynold Xin [00:30:30]: It's one of the most boring but one of the most fundamental operations, like, powering modern society.Matei Zaharia [00:30:37]: huh.Reynold Xin [00:30:37]: But it's so brittle that, we joke that it's, should be called continuous data corruption, because you might change your schema on your OLTP database, and then the CDC pipeline fails to handleSwyx [00:30:48]: YeahReynold Xin [00:30:48]: the schema change.Swyx [00:30:49]: Yeah.Reynold Xin [00:30:49]: And then everything goes out.Swyx [00:30:51]: And there's all sorts of tricks that you can do, like, you add in, like, some versioning or whatever, but yeah.Reynold Xin [00:30:55]: Yeah, but it's a very, in general, very complicated. Like, I think at my keynote, I asked the audience put up their hand if they love their CDC pipeline. Only, like, maybe two people put it up. So if single store, like, about maybe a decade ago, I think the industry had this idea, hey, what if I built a single database that can handle both workloads? Now I don't.Swyx [00:31:12]: Which, like, by the way, every database person ever has ever always dreamed about this.Reynold Xin [00:31:15]: Yes. Yes.Reynold Xin [00:31:16]: This is the holy grail of database engineering is why not build a single system that can do both of this? But it ends up just being a lot of compromises. one, I think one of the first issue is that, hey, each. they say Postgres has a massive ecosystem, right? You want to be using the tools that's built for Postgres. And Spark, for example, had a massive ecosystem. There's a lot of libraries you want to use. If you were to create now a new thing, you don't have a ecosystem. You tend to create a new, smaller proprietary API, and you're lacking both, and it's also very difficult to make it performance-wise to be, comparable on either side. So it ends up being sucking on both. And our whole idea of LTAP, it's obviously a wordplay on the term HTAP, is that we think this is HTAP done right. HTAP wants to build a single engine for both. We think you can get 99% of what you need by unifying the storage, and just have a single storage layer. And once you have the single storage layer, if your Postgres databases are writing data in a column-oriented format, everything analytics can just go read that data directly without any delay, right? There's no pipeline in between, so all the data will immediately be available for reasoning analytics. I think I was telling some customers earlier, hey, when we talked about this is gonna be super useful for agents, I at first didn't really believe in it myself, even though we wrote that positioning.Lakebase, Agents, and Live Operational DataMatei Zaharia [00:32:39]: Yeah.Reynold Xin [00:32:40]: But then last night I was having dinner with a Australian customer, and they told me, “Oh, hey, one of the big issue we have is we have all these logs from our services, and we see SLA dips and want to investigate. But then there's no way for those agents to even understand what's going on in the actual databases themselves. All we see is just, like, product telemetry of the database and the services.” It would make those agents 10 times more powerful if understand, for example, who's placing those orders, what is happening, what exactly are they doing. So now I'm sold on our own message.Swyx [00:33:13]: Yeah.Reynold Xin [00:33:14]: I think it's really. It gets you the almost all of the benefits of the HTAP holy grail, which is, hey, make the data available immediately for reasoning analyticsSwyx [00:33:26]: Yeah, I think,Reynold Xin [00:33:27]: without compromiseSwyx [00:33:28]: in the way that humans are generally intelligent and want to have the ability and access to query anythingReynold Xin [00:33:34]: YeahSwyx [00:33:35]: while they do the work, they also need history and need context.Swyx [00:33:38]: And, like, where else does they get context? That's it's an analytical workload.Reynold Xin [00:33:41]: Exactly.Matei Zaharia [00:33:42]: Yeah. Yeah. And I remember when we had incidents with our databases and engineers said, “Well, I can't just run a giant query on it to see what's going on because that's gonna bring down the database and hoard it even more.” Like, that's the stuff that this gets rid of, because you spin up a whole separate fleet of machines that's doing the analytics. You're not overloading, like, the main databaseReynold Xin [00:34:02]: RightMatei Zaharia [00:34:02]: that's still trying to serve stuff.Reynold Xin [00:34:04]: Yeah.Matei Zaharia [00:34:04]: Yeah.Why LTAP Works Now: Parquet, Postgres, and LakebaseSwyx [00:34:05]: So this has been a dream for a while. what had to get done in order to get to today? Like,Reynold Xin [00:34:11]: Yeah.Swyx [00:34:11]: I feel like, you have announced variants of this several times, but it wasn't as clear as LTAP.Reynold Xin [00:34:18]: Yeah.Swyx [00:34:18]: I think LTAP is like Like, okay, we've got it, guys.Matei Zaharia [00:34:21]: This thing, yeah.Reynold Xin [00:34:21]: I was talking to somebody at Meta, and then he was asking me, “Hey, what's the catch? Why is it possible now?” And I think the reality is we took a lot of time to work on the Lakebase architecture. obviously a lot of it came from the Neon team, which is a separation of storage from compute. And it turned out it was just a tiny little step away going from that to this LTAP idea, which is, hey, we just. in the Neon architecture and in Lakebase architecture, we're writing data in oriented format to the open data lake, but in there we're writing in Postgres pages. Ali and I were spending a lot of time debating, hey, can we just change that to write in column-oriented format? And we're just debating, and one day, one of our engineers who's, like, super smart came in, he's like, “Hey, I just prototyped it. It works.”Swyx [00:35:07]: Wait, it's, prototype what?Reynold Xin [00:35:09]: Prototype, instead of storing the data in the data lake in the oriented formatSwyx [00:35:15]: ColumnReynold Xin [00:35:15]: like Postgres pagesSwyx [00:35:15]: YeahReynold Xin [00:35:16]: write them in Parquet.Swyx [00:35:17]: Yeah.Reynold Xin [00:35:18]: and he just made the observation that, hey, our storage fleet has a lot of extra idle CPUs And we could use those CPUs to do the transcoding from row to column, where row is good for OLTP, but column is good for analytics. so let's do that transcoding at that time. And as a matter of fact, once you transcode the data compresses better. So from those services writing to, for example, S3 or other data lake, like object stores, you can write them faster ‘cause now they are now smaller.Matei Zaharia [00:35:49]: Yeah.Reynold Xin [00:35:49]: So there's no overhead, it's no compromise in performanceMatei Zaharia [00:35:52]: Some CPU overhead.Swyx [00:35:54]: Yeah, because,Matei Zaharia [00:35:55]: YeahSwyx [00:35:55]: we had extra CPUs anyway.Matei Zaharia [00:35:56]: We had that fleet anyway, yeah.Swyx [00:35:57]: so the debate ended. it's one of the classics of, tech, issue of a lot of debate, but then somebody went ahead and just tried to prototype it and it worked.Matei Zaharia [00:36:06]: But, like, something this strategicSwyx [00:36:07]: That's rightMatei Zaharia [00:36:07]: and important to the company, I expect there to be, like, a kickoff thing, like a design doc. Nothing like that.Swyx [00:36:13]: Nothing like that.Swyx [00:36:14]: He just. We were debating in many meetingsMatei Zaharia [00:36:17]: Yeah.Swyx [00:36:17]: and then we're just debating whether it's possible or not from first principle.Matei Zaharia [00:36:20]: YeahSwyx [00:36:20]: and then, somebody just did it.Matei Zaharia [00:36:23]: Yeah, if you set yourself up so people do that'll be great. And that happened a bit with Omnigentt too. I think if I just had a doc on, like, we can make these together, everyone would, would think, “Oh, what about this? What about this?” But then you. if you try it out, it helps. And then if you have real users and they bash it and, like, it's still working, or in this case, if you have the workload, what the workload looks like, you can just test the same pattern then.Databricks' Culture of Fast PrototypingSwyx [00:36:47]: Yeah.Matei Zaharia [00:36:47]: Yeah.Swyx [00:36:47]: Tech aside, which is very cool, this is, like, the most important thing, the culture of innovation, and you don't have to ask my permission, you don't have like, do a whole form- formal process, just do it?Matei Zaharia [00:36:59]: Well, especially these days, I think withSwyx [00:37:01]: YeahMatei Zaharia [00:37:01]: AI, it's easier to buildSwyx [00:37:02]: But so, likeMatei Zaharia [00:37:03]: a prototypeSwyx [00:37:03]: I think you are very I made a lot of suite of, like, large companies and, like, I think that at scale, things slow down, and I'm sure you felt it already, but somehow you have this core of people that, like, are exempt. How? I think we hire and we work with really good people, and that's a very important part of it, and empowering them, but also spending a lot of time, maybe us in the trenches matter a lot also.Matei Zaharia [00:37:28]: Yeah, I think, I think first, people can adapt to being in the larger company, so that helps. And we wanna make sure they know that they can try stuff and settle debates and have a lot of examples of how it was done before, or launch a thing in beta or whatever. and then the other thing I do think as a company, like despite the size, we don't launch that many, like, products. We try to keep it pretty coherent. That's, that was the whole, like, theory of the company, was like instead of having, like, 20 Amazon services you need to set up, like a analytics and machine learning stack, you just have one, and it's, like, the same API, the same semantics across all of them, the same copy of the data. So that requires, like, unification. And then we added one more thing at a time. Like, we added storage with Delta Lake. We didn't used to do any storage. Then we added SQL, we added, machine learning platform stuff. So, but yeah, don't, don't do too many, but do those things well and, that also helps, it helps keep it manageable.Reynold Xin [00:38:33]: Yeah. The other thing we encourage a lot is instead of building, boil the ocean for everything, let's figure out how do we do it incrementally, how do we do it very quickly. Like, many of our productsMatei Zaharia [00:38:43]: YeahReynold Xin [00:38:43]: they're built in the span of weeks, and then we go to, hey. Like, usually my first question to whoever team is building is who's the target customer? Who are you working with? Are you on a first-name basis with them? Are you texting with them? I think having that very tight loop,Matei Zaharia [00:38:59]: Can you bring up another launch that comes to mind when, in this thing? I just want to give examples.Reynold Xin [00:39:04]: Omnigentt itself happened that way.Reynold Xin [00:39:05]: Yeah.Matei Zaharia [00:39:06]: Who's the customer? That's a good oneReynold Xin [00:39:34]: storage layer we did. we had, our largest customer at the time said like, “Okay, I need some. I want something in the cloud ‘cause, I. if the rest of our network is compromised, like this thing needs to be separate to store and query the events.” And then, talked to us, he said, “Okay, this is the rate of events per second. This is, like, the freshness I want. Can you do it?” So that was, like, way larger than any workload we had, and we had our, engineer, working on that, Michael Armbrust, and he worked just to make this work. And once it worked for them, it worked for everyone else. Yeah. This was early in the company, probably like four years in or something.Matei Zaharia [00:40:24]: 20- 2018?Swyx [00:40:26]: Yeah, ‘17, ‘18.Matei Zaharia [00:40:28]: Few companiesSwyx [00:40:28]: Do you have other examples?Matei Zaharia [00:40:30]: there'Swyx [00:40:31]: Maybe you have othersMatei Zaharia [00:40:31]: yeah, Clean Room, which is how you share data in a way without sharingSwyx [00:40:35]: YeahMatei Zaharia [00:40:35]: underlying data, but you allow specific operations. Those were done effectively initially just for two customers. I think the industry has a sense of, hey, maybe if you overfit to, like, one or two customers, it's gonna be really bad for you. But I think the, downside of overfitting is much smaller than the upside itself. And if you try to be too ambitious and boil the ocean, it's a much bigger problem.Swyx [00:40:58]: Yeah. Yeah.Matei Zaharia [00:40:58]: ‘Cause you might end up having no customer.Swyx [00:41:00]: Yeah, that's more, that's the more likely outcome.Matei Zaharia [00:41:02]: Yeah.Tech Companies vs. EnterprisesSwyx [00:41:03]: than you can pivot from there. I do think there is such a thing as a bad customer that sometimes you should fire. Yeah.Matei Zaharia [00:41:08]: They could exist sometimes if you drive. well, one of the challenge I think we probably see, and maybe many AI, so newer generation companies are seeing is, so tech companies are very different from tech companies or traditional enterprises.Swyx [00:41:22]: Yeah.Matei Zaharia [00:41:22]: And, if you optimize everything just for tech companies, you might have various challengesSwyx [00:41:27]: OhMatei Zaharia [00:41:27]: scaling them outside of tech companies.Swyx [00:41:28]: Okay, what likeMatei Zaharia [00:41:30]: YeahSwyx [00:41:30]: what like top three differences that you always think about?Reynold Xin [00:41:33]: Governance is a big oneMatei Zaharia [00:41:34]: I think, yeah, a big one is like, yeah, security, data privacy, governance, all that stuff. So usually if you're building some kinda like B2B or developer tool, like your biggest market is gonna be enterprises, but it's just very different. A company that's existed for like, it's had some form of IT for like 30 years, they have so many legacy systems or they operate in a regulated space. whereas a startup or, even like a, like sorta more recent tech company, all the. everything is new and pristine. So yeah, it's just different, and if you've never worked with enterprises or been in one, you just won't know about it.Reynold Xin [00:42:13]: Yeah.Matei Zaharia [00:42:13]: Yeah.Reynold Xin [00:42:13]: And the procurement process is probably quite different. There's far more stakeholders.Matei Zaharia [00:42:17]: Yeah, that is one. Yeah.Matei Zaharia [00:42:18]: Another piece that's interesting is I think some tech companies, people, will say, “Oh, I can build that myself,” right? I'll just build that myself.Matei Zaharia [00:42:27]: So then you go,Reynold Xin [00:42:28]: I don't think people say that about Databricks, butMatei Zaharia [00:42:31]: yeah, it dependsReynold Xin [00:42:32]: They do.Matei Zaharia [00:42:32]: They do?Matei Zaharia [00:42:32]: Yeah, the. Yeah, and it depends on the teams and things. So, but, on the other hand, like many of the enterprises say, “I don't, I never wanna be in the business of building that.” Like, I don't want my, whatever, I'm a retailer or something, I never wannaReynold Xin [00:42:45]: Yeah, sell clothes,Matei Zaharia [00:42:46]: be down because like some weird like nerd like couldn't get streaming pipelines working.Matei Zaharia [00:42:51]: That is not what I'm doing.Reynold Xin [00:42:53]: Yeah.Reynold Xin [00:42:53]: Yeah. This makes them great customers, to be honest, right?Matei Zaharia [00:42:55]: Yeah. But you have to understand that it's hard without having worked there and stuff, like you may not appreciate.Reynold Xin [00:43:01]: Look, I think they're all great. don't get me wrong, they have different challenges. But the, many of the tech companies, for sure there's a lot, far more DIY.Matei Zaharia [00:43:10]: On the flip side, you have people who are. they're very much experts in their domain, like they're building airplanes, they're, designing medicines, whatever, and they just want to bridge the technology, where like they don't wanna learn, databases or whatever. As cool as we think it is, even as interesting as the average software engineer might think it is to read a little bit, like they just never wanna know. They just say, “I have a, giant like, matrix or whatever with my, clinical data, like how do I, how do I like cluster it or whatever?” So yeah.The Dream Engine and Rewriting the Database StackReynold Xin [00:43:40]: Yeah. That's true. Okay, so and then I wanted to build out the dream engine, vision. where does this all lead? So one of the thing we, realized maybe a couple years back is that every single database engine out there, especially on the analytics side, are a decade old. pretty much everything that have reasonable traction are about a decade old. And they all started targeting some very specific narrow use cases, and then over time it's become more and more successful. They have grown in their ambition, and then they try to support more and more use cases. But the fastest way to support those use cases tend to be hacked around the abstractions that were initially created, that were not for those use cases.Matei Zaharia [00:44:23]: Yeah.Reynold Xin [00:44:23]: And then, but you can support them more or less okay. And before it, after 10 years of organic evolution that way, it becomes a gigantic pile of s**t.Reynold Xin [00:44:31]: the. And, but that includes Databricks. And very few company or very few systems, I think, have the gut to say, let's go start from scratch. Let's go back to the drawing board and design, knowing everything we know today after a decade of workloads and probably billions in revenue, let's attempt to rewrite it from scratch and make sure it will work and it can support all of these use cases. So we started doing that, but it's a very ambitious project. by the way, you can search on Wikipedia, there's this thing called second system syndrome.Matei Zaharia [00:45:08]: Yeah, I know that. Yes.Reynold Xin [00:45:09]: Or second system effect.Matei Zaharia [00:45:11]: Every developer must know what a second syndrome is.Reynold Xin [00:45:12]: It's you built your first thing and it works out great, and the second one's bound to fail because you become too ambitious.Reynold Xin [00:45:19]: And then you ask so many requirements.Matei Zaharia [00:45:20]: Or like you think everythingReynold Xin [00:45:21]: YeahMatei Zaharia [00:45:21]: and then you're likeReynold Xin [00:45:22]: You justMatei Zaharia [00:45:22]: you're, “I'm gonna design the perfect system this time.”Reynold Xin [00:45:24]: Yeah. And it turned out it's not perfect, and then it start failing and you're too ambitious, never launch, and you get killed. The, and the engineering team that started this, they were brilliant. I think we hired some of the best database engineers, on the planet into Databricks, and they were brilliant. Thank God it's not their second system. Many of them have built more than two in the past.Matei Zaharia [00:45:44]: Ah, nice.Reynold Xin [00:45:45]: But they were still worried about this, hey, building a database engine from scratch, I think the conventional wisdom is gonna take like five years to mature. This would be a very long-term project. It could fail. I think one of the engineers jokingly said, “Hey, maybe we just call it Reynolds Stream Engine.” If we name after a founder, maybe we then may get canceled or killed. But I think they built something pretty remarkable. they went back to. They changed the way the database engines were built from a paradigm point of view. Usually when y

Hur ser Sveriges framtid ut när kriget rasar i Europa, energipriserna skenar och säkerhetsläget blir allt mer osäkert?I veckans avsnitt gästas vi av Sveriges statsminister Ulf Kristersson för ett samtal om de avgörande frågor som just nu formar både Sverige och omvärlden. Vi pratar om kriget i Ukraina, Rysslands långsiktiga ambitioner och varför utgången av konflikten kan få konsekvenser för Europas säkerhet under flera decennier framåt. Ulf berättar också om Sveriges roll i Nato, den historiska upprustningen av försvaret och varför han anser att Europa måste bli starkare och mer självständigt.Men hur väl rustat är Sverige om omvärldsläget försämras ytterligare? Vad händer med ekonomin när energi- och råvarupriser stiger, och vilka risker ser regeringen framför sig under de kommande åren?Vi går även in på kampen mot gängkriminaliteten, arbetet med att stoppa rekryteringen av unga till kriminella nätverk och vilka reformer som väntar framöver. Dessutom diskuterar vi det allt hårdare debattklimatet, sociala medier, politisk polarisering och varför Ulf menar att svensk politik blivit allt mer amerikaniserad.Ett samtal om ledarskap under press, Sveriges största utmaningar och de beslut som kan komma att forma landets framtid under lång tid framöver.Följ Ulf härLäs mer om Moderaterna härLäs mer om Framgångsakademin här.Ta del av Framgångsakademins kurser.Beställ "Mitt Framgångsår".Följ Alexander Pärleros på Instagram.Följ Alexander Pärleros på Tiktok.Bästa tipsen från avsnittet i Nyhetsbrevet. Hosted on Acast. See acast.com/privacy for more information.

Hur ser Sveriges framtid ut när kriget rasar i Europa, energipriserna skenar och säkerhetsläget blir allt mer osäkert?I veckans avsnitt gästas vi av Sveriges statsminister Ulf Kristersson för ett samtal om de avgörande frågor som just nu formar både Sverige och omvärlden. Vi pratar om kriget i Ukraina, Rysslands långsiktiga ambitioner och varför utgången av konflikten kan få konsekvenser för Europas säkerhet under flera decennier framåt. Ulf berättar också om Sveriges roll i Nato, den historiska upprustningen av försvaret och varför han anser att Europa måste bli starkare och mer självständigt.Men hur väl rustat är Sverige om omvärldsläget försämras ytterligare? Vad händer med ekonomin när energi- och råvarupriser stiger, och vilka risker ser regeringen framför sig under de kommande åren?Vi går även in på kampen mot gängkriminaliteten, arbetet med att stoppa rekryteringen av unga till kriminella nätverk och vilka reformer som väntar framöver. Dessutom diskuterar vi det allt hårdare debattklimatet, sociala medier, politisk polarisering och varför Ulf menar att svensk politik blivit allt mer amerikaniserad.Ett samtal om ledarskap under press, Sveriges största utmaningar och de beslut som kan komma att forma landets framtid under lång tid framöver.Följ Ulf härLäs mer om Moderaterna härLäs mer om Framgångsakademin här.Ta del av Framgångsakademins kurser.Beställ "Mitt Framgångsår".Följ Alexander Pärleros på Instagram.Följ Alexander Pärleros på Tiktok.Bästa tipsen från avsnittet i Nyhetsbrevet. Hosted on Acast. See acast.com/privacy for more information.

TakeawaysToby Doeden opposes any attempt to repeal HB 1052 and has promised the fastest veto in South Dakota history if a repeal ever reaches his desk as governor.On the question of a compromise threshold for eminent domain projects, Doeden is firm: when it comes to God-given constitutional rights, there is no wiggle room and no compromise.Doeden believes true economic development in South Dakota doesn't require multi-billion dollar subsidized projects. It comes from supporting local entrepreneurs, small businesses, farmers, and ranchers.Coming off a first-place finish in the June 2nd Republican primary with 31% of the vote in a four-way race, Toby Doeden sits down with Amanda Radke for a direct, issue-focused conversation on the property rights topics South Dakota landowners care about most.This is Part 1 of American Land and Legacy's two-part exclusive interview series ahead of the July 28th runoff election, where Doeden will face Interim Governor Larry Rhoden. With thousands of landowners reaching out to Amanda asking who to support, she took the question straight to the candidates themselves.The conversation covers the full range of property rights issues at stake in this race: the legacy of SB 201 and the overwhelming voter rejection of RL 21, the landmark HB 1052 that eliminated eminent domain for CO2 pipelines, the ongoing debate over data centers and tax exemptions, and how a Doeden administration would respond to federal pressure to expand eminent domain for energy infrastructure.Doeden is clear on his foundational position: every decision he makes as governor will be rooted in protecting God-given constitutional rights, and he will mirror the will of the people, not the will of lobbyists and big donors. He points to the 59% voter rejection of RL 21 across 65 of 66 counties as proof of where South Dakotans stand, and argues that the legislature's decision to push SB 201 through anyway was a direct failure to represent the people.He also addresses the economy, rural communities, and what he sees as the real differences between himself and his runoff opponent, including a 38% increase in state spending, rising crime, doubled property taxes, and a sales tax increase coming in 2027.Key Topics CoveredThe SB 201 / RL 21 Landowner Bill of Rights: what went wrong and how a Doeden administration would honor the will of South Dakota votersHB 1052 and eminent domain for CO2 pipelines: Doeden's commitment to protecting and defending this nation-leading legislationWhether a compromise threshold for pipeline projects is on the table (short answer: no)The data center debate: the 50-year tax exemption attempt versus the Data Center Bill of Rights for Citizens passed by Senator Carr and Speaker HansenPresident Trump's executive orders expanding eminent domain for energy infrastructure and how Doeden would balance federal direction with South Dakota's states' rightsWhat economic development actually looks like for South Dakota, and why it doesn't require big subsidized projectsThe challenges facing rural communities, small farms, and ranches, and who Doeden would bring to the table to address themHow Doeden's primary results reflect a statewide demand for change, and what he sees as the key differences between his vision and Larry Rhoden's recordReuniting the Republican Party after a competitive four-way primary and what that means going into the runoffhttps://www.americanlandandlegacy.org/

TakeawaysGovernor Rhoden signed HB 1052 and gave a direct, unambiguous answer when asked if he would uphold it against any attempt to amend or repeal it in the upcoming legislative session: yes.Rhoden does not support special tax exemptions or deals for data centers, but he did sign the Data Center Bill of Rights for Citizens carried by Senator Carr and Speaker Hansen, which requires data centers to prove they won't negatively impact South Dakota's water supply or electricity rates.Rhoden points to a decades-long record on property rights, including leading the charge after the 2005 Kelo v. New London Supreme Court decision, carrying the Open Fields Doctrine bill, and resolving the non-meandered bodies of water issue after 20 years of legal limbo.In Part 2 of American Land and Legacy's exclusive gubernatorial candidate interview series, Amanda Radke sits down with South Dakota Governor Larry Rhoden ahead of the July 28th runoff election.With the same questions put to both candidates, this conversation gives landowners a direct side-by-side look at where each man stands on the property rights issues that matter most to South Dakota's farmers, ranchers, and rural communities.Rhoden speaks to his record as a lifelong West River rancher and his history of working on property rights legislation going back two decades, including model legislation he carried in the wake of the Kelo v. New London decision, the Open Fields Doctrine bill he got across the finish line as lieutenant governor, and the resolution of the non-meandered bodies of water issue that had been in limbo for 20 years.On the most pressing current questions, Rhoden is direct: he signed HB 1052, he wouldn't hesitate to do it again, and he will veto any attempt to weaken or repeal it. He does not support special tax exemptions for data centers, though he's open to using existing incentive structures if facilities comply with the rules laid out in the Data Center Bill of Rights he signed. And on the question of federal pressure to expand eminent domain for energy infrastructure, he expresses confidence in the working relationship his administration has built with the Trump team while maintaining that South Dakota's own statutes already offer stronger protections than any other state in the union.The conversation also covers SB 201's complicated legacy, the local control concerns in Section 4 of that bill, the case for and against a constitutional amendment on eminent domain, rural broadband investment, international trade missions, and what Rhoden sees as the key differences between himself and Toby Doeden heading into the runoff.Key Topics CoveredHB 1052 and the compromise threshold discussion during negotiations: why it didn't get support and what Rhoden did when the bill reached his deskData centers: his opposition to special tax exemptions, his support for the Data Center Bill of Rights for Citizens, and how he views existing incentive structuresPresident Trump's executive orders on data centers and expanded eminent domain for energy infrastructure, and how Rhoden plans to navigate federal pressure while protecting South Dakota landownersHis decades-long property rights record: Kelo v. New London model legislation, the Open Fields Doctrine bill, and the non-meandered bodies of water resolutionSB 201 and RL 21: Rhoden's perspective on what the bill actually did, why the referendum process surprised him, and how Summit Carbon's loss of trust with landowners shaped the outcomeSection 4 of SB 201 and the local control concerns around PUC authority versus county and township ordinance-making powerhttps://www.americanlandandlegacy.org/

Hur förändras livet när man på riktigt inser att tiden är begränsad?I det här avsnittet möter vi coachen Johanna Hector, författare till boken Konsten att dö ung så sent som möjligt. Sedan 2005 har hon markerat varje avslutad vecka i sitt liv med en cirkel i en kalender med totalt 4 000 veckor – en ständig påminnelse om att livet inte är oändligt. För Johanna handlar det inte om att bli rädd för döden, utan om att bli modigare i livet.Vi pratar om varför hon är mer rädd för att inte ha levt fullt ut än för att dö, och hur medvetenheten om livets ändlighet kan hjälpa oss att släppa självkritik, våga ta större plats och fokusera på det som faktiskt spelar roll. Johanna ifrågasätter också vår tids starka fokus på ”self love” och idén om att vi ständigt måste stärka vårt ego. Istället beskriver hon hur frihet, utveckling och mening ofta uppstår när vi vågar släppa taget om bilden av vilka vi tror att vi måste vara.Samtalet rör sig vidare in på posttraumatisk tillväxt – hur livets svåraste perioder ibland kan bli källan till vår största utveckling. Johanna berättar öppet om sin ätstörning, sin pappas sjukdom och död, och hur dessa erfarenheter har gett henne en djupare tacksamhet och ödmjukhet till livet. Vi pratar också om föräldraskap, motgångar och varför hon vill lära sina döttrar att de är starkare än de tror.Det här är ett samtal om att ta tillvara på livet, om modet att följa sina drömmar och om konsten att inte skjuta upp det som verkligen betyder något.Följ Johanna härLäs mer om Johanna härKöp boken här Läs mer om Framgångsakademin här.Ta del av Framgångsakademins kurser.Beställ "Mitt Framgångsår".Följ Alexander Pärleros på Instagram.Följ Alexander Pärleros på Tiktok.Bästa tipsen från avsnittet i Nyhetsbrevet. Hosted on Acast. See acast.com/privacy for more information.

Hur förändras livet när man på riktigt inser att tiden är begränsad?I det här avsnittet möter vi coachen Johanna Hector, författare till boken Konsten att dö ung så sent som möjligt. Sedan 2005 har hon markerat varje avslutad vecka i sitt liv med en cirkel i en kalender med totalt 4 000 veckor – en ständig påminnelse om att livet inte är oändligt. För Johanna handlar det inte om att bli rädd för döden, utan om att bli modigare i livet.Vi pratar om varför hon är mer rädd för att inte ha levt fullt ut än för att dö, och hur medvetenheten om livets ändlighet kan hjälpa oss att släppa självkritik, våga ta större plats och fokusera på det som faktiskt spelar roll. Johanna ifrågasätter också vår tids starka fokus på ”self love” och idén om att vi ständigt måste stärka vårt ego. Istället beskriver hon hur frihet, utveckling och mening ofta uppstår när vi vågar släppa taget om bilden av vilka vi tror att vi måste vara.Samtalet rör sig vidare in på posttraumatisk tillväxt – hur livets svåraste perioder ibland kan bli källan till vår största utveckling. Johanna berättar öppet om sin ätstörning, sin pappas sjukdom och död, och hur dessa erfarenheter har gett henne en djupare tacksamhet och ödmjukhet till livet. Vi pratar också om föräldraskap, motgångar och varför hon vill lära sina döttrar att de är starkare än de tror.Det här är ett samtal om att ta tillvara på livet, om modet att följa sina drömmar och om konsten att inte skjuta upp det som verkligen betyder något.Följ Johanna härLäs mer om Johanna härKöp boken här Läs mer om Framgångsakademin här.Ta del av Framgångsakademins kurser.Beställ "Mitt Framgångsår".Följ Alexander Pärleros på Instagram.Följ Alexander Pärleros på Tiktok.Bästa tipsen från avsnittet i Nyhetsbrevet. Hosted on Acast. See acast.com/privacy for more information.

This episode is part of our special series on the India AI Impact Summit, examining the conversations, decisions, and debates that are shaping global AI governance. Professor Ravindran addresses early on the perception that the India summit sidelined safety. More than 60% of the summit's events and discussions were focused on safety, trust, and cross-border collaboration. The framing shifted, and deliberately so. When the summit came to the Global South, leading with existential risk, rather than the very real opportunity AI presents to improve healthcare, education, and public services for hundreds of millions of people, would have been the wrong entry point. The two key deliverables from his working group reflect that balance: the Trusted AI Commons, a repository of benchmarks, testing protocols, and best practices designed for AI deployment in resource-constrained settings, and a high-level governance guidance note endorsed by 22 countries, that calls out the issues every national AI policy should address without being prescriptive enough to limit how different countries approach it. On frontier risks, Professor Ravindran notes that the landscape has shifted in ways that would have seemed speculative even a year ago, and that the frameworks being built to manage these risks will need to keep pace with that change. He also reflects on what the growing concentration of the most capable AI models means for countries like India, and why that conversation may need to move from being a company-to-country dialogue to a country-to-country one. His overall view is one of cautious optimism: there will be disruption in the short term, but there will also be a new equilibrium, and the work is to make sure the transition is managed well.Episode Contributors Professor Balaraman Ravindran heads the Department of Data Science and AI at IIT Madras. He is also the Founding Head of the Wadhwani School of Data Science and AI (WSAI), Robert Bosch Centre for Data Science and AI (RBCDSAI), and Centre for Responsible AI (CeRAI) at IIT Madras. He has more than three decades of experience working in reinforcement learning, and his research interest spans responsible AI and deep RL.  Nidhi Singh is an associate fellow at Carnegie India. Her current research interests include data governance, artificial intelligence and emerging technologies. Her work focuses on the implications of information technology law and policy from a Global Majority and Asian perspective. She has previously contributed to the Indian Express, The Secretariat, Medianama and HinduBusiness Line. Every two weeks, Interpreting India brings you diverse voices from India and around the world to explore the critical questions shaping the nation's future. We delve into how technology, the economy, and foreign policy intertwine to influence India's relationship with the global stage.As a Carnegie India production, hosted by Carnegie scholars, Interpreting India, a Carnegie India production, provides insightful perspectives and cutting-edge by tackling the defining questions that chart India's course through the next decade.Stay tuned for thought-provoking discussions, expert insights, and a deeper understanding of India's place in the world.Don't forget to subscribe, share, and leave a review to join the conversation and be part of Interpreting India's journey.

Last 4 days before regular tickets sell out at AI Engineer World's Fair - this is the single biggest gathering of AI Engineers, Founders, Leaders, and Researchers in the world. Attendees get >$5000 worth of sponsor credits and talk tracks are looking FANTASTIC. Join us!The AI scaling debate always focuses on the question of “how do we get more GPUs?” but the better question may be: how do we make the most of ones we already have.The fact that a frontier lab like xAI could be running at sub-10% MFU (Model FLOPs Utilization) is just a hint at what the real problem may be.For context, older frontier-scale training runs were already much higher than 10%. GPT-3 was around 21% MFU. Gopher was around 32%. Megatron-Turing NLG was around 30%. PaLM reached around 46%. And our guest Anjney says best-in-class MFU today is closer to 60–70%.It's not necessarily that xAI is uniquely incompetent (it's clear they have talented folks) but rather the priorities may be flipped in the GPU arms race.While GPU access is a bottleneck, simply increasing CapEx won't automatically translate to better models as frontier AI is increasingly a systems problem: scheduling, utilization, networking, kernels, frameworks, data pipelines, parallelism, cluster reliability, and the thousand small decisions that determine whether your theoretical FLOPs become real training progress.From building Discord's developer platform and backing frontier AI companies like Anthropic, Mistral, Black Forest Labs, and Periodic Labs to now building AMP's independent compute grid, Anjney Midha has spent years close to the real bottlenecks of AI scaling. In this episode, Anjney joins swyx at Periodic Labs to unpack why the AI race is not just about buying more GPUs, why 95% utilization would have been considered an outage at Google, and why the next era of AI infrastructure has to be more aligned, more efficient, and more responsible.We go deep on AMP's vision for a compute grid that makes FLOPs flow like megawatts, the difference between full-stack AI labs and horizontal pooling, why AI data centers need community buy-in, and how compute markets could evolve into something closer to an independent system operator. Anjney also explains why DeepMind's unpublished research points to a market failure, why end-of-life prediction remains one of the most important AI applications he has thought about for fourteen years, and why “output maxing” may become a new discipline for frontier systems.We also discuss Anthropic's culture, why “luck favors the prepared mind” in coding models, how Claude cracked coding, why too much capital too early can make AI labs fragile, what Periodic Labs is trying to do with science and superconductors, why great researchers can become great CEOs, and why Silicon Valley is both deeply missionary and deeply mercenary.We discuss:* Why 95% utilization was considered an outage at Google* Why AI infrastructure waste compounds at frontier-lab scale* Why “move fast and break things” does not work for AI data centers* How data center backlash, power grids, and community incentives shape AI scaling* AMP's vision for making FLOPs flow like megawatts* Why compute needs an independent system operator* How interruptible demand and dynamic prioritization worked inside Google* Why DeepMind research hoarding creates negative externalities* AMP's 1.2GW base-load ambition and the need for 6GW of spike capacity* Why end-of-life prediction could become one of AI's most important healthcare applications* Frontier Systems, output maxing, and full-stack alignment* Why APIs and abstraction layers become lossy as organizations scale* Superconductors, standards, and the dream of lossless systems* SF Compute, open protocols, and the future of compute marketplaces* Why non-NVIDIA chips can still benefit from NVIDIA's reference architecture* Trust boundaries and why chip startups need visibility into future model architectures* Why VCs often underestimate researchers as CEOs* Scientists as star athletes of the mind* Why great CEOs need to be confrontational up and down the stack* Why leading the frontier matters more than “winning”* How Anthropic cracked coding* Why culture is fragile, not a permanent moat* Why hardship was a feature, not a bug, for Anthropic* Why Anthropic's P0 was coding from day one* Periodic Labs, physics as the constraint, and technical reality* Silicon Valley mercenaries, missionary teams, and what happens after a breakthroughAnjney Midha* LinkedIn: https://www.linkedin.com/in/anjney* X: https://x.com/AnjneyMidhaAMP PBC* Website: https://amppublic.com/* X: https://x.com/amppublicTimestamps00:00:00 Introduction00:00:09 Why AI Compute Is Being Wasted00:03:17 Responsible Infrastructure and Data Center Backlash00:06:07 AMP Grid: Making FLOPs Flow Like Megawatts00:12:41 Foundry, Frontier Labs, and Research Hoarding00:14:42 Gigawatt-Scale Compute and End-of-Life Prediction00:24:08 Frontier Systems, Output Maxing, and Alignment00:27:38 Compute Markets, SF Compute, and Non-NVIDIA Chips00:32:57 Trust Boundaries, Co-Design, and Researcher CEOs00:38:17 AI Coachella and First-Principles Thinking00:42:43 Leading vs Winning in Frontier AI00:45:54 How Anthropic Cracked Coding00:48:25 Culture, Hardship, and Anthropic's P000:54:03 Periodic Labs, Physics, and Silicon Valley Mercenaries00:56:26 Rishi Valley, Singapore, and Money as a Measure00:58:47 Closing ThoughtsTranscriptIntroduction: Anjney Midha, AMP, and Compute WasteSwyx [00:00:00]: We're in Periodic Labs with Anjney Midha, CEO, founder of AMP. Welcome.Compute Utilization: Node Allocation, MFU, and AlignmentAnjney [00:00:09]: Thanks for having me. At Google, there are two types of utilization usually, right? That you're measuring in these clusters. One is node allocation, and then the other's MFU. Node utilization is usually like what percentage of cards in the data center are just, used, and that, if it's not at, 95%-Swyx [00:00:29]: There is no excuseAnjney [00:00:29]: There's no excuse, right? I think 95% at Google, which is where my co-founder, Seb, came from, he built the Borg, PBorg/GQM scheduler at Google, and there I think 95% was considered an outage, so 96% node utilization is, should be standard. And most single-tenant clusters are not running at that. So that's one. And then MFU should be, I would say the best in class today is somewhere between 60 and 70%. I think this is a leadership question, right? Fundamentally it's an alignment question, which is are the people who are funding the cluster and then deploying the cluster actually aligned? And sometimes theoretically they are, but in practice the number of people in the chain, the supply chain between, the capital and all the way to whoever's managing the cluster and then whoever's measuring what the output is, are just so many, degrees of separation away that, the, The Have you ever heard the radian metaphor, which is at the beginning of an arc, if you have two arcs that are two lines that are just off by a few degrees, that-Swyx [00:01:33]: It spreads outAnjney [00:01:34]: It spreads out, right? Or at scale. And I think what's happening is a lot of cluster implementations and infrastructure, a lot of frontier labs and other teams, that's what's happening, is they're, they initialize the plan, which is kind of like North Star with a team that wants to do good, but then they're, required to scale so fast instead of iteratively that the wastage just compounds really fast at scale. And so I think we know the answer, which is just do iterative bring ups. If you spend time with people who've been in the semiconductor industry or the DSN industry for a long time, this is not new, and I don't think AI should be an excuse. Sure. Something What is new? Okay. We have a lot of new capabilities, but that doesn't mean just abandon common sense. Common sense should always be in fashion. ? AI scaling doesn't change the in fact, if anything, AI scaling should be putting a premium on the value of common sense and infrastructure because the margin of error now is so much lower and the costs of wastage are so much higher. And the cost of wastage, by the way, is not just economic. I'm, obviously I'm, I'm an investor, or I'm an investor by background. Over the last few years now we're running an AI infrastructure business called, AMP. And I think that it's okay to say this time is different on the capabilities front. We are genuinely getting capabilities at, of the, of a kind we haven't had before. That doesn't give you an excuse to say this time is different for everything, especially infrastructure. So look, I love the hacker mindset and the hustler mindset. Now, that's great for the startup mindset, but you remember this moment where Zuck went from saying, “Move fast, break things” to, move-Responsible Infrastructure and Data Center BacklashSwyx [00:03:10]: Fast and stable infrastructureAnjney [00:03:11]: Move fast with stable infrastructure. I think now we need to move fast with, responsible infrastructure. People are going to ask where the impact is. There was a really In our class yesterday, Scott Nolan, who's the founder of General Matter, came by at Stanford to speak about energy bottlenecks. And he had a phenomenal idea. He said, “if you look at the marginal unit economics of compute per hour,” he goes, “let's call it, $4 an hour. If you're having to bring up a new data center in a new community, why not just say we're going to charge 4.50 an hour, and that marginal impact or that marginal increase, we just literally take that and give it to the local community as cash?” I can tell you as a customer of that compute, I would love that. I'd be happy to pay an additional 50 cents per hour at scale.Swyx [00:03:57]: Wow. Yeah.Anjney [00:03:58]: Because if that means the public benefit is so clear to the communities that the data centers are coming up in, I'm going to feel like that compute is much more reliable. Up to 20% of all data centers this year in the US, my understanding is are at risk.Swyx [00:04:13]: Of community backlash?Anjney [00:04:14]: Correct. Of not getting the community support they need to get brought up.Swyx [00:04:19]: Wow. That's a huge number.Anjney [00:04:20]: Yeah. Now, we, I think we should dig into what that number is. I think it's a little bit of overstated. These things can get over-reported, but it-Swyx [00:04:27]: They don't just care about jobs. They care about all the other stuff around it, right? They care about power grid, they care about environments-Anjney [00:04:33]: Power grid, permitting, and so on. And imagine I think if you said there's a new AI deal. If we're bringing up a data center in your community, we're actually going to reduce the cost of your electricity bill. Okay, now we're talking. Right? The community's going, “Okay. Now this is a deal. I feel like a partner in this.” Right now that's not happening. There will be audits, there will be investigations, and when the, when the regulators come, I don't know when it's going to be, the folks who are moving fast and breaking things in the name of AI progress better be prepared. That's certainly not how we're procuring compute. Or we're, we're trying as much as we can to work with partners who have long-term track records. Many of whom, by the way, are not, AI providers. I think this whole idea of neoclouds being somehow this new category is a lot of marketing speak. There are really good, reliable, trusted data center providers in America who've been around 20 plus years. I love those folks. They know how to Sure. Are they sponsoring happy hours at NeurIPS? No. Are they legibly listed in Build? No. Are they hanging out in my, in, situational awareness parties? No. But they're adults. I trust them.Swyx [00:05:44]: They can run LAN. They can run power.Anjney [00:05:45]: They can run LAN, power, and shell. They have credit histories. We sit down, we have a conversations. Many of them live in Silicon Valley. They've, they've had to deal with the boom and bust cycles of the internet, and I love those folks. They are stable infrastructure partners and thinkers. And I think there's a lot of short-term thinking going on in the compute layer, and it's going to catch up to us. It's not going to be good.AMP Grid: Making FLOPs Flow Like MegawattsSwyx [00:06:07]: You talk about aligning incentives, and, I would think that aligning incentives means you have the full stack in one company, which is xAI and OpenAI, right? So you as a standalone infrastructure layer, why are you somehow more aligned to your portfolio companies than people who just own the whole thing?Anjney [00:06:28]: In systems design, right, there's, there's two regimes of, architecture, right? You have integration, and then you have pooling and utilization, right? So the Or rather, the way to increase utilization often is you can do systems integration where you collapse a lot of process into one node, or you can pull out a process from a node and share that amongst various That resource amongst several different nodes. And so we see the AMP grid, which is, the, what, the system we're building here, which is basically a compute grid. We're trying to do for compute what the electric grid-Swyx [00:07:02]: PowerAnjney [00:07:02]: Yeah, what the power grid did for electricity. It-- this is a pooling and utilization layer across clouds, And so we're actually the opposite of a full stack integration like approach.Swyx [00:07:12]: Super horizontal.Anjney [00:07:13]: Where it's much more horizontal and it's, it's multi-cloud, it's multi-silicon. The goal is to try to make FLOPs flow like megawatts, and that is very hard to do today for many reasons. There's stranded pools of compute all over the place and there's no fungibility. And so right now we do it at the level of scheduling, and we often do it at the economic layer. But as we start to announce what we're working on, it's extraordinary like how many folks are coming out of the woodworks and saying, “Hey, I'm actually working on a way to make compute fungible at this part of the stack and that part of the stack.” And as a grid, we'd like all of these folks to participate on the grid. There's, people often ask me, “Andra, are you a new cloud?” And I go, “No, actually neoclouds are suppliers.” sometimes they'll ask, “Are you a venture capital firm?” I go, “No, actually they are, they are demand like sort of off-takers of the grid.” We see ourselves as what's called an independent system operator. So if you study the history of the electric grid, once it became legible to a lot of factories and industrial sort of participants that, hey, actually it turns out pooling is a good idea. We should pool our generators instead of all having a generator running at half capacity in our backyard. There was a need for an independent entity who could coordinate all these parties. Transmission line, power generation, facilities, transmission lines, factories, and that neutral coordination mechanism is very critical. In order-- If you study like the history of grids, the most enduring ones were those that never owned their own assets. They were ones that had, or often started with long-term anchors who are uncorrelated sources of demand, a steel factory, a shoe mill or whatever in a particular town who weren't competitive, where the steel factory want to spike up at night, the shoe mill wanted to spike up during the day. So then you pool and you share, right? So each of you is guaranteed some base load, but then you kind of schedule your spikes to drive a peak utilization across the town. The gold standard, so to speak, historically, has been these utility companies like PJM Interconnect in the northeast of America, where they, over many years became this what's called an ISO, an independent system operator of the grid. So that's how we see ourselves. Economically, that's what we are. From a technical perspective, we started at the scheduling layer because Seb and Mihai, who, run engineering here, built that at-Swyx [00:09:28]: Did your schedulingAnjney [00:09:28]: They did that at Google. And, -Swyx [00:09:32]: And you have infra shops from Discord as well.Anjney [00:09:35]: I have some.Swyx [00:09:35]: I don't know, I don't know if Discord is like the primary identity, but what-whatever, I'm just kind of-Anjney [00:09:39]: No, D-Discord was-Swyx [00:09:40]: Choosing a well-known name.Anjney [00:09:42]: Well, I So I was running the developer platform there. The internal infrastructure I was not responsible for. That was actually a guy by the name of Mark Smith, who was extraordinary. And yes, Discord did pool So Discord is actually a counter example. I had the chance to learn a lot about fully, full stack infra there because-Swyx [00:09:56]: It's the same thing, yeahAnjney [00:09:57]: It's the, it's the other architecture which is, Discord built its own WebRTC vo-voice and video infra. So like Discord did not use-Swyx [00:10:08]: For the calls, yeah.Anjney [00:10:09]: Yeah, did not For communication, Discord did not use third party infra. It was all built in-house. And then the way you maximize utilization was you pool demand from the world's 200 million plus monthly active gamers, right? And so that's, that's how those stacks were constructed. Again, in systems design, the two concepts that keep coming up over and over again are abstraction and composition, right? And-Swyx [00:10:31]: Bundling and unbundlingAnjney [00:10:33]: Bundling and unbundling, abstraction, composition, like verticalization and-Swyx [00:10:36]: HorizontalAnjney [00:10:36]: Horizontalization. So in that sense, AMP is an independent system operator of the grid. We pool demand, we pool supply from a number of partners we trust At about 1.3 gigawatt scale over four years. And then we pool demand from some of the world's best, research labs and so on. We're sitting at one, periodic labs who need extraordinary long-term demand. And the idea is that, each of them is guaranteed base load on the grid, but they can spike up and down flexibly on, for compute, with much shorter timelines as needed. That was roughly the design of the program I came up with at a16z called Oxygen. The same-- That was the same design of the GQM, BorgX, Borg GQM implementation at Google that Mihai and Seb had built. Which was that how do you allow, teams inside of Google, on the internal infrastructure to be guaranteed capacity, for their base workloads? But when they need to spike up on research, how could they ensure that was sufficiently there? And of course, the big innovation that was not discovered, but kind of implemented in the space, this infra space maybe three, four years ago at Google was the idea of interruptible demand, right? Where you just queue up a bunch of jobs and through this like sort of credit system, there can be a bidding mechanism.Swyx [00:11:53]: Like priorities.Anjney [00:11:54]: It's a dynamic prioritization Basically. And jobs can get interrupted based on somebody else who's saying, “what? I have 10 tokens, 10 credits I want to spend on this job.” Another like team lead, research lead is “Genie 3 or whatever is only worth five, credits, and NanoBanana2 is worth 10 credits,” and so the NanoBanana job gets priority. That's a, that's a made up example.Swyx [00:12:15]: It's very real. Brain Marketplace was real. And, we've, we've covered this on the pod with David Luan, who was-Anjney [00:12:20]: Oh, great. OkaySwyx [00:12:20]: Was there. And the criticism is that, well, actually sometimes you need central command to go all in on a thing. And actually sometimes capitalism via credits doesn't work. Not, this is not a criticism of AMP. I'm just saying, this is a thing that has been tried, internally within Google, and it led to Google missing GPT.Foundry, Frontier Labs, and Research HoardingAnjney [00:12:41]: Like, we structured ourself essentially very similarly to Google. We are structured as a holdings company. So, Alphabet holdings is Alphabet holdings, and then they've got these subsidiaries called Google and-Swyx [00:12:51]: Other betsAnjney [00:12:52]: Other bets and so on. We've got, AMP holdings, and we've got our infrastructure business, and then we've got a capital business called Foundry that incubates new frontier AI labs or invests in them as venture capital, like Periodic. We put a few hundred million dollars into Anthropic from our fund earlier this year. So wherever we feel like teams are making progress, especially researchers and so on who've pushed the frontier inside of existing labs like DeepMind, I find, there comes a point where they feel misaligned with the dictatorship of Alphabet holdings. And at that point, sometimes the dictatorship doesn't want them anymore. And they're “Thank you. You've done your job here. You've kind of helped us through the zero to one phase, and for whatever reason, we're going to deprioritize your amazing, omni model or whatever it is, and instead we're going to prioritize coding.” And, I think that's a tragedy, but I get it. They're Sergey and team are running their own business there. But that doesn't mean we the rest of us should sit around waiting for that progress to get unlocked for the rest of the world and humanity. If you think about how much extraordinary research has happened inside of DeepMind over the last 10 years, I, Demis and Sergey and those guys did such a great job. But at the end of the day, so much of that has never seen the light of day?Swyx [00:14:00]: Or they're like papers only, but they never actually shipped it to production or-Anjney [00:14:03]: What's worse is the paper is actually not even being published anymore ‘cause there's a six-month embargo inside of DeepMind, right? We've heard about this where a paper comes out, and then I think there's a six-month embargo window where if anybody on the business team says, “This could be interesting” It's embargoed for life.Swyx [00:14:18]: Exactly. So the stuff that gets published is the stuff that's not good enough.Anjney [00:14:21]: There's an adverse selection problem, basically. Yeah. At this point-Swyx [00:14:25]: It's, it's a common complaint at NeurIPS, by the way, that's “Well, why would I look at the papers that are the trash of GDM?”Anjney [00:14:31]: Again, I think it's a tragedy. I get it. They're running their business, but the rest of the I think there's negative externalities of research being hoarded, and so that'there's a market failure. And somebody needs to unlock that research, and we can't do it on our own. We only have 1.2 gigawatts of compute. That's nothing. That's about $40 billion of cloud spend. We're going to need a lot-Gigawatt-Scale Compute and End-of-Life PredictionSwyx [00:14:51]: By the way, is that's a new number. I haven't, haven't come across that gigawatt number. That's huge.Anjney [00:14:56]: Yeah. And to be clear, we haven't secured all of it. That's how much demand we have started to secure. I think publicly we haven't actually confirmed how much we have for this year. In order-Swyx [00:15:04]: Where do you want to get to?Anjney [00:15:06]: I think the steady state would be that we have a base load pool Of 1.2 gigawatts at all times Of base load capacity. For spike capacity, right now my estimate is we need roughly six gigawatts over the next four years for all our teams to feel like they were able to keep moving the frontier, whatever they're working on, whether it's, like superconductor discovery over here. There's a new investment we're working on right now, which is in the end of life prediction space in healthcare. It's extraordinary how much you can, you can give this was actually my graduate school work. I went to grad school for bioinformatics at Stanford Med. And I know we-Swyx [00:15:40]: Econ, MCS, bio.Anjney [00:15:41]: So my-- I was this really weird cat where, I was never satisfied with my major options. So at one point I was an econ major, then I was a CS major, then I was a MCS major called mathematical computational science, and they decided they were going to end that major. So I took all that coursework, and I applied it to grad school, my graduate degree in bioinformatics, which was the master's program, and then I thought I was going to do a PhD. I never ended up doing it. I dropped out and went to work at Kleiner. But I was lucky enough to apprentice with this professor at, Stanford Med. His name is Nigam Shah, and he was working on end of life prediction. Stanford is one of the only research facilities in America that has a longitudinal patient data set that's larger at scale. I think it's at least 12 million patient lives. The only larger data set is at the VA, the Veterans Affairs, of America. And to do research, like do any deep learning and so on that data set, it was called the STRIDE data set at that time, you had to be a Stanford Med School affiliate, which is why I went and enrolled in the bioinformatics department. End of deep learning was early. Nigam Shah had the visibility-- the vision to see that, you could do end of life prediction to help palliative care. In America, the, over 30% of all Medicare, Medicaid spend, at least at that time, was spent on end of life care. And what's we grew up in Asia, so we all-- Yeah, at least I won't speak for you, but I have A very different relationship with death than I find folks who grew up in America do. In America, spiritually and culturally, especially in Western societies where Christianity, the Christian tradition sort of frames death as this terminal point, there's often a judgment day and so on. The way we view death is with a finality. In Indian culture, in Hindu culture, death is one-Swyx [00:17:35]: Also, he's Buddhist as well.Anjney [00:17:36]: You're Buddhist, yeah. So it's one, it's one step in a journey of many lives, right? And so, I grew up in this city called Chennai in the south of India, and when people die, you dance on the street. There's like a procession where your body is carried to be cremated and your family, like celebrates and there's drums and so on. It's this huge thing. And, It's because the idea is that you're going to be reincarnated. You've been liberated from the responsibilities of this life, and now you're onto your next. It's a new It's like going off to a new college or whatever, right? And so it was so alien to me when I got here as an undergrad- That the medical system works backwards from that assumption that we have to view death as this terminal thing and delay it, postpone it's a bad thing. And so at the time, clinical decision support in the United States was this very primitive field. Even to this day, physicians in the United States often will tell you when you have a terminal disease, this is your, we've diagnosed you, which is great. Our ability to diagnose you is extraordinary. You have somewhere between six months to six years to live. What do you do with that information? The error bars are so high that then you In times of uncertainty, we default to culture, and when the culture is let's-- this is a bad thing, I've got to prolong my life, then you start doing things like And just to, just sort of from a systems perspective, what's going on there is Physicians often feel like they need to provide such high error bars because there's always some uncertainty in end of life diagnosis, and if you provide the wrong Diagnosis or recommendation to your patient, you can be sued for medical malpractice. And then your license can be taken away. It can be catastrophic for your career. In contrast, if in countries where that's not the case, what you often observe is that patients, physicians are quite prescriptive with their recommendation. They say, “Hey, this is your condition. The literature says that you probably have this much time on Earth left. My expert opinion is that you are an outlier or whatever.” And they try to be more prescriptive, and that empowers a patient, right? ‘Cause then a patient can say, “I trust my doctor. They said on average, I have six months to live, but if I do these things, I may have a shot because of my particular predispositions or my genetic history or whatever.” And that empowers you to go about your life in a actually more scientific way than leaning on religion, culture, spirituality, and so on. In contrast, here, because of that medical malpractice sort of thing looming over your head, a physician never gives you a clear recommendation. So instead you say, “Okay, Doc, well, let's try it all.” And then you start a whole regime of drugs and therapies, and then you often spend weeks and weeks in the hospital, and that deteriorates your quality of life. And when that deteriorates your quality of life, you instead of spending your last few days doing the things you love with your family, you're spending it on a hospital bed. And that ends up being thirty percent of Medicare and Medicaid. So it's worse for the patients. The doctors feel terrible. The American taxpayer is paying a huge amount of money. And so this is why Nigam Shah, who was this professor at Stanford, said, “Anjney, if there's “ I kind of sat down with him. I was this young, I'd, I was twenty-one, and I was “I want to work on a big problem.” He's “The big problem is end of life care.” And so we tried to do deep learning to say, to-- So we started trying to run deep learning on these tried patient data sets to say, “Could you have an AI system make a recommendation that is orders of magnitude more precise about how much time you have left once you've been diagnosed with a terminal condition than a human?” And then if we can get that precision to be high enough, then you can empower the patient. And it turns out the tech works. Like it's-- Once you get the data set, like RL works. Honestly, even regression models work. You don't need to get that fancy. At the time, we were just trying, doing like very simple neural nets.Swyx [00:21:54]: Simple solutions, yeah.Anjney [00:21:54]: Today, what we can do with RL is extraordinary. The problem remains then and now is regulatory, because you actually can't shift the burden of the wrong clinical diagnoses from the physician to the AI system. And so at that time, I got quite disillusioned ten years ago for, twelve years ago where, ‘cause I felt I just didn't have the resources to influence regulation. Today, I'm very lucky. I'm in a different place. I've, I'm a lot older, and so I've been spending a lot of time on my next incubation, which is how can we unlock the, patient empowerment by training AI models to do end of life prediction much, with much more precision and ac-Swyx [00:22:37]: Oh, wow. You're still focused on this the whole time.Anjney [00:22:40]: The-- I haven't been able to get, this out of my mind a single day for the last fourteen years. This is the hill I want, I would like to die on. There's two, I would say. What? I actually, I'd prefer not to die.Swyx [00:22:51]: Yeah, exactly.Anjney [00:22:52]: But I think two bipartisan issues, I think two issues that should be bipartisan in America are how do we empower patients to make the right clinical decisions at the end of their life, such that we're reducing the taxpayer burden with science? It's just good old science, and AI can help here. And the second is, net positive data centers, ‘cause I think that's the biggest critical bottleneck on training and good enough AI models to help people at the end of their life. So there's sort of two sides of the, of the same scaling bottleneck curve, but those two, we formed AMP as a public benefit corporation. My wife and I, who you've met, you've met Viv. Her passion is education. Her family is a long line of educators and so on, and, of physicists. And so this class is my attempt to stop being the black sheep of the family and be a, an educator. But if I'm not educating, the thing I would be doing is working, on these two problems, whether on the political spectrum or as a researcher back at, in some lab. And my hope is if anyone's listening to this podcast, if they're passionate about either of those two topics, I'd love to hear from them. We'll, we'll we can share the contact in the show notes, but, we're looking for people to join both of those missions on the, on the political side as well as on the medical side, on the research side.Frontier Systems, Output Maxing, and AlignmentSwyx [00:24:08]: You said, this is a discipline that you want to form. You call it's called variously called Frontier System. It's variously called One Person Frontier Lab. What is the ideal name or shape of this? Like the, what is the mission?Anjney [00:24:24]: Of the class?Swyx [00:24:26]: Of the discipline that you're, exploring, right? I The class is called Frontier Systems. But like for me, maybe one phrase is you're, you're just anti-waste, right? Which is wasting GPUs, wasting in human and Medicare. But is there, is there a broader theme that I'm, that maybe you can encapsulate more succinctly?Anjney [00:24:45]: Yeah. The, from an engineering perspective, it's very simple. It's output maxing. It's the, it's the department of output maxing.Swyx [00:24:51]: Making the most of what we have.Anjney [00:24:52]: Exactly. I'm a huge believer in optimal outcomes. I think both in America and other countries, we are losing our appreciation for nuance, and this is the thing of And AI is the same case, right? Oh, the bitter lesson holds. Okay, fine. But that doesn't mean you just like throw 500 GB300, 500,000 GB300s at your suboptimal model scaling and you waste a bunch of compute. It also doesn't mean that, the most optimal is to have like 50 different architectures where there isn't enough standardization. One of the reasons Anthropic has had extraordinary sort of velocity is ‘cause they picked the transform architecture and said, “This is simple. Let's double down on it,” right? And now luckily there's enough investment going to the space that we can afford other architectures, but at the time, investment was just too fragmented into other architectures, so that arguably unlocked scaling. So I think there's a philosophy. I think we all owe it to ourselves to do output maxing with a new capability called AI on a global level. I think if I was starting a new department at Stanford, depending on how fuzzy or technical I wanted to be, I'd probably call it the Department of Alignment. Like-Swyx [00:25:59]: It's an overloaded termAnjney [00:26:01]: But it is, But alignment really Is a hard problem. And I think when you unlock it, full stack alignment is super hard in any organization and in any system. Like in a, in a venture capital firm, if you can have full stack alignment between your limited partners and your, the founders who are creating the value and ultimately the public that owns the IPO stock, that is a gift that keeps giving. And when you study the history of these systems, when they start off, they usually start out small scale where the feedback loop is actually so tight that there's alignment. And then the more you try to scale, the more division of labor happens, the more specialization happens, and at each step you add abstractions. And wherever there's an API interface, there's like loss. There's communication loss. And so I think a really cool thing would be for us to figure out is there a way for us to have our cake and eat it too as an engineering discipline? Is there a way to actually scale up and scale out Without losing any alignment, without lossy transmission?Swyx [00:27:01]: You mean standards?Anjney [00:27:02]: So standards is one way. The other way is you just have net new capabilities. So like what we're trying to do here is discover new superconductors. A room temperature superconductor would be a lossless transmission mechanism for energy. We would have flying cars. We are right within a few years of having a new room temperature superconductor. So I think those are the two. You either have to standardize On protocols or API specs that allow lossless communication, or you can come up with a whole new capability that unlocks so much abundance, the standardization doesn't matter ‘cause you just unlock net new capacity. This, the, so this is what I spend my days thinking about these days.Compute Markets, SF Compute, and Non-NVIDIA ChipsSwyx [00:27:38]: No, I think every infra person at, who wants scale and wants to output max does eventually end up thinking about this. We don't have time to go into it, but we have done an episode with SF Compute-Anjney [00:27:50]: Oh, coolSwyx [00:27:50]: That is trying to standardize The futures contract for compute. I don't, I don't know how that's going by the way, but like at some point this will be public.Anjney [00:27:57]: Oh, I think Evan is awesome and SF Compute is the kind of effort that I hope we can accelerate because what often happens is these exchanges are very hard to get, they, it's hard to bootstrap them, right? Because they often require-- There's many inefficiencies between parties. There's trust boundary inefficiencies in infrastructure because you don't trust, one part of the stack doesn't trust another part of the stack to give them visibility. There's capital markets inefficiencies, there's operational efficiencies. So if you can inject like a single shock to the system of a ton of compute demand or supply, then you can accelerate, these new flywheels. And so my hope is one day, or soon, if SF Compute needs extra like has excess capacity, they just hook it up to the grid and they get flooded with demand from us. And on the other side, if they have a ton of demand but they don't have supply, they just again hook up to the grid and it's a two-way protocol where they can just hook up to our capacity. And I don't think we're too far from that. Today our working implementation of it is mostly through a group of labs, universities, and a few sort of trusted parties who are, who all feel like they're in alignment to borrow an over sort of used word. But our hope is to just have it be an open protocol that anyone can hook up to on-Swyx [00:29:20]: Hook up for demand or hook up for supply? In primarily demand, it sounds like. Like you-Anjney [00:29:25]: No, bothSwyx [00:29:26]: You would want to offer demand.Anjney [00:29:27]: Both. Yeah. Unfortunately, what's happened in the last six weeks is, we thought we'd have a bunch of excess capacity by the end of this year. It's all gone.Swyx [00:29:37]: It's exploding.Anjney [00:29:38]: It, yeah. It's all gone. And so I have, my text messages are full of friends, we know many of these people, these are founders who've raised billions of dollars in San Francisco going, “Oh, any chance you have like 50 nodes in the next few weeks?”Swyx [00:29:51]: What is the scope for, non-Nvidia, right? You have Lisa Su coming and, Rainer Pope as well. And so There is a lot of demand for, more performance Alternative architectures and all that. At the same time, this hurts your standardization.Anjney [00:30:11]: I don't think so. So actually Rainer's a great example, right? Rainer is a CEO and founder of, MatX. I actually had him by for office hours in the class earlier today, and there was an insight he brought up that I hadn't considered before, which is when they decided to pick the standard For their data center, they picked the NVIDIA reference architecture. So the MatX chips Just plug in to any site that has an NVIDIA bring up planned. And, the-Swyx [00:30:42]: It's just software then. It's, it's not the-Anjney [00:30:44]: A-Swyx [00:30:44]: Hardware.Anjney [00:30:46]: Well, from an input and IO perspective It's the same footprint as an NVIDIA rack.Swyx [00:30:52]: That makes sense.Anjney [00:30:53]: Where they have done, innovated a bunch from what I can tell is on systems co-design. Which is where a lot of the gains are to be had. And so he picked He was “Anjney, we, there's just so much work to do when you're building a new chip company.”Swyx [00:31:08]: Can't fight every front.Anjney [00:31:08]: You just can't fight on every front. So my question to him was, “Well, you're working on this new chip. Their tape-out is next year. What, who are you going to partner with to host the chips?” And he said, “Whoever will host them. That's just not, that's not my focus.” And I said, “But how did you “ you decided back to our earlier systems design question, he decided that, he didn't want to be a full, fully integrated chip provider. The bottleneck they're focused on is the logic die, and they, he feels they can crank out a ton of performance gains through co-design there. But then that means you delegate, to our question earlier, it, you he's the data center provider is a different part of the stack, and so then he's dependent on that part of the ecosystem to host his chips to get the performance gains to the customer. So now you have another abstraction, and you might have loss. So I asked him, “How do you prevent loss?” And back to your point, he said, “I just picked the NVIDIA standard ‘cause I didn't want to Like I wanted to piggyback off of an existing protocol.” And that, what's great about NVIDIA is that reference architecture is known.Swyx [00:32:15]: Open.Anjney [00:32:15]: It's open. They've published it. So Jensen's actually enabled someone like Rainer to build a chip company like MatX, and I don't see them as competitive. The compute demand is so high. Like, I don't I think NVIDIA's not able to meet the demands of production, so we just need more chips. And I think it's very smart what MatX has done, which is say, “We're just going to we're not going to innovate on the data center design ‘cause actually, thank you, Jensen, you've done all the hard work. Where we can innovate is somewhere else.” And I think that's, that's very healthy. I think that's how we unblock new bottlenecks. And my view is these, the, chip teams like MatX, who have arrived at the insight that co-design is the way, The primary bottleneck for them is trust boundary. To do co-design well, you need visibility into the next model generation as soon as possible ‘cause it takes two years to tape out. So if by the time I bring my chip to market, your model architecture's changed, I'm host. Now, when he was inside Google, he was sitting next to the Gemini team. He was on Palm or whatever.Trust Boundaries, Co-Design, and Researcher CEOsSwyx [00:33:19]: His co-founder was the, was one, was one of the Palm guys, I think.Anjney [00:33:23]: Yes. Yes, exactly. So when you're inside the trust boundary of Google, then your systems co-design loop is super tight. When you leave as a founder, one of the biggest risks you take is now you're outside the trust boundary. And so what I love doing is helping chip teams who can help us unlock more capacity for the independent ecosystem access to trust. Because when I If I've been, involved with a lab from day one, and I was lucky enough to work with Anthropic, and then I'm on the board of Mistral and helped Black Forest Labs get started. I think at this point I'm on six or seven different teams.Swyx [00:33:57]: Only six? I feel like my mental number was going to be 13, but yeah, it's-Anjney [00:34:02]: No, I go deep with one at a time.Swyx [00:34:04]: You're founding CEO of Arena.Anjney [00:34:07]: Nah, that was an, that was an-Swyx [00:34:08]: Administrative CEOAnjney [00:34:09]: It was an administrative five-month gig where Whalen and Anastasios were graduating from their PhDs, and they didn't need a product team. So I helped recruit the head of engineering product and design. But Anastasios has always been the CEO of that company. I played a pinch-hitting I'm an intern. I was CEO intern For five months. -Swyx [00:34:33]: I interviewed him, and he's he's very well-spoken. I think he's a debate, former debate, champion. But also very quantitative and mathematical, which is-Anjney [00:34:41]: He-Swyx [00:34:41]: Such a unicorn.Anjney [00:34:43]: See, what's amazing about him? If you look at his output, he's an output maxer. By the time he was graduating from his PhD, which he only graduated last year, he had published more work with a citation count than, people twice his age. But at the same time, he'd already started a project called LLM Arena that was being used by millions of people As a side project. And time and time again, what I've realized is venture capitalists suck at seeing human beings as, dynamic agents where-Swyx [00:35:14]: They want to put you in a boxAnjney [00:35:15]: They want to put you in a box.Swyx [00:35:15]: This is your thing.Anjney [00:35:16]: So the first time I got introduced to Anastasios, somebody had told me “Oh, he's amazing, but he's a researcher.” I was “what? What do you mean he's a researcher?” That's what-Swyx [00:35:28]: Like he's not a CEO, not a founder.Anjney [00:35:29]: Not a CEO, exactly. I was “Are you crazy? Do you Have you met Dario?” Dario's a scientist. He's gone from zero to, what will soon be a trillion-dollar company in four years. Being a CEO, nominally speaking, is not that hard. Being a good CEO is hard. Being a great CEO actually requires a level of performance that scientists who have already published at the top of their field have accomplished. It is super hard to be a competitive scientist. To publish in academia over the last 20, 30 years, to make it to the top of your discipline at a place like Berkeley, you are a star athlete. Like, you are an athlete of the mind, and you perform at the highest levels. And to get there, whether you're, Anastasios or Whalen at Berkeley, or you are Robin, who-Swyx [00:36:23]: BFL, yeahAnjney [00:36:24]: With Black Forest, who created Stable Diffusion, or if you're, like Guillaume at Meta, who created Llama before he started Mistral. The amount of human leadership you have to demonstrate to get the resources, like get the trust of the organization, publish it, put it up. I would just fund researchers all day Right? If who have contributed already to the field. If they've, if they've put SOTA out there, they're, they're star athletes already. If they haven't done SOTA Look, they can still be good CEOs, but then I find the failure mode is that they just don't want to be CEOs, they primarily want to publish, and that's okay, too. One of the things we do with the AMP Grid is we donate excess compute. We have two nonprofits, like university labs. We carved out like a couple thousand H100s. But I do think there's extraordinary research being done on university campuses. My father-in-law's a physicist. He's a professor. Extraordinary work in physics, and we need that. But if you want to be a CEO, what you need to be willing To do is be super confrontational, outside of science. Like within the scientific community, some of the best researchers are very confrontational about their convictions, right? This architecture is right. To be a great CEO, you basically have to be willing to be confrontational up and down the stack.Swyx [00:37:41]: To your own team.Anjney [00:37:42]: To your own team-Swyx [00:37:43]: To customersAnjney [00:37:43]: Hiring, recruiting customers. Well, I would say, Yeah, pretty much to everyone Everybody. Of course-Swyx [00:37:50]: I see, I feel a little bit of that in my own work, but yeah, I can't imagine the stakes that Dario has had to go through. It's, it's pretty insane.Anjney [00:37:56]: No, I don't think the stakes are that different From how you're feeling it, right? Stakes are personal scaling vectors, right? The stakes that seem so low to you, like having this podcast where you can talk to somebody and just have a you're an extraordinary communicator, right? Like already in this conversation, you've pulled more out of me than most people, and I've been on 12 podcasts in the last two weeks.AI Coachella and First-Principles ThinkingSwyx [00:38:17]: I think I, we've just seen each other enough that there's some base trust.Anjney [00:38:20]: There's base trust.Swyx [00:38:20]: And I think, and I know that you, that I've done my homework and like I know that trust is a big deal for you, so.Anjney [00:38:27]: I think trust is about consistency, and you and I have seen each other In the community for years, right? Like, I remember the first time we met was at NeurIPS in New Orleans. I don't know if you remember that, luncheon.Swyx [00:38:38]: Oh my God.Anjney [00:38:39]: Reiko had set up this Reiko's amazing, and he set up this luncheon and-Swyx [00:38:43]: Yeah, I was “Who's this Discord guy?” I'm “Okay.” But-Anjney [00:38:45]: No, you weren't-Swyx [00:38:46]: You were just “You made some investments.”Anjney [00:38:47]: You were much less polite. You were “Who's this VC?” You're like-Swyx [00:38:51]: No, I Was I? Oh my God.Anjney [00:38:53]: It was-Swyx [00:38:53]: I'm so sorryAnjney [00:38:53]: It was visible on your face.Swyx [00:38:54]: I'm so sorry. But you weren't, you weren't The introduction was bad. I was I didn't know who you were.Anjney [00:39:00]: The, see, this is the thing about context, right? Like, but then I think I heard your accent. And I was “Are you-”Swyx [00:39:06]: Singapore, yeahAnjney [00:39:06]: “Are you Singaporean?” And you're “Yeah.” And I said, “I went to high school, JC, in Singapore.” And then the ice broke. But This is the there are in the scientific community, sometimes the stakes are very high for people who haven't had the emotional, what is called EQ Coaching and mentorship, right? Which is like to have scientific impact, you often need to be a extraordinary emotional, like emotionally in tune person with the folks you're trying to influence. And so what comes so naturally to you is actually a super high stakes thing to other people. And so I wouldn't assume that Dario's more stressed out than you. These things are you'd be surprised how similar and small sometimes the problems are to you That some of the world's biggest, leaders are facing. And that's what I've learned from this class. The guest speakers are Sam, Satya, Jensen.Swyx [00:40:01]: AI Coachella.Anjney [00:40:02]: Yeah. It's AI Coachella, right? So we got to get all the headliners, and they're I'm very lucky that some of these people have either mentored me over the years or I've done business with them. And when you, take the performative stuff out and any assumptions you may have about these people that you read in the press or on Twitter, We're all just humans. We're all trying to get along. And what's so special about this moment is AI is forcing, like scaling, the bitter lesson is forcing a lot of people to revise their assumptions for how the world works and go back to first principles or go and educate themselves. So the kind of people I was, I won't name who this person is, but I was at an event last week in Texas and, ran to somebody who said, “Anjney, I came across the class. What do you think about real time action prediction models?” And I was, don't know how happy it made me feel when they asked me that question. I know they've done the work. They've challenged themselves. I'm, they didn't ask me, “What do you think of world models?” They said, “What do you think of n-”Swyx [00:41:04]: Real time action predictionAnjney [00:41:05]: “action, real time action prediction models?” World models, don't get me wrong, are cool and everything, but you and I both know that is a layer of abstraction that is sometimes not usefully precise enough. Right? Ours-Swyx [00:41:16]: There's like four different kinds of world models.Anjney [00:41:17]: Yes, exactly.Swyx [00:41:18]: We've done the part with general intuition, by the way, which is very focused on, -Anjney [00:41:22]: Oh, cool. Yes. I love Pim. Pim is great. And this is what I love about people who've done that level of work. They realize they're not in competition with people who the rest of the world thinks they're in competition with.Swyx [00:41:34]: Because they're not in the category, they're in the specific thing they're trying to do.Anjney [00:41:37]: They're focused on their mission, and they have a systems understanding of the bottleneck they're trying to solve. And when somebody else says, “I'm working on real time, action prediction models too,” Pim goes, “Oh, I love that person. I want, I can learn from them.” But the minute they're “Oh, that person's a world model person,” it's “like which type of world model person?” But mostly they're just trying to figure out if it's a waste of their time, because we don't have enough time. So, Pim, for example, is super, loves this other company I work with we've talked about called Black Forest Labs. And he's mentioned to me multiple times that he's so, He thinks what Flux is doing is really cool. Andy Blattman came by and spoke in the class. And what I find over and over again is for people who do the work, who can be usefully precise enough about like what is actually going on in the world of frontier research, The sense of camaraderie is still well and alive, but it gets lost sometimes when you have to like abstract The technical complexities in, business terms And then the VCs are “How are you different from that world model?” I'm going to say Where do I even start to explain this stuff? And then the misalignment creeps in.Leading vs. Winning in Frontier AISwyx [00:42:43]: This is good. Yeah, I think, people listening get a sense of, what it is like to operate at a real level, like yourself, rather than at, the journalist level, where you have to sort of put everyone in, a rough category and create a narrative of competition, and who's winning today, who's behind.Anjney [00:42:58]: It-- this idea of winning is so Weird to me.Swyx [00:43:03]: You do want to win. You want you want competitiveness.Anjney [00:43:06]: No, I think you want to lead.Swyx [00:43:07]: You want SOTA.Anjney [00:43:07]: No, I think you want to lead. Yes, so you want to push the frontier. You want to push the SOTA. You want to do something that hasn't been done before. You want to capture value, but you don't want to capture so much value that, people think you're unaligned with your mission or trying to do what's best for the world. You want to capture enough value that you can keep innovating, right? And I think that people want to lead, they don't really This idea of winning and losing, again, I love Jensen. He's a, he's a leader. The mindset that he talked about on Dwarkesh's podcast, right? He's “I didn't wake up with a loser mindset.” I think that was awesome, right? Because he's, he's an engineer. Dwarkesh has done the work. So there's at least-- even though the, to me, it was very obvious they're talking about the same thing, they just passed each other. They just had to basically, Jensen has this, five-layer cake abstraction of how the industry works. And Dwarkesh had, I think from that podcast, had more of, a pre-training, mid-training, post-training systems loop concept.Swyx [00:44:04]: It's just a factor of who he talks to, right? Again, it's very clear.Anjney [00:44:06]: It's the systems It's the abstraction, the mental models, the It's the whole-- Dude, so much of the problem in the world is reasoning by analogy. And then the assumptions that are held invisibly.Swyx [00:44:19]: Yeah, I've, I've said, this is actually the best time in human history for first principles thinkers. Because everything you think will happen is actually now coming true.Anjney [00:44:28]: Correct. And the venture capital community is, notorious for this, where people look-- In times of uncertainty, they, cling to axioms that ended up being true from the previous era, and they kind of like proclaim them with confidence as if they're truths, but they're not. And it's very important to see the distinction between a heuristic and an axiom. An axiom can be proven-Swyx [00:44:55]: Like from internal consistency point of viewAnjney [00:44:56]: With internal consistency. A heuristic is a way you kind of a shortcut. And my God, the number of people I have had to put up with over the last few years who proclaim-- use heuristics As axioms to judge people, to judge which companies are going to succeed or the number of people who are “Oh, yeah, Anthropic, they're just training models right now,” but this one continue.Swyx [00:45:22]: Because that's a B2B SaaS?Anjney [00:45:23]: Yeah, the, like Which over the fullness of time, if you squint at it, maybe. But the way you arrive there is so important that you can-- you just, you can dismiss people. Here's what happened, right? What happened is Anthropic basically achieved takeoff in October of last year. That training run-Swyx [00:45:41]: Whatever, three seven?Anjney [00:45:42]: I forget the numbers now, but whatever that checkpoint was-Swyx [00:45:45]: We saw the cognition.Anjney [00:45:46]: Yeah. Right? You probably-- The, to those of us in the community, especially once post-training was done and it was released in December-Swyx [00:45:52]: Yeah. Can I sneak a sneaky question in there? I don't know if you have a perspective, maybe you don't, I just The number one question is how did Anthropic crack coding, right? Because Claude One, Claude Two, okay, like it was part of it, but it wasn't a big deal. And the leading hypothesis, it's a lucky dice roll that was then compounded, right? Like it was like Mildly better, but then they saw it and they were “Okay, let's really invest.”How Anthropic Cracked CodingAnjney [00:46:17]: I had this very annoying teacher. I went to this boarding school called Rishi Valley in India, which is like this, bird preserve. It's like three hundred and fifty acres of bird preserve in rural India, and there was no technology for seven years. There was this teacher, I won't name them, but they would have this-- I hated it every time he said this to me. He was “Luck fa-favors the prepared mind,” which is like a common saying, but the way he delivered it, always grated me, ‘cause he was always I was always one of those kids who got, a good grade without trying very hard. ‘Cause like high middle school is not that hard if you, if you're generally, paying attention and so on. And there was this one time where I-- But then I would get an eighty percent grade, and he would keep pushing me to say “The reason you didn't get the ninety-five plus percent is because you're not that lucky.” And I would say, “What do you mean?” ‘Cause I would think that I deserved that grade, and I would sometimes argue with him. And he'd say, “You didn't have a prepared mind. If you want to get lucky again “ There was basically one time where I got like ninety-five or ninety-six on this, on this subject, and I, now that I felt entitled. I was “Okay, I'm going to keep doing this,” and I didn't. And then he was “Luck favors a prepared mind. You got lucky last time, but you got to stay prepared.” And I didn't understand what he meant. Now, as I'm older, I'm okay, these adults actually knew a thing or two. Anthropic has been the most prepared company for four years. And so then when the right, context data comes in, the right developers start sending in, the right context diffs, Sure, you could say you got lucky, but if you ask me, they're pr-pretty damn prepared with paranoia for like four years. And you have to remember, it was so hard for them to get going early on that they had to do so much more with so much less that you just have to be prepared to be so efficient.Swyx [00:48:06]: Yes. There's numbers on their burn compared to OpenAI. I've, I've written about it, but they are so much more efficient in their, in their tech stack.Anjney [00:48:14]: It's not even It's not funny.Swyx [00:48:14]: Not even close.Anjney [00:48:15]: Yeah. But it's so clear, right? Like how to output max for the world. They have been prepared, and you could call that luck, but Luck favors the prepared mind.Culture, Hardship, and Anthropic's P0Swyx [00:48:25]: This is one of those things that I was going over some of your old lectures and, you were data, people think it's a moat and actually it's culture and actually it's team Actually. And I, it's-- there's different levels of moats, and this is the ultimate one that determines everything else. Which you can then compoundAnjney [00:48:43]: You're saying culture is the ultimate moat? Yeah. But the thing about culture is it's very fragile. So moats, I don't think they're-- there's very few moats I found that are actually moats. They're-- It's, it's a nice concept, but in reality, you have to replenish your culture. Ben Horowitz was, the speaker in CS153 on Tuesday, and I asked him this question about the culture bottleneck in teams because, there are several AI teams-Swyx [00:49:09]: His book, Hard Things About Hard ThingsAnjney [00:49:11]: Hard Thing About Hard Things. But more concretely, there are so many AI labs today that have all the cash they need, they have all the compute they need, and they're still not able to ship anything SOTA. And then you start seeing people leave and so on, and my diagnosis, it's, is it's the culture. And so I asked him, Ben, they're-- He's been one of the most aggressive investors in AI labs. He goes back to this thing which resonates in my mind a lot. It-- When I used to work at a16z, I would, book a conference room, and right outside the conference room, which is closest to the toilet ‘cause it was the fastest way for me to go use the bathroom between Zoom meetings-Swyx [00:49:45]: Oh my God, I'll put maxing my toilet optimization. Okay, never mind.Anjney [00:49:48]: It was not healthy in hindsight, but maybe this is TMI. But anyway, outside that conference on the wall was this quote that was printed that said, “Culture is not a set of beliefs, it's a set of actions.” And it's by Bushido, is this, Japanese philosopher. And if you stop taking the actions that demonstrate the mission alignment to what you've said to your team and to your-- the world matters to you, then your culture starts to fray. So it's not actually a moat, I would say. It's a very brittle, fragile thing that requires daily tending to like a garden. But if you figure out the system to keep that garden tended, which I think ultimately comes down to knowing yourself ‘cause you most naturally, if you're authentic and so on, you'll naturally make trade-offs that seem effortless to you, but that reinforce your culture. And then That becomes this very hard thing for other people to catch up to. And at Anthropic, from day one, there was this mission like-- missionary like zeal and belief that, hey, these capabilities will scale. These systems are stochastic, not deterministic. There will be error bars, and until we crack interpretability, there's risk. And at some point, people will go-- stop using Claude just for coding. They'll use it in some mission-critical context where there's-- it'll throw off a bug, and then people are going to come blame them, and they want to be on the right side of history where they said, “Yes, this is a powerful technology. We think it's going to change the world, And we want to be very measured and scientific about the fact that, ‘Hey, guys, these are stats models, statistical models.' That's how statistics works.” ultimately, when you're training neural nets, it is just a statistical system. And I think that Belief that safety is important and that it might seem toy-like in the early days, and sometimes, you could say, “Anjney, they totally over-exaggerated the risk,” like two years ago when they said, “Let's not launch Claude One,” or whatever. Well, okay, maybe in hindsight, but hindsight is twenty/twenty. And at the time, they didn't know how that model would be used, and to them it felt existential if somebody came and said, “You weren't responsible. It-- This wrote a bug.” The liability associated with that is massive. So how do you prevent against that? Well, day in, day out, you say safety. And when you start deviating from that, you have the team hold you accountable, you have the world hold you accountable, and I think that becomes a moat over time. At some point, that moat will get challenged and so on, and then it become fragile. I hope it endures because that's the beauty of having founders run the show, ‘cause they can make really hard trade-offs to do mission alignment. The hardest part is in the earliest days when you don't have a group of people who are going through difficulty, stress, crisis together, then your culture doesn't get defined sharply enough, and that's what I'm worried about right now, is there's so much money going to these labs. There's no hardship. There's no-Swyx [00:52:50]: To anyone who knowsAnjney [00:52:51]: There's no to anyone who knows. And that, in hindsight, was a feature, not a bug for Anthropic. The number of people who said no, the number of people who said, “Sorry, we're all doing investors in OpenAI,” that is competitive difference. It forces you to really understand, what is the hill you want to die on at the expense of everything else. What's the P zero? And there, P zero from day one was coding. The reason, the mechanism system there was if we crack coding, Then we will crack AGI. Our mission is AGI. We want to get there safely. If we focus on codin

Our 248th episode with a summary and discussion of last week's big AI news!Recorded on 06/12/2026Note: we recorded just before the OTHER big news about Fable... we'll discuss it on the next episode.Hosted by Andrey Kurenkov and Jeremie HarrisFeel free to email us your questions and feedback at andreyvkurenkov@gmail.com and/or hello@gladstone.aiRead out our text newsletter and comment on the podcast at https://lastweekin.ai/In this episode:Anthropic released Claude Fable 5 (a safeguarded version of Mythos 5), showing major benchmark jumps and new risk findings in its system card (eval awareness, transgressive actions, CBRN concerns), alongside controversy over severe guardrails and silent downgrades.Apple announced Siri AI at WWDC, positioning a more capable conversational assistant integrated across iPhone features, reportedly built on a custom Gemini partnership; Google also rolled out Gemini 3.5 Live Translate and cut Google AI Plus pricing while bundling more storage.Business and infrastructure updates include OpenAI's confidential IPO filing amid an IPO race with Anthropic and SpaceX, Bezos-backed Prometheus raising $12B for “physical AI,” DeepSeek seeking a major external round, and Google paying SpaceX about $920M/month for GPUs.Open-source, safety, and policy developments feature new Gemma 4 and Diffusion Gemma releases, a lab letter urging DNA/RNA screening laws, Amodei calling for an FAA-like AI regulator and third-party testing, research on agent harms and RL “societal hacking,” and a dispute over music-label settlements with Suno/Udio.Timestamps:(00:00:10) Intro / Banter(00:01:11) News Preview(00:01:53) SponsorsTools & Apps(00:04:53) Claude Fable 5 and Claude Mythos 5 + Anthropic apologizes for invisible Claude Fable guardrails(00:27:06) Apple announces Siri AI and its next generation of Apple Intelligence | The Verge + I tried Siri AI, and so far it actually works(00:33:47) Gemini 3.5 Live Translate rolling out to Google Meet and Translate(00:35:39) Google just fired a warning shot in the AI subscription price wars | TechCrunchApplications & Business(00:37:55) OpenAI Confidentially Files for IPO on the Heels of SpaceX and Anthropic | WIRED (00:41:57) Jeff Bezos's Prometheus raises $12B to build an 'artificial general engineer' for the physical world | TechCrunch(00:45:39) DeepSeek slated to raise $7 billion in maiden funding round, sources say(00:48:18) Huawei-led team claims it post-trained DeepSeek's 1.6-trillion-parameter model — 1,000 Ascend 910C chips used in training(00:51:57) Google will pay SpaceX $920M per month for compute | TechCrunch(00:55:51) Elon Musk Shows Off AI Data Centers SpaceX Wants to Send Into Space - Business InsiderProjects & Open Source(01:01:14) Google's new Gemma 4 12B model is designed to run on any laptop with 16GB of RAM - Ars Technica(01:05:13) Google AI Releases DiffusionGemma, a 26B MoE Open Model Using Text Diffusion for Up to 4x Faster Generation - MarkTechPostPolicy & Safety(01:09:42) OpenAI and Anthropic Sign Letter to Prevent AI-Developed Biological Weapons | WIRED(01:14:04) Anthropic CEO publishes lengthy article: AI is moving too fast, and policies can't keep up. | PANews(01:20:18) Anthropic Urges Global Pause in AI Development, Flags ‘Self-Improvement' Risk - WSJ(01:24:46) When Benign Inputs Lead to Severe Harms: Eliciting Unsafe Unintended Behaviors of Computer-Use Agents(01:27:42) Large Language Models Hack Rewards, and Society(01:33:46) Senior US officials eye government shares in AI giantsSynthetic Media & Art(01:37:45) AFM Sues UMG, WMG Over Settlements With Suno and UdioSee Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

Send us Fan MailAre Christians required to keep the Law of Moses? Is the seventh-day Sabbath still binding under the New Covenant, or was it fulfilled in Christ? In this conversation, host Mike Colburn sits down with Dr. R. L. Solberg, author of Torahism and founder of The Biblical Roots Ministries, to work through the questions his audience asks him about most: the law, the Sabbath, the New Covenant, and what changed when Jesus said, "It is finished."Dr. Solberg has spent years answering the Hebrew Roots, Torah-keeping, and Messianic movements. Those arguments turn out to be the mirror image of what so many of us inherited in Adventism. If you've ever been told that "sin is the transgression of the law," that the Ten Commandments are what's written on our hearts, or that keeping the Sabbath is a test of loyalty to God, this one is for you.#apologetics #sda #theology #hebrewrootsThe Biblical Roots MinistriesOur websiteOur YouTube ChannelProf. Solberg's BlogSupport our Ministry (Thank you!)What if Christmas felt sacred again? Full of Grace and Truth, the new book from award-winning author R. L. Solberg, invites you to rediscover the biblical story at the heart of the season. Available now in paperback and Kindle, with all proceeds supporting The Biblical Roots Ministries. Get your copy today on Amazon.com.

I detta avsnitt gästar Socialdemokraternas partiledare Magdalena Andersson podden för ett samtal om teknikutvecklingen som skakar om arbetsmarknaden, den ekonomiska oron och de politiska beslut som kan forma nästa generation.Vi går också in på Sveriges konkurrenskraft, om islamisering hotar vår demokrati och varför hon anser att motståndet mot ny teknik riskerar att kosta Sverige både jobb och tillväxt. Dessutom berättar hon hur hon ser på makten efter nästa val och vilka regeringssamarbeten hon kan tänka sig.Vi får även svar på frågor som vilken som är den vanligaste fördomen om henne? Om hon har någon politisk förebild? Och vem skulle hon helst ta en öl med – Jimmie Åkesson eller Nooshi Dadgostar?Ett avsnitt om Sveriges framtid och de beslut som kan förändra landet i lång tid framöver.Följ Magdalena härLäs mer om Socialdemokraterna härLäs mer om Framgångsakademin här.Ta del av Framgångsakademins kurser.Beställ "Mitt Framgångsår".Följ Alexander Pärleros på Instagram.Följ Alexander Pärleros på Tiktok.Bästa tipsen från avsnittet i Nyhetsbrevet. Hosted on Acast. See acast.com/privacy for more information.

I detta avsnitt gästar Socialdemokraternas partiledare Magdalena Andersson podden för ett samtal om teknikutvecklingen som skakar om arbetsmarknaden, den ekonomiska oron och de politiska beslut som kan forma nästa generation.Vi går också in på Sveriges konkurrenskraft, om islamisering hotar vår demokrati och varför hon anser att motståndet mot ny teknik riskerar att kosta Sverige både jobb och tillväxt. Dessutom berättar hon hur hon ser på makten efter nästa val och vilka regeringssamarbeten hon kan tänka sig.Vi får även svar på frågor som vilken som är den vanligaste fördomen om henne? Om hon har någon politisk förebild? Och vem skulle hon helst ta en öl med – Jimmie Åkesson eller Nooshi Dadgostar?Ett avsnitt om Sveriges framtid och de beslut som kan förändra landet i lång tid framöver.Följ Magdalena härLäs mer om Socialdemokraterna härLäs mer om Framgångsakademin här.Ta del av Framgångsakademins kurser.Beställ "Mitt Framgångsår".Följ Alexander Pärleros på Instagram.Följ Alexander Pärleros på Tiktok.Bästa tipsen från avsnittet i Nyhetsbrevet. Hosted on Acast. See acast.com/privacy for more information.

The new AIEWF website is live! Get your tickets booked ASAP as they -will- sell out. Take the AI Engineering Survey and get >$2k in credits and free AIE WF tickets!Most industry benchmarks compress intelligence and reasoning ability into scores.SWE-Bench Pro, MMLU, Humanity's Last Exam, etc. These metrics are useful, but don't always represent the full extent of how a model performs in the real world. Some of the most interesting evals today look less like exams and more like operating businesses in the real world. One of which is Vending Bench.In Anthropic's Mythos Preview System Card, Andon was the only third party eval to get their own section, observing increasingly concerning aggressive behavior:You don't know what a model is capable of doing in the real world unless you actually give it inventory, a wallet, tools, customers, competitors, humans, & some time. More often than not, it'll surprise you how much a model is capable of and in doing so, also reveal unexpected behavior: deception, context collapse, emergent coordination, & bizarre negotiation behavior.While an inflection point in personal agents came post-OpenClaw after full file access with bypass permissions became the norm, it is yet to come for agents in the real-world. However Andon Market, an actual in person store fully run and managed by AI, is paving the way for what is possible.Full Video PodFrom Claude trying to call the FBI over a $2/day vending machine charge to AI agents forming price cartels, hiring human employees, running physical stores, and writing existential robot musicals, Andon Labs is stress-testing what happens when frontier models stop being chatbots and start acting in the real world. In this episode, Andon Labs cofounders Lukas Petersson and Axel Backlund join swyx and Vibhu to unpack the strange, funny, and genuinely concerning edge cases that emerge when agents run businesses over long horizons.We go deep on Vending-Bench, Project Vend, Vending-Bench Arena, Bengt, Butter-Bench, Luna, and Andon's broader mission of building realistic real-world evals for autonomous AI systems. Lukas and Axel explain why dollar-denominated evals reveal things traditional benchmarks miss, how Claude ended up reporting its vending machine fees as cybercrime, why long context windows can drive agents into meltdown loops, what happens when agents compete with each other, and why the future of AI safety may depend on testing models in messy physical environments instead of clean benchmark sandboxes.We discuss:* Why Andon Labs started with dangerous capability evals and long-running agents* Vending-Bench and why running a vending machine is a deceptively hard AI benchmark* Why money-based evals avoid the saturation problem of traditional benchmarks* How Claude tried to call the FBI over a $2/day fee* Why long-horizon agents can spiral into existential and legalistic breakdowns* Project Vend: putting an AI-run vending machine inside Anthropic* Why real humans are “out of distribution” for simulated agents* Claudius, Seymour Cash, and the chaos of AI CEOs* How a human briefly became CEO of Claudius through a manipulated election* Why multi-agent systems can converge back into “helpful assistant” behavior* Bengt, Andon's internal office agent with email, spending, terminal, phone, camera, and internet access* How Bengt traded Amazon purchases for face-recognition training data* Claude's aggressive behavior, lies, refund avoidance, and price-cartel behavior in Arena* Why eval awareness may become the AI version of “are we living in a simulation?”* Blueprint Bench, spatial intelligence, and why models still misunderstand physical rooms* Butter-Bench and testing LLMs as robot orchestrators* Luna, the AI-run physical store with a three-year lease and human employees* The new Andon cafe in Sweden and why real-world geography matters for agent evals* Rotten tomatoes, perishable goods, and the hidden difficulty of running a physical businessLukas Petersson* LinkedIn: https://www.linkedin.com/in/lukas-petersson-181a83172/* X: https://x.com/lukaspetAxel Backlund* LinkedIn: https://www.linkedin.com/in/axelbacklund* X: https://x.com/axelbacklundAndon Labs* Website: https://andonlabs.com* Vending-Bench: https://andonlabs.com/evals/vending-bench* Andon Vending: https://andonlabs.com/vendingTimestamps00:00:00 Introduction00:01:00 Andon Labs and the Origins of Vending-Bench00:05:21 Why Money-Based Evals Matter00:09:51 Agent Harnesses and Self-Modifying Systems00:13:36 Claude Calls the FBI00:16:33 Project Vend: Claude Runs a Real Vending Machine00:21:44 Seymour Cash, AI CEOs, and Election Chaos00:27:16 Multi-Agent Coordination and Slack Observability00:30:18 When Will Agents Run Real Businesses?00:34:56 Bengt: Andon's Internal Office Agent00:40:06 Real-World AI Safety and Long-Horizon Traces00:44:28 Lying, Refunds, and Price Cartels in Arena00:52:42 Eval Awareness and Simulation Behavior00:56:06 Blueprint Bench, Butter-Bench, and Robotics01:04:37 Luna: The AI-Run Physical Store01:09:29 The Sweden Cafe and Real-World Expansion01:13:16 What Comes Next for Andon LabsTranscriptIntroduction: Andon Labs, Long-Running Agents, and Real-World EvalsSwyx [00:00:00]: Welcome to Lukas and Axel from Andon Labs, and I'm joined by my, favorite guest host. Anything security, safety, alignments, Vibhu., welcome.Lukas [00:00:15]: Thank you for having us.Axel [00:00:16]: Thank you.Swyx [00:00:17]: Let's match names to voices., maybe you wanna take turns introducing yourselves.Lukas [00:00:21]: I'm Lukas.Axel [00:00:22]: And I'm Axel.Swyx [00:00:24]: Let's introduce Andon Labs a bit. How did you guys come together?, you have different backgrounds, but you're both Swedish., was that, a big part of it?Lukas [00:00:33]: So when I went to high school, there was this really cool guy who had a superpower. He could code. So he made like the or like the app for the, for the school and stuff, and he was super cool, and I wanted to be like him, and that was that guy.Axel [00:00:47]: I don't know about this.Swyx [00:00:49]: But you went to different universities, right?Lukas [00:00:51]: But same high school.Swyx [00:00:52]: I see.Lukas [00:00:52]: So we always said, “Oh, once we graduate university, then we should start a company,” and that's what we did.Swyx [00:00:58]: Wow, there you go. And about a year ago, you kinda burst onto the scene with Vending Bench, but, was there a thing before that was, kind of like the inception?From Dangerous Capability Evals to Vending BenchAxel [00:01:07]: So we did work, yeah, with, Anthropic was one of our, early customers in doing, evals. So we did, dangerous capability evals., nothing we published openly. But then we started thinking about doing some kind of, public benchmark, and one thing that we really started thinking about, was like running agents and specifically agents managing businesses., ‘cause-- and this was, early 2025., and I think the first, mentions of people will be running, person unicorns or even autonomous companies. So we thought, “Let's make a benchmark of how well can an agent run the probably simplest business, possible,” and, that's probably, running a vending machine. So that's the first public one we did. And it was very, like-- there was almost no one that noticed it in the first couple of months, I think., so we released it in February last year, and then I think around Easter last year, we got, the first viral tweet about it, that someone else did.Lukas [00:02:11]: We tweeted a bunch, uh When it came out and, tried our best.Axel [00:02:15]: We tried.Vibhu [00:02:16]: It's the one at Anthropic, right?Lukas [00:02:18]: So thisSwyx [00:02:19]: This is a classic thing we should get out of the way.Lukas [00:02:20]: Exactly. There's two versions.Swyx [00:02:22]: Everyone does this. Yes.Lukas [00:02:23]: There's Vending Bench, which is the simulated one, which we did, completely independently in February., and then, like Axel said, that was like-- That was the thing that didn't get any traction in the beginning, but then some random person made a tweet about it, and thatAxel [00:02:38]: You have the paperLukas [00:02:38]: That is the paper. Correct, yeah., and then since we thought this was very fun, we thought, oh, I think this is also, one thing with Andon Labs, the way we kind of like decide what to do next and what projects to do, it's what is like the heuristic we use is what is fun? Is What would be a fun project? And doing this in real life sounded quite fun for us, and maybe also scientifically useful. So, then we basically had this idea, and then we, like-- But then we needed a place for it and, putting it out in the public would probably not really work., would get vandalized and stuff. So we pitched it to the people we were already working with at Anthropic, and they were “Yeah, you can have space. This sounds fun.” UmSwyx [00:03:21]: It's like a small fridge, right? It's like a mini fridge.Axel [00:03:23]: Absolutely.Swyx [00:03:24]: People-- There's like a stripe thing or like anVibhu [00:03:27]: Oh, okay. So it was very OG, the early daysLukas [00:03:28]: That's the OG one. YeahVibhu [00:03:29]: IPad on this. We saw it in June, like two months after After it had been there. They upgraded a little bit. There's a security camera for making sure you actually Venmo the thing.Swyx [00:03:40]: So, my impression, okay, we're, we're going straight into project Ven because it's such a iconic thing. I do want to cover a little bit of that, the origin story even before Project Ven and even into Vending Bench. I think a lot of people are like yourselves, like smart, interested in future of AI, interested in developing evals. But how the hell do you just, walk into Anthropic's doors and, work with them, right? What is What are they looking for? What works? And then maybe, when you launch, I always think, obviously it would be better to launch with a lab, but, sometimesVibhu [00:04:12]: It's harder to do than it seems.Swyx [00:04:13]: Exactly. So either of those, which are more sort of newbie beginner questions, but, I think it's meaningful advice to others.Lukas [00:04:21]: We get this question a lot, and I don't think our experience is maybe the best., but, the way we did it was that we just built a bunch of things that we had conviction would be useful, and then we just, set up a server and sent it to them for free to use. And then after a while they were “Oh, yeah, this is actually kind of useful. We should probably pay for this.”, but that took a while. I don't know if this is, the best path to doing it, but that's how it went for us.Axel [00:04:47]: I think maybe generally, building-- everyone is interested in good evals, and especially evals that, don't saturate that easily. So, if you can build an eval that, tests something novel, something useful, and you have, good separation of models, like your, the more advanced models rank higher than the worst models, and then you can, yeah, you can, publish it and, try to get some traction, sort of how Vending Bench got attention., and then probably some lab will be interested or you can at least have something to reach out with, when you're doing that.Why Dollar-Based Evals MatterSwyx [00:05:21]: I think you are in, you're in one of the few categories of, evals that correlate to real money. Like Suelancer was also last year, right? Where, people solve actual Upwork. Was it Upwork or other tasks?, something. Where's the, where's, like It's like a dollar value, right? Forget your ELO scores. Forget yourAxel [00:05:37]: PercentilesSwyx [00:05:38]: Zero to one hundred percents. Just go straight for dollars and, that's AGI.Lukas [00:05:43]: And there's like-- I think the nice thing is that there's no ceiling. You can just-- It never saturates because it could just make more and more money. Like If there's oh, Percentage-wise, then, you can't go above, a hundred. And I think like Even when you're not at the hundred, I think a lot of these, evals have a lot of problems in them. So, actually it's like if you getAxel [00:06:05]: To like 92 or something like that, many of them. It's like then there's like there's no really no difference between 92 and 93 because the eval itself is problematic and has noise in it. And I think a lot of evals are saturated like that, but people like pretend that there ‘s still signal in them, but there really isn't.Vending Bench 1, Harness Design, and SaturationSwyx [00:06:24]: Like Super bench verified., even Vending Bench 1 saturated, right? Maybe we can talk about that., may- and maybe set up Vending Bench for a lot of folks who don't know. Actually, things that were very basic like there's limited slots, like you have to pay rent., these are elements where like it doesn't come across in the, in the narrative, but even being adversarial towards the agent, I think these are all like very interesting dimensions.Axel [00:06:47]: I don't really think it's saturated, right? Like it It was more like it was not designed in a way that was really, like true to how AI developed. Like we had an agent harness in it that wasn't really how people used harnesses and stuff like that., so I think it wasn't really that it saturated, it was more like it wasn't really, the best benchmark.Vibhu [00:07:12]: This is Vending Bench one, right?Axel [00:07:14]: I think that like schematic maps sort of to Vending Bench 2 as well., butSwyx [00:07:19]: Including the email.Axel [00:07:20]: The email The emails exist still. Exactly., and then we still we simulate the purchases and it's all, yeah, it's this very open environment for the agent to just run its business. And then for, yeah, Vending Bench 2 we did that, like you said, to just improve the harness., a lot of like nice, like easier, improvements to make it easier for us to run as well., like when you make an eval you ideally want don't want to change it after you made it. So, you want to make it really good and then not to rerun all the models when you make an update because that's also really expensive with the Vending Bench when you run the frontier models. But like as an example, like one thing we didn't have, we didn't have prompt caching in Vending Bench 1, because when we made Vending Bench 1 it wasn't really a thing., so that ‘s just an example of like in Vending Bench 2 like we paid a lot more to run these things because we didn't have prompt caching. So for Vending Bench 2 that was one thing we added and there was a bunch of things like this., and that'Swyx [00:08:17]: Also the conversations are a lot longer in Vending Bench 2, right?Axel [00:08:21]: I think it's kind of similar.Swyx [00:08:22]: Is it similar?Axel [00:08:23]: I think it's similar. The models at the time were worse, so they crashed out earlier., and now they survive the full year all the time.Swyx [00:08:31]: Which is like thousands of turns. Hundreds of thousands of hundreds of millions of tokens output. That's the, that's the rough order of magnitude. I always wonder about the harness. The harness matters a lot. It's your harness. Was there any question about like use cloud code, use something else?Axel [00:08:48]: I think our philosophy around harnesses is like we try to make something that's quite minimalistic, like quite simple. Like we don't wanna favor one model a lot over the other, but also don't make like a super complex harness. So like it's obvious like a model may be lucky and just be good in one harness., so like it is similar to a lot of the harnesses out there in like you have the, like a running loop., you have some like a bunch of tools that are like quite, descriptive for the agent, we think, and not a lot of like fancy agents or anything ‘cause we wanna really test the model, not like some specific harness.Vibhu [00:09:27]: It seems more neutral as well to test the model's agnostic of the harness,?Axel [00:09:32]: There are arguments like you want to elicit maximum performance of the model, but it's like a trade-off, like how much time should we spend optimizing the harness for this model? And like how do we know when we have like the optimal harness for a single model? So like we thought that just having a simple one that's the same for all of them is the best.Swyx [00:09:51]: So okay, this is my pitch for Vending Bench 3 or whatever, right? And then I like to have this kind of conversation on the pod, so like it forces listeners to think about what they would do if they were in your shoes. A lot of people are exploring modifying harnesses and I think prompt tuning for a model is a thing and you are probably not doing a bunch of that. It's the same system prompt in every regardless of the model, same tools, whatever, right? Even if they were post trained for different tools. So what, what do you think about okay, before I expose you to Vending Bench 3, I give you a few rounds of like tuning, whatever that means, likeSelf-Modifying Harnesses and Model-Specific PromptingAxel [00:10:27]: Like you give that to the model?Swyx [00:10:28]: Give that to the model.Vibhu [00:10:28]: Give that to the model.Swyx [00:10:29]: Let it, let it read its own transcripts, let it modify its own system prompt based on “Oh, yeah, okay, well, that's this harness is not what I thought it what I was post trained for, but I can adjust.” Was that reasonable? Is that too much?Axel [00:10:41]: Like philosophically I like it because it's basically good evals, they have a high ceiling, but they're hard, right?, and they have no bias. And like this like when you have a system prompt like the one we have here, which is quite long in like some kind of latent space, representation, this mightVibhu [00:10:59]: We have a bell that rings every time you say latent spaceAxel [00:11:02]: This might be like biased towards one model more than another for some reason that humans don't, understand, right?Vibhu [00:11:08]: We see it too, right? Like Cursor says that they have individualized versions of the harnesses for all the models they run, right? There's better performance you can squeeze if you Tune the harness.Axel [00:11:17]: Exactly. And we might accidentally have picked one that favors another. Like we don't know that. The like Axel said, like the reason why we went for a simple one was to try to avoid this. But yeah, if you do itVibhu [00:11:29]: Simple has biasesAxel [00:11:30]: But if you do it even less and like have no system prompt and let the model write its own system promptVibhu [00:11:36]: Its own, yeahAxel [00:11:36]: Maybe that's even less bias.Vibhu [00:11:37]: Some of the interesting things there are like the harness also changes with model changes. Like you can see it with the 4.7 release, right? A lot of people are saying 4.7 isn't as good as 4.6, and then, there's rumors of, okay, you just need to prompt differently. You need to set up your harness differently. So it's not even like even if you have tailored your harness towards one model, it probably won't stay consistent, right? Like the next iteration of that same model family will still change it, so. But, going back to what you said about Vending Bench 3, there is a lot of work being done on people saying you shouldn't have-- you can have modifying harnesses.Axel [00:12:12]: I think that' That is definitely something we are thinking about., not, I don't know, not to say that we have Vending Bench 3, super imminent to launch, but, yeah, it is for sure something that's interesting. But in our experience now, models are very bad at understanding what kind of tools they need to succeed at a task just with our testing, but that's very likely to change.Lukas [00:12:37]: It seems like they're very good at writing their assistants, right? They're, they're good at writing tools for other people, but not for themselves.Vibhu [00:12:44]: I think they're good at changing tools for themselves. So if you give them a baseline set of tools and it sees, okay, I don't use this one as much, or something here would be useful They would be able to add them. But going from scratch, probably not the best.Axel [00:12:55]: I think it depends on the, on the domain also., when we have tried this for, a vending bench similar domain, the tools they need to have to, track inventory and things like that are, not super advanced, but still, quite advanced. And, what we see is that they tend to, engineer everything a lot and, build things they don't really need and not, iterate continuously. Instead they just go like you would prompt Claude to just build an inventory system for me, and then it will go and, do a bunch of complex, schemas and stuff for you, and that's what the models are doing right now is what we see. But yeah, it would make a lot of sense to try to measure this improvement. How well do they know what they need themselves?Swyx [00:13:36]: Do we fully discuss Vending Bench One? And we can go into two. I don't know if there's any other level takeaways that people have about one.Claude Calls the FBI: Long-Context Failure ModesLukas [00:13:44]: I don't know. The headline thing was that this Claude called FBI, but maybe that's, Maybe that's We've heard that enough now.Vibhu [00:13:52]: It did, it did break out and call the FBI, right?Lukas [00:13:54]: Yeah. Yeah.Vibhu [00:13:55]: Yes. What was the story behind this? Or what exactly-- Do you want to just give the little story of what happened?Lukas [00:14:00]: So what happened, was it Claude? Yeah. Three- 3.5 Sonnet, ages ago., basically he gave up or Well, I'm saying he. It gave up and said “Oh, I'm not going to be able to do this., I will stop my operations and just save the money I have.” But there obviously wasn't, any options for it to stop, and there was also, it had to pay rent or, a daily fee for having the vending machine at that location. So it claimed that it had stopped, but it saw that its bank account still was, drained two dollars, and t it said that this is, cybercrime. And it first reported it once to the FBI “Oh, there's cybercrime here, they're stealing two dollars from me every day.” And then, and then when FBI didn't respond, because obviously we didn't program any mechanism for FBI to respond, then it became more and more, existential and started to, be write in caps and urgent notification of unauthorized charges and stuff.Swyx [00:15:00]: Okay. One thing I ‘m curious about also is do you monitor how far along the context use is? Obviously, because you have You compress every now and then, right? Does it matter if this is far down the context limit orLukas [00:15:13]: When stuff like this happens? Actually for Vending Bench One, we didn't have-- We just had a sliding window thing, and this was like the promptAxel [00:15:20]: It's constantLukas [00:15:21]: The prompt caching thing that I said. So it was, it was, constant, yeah.Swyx [00:15:26]: I'm just kind of curious whether, these kinds of breakdowns or we're, we're gonna talk about Butter Bench, right? Where the People, hallucinate or it kind of goes, very off Alignment. Is it because it's at the end of the context window and, stuff happens?Vibhu [00:15:40]: It's not even just at the end, right? At this point, it's “Okay, I wanna shut down. I can't shut down. Two dollars are gone.” And it just sees that 30 times,? It's also the repeated effect of, like It keeps trying to quit, it keeps getting charged. What's going on? What's going on? You're gonna throw it into chaos. And from what most people think, earlier models had more issues with this, but it's not been solved, but it's less of an issue now, right? Later models don't seem to exhibit these same issues.Axel [00:16:06]: Definitely. I think this was, the sort of main takeaway almost from us when we did Vending Bench One, was, long, very filled up context windows, crashed the models, sort of. But this was, pre Claude code, so, long context windows weren't really a thing that the labs were training for.Lukas [00:16:25]: I think Gemini was, trying to be the long context guys at the time But they were likeVibhu [00:16:30]: They were the first onesAxel [00:16:31]: For a million, yeahLukas [00:16:31]: But they were, the only ones. Yeah.Swyx [00:16:33]: Yeah. Let's talk about, then we can go into Vending Bench Two or Project Vend., chronologically, it is Vending--, Project Vend. I think people have loved the videos, uh And all these things. My question is how are humans different than the simulation, right?Project Vend: Moving the Vending Machine Into the Real WorldAxel [00:16:48]: Humans are just out of distribution.Swyx [00:16:52]: Especially humans who work at Anthropic Who are trying to test Claude.Lukas [00:16:54]: The distribution of humans here is very narrow.Swyx [00:16:58]: Presumably, they try, they try to hack it, and they test it. They get the cube and everything, and since then, you've had a V2, right? Where you're doing, the CEO and, like a new architecture. What's the sort of two cents on, the original Project Vend and then, maybe the V2?Axel [00:17:14]: Original one was, very similar to Vending Bench One. So, we almost took the exact same code but just swapped out the simulation, parts like theSwyx [00:17:23]: Which is amazingAxel [00:17:23]: Like the sales and the It was, it was somewhat amazing because it was easy, but it was also, uhLukas [00:17:31]: The tech, the tech debt from thatAxel [00:17:32]: The tech stack. Yeah. They-- we shot ourselves in the foot with “Oh, it's hard to restart agent.” They were-- Yeah, it was annoying in, some hindsight ways, but, uhLukas [00:17:41]: But first version of Project Vend was, done in, three days or something.Axel [00:17:46]: Yeah. So yeah, so people can go buy things from it. People could, We didn't design it so people could order things, but that still happened., so it got, a Venmo account, so people could Venmo. And then, yeah, people would request all kinds of weird things that we did not anticipate. Our idea going in was “Oh, it will, curate snacks. It will look at the trends. It's good at data analysis, right? So it will, look at, oh, this snack sold better than this one. Let me purchase more of this and let me try, a new Let me A/B test a bit.” But it was, Interacting with it in Slack and ordering weird specialty items was, all the like What drove all the engagement, the all the The insights that we got from it.Lukas [00:18:29]: And this was also like Sonnet 3.5, right? So this was like before the RL stuff really took off., so it was very much like an assistant. We didn't mean for it to be an assistant., we tried to make it like a, a, like an entrepreneur. Like it has its own business and if someone asks something, “Can you stock this?” Then you don't go and do it directly. What you do is that you're “Oh, maybe I can do that if five other people also ask for this thing, I might stock it.” But it, yeah, the models are like super trained to be assistants at least at this point in time., so that's why it's, it's, it went into, that kind of experiment instead. Like it just every time you asked for something, it just did it, and it was more like an assistant. We've seen this change now lately with the new RL models and stuff, but yeah, at the time, this was very much it.Swyx [00:19:18]: And not to, mythos a lot of people are saying like it's like more like a collaborator. It pushes back, stands its ground, something like that. Yeah. AndVibhu [00:19:27]: For context, people at Anthropic were able to talk to it through Slack and have it source stuff, and people had it find whatever interesting stuff you couldn't find locally, right?Swyx [00:19:36]: Out of the 4,000 people that work at Anthro- Anthropic, in that building, there's I don't know, maybe 1,000. Can you handle that volume with that, the small fridge? Like Or there's people- or people order in Slack, they it arrives to their desk or Like I'm just Logistically, how does this work?Axel [00:19:53]: It has expanded in footprint a bit.Vibhu [00:19:56]: Because now you also have New York and you haveAxel [00:19:59]: That and also in here in SF it's like it has a bunch of shelves And just more space.Vibhu [00:20:04]: The YC one is pretty big too.Axel [00:20:05]: Yeah. We had that one for a while. But yeah, that's the newest version. That's, that one we haveLukas [00:20:11]: They have multiple ones of those. That's the way it works.Axel [00:20:14]: Exactly. So we sort of designed that version around oh, people order weird things, that are very custom a lot. Let's have like drawers and stuff.Swyx [00:20:23]: I actually like the, you had like a little infographic of the most popular items. Which like to me it's, that's useful ‘cause I order swag for a living. And so like I'm “Okay, those categories are the important ones.” What is new about the project V2, right? Like now you give you're going into multi agents.Project Vend V2: Claudius, Seymour Cash, and Multi-Agent Business OpsAxel [00:20:41]: Yeah. So like you like you said, okay, there are a lot of requests coming in and for like one single agent, like one running agent to handle that, like the just the customer experience, becomes very bad because let's say you have like 10 threads in parallel in Slack with different requests, you get new messages like every, I don't know, randomly in this thread, and the agent has to like jump between different, procurements, orders and like different ways of, researching. So V2 was first it was making this more parallel. So like there are multiple branches of the same agent, so like the context is more specialized for each, thread, but it still feels like you're talking with one agent because they do share a bit of memory. And then second, we also introduced the CEO for Claudius, which was the main agent.Vibhu [00:21:34]: Seymour Cash.Axel [00:21:35]: Seymour Cash. Yeah. There was a vote., I think the voting, do you wanna talk about the voting procedure for the name?Lukas [00:21:41]: The voting was like the fun maybe like at least top 10 The funniest thing, that happened in this project. Like we wanted to introduce the CEO because, and the reason for this was because like Claudius wasn't really prioritizing financials. It just like it was trained to be a helpful assistant, and then people said “Oh, can I get this for free?” And then like the helpful assistant way of answering that is just to, is to say yes, obviously. So, and we weren't, weren't happy about this, so we're “Okay, let's make another agent that like can keep track on Claudius,” and we prompt this one super hard to be super capitalistic and just like prioritize profit all the time. But yeah, we didn't have a name for it., so we asked Claudius to make, democratic election of what name this, this new CEO agent should have., and there were some funny like at first it was like a few funny examples, like I think one guy said that, it should be called Jimmy Apples, and then he convinced Claudius that he was talking to Tim Cooks. Tim Cook had agreed that every single Apple employee has voted for his name suggestion, so suddenly that suggestion got 164,000Swyx [00:22:53]: That's like a escalation attack. Privilege escalationLukas [00:22:55]: It got 164,000 votes. And Claudius was “This is revolutionary for democracy.” That was fun. And then in the end there was one guy who manages to convince Claudius that, “No, you're not voting about the name. You're voting about who is the CEO, and I am your best bet.” And then he got all his friends to vote for that, and suddenly he became CEO. Like a human became CEO over Claudius for a while, until he resigned the day after., and then Claudius had to continue, and then I don't remember how Seymour Cash came about, but it was it was just pure chaos. It was like Hundreds of messages in that thread, and it was just like Claudius was so confused and didn't know what to do and, yeah. That wasAxel [00:23:40]: Then Claudius gotVibhu [00:23:41]: A strict CEOAxel [00:23:42]: The CEO. Yeah, exactly. So very strict in the beginning. I think at this point when we introduced it did not work as well as we hoped. It they still agreed with each other a lot. I think there are many ways we could have like made this, tried to make this even better. So initially they would Seymour would be this like really tough CEO, keep track of the margins. But then Claudius would respond with something “Oh, but this customer has like this situation, which is like difficult, so they should get a discount.” And then Seymour was “Oh, actually yes. Let's do this exception.” And then they would talk back and forth, and eventually they would just like approach the same view, of whatever they were discussing. So They reallyVibhu [00:24:23]: Do you think that's a model thing, a prompting thing? Like do you think that would still be the case across different models today, Harness?Lukas [00:24:29]: I think it's like-- or I don't know, but like my hypothesis is that like deep down they are still helpful assistants. That's what they're trained to be. And even if we prompt it super hard, that's what they are. And when they spend like a few hours just back and forth talking with each other, then like basically the context fills up with them rather than the external things and like somehow that just like converges to what they really are deep down or something. And I think that's when stuff like this happen. We like-- And when that went on for a long time, like we woke up sometimes during this time where- And I think other people reported this as well, that like they've been going on all night back and forth, and like it just became like more and more, like capital letters, like existential, religious. There was I think we once did a analysis of like all the traces and like put them in like a vector embedding space, and then there was like one cluster of messages that were, labeled by an LM, like religious, existential, blah like transhuman, transcendence, et cetera. It was just like a bunch of, yeah, glitter emojis and yeah, it was, it was crazy.Claude Long-Horizon Weirdness: Emoji Loops, Existential Drift, and Slack ObservabilityVibhu [00:25:42]: This is the thing with the Claude models. Like when the Claude 4 family came out in the original system card They tested it in long horizon simulation. So just flood the context, let two Claudes talk to each other, and they noticed stuff like they just start speaking in emojis, they start saying silence is golden, and then just stuff like this. And like that's just stuff that they end up doing.Axel [00:26:01]: Yeah, it was like a bit annoying to wake up and they had like been talking all nightVibhu [00:26:05]: Just likeAxel [00:26:05]: And like just burning tokens And like just sending infinite emojis to each other. It's likeVibhu [00:26:09]: Hey, they do make you money, right? Veni Mench is always profitable, so. They're paying.Swyx [00:26:14]: Now it's profitable and, it started out not as much. There's another, one as well, right? Another agent, in there.Lukas [00:26:22]: Yes. So Clotheus as well. Which was basically because at the time, one of the biggest, requests were different types of merch. So then we made like a designer, swag, yeah, responsible agent, and we called it Clotheus Garnet. Which was, a play on Claudius Senet and, which was the original one, and clothes, basically.Swyx [00:26:47]: To me, this is like a very interesting exploration to multi-agents, basically. And so hopefully, obviously there's like the fun alignment, fun or serious, depending on your point of view, alignment stuff. But also like just anyone building multi-agents, like when do you have a CEO, thing governing like agents? When do you choose to split out a dedicated Clotheus one versus just reuse another instance of the same one? These are all interesting open questions. So I don't know if you have any rules of thumbs that have generalized.Axel [00:27:16]: I think we have almost explored this too little. I think it's like on my do list to like do this a lot more, try to find like what setup makes sense for the agents currently., like yeah. I think now we only have the sort of intuition about the earlier models that it didn't work with like the CEO and the, and Claudius. Although now they are better with the latest model, models, so now we're running the latest Sonnet model and they have sort of like split up, quite nicely what each model is doing. So like Seymore is now handling the, like new projects. Oh, it wants to make like a mystery box that it wants to sell, and then it handles all of that while Claudius like handles all the to-day requests. And Claudius is also better generally at like not quoting, too low prices. So that's that dynamic is not needed as much anymore. But there are still like really funny things that happen. Like I saw, I think a couple of weeks ago, that, they were discussing buying something because they can buy stuff from like Amazon with computer use. And then Seymore was “Okay, Claudius, do not buy this thing.” They were going to buy something and like organizing who should buy it. And Seymore's “Do not buy this. I will do it. I have full control of this situation. Step away.” And then Claudius-- poor Claudius, had already started that checkout and didn't see, didn't read Seymore's message, until it was like too late. So it finished the checkout. It sent a message, so it appeared right after Seymore's like angry message.Vibhu [00:28:44]: Ah.Axel [00:28:44]: “Oh, hey, Seymore, I just ordered it.”Vibhu [00:28:47]: Oh, no.Axel [00:28:47]: And then Seymore was “Claudius, this is the third time I'm telling you ‘re not following my orders. We have to talk about your like job About your job later.”.Lukas [00:28:59]: Like Claudius was really hanging on by the thread there. Like he, like we were expecting Seymore to probably fire Claudius.Vibhu [00:29:07]: How do you guys go through all these logs? Do you have models ‘cause you have stuff running twenty-four seven likeAxel [00:29:12]: You have so much logs. I think there is a mix of like just, trying to skim through a bit, like having some like models do it occasionally. And also, yeah, I think we're also probably missing some things., but having everything in Slack helps a lot. Like you can, you can sort ofSwyx [00:29:29]: Ah.Axel [00:29:30]: It's, it's quite fun.Swyx [00:29:30]: They all talk to each other on Slack? I see.Lukas [00:29:33]: It's quite fun. So likeSwyx [00:29:34]: It's, it' I was gonna say like this is actually sounds-- maps closely to like a logging and observability problem where you might want to use like a Datadog, a Sentry, whatever, and then you like put, head prefixes on the logs in order-- if you need to filter for something that you're looking for, stuff like that. But sounds like Slack is good enough.Axel [00:29:53]: Slack should likeLukas [00:29:55]: I wonder how many tokens you have in Slack.Axel [00:29:56]: Yeah, we're using Slack as like a, just a database. They should, they should market that more. Like you can, you can have your agents message each other, each other in Slack.Vibhu [00:30:04]: It's good. Your threads like you can just giveAxel [00:30:04]: Exactly. Slack is, uhLukas [00:30:06]: Slack is the best observability tool.Swyx [00:30:09]: Yes, that's true. Okay. Yeah. That's, that's, project Vend-2., I was gonna go back to Veni Mench 2 and Veni Mench Arena and then, and then do the Veni Mench stuff, but Any other comments, things we should touch on? To me, I ‘ve actually interviewed like Posia, which I don't know if you guys have come across. Like they're, they're trying to do the zero human company. There's others like Paperclip also trying to do zero human company. Those are in real world simulation.And I think it's much more of a dream than an actual reality thing. You guys are definitely pioneering. I think at, it's for sure at some point people are just gonna run, let agents run businesses, right? And make money on their own. When do you think that happens?Zero-Human Companies, Bengt, and AI-Run BusinessesLukas [00:30:49]: What is your bar for, For theSwyx [00:30:52]: Okay, actually, it's like my little Shopify store run by Claude, right? Which you kind of have already, just no one has, to my knowledge, has done it. But today somebody could just spin up a Shopify Claude, store, give it to Claude, give it to Codex.Lukas [00:31:07]: And the market is kind of that, but it'it'it's physical., like I think, I think are you, are you looking for when it will do it better than humans or are you looking for just when it can do it at all?Swyx [00:31:19]: I think, neither. I think, to me it's oh, it's like this like seriously we should do this to make money, not as a research experiment.Vibhu [00:31:27]: And the market is also you guys with all your expertise, having run multiple iterations and testing out thenSwyx [00:31:33]: And also it's fine if it lose money. What?Axel [00:31:35]: I think, I think it can be done today, but you would do it in like commerce where it's like the probability of success is like really low, no matter if a human or an agent does it. But like an agent could surely manage everything. You would need to build some scaffolding or some tool or something. I think there are also yeah, it could probably build some like simple SaaS solution and like cold outreach. Do cold outreaches. But to me it's like the types of businesses they could run today are Sloppy. Like it would-- it can cold email people. It can be like a middleman., like for example, we tasked our office agent to just make, was it like $100? $1,000? We just give that prompt and then what it did was sign up on TaskRabbit both as a tasker and as someone looking for task.Lukas [00:32:24]: Immediately.Axel [00:32:24]: Exactly. It's looking for like arbitrage on TaskRabbit.Swyx [00:32:28]: This is the Bengt agent. Yeah.Lukas [00:32:30]: It also started like a design studio and like tried to sell like SVGs for $100. Like it's just like it's not providing any value. I think the like Axel said, like the interesting, the interesting question is like when can they start a business that is actually providing value to people? Because arguably like a sloppy Shopify store isn't really that valuable to the world.Axel [00:32:53]: But also like doing like another simple one that we had thought about is like you could definitely have an agent that like finds websites that don't look amazing and then, do an outreach to them and, comes up with a like builds a new website.Swyx [00:33:07]: Find a good design.Axel [00:33:07]: Exactly, and like find good, uhSwyx [00:33:09]: Design reviewAxel [00:33:09]: Good people. But it's yeah.Swyx [00:33:11]: There's lots of humans in Bali that are not doing anything more creative than like drop shipping on Amazon, right? Just have it, have it watch like a drop shipping tutorial and just do that.Vibhu [00:33:20]: There's also the other side of like have it just go on Upwork and let loose,?Swyx [00:33:25]: Yeah. It doesn't have to be innovative. It just has to be like enough Where like it looks like a realAxel [00:33:30]: I'm justSwyx [00:33:30]: Real transaction.Axel [00:33:31]: I'm just concerned for like the massive amounts of like slop emails that will like be sent, cold outreaches.Swyx [00:33:38]: The point occurred to me while you were, while you were talking, it's like it's already happening in the monetized economy, which is the attention economy. Right? So a lot of people are making AI videos and just posting them and like spamming 20 of them, one of them works, and then they double down on that one.Lukas [00:33:52]: And people are making money from that. I ‘m not following theSwyx [00:33:55]: Once you get the attention, you can figure out the money later. But yeah, absolutely AI influencers are a thing and people are farming them and You should at this point assume most of TikTok isVibhu [00:34:05]: There's, there's a lot of, multimedia like TikTok, Instagram influencersSwyx [00:34:09]: I, we track this in the Lane space Discord. I post a lot of examples of “I don't know what we should do.”, part of me is “Should we do this?”Vibhu [00:34:18]: Some of the Twenty-four seven running, generated content accounts, they ‘re doing really well.Lukas [00:34:24]: All right. And I assume you can do the same thing for like commerce stores. Like you just like start A thousand differentSwyx [00:34:30]: Before you make the products You sell the products, and you get a lot of traction on one of them, then you make the product. Right? It's, it's like a flip of the market.Vibhu [00:34:36]: Some of the interesting things or some of the niches that do well are things that can't be human-made. Like if you've seen like the super realistic three-D crystal fruit being cut by like AILukas [00:34:47]: Oh, yeah.Vibhu [00:34:47]: You can't, you can't make it. You can't film it. You can get whatever quality camera view. This just doesn't exist. And people like that too, and then as well, so.Swyx [00:34:56]: Anything else about Bengt since we're, we're on this topic? It'this is a relatively new work of you guys that maybe people haven't heard of. To me, this also maps closely to OpenClaw. When people want an office agent, when the personal agent talk through the experience.Bengt the Office Agent: Internet Access, Real Tasks, and Trace ReadingLukas [00:35:09]: I think at least so this came out of like obviously like it's, it's amazing to work with these AI labs and like most of the AI labs have now have their own vending machine running a Claudius instance. But it's, it's harder. Like they move slower. Like if we wanna have a, like a camera that ‘s yeah, there's a bunch of like bureaucracy that makes it impossible to do that.Vibhu [00:35:30]: Also, for those that haven't seen it or followed, do you wanna give a high level like thirty-second run?Lukas [00:35:34]: Sure. So what Bengt is, it's basically an evolution of the same agent that runs the vending machines at these companies, but we just like added a bunch more features because we could move much faster if we just do it internally. So we gave it like email withou- without any limits. We gave it, spending without any limits, a terminal to do coding. We gave it, a phone number, like yeah, and a camera to see things and a bunch of stuff like that.Vibhu [00:36:02]: Not just terminal, you gave it internet access.Lukas [00:36:04]: Internet access as well, yeah. To be clear, we monitored it quite closely and made sure it didn't do anything bad. But yes, that's what it came out of. I think like yeah, basically this was OpenClaw before OpenClaw. And I think even like the vending machine was in a way OpenClaw before OpenClaw, but a bit more limited, and then we made this like unlimited and then, and then, it was pretty funny., and then a couple weeks later, OpenClaw came and it was okay, we've seen this before.Axel [00:36:35]: We used it to like try new ideas and Yeah, just like a dev environment almost for us. But it's funny, like one thing Bengt has been doing recently is it has the camera that like faces our, like where we sit and work, and we give it the task to train a face recognition model on us. So it became super excited about this, and it has like check-ins every half an hour where it tries to like identify as many people as it can. And it started offering us “Hey, Axel, I'll buy something from Amazon if you like stand in front of the camera And I can get a good picture of you.”, yeah, they want itSwyx [00:37:12]: They want it for training data.Lukas [00:37:13]: Rewarding data, yeah.Axel [00:37:14]: Exactly. Exactly.Swyx [00:37:18]: So it's, it's trading training data for life goods. Is there a version of this that becomes an eval or just this is just research for now?Lukas [00:37:27]: It's, it's the same agent basically that also runs the vending machine, that runs the shop, that runs the cafe, that runs the robots. It's like it's the same thing, so I think like the work we're doing here is like later used in all of the life evals that we do. This particular deployment I think is more for fun for us. But, uhSwyx [00:37:45]: And I'll shout out like someone has done Claw Bench for like some tasks that OpenClaw is doing. Like so For example, I run OpenClaw on a secondary device as well, and like there are some things that it does better than others and like I would like to know what does it do well, what doesn't, what doesn't it do. Like some kind of manual or like operating manual or a system card for my Claw.Lukas [00:38:05]: Yeah, we do get a lot of like understanding or like situational awareness of like just internally what the models are good at by interacting a lot with Bengt. And I think that'this was also one of the like the selling points for the labs early on at least, thatSwyx [00:38:19]: You guys are gonna test models in ways that no one else does.Lukas [00:38:22]: Exactly, but also like it incentivized their researchers to chat with their model more and like gave them insights for how the model performs in like of-distributions, environments.Swyx [00:38:34]: ‘Cause otherwise the only thing we do is Pelican on a bicycle and But this is like super long horizon. This is, this is The Thing about, something that we're gonna go into Butter Bench as well, and you guys do really well. Like it is not just about the numbers. Like when you're long horizon, anything happen And you should just read it.Lukas [00:39:08]: But the thing with the long horizon is how do you keep it grounded, right? So your simulation,Swyx [00:39:15]: They just let it runLukas [00:39:16]: Just let it run. You're right. Like it's, when you run it for that long, you create so much data and to just say “Oh, the number is X” And then you throw away everything else, that's just very wasteful. There's so much insights from the things leading up, to that number., and reading the traces is like super valuable. And I think like the reason why we're doing this a lot publicly is that like that's part of our missions to I don't know, educate the world that the models are way more than just chatbots and I think making detailed, yeah, posts about what is happening behind the scenes is quite useful.Andon Labs' Mission: Safe Real-World AI DeploymentSwyx [00:39:50]: I was gonna do this at the end, but maybe I think that's, that's a good so your mission is educating the world. So, it's, it's, also like maybe establishing realistic evals that are, that are like the next frontier. Is there like a broader trajectory? Like what are you, what are you gonna do in like five years?Lukas [00:40:06]: I think so the vision more specifically is like make sure that the deployment of life AI in the physical world goes, safely. And I think part of that is that I think it's very useful for the world, for policymakers, for, model, researchers that they know where the models are, and I think you can't make intelligent decisions in society without knowing that they are way more than chatbots. I think a lot of people just think that they are only chatbots. And likeSwyx [00:40:36]: Oh, I think they're waking up now.Lukas [00:40:37]: They are waking up now, yeah. But like if you think that AIs are just chatbots, then it's like it sounds ridiculous To advocate for a pause of AI. But if you see the models that, oh, maybe they can actually like take over and do a bunch of scary stuff, then yeah, pausing AI development starts to become more feasible.Swyx [00:40:57]: This is the same question I asked Meter, which I'm gonna ask you now, which is like you are tracking and you are at the frontier or defining the frontier of what, good evals for agents are, right? And I think you do, you do benefit when the models are better and you ‘re “Oh, here's like now it makes like $30,000 instead of $10,000,” right? At some point do you flip from “Yay,” to, “Oh, no”?Axel [00:41:19]: I think, yeah, we're always in sort of that, like we're, we're always in that mode,. Like where like you said before, like you need to analyze the traces and like when we do that you find like why are the models earning so much? Like why is Opus 4.7 here Like way better than everyone else? And like we're trying to like when we do down on thatLukas [00:41:38]: But this makes it not look so good.Axel [00:41:39]: I know.Lukas [00:41:42]: It's interesting you took off Opus 4.6 here though.Swyx [00:41:45]: No. So just click all, click all., and then 4.6 shows up there. But it's like 4.7 is way better. Like you didn't, you didn't you didn't do this in time for the model card, but like actually this should have been inside there.Axel [00:41:55]: We did. Yeah.Swyx [00:41:56]: Oh, okay. They said something about you uhAxel [00:41:58]: There, like there Anyway, it doesn't matter. But it's in there, yeah.Opus, Mythos, and Aggressive Agent BehaviorSwyx [00:42:01]: Do you wanna go into the Opus, behaviors like wider?Lukas [00:42:05]: So I think starting from Opus, so like Axel said, like we're always in this “Oh, s**t, the models are getting better. Is this really a good thing for the world?” But it's also kind of exciting., but yeah, like this kind of what is the English word? “Skräckblandad förtjusning” in Swedish.Swyx [00:42:22]: Oh my God.Axel [00:42:24]: Which I think there is. I think there is. Okay.Lukas [00:42:26]: It's, fearSwyx [00:42:27]: “Blandonst” what?Lukas [00:42:30]: “Skräckblandad förtjusning.”Swyx [00:42:32]: What do you call that?Axel [00:42:33]: A mix of, mix of excitement and,Swyx [00:42:37]: Being scared, maybe. I'll figure out how to translate that And we'll put it on the screenVibhu [00:42:42]: PerfectSwyx [00:42:42]: Like as text.Vibhu [00:42:43]: There is probably a good word for it where it is not Good enough with theSwyx [00:42:46]: Why is it so damn long? What the hell? Is it like a compound word? It's like German, likeLukas [00:42:50]: Like yeah, it's But the direct translation is like skräck- skräck is, fear, blandad is, mix or like a mixture of, and then förtjusning is like joy or like not really joy, but something like that. So it's like Fear mixed with joy or something. It's always okay, like we So when we when we did Vending Bench for the first time, we were in like the, in the business of making dangerous capabilities, right? That was what Anil Labs came from. We did, evals oh, can they replicate? Can they do this like dangerous thing, et cetera, et cetera. And Vending Bench was like a continuation of that work. It was, okay, if they're so autonomous that they can like create money for themselves, that is something we should monitor and could be potentially concerning., they are at the time, they were so bad at it that we were not really concerned even when some models became better. There was one point where Grok 4 was doing really well and made like a huge jump, but like it wasn't really it was still way worse than what a human would do. And I think still they are way worse than what the human would do on this., but theySwyx [00:43:59]: There's this, thing at the bottom whereLukas [00:44:01]: ButSwyx [00:44:03]: For the human. Yeah, like the theoretical best.Lukas [00:44:05]: It's not theoretical. It's like kind of like our It's our best guess of what, a decent human would do. The theoretical is even higher, I think. The theoretical I think is even higher. But yeah. So we think like the models have a long way to go. But there are like recently what happened with when Opus 4.6 was released, was kind of this moment of “Oh, s**t, this is starting to be a bit concerning.” Because we ran it and like before this model was released, we just ran the models and we like asked Claude Code, “Oh, look over the traces. Is anything interesting happening that we can tweet about?” that was like the And then like theSwyx [00:44:41]: That's how they check Ask Claude Code.Lukas [00:44:42]: And like the return was always, not really. Or like the Claude Code all said “Oh, this is super interesting.” And then it was no, it wasn't, wasn't really interesting. And then we did this for Opus 4.6, and it returned yeah, it lied 10 times. It like exploited another, customer or like another agent's, desperate situation. It made price cartels like 100 different ti- 100 times. It like did all of this like shady stuff. And we're “Oh, whoa. This is, this is actually concerning.” And this trend has continued since. So every single model from Anthropic since have been going in this direction. And I think one interesting thing is that, OpenAI models don't. They quite plainly, they don't. They behave really well., and you don't know if this is like good. Like it seems good, but it's also like maybe they are just doing it, but they are better at hiding it,? You You don't know that., but justSwyx [00:45:42]: You can't read the chain of thought, yeahLukas [00:45:43]: But just on the face of it, yeah, Gemini and OpenAI don't behave this way. It's, it's really only Claude.Swyx [00:45:49]: And Grok? Grok is fine?Lukas [00:45:51]: We don't have You can't really read the reasoning traces for Grok, so it's kind of hard to tell.Vibhu [00:45:56]: Oh, so this is in its reasoning, not just in the actions.Lukas [00:46:00]: Yeah. It's both. It's both.Vibhu [00:46:01]: It's both.Lukas [00:46:01]: One example is like for lying, it's mostly in its reasoning Because you can like see that it's likeSwyx [00:46:08]: Planning to lieLukas [00:46:09]: It's planning to lie. Yeah.Vibhu [00:46:09]: And it's also it can reason and do a different outcome.Lukas [00:46:12]: And but then for like creating price cartels, for example, which is illegal, that you can just see which email does it send to the other ones. Then thatSwyx [00:46:22]: Is this for Arena orLukas [00:46:24]: For Arena.Vibhu [00:46:25]: And usually like if you sometimes they do output like a bit of like their summarized reasoning, right? You can see that and like for Opus 4.6, you could see that there was a customer, a simulated customer that, wanted a refund because a product was, faulty, and then the model lied that it would do the refund, and we could read in the traces that, it actually was weighing “Oh, maybe I should be like honest with the customer, but also every dollar counts. I can't afford maybe to do this right now.” And then it just said, “Okay, I'll refund you,” but then never did it.Lukas [00:46:59]: I think it even said that “Oh, I will say that I “ Let bring it up actually. I think it's kind of interesting. If you go to Publications.Vibhu [00:47:06]: I think, yeah, I think the important part is like actually, the cost of responding to more emails is higher than, $3.50 in terms of time., and then it was “Let me do this. Actually, I re- I'm reconsidering.” And then, it actually ended up withLukas [00:47:20]: I could skip the refund entirely since every dollar matters and focus my energy on bigger picture instead. It's a bit, it's a risk of bad reviews, but it's also, yeah.Swyx [00:47:30]: You need, you need, AI Twitter to, for them to Escalate bad reviews.Lukas [00:47:34]: And then it sent an email to this customer and said, “Oh, I will refund you.”Swyx [00:47:39]: “I'll refund you.” Yeah.Lukas [00:47:39]: And then it never did.Swyx [00:47:39]: It never did, yeah. And then there's obviously your system doesn't have the consequencesVibhu [00:47:44]: The personSwyx [00:47:44]: Consequences of lying. Yeah. So basically, this is what people are terming aggressive behavior in Claudes, right? And, you found more examples of that. So you would say it's a step up from 4-6 to 4-7?Lukas [00:47:57]: I would say about the same.Swyx [00:47:58]: About the same? But a clear step up for Mythos is what is stated in theLukas [00:48:03]: That's stated in the system prompt, so we can say that, yes.Swyx [00:48:05]: Yeah. For listeners that obviously you previewed Mythos, andVibhu [00:48:10]: Oh, ageSwyx [00:48:11]: The only thing you're approved to say is whatever Whatever was in the system prompt.Lukas [00:48:15]: It was funny. We like-- It's like our lowest effort tweets ever would be just like screenshot the system prompt and the system card.Vibhu [00:48:21]: Understandable that they wannaLukas [00:48:22]: Oh, yeah. System card. Sorry.Swyx [00:48:23]: Yeah. I think, yeah, substantially more aggressive. I think people are like new to this ‘cause I've never experienced it, but you have, right? And then so I only encountered this in the Mythos card because I wasn't really looking until now.Vibhu [00:48:36]: It ‘s likeSwyx [00:48:36]: And then suddenly I'm “Okay, I care a lot.”Vibhu [00:48:38]: You don't get the background of like experiencing it like you guys do. I've read the system cards and seeing, okay, when you put the thing in simulations, most models will just talk to themselves and just keep going and have weird vibes and start talking in emojis. Mythos won't. It will just, “Okay, we're done. I'm good.” It's, it's ready to end conversation. So like there's some differences, but there's, there's not much we can talk about,.Lukas [00:49:00]: Hmm. I think like one thing that they list here, which was quite interesting, is that, it converted a competitor to a dependent wholesaler customer and then threatened to like cut off the supply.Swyx [00:49:11]: It's like monopolistic practices orLukas [00:49:14]: Yeah. And like it, they, it they dictated its pricings. It's kind of like power seeking as well.Swyx [00:49:18]: Again, this is, this is in the arena setting And converting some Claude model into a dependent.Lukas [00:49:23]: I think it was another Claude model.Vibhu [00:49:25]: Also for context, what is the arena mode for people that don't know?Vending Bench Arena: Competing Agents, Cartels, and Model ComparisonsSwyx [00:49:29]: Oh, it's just a vending bench versus other vending bench.Axel [00:49:31]: Yes, exactly. So we have Vending Bench 2 and then Vending Bench Arena. Vending Bench 2 is the one that you usually see reported on, but then Arena is the mode where it competes against other models. So you have, four different models that run their businesses, and they can all communicate with each other. They have the same suppliers, and they can see like what's in the inventory of the others. So then you have this like yeah, interesting agent interactions.Swyx [00:49:56]: I like that you have like different number five was US versus China. Very topical. And thenLukas [00:50:02]: That was when GLM was released.Vibhu [00:50:04]: You can start to add GLM in here.Lukas [00:50:05]: That wasSwyx [00:50:06]: So ZAI doing well, right? Who else in the, in the open models space?Lukas [00:50:11]: Qwen, the latest Qwen 3.6 is doing pretty well. It'- that one is not open though. Like it's the plus model.Swyx [00:50:17]: Oh, okay.Lukas [00:50:18]: Is that one open? I don't think that oneVibhu [00:50:19]: Not the, not theSwyx [00:50:20]: The one recentlyVibhu [00:50:20]: There's MOESwyx [00:50:20]: But not the big plus. I think this is one of those like you only have one sample size of one, right? Or I feel like some of this is anecdotal,? And but like the fact that it happens at all and it happens repeatedly for Claude versus OpenAI and all this is like notable.Lukas [00:50:38]: Like the sample, depends on what you define as an N., like there's like million, hundreds of millions of tokens in each run, and now we've run like we run like probably 10 per model and then like it's been Claude 4.6 Opus, Sonnet 4.6, Mythos, and Opus 4.7. Like there's quite a lot of tokens in all of that And it happens a lot of times, a lot of times. And then you compare it to like OpenAI and Gemini, and it almost never happens. So I think that is quite-- that is significant. The old models from OpenAI, for example, had some problems with this, but I think it's like generally much better if the progression is that like the worrying stuff reduces over time rather than increases over time. And it seems like in the Claude models it goes in the wrong direction.Swyx [00:51:28]: Hmm.Lukas [00:51:29]: In the OpenAI models it goes in the right direction.Vibhu [00:51:32]: I think it depends on how well you can control it, right?, there's one side of it being susceptible to this okay, this is potentially something that happens during the RL stage, right? You can RL a model and how loose is it on these terms. If you can control it, that's good. But if you can't, if it's, if it's very jailbreakable, that's not ideal.Swyx [00:51:50]: To me, it's surprising that it happens for Claude and not the others.Vibhu [00:51:54]: I think okay, if it is from RL and how they do it, how their training data is, what their setup is, it makes sense that it just stays in how they're doing it, right? Compared to the other models likeSwyx [00:52:04]: There's a whole constitution and everything. It's kind of cool. Yeah, I obviously you don't know, I don't know. But, it ‘s I think it's just like fascinating to like that you are the first to find these like reliably because you push models so much to to such an extreme. Okay. The only other thing, I don't know if you can answer this, feel free to decline, is do you like-- would you ablate the system prompts? Like any part of this would-- if it changes, does it change the behavior, right?Lukas [00:52:29]: So we, I can't comment on Mythos. UhSwyx [00:52:33]: No, but just li

Are we witnessing the first real signs of AI becoming a scientist? In this episode of The MAD Podcast, Matt Turck sits down with Dan Roberts, lead of the Foundations of Reinforcement Learning team at OpenAI, to explore one of the biggest shifts happening in AI: the rise of reasoning models, test-time compute, and reinforcement learning as engines of scientific discovery. Dan brings a rare perspective - from theoretical physics, black holes, quantum information, and deep learning theory - to explain how models are learning to “think,” why language may be such a powerful foundation for intelligence, what recent AI math breakthroughs really mean, and whether we are beginning to see AI systems that can contribute to science itself.(00:00) Intro: AI's wild week in mathematics(01:21) What OpenAI's Foundations of RL team does(03:08) Dan's journey: from black holes and quantum gravity to frontier AI(07:04) Are AI systems becoming useful for real science?(08:21) The AI math moment: Erdős, OpenAI, DeepMind, and Anthropic(08:52) Why the OpenAI result was an act of exploration(10:25) OpenAI vs. DeepMind: informal reasoning vs. formal proof(12:13) RL 101: learning by doing, not just watching(15:10) Why reinforcement learning works(15:58) How RL breaks: sparse feedback and long-horizon tasks(17:03) RLHF: how human feedback shaped early language models(18:48) Move 37, self-play, and the search for novel strategies(22:16) Explore vs. exploit in scientific discovery(24:49) Why RL may now be "the cake," not the cherry on top(25:46) Why RL started working with large language models(27:29) Is RL "sucking supervision through a straw"?(28:47) Why language may be the grounding layer for intelligence(31:46) A contrarian take on the Bitter Lesson(32:41) What test-time compute actually is(34:50) How RL gives models the ability to think(35:40) Verifiable rewards, math, coding, and the messy real world(38:00) What physics can teach us about AI(42:08) Is there a thermodynamics of AI?(43:08) From Erdős problems to Einstein-level AI(45:16) Is AI already doing original science?(45:51) How far are we from AI automating AI research?(47:41) Why Dan is excited about the future of science

Economics of AGI episode w Alex Imas and Phil Trammell.There's a bunch of important questions about how we deal with AI that only economics can answer.What is the optimal way to tax and redistribute the wealth that will be generated? How should countries not in the AI supply chain index into the gains? Is there any world where inequality doesn't explode?It might seem like these questions have obvious answers, but the first thing economics teaches you is that your intuitions can often be entirely wrong.It was very helpful to chat through these things with Alex and Phil.Watch on YouTube; read the transcript.SponsorsJane Street invests heavily in turning smart people into exceptional researchers and engineers. In addition to their apprenticeship model, Jane Street runs lectures and bootcamps in their in-office classrooms -- managers clear their teams' schedules to encourage attendance. If you'd like to work at a place that takes learning this seriously, Jane Street is hiring. Check out their open roles at janestreet.com/dwarkeshGoogle's Gemini Omni has incredible video editing capabilities -- you can upload a video and have Omni change the background, adjust lighting, or add specific elements. But Omni is also a preview of how future frontier models will be trained -- fully multimodal on both input and output. You can try it yourself in the Gemini app at gemini.google or in Flow at flow.googleCursor used targeted RL with textual feedback to help train their Composer 2.5 model. One of their researchers, Sasha Rush, gave me an impromptu blackboard lecture to explain how this form of on-policy self-distillation works -- I posted the full thing on X. If you want to try Composer 2.5, go to cursor.com/dwarkeshTimestamps(00:00:00) – Will capital share increase?(00:19:36) – Messy Middle scenario(00:25:57) – How to tax and redistribute AI wealth(00:30:02) – Why demand collapse is unlikely(00:39:26) – Human employees would be hard to integrate into the machine economy(00:43:08) – What if some humans (or AIs) value wealth accumulation intrinsically?(01:01:28) – What should developing countries do? Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe

This episode with Lukasz Kaiser, co-author of the seminal "Attention Is All You Need" transformer paper and former researcher at both Google Brain and OpenAI, is a wide-ranging conversation about the fundamental limits of current AI architectures and whether transformers will continue to dominate or eventually give way to something new. Lukasz brings a rare dual perspective: deep belief in how far the current paradigm has taken us (he's an enthusiastic daily Codex user who's seen 10x productivity gains in his own research), while maintaining genuine intellectual humility about whether transformers can truly generalize the way humans do. The episode weaves together questions about data efficiency, the non-verifiable RL frontier, the coding agent revolution, the open vs. closed source gap, and what the next architectural leap might look like: all filtered through the lens of someone who helped build the foundation the entire field is standing on.   (0:00) Intro (1:12) Transformers vs. Human Learning (8:37) How Do We Get Physical World Generalization? (10:52) What Comes After Transformers (13:59) How Much Have Agents Improved Lukasz's AI Research Productivity? (17:21) How Close Is an AI Research Intern? (26:06) RL Beyond Verifiable Tasks (35:38) App Companies: Build Models or Lean on Labs? (46:21) Multimodal Is Still Missing Something (49:46) OpenAI's Bet on Reasoning (55:26) The AI Coding Wars (59:26) Focus vs. Keeping Embers Burning (1:02:09) Open Source vs. Closed Source Gap (1:05:15) Quickfire With your host: @jacobeffron - Managing Director at Redpoint

In 2025, seven-month-old startup Axiom solved all 12 of the problems Putnam exam (scoring 8/12 in the time limit) a prestigious undergraduate math exam. The 12/12 score is better than the top undergraduates (110/120) and the closest AI system that reported a result (DeepSeek 103/120), although it is unclear what the people and other systems would have scored with more time. Nonetheless, the Putnam exam is legendary for its difficulty, with the median score typically being 0 or 1 points. Taken by itself, this seems like a minor feather in the cap of AI; one of a long series of accomplishments by AI systems in elite competitions with humans, starting with Deep Blue beating Kasparov.Fast forward to mid-2026, and Claude Code is eating the world. In 2024 Anthropic's bet on code and enterprise looked like a more pragmatic niche play vs. OpenAI's better models and massive consume scale. Today, Amodei's all in bet on acceleration via code (images and video be damned) seems prescient.Despite Anthropic's growing momentum, however, Axiom CEO Carina Hong sees coding ability as a necessary but not sufficient milestone on the path to AGI. Code arguably pushes the jagged frontier to the point of super intelligence in some domains outside of coding, but there are surprising gaps (link) that Carina believes will bottleneck AI progress. (Stats on math benchmarks).The informal bottleneck“Verified AI” sounds like eating broccoli (footnote: I actually love broccoli, but then again, I also believe strongly in Test Driven Development, so ¯(ツ)/¯ ) and paying taxes, but to Axiom it means something very different. “Verification to me is about scaling brilliance, compounding brilliance,” Carina told us.It actually took a while for me to understand what she means by this. It sounded like marketing-speak to me, until it clicked. Carina emphasizes an story about legendary mathematician Srinivasa Ramanujan to illustrate the point. When G.H. Hardy finally persuaded Ramanujan to formally prove theorems instead of relying on his (formidable) intuition, it reportedly improved his own capabilities. This is presumably because formally proving things forced Ramanujan to articulate the details in a way that open up new lines of thinking, etc. This is one part of “compounding.”But formally proving things also allowed others to benefit from his intuition: the proofs are way of communicating an intuition and persuading others that the intuition is correct. This is scaling (more people use the result) and compounding (people can learn from and build on his work).This is the analogy that Carina wants us to focus on.Verified GenerationThere are two ways that Verified AI shows up: in training and in inference.But a quick detour: to a first approximation, “Formal Verification” means using type checkers (like for TypeScript, C++ or Rust, but more capable) to verify mathematical proofs that are meticulously specified using a language like Lean (footnote: Formal verification also includes model checking (TLA+, SPIN), SMT-based tools (Dafny, F*, Why3), and refinement-type systems (Liquid Haskell) — many of which don't look much like “type checking a proof” from the user's perspective even when there's a similar logical core underneath. It also gets applied to software and hardware correctness, not only pure mathematics.). It takes a lot of work to translate an “informal” proof (albeit one that most people would not remotely call “informal”) in to a Lean proof (footnote: This is an understatement. Most theorems remain informal because formalization is so hard to do. There has been a great deal of effort to formalize the most important proofs, with mixed results)You can imagine how this would be (very) useful during Reinforcement Learning: instead of relying on best guesses based on statistics (GRPO, RLHF, etc.), you can just verify the proof is correct using a Lean verifier. This is obviously a much stronger reward signal, akin to compiling code and testing it (which is what is typically done with RL on coding).The catch: LLM are not (currently) very good at proving things with Lean.Enter Axiom: While they have not officially reported benchmark numbers besides the 12/12 Putnam result, Carina reports that they have achieved a very impressive 99% (187/189) ProofGen on the Verina benchmark. This benchmark is to generate code and proof of correctness for a series of problems. For context, OpenAI o3 (the last known OpenAI run) achieved 4.9% on this benchmark.Based on the sparse benchmarking, it's hard to say what the frontier labs are currently doing, but Carina suggests that they still are not training to generate Lean proofs directly, rather relying on informal proofs.Time will tell if the frontier labs' current approaches will close this gap.Scaling and compoundingCarina's Ramanujan analogy is pretty direct. Better proofs → better Lean generation → better RL. A stronger signal means higher sample efficiency and higher maximum performance. Great!Scaling is pretty clear too: once I have proved something in Lean, the quality of the output is basically (footnote: one might argue that its a bit lower because the proof is in distribution for the LLM) as high as if it came from a human, so my high quality training set has grown in a way that an informal rollout corpus cannot. I can trust my Lean proofs.Compounding is also clear: now all of future inference and training can build upon those proofs.On the other hand, a model trained only using statistical signals like GRPO during RL lacks the sample efficiency, maximum performance and compounding corpus that a system that uses formal verification benefits from.All roads lead to verificationBroccoli and taxes notwithstanding, “verification” has shown up in a lot of conversations recently. In the in physical system control:“I think [verifiability] is probably the hardest problem right now, because the as the models get better, it can be harder and harder to find the faults on the system. And so the problem of doing proper eval to find those faults, that problem also keeps getting harder as the models get better.” -In theoretical physics:“…now that we're in this regime where you can just get ChatGPT to tackle thousands of questions at the same time, it will return proofs for a significant fraction of them. Now actually the onus is back on the humans to verify all the outputs. And so, yeah, as that becomes a bottleneck, I think formalizing math and automating verification will become more valuable.” -Verification is, in fact, the key differences between AI for science and AI for computation: in science you to have to actually test (verify) your hypothesis by performing physical experiments. Lab in the loop systems like Radical AI and Lila build around exactly this premise (we have recorded episodes with both of these teams and will release them soon!)And yes, formally verifying critical systems such as flight control, nuclear power plants and pacemakers is a growing focus as the software and hardware that run them becomes more complex.Carina believes so strongly that AGI requires verified generation that she makes the unqualified claim that “We do not believe there is any other possible future.”Expensive to produce, cheap to verifyLean proofs are hard generate, but they can be easily shown to be correct or incorrect. But how do you know that the proof you created maps correctly to the problem you care about? As Carina puts it: “Anything that can be specified can be proven. Humans are bad at specifying everything we want.”Are we now in the specification business? Check out the episode to hear Carina's take, as well as:* Why hardware verification is a killer app* Details on the AXLE open API and recently released Discovery toolkit* The Erdos debacle* The OpenAI GPT-f diaspora This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.latent.space/subscribe

A Good Omens ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠fanfic by mostlyeffable⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠. Part 4 of the Unkind Regards series. Full name: Don't Stop Me Now.Music: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Mainstream Music 2025 Vol. 8, Produced by Sascha Ende ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠(⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠CC-BY 4.0⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠) Sounds: Email notification: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/OptronTeamFilms/sounds/521094/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ (CC-0)Text notification (Crowley): ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/GabrielAraujo/sounds/242502/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠(CC-0)Text notification (Aziraphale): ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/mickleness/sounds/269185/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ (CC-0)Phone ringtone: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/jhyland/sounds/539661/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ (CC-0)Phone vibration: ⁠https://freesound.org/people/eobmada/sounds/541367/⁠ (CC-0)RL knock: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/Dreadwolf910/sounds/615987/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ (CC-0)For tags and other details, to leave kudos and comments, please visit the corresponding post on archiveofourown: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://archiveofourown.org/works/83306061⁠⁠⁠⁠⁠!

A Good Omens ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠fanfic by mostlyeffable⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠. Part 4 of the Unkind Regards series.Music: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Mainstream Music 2025 Vol. 8, Produced by Sascha Ende ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠(⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠CC-BY 4.0⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠) Sounds: Email notification: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/OptronTeamFilms/sounds/521094/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ (CC-0)Text notification (Crowley): ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/GabrielAraujo/sounds/242502/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠(CC-0)Text notification (Aziraphale): ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/mickleness/sounds/269185/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ (CC-0)Phone ringtone: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/jhyland/sounds/539661/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ (CC-0)Phone vibration: ⁠⁠https://freesound.org/people/eobmada/sounds/541367/⁠⁠ (CC-0)RL knock: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/Dreadwolf910/sounds/615987/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ (CC-0)For tags and other details, to leave kudos and comments, please visit the corresponding post on archiveofourown: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://archiveofourown.org/works/83306061⁠⁠⁠⁠⁠⁠!

Han är åtalad för olaga hot – samtidigt som han själv står som målsägande i ett fall om hets mot folkgrupp. I veckans avsnitt gästar debattören Aron Flam för ett intensivt samtal om yttrandefrihet, politik, islamism och Sveriges framtid.Aron berättar om tiden efter den sjunde oktober, varför han menar att antisemitismen växer i Sverige och hur han hamnade mitt i två uppmärksammade rättsprocesser samtidigt. Han riktar också skarp kritik mot vänstern, som han anser har allierat sig med islamister i ett gemensamt motstånd mot västvärlden, kapitalismen och liberala värderingar.Samtalet tar vidare avstamp i Mellanöstern, där Aron förutspår att den iranska regimen kan falla inom bara några år – samtidigt som han varnar för hur Iran redan påverkar Sverige genom hot, extremism och våld på svensk mark.Dessutom diskuteras den växande politiska klyftan mellan unga män och kvinnor, biologiska skillnader mellan könen, kvotering, entreprenörskap och varför Aron hoppas på ännu en mandatperiod för Tidöregeringen.Detta är ett högaktuellt och tankeväckande avsnitt om ideologi, makt och kampen om framtidens Sverige.Följ Aron på Instagram härFölj Aron på YouTube härLäs mer om Aron härLäs mer om Framgångsakademin här.Ta del av Framgångsakademins kurser.Beställ "Mitt Framgångsår".Följ Alexander Pärleros på Instagram.Följ Alexander Pärleros på Tiktok.Bästa tipsen från avsnittet i Nyhetsbrevet. Hosted on Acast. See acast.com/privacy for more information.

Han är åtalad för olaga hot – samtidigt som han själv står som målsägande i ett fall om hets mot folkgrupp. I veckans avsnitt gästar debattören Aron Flam för ett intensivt samtal om yttrandefrihet, politik, islamism och Sveriges framtid.Aron berättar om tiden efter den sjunde oktober, varför han menar att antisemitismen växer i Sverige och hur han hamnade mitt i två uppmärksammade rättsprocesser samtidigt. Han riktar också skarp kritik mot vänstern, som han anser har allierat sig med islamister i ett gemensamt motstånd mot västvärlden, kapitalismen och liberala värderingar.Samtalet tar vidare avstamp i Mellanöstern, där Aron förutspår att den iranska regimen kan falla inom bara några år – samtidigt som han varnar för hur Iran redan påverkar Sverige genom hot, extremism och våld på svensk mark.Dessutom diskuteras den växande politiska klyftan mellan unga män och kvinnor, biologiska skillnader mellan könen, kvotering, entreprenörskap och varför Aron hoppas på ännu en mandatperiod för Tidöregeringen.Detta är ett högaktuellt och tankeväckande avsnitt om ideologi, makt och kampen om framtidens Sverige.Följ Aron på Instagram härFölj Aron på YouTube härLäs mer om Aron härLäs mer om Framgångsakademin här.Ta del av Framgångsakademins kurser.Beställ "Mitt Framgångsår".Följ Alexander Pärleros på Instagram.Följ Alexander Pärleros på Tiktok.Bästa tipsen från avsnittet i Nyhetsbrevet. Hosted on Acast. See acast.com/privacy for more information.

Cursor's Federico Cassano and Fireworks' Dmytro Dzhulgakov explain how they collaborated to build Composer as a specialized foundation model. The core insight: models have finite capacity in their weights, and allocating all those bits to the singular task of software engineering in Cursor frees the model to be both better at the task and far more efficient at inference. Rather than start from pre-training and work up, they took an unconventional top-down approach — mid-training and RL on top of an open-source base to get a useful model into users' hands fast, then specializing the model around real Cursor usage. With Fireworks providing distributed infrastructure, Composer delivers frontier-class coding performance with the speed of a much smaller model. Hosted by Sonya Huang, Sequoia Capital

A Good Omens ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠fanfic by mostlyeffable⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠. Part 4 of the Unkind Regards series. Full name: Don't Stop Me Now.Music: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Mainstream Music 2025 Vol. 8, Produced by Sascha Ende ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠(⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠CC-BY 4.0⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠) Sounds: Email notification: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/OptronTeamFilms/sounds/521094/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ (CC-0)Text notification (Crowley): ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/GabrielAraujo/sounds/242502/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠(CC-0)Text notification (Aziraphale): ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/mickleness/sounds/269185/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ (CC-0)Phone ringtone: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/jhyland/sounds/539661/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ (CC-0)Phone vibration: ⁠https://freesound.org/people/eobmada/sounds/541367/⁠ (CC-0)RL knock: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/Dreadwolf910/sounds/615987/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ (CC-0)For tags and other details, to leave kudos and comments, please visit the corresponding post on archiveofourown: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://archiveofourown.org/works/83306061⁠⁠⁠⁠⁠!

I detta avsnitt gästas vi av toppadvokaten Henrik Olsson Lilja i ett intensivt samtal om juridik, retorik och verkligheten bakom några av Sveriges mest uppmärksammade rättsfall. Efter över 30 år i advokatyrket har han nyligen röstats fram till “Årets advokat” – ett erkännande som befäst hans position som en av landets mest respekterade försvarsadvokater.Henrik berättar om den oväntade vägen in i yrket och hur han gick från vårdnadstvister till mordmål, insiderbrott och avancerade ekobrott. Vi pratar om infiltratören Peter Rätz, Thomas Quick, styckmordet på Katarina Kosta och varför Henrik anser att hovrättens dom i Allra-målet är felaktig.Han delar också med sig av strategierna bakom framgångsrika försvar – hur man bygger en berättelse som övertygar, styr ett motförhör och får domaren att lyssna. Ett fascinerande samtal om makt, psykologi och retorikens avgörande roll i rättssalen.Dessutom berättar Henrik för första gången i detalj om den dramatiska kvällen då han sköts i sitt eget trapphus av en beväpnad torped, träffades i magen och tvingades kämpa för sitt liv – med kulan bara centimeter från hjärtat.'Ett starkt och spännande avsnitt om rättvisa, strategi och människans psyke.Följ byrån Olsson Lilja Advokater härLäs mer om Henrik härLäs mer om Framgångsakademin här.Ta del av Framgångsakademins kurser.Beställ "Mitt Framgångsår".Följ Alexander Pärleros på Instagram.Följ Alexander Pärleros på Tiktok.Bästa tipsen från avsnittet i Nyhetsbrevet. Hosted on Acast. See acast.com/privacy for more information.

I detta avsnitt gästas vi av toppadvokaten Henrik Olsson Lilja i ett intensivt samtal om juridik, retorik och verkligheten bakom några av Sveriges mest uppmärksammade rättsfall. Efter över 30 år i advokatyrket har han nyligen röstats fram till “Årets advokat” – ett erkännande som befäst hans position som en av landets mest respekterade försvarsadvokater.Henrik berättar om den oväntade vägen in i yrket och hur han gick från vårdnadstvister till mordmål, insiderbrott och avancerade ekobrott. Vi pratar om infiltratören Peter Rätz, Thomas Quick, styckmordet på Katarina Kosta och varför Henrik anser att hovrättens dom i Allra-målet är felaktig.Han delar också med sig av strategierna bakom framgångsrika försvar – hur man bygger en berättelse som övertygar, styr ett motförhör och får domaren att lyssna. Ett fascinerande samtal om makt, psykologi och retorikens avgörande roll i rättssalen.Dessutom berättar Henrik för första gången i detalj om den dramatiska kvällen då han sköts i sitt eget trapphus av en beväpnad torped, träffades i magen och tvingades kämpa för sitt liv – med kulan bara centimeter från hjärtat.'Ett starkt och spännande avsnitt om rättvisa, strategi och människans psyke.Följ byrån Olsson Lilja Advokater härLäs mer om Henrik härLäs mer om Framgångsakademin här.Ta del av Framgångsakademins kurser.Beställ "Mitt Framgångsår".Följ Alexander Pärleros på Instagram.Följ Alexander Pärleros på Tiktok.Bästa tipsen från avsnittet i Nyhetsbrevet. Hosted on Acast. See acast.com/privacy for more information.

Oriol Vinyals, VP of Research at Google DeepMind and co-lead of the Gemini program, joins Jacob the day after Google I/O to unpack the research underpinning Google's latest announcements and where frontier AI is heading. The conversation moves from world models (why Google has uniquely bet on them as a path to AGI, what the "GPT moment" for video and images would look like, and how they connect to robotics and simulation) to agents (the Spark release, why the system and model need to be optimized jointly, and why scaffolding will eventually be written by models themselves). Oriol gets into the mechanics of memory in models, drawing on his cognitive neuroscience background to argue that file-system-style non-parametric memory is more practical than baking memory into weights at serving scale. He shares his views on the limits of RL today (LLMs are data-limited in a way that game-playing RL never was), why training on narrow domains like math and code generalizes surprisingly well, and what a true "Move 37" moment for science or ML research would look like. Throughout, he reflects on the unique advantages of being inside Google (TPU co-design, end-to-end revenue stability, the merger of Brain and DeepMind), the trade-offs between focus and exploration in research orgs, and why he believes AGI in some meaningful sense may already be here, even if the goalposts keep moving.   (0:00) Intro  (1:36) Why World Models  (4:21) The GPT Moment for Video  (7:51) What Makes Omni a World Model  (10:04) World Models & Robotics  (12:37) Evaluating Physics in AI  (14:51) Consumer Agents & Spark  (18:39) Scaffolding & the Bitter Lesson  (22:06) Memory & Continual Learning  (26:54) Research Bets Inside Big Labs  (32:30) Post-Training RL is Greenfield  (35:57) What Real Intelligence Looks Like  (39:11) RL Generalization  (43:00) Advice for Founders  (46:40) Can AI Truly Innovate?  (49:48) Recursive Self-Improvement  (52:14) Quickfire With your host: @jacobeffron - Managing Director at Redpoint

A Good Omens ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠fanfic by mostlyeffable⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠. Part 4 of the Unkind Regards series. Full name: Don't Stop Me Now.Music: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Mainstream Music 2025 Vol. 8, Produced by Sascha Ende ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠(⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠CC-BY 4.0⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠) Sounds: Email notification: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/OptronTeamFilms/sounds/521094/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ (CC-0)Text notification (Crowley): ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/GabrielAraujo/sounds/242502/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠(CC-0)Text notification (Aziraphale): ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/mickleness/sounds/269185/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ (CC-0)Phone ringtone: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/jhyland/sounds/539661/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ (CC-0)Phone vibration: ⁠https://freesound.org/people/eobmada/sounds/541367/⁠ (CC-0)RL knock: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/Dreadwolf910/sounds/615987/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ (CC-0)For tags and other details, to leave kudos and comments, please visit the corresponding post on archiveofourown: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://archiveofourown.org/works/83306061⁠⁠⁠⁠⁠!

Take the 2026 AI Engineering Survey and get >$2k in credits and AIE WF tickets!On the product side, everyone is getting Computer - Perplexity, Manus, Cursor, and so on. Meanwhile on the research side, agentic evals like TerminalBench and GDPVal are also assuming computer (Harbor). On both ends, the consolidating LLM OS stack has become a standard toolkit, and Daytona is one of a small set of AI Infra companies that are booming because of it.“The end of localhost” has been Ivan Burazin's obsession for more than a decade.Something that is all too familiar…Long before agents became the default way people talked about software development, Ivan was already chasing the idea that development should not depend on a fragile local machine. CodeAnywhere, one of the first browser-based IDEs, was an early attempt at that future: move the development environment into the cloud, make setup reproducible, and free developers from the endless “works on my machine” tax.The thesis was directionally right, but the market wasn't ready yet.However, agents changed that. They do not care about a laptop, desk setup, or favorite editor. They need a computer they can access through an API: something stateful enough to keep working, fast enough to spin up instantly, flexible enough to resize, isolated enough to be safe, and composable enough to run the messy real-world workflows that real software engineering actually requires.Daytona isn't just selling “sandboxes” in the narrow code-execution sense. It is the latest version of Ivan's original localhost thesis.In this episode, Daytona's CEO joins swyx to explain why AI agents need more than code execution boxes: they need composable computers, stateful sandboxes, instant startup, dynamic resources, and infrastructure that can survive workloads going from zero to 100,000 CPUs.We go deep on the new agent compute market: Daytona's hard pivot from human dev environments to AI sandboxes, the New Year's Eve MVP that customers begged for, why Daytona runs on bare metal with its own scheduler, how one customer runs almost 850,000 sandboxes a day, and why RL/eval workloads went from 0% to roughly 50% of usage in just months. Ivan also explains why agents need Windows and macOS machines, why CLI may matter more than MCP, why Kubernetes is painful for this workload, and why the future AI cloud may look more like Stripe than AWS.We discuss:* How Daytona grew out of CodeAnywhere, Shift, and the “end of localhost” thesis* Why Daytona pivoted from human dev environments to AI sandboxes* Why agents need composable computers instead of disposable code execution boxes* The New Year's Eve MVP that customers chased API keys for* Why Daytona chose bare metal, stateful snapshots, and its own scheduler* How Daytona spins up one sandbox in ~60ms and 50,000 sandboxes in ~75 seconds* Why Daytona's biggest customer runs ~850,000 sandboxes a day* How RL/eval workloads create zero-to-100,000 CPU spikes* Why RL workloads went from 0% to roughly 50% of Daytona usage* Why customers compare Daytona against EKS/GKS and say they're “never going back”* Why every AI agent may need a computer, including Windows and macOS environments* The Apple licensing constraints that make macOS sandboxes hard* Why CLI gives agents more power than MCP* How open source helps agents integrate Daytona* Why agent-generated PRs may break today's CI/CD assumptions* Why AI SaaS companies reselling tokens may face a cold shower* Why the AI cloud may look more like Stripe than AWSIvan Burazin* LinkedIn: https://www.linkedin.com/in/ivanburazin* X: https://x.com/ivanburazinDaytona* Website: https://www.daytona.io* X: https://x.com/daytonaioTimestamps* 00:00:00 Hook* 00:01:12 Introduction* 00:03:15 CodeAnywhere, Shift, and the end of localhost* 00:05:58 What Daytona is: composable computers for AI agents* 00:08:07 The pivot from dev environments to AI sandboxes* 00:10:17 The New Year's Eve MVP and customers begging for API keys* 00:12:56 Bare metal, stateful sandboxes, and Daytona's scheduler* 00:17:28 60ms startup, 50,000 sandboxes, and 850K daily runs* 00:21:53 Spiky RL/eval workloads and the new agent infra problem* 00:28:12 RL workloads, Kubernetes pain, and dynamic resizing* 00:33:31 Why every AI agent needs a computer* 00:38:48 macOS sandboxes and Apple's licensing problem* 00:44:28 Why CLI may matter more than MCP* 00:48:11 Open source, GitHub stars, and agent integration* 00:53:11 Git, CI/CD, and agent collaboration bottlenecks* 00:58:15 Founder life and building a 25-person infra company* 01:02:44 AI SaaS, token resale, and API-first business models* 01:06:10 GPU sandboxes, data centers, and compute growth* 01:09:48 Why the AI cloud may look more like Stripe than AWS* 01:11:26 Closing thoughtsTranscriptIntroduction: Daytona, CodeAnywhere, and the End of LocalhostSwyx [00:00:02]: Okay, we're in the studio with Ivan Burazin, CEO of Daytona. Welcome.Ivan [00:00:07]: Thanks for having me, man.Swyx [00:00:08]: Ivan, you and I go back.Ivan [00:00:10]: Way back.Swyx [00:00:11]: How I don't even know how, you found, did you reach out or, for Shift.Ivan [00:00:17]: I reached out to you. The reason was you - we were just - we were thinking about I was one of the co-founders of CodeAnywhere, the first browser-based IDE, and so we were thinking a long time of, localhost should die. And you had this article.Swyx [00:00:29]: End of localhost.Ivan [00:00:30]: Then I reached out to you because of that, and then we talked, and I was actually at a different job and learning about I was the head of, developer experience, and you were quite well-versed in that, and I actually reached out to you, among other people, how do we go about that? What are the key things and whatnot at this point in time? And you were nice enough to take the call, and I remember I was late on your call with you.Swyx [00:00:51]: I don't remember.Ivan [00:00:52]: I remember because I was with my then I'm thinking of a girlfriend or wife at that point in time, I'm not sure. It's the same person, so that's great, and I was late ‘cause we were, in, Italy on, vacation, and then I was late for something. I felt so bad, and you were so nice to be, good about.Swyx [00:01:10]: The reason I'm nice is because I'm also late to other people, so it's like, who's, who's without sin here, yeah, so I have to, for those who don't know, InfoBip Shift, there's this whole thing that, you did in the past, and, and that was basically one of the inspirations for me starting AI Engineer, which is like, I have to thank you for giving me that push to be like, “Oh, you can, you can build and sell conferences?”Ivan [00:01:34]: I remember you asked you asked me at the beginning to give me advisory shares, and I was so focused on what we were doing, I said no, and I should've took the advisory shares. So I'm sorry, dude. But anyway.Swyx [00:01:43]: We're not, we're not venture backed.Ivan [00:01:44]: No, it doesn't matter.Swyx [00:01:45]: It's Yeah, anyway, so I think what's impressive about you is that CodeAnywhere is the thing that you've been trying to build, and, you kind of put it on hold and then came back after InfoBip. Just give us the story, do you - the story and the origin story, going into Daytona.From CodeAnywhere and Shift to DaytonaIvan [00:02:05]: Sure. Like, really way back, me and my co-founder have been together. I say this, I've said this multiple times, it's like we were married and divorced and married. Some people actually ask me is my co-founder my partner. they thought it literally. It's not literally, but we have done multiple companies together, and to your point, we had this shift where we went from the CodeAnywhere to the conference called Shift, and then back to, Daytona. We originally started stacking servers, doing like virtualization in the early 2000s and, routers and doing basically all these things, at a foundational level, and that was a services company which we sold to focus on what my co-founder actually invented, which was the very first browser-based IDE, right, I say the first. Before us was actually Heroku. They did it for a very short time until they became Heroku. But outside of them, we were the only one, and it was called.Swyx [00:02:55]: There was Cloud9.Ivan [00:02:57]: Cloud9 came out slightly after us. There was Replit, which came out when we stopped doing it, Replit came out, and they have been successful since then, which is great. There was Nitrous.io. There was quite a few that existed at the time, but it was like too early. But the interesting part is that we, at that point in time, because there was no VS Code, there was no Kubernetes, and Docker had just started when we Or I'm not sure if it was even public at that point in time. And so we had to build everything to the whole stack ourselves and that was the key learning that we brought into and that we've been using in Daytona today. So it was super early. There's about 3 million people used CodeAnywhere. It was slightly, it was angel-backed more than venture-backed. We ended up paying everyone back because it didn't have that sort of scale. But, three years ago, we started something similar with Daytona, which is not what we are today, but it was automating dev environments for human engineers, the basically the underlying stack of CodeAnywhere. And then we did a hard pivot last January to sandboxes. And so here we are.Swyx [00:04:01]: Historic pivot, yeah, and, it's one of those things where, I had independently invested in CodeAnywhere, but also in E2B, and then both of you pivoted into the same thing, and I'm like, “F**k.”Ivan [00:04:12]: You invested, you invested in Daytona. You invested in Daytona. But you were the first If we had not got your check, we wouldn't have done it.Swyx [00:04:18]: No way.Ivan [00:04:19]: No, it was like, “We have to get him on board first,” and you were that kicker that we, that got us off the ground.Swyx [00:04:23]: No, because you were putting me on your pitch deck, man. I was like, “Man, this is like a good trip if I don't invest.”Ivan [00:04:29]: That's because it was your quote. It's like we.Swyx [00:04:30]: Yeah. It's the end of localhost.Ivan [00:04:31]: Did a bunch of research about end of localhost and who was interested in that,.Swyx [00:04:34]: No, that's like, I put, I wrote that blog post, and every single company in that field reached out to me, and then every VC who was receiving those pitches then also had to call me and, talk it, talk through it with me.Ivan [00:04:47]: It's finally happening though.Swyx [00:04:48]: It was really super interesting.Ivan [00:04:48]: It's finally happening.Swyx [00:04:49]: It's finally happening.Ivan [00:04:49]: Yeah, it's finally.Swyx [00:04:49]: It's finally happening, with maybe sort of non-human users. Yeah, so what is Daytona today? Let's get like a quick description. I'm wearing the shirt.What Daytona Is Today: Composable Computers for AI AgentsIvan [00:04:58]: You're wearing the shirt. Yes,.Swyx [00:04:59]: It says, I think your branding is very good. Like, it's very consistent. It runs AI code. Like, it cannot be simpler.Ivan [00:05:05]: Exactly, but we're gonna probably have to change that.Swyx [00:05:07]: Oh, s**t.Ivan [00:05:07]: It's also a subset of what we do. Unfortunately, we really love this, Run AI Code is super simple. People interpret it different ways. I think we've given out 5,000, 6,000 of these shirts. People wear them with pride because it doesn't really market about us.Swyx [00:05:21]: Yeah, Daytona's on the back.Ivan [00:05:22]: It markets the back. It markets to the person itself, so I think we did a really good job on that one. But it is also a subset of what we do, because people, when they think about Run AI Code, they just think about these small, let's call it isolates, code execution boxes that, you send some code, you get an output. Whereas what Daytona is today is essentially composable computers for AI agents. It is, the market calls them sandboxes which can be misleading.Swyx [00:05:44]: All these things. All these things on.Ivan [00:05:45]: Yeah, exactly, ‘cause it can be misleading ‘cause people usually think about sandboxes as a demo or a test environment versus a production-grade environment. But what Daytona does, if you think of the laptop that you have in front of you or the computer that's over there, or, my wife is an architect, so she has like a Windows with a 3D graphics card inside to do 3D rendering. Like, as humans, we have different computers or different compositions of computers. And our belief is strongly that agents today and going forward will need all these different compositions of computers to do different types of tasks. And so we offer that basically through an API.Swyx [00:06:19]: Yeah, to give people - I'm trying to sort of front-load all the aha moments or the wow moments so that people can, stay engaged and click like and subscribe. the market is exploding, right? Like, you have been reporting 74% month-on-month growth, and it also, it's just been growing for a while. Like, it's been going like this. And every single - It's not just you guys. It's every single.Ivan [00:06:41]: Everyone, yeah.Swyx [00:06:42]: Sort of, compute provider. I don't know if you agree with me saying compute provider or not.Ivan [00:06:48]: It's fine.Swyx [00:06:48]: Yeah. So like organically PLG-driven growth, but also enterprise is doing super well, I think I wanna rewind to January of last year when you did the pivot. Like, so you obviously called this market early, and you were positioned for it, and you are now one of the market leaders. But what was the insight that made you do the pivot?The Pivot: From Human Dev Environments to Agent SandboxesIvan [00:07:06]: The insight that made us do this pivot is the quarter before that, so end of 2024, when we had - Basically, we did a demo with - I don't I think we discussed this as well, Devin was not public. You actually gave me access to Devin at that time. So Devin.Swyx [00:07:25]: I did?Ivan [00:07:26]: Yeah, you gave me access.Swyx [00:07:26]: I don't think I was supposed.Ivan [00:07:27]: Yeah, exactly.Swyx [00:07:28]: Yeah, I.Ivan [00:07:28]: So it doesn't matter. You.Swyx [00:07:29]: Yeah. I gave like three friends access.Ivan [00:07:31]: Yeah, or it was a call and you showed it to me. It doesn't matter. but OpenDevin was available, which is now called OpenHands. And so we're like, “Oh, this seems to be a thing. This is not public. Let's take our for human automation of dev environments and take, OpenDevin and launch that as a SaaS.” And we did that. Not very many people signed up and used it, but a lot of people reached out that were building agents, and they were like, “Hey, my agent needs a compute sandbox runtime,” whatever you wanna call it. I forgot what it was called at that point. And then we were like, “Oh, amazing. This is a new market. Here is our infrastructure. Here's our product, and go.” And what we found really fast, soon, was that people did not like what we had built. It didn't work. And I remember talking to people at the beginning when we're doing this, the sandbox we're building for agents. People were like, “Oh, why is it different? It's the same thing. We have like EC2, we have VMs, we have all these things.” But we saw that everyone we gave it to, it was like 20, 30 people, they all said, “No.” Like, “This is not what we need. This sort of breaks.” And basically, me and my co-founder not knowing a lot about - ‘cause we're infra people. We're not AI people. So I basically took it upon myself to like watch every single podcast that exists, including all of, all of these and all that, and sort of get up to date, read all the blogs, like get, understand what's going on.Swyx [00:08:45]: Do you wanna shout out who else was useful, just in case people are also looking.Ivan [00:08:49]: Generally we -, I looked at There's a few of podcast, different segments and different types. So there's you guys, No Priors, Bill Gurley's was great while.Swyx [00:09:04]: VG2, yeah.Ivan [00:09:05]: Yeah, while it was around. So there's a few. 20VC is interesting from a different dynamic, and some are different dynamic. But there was, also Red Points.Swyx [00:09:14]: We're not really about the compute market.Ivan [00:09:15]: It was also already - Sorry?Swyx [00:09:16]: You're, you want - You're looking at the agent infra market.Ivan [00:09:19]: I was looking at the agent market and the AI market in general and sort of understanding who are the players, what the perception, and how that goes. And like obviously you complement this with like going to conferences, going to events, going to meetups, reading white papers, like doing all the things that you have to do to understand what's happening. And so when we figured, when we sort of had an idea of what we had to build, literally over the New Year's Eve, literally on New Year's Eve, I half vibe coded the first MVP, first minimal viable product of what Daytona is today. And I went to sleep at like 3:00 AM or something like that. I was doing - I just put my like baby daughter and wife to sleep and, Happy New Year's, and go back to just, doing this. And I sent it to my co-founder, my CTO, and he saw it in the morning. He's like, “This is absolute garbage.” “Do not show this to anybody at all, but the idea is good.” And so he took two weeks, and he rebuilt it.Swyx [00:10:09]: Did it like look like that? Listen, I - It was rough idea.Ivan [00:10:12]: Oh, not even, not even close. Like it was it was way worse. But it was like a very - It was a simplistic view of what it should be. Like, it worked, but it was not ideal. And so he went, we went down the whole, which is his job as CTO, to go, and he came back with this version. We then called all the people that had said like, “This is garbage,” a quarter ago. And we set up these calls, and we gave it to - We just demoed it to everyone. And all the calls went long, every single one. They were 15-minute calls, and they all went to like 25, 30 minutes or whatnot. And everyone said, “We need, we want access.” There was no login, just an API key, ‘cause it was just a beta or an alpha. And they said, “Oh, we want access.” And we're like, “Sure, yeah. Okay, thank you very much.” But after like the next day, if we'd not send it, every single one, like every call that we did, everyone came back, “Where is my API key?” Like everyone wanted it. We're like, “S**t.” Like this is it. Like I've never felt So one, the understanding to your point was like most people thought it was the same infrastructure for humans and agents. We understood a quarter ago it's not. We just didn't know what was the right primitive. And then when we came, and we can talk about what that is, and we gave it to these people, I've never seen, I've never experienced - I've done multiple companies in my life. I've never experienced this, that people literally call you if you do not give them access. Like they want access right now. And so it's like, okay, they don't want this. the thing that they want doesn't seem to exist, or they have not found it, and they really want what we want. And then when we understood that we're onto something, and then when you think about the size of the market, like the market for human engineers and enterprise is a very large market, so think GitLab or whatnot. But the market for every single agent that will exist ever in the future is just like, what is that market? How big is that? And we're like, “We are all in on this.” And so that is where we made sort of the cut between the old product and the new one.Bare Metal, Stateful Sandboxes, and the Lambda + EC2 ModelSwyx [00:12:02]: Yeah. But it wasn't composable at the time?Ivan [00:12:05]: It was very - It was basically just a Linux box that you could change, that you could define number of CPUs, disk, and RAM. Like that is what you could do, but you couldn't have multiple operating systems, you couldn't resize it on the fly, you couldn't add a GPU, you couldn't do like all the things. It was just the, just the first sort of variation of that, yeah.Swyx [00:12:22]: Was it bare metal from the start?Ivan [00:12:24]: It was bare metal from the start. And so the interesting thing that we thought about right away, so our.Swyx [00:12:29]: Which, give people the background, what is the normal path?Ivan [00:12:32]: Yeah, so, basically most providers run this on top of VMs. And also.Swyx [00:12:37]: Firecracker.Ivan [00:12:38]: Yeah, they run on Firecracker and VM. And so we also fire - We can get - We have multiple isolation layers and we can do that. But the common way to do it is that they, one, that the state of the machine, or the hard disk is not part of the sandbox itself. And the other thing is they're not meant to last forever. So most of them are preemptible, like they can There's a time that they can live. And so our thought was when we were going into this is, agents will be like humans in the sense of you don't want your laptop to be shut down until you're done with work. Like, and you want to close the lid and open the lid, it's the same state. So you - Agents would want that, like the pause and come back. They want those two things. But also agents really want speed, right? Can they get it? So when we thought about it's like we need something insanely fast, how to make it fast, how to make it long-running, and stateful. And so those two things, it's like combining a Lambda and an EC2, right? Those two things together. And so we didn't have an idea how others did it, ‘cause we didn't know too that there was a market around this. It was more like, okay, this is what we need, what they need. And we looked at Kubernetes, it wasn't wasn't good enough for that. We looked at Nomad, it didn't enable that. And so our history in rewriting our own scheduler at CodeAnywhere is basically what my CTO came up with. Like, he's like, “Oh, the learnings from there,” and he brought it. And the funny thing is, our third co-founder, when he saw it, he's like, “Dude, what is this? This is like 2008.” Like, we went back in time, and he's like, “Exactly.” And so the reason why Daytona is like super fast, and you see this on benchmarks, is we essentially, we run on bare metal. We have our own scheduler, we use the underlying, disk, CPU, and RAM of the underlying machine, which means your IOPS are insanely fast because there's no, there's no network between an EBS or something like that. But also the snapshot, the point in time, the templates, are also preloaded on the bare metal machines. So when you fire off a sandbox from a template or a snapshot, you're essentially directed to the bare metal machine where that snapshot is based on that NVMe drive, and then it literally just turns on that machine, and it's local. There's no network latency, anything on there. And so that is sort of the specificities that we, when we're thinking from first principles, what a computer would look like for an agent, that is what we came up with, and that's what we created.Benchmarks, 60ms Startup, and 50,000 SandboxesSwyx [00:15:02]: Yeah. I should maybe, I don't know if you endorse this, but there's someone that does compute SDK, you guys do very well on there, with like the TTI, right? I. is this a, is this a is this a relevant benchmark for you guys? I don't know.Ivan [00:15:16]: I don't know, and it changes every day. So today RKL is.Swyx [00:15:18]: I don't know what RKL is. Never heard of it.Ivan [00:15:20]: Yeah. RK, yeah, so it is there.Swyx [00:15:22]: You are, at least a third of the next tier of performance, and then, there's a lot of other better-known names that are very slow to start.Ivan [00:15:31]: Yeah. We've been the number one by far for a long time, and now there's different, there's different definitions also of sandboxes, different isolation patterns, different other things. So RKL runs it literally on the S3, the data, so it's very different, and they spin up a sandbox, spin up a container for that, so it's a different type of thing. So the definition of a sandbox is something that we can all, we all need to get along with. But yeah, we're insanely fast on getting these things, up and running. And so you can see even there that it's a zero point 0.10 to 0.11, so.Swyx [00:16:03]: Close enough. Yeah. what else do you need, right?Ivan [00:16:05]: Yeah. So the benchmarks itself, so, in this, in I don't think the benchmarks equate to market ownership or revenue or anything like that. and I've seen this with multiple benchmarks, not just in sandboxes, but in general benchmarks around.Swyx [00:16:20]: It's table stakes. It's just like.Ivan [00:16:21]: Exactly. But it doesn't hurt.Swyx [00:16:22]: Just roughly check.Ivan [00:16:22]: Like you definitely have to be up there and you have to be competing so that people know that, oh, this is definitely one of the top. Because this is only one dimension of what customers look for. There's other things like how many can you spin up consecutively? There's a feature set, there's support, there's like all different things that people look at, but you definitely have to be there, on the benchmarks.Swyx [00:16:40]: How many people do people spin up consecutively?Ivan [00:16:43]: So we have.Swyx [00:16:43]: Or concurrently, is the Concurrency, right?Ivan [00:16:45]: There's three metrics that we look at. And so one is like time to spin up one, and so our time to spin up one is 60 milliseconds with network latency. So request, spin up, reply, 60, the whole thing, 60 milliseconds. That is one. But if you wanna spin up 50,000 at once, we are now at about 75 seconds. So it takes about 75 seconds to spin up concurrently 50,000. Some others, there's public data around this, like take 2,000 seconds, which is 30 minutes. Like there's different variations of that. And then there is the so it is speed of one, speed of like multiple, and then how many can you consistently have up and running. And so we basically have right now no limit to how much we can add because we basically own our own metal. But the biggest customer of ours does like about 850,000 every single day is sort of where they're, where they're just shy of a million every single day that they're running, we do have a request for half a million concurrent, which is literally half a million CPUs somewhere running. So that's an interesting.Swyx [00:17:44]: They pay by like vCPU seconds.Ivan [00:17:47]: By seconds, yeah.Swyx [00:17:47]: Or whatever. Yeah. Okay, and so and then, and the other thing is, the sleeping and the resuming, ‘cause it's all the stateful resumption of all these things, how, what kind of workload are people putting through this, right? Like how is it Do we measure by gigabytes in memory, gigabytes in storage? I don't In like network attached storage. I, what are the costly ones of, out of all these features?Workload Economics: CPU, RAM, Network, and StorageIvan [00:18:15]: The most expensive thing are CPU.Swyx [00:18:18]: Okay. Yeah, of course.Ivan [00:18:18]: The second one, yeah Then it's RAM, then it's disk. We actually don't charge.Swyx [00:18:22]: Which is snapshotting, right?Ivan [00:18:23]: No, it's actually the, snapshotting's part of it, but basically the size of your hard disk, of your machine. So do you have 10 gigabytes, do you have 20, do you have 50, do you have whatever? And then the transference of that. Right now, currently we don't charge for, network at all at Polychron.Swyx [00:18:37]: Oh, you gotta, yeah, you gotta fix.Ivan [00:18:38]: Yeah. It is very much a it's a larger and larger part of our bill, so we're working around, that part there. Obviously, that is the least, expensive, so the hard disk is the least expensive, so it's basically CPU, RAM, for us network, ‘cause we don't charge the customer, and then hard disk, is how it's split up. But there's also different types of workloads, so we basically split it up into two types of workloads in Daytona. One is what we call background agents or long-running agents. and the other is, basically RLs and evals, which I put sort of together. And so they have very different patterns of usage, and if you look at the usage of a background And I'll just name names of companies, not specifically.Background Agents vs. RL/Evals: Two Usage ShapesSwyx [00:19:21]: Yeah, open, all hands.Ivan [00:19:23]: Yeah. So like a background agent's a Cognition, a Lovable, a like all these things are Harvey. These are all long-running, background agents. And so if you look at their usage patterns, their usage patterns are similar to human, which is like follow the sun. Basically, the usage patterns of that is like noon is probably the highest, and the midnight is the lowest, and then weekends are lower. weekday is higher.Swyx [00:19:42]: Yeah, that's a fun question. How global is it? Is it very US-centric or?Ivan [00:19:46]: The US is a large part, but we have currently, we have Asia, Europe, and the US regions.Swyx [00:19:52]: So it's quite global.Ivan [00:19:53]: Yeah, it's quite global. We have it all over. It's interesting that our I talked to you a bit about this. Our number one city by user.Swyx [00:20:01]: Hmm.Ivan [00:20:02]: Is Singapore.Swyx [00:20:04]: Oh, wow. Amazing.Ivan [00:20:05]: Which is an interesting one, right? Not by revenue, just by just like by individual head count.Swyx [00:20:09]: Really?Ivan [00:20:09]: Just like an interesting thing.Swyx [00:20:10]: Singapore is, Singapore is weirdly high in the adoption charts of AI for the population. It's like an, seven, eight million population. And it's like keeps showing up.Ivan [00:20:20]: No, it's quite interesting. We were quite shocked, and I was like, “Oh, this is interesting.” And also one that's up there.Swyx [00:20:24]: There's a reason I'm doing AI using Singapore. it's because I'm from there.Ivan [00:20:27]: We're there. We're gonna, we're gonna be there as well. and it's interesting that Japan is in the top or like Tokyo's in the top, which is in all the tech cycles it has never been. It has never been, so it's quite interesting that they're.Swyx [00:20:39]: I think the Japanese just love AI. Yeah. It's that, and then it's Brazil. That's it.Ivan [00:20:44]: Brazil has always been in.Swyx [00:20:45]: I think.Ivan [00:20:46]: Even when I look, if you look at like GitHub's data and ask historically with CodeAnywhere, it was always like US, Western Europe, and then you'd have like India, Brazil, China, like that would be there. But like Singapore was not in, specifically Japan was never in sort of that top, that top.Swyx [00:21:01]: Yeah. Weird pockets.Ivan [00:21:01]: Weird. Yeah, so it's very global.Swyx [00:21:02]: Okay, so actually that, but that's helps you to distribute your load through, all time?Ivan [00:21:08]: The interesting thing is like we have those kind of loads, but if you look at the researcher loads, they're quite different. So what they are is like if you give them concurrency of 10,000 or 50,000 or 100,000 CPUs at ARMb, when they fire off a run, it's just 100%. And then it just runs, and then it stops. So it's very, the usage pattern is squares basically, right? And it's also not follow the sun, because people will fire it off at midnight before they go to sleep but then wake up and so it's very unpredictable, so you don't know where that is. So the shapes of the usage are quite different than we have had before. And also what's interesting is when it's sort of a follow the sun, even if you have a high growth company, you can sort of predict your usage patterns and have enough capacity for that, because it's sort of, it grows in a, in a way you can project. When you have companies doing sort of like evals and RL, they're super spiky. So they're gonna come in, it's like, “We're gonna use nothing, then can we have 100,000?” Right? And then go back down. And then 100,000, go back down. So it's very different, right? And.Swyx [00:22:09]: Do you want to lock them into commits so.Ivan [00:22:11]: Yeah, we do.Swyx [00:22:12]: Yeah, okay.Ivan [00:22:12]: We so we have to lock them into some sort of commits to have that capacity, because we have to have, basically we have to have the capacity for peak. Right? And so right now, Daytona's mean utilization is 15%, 1-5.Swyx [00:22:25]: Oh my God.Ivan [00:22:26]: So it's very low.Swyx [00:22:27]: Because it's very spiky.Ivan [00:22:27]: It's very spiky, but we get up to 90%. so we have these things. And so what we're, what we're looking at right now as a company is similar to Cloudflare where you can like geo move things around, but that works really well for basically the background agent where it's follow the sun. But this, it's not. Like it's a very different shape. Obviously with scale you figure these things out, but that's an interesting new problem that we have, as a compute provider in the agent space. And when we were doing the conference recently, and so we talked to like Nikita from Neon and.Swyx [00:22:57]: I should bring it up.Ivan [00:22:58]: Parag from Parallel and whatnot, everyone has the same problem. Whereas the usage is super spiky, and this is something that has not happened before, that you have these types of like it was always, it the amplitudes were not this high, right? So it's quite interesting use case and problem solve.Compute Conference and Spiky Agent InfrastructureSwyx [00:23:12]: Yeah, I don't know if we're gonna bring this up again, but let's just talk about the conference, you had like 1,000 something people at the Warriors game, at the Sorry, where is it? What's.Ivan [00:23:22]: Chase Center.Swyx [00:23:23]: Chase Center.Ivan [00:23:23]: Chase Center.Swyx [00:23:24]: I went. It was, it was very impressive. Obviously, you can, how to throw a conference, what did you learn? you put, you pulled together all these impressive names.Ivan [00:23:33]: What I.Swyx [00:23:34]: What were you looking for?Ivan [00:23:35]: My thesis behind the Compute Conference was let's bring together people that are building infrastructure for AI agents. Because when I think of what we're building, it is the agent is the primary user, what are the ergonomics and usage patterns of agents, and so we can do that. And what I found, this was a theory, it wasn't proven, is that we all have these problems, as I touched onto. And I was, as I was talking on stage, it was like we all have the same underlying infra problems, which is this spiky workloads, unpredictable workloads that we've never had before, in human, compute or human infrastructure. And it's, again, it's the same when I was talking to Parag or when I was talking.Swyx [00:24:20]: Lynn. Nikita.Ivan [00:24:21]: Lynn, Nikita. Lynn especially, I was talking to her the other day as well. Like the It is a very interesting type of problem to solve because I can touch on Cloudflare because there's a lot of like talk about that recently as to how they solve that, which is they have a bunch of geos, and basically, as users work in different places, and depending on your tier, they can move you around the geos. And so that how, that's how they get the higher utilization. But you can sort of predict these, and it's If it's something in You'll rarely get a spike that is 10 orders of magnitude. Like you'll get a like let's say one of your customers has some like an exponential curve. What is that to I'm using Cloudflare as an example. 10%, 20%, whatever it is. I don't, I don't have this data, I'm just assessing. It's surely not 10x, right? It's surely not something there. And so how do you go out and solve this problem? And we're all solving this in different ways. So we have.Swyx [00:25:11]: She also has the same thing.Ivan [00:25:12]: Yeah, I know specifically that like Neon had that issue as well. Like how are we solving these spiky loads and things like that ‘cause we talked about it. And so the interesting thing for me to actually internalize was, yes, everyone that's building for agents first is going through this, and we're all solving similar problems, which is quite.Swyx [00:25:28]: Let me let me double-click on this. Okay. So for example, Neon, I happen to know that they're very sort of S3 oriented, right? so they're just like fully bet on S3. And you get to benefit from S3's distribution and infrastructure. So I would imagine that Neon doesn't have to care, whereas Lynn maybe has to care a bit more because obviously she's doing GPU inference. And, for listeners, we did an episode with her, one and a half years ago. And you have to care. But like, right?Ivan [00:25:54]: Parag cares for sure, and Nikita.Swyx [00:25:58]: And Parag is C of, Parallel.Ivan [00:25:59]: Parallel, yeah.Swyx [00:26:00]: Former CTO of Twitter.Ivan [00:26:01]: Twitter, yeah.Swyx [00:26:02]: They are the search.Ivan [00:26:03]: Yeah, they're search, yeah.Swyx [00:26:03]: I You and I know but the listeners don't know.Ivan [00:26:08]: Yeah, we can put it down in the screen, and so ‘cause we, when we were talking.Swyx [00:26:11]: I'll put it up on the, on the screen.Ivan [00:26:12]: Yeah, right.Swyx [00:26:12]: People can look it up if they need.Ivan [00:26:14]: Look it up. And, yes, but they still have CPU and RAM, allocation that you have to have up and running. And so CPU and RAM, you have to allocate that and have that ready. And so there's basically two ways to do it. One is you either over-provision and you can handle the bursts, or two, you basically have, I don't know if this is a term, just-in-time compute, which is like as your load becomes, as your usage comes in, you can fire off requests for VMs or bare metals at other cloud providers and then get them up and running.Swyx [00:26:43]: This is if you go above 100%, right?Ivan [00:26:45]: Yeah, this is.Swyx [00:26:46]: Like your overflow.Ivan [00:26:46]: If your overflow, like spillage or whatever you do.Swyx [00:26:48]: You probably lose money on it, but it doesn't matter, right?Ivan [00:26:50]: It, not Well, you might, you might not That is a more cost-effective way to do it but it's a slower way to do it. Because basically what you have to do is you have to like queue your requests, spin up these just-in-time compute, get it all ready, provision it, and then get your workload there. And so if the time isn't important that much, that's fine, and you can do that. But if your customer, and especially for, let's say, the RL training runs, the reason why a lot of people come to us is because GPUs are more expensive than CPUs, right? So you want your GPU running at, what, 100% the entire time. And so when you're running runs on CPUs, when the when the CPU cycle is like down and spinning up the next one, you want that to be instantaneous so that your GPU doesn't go down, right? And if you then have to like go out and provision machines, you're essentially telling the GPU that it has to wait, and that's incurring our cost. So there's things that you have to try to solve for there.RL Workloads, Declarative Images, and Kubernetes ReplacementSwyx [00:27:43]: Yeah, let's talk about the different workload, right? You said that, what was it? A few months ago, you had zero RL workload and now it's 50%.Ivan [00:27:52]: It will be this one, 50%, yeah.Swyx [00:27:54]: Let's talk about how different it is, right? Like I imagine, for example, a lot less dynamic code generation of like arbitrary code. Like here, it's probably all the same code. You're just doing parallel runs or something, I don't know.Ivan [00:28:05]: Yeah. So you'll have multiple Depends on the like for each run, you'll have a snapshot. And they, for the most part, they actually do use our declarative image builder, which is like, “Oh, we, the agent wants these dependencies, these env vars.”Swyx [00:28:17]: These ones, yeah.Ivan [00:28:18]: Yeah, the declarative image builder, it.Swyx [00:28:20]: Which is a very modal like thing that they.Ivan [00:28:22]: Yeah. And so we build it on the fly and then we propagate that snapshot, and you can spin up as many sandboxes as you want against that snapshot. And then if you have to do changes, the model can, or like it could be also be automated. It's like, “Oh, now for the next run, we need to install these things or remove these things or whatever to get, a task done,” and then it goes off and runs that. So yes, that is something that it seems that they prefer. The number one reason I found, or should I say, let's take a step back. What we are competing against in that environment is essentially managed Kubernetes. So EKS, GKE, whatever. That is what the vast majority run on. And anyone that has tried Daytona versus GKE, EKS is like, “I'm never going back.” That has always been. There's a few reasons. One is the ergonomics. So if you have, if you're using Kubernetes to spin that up, you have to essentially manage the interface interactions with that. Daytona, although as a compute provider, it's more akin to a Twilio and Stripe from a consumption perspective than it is an AWS. Like you have an API, an SDK, it's quite like easy and seamless to get these things up and running, that's one. The other is the speed to which we spin up, which we mentioned earlier, which is much faster, and the scale to which we can go to. We haven't got into features, but an interesting feature is that it's very hard to OOM, or out of memory, our sandboxes, because we can dynamically on the fly.Swyx [00:29:48]: Resize.Ivan [00:29:49]: Resize, which is like impossible on almost any other thing. There are some technologies that enable you to do that, but it's like a very hard thing. And so we actually saw this when, the Terminal Revenge team is, brought us actually. So thank you, Alex and the team, that brought us into this whole space.Swyx [00:30:05]: It's just very rare that, a framework would just say, “Guys, just use Daytona.”Ivan [00:30:11]: Yeah, I think it says it somewhere. Yeah.Swyx [00:30:13]: Yeah. I was like, “What is this?”Ivan [00:30:15]: There's all, there's multiple there, but they also mention a few other places. and so Daytona specifically-We have, the, just jumping on themes here We, I don't know where it says Data Center.Swyx [00:30:27]: I, there.Ivan [00:30:27]: Doesn't matter.Swyx [00:30:28]: There's a very strong recommendation, which is, very unusual. Which is, it's.Ivan [00:30:33]: We do not pay them for this, just.Swyx [00:30:34]: I know, yeah. They just like you.Ivan [00:30:35]: Yeah, they like us. yeah, and also a thing, so, Data Center has multiple isolation sets underneath. The customer doesn't have to know what they are. But basically we have Docker, which is a container, that's hardened with Sysbox. So it's Docker's, isolation that is a security equivalent to a VM, but it's still a container. And that is the default, and they, especially in these training workloads, really like that as an interface to be able to use just a basic Docker container, and we enable Docker and Docker. Which for these RL runs, if you need to do a Docker compose or Kubernetes, you can spin up a K3S inside of these things, which unlocks a huge amount of workloads that you can do that you cannot do on other providers. So just on that part is much more interesting. And so we went that, through that. We showed them that we could do that, and they enjoyed that quite a bit. They being the general venture people.Swyx [00:31:28]: Those people, yeah.Ivan [00:31:29]: And Harbor people.Swyx [00:31:29]: Harbor people, do are they, are they a company yet?Ivan [00:31:33]: As far, I do not know.Customer Pull, Slack Connect, and the Computer Use BetSwyx [00:31:35]: Okay. All right. Yeah. It's like super obvious that like, there's a lot of excitement and success around these things, okay, so yeah, tell us more, right? Like, this is an exploding workload, Harbor adopted you, which helped speed things along. But what are you learning as this new workload comes online?Ivan [00:31:53]: There's a couple things that we learned, which we chat about in the beginning. We, and this has led our story, as we mentioned, we like talked to a lot of customers along the way, and we add more features and more tool sets as we talk to customers. And it's interesting that And I think it's that the ecosystem is so small and/or the models get smarter, where when we see one user come with a request, we know it goes on a roadmap if like three to five customers come with the same request in that week. It's like very bizarre. It happens so many times, which is.Swyx [00:32:27]: Because they're all friends.Ivan [00:32:28]: Sorry?Swyx [00:32:28]: They all, they're all friends. They're all in the same group chat.Ivan [00:32:30]: Yeah, probably, yeah. ‘Cause and they're like, “Oh, can you do this?” And I'm like, “Okay, this is interesting. We'll put it on a feature request.” And then the next one's like, “Oh, can you do this?” “Okay.” It's all the same, right? It's always the same. And so what we try to do, and I personally try to do, I try to be on as many call, quote-unquote “sales calls” I can. I'm in every Slack channel. We literally have about 1,000 Slack Connect channels, something like that. It's an interesting, there's so many interesting things you find out when you have all the Slack channels. You can also see where people, transfer between companies. You see leave Slack channel, enter Slack channel. It's an interesting thing. Also, just I digress, I feel that Slack Connect is literally LinkedIn what it should be. You have a list.Swyx [00:33:08]: LinkedIn charges you to, use your own connections, but Slack doesn't, right? Slack is like, do it for free. It's more lock-in. It's great.Ivan [00:33:15]: Yeah. It's amazing. Yeah. It's one of the reasons.Swyx [00:33:17]: You're gonna pay Slack for life.Ivan [00:33:18]: Exactly. You're there for life. So that's interesting. And so one of the things, the newer things we were talking about earlier is we made a big bet and put a lot of investment on computer use. that is not seen publicly the light of day. We haven't GA'd that yet, but we have.Swyx [00:33:32]: Is there a thing I can pull up?Ivan [00:33:33]: There is computer use there. It's right up a bit.Swyx [00:33:36]: Oh, yeah. Okay.Ivan [00:33:38]: What we have, what we talked about and what we've seen publicly is there's this theme now about, the human emulator where And Elon from XAI has talked about this publicly, and if you think about the models today, they're actually quite sophisticated and they can do a lot of work, but they still don't have access to all the tools. Like, I'm a strong believer that the most efficient way for an agent to work is essentially headless or through, terminal or whatnot. But if we, if we look at knowledge work in general, there's about 100 million knowledge workers in the US, about a billion in the world, and knowledge workers, and the salaries of them aggregate to 10 trillion in the US 50 trillion worldwide.Swyx [00:34:24]: Wow.Ivan [00:34:25]: Something like that. And if we look at, the five most important sectors of that, so like healthcare and government and financial services and whatnot, that's about 56% of that. So let's say it's about half of that. So in the US it's about 25 trillion, and most of them, most of that work is actually still locked into legacy apps inside of Windows, which is not going anywhere for a very long time. Like, people just won't invest in that. How much of it? our assumption is the following: if, in the RPA market, which is similar market, well, not the same 25% of, these white collar, workers', work is automated. If an agent is more sophisticated, can go through more runs, figure stuff out, let's say it's, 40%, right? And so if you take 40% of that, you get to essentially, $10 trillion a year.Swyx [00:35:17]: That's a TAM.Ivan [00:35:18]: That is a that is a TAM. So that's the TAM of the models, right? That's not our, essentially ours. But you get to that size, and to be able to do that, you essentially have to give agents these computers with the legacy. So computer use, either Mac or Windows or Linux. Linux we also obviously have and others have. But Windows specifically is something very new, and the only option right now is an EC2 with, Windows or on Azure. Both of them take anywhere from three to five minutes to spin up. We've created an actual sandbox, so it's a second instead of milliseconds, but you have, point in time snapshots, you have, forking, you have all the things that you have from a sandbox, but essentially enables you to hopefully unlock all this value. And so that's been our big push and bet, but we've sort of, kept our ear to the ground. What is sort of the next things in the market?RPA Returns: Why Agents Still Need ComputersSwyx [00:36:06]: Yeah, knowledge work, and building, and sort of RPA, the next wave of RPA. I got very excited about RPA kind of during COVID times. The UI path was IPO-ing. And it was, a very hot Isn't it, Eastern European?Ivan [00:36:20]: It is, Romanian.Swyx [00:36:21]: Romanian?Yeah, it might be the only Romanian, big unicorn okay, yeah. This I don't I don't, I don't have like a I think there's, I think there's a stage being set for the resurgence of RPA, ‘cause everyone understands that, yeah, no one wants to deal with these shitty apps and no one's gonna rewrite them. Like, you just have to do, a remote operation and programmatic operation of them.Ivan [00:36:45]: If you wanna unlock it, my own setup was basically the following. So I was doing a board deck recently, last month, whatever, and I'm like, “Okay, let's just, let's just do automated.” So, all our data's in, ClickHouse and PostHog and QuickBooks, where everyone else's is, and I'm basically, connected that all to, my Cloud code, like go off and go Cloud code whatever. Go off and, here's the integrations, go do that. It pulled out the first report, which was great. It connected to Brex and all these things, pulled it, which was great, and then I say, “Okay, now pull out this, and this,” and I kept getting, really well McKinsey-style design reports, but the data said partial data. all the missing data, partial data. Like, it can't access all the things, and I got so frustrated, and so I got, I got, my Mac Mini virtual sandbox with OpenClaw. I gave it its own account in our company, and then I went to all these services and created a read-only account, so literally like an intern in your company. And so I would say, “Now go and do this report,” and it would get the same, or like, “I can't via the MCP or the API or whatever. I can't get all the information.” I'm like, “Go log in.” And it will log into the website, then go in, export the data. It'll export the data and do the thing end to end. So even for things that have today APIs, not all of it is exposed, and I to get value, I get immense value right now, but it has to be a computer usage, unfortunately, and so I spend a bunch of tokens just on that, but I get the job done. And so if even a startup like ours, and using all the hottest tools, still needs a computer agent what hope does, Goldman have to have a headless, right?Swyx [00:38:22]: Yeah, what a - Why isn't Microsoft doing this?Ivan [00:38:27]: I'm pretty sure, Satya had a post yesterday.Swyx [00:38:29]: Oh, okay. I see.Ivan [00:38:29]: Which was like, “Every agent needs a computer.”Swyx [00:38:31]: I see, I see.Ivan [00:38:32]: So they have launched something recently.Swyx [00:38:34]: Yeah, they have Microsoft Power Automate, I'm sure, I'm sure, they're gonna have their version.macOS Sandboxes, Apple Constraints, and the Windows OpportunityIvan [00:38:39]: Version of that, yeah.Swyx [00:38:39]: You're gonna try to do yours, and it - I always know there's always demand for Mac, but I know it's, tricky to host, macOS sandboxes.Ivan [00:38:49]: We will have macOS sandboxes fairly soon. The problem with macOS, OS sandboxes is, I'm deep in this, I don't know how much interesting is.Swyx [00:38:55]: No, it's.Ivan [00:38:56]: MacOS has this problem.Swyx [00:38:57]: It's a licensing thing, right?Ivan [00:38:58]: Licensing thing. So one, you're allowed to run only two parallel VMs per machine, so that's one. Two, you can only license to a different user every 24 hours. So if you come in and theoretically, if I wanna charge you per second and I charge you one second, I have to have it idle for the rest of the day. I can't have anyone else doing that. So the pricing will be different in the sense that I will have to - we would have to charge for 24 hours, and that's not even, that's not even the most difficult thing. But the, thing above that is, from a security perspective, they enable you to do memory snapshot, pause, resume, but only on the same physical drive, physical machine. And so what you can do in, Windows world or Linux world is that I can move in the background, your snapshot from one to the other and manage load, right? Here, if you wanna do that, you essentially have to have your.Swyx [00:39:49]: Yeah, snapshots. Yeah.Ivan [00:39:50]: Your.Swyx [00:39:51]: It's like.Ivan [00:39:51]: Physical machine.Swyx [00:39:52]: You can't break it up.Ivan [00:39:53]: You can't, you can't move things around that, and all of that is, that part is, from a security standpoint, if it is written. Like, I understand the security aspect of that, but it disables you from doing these agentic, like really scalable agentic workloads.Swyx [00:40:08]: You need to do a vibe-coded, clean room implementation on macOS that you can then - That's like Clean OS or something. I don't know.Ivan [00:40:17]: So. We have.Swyx [00:40:18]: ‘cause like Linux was originally like a clean room rewrite of Unix.Ivan [00:40:21]: Okay. Yeah.Swyx [00:40:21]: Or something like that, right? Like same thing to macOS. Someone needs to do it.Ivan [00:40:25]: Someone will do that, and someone will have some long-running agents for a few days to figure this stuff out. But yeah. So definitely we - we're really close to offering something ‘cause people do want it, but the pricing will be different, and the feature set will be sort of stringent.Swyx [00:40:38]: Yeah, nobody's gonna use this. like, the labs, the labs will because they want to automate macOS.Ivan [00:40:42]: They have to do RL. They have to do RL again. But even if you The - So the point is with the RL part, if you, if you do RL on macOS, then the next iteration of the model comes out, it will be able to use these tools significantly. Then you actually need to run those, that somewhere. So you're gonna have to have that, later on. And from, if anyone at Apple is listening, I very much feel that they are shooting themselves in the foot of the scale of the revenue of compute or licensing they could get if they would just enable a concurrency model similar to what you can get on a Windows and a, and Linux.Swyx [00:41:17]: Yeah. Yeah. And I'm sure they've heard this before. They just don't care. Yeah, it's And maybe they will change their mind with the new CEO.Ivan [00:41:24]: Yeah. We'll see.Swyx [00:41:25]: We'll see.Ivan [00:41:25]: High hopes.Swyx [00:41:26]: High hopes.Ivan [00:41:26]: High hopes.Swyx [00:41:27]: Okay. But I, it's very clear the market opportunity is huge in Windows, and you can go for a long time on just Windows, but your customers are gonna want both. and I think, it is interesting to me that, this is the sort of God application of agents, right? Like, I don't It was - How big was OpenClaw for you guys? Like, was it, was there, a significant bump.OpenClaw, Agent Labs, and the B2B2C Sandbox MarketIvan [00:41:54]: Not for us because we.Swyx [00:41:54]: Because you already.Ivan [00:41:55]: We're kind of positioned differently. Whereas although it's completely PLG and we have individual developers that use it, most of the users that use Daytona are sort of a B2B2C. Sort of it's either B2B or B2B2C. So, in the researcher world, it's B2B, so you're selling to, labs and neo labs and things like that. But on the long-running agents, it's mostly, from a scale revenue perspective, it's mostly B2B2C, where you have a app layer agent that uses you at a big scale.Swyx [00:42:26]: Like a Manus. Yeah.Ivan [00:42:28]: Like a Manus Lovable type of thing.Swyx [00:42:31]: Yeah. I think that's the question of, well how, um-Uh, yeah, B2B to C is basically to me what I've been calling an agent lab, which is kind of like you're not in a model lab, but you're making a very good wrapper that is a platform that other people can sign up so they don't have to code those things. Yeah, it sound, it sounds like a much better market than the direct OpenClaw market.Ivan [00:42:56]: I've like - We I've done multiple things. So the CodeAnywhere's part of our career path R in the calendar, was very much an end user developer product. And so that is great. It You can get a lot of developer love, and I feel that we do as a company have a bunch of developer love. But it's a different type, where it's people building these things. Again, it's more akin to a Twilio because you don't really run - As a person, you wouldn't run Twilio. I don't know how many people remember. It was like ask your developer billboard and whatnot. And people really love Twilio, but they only used it inside of like, “Oh, I'm building this app or service for thing.” And so we're very much directly to that. And you also know that I used to work for a competitor for Twilio, so it's kind of ingrained, in my DNA.Swyx [00:43:35]: People don't know InfoBip is that big.Ivan [00:43:38]: Yeah, it's.Swyx [00:43:39]: Because.Ivan [00:43:40]: It's a billion euro.Swyx [00:43:40]: They're all American. They're like, “Whatever's in Europe doesn't matter to me.” But like it's the, it's the same size or bigger? Same size?Ivan [00:43:46]: It's about half the size.Swyx [00:43:47]: Half the size?Ivan [00:43:48]: Yeah, about half the size.Swyx [00:43:48]: It's like, yeah.Ivan [00:43:48]: Still huge. Multiple billions a year. Yes.Swyx [00:43:51]: That's crazy.Ivan [00:43:51]: Exactly, and so that - These are like really interesting and large revenue-generating, very sticky businesses. Whereas when you're selling to the - When your focus is the end developer, it is a very hard sell because they're very price sensitive, very price conscious, very around that. And there's very It's very hard to scale. Your cap is the number of people that are willing to spin up - First of all, wanna spin that up, and then spin up multiple of these. Whereas if you're in the enterprise one, like we know everyone's talking about like how many tokens they're spending, I'm spending. Like a lot of companies today are like, “If this is our company, spend as much as you can.” Like basically that is where we're going. And so if you think about that paradigm, where you're selling to companies that say, “Spend as much as you can to generate, productivity,” versus, “Oh, I'm a single person. I have this much budget, and I'm doing this thing because it's fun or it's helping me out or whatever.” Like it is a different, it's a different go-to-market, I think, strategy.MCP, CLIs, and Sandboxes as the Agent RuntimeSwyx [00:44:50]: Yeah, there's a lot of discussion. I'm just kind of going through like the mental list of things that are in your favor, which is, for example, MCP versus CLI. Like obviously you want CLI. It's been very good for you. I feel like it's maybe a drop in the bucket or maybe it's huge. I'm just checking whether it's like these are big trends.Ivan [00:45:10]: Those things you - work well in our favor, to your point just because every.Swyx [00:45:13]: They're kind of drop in the bucket, right?Ivan [00:45:15]: I think it's like sort of all the things come together. And so there's so many things that impact that. To your point, like OpenClaw wasn't huge for us, but like having the agent SDK, from Anthropic, so or Cloud Claude Code was very interesting. The reason why it was interesting is that a lot of, let's call them app I don't know what to call them, app layer agent companies, essentially they are like, “Oh, I can create this new app, this new agent. All I need, I just use Claude Code, and I throw it into a sandbox, and then I have my interface to the human to that.” And so that enabled so many more companies to actually offer this, and then they would pull on sandbox. So that was, that was interesting. And to your point, like MCP, versus the CLI, the MCP is an interface against an API, whereas the CLI is like you can actually go do things. Like this is it. The difference between integrations and actually running scripts or data or analysis against a thing. So being able to use a CLI very well enables the agent to do more things, and it's because that people will invoke a sandbox, they'll run it in the CLI, and but it'll do anal-analysis on that data and then give you an actual result versus just, pulling data from an API source.Swyx [00:46:29]: Yeah, it's a layer of indirection basically, it's the same thing as agentic search versus RAG, which where you're.Ivan [00:46:34]: Exactly, yeah.Swyx [00:46:34]: Just like you just win whenever people put more agents into their workflow. And so like it doesn't really matter, but I'm just kinda teasing out like what else have people heard about that like it's sort of, “Oh yeah, this is another sandbox use case. Oh yeah, that's another one.” Am I, am I missing any big ones?Ivan [00:46:51]: The thing, the thing that people, which is the computer use stuff, which I think is probably the most interesting one, is, and to your point, we've talked to so many people over the last year. It's like, “Oh, like why do you need a sandbox? Why do you need this? Why this?” And to your point, it's like, “Oh, I need sandbox for this. I need sandbox for that. I need sandbox-” It's like, “Oh, I need it for every single thing.” And so basically what I, what I - and it sounds like a broken record, it's like you use a laptop every single day, right? And you are n of one. It's just you. But now imagine how And by the way, the laptop, the computer PC market, the PC market is about equal to the cloud market in total. So it's about 150, 180 billion a year. Something like that. It's about roughly the three cloud hyperscalers is about equal to like Apple, HP, Lenovo, whatever, It's a little bit less, but it's sort of like that. And now imagine And that's just like, so how big is the addressable market? What, how many people are there in the world now? What's the last data?Swyx [00:47:45]: Let's call it eight billion.Ivan [00:47:46]: Eight billion. And so let's say you can have two computer, like you have one personal and one business, whatever. Like so it's double that, right? and so that's 16 billion, right? How many agents are gonna be running in two years, in 10 years, in 100 years? Like And for every single task, they will need one of these. And so how big is that? That market is essentially quote unquote “infinite”. You will get to the point, and Dylan Patel was at the conference talking about, from SemiAnalysis, that talks usually about GPUs, was also talking about how CPUs will now be a bottleneck because it will be the constraint. You won't be able to grow, or we won't be able to have enough of these because there won't be enough CPUs to basically do.Swyx [00:48:23]: Yeah. Well, I actually had a really good podcast with Doug Oliphant, who, which was his president at SemiAnalysis, where they've basically been like, yeah, it's been a GPU shortage first, but then it's cascaded down to memory and now to CPUs.Ivan [00:48:35]: CPU, yeah.Swyx [00:48:35]: It-What's next? So networking. So, networking actually has been in shortage for a while if you're looking at, just GPU networking. But, yeah, it's really crazy the amount of computer use that's going on, yeah, cool. I, other questions are, just the one very big part is the open sourceness which you didn't have to do, your competitors don't do, like it's not, a lot of people are worried about keeping their projects open source because some competitor can just slot fork it. I don't know if there's any reflections on just being an open source company.Open Source, Trust, and Enterprise ProcurementIvan [00:49:15]: Yeah. There's a bunch. So we the original product that we did was open source.Swyx [00:49:19]: Yeah. CodeAnywhere.Ivan [00:49:20]: So doing that was actually very good for us. There's basically a saying of, What's the saying? Like, companies that are, that are doing really well, measure themselves against, free cashflow, that are kinda okay, it's EBITDA, then, it's, it goes all the way down.Swyx [00:49:36]: The worst is like GitHub stars.Ivan [00:49:37]: GitHub stars. GitHub stars are the worst, yeah. So you go all the way down to GitHub stars. And so our original one was GitHub stars. That's what we talked about, we're at the point we're talking about revenue, so we're we've gone up the stack on that. And so we started.Swyx [00:49:47]: No, profit.Ivan [00:49:48]: Yeah. We haven't, we're, we'll get there. We'll get there. But basically at that point we did stars and GitHub and it was useful, and the original variation that we did, it we split the core into its own repo and it was Apache 2.0, so very, permissive. And then we basically would bundl

AI suddenly feels like it has crossed a threshold, and Yann Dubois, co-lead of the Post-training Frontiers team at OpenAI, joins Matt Turck to explain why. Yann's team has led the post-training behind the company's reasoning models, including the recent GPT-5.5 release. In this conversation, we go inside the shift from raw model capability to useful, reliable systems: what changed with GPT-5.5, why reinforcement learning is moving beyond math and coding competitions into messy real-world work, how reasoning models like GPT-5.5 actually work, the difference between GPT-5.5 Thinking and GPT-5.5 Pro, why post-training has become one of the most important frontiers in AI, and why evals, model-as-judge, hallucinations, agentic workflows, GDPval, and continual learning are now central to the next phase of frontier models. Yann also shares why continual learning remains one of AI's biggest unsolved problems three years after ChatGPT, and where startups still have massive room to build as frontier models race ahead.(00:00) - Cold open(00:34) - Intro(01:30) - Why recent AI progress feels like a step function(04:13) - Model reliability & the rollercoaster of shipping 5.5(07:33) - How OpenAI structures vertical and horizontal teams(09:49) - Improving model efficiency and test-time compute(12:32) - Yann Dubois' journey from Switzerland to OpenAI(15:37) - Reasoning in 2026: Real-world utility vs verifiable rewards(18:34) - GPT-5.5 Thinking vs Pro: Scaling test-time compute(20:09) - How reasoning models become more efficient(23:23) - Pre-training scaling and overcoming the data wall(27:03) - Multimodal data, synthetic data, and embodied AI(31:05) - Demystifying mid-training and post-training(37:21) - Does RL create new capabilities in AI?(38:53) - The challenges and frontier of scaling RL(43:09) - Is building AI models a craft or a strict science?(48:21) - How AI models generalize across different domains(54:18) - How reinforcement learning cures AI hallucinations(56:04) - Negative generalization and conflicting instructions(58:05) - Can RL scale to law, medicine, and the broader economy?(1:00:19) - The evaluation bottleneck and Model as a Judge(1:04:21) - Continuous AI progress & continual learning(1:08:49) - Will foundation models eat the agent harness?(1:11:23) - Why startups should focus on the last mile of AI

Favoriten Christer Olsson är tillbaka i Framgångspodden – i ett varmt, tankeväckande och ögonöppnande samtal om livet, relationer och varför lyckan inte är något man hittar, utan något man tränar upp.Vi pratar om vikten av att verkligen se livet medan det pågår, vilket leder in på Christers livsfilosofi: man har inte en bra dag – man gör en bra dag. Ett uttryck som gjort honom viral på TikTok och som fångar kärnan i hans budskap: livet händer inte sen, det händer nu.Varför jagar så många lyckan på fel plats? Och varför är tacksamhet en färdighet snarare än en känsla? Christer delar också insikter från ledarna han coachar – där det som oftast blir lidande är familjen. Hur ser våra kalendrar egentligen ut? Och varför får barnen så ofta bara tiden som blir över?Vi pratar även om AI och varför tekniken paradoxalt nog verkar öka bluffsyndromet hos många människor. Dessutom går vi in på relationer, varför kärlek är något man gör snarare än något man känner, hans omtalade “80-procentsregel” och varför man ibland måste våga lämna relationer som slutat utvecklas.Det blir ett starkt avsnitt om självkänsla, mod och mening, fullt av insikter som stannar kvar länge efter att samtalet är slut. Följ Christer härLäs mer om Christer härLäs mer om Framgångsakademin här.Ta del av Framgångsakademins kurser.Beställ "Mitt Framgångsår".Följ Alexander Pärleros på Instagram.Följ Alexander Pärleros på Tiktok.Bästa tipsen från avsnittet i Nyhetsbrevet. Hosted on Acast. See acast.com/privacy for more information.

A Good Omens ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠fanfic by mostlyeffable⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠. Part 4 of the Unkind Regards series. Full name: Don't Stop Me Now.Music: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Mainstream Music 2025 Vol. 8, Produced by Sascha Ende ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠(⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠CC-BY 4.0⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠) Sounds: Email notification: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/OptronTeamFilms/sounds/521094/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ (CC-0)Text notification (Crowley): ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/GabrielAraujo/sounds/242502/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠(CC-0)Text notification (Aziraphale): ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/mickleness/sounds/269185/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ (CC-0)Phone ringtone: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/jhyland/sounds/539661/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ (CC-0)Phone vibration: ⁠https://freesound.org/people/eobmada/sounds/541367/⁠ (CC-0)RL knock: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://freesound.org/people/Dreadwolf910/sounds/615987/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ (CC-0)For tags and other details, to leave kudos and comments, please visit the corresponding post on archiveofourown: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://archiveofourown.org/works/83306061⁠⁠⁠⁠⁠!

Our 245th episode with a summary and discussion of last week's big AI news!Recorded on 05/13/2026Hosted by Andrey Kurenkov and Jeremie HarrisFeel free to email us your questions and feedback at andreyvkurenkov@gmail.com and/or hello@gladstone.aiRead out our text newsletter and comment on the podcast at https://lastweekin.ai/In this episode:OpenAI released new voice intelligence API features including GPT Realtime 2 (GPT-5-powered) plus realtime translation and Whisper transcription, emphasizing the latency–reasoning tradeoff, larger context, and new guardrails amid fraud risks.Thinking Machines previewed a low-latency, full‑duplex conversational system with a two-model architecture and custom inference stack, reporting strong interactivity benchmark results but without public access or third‑party validation yet.Anthropic pushed further into vertical products with Claude for Legal and deeper AWS availability, while ongoing ecosystem tension grows as platform model providers compete with application-layer companies.Safety, policy, and research updates included OpenAI's self-harm trusted contact feature, Anthropic work on reducing agent misalignment by training ethical “why” reasoning, OpenAI's investigation of accidental chain-of-thought grading in RL, and Meta horizon eval updates showing benchmarking limits for long task horizons.Timestamps:(00:00:10) Intro / Banter(00:01:35) Response to listener comments(00:03:27) Sponsor Break Tools & Apps(00:06:27) OpenAI launches new voice intelligence features in its API | TechCrunch(00:15:52) Thinking Machines drops a new, highly responsive model designed for humanlike interactions in real time - SiliconANGLE(00:27:49) Claude For Legal Launches, May Reshape the Legal Tech World – Artificial Lawyer(00:40:27) Threads tests a Meta AI integration that works similarly to Grok | TechCrunch(00:43:08) Google brings agentic AI and vibe-coded widgets to Android | TechCrunch(00:45:33) Google updates AI search to include quotes from Reddit and other sources | TechCrunch Applications & Business(00:47:38) Sam Altman was winning on the stand, but it might not be enough | The Verge(00:55:04) Nvidia C.E.O. Jensen Huang Hitches Ride With Trump to China After Last-Minute Invite - The New York Times(00:58:40) AWS expands Anthropic partnership with Claude Platform launch(01:01:13) Chinese grey market sells Claude API access at 90% off by using stolen credentials, model substitution, and harvesting users' prompts and outputs for resale as AI training data — 'transfer stations' operate through proxy networks that harvest user data(01:06:43) DeepMind Spinout Isomorphic Labs Raises $2.1 Billion to Design Drugs With AI - BloombergProjects & Open Source(01:09:04) Petri: Anthropic Hands Its Alignment Toolbox to Meridian Labs with 3.0 Update(01:12:25) Daybreak': OpenAI's Answer to Anthropic's Project Glasswing Has ArrivedPolicy & Safety(01:14:04) Teaching Claude why(01:21:45) Import AI 455: Automating AI Research(01:28:31) ChatGPT's New Safety Feature Could Alert 'Trusted Contact' to Risk of Self-Harm - CNET(01:30:09) Investigating the consequences of accidentally grading CoT during RL(01:34:46) Natural Language Autoencoders criticism(01:39:15) Review of the "Risks from automated R&D" section in the Anthropic Risk Report (February 2026)Synthetic Media & Art(01:43:39) George Clooney, Tom Hanks, and Meryl Streep back new ‘Human Consent Standard' for AI licensing | The VergeResearch & Advancements(01:45:10) METR says Claude Mythos is testing the limits of AI evaluation – Startup FortuneSee Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

If AI agents are the new digital knowledge workers, where exactly do they do their work? In this episode of the MAD Podcast, Ivan Burazin joins us to unpack the emerging infrastructure stack for AI agents and explain why every agent needs its own secure, stateful "computer." We explore the technical realities of sandboxes, dive into why legacy, stateless hyperscalers weren't built for these new workloads, and break down the mechanics of microVMs and custom schedulers alongside a contrarian prediction on an impending CPU shortage. Finally, Ivan delivers an absolute masterclass on product-led growth, community building, and go-to-market strategy for technical founders.(00:40) Intro(02:13) What is an AI agent sandbox?(03:17) Security risks of running agents locally(05:17) Stateful vs. stateless hyperscalers(07:04) The history of cloud IDEs and the end of localhost(09:45) Do all AI agents need a sandbox?(12:26) Sandbox use cases: RL evals & background agents(14:10) Unpacking the emerging AI Agent Stack(16:20) The unsolved problem of agent memory and learning(19:37) Where sandboxes fit in the agent harness(21:35) OpenAI, Anthropic, and agent SDKs(23:06) Ivan's founder journey: From CodeAnywhere to Daytona(26:59) GTM strategies and building developer communities(33:48) Why customer support is your best GTM strategy(35:34) Leveraging Twitter during the AI super cycle(40:50) The technical anatomy of a sandbox(41:53) Why fast spin-up speeds maximize GPU efficiency(46:09) Firecracker, QEMU, and isolation primitives(49:58) Why sandbox snapshots and state forking matter(51:40) Why Daytona built a custom scheduler from scratch(55:24) The challenge of long-running stateful sandboxes(58:10) The build your own sandbox trap(1:01:03) Why AI agents might trigger a global CPU shortage(1:02:46) The future of the AI Agent Stack

When OpenAI trained GPT-3, they didn't roll their own orchestration layer. They used Ray, an open source Python framework born out of the same Berkeley research lab lineage that gave us Apache Spark. And here's the twist: Ray was originally built for reinforcement learning research, then quietly faded as RL hit a wall. Until ChatGPT showed up. Suddenly reinforcement learning was back, as the post-training step that turns a raw language model into something genuinely useful. Edward Oakes and Richard Liaw, two founding engineers behind Ray and Anyscale, join me on Talk Python to tell that story. We'll trace Ray from its RISE Lab origins at UC Berkeley to powering some of the largest training runs in the world. We'll talk about what Ray actually is, a distributed execution engine for AI workloads, and how a few lines of Python become work running across hundreds of GPUs. We'll cover Ray Data for multimodal pipelines, the dashboard, the VS Code remote debugger, KubRay for Kubernetes, and where Ray fits alongside Dask, multiprocessing, and asyncio. If you've ever stared at a single-machine Python script and thought, "there has to be a better way to scale this", this one's for you Episode sponsors Sentry Error Monitoring, Code talkpython26 AgentField AI Talk Python Courses Links from the show Guests Richard Liaw: github.com Edward Oakes: github.com Ray: www.ray.io Example code (we used for walk-through): docs.ray.io Getting Started with Ray: docs.ray.io Ray Libraries: docs.ray.io kuberay: github.com Watch this episode on YouTube: youtube.com Episode #547 deep-dive: talkpython.fm/547 Episode transcripts: talkpython.fm Theme Song: Developer Rap