Podcasts about Oof

Why did Earth's magnetic field crack open this week, and why does your body feel like it's picking up on it? What if the chaos swirling around you right now isn't a crisis to survive, but an invitation to come home to yourself? Michael welcomes back evidential medium and former naval officer Suzanne Giesemann, who connects with her guide Sanaya, a collective consciousness she's channeled consistently since 2010. Recorded the same week scientists confirmed a portal opened in the magnetosphere, this conversation digs into why so many people are feeling unmoored right now and the one practice Suzanne says can anchor you no matter what's happening on the outside. This isn't about predicting what's coming. This is about learning to return to the base beneath the story. Key Topics: What Sanaya says is actually happening to humanity right now, and why the message hasn't changed since 2010. The "widen the lens" teaching: why focusing on your personal storm makes everything look worse than it is. Suzanne's "return to base" (RTB) practice, and the overhead-projector analogy that makes it click: you are the light, your story is just the overlay. "Oof vs. Ah": the difference between living from outward objective focus and living from the base that's always already here. Why caregiving for a parent in chaos, and a mother living in the "honeymoon period" of Alzheimer's, became its own teaching on compassion over judgment. The empath's dilemma and how a simple nervous system reset changes everything. Why what looks like "a split" in humanity isn't separation at all, just different levels of attention to the overlay. How to build undeniable trust in your own intuition with a "miracle journal" of NOEs (No Other Explanations). Two real evidential readings, including a message that arrived eight years after it was first given, and a rainbow that stopped two grieving parents cold. What happens, according to hundreds of readings, to souls who die by suicide, and why none of them describe anything resembling hell. What Sanaya says about AI: a genuinely useful tool, as long as you never stop turning to spirit first. Suzanne's October 11th Channeling for Charity event benefiting the Jed Foundation's suicide prevention work. You are not your story. Beneath every title, every role, every "I have to," there is a base that never shifts, and that base is where your loved ones, your guides, and your own peace have been waiting the whole time. You can't stop the waves of this moment. But you can learn to return to base, turn up the love, and surf.

Send us Fan MailWhat If Their No Isn't About You?Ever explain something you're wildly excited about… only to be met with a blank stare, a lukewarm response, or the dreaded “Yeah, that's not really for me”?Oof.When you care deeply about what you're creating, a “no” can feel surprisingly personal. Suddenly you're questioning the idea, your approach, maybe even yourself.But what if rejection isn't the problem?In this episode, Alex gets real about an enrolment conversation that didn't go the way she hoped and the frustration that followed. Instead of spiralling into “What am I doing wrong?”, she got curious.Because our brains love to make meaning out of rejection. But curiosity gives us another option.Are you actually connected to the deeper why behind what you're asking for?And are you listening closely enough to understand what matters to the person on the other side?Sometimes the answer isn't pushing harder. It's getting clearer.Get curious. Find the gap. Make the ask.

Welcome to another episode of Before the Box Score! Oof what a tough loss. But what can we take away from this? Should we feel bad or hopeful? Nate and Nathan try to help answer those questions and more as they break down Mizzou's tough loss on the road at Mississippi State. Subscribe to Rock M+ for access to Mizzou insider info, discussion boards, special live podcasts just for subscribers, and more! You can follow members of today's show on Twitter @BurstaHurst & ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠@NateGEdwards⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠. Have a question for one of our podcasts? Leave a 5 star review with your question and that show just might answer it in an upcoming episode! Do you like Rock M Radio? Drop us a Review and be sure to subscribe to Rock M Radio on your preferred podcasting platform. Be sure to follow @RockMNation and @RockMRadio on Twitter. And if you aren't subscribed yet, please subscribe to our YouTube channel! Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

From the earliest days of open-weight models to becoming the neutral routing layer for more than 10 million developers, OpenRouter is one of the clearest bets that the future of AI will be multi-model. In this episode, OpenRouter co-founder & CEO Alex Atallah, with AMP's Anjney Midha returning with swyx to unpack how OpenRouter emerged from the first wave of Llama, Alpaca, Mistral, and Midjourney, why model diversity mattered before it was consensus, and how a company dismissed as “just a wrapper” became critical infrastructure for the AI ecosystem.We go deep on the product and distribution lessons behind OpenRouter: why model labs can spend billions training a checkpoint and still struggle to get it into developers' hands, how Mistral helped prove the value of a competitive inference marketplace, why OpenRouter chose focus over expanding into fine-tuning, memory, and other adjacent products, and how its rankings became a real-time map of how AI usage was changing. Alex also explains OpenRouter's early experiments with model fusion, why they deleted the first version and brought it back years later, and how the platform grew to more than 10 trillion tokens per day.Finally, Anjney explains why Stripe and OpenRouter fit together, why token fraud may become one of the defining security problems of the AI economy, and why the next wave of fraud won't just come from humans but from autonomous agents attacking increasingly valuable token flows.We discuss:* Why OpenRouter bet early that no single AI model would win everything* Alpaca, Llama, and open models becoming impossible to ignore* Why Discord's early AI deployments exposed the limitations of closed models* Why model labs can spend billions on training and still fail at distribution* How OpenRouter became a neutral distribution layer for model developers* Why VCs dismissed OpenRouter as “just a marketplace” or “just a wrapper”* The Mistral price war and the first real proof of an inference marketplace* How Midjourney scaled through Discord and what it taught the AI ecosystem* Why crypto infrastructure became a dress rehearsal for generative AI* OpenRouter vs. LM Arena and why their missions are fundamentally different* Why focus became one of OpenRouter's biggest strategic advantages* Anthropic's early focus on AI pair programming and coding* The OpenRouter products that were prototyped but never launched* MOM, OpenRouter's early Mixture of Models experiment* Why model fusion failed in 2024 — and why it works much better now* How OpenRouter's leaderboard became a live map of the AI industry* OpenClaw, auto-routing, and agents reshaping AI usage* How OpenRouter reached 10+ trillion tokens per day* Why inference gateways are increasingly becoming targets for fraud* Why Stripe's fraud infrastructure is strategically important to OpenRouter* The coming rise of agentic fraud and attacks on the token economy* What changes and what stays the same as OpenRouter joins StripeAlex Atallah* LinkedIn: https://www.linkedin.com/in/alexatallah/* X: https://x.com/alexatallah* Website: https://alexatallah.comAnjney Midha* LinkedIn: https://www.linkedin.com/in/anjney/* X: https://x.com/AnjneyMidha* AMP: https://www.amppublic.com/Timestamps00:00:00 Introduction00:02:12 Alpaca, Llama, and the Multi-Model Bet00:06:04 Discord, Open Models, and OpenRouter's Origins00:14:28 Why “One Model Wins” Was the Wrong Bet00:17:27 Why Model Labs Struggle With Distribution00:23:04 “Just a Wrapper”: Why VCs Misunderstood OpenRouter00:27:58 Bootstrapping OpenRouter Through Community00:36:16 Crypto, Midjourney, and the Early Generative AI Ecosystem00:43:38 Mistral and the Birth of the Inference Marketplace00:47:10 OpenRouter vs. LM Arena00:52:08 Focus, Anthropic, and Roads Not Taken00:59:34 Mixture of Models and Model Fusion01:02:44 Sonnet, OpenClaw, and OpenRouter's Explosive Growth01:09:03 Why Stripe Acquired OpenRouter01:12:45 Fraud and the Emerging Token Economy01:17:47 The Coming Wave of Agentic Fraud01:19:07 What's Next for OpenRouter at StripeTranscriptIntroduction: OpenRouter, Marketplaces, and Pub-Sub as a Product PrincipleSwyx [00:00:00]: Okay, we are here in Anja's house, which is where all big startups in San Francisco start.Anjney Midha [00:00:08]: Howdy.Swyx [00:00:08]: And, congrats on Cursor, Mistral. I don'- God knows what else. You got so much stuff going on.Anjney Midha [00:00:17]: There's, there's a lot going on. Well, OpenRouter is probably the - has been the most, I would say, like, one I'm excited about recently.Swyx [00:00:24]: Yeah. And we have Alex, first time on the pod, but,Anjney Midha [00:00:27]: Thanks for having me.Swyx [00:00:27]: You've been in the IE a few times. I appreciate every time you've shown up, for the community. Congrats. I just, like, what a journey. When I was looking back at your past posts, one of the earliest principles that I saw you write as a product person is sub as a product principle. And I wanted - you to maybe explain how you think about what should exist in the world.Anjney Midha [00:00:49]: Yeah. The sub piece, which was early 2023, I didn't think about it until we talked like 10 minutes ago, is about how there is like a way of thinking about products as an intersection between subscribing to data and publishing data. And marketplaces are an easy example of this. You have suppliers that are publishing some product to a SKU. And the SKU is like a sub topic that a consumer is subscribing to and just going to, like, consume whenever they want. And humans consume in a very, like, discreet, ad hoc way. It's not very scalable. all their attention is on the topic when they're buying the thing, and their attention is nowhere else when that happens. agents and consumers of inference don't act like that. They're consuming continuously, and they're changing the SKUs that they consume from all the time. So OpenRouter is like a blend between a normal API experience and a marketplace where we create model slug. We have the auto router. We have all kinds of, like, product SKUs that you can subscribe to. And then you can, like, continuously add, like, derive value and make decisions based on those consumers.Alpaca, Llama, and the Multi-Model BetSwyx [00:02:11]: Yeah. This is something that was more consensus now, but not consensus when you guys started, which was that there is such a demand for swapping models and changing things out and, that people would not use the native SDKs. I guess, for each of you, what was your realization moment that this would be it? I, - You've, you've given a talk at EIE about Alpaca as,Anjney Midha [00:02:33]: Yeah.Swyx [00:02:33]: One of your inspiring moments.Anjney Midha [00:02:35]: Alpaca, I can, like, rehash the Alpaca moment for a sec. Like, the very beginning, at the end of 2022, OpenAI was the only game in town. There was, like, OpenAI, Cohere,Swyx [00:02:47]: Yes.Anjney Midha [00:02:48]: And then a smattering of, like, early attempts at open weight models.Swyx [00:02:54]: Yeah.Anjney Midha [00:02:54]: When Llama came out in January of 2023, it was like, “Wow, really exciting. This is really big.” It outperforms 3 on, one or two benchmarks. but you can't chat with it. It wasn't like - It wasn't an engaging model, but it seemed like someone just needed to fix a couple things and do some RLHF on it to get it all the way there. And Alpaca was the first model that I saw that did that. It only took $600 to do. A team at Stanford generated a bunch of synthetic data, tuned Llama, and made Alpaca, billion parameter model. Or was - Maybe it was thirteen billion parameters. And it was so good. Like, I was just, like, on an airplane using it. I, - in many cases, I, like, you could not discern a ChatGPT versus an Alpaca result. And I figured if it was this easy to make a model, one, we have a whole new way of monetizing data for the first time. you can just, like, take really valuable data and turn it into a service in $600. and that cost will probably go down over time.Swyx [00:04:03]: When you - So sorry. when you say monetizing your data as, what eventually will become an MCP endpoint or as a training data for a model?Anjney Midha [00:04:12]: Yeah, training data for a model.Swyx [00:04:13]: Awesome.Anjney Midha [00:04:13]: Like, an abstract way of saying like, “Hey, I have this data.”Swyx [00:04:15]: Compress it into a model.Anjney Midha [00:04:16]: Like, it makes sense for me in my product, but, like, I could repackage it in the form of a model and sell it. And so it's just a whole new business model for the economy. It also, of course, provides, like, a way of following what Frontier Labs are doing, but in a way that, like, a single developer or a small team of developers can roll on their own. And so - Whenever you have an example of that, like a breakout app that's doing really well, and then some framework for imitating it with - in your own flavor, you have an immediate ecosystem of, like an immediate ecosystem, like, should arise because there's just a huge gap between the, like, decisions that the single company is making and all of the variations in those decisions that, like, a wider ecosystem can create themselves. And so then, you need a marketplace to, like, discover all of those, services and all of those products. There wasn't any place on the internet that, like, was like a home base for LLMs in terms of seeing how much they were being used and seeing who was using them and why.Swyx [00:05:29]: The closest would be Hugging Face.Anjney Midha [00:05:30]: Hugging Face was the closest at the time, yeah.Swyx [00:05:31]: They just started Hugging, like, a few years ago before that.Anjney Midha [00:05:34]: Yeah, and Hugging Face also didn't have the closed-source models.Swyx [00:05:37]: Yeah.Anjney Midha [00:05:38]: And they didn'- you couldn't use the models at the time. and there wasn't data about who was using them. There were, like, a bunch of differences between OpenRouter and Hugging Face, and those differences felt really critical to me, especially when I was just trying to learn about LLMs and, like, why people are choosing, like, Different little ones that are emerging over time.Discord, Open Models, and the Origins of OpenRouterSwyx [00:06:03]: Got it. And then, Ansh, no stranger to wanting more model diversity, at the time, you're a couple of years into your Anthropic journey, which we covered in the previous podcast as well. What was your introduction to Alex?Alex Atallah [00:06:16]: Well, the introduction was, I think, thirteen years before that.Swyx [00:06:20]: Oh.Alex Atallah [00:06:20]: But the OpenRouter handshake happened right over there, if you remember.Anjney Midha [00:06:23]: Yeah.Alex Atallah [00:06:24]: Which - So Alex and I, met, I believe as sophomores now, if I remember at the Stanford Review,Anjney Midha [00:06:32]: That's rightAlex Atallah [00:06:32]: Meeting for the first time.Anjney Midha [00:06:33]: I think so, yeah.Alex Atallah [00:06:35]: Yeah.Anjney Midha [00:06:35]: Yeah.Alex Atallah [00:06:35]: So Stanford Review was the libertarian newspaper on campus at Stanford that Peter Thiel started back in the day. And, whatever-- for whatever reason, I, Alex and I both showed up to one of the meetings, and I remember, the editor-chief was a mutual friend of ours. Lisa was really a really great editor-chief, where, part of an editor-chief's job is to assign responsibilities to people and make sure the work gets done. and I, I may be misremembering the details, but I remember wanting to. It was surprising to me that at the time there was no dedicated technology section in the newspaper.Alex Atallah [00:07:11]: YouSwyx [00:07:13]: Because it's political, right?Alex Atallah [00:07:14]: It is primarilySwyx [00:07:14]: Like, it's talkingAlex Atallah [00:07:15]: It originally started as like aAnjney Midha [00:07:16]: Yes.Swyx [00:07:17]: Yeah, states and all those things.Alex Atallah [00:07:17]: Correct.Swyx [00:07:18]: Yeah.Alex Atallah [00:07:18]: But it, - To take us back in time, you may remember this, but, there was this technology, legislation that was being debated called, the Net Neutrality Act. And net neutrality is, like, inherently this political concept, right? It's, it's about the regulation of - internet broadband access. And so there was a community of us who were technologists, but also debating the politics of the technology. And I thought the Review would be a great place - to, like, write about that. And I was working on, I think, a net neutrality article, and I remember proposing, “Well, maybe we should start a technology section.” And Alex was one of the only people who said, “Yes, that would be cool.” And said. I forget whether we ended up writing stuff together, but - that's when we first met,Alex Atallah [00:08:03]: Was 2011 or twelve. I forget which year it was. It was one of those.Anjney Midha [00:08:09]: Yeah.Alex Atallah [00:08:09]: It was at Old Union, if I remember correctly.Alex Atallah [00:08:11]: That's where we used to meet. But, along the way, Alex and I have had a chance to, To hang out often. And probably the time when we had the most professional overlap was when I was running the platform at Discord, and it had become this explosive platform for cryptoSwyx [00:08:32]: YeahAlex Atallah [00:08:32]: And NFTs in the middle of the pandemic.Swyx [00:08:35]: Which also, by the way, you were in charge of safety and security as well, right?Alex Atallah [00:08:38]: I was the head of platform, which meant all of the crypto - the DAO and NFT launch security debugging fell onSwyx [00:08:45]: And their phishing and.Alex Atallah [00:08:47]: The phishing, the social engineering attacks, the katana DDoS that we were getting hit by. but it's around the time I first started teaching security at scale at Stanford, CS 153. And Alex was on the, - at OpenSea at the time, and I was trying to figure out how we could defend against all these attacks that we were. Like, and at peak, I forget, if you remember how much NFT volume was running throughSwyx [00:09:10]: DiscordAlex Atallah [00:09:10]: Discord, but it was, like, a meaningful amount of, like, it was, like, several billion dollars in NFT volume of GMV, so to speak, were running through the platform, and it was all coming from OpenSea. It was these, like, buy, sell,Swyx [00:09:20]: TheAlex Atallah [00:09:21]: ServersSwyx [00:09:21]: The D in DAO is Discord.Alex Atallah [00:09:25]: Yes. And so that's when I think we had hung out professionally. But a year after that, OpenAI gave Discord early access to GPT. Sorry, three. No, it was five. Yeah, five, which is the RL version of three. And that's around the time we made a Discord bot with, OpenAI for internal deployment, and that's when I realized we would need. Like, since I was part of the deployment team.Anjney Midha [00:09:50]: What was the use case?Alex Atallah [00:09:51]: There were two that were. And there's, there's a post now called “Discord is Your Place for AI with Friends” that somebody sent me recently that I wrote, and published in twenty-three. But There were two use cases. One was Clyde, which was the - like, a party friend inside of Discord that could help you set up your Discord server and talk to you about onboarding and get your friends to hang out more. and then there was content moderation. And one of the realizations we had with content moderation was - it would refuse to moderate. Like, it would just refuse our prompts because the The training was. We were very early in the training era, and it would just. Our prompts would trigger it, its, like, guardrails. And we told OpenAI, “Hey, guys, we need access to the weights because if we're gonna be doing content moderation at scale, we had 250 million monthly active users, we need more reliability that the model will do what we need it to.” And they said, “Well, sorry, guys, that's not how this works. We're a closed-source company.” And so that was my first realization that we needed open models, and the enterprises would need more control over capabilities, and then ultimately would need some control plane or management system to orchestrate these open models. But there weren't no good - there were no good open alternatives until maybeAlex Atallah [00:11:10]: Six months later when Llama came out. And six months after that, I led the series A into Mistral, which was started by Guillaume and the Llama team. And - That, - Around that time is when I remember hearing about Alex launching OpenRouter and going, “These worlds are gonna collide, and I don't know when it'll make sense to team up.” But Alex was so early and could see. I think he was totally right about this ecosystem starting with Llama that then needed, like, a, an easy layer to manage for, especially for. I was approaching it from the enterprise perspective because I had been that, like, the. As the VP of platform at Discord, it was my job to ensure that when we deployed models to, like, 250 million users, they did what we wanted them to. And that was very hard, because if you outsourced it to the labs and they controlled the guardrails and their guardrails are their safety policies. Forbid the model from responding to your prompts. That was quite catastrophic.Swyx [00:12:05]: Yeah. But what, a moderation is the thing that they want to support. And obviously, beyond that, they would - OpenAI would work with you, presumably to give you a moderation endpoint, which they offer for free.Alex Atallah [00:12:16]: It was an interesting use case, that - So they did give us a moderation endpoint. However, as you guys know, every Discord server is like a mini deployment of itself. And so the use case was instead of having human moderators that have to interpret the norms of the community, you just give the, - Often, like every, subreddit, Discord servers, public ones have their own rules that the user, the users create.Swyx [00:12:41]: Oh, yeah. We run the LinkedIn Discord in. Yeah.Alex Atallah [00:12:43]: And then humans used to read those norms and then enforce it every day manually, like observing each message in these communities. And these communities have like millions of users. So we had a 5,000+ person team globally in the, on the Discord content moderation team. These are outsourced contractors who had a really tough job. And so the idea was instead, if you could give the norms of that server To the LLM, then the LLM would do custom moderation for that server. It's almost like a, like context moderation for that server. And many of those servers' norms just violated OpenAI's rules. And so - It was like we had our own custom eval. So each server had its own custom eval. But Discord-- at the time, OpenAI's evals, we were all soAlex Atallah [00:13:28]: Primitive in our thinking about how to deploy these LLMs that often the training prompts were super handed. It said, “Oh, anything about Harry Potter, anything that has trademarked content, don'- refuse.” And if it was a fan - Harry Potter fan community, this is a real use case, that had content moderation, the LLM would just refuse.Swyx [00:13:48]: Yeah.Alex Atallah [00:13:49]: And that was just not precise enough.Anjney Midha [00:13:52]: Another one that we heard was like if someone was trying to write like a detective story, and there's one chapter with a lot of violence, like maybe someoneAlex Atallah [00:14:01]: RightAnjney Midha [00:14:01]: Like kills someone, the LLMs would just refuse to, like, help with that part of the story.Alex Atallah [00:14:07]: Yeah.Anjney Midha [00:14:07]: And then - like, we used to be like, okay, this is not like structurally inherent to LLMs. There must be, like, some choice out there so that I can, like, switch to another model, when I'm getting, like, a refusal or a bad result from the main one that I have. And that, like, tension also drove me for a marketplace.Why “One Model Wins” Was the Wrong BetSwyx [00:14:28]: Yeah. I think that is well accepted now. What was it like back then when you were raising or, starting this? did people get it? what was the, some of the struggles? I like getting stories out of him about how other VCs don't get it. So like anything you wanna, talk about, now - Let's, let's call it, that the early journey of OpenRouter is done, right? You can obviously talk about some of the early days stuff.Anjney Midha [00:14:54]: Well, I was gonna say that, like, the biggest objection we got is big model win, which is - all of theSwyx [00:15:03]: Scaling laws.Anjney Midha [00:15:04]: Huh?Swyx [00:15:04]: Scaling laws.Anjney Midha [00:15:05]: Yeah, scaling laws, and natural network effects are just gonna accrue to one company, which will be - It'll be a Google-style monopoly, just like how Google won the search market, by a large margin, and you'll just be fighting for scraps at the end. That was probably the biggest objection we got. it is interesting that Google won the search engine race with such a huge margin. I think, like, had there been more interesting benchmarks or had, like, search engines been, - had people, like, seen them a little bit more like LLMs where they're services that you can build companies on top of, that might not have been the case. but LLMs don't merely have a user interface. They're also, like, ways of building entirely new businesses. And, a Google-level monopoly would be like the Dutch East India Company times, quadrillion in magnitude because the whole economy ends up, like, depending on the one monopoly as well. So it didn't seem like would be a really crazy outcome if that happened. And it's also less likely because the economics of, like, creating good competitors are much, like, much more decentralizable.Alex Atallah [00:16:25]: Everything Alex said is true, And I came at it from a completely different perspective, whichSwyx [00:16:31]: Yes, this is why we're here.Alex Atallah [00:16:32]: The scaling laws were never - In my mind, were always a feature, not a bug for why OpenRouter would be very valuable. Because, I was one of the first investors in Anthropic, and it was obvious to me that other researchers in our friends - I went to grad school for machine learning, and I just had a lot of friends in the ML community who it was very obvious to us that the bitter lesson holds. And so I was like, “Oh, fantastic. Now we have at least two proof points that compute scaling works.” It was OpenAI and Anthropic. and by the time I think we decided to team up on OpenRouter, I had already invested in Mistral and Black Forest Labs and Luma. So there was multiple model companies and teams that I was, working with.Why Model Labs Struggle With DistributionSwyx [00:17:14]: But you did other modalities, whereas this is literallyAlex Atallah [00:17:16]: Across different modalities, yesSwyx [00:17:17]: Text.Alex Atallah [00:17:18]: Exactly. And it was so obvious to me that an ecosystem of different kinds of models were being created, and that this whole narrative of, like, Only one company will dominate like Google was, well, like maybe true, but one, I don't believe that. But two, there was so much extraordinary innovation happening across several different research teams. But the shared problem I was noticing across all of them was often, the research teams were fantastic at figuring out how to reason about new capabilities. They think in terms of capabilities, but never - like, are not developer mindset-oriented. Like, what happens after the training is done and the checkpoint comes out? Like, you'd be shocked how, like, similar the early training teams at OpenAI, sorry, Anthropic, BFL, Mistral, were in their, like, default approach to. Taking their research out of the, lab and scaling their impact, which is often, oh, the checkpoint is done, put it out as an API, done, and then there'd be crickets. in the case of Claude, the first Claude checkpoint was done a year before they released it internally. And then ChatGPT came out, and we decided, okay, yes, it's a good idea to release a Claude version externally.Alex Atallah [00:18:34]: And they had no plan, like no plan for how to get developers to try it out. And so if you go to the Claude one blog post, you'll notice there are, like, three developer examples for users of the API, and one is a Discord bot, and the second is Vivian, my wife's startup called Juny Learning, ‘- And then there was, like, Notion, because these were all friends of, like, the Anthropic Because that's how - like, last minute the planning was around, hey, once the model's done training, how do you get it out to the world? There was no distribution platform that understood what developers needed, all the key management, provisioning, like, simple, like, endpoint management, versioning control. Like, all these things that the scientists and researchers go, “ that's plumbing. I don't really think about it.”Swyx [00:19:15]: Implementation detail.Alex Atallah [00:19:16]: Right. And instead, Alex came at it from that perspective. And so, it was so obvious to me that, like, every single lab I was funding would spend - like, literally sometimes billions of dollars into training, and then a checkpoint would be done, and there'd be crickets, like, during early access because they're like, “Oh, that's right.”Alex Atallah [00:19:35]: It's hard to use a checkpoint to make anything. You need a whole bunch of plumbing around it to make it usable by a developer. And so by the - I think - it was so obvious to me that a distribution platform like OpenRouter was critical to have in the ecosystem if we wanted there to be competition to Google. Like, unless-- ‘cause with Google, DeepMind is done training a new checkpoint, and then they push a button, and it gets blasted out across all their surfaces from Google Docs to,Swyx [00:20:01]: Everywhere, even if I don't want it.Alex Atallah [00:20:02]: Everywhere. You wanna know about, like, on Android, like, overnight, they can deploy a new checkpoint to, like, a billion devices, right? And that invisible infra advantage, distribution advantage, most people don't realize, but until OpenRouter showed up, - you had to think about all of that yourself as a model lab. And it was very daunting. at Anthropic, I think it took, well, more than twelve months to get to our first 10 million in revenue. And in contrast with Black Forest Labs, I remember the early days, you guys had a conversation with the BFL team, and, it was so simple for OpenRouter to say, “Oh, no problem. Like, the day you launch, we can send 1 million developers to you.” that was crazy. That was like a step function change in, like, an hour.Swyx [00:20:46]: Is that a real number, a million?Alex Atallah [00:20:47]: I,Swyx [00:20:48]: Okay. All right.Alex Atallah [00:20:48]: I think today it's, like, 4 million. How many developers are on OpenRouter today?Anjney Midha [00:20:52]: Over ten,Alex Atallah [00:20:54]: Yeah.Anjney Midha [00:20:54]: Over 10 million, but, like, it's, it's hard to, youAlex Atallah [00:20:59]: I, yeah, I don't know how to. Yeah.Anjney Midha [00:21:00]: We do a lot of, like, account duping work, but, noAlex Atallah [00:21:04]: If you could get 1,000 developers, just to put in context If you get 1,000 developers who try the model on day one after you release it and just, like, do inference and give you feedback, that's a thousandAnjney Midha [00:21:15]: That's hugeAlex Atallah [00:21:16]: More developers than they knew how to get to on their own.Swyx [00:21:19]: Well, BFL had a reputation, but yes.Alex Atallah [00:21:21]: They had one in Stable Diffusion.Swyx [00:21:22]: Yeah.Alex Atallah [00:21:23]: And with Mistral, I don't know if you guys remember, but the first checkpoint they released was, like, torrents. It was, like, torrent weights.Swyx [00:21:31]: Yeah, they just put up a magnet link.Alex Atallah [00:21:33]: Yeah, there was no API.Anjney Midha [00:21:34]: Yeah.Alex Atallah [00:21:34]: Because they didn'- they weren't infra people.Alex Atallah [00:21:37]: ? Like, it's like, okay, download these weights, and you guys go figure out how to host it.Swyx [00:21:39]: Well, he has a story on his side, yeah.Anjney Midha [00:21:41]: Yeah, in addition to the, like, building a really good developer experience around it, the marketing that we do on, like, for different models is totally different and perceived totally differentlyAlex Atallah [00:21:54]: RightAnjney Midha [00:21:54]: From the marketing that a model lab does for itself.Alex Atallah [00:21:56]: Yes, 1,000%.Anjney Midha [00:21:57]: Right? We are like a, neutral layer looking at this market like it's a big dark room with all the corners completely obscure to users, and users are walking into the room and, like, feeling aroundAlex Atallah [00:22:09]: YeahAnjney Midha [00:22:09]: And trying to figure out what objects to grab off the tables and, like, build into, their companies. And it's just an insane way of working. Like, models are not products where you can just enumerate all their features onto a web page. They're all black boxes, including the open weight ones. So you need to, like, shine lights on all corners of this room, so that people can see what makes this model good, and you need the company shining that light to be a neutral third party, which is what we specialize in. So the, like. It'- In addition to developer experience, there's also, like, a very important, like, marketing and product packaging componentAlex Atallah [00:22:50]: YeahAnjney Midha [00:22:50]: And a way of, like, routing and discovering models becomes, like, critical to your market as a provider or a model lab or a server tool and more in the future.“Just a Wrapper”: Why VCs Misunderstood OpenRouterAlex Atallah [00:23:03]: And this value, to your earlier point about how many VCs, like, just don't. One of my biggest frustrations is that venture capitalists, many of them, like, just don't have any operating experience in the field. so unlike a traditional investor who's just maybe come up through the ranks as, like, a associate working on financial modeling or maybe hasn't been a real operator in the field for, like, more than ten years, which is a big part of the industry now, I had just arrived at a16z, like, a year after running the platform. And so I knew what the challenges were of, like, building a real - great developer experience and like, being able to create a working piece of software with a model. And there were a few, I won't name names, but there were investors who were looking at OpenRouter, and, felt at the time, like, when I would compare notes with people, that it was just, I quote unquote, “just a marketplace.”Swyx [00:23:59]: Yeah, just a thin layer, just aAlex Atallah [00:24:00]: CorrectSwyx [00:24:00]: JustAlex Atallah [00:24:01]: A wrapper or whatever on other people's APIs. And I was like, “You have no idea how strategic the value that OpenRouter has created by being able to orchestrate even three.” APIs in production. The amount of both engineering work and community design that goes into getting that live and running in production at the scale the OpenRouter team had started just doesn't happen by default. And that was one of the things that stood out to me about Alex from the earliest days. Like, he just understood, like, - from a systems perspective, like, how do you get these flywheels going? Like, that stood out to me with OpenSea when we were working together on the NFT integration at Discord. Like, Alex had a level of community-- like, systems thinking on how you get these flywheels going that most scientists and machine learning people just don'tAlex Atallah [00:24:48]: Think of. Like, we often think in terms of training.Swyx [00:24:52]: It's a linear stage.Alex Atallah [00:24:53]: It's this linear pipeline.Swyx [00:24:53]: There's no loop yet.Alex Atallah [00:24:54]: Yeah. It wasn't until much later that the modern context feedback loop cycle really got standardized in the industry. But at the time, if you remember, machine learning was like. Like, mostly we did a lot of ML, like, when I was in grad school on a laptop. So you just, like, download a dataset, ran some ablations, and you looked at the loss curves, and you're like, “Great, I made AI.” And the idea that you have to, like, deploy those capabilities, collect feedback trajectories, then, like, put those into a continuous loop, like, came much later. And it was very counterintuitive to the - like, the traditional AI mindset. I do remember doing the investment phase for, OpenRouter, I just didn't try and educate a bunch of other VCs on why it was not just a marketplace. I was like, “ what? I'm just gonna invest.”Anjney Midha [00:25:41]: Yeah.Alex Atallah [00:25:41]: And I'm going to, like, take the opportunity to partner with Alex, and if - no other VCs get it, that's totally fine. ‘Cause at the time, - it was not obvious, I think, to several of the investors that, like, OpenRouter was not more than just a wrapper around APIs. And - that infuriated me. And I was like, “ what? I don't have time to debate you. I'm - we're gonna, we're gonna invest.” And then I think, like, a month later, Matt Murphy marked it up by 10x. Like, - I think. I forget what the exact money was and so on, but, to his credit, Menlo Ventures realized, “Okay, there's much more strategic value here as well.” Maybe you didn't hear all these conversations behind the scenes But that frustrated me a lot. there's a lot of this, like, opining about wrappers. and if you're like, “Oh, an app is just a wrapper on a model,” then, like. And, OpenRouter is, like, this wrapper on top of other APIs, and this is the most stupid, reductive framework.Alex Atallah [00:26:31]: And so it's clearly somebody who has no experience deploying product at scale.Swyx [00:26:34]: It's the thing you dismiss other things with. Like, you're a - everyone's a wrapper on everything, right? Like, and there's, there's some Some wrappers have value.Alex Atallah [00:26:40]: Investors are wrappers and LPs, right?Alex Atallah [00:26:42]: Like venture capitalists. So, yeah, it's all wrappers down, all down to bare metal, I guess, and like energy.Swyx [00:26:46]: Yeah, there - When I started the whole AI engineer, I guess, the coining, in 2023, like, that was, like, the number one pushback is that this is no value. You should just train models.Anjney Midha [00:26:56]: Right.Swyx [00:26:57]: And, yeah, obviously this is, like. you guys are one of the testaments to the fact that you can build very valuable wrappers, but also very valuable model companies.Alex Atallah [00:27:06]: It's so, hard to be. Like, the day a model launches, the fact that you have an OpenRouter, endpoint for that model frequently at the top of Hacker News on day one, people don't realize the amount of work that goes into accomplishing that. And OpenRouter used. Like, that would happen over and over again, and I remember going, “People have no idea how hard that is.”Alex Atallah [00:27:30]: That's not.Swyx [00:27:31]: Yeah, we've covered some of the inference engineering that goes behind,Alex Atallah [00:27:34]: YesSwyx [00:27:34]: Some of - with Base Ten and all those. Well, today you have, all those, like, cool code name things that people guess what Oxy Alpha is and all those things. But, like, I guess one of the things that you're teasing is, how do you get that initial flywheel going, right? Because today you have your scale and your reputation, all these things, so obviously you - you're driving immense distribution. But when you were early on, when it's mostlyBootstrapping OpenRouter Through CommunityAlex Atallah [00:27:55]: The bootstrap, yeah.Swyx [00:27:56]: Yeah.Alex Atallah [00:27:56]: What was the bootstrap like?Anjney Midha [00:27:58]: To bring it back to early Discord days, I think we, like, initially connected with. This is an OpenSea story, technically. But, and we initially connected when you were at Discord, and we talked about, like, - the Axie Infinity server.Alex Atallah [00:28:13]: Oh, yes. Yes.Anjney Midha [00:28:14]: This server was, like, the biggest server at theAlex Atallah [00:28:17]: YeahAnjney Midha [00:28:17]: At Discord.Alex Atallah [00:28:18]: That's right.Anjney Midha [00:28:19]: And you were like, constantly bumping up theAlex Atallah [00:28:22]: The limits on the server. Oh, my GodAnjney Midha [00:28:24]: Of how many people could be in the server.Swyx [00:28:24]: For those who don't know, like, 10% of Philippines was Axie.Alex Atallah [00:28:29]: Was on that server. That's a big hit.Swyx [00:28:31]: It was, like, a meaningful contributor to the GDP of the country.Alex Atallah [00:28:33]: It was an NFT, like, crypto game, but itSwyx [00:28:35]: It was like a Pokémon breeding thing.Anjney Midha [00:28:36]: Yeah.Alex Atallah [00:28:36]: Yeah. Similar. Yeah. There was battling, there was breeding, and then there was, like, a marketplace for trading.Swyx [00:28:43]: Earn as well.Alex Atallah [00:28:45]: Yeah, earn. And, like, the graphics were really cute and fun, and you like, you get emotional about your Axie that you make. So to, like, start a community like that, which we had to do many times at OpenSea with every early project, for us to create a marketplace for it, we need to make sure that the, like, the community wants it.Anjney Midha [00:29:09]: Right.Alex Atallah [00:29:09]: And it's like building something that people want and going and telling them about it. Like, you can do that on a one basis, but there's way higher leverage to do that in a community where everyone can talk to you at the same time. So we spent a lot of time, like, building things that the community really wanted. We did the same thing for OpenRouter. And, like, the Axie community was one of, like, a zillion communities we did that with. And Anj, like, saw us doing it and. ‘Cause you could just see people sharing OpenSea links constantly in that Discord. Like, users sharing links is a really clear indicator that, like, something important is going on. So we spent, a lot of time, like, first figuring out what the gap is in the technology that people care about. Like, what was the actual problem that needs to be solved? in early LLM days, it was, OpenAI refusing to finish the prompt or,Anjney Midha [00:30:09]: YeahAlex Atallah [00:30:10]: To, like, complete the task. It was also.Anjney Midha [00:30:13]: Inability to customize models. and so there are communities that, like are just completely blocked on that issue, and those are the communities that are most useful to learn about and dive into and explore.Alex Atallah [00:30:28]: Something that really struck me at that time, - as I was just hearing your talk, I remember noting - you may not remember this, but we - we had these, like working, Zoom calls that we were doing a sprint around for, like this OpenSea integration with Discord. and, we'd, we'd - it was myself, my engineering team. I think you were there. And I remember, Alex, in the middle of one of those calls, just like there was like silence. we were all like, “Oh, yeah, this totally makes sense. Let's do this.” And then there's - every, like everybody aligned. And Alex was like, “No, this makes no sense to me.” And everyone's - I remember going, “What? Like, it works. Like, you click on a link and this, then it bounces you out to, like, OpenSea.” And he was like, “It's not a good user experience. Yeah, we should not do this.” And I remember going, he was the only one person out of all of us to raise his hand and go, yes, it made sense from a technical implementation perspective. Like, we were bouncing the user out into the, into OpenSea. And so it kinda checked the box of the product manager's requirements on both sides. But Alex went one step further and was like, “ what would be better, guys? If we just embedded the experience right here inside of Discord so the link opened up as an embedded iframe, and you can just check out right there.”Alex Atallah [00:31:47]: And not one person on the call, and there's like seven of us who had met, like, week after week.Swyx [00:31:52]: And it's the guy who doesn't work for Discord.Alex Atallah [00:31:53]: And it's the guy who doesn't work for Discord.Swyx [00:31:55]: Like, technically, you benefit if they bounce.Alex Atallah [00:31:57]: Exactly. And that was, like, adversarial. To keep the user inside of Discord would be adversarial to OpenSea. And yet Alex put that user experience first. And I was like, “That's special.”Swyx [00:32:08]: Wow.Alex Atallah [00:32:08]: Because it's very hard to have somebody who's technical like Alex and understands the developer flow, but also understands the best user experience and wants to prioritize that. And that's two sides of the flywheel that if you can get spinning, like is often hard to stop. And you just reminded me, like that one was one of those moments where I go, I - I realized I gotta be better at user experience because I should have been the one who came up with that, and I didn't. And I learned from you. And, I think that went into one of our case studies for the PM training program at Discord.Swyx [00:32:34]: Whoa.Alex Atallah [00:32:36]: I don't know if it there is Because ofSwyx [00:32:38]: You need an Alex is the conclusion.Alex Atallah [00:32:40]: Yeah. You need an Alex. And this is why I'm not, nobody should be surprised why Stripe decided like they had to buy OpenRouter because it's a really rare combination of people who understand the machine learning community, the developer experience, and the user experience. And putting all that together has resulted in this extraordinary scale that very few other marketplaces have been able to achieveWindow AI, BYOM, and Finding the Right Form FactorSwyx [00:33:02]: Yeah.Alex Atallah [00:33:02]: Over the last, five years.Swyx [00:33:04]: Yeah. Well, we should talk about the other reasons for acquisitions, whichAlex Atallah [00:33:07]: Yes, we should.Swyx [00:33:07]: You've written about. I wanna proceed somewhat chronologically as well. So - there is a point that, one of the questions that, Dave from H of Zero sent in was, when did it - really started to work? And you brought up Mixtral. I don't know if you wanna bring up that story.Alex Atallah [00:33:22]: Oh, yeah.Swyx [00:33:23]: Which obviously you overlap with, so.Anjney Midha [00:33:26]: Yeah, the MoE was. I don't know when. there's no like one moment where I was like, “Oh, this is, officially starting to work.” It wasSwyx [00:33:36]: The moment where you had a Chrome extension, like, really super early on.Anjney Midha [00:33:39]: Oh, yeah. But, well, - yeah. So before OpenRouter, I wanted to, like, explore a bring-your-own-model experiment. And,Swyx [00:33:47]: Which anyone familiar with crypto is like, yeah, Phantom and all these things.Anjney Midha [00:33:50]: Yeah. So it felt like doing a MetaMask analogy for AI would be a fun way of exploring that. And at the time, there were no AI apps. There were probably as many AI apps that were, like, hitting AI - like, hitting an LLM via an API call as there were, like, games just doing it in JavaScript. like there was a, there was a moment in time where it could have been the case that web apps call LLMs through the browser, like through some desktopAlex Atallah [00:34:27]: Yes.Anjney Midha [00:34:27]: Managed app that is controlled by the user. and of course, there are like, I think, many reasons that did not happen. But back when the days were that primordial, I built a Chrome extension called Window AISwyx [00:34:43]: With Plasmo.Anjney Midha [00:34:44]: With Plasmo.Swyx [00:34:45]: I had come across early on, and I was like, “Who's gonna use this?” You did.Anjney Midha [00:34:49]: Plasmo had a couple, like, I think Phantom was using it. there were some other, like real companies using it.Alex Atallah [00:34:56]: It was like a shim.Swyx [00:34:57]: React for Chrome extension. It compiles to allAnjney Midha [00:35:00]: Yeah.Alex Atallah [00:35:00]: I see.Anjney Midha [00:35:00]: Like Next.js for Chrome extensions.Swyx [00:35:01]: Next.js, Next.js.Alex Atallah [00:35:02]: Okay.Anjney Midha [00:35:03]: And yeah, built Window AI on top of it. The creator of Plasmo, like started contributing code to Window AI, in GitHub, and that turned out to be Louis VicchiAlex Atallah [00:35:15]: Oh, you'Anjney Midha [00:35:15]: Who is the founder of OpenRouter.Alex Atallah [00:35:17]: That's right. You have told me this is how you met Louis. Yes.Anjney Midha [00:35:19]: Yeah.Alex Atallah [00:35:19]: Okay.Anjney Midha [00:35:20]: So, that allowed users to like configure which model they wanted to use for a web page in their browser, and then, like the app would just call out to that model when it needed to do things. not the right form factor for LLMs, but, it's like fun experiment. You learn a lot, and like I open sourced it. And the main learning is like, okay, this has to be an API, and it has to look a little bit - like, there has to be more of a developer experience here and more of a discovery experience as well. Like, I don't know where to use these models, and a little Chrome extension is not gonna help me discover. It's not enough real estate. I need more space. I need visuals. I need graphs. I need, examples. I need images. I need to, like, I need to be able to, like explore both as a human and as an agent.Crypto, Midjourney, and the Early Generative AI EcosystemAlex Atallah [00:36:10]: Yeah.Anjney Midha [00:36:10]: So that's how OpenRouter came to be.Alex Atallah [00:36:13]: A meta point that.Alex Atallah [00:36:16]: I think is underappreciated, but Alex is reminding me, is that we were quite lucky that we were so. we were, like, adjacent to the crypto community in those days. Because in hindsight, crypto ended up being like a dress rehearsal for generative models, right? If you think about the Axie experience, Alex is totally right, there were not that many AI apps at the time. And while I was dealing-- my job was to be the head of platform at Discord, which meant to be a general purpose place for communities and friends to create-- for developers to create apps and bots and, other services that could be deployed across Discord. And while 80% of the attention at the time was being spent on crypto, because that's where all the NFT volume was, there was, like, twenty percent of my time I was spending with a friend, who would get hotbot with me and ask me for. We would play Magic: The Gathering on weekends, and he was working on a little Discord bot that could take a text input and turn it into an image, and it was called Midjourney. YouSwyx [00:37:15]: Is that David?Alex Atallah [00:37:15]: It was David Holz.Alex Atallah [00:37:16]: He was a good friend. And David and I have both been failed ARVR founders, in the before that. And, I remember this. Midjourney was one of the fastest-growing communities we had after Axie Infinity started to peter off. And many of the, like, the abstractions and the infrastructure decisions we made to scale Axie happened just in time because they. Axie did this and then fell off a cliff. And then as Midjourney was taking off, we, like, explicitly decided to help David make the server, the Midjourney server, as the primary place for interaction with the model, because it was very hard for people to understand how to use the model if they couldn't see other people using it and copy them. And so the single-player Midjourney web app on its own, like midjourney.com, had, like, terrible retention because people would show up, they'd see this empty field. It's like E 2, and they would type in, like, cat or dog. And it was, like, paralyzing for them to have this blank canvas that they had to fill because they'd never used an AI model before. But instead, in a Discord server, you could see other people using it and riff off of their prompt, and the engagement was off the charts. And so scaling, Midjourney from zero to, like, 10 million monthly actives was a much smoother approach Axie Infinity. And so,Swyx [00:38:29]: Don't forget the best of four pictures, and you choose one.Alex Atallah [00:38:31]: The best, yeah, and then the other, weSwyx [00:38:32]: Which is the feedback loop.Alex Atallah [00:38:33]: The RLHF feedback loop, which, by the way, separately, like, Tom Brown, David and I used to play Magic: The Gathering on weekends. And so, like, it was one group of friends would hang out, and we'd. Like, these concepts were all being discussed all the time. But, there was.Alex Atallah [00:38:47]: I think there were few of us who bridged both the crypto worlds and the AI worlds. And compared to crypto, where it was - the question was always, what's the use case, for this technology? There was never any need to ask that for AI because it's, like, the use case was so visceral. It was like, I can create now anything at - I can imagine. I can write novels, I can code. And the infrastructure that those of us who believed in the distributed systems, like, value of crypto, like the censorship resistance part, found this use case that was explosive. And I think between Midjourney, the, Claude was a Discord bot launch, that we were using internally as an LLM. ElevenLabs had a TTS model that we had on Discord as well. Like, Discord became this petri dish for, like, early apps to innovate. And I don't think it's a coincidence that they found a home there before OpenRouter gave the world, like, a public home store or, like, a, storefront. Discord was this, like, almost petri dish storefront that - had, like, piggybacked on the infra we'd built for crypto communities. And then I think Alex was one of the first people to realize, wait a minute, like, these apps need their own home, on the internet. And then OpenRouter, to me, was a continuation of that community's needs. And of course, there was the crazy distribution that you enabled for a lot of these developers.Why OpenRouter Couldn't Just Live Inside DiscordSwyx [00:40:07]: So then my question is, how come you were. My perception is OpenRouter is not that Discord-centric, right? You have a Discord.Anjney Midha [00:40:14]: Yeah.Swyx [00:40:14]: And you use it to engage your community, but it's not like Midjourney where, like, no, that is like the primary way people experience OpenRouter.Anjney Midha [00:40:21]: Yeah, Midjourney, like, it really helps to see visually really quickly how people are using the model and how to prompt it.Swyx [00:40:29]: Yeah.Anjney Midha [00:40:29]: And I think that is partly why the server was so critical. It's like it is the user experience. It adds a ton.Swyx [00:40:36]: Yes.Anjney Midha [00:40:37]: And you can go the whole mile with just, like, prompting via Midjourney, like, the, via the Midjourney Discord server, getting your images and then sharing them and having fun. For OpenRouter, for LLMs, like, you need a lot of user experience around LLMs to make them, like, really usable.Swyx [00:40:54]: Charge point.Anjney Midha [00:40:55]: And yeah.Anjney Midha [00:40:57]: The, like, seeing the examples of other people is also not as useful because it's a lot of stuff to read. It takes a long time.Swyx [00:41:03]: Yeah.Anjney Midha [00:41:04]: You need, like, based integration. Not possible to do in a Discord server. You need, Or technic- it's possible. I shouldn't say that. It's just not a great developer experience. you need, like, - you need governance for. At the point where you got based integration, now you need governance for managing the LLMs that have access to it, the data policies, which teams. All that stuff needs a lot more than a Discord server can provide. So it's justSwyx [00:41:30]: YeahAnjney Midha [00:41:30]: It's not the right.Alex Atallah [00:41:32]: Well, in addition, you're not wrong, but also there's the very important distinction that, Midjourney was an end user application.Swyx [00:41:40]: Right.Alex Atallah [00:41:40]: And, that's why Discord, which has 250 million monthly end consumers, made, it made sense for Discord to be a host for that application experience. What I knew was gonna happen soon after Midjourney found explosive product-market fit, because we. I think when Midjourney launched, from launch to $100 million revenue run rate, it was less than eight months. And shortly thereafter, Stable Diffusion launched. And, all of us used to hang out in the Discord server. There, I think it was the,Swyx [00:42:13]: The Stability Discord?Alex Atallah [00:42:14]: It was theSwyx [00:42:16]: Yeah, LAION.Alex Atallah [00:42:16]: Yeah, the LAION Discord server.Swyx [00:42:17]: The image community that spawned Stable Diffusion.Alex Atallah [00:42:19]: The image community. Yeah. And so when Stable Diffusion came out, I realized- Oh, now other people can build their own Midjourney.Alex Atallah [00:42:27]: Because until then, Midjourney did not have an API, so they were a stack company, right? They were training their own models, and they were deploying them as an application. But if you wanted to build your own Midjourney, there was no API of that quality. and I think E two was still quite primitive. Like, Midjourney had great quality. And then when Stable Diffusion came out, suddenly there was this new person who - there was - this new capability in the world, which is a developer could create their own Midjourney. And that, I think, created the need for something like OpenRouter, because then you need an API to. If you - if you had the creativity of David Holz and you had Stable Diffusion as the model and you wanted to put these things together, how could you do that without having to figure out how to host the weights? And what OpenRouter, - the shape of OpenRouter enabled is that. Right? When you have open model alternatives to closed applications, OpenRouter's value in the world becomes extraordinary because now any developer can just show up and use theStable Diffusion and the Need for a Model API LayerSwyx [00:43:20]: You just love model diversity.Anjney Midha [00:43:21]: Did you just say the shape of OpenRouter?Alex Atallah [00:43:23]: Oh, no.Anjney Midha [00:43:25]: Were you in cloud? What is this the real Han?Alex Atallah [00:43:26]: I've been, I've been - I'm, I'm misaligned now. I've been overtrained. I've been using Cloud way too much, haven't I?Swyx [00:43:34]: Claude-ish is what people would say.Alex Atallah [00:43:35]: Claude-ish. Oh, God, I gotta untrain myself.Swyx [00:43:38]: Okay. - And I just wanna cap off the Mistral side. my TLDR is there was a Mistral price war, is what they called it, right? Like, round about NeurIPS is twenty-three or twenty-four.Mistral and the Birth of the Inference MarketplaceAnjney Midha [00:43:47]: Yes. DecemberSwyx [00:43:48]: They launched, the Mistral 8x7B, and like the price went down like 80%.Anjney Midha [00:43:54]: Yeah.Swyx [00:43:54]: To me, that's very positive because it's like the first, like, real competition to host Mistral. Is there more?Anjney Midha [00:44:01]: Yeah, that was. I'm, like, trying to remember it, all the things that happened. It. Like, we saw that model come out and immediately saw people say that it was the best model in the world.Alex Atallah [00:44:15]: Yes.Anjney Midha [00:44:15]: Like, this was, to my knowledge, the first time an open weights model was called that in real seriousness.Swyx [00:44:22]: It's hype, right? Is it?Anjney Midha [00:44:25]: It was hype. It was hype. It was also, like, hype from AI influencers at the time. And there were many examples where it was, like, outperforming four. So people really wanted to try it out and see, is this gonna be true for me too? And if so, at what price? And, the, like, inference landscape was really messy.Alex Atallah [00:44:49]: Yes.Anjney Midha [00:44:50]: We cleaned it up. - it allowed, like, providers to compete on price, so we could give you just the best price in one spot. And so it was, I think, the first clear example of, like, a provider marketplace working in a way that adds value to end developers.Alex Atallah [00:45:08]: Sean, you may not remember this, but I think we met for the first time a few days after Mistral came out at NeurIPSAnjney Midha [00:45:15]: Yeah.Alex Atallah [00:45:15]: At a luncheon.Swyx [00:45:16]: Yeah. That's where I also met BFL as well. Yeah.Alex Atallah [00:45:18]: And Guillaume was there.Swyx [00:45:19]: Yeah.Anjney Midha [00:45:19]: I was at NeurIPS at that time.Alex Atallah [00:45:20]: You were there too. And, we had just announced the Mistral investment, and I remember Guillaume was over there, and I remember turning to Guillaume and asking him, Like, “Is it is all the. Like, how are you feeling after the launch of Mistral and seven B?” And, him in his typical French fashion was like, “ it's a, it's an okay model. It's not that good.” And I was like. It was so, in contrast. But I remember him also saying that part of the reason he felt a lot of people Thought that it was better than four was because of the speed. - it was an MoE model that they had, like, absolutely figured out how to make super efficient. It was on the Pareto frontier. And this is an important thing about LLMs, right? Sometimes when they're faster, you think they're smarter, even though, like, if you did, N of, these common, like, evals that are - you do seven tries, and I don't remember. I think we should go back and figure out what the data says, but I wouldn't be surprised if it turns out, oh, on an N of seven attempts, four was smarter on evals, but the perception of on, like, or correctness would be smarter or more accurate. But, people, like, from a human preference perspective felt that it was faster because it - or smarter because it's so fast.Swyx [00:46:36]: Yeah. And most queries do not take that levelAlex Atallah [00:46:39]: Don't take that. That's true.Swyx [00:46:40]: Right? So this is the start of humans as routerAlex Atallah [00:46:42]: Yes.Swyx [00:46:42]: Which then eventually becomes OpenRouter as router of like theAlex Atallah [00:46:45]: Oh, that's interesting way to think about it. Yeah.Swyx [00:46:47]: Like, because humans are the routing mechanism. Like, I will ask the fast model first, and then if, like, oh, not good enough, I'm gonna upgrade manually.Alex Atallah [00:46:52]: Yes.Swyx [00:46:53]: But then he's gonna auto it.Alex Atallah [00:46:54]: I didn't, I hadn't thought of it that way, but that makes sense.Swyx [00:46:57]: Which then there's, there's a lot more techniques, like fusion. Fusion is the thing that we should talk about. Before I move on to those things, I just want to close off the early years. one thing that I observe, which you are also an investor in Arena.OpenRouter vs. LM ArenaAlex Atallah [00:47:10]: Right.Swyx [00:47:10]: And we talked about Midjourney having that feedback loop of, A, B, C, D, and choosing that very. being very important. And you understand the flywheel. So how come you didn't build Arena, and how come Arena didn't build OpenRouter?Anjney Midha [00:47:23]: Well, Arena started before OpenRouter, right?Swyx [00:47:27]: They had the school projectAnjney Midha [00:47:29]: Yeah, LMSwyx [00:47:29]: And then it became a company.Anjney Midha [00:47:31]: LM Arena, yeah.Swyx [00:47:32]: So, but, and I know you had some Arena experiences, like the up comparison type things.Anjney Midha [00:47:37]: Yeah.Swyx [00:47:37]: But you never really went as hard as Arena did.Swyx [00:47:40]: And,Anjney Midha [00:47:40]: In doing up experiences?Swyx [00:47:42]: Yes. And LM Arena did have a router project based on LM Arena ELOs, which they never commercialized.Anjney Midha [00:47:48]: It's hard to do a company that does both because one company is taking data and selling it, and the other company really can't by default. So, I think there is, like, a branding reason that there are two companies here. like, when you set up OpenRouter, there's no training, there are no prompts, right, aside from what your provider policy set. Like, OpenRou- like, OpenRouter can't see your prompts or completions. If you want to see that as an org, you have to opt into it and enable it. And so we're, like, pretty conservative and careful about data policy and security. And privacy. And LM Arena is like, their business model is like oriented around the labs and,Swyx [00:48:34]: Because they give it for free, right? You don't give it for free to give it for free.Anjney Midha [00:48:37]: Yeah.Anjney Midha [00:48:38]: But we do give some. We like have free endpoints too, but like those free endpoints, we, I think we're not collecting any prompts. We're not like monetizing the data unless you, opt into it for some reason.Alex Atallah [00:48:48]: This comparison. you're not the first person to ask me this, and Alex knows this, but I was the interim, like the founder, like first CEO of Arena for the first five months when, and we were helping Anastasios and Waylin spin out of Berkeley. And, I did invest in that before, OpenRouter, but it was very strange to me the comparisons that outside, folks would make between the two projects because the missions were completely different. The founding entity for Arena, we called it the AI Reliability Institute because it was there as an eval service. Like the data, so to speak, that they were originally, offering the labs was how do you make the evaluation of models more reliable than like the state of the art at the time, which was like really just finger in the wind.Alex Atallah [00:49:38]: That's what Anastasios and Waylin's PhD work was as scientists at Berkeley, was on statistical methodologies for correcting, eval estimates, based on like intrinsic biases and how you collected the data.Swyx [00:49:54]: Yes.Alex Atallah [00:49:54]: AndSwyx [00:49:54]: Style control.Alex Atallah [00:49:55]: Style control and stuff like that. And which is very much like a, hey, how. If you're a scientist and you're trying to. the highest expectation customer for Arena was always like a training and, like a researcher at a lab. Whereas the highest expectation customer from my perspective that Alex like really understood and was the mission was to serve was like a developer, right? Who then takes the result of the research and then produces an application that's deployed to the world. It was a completely different problem and person that these two teams were focused on. And so from the outside in. I don't know if you remember this, but I have a distinct memory of a few weeks before we did the term sheet, together for OpenRouter, I'd given you a call because we were trying to get a pooled data set together from OpenRouter and from Arena to, create like an open source repository of prompts. these projects were so different in their goals that it was totally normal to me to be like, “Oh, yeah, let's call Alex and see if he'd want to team up on pooling data,” because they're so different. We need. We don't have that data at all. We. Like, we didn't have API prompts. We didn't, we didn't have like what developers want to do with the models, which is very different from what researchers inside a model lab want to do before releasing the model.Swyx [00:51:15]: Yeah.Alex Atallah [00:51:15]: Does that make sense? And so to this day, I think you see that this difference, even though at a 30,000-foot level you could. I guess you could conclude that Arena and OpenRouter are adjacent, but, the roadmaps, the missions and so on at the time at least were like in very different directions.Swyx [00:51:36]: That ideal customer, I get. I totally get that.Alex Atallah [00:51:39]: Yes.Swyx [00:51:39]: As a founder, I want to own everything, right?Alex Atallah [00:51:41]: That's possible.Swyx [00:51:42]: Like this is clearly an adjacency that I'm like gonna explore that.Anjney Midha [00:51:45]: Own everything meaning like you don't know what to do yet, so you wanna like make sure you catch PMFocus, Anthropic, and Roads Not TakenAlex Atallah [00:51:51]: No, I think what heAnjney Midha [00:51:52]: As quickly as possible.Alex Atallah [00:51:53]: You want to own the entire infrastructure space, and so you expand to whatever demand you can capture.Swyx [00:51:58]: You want to have a play in each end.Alex Atallah [00:51:59]: Yeah, I think that's, that's hard, in reality, because serving multiple customers is difficult.Swyx [00:52:05]: Clearly, this is the one focus, right?Alex Atallah [00:52:08]: Yeah.Anjney Midha [00:52:08]: Yeah. I still think even in the age of AI, like focus is,Alex Atallah [00:52:12]: Is criticalAnjney Midha [00:52:13]: Underrated and critical, not just because you end up with a better product by focusing your humans on it, but also because the world knows what your focus is.Alex Atallah [00:52:22]: One thousand percent.Anjney Midha [00:52:23]: The world can map like, “Oh, I have this issue. Which brand out there is going to help me with that issue? This is the brand that's known for that focus.”Alex Atallah [00:52:31]: Yes.Anjney Midha [00:52:32]: So like if I want real attention on this issue, like this really matters to me, I should go with the brand that cares the most about it.Alex Atallah [00:52:39]: To underscore Alex's point about how important focus is, in the early days of Anthropic, it was not easy to. Like people think that the early days of Anthropic were like super easy because they were on their 3 guys who left, but it was very

The fabulous Samm Deighan is back with us this week to delve into a couple of great, underseen French films - and also to tell us all about her new book on the work of that great master of cinema insanity, Andrzej Żuławski. Exciting! We kick off our double feature with Żuławski's frantic ode to mad French mobsters, L'Amour Braque (1985) and we follow that up with La Cérémonie (1995), directed by Claude Chabrol - a domestic thriller about class, shame and sociopathy that comes on like a television procedural but hits you like a gut punch when you least expect it. Oof!!! L'Amour Braque is so dense and confusing that we'd probably recommend listening to this before you watch it - and spoilers are made up for by being properly prepped for its unique vibe - but Spoiler Territory was ESSENTIAL for La Cérémonie. Skip ahead to 1:58:34 to avoid spoilers for that one. Join our Patreon! As a member of our Mario Adorf Allstars tier you get access to hours of extra content including two audiocommentaries every month and other bonus episodes! Find it here. Buy Samm's new book here: Hysterical Excess and Inhuman Ecstasy: The Cinema of Andrzej Żuławski Want to get in touch? You can reach us on caliber9fromouterspace@gmail.com Theme music: "The Cold Light of Day" by HKM. Check out HKM on #SoundCloud or Bandcamp "Idiot Heart" by Sunset Rubdown

What happens when the life and business you've built is GOOD… but there's still a little voice inside you asking:What if there's more?In this episode of Keeping It Real, I'm sitting down with my longtime client, Business Circle member and friend Amanda Dawe, owner of The Natural Emporium.And friends, this conversation is about SO much more than business.After eight years of building The Natural Emporium and an incredible community in Newfoundland, Amanda made a massive decision: close the chapter she knew and loved, pack up her life, and take her business to Toronto's Distillery District.A dream she had once written down in a journal and quietly tucked away as a “maybe someday.”But getting the dream didn't mean everything suddenly became easy.We talk about the part of growth that doesn't get talked about enough.The grief.The fear.The moments you wonder if you've made a gigantic mistake.The temptation to stay where things are comfortable because you should be grateful for what you already have.And the courage it takes to trust yourself when you have absolutely no evidence that the thing you're stepping into is going to work.Amanda shares what it looked like to go from being so exhausted she was considering walking away from entrepreneurship altogether… to reconnecting with what she actually wanted and making one of the biggest moves of her life.We also get into:✨ The difference between running AWAY from something and moving TOWARD something✨ Why you can be wildly grateful for what you have and still want more✨ The very real grief that can come with growth✨ Making big decisions from your power instead of exhaustion or desperation✨ The small daily rituals Amanda uses to ground herself and regulate her nervous system✨ Why comparison can pull you away from the very thing that makes your business special✨ Learning to trust your intuition when there's no proof your next move will work✨ Being willing to not be for everyone✨ Why celebrating the tiny moments matters just as much as celebrating the giant milestones✨ And what Amanda has learned about going after a dream at 50One of my favourite moments in this conversation is when Amanda says:“Not doing it is a bigger cost than doing it.”OOF.Because sometimes the question isn't:“What if I do this and it doesn't work?”Sometimes the better question is:“What will it cost me if I never find out?”This is a conversation about business, yes.But it's also about trusting yourself enough to follow the thing that keeps tugging at you.About realizing you don't need permission.About understanding that fear and grief don't automatically mean you're on the wrong path.And about becoming the version of you who is willing to find out what's possible.If you're standing at the edge of a decision, a pivot, a dream or a next chapter that feels exciting AND terrifying…this episode is for you.CONNECT WITH AMANDA:Follow Amanda and The Natural Emporium on Instagram: @EmporiumontheSquareYou can also find The Natural Emporium on Facebook and visit Amanda at her new location in Toronto's Distillery District.And make sure you get yourself on Amanda's newsletter to follow along with this next chapter and everything she has coming.

AIUC first got our attention with the NFDG backing, and have just announced a $40M series A today, with the most impressive industry advisor list we may have ever seen for an early startup behind AIUC-1, their agent standard backed by real insurance:From being Anthropic's first product hire to building the standards, testing, and insurance infrastructure meant to make frontier AI deployable, Rune Kvist is betting that the biggest constraint on AI adoption won't be capability it will be trust. In this episode, the AIUC cofounder joins swyx and Vibhu to announce a new $40M round and explain why companies like Cursor, Harvey, Lovable, and ElevenLabs are increasingly confronting a problem that gets harder as AI gets better: who is responsible when autonomous systems fail?We go deep on AIUC-1, the emerging standard for agent security, safety, and reliability; how AI agents are stress-tested for jailbreaks, hallucinations, and data leaks; and why Rune thinks standards and insurance could become critical infrastructure for AI. We also discuss the growing trust gap between governments and frontier labs, AI-enabled cyber and biological risks, why every model can ultimately be jailbroken, what happens when a $20 coding agent causes $200M of damage, whether AI engineers should be certified, and why even after AGI there may be one job the labs can never do themselves: be their own watchdog.We discuss:* Why risk, liability, and trust may become the binding constraint on AI adoption* Rune's path from reading the Scaling Laws paper to joining Anthropic in its earliest days* What Anthropic understood about scaling, compute, and the future years before it became obvious* Why Waymo illustrates the gap between AI capability and real-world deployment* AIUC's $40M round and work with Cursor, Harvey, Lovable, ElevenLabs, and other frontier AI companies* AIUC-1: a standard for AI agent security, safety, and reliability* How agents are tested for jailbreaks, hallucinations, and data leakage* Why most AI companies optimize the happy path without seriously stress-testing adversarial cases* Why AI standards may need to update every quarter instead of every decade* The emerging trust gap between frontier AI labs and governments* Cybersecurity, child safety, biological weapons, and the expanding frontier-model risk surface* Why standards and insurance may need to evolve together* How Lloyd's of London can insure AI systems and bring trust to enterprise deployment* What happens if a $20 Cursor subscription contributes to a $200M plane crash* The Air Canada chatbot case and how AI failures are beginning to clarify legal liability* Why copyright may be one of the hardest AI risks to insure* Evals, mechanistic interpretability, monitoring, and models becoming aware they're being tested* The impossible CISO mandate: adopt AI fast, but don't let anything go wrong* Why robotics will make AI liability dramatically more consequential* Whether AI engineers should have Level 1, 2, and 3 certifications* AIUC's roadmap across agents, frontier models, robotics, and universal red teaming* Why AGI could become a question of national sovereignty* Why the labs can never fully serve as their own watchdogs* The Big Short problem: how do you stop competing watchdogs from racing standards to the bottom?Rune Kvist* LinkedIn: https://www.linkedin.com/in/runekvist/* X: https://x.com/RuneKvistAIUC* https://aiuc.comTimestamps00:00:00 AIUC's $40M Round and the Risk Bottleneck for AI00:01:07 From Scaling Laws to Early Anthropic00:07:58 Why Trust, Not Capability, Could Limit AI Adoption00:12:19 Founding AIUC and Building AIUC-100:18:52 How AI Agents Are Audited and Stress-Tested00:25:26 Frontier Models, Government, and the AI Trust Gap00:33:32 Cyber, Child Safety, and AI-Enabled Biological Risk00:38:14 Why Standards and Insurance Belong Together00:41:45 What Does an AI Insurance Policy Actually Cover?00:50:44 The $20 Cursor Subscription and the $200M Plane Crash00:53:53 AI Liability, Monitoring, and Earning Enterprise Trust00:56:21 From AI Agents to Models to Robotics00:58:29 Copyright, Adverse Selection, and AI Insurance01:03:28 Evals, Mechanistic Interpretability, and Eval Awareness01:08:36 The Impossible Enterprise AI Mandate01:11:52 Prediction Markets vs. AI Audits01:14:43 Should AI Engineers Be Certified?01:19:10 AIUC's Roadmap, AGI, and Who Watches the Watchdogs?TranscriptIntroduction: AIUC, the $40M Series A, and Risk as the Adoption BottleneckSwyx [00:00:00]: Okay, we're in the studio with Rune from AIUC, the Artificial Intelligence Underwriting Company, with our trusty co-host, Vibhu. Welcome.Rune Kvist [00:00:10]: Thank you. Thanks for having me. Thank you.Swyx [00:00:11]: What are you announcing today?Rune Kvist [00:00:12]: We have raised $40 million, led by Ribbit Capital and First Harmonic.Swyx [00:00:17]: You first came to my attention when Nat and Daniel invested in you guys. Is the story, like, pretty much the same? Like, what are you today versus what you thought you were back then?Rune Kvist [00:00:26]: When we raised our seed round, we had a hypothesis that at some point risk was going to hold down adoption. At that point in time, that felt kind of hypothetical, and I think that is now over. Clearly, the moment is now with Mythos and Fable. It's pretty obvious that literally the binding constraint on adoption is risk. And so for us, it feels like this is a natural continuation of the same hypothesis, but where previously it was speculation, now it feels like fact.Swyx [00:00:54]: And let's get a list of the customers that you're highlighting as part of your Series A.Rune Kvist [00:00:58]: Totally. Yeah. So we are now working with folks like Cursor, Harvey, Lovable, ElevenLabs.Swyx [00:01:05]: Yeah. Amazing. Congrats.Rune Kvist [00:01:06]: Thank you.Swyx [00:01:07]: So you were famously one of the first hires involved in GTM and product. I'm just kind of curious: what was your path into AI? Just recap.Rune's Path Into AI: Scaling Laws, Capital, and AnthropicRune Kvist [00:01:18]: Yeah.Rune Kvist [00:01:19]: Late 2021, I sold a company, my first company, an edtech company. I had a bit of time to think about what was next. I came across the Scaling Laws paper, and that just struck me like lightning. I was just like, “This is a big idea.” In short, the Scaling Laws paper just says the bigger the model, the smarter the model.Swyx [00:01:38]: So this is the Kaplan one, not the Chinchilla one?Rune Kvist [00:01:40]: Exactly, the Kaplan one.Swyx [00:01:42]: Yeah.Rune Kvist [00:01:42]: And the important thing that clicked for me there was, oh, now capital will understand this. If you put in more money, you get more money out, and so that will kick off a hype cycle. And so you get a sense of predictable returns, which is, in fact, what's played out. And so I just packed my bags. I'd never been to San Francisco. I'd never been there. I just packed my bags, flew out here to find the people who had written it. And at the time, they had just started a small lab called Anthropic. There were around 40 people at the time or so. Drank a bunch of coffee until I eventually got introduced to Dario. And at the time, they were wrestling with some of these questions of, like, should we deploy our models? Should we make revenue? How should we engage with the rest of the world? They'd just broken off from OpenAI, and it's been publicly reported that they were kind of concerned with how they were dealing with deployment. So they were wrestling with some of those questions. At this point, this is early fog of war, like early 2022. The hottest product at the time was, like, Jasper. Like, there's nothing out there. So where value was going to accrue, and what the different parts of the stack were going to be, were all open questions.Swyx [00:02:48]: I want to highlight to people, you ask these questions because you have a PPE background.Rune Kvist [00:02:52]: Yes.Swyx [00:02:52]: I actually was in Singapore in one of the sort of feeder programs for prepping people for PPE. So I had a tutor. We learned, you know, philosophy and politics and economics. But, like, I think your kind of background matters. Machine learning people who read the neural, Scaling Laws paper would not necessarily draw the same conclusions that you did. Whereas any capitalist would read that and go, “Holy s**t.”Rune Kvist [00:03:19]: Correct.Swyx [00:03:20]: Right?Rune Kvist [00:03:21]: Yes.Swyx [00:03:21]: Who tipped you onto that paper? Because it's not a paper that you normally read, right, like, in your circles?Rune Kvist [00:03:26]: Yeah. I think I'd actually, ever since AlphaGo, had some appreciation that AI was a big deal.Swyx [00:03:36]: Yeah.Rune Kvist [00:03:36]: But it kind of felt like it raised all these kind of interesting philosophical questions, but it was kind of not clear from afar where exactly that would go. But it was obvious enough that it was like, this is going to be a big thing if we find the kind of right mechanism to kind of get the techno-capital machine to work on this. But it was just not clear. And so I think there was some way in which, like, that became obvious, and also it wasn't as obvious at the time than it is now, right? Like, it was just like, wow, this is so interesting. But it still felt, coming from kind of a philosophy and economics background, it felt like if this turns out to be true, you're going to be wrestling with all of the big questions in society. Everything you've learned about politics gets thrown out of the window. Everything you've learned about economics at least gets challenged. And so what felt interesting was to be at that frontier that has ramifications across everything. So that's why I sought it out.Swyx [00:04:32]: I mean, clearly really good insight. For people who don't know, the PPE program is, like, where prime ministers are born. So then you end up meeting Dario.Rune Kvist [00:04:41]: Yep. First Dario, yeah.Swyx [00:04:43]: Yeah. Well, I mean, like, so did you get extra insights from talking with them that you didn't get from your original hypothesis?Anthropic's Early Conviction and the Scaling Laws Crystal BallRune Kvist [00:04:50]: If you read the Scaling Laws paper, you get this, like, very vague sketch of like, wow, this seems kind of important. There are some lines on a chart. This seems kind of important. And what I think the team at Anthropic had thought more about than anyone was like, what are the implications of this if you really play this out? And back then they had, kind of vision documents for what the world would look like in 2026, and they were kind of in vivid detail playing out how much compute is going to be needed, what the CapEx was going to look like, what some of the societal concerns were going to be, but also what is the amount of economic value coming out here? And so it kind of felt like they held a crystal ball that in hindsight turned out to just be dramatically correct. And they weren't holding it like they were obviously correct. They were just like, “Take this hypothesis really seriously.”Swyx [00:05:38]: Think it through, yeah.Rune Kvist [00:05:38]: And think it through in the same way as the kind of situational awareness that isSwyx [00:05:43]: Across the street.Rune Kvist [00:05:44]: Across the street.Swyx [00:05:44]: Your office, yeah. Oh my God, we're all living across the street in the same one square mile.Rune Kvist [00:05:50]: Correct. And that's now a couple of years old, but also people keep referencing it these particular weeks with Fable and Mythos, and it's like, wow, if you take this one idea seriously- For the Scaling Laws, a lot of things fall into place.Vibhu [00:06:03]: And keep in mind, at this point, this is the same team that did GPT-1, GPT-2, and GPT-3.Rune Kvist [00:06:08]: Correct.Vibhu [00:06:08]: Which is also, like, it's not just some experimentation. Like, this is a real model that we just scaled up.Rune Kvist [00:06:14]: And they had deep conviction in this idea: if you take a big blob of compute and data, it just wants to learn, and out of that will come smarter and smarter models. And all the particulars were not clear.Vibhu [00:06:26]: Yeah.Rune Kvist [00:06:27]: And all the implications were not clear. But their deep conviction in this, like, core thesis, and that was kind of dizzying. It was both phenomenally interesting and exciting, and also very quickly you get to, like, the world we know today will no longer be if this hypothesis holds. So it also just felt, like, important in some kind of grand sense.Vibhu [00:06:48]: What kind of shaped you there? So that was early 2022. Not only had GPT-1, GPT-2, and GPT-3 come out, but, you know, the amazing founders of Anthropic that have never split up, the only ones, they actually had the conviction to leave OpenAI, start their lab. You said there were about 40 people there. What was the time like there?Inside Early Anthropic: Mission, Deployment, and RiskRune Kvist [00:07:06]: It was kind of remarkably like what it looks like on the outside today. Extremely cohesive, extremely mission-oriented, and living in this tension between their two ideas, which is AI could both go really well and really bad, and we want to be part of building it. That creates astounding amounts of tension. And they were wrestling with this incentive challenge where they know they're in a race that they're in where you might get forced to cut corners, but it also felt very important to them to be at the forefront of technology. And all of those ideas were just present at that time. It kind of feels like that line has been just very clear, and I think kind of love them or hate them, they have really stuck to their guns. There's a core set of beliefs that they hold more deeply than most companies hold any beliefs.Vibhu [00:07:58]: Yeah. Fast-forward to today.Rune Kvist [00:08:00]: Yeah.Vibhu [00:08:00]: What does that lead us to AI underwriting company? What are you up to? What motivated you to start this?From Waymo to AIUC: Confidence Infrastructure for AIRune Kvist [00:08:05]: Yeah. AIUC builds confidence infrastructure for frontier AI through standards and insurance. The link from Anthropic to building confidence infrastructure, looking out the windows at Anthropic offices and seeing Waymos driving by. Already back then, early 2022, Waymos were in some ways like AGI for cars. Like, they were superhuman drivers, but you couldn't take one to the airport. And now, four and a bit years later, you still can't take your Waymo to the airport, despite now everyone having kind of looked at the evidence and being like, “They're better drivers than humans.” So in that particular instance, what's clear is that the binding constraint on AI being useful is not capability, but is that liability or risk or trust. That problem is, general. The reason why right nowRune Kvist [00:08:52]: Fable is not open for access is not because it's not a good model, it's because it's a very good model. It's just hard to make promises about what it will or will not do. And this problem gets worse as AI gets better. Basically, more intelligent AI can be more autonomous. That's more valuable, but also the risk surface grows. And so - what Waymo illustrates is that unless you build the confidence infrastructure to make promises about AI, or at least bring light to the risks, you grind adoption to a halt. Governments, banks, hospitals, militaries need to have some sense of what AI will and will not do to be able to operate for them to incorporate it. And that's the problem that we're trying to solve. Now, why standards and insurance? If you trace this problem back through history, every technology wave has had some version of this problem. So if you go back to, like, year 1900, electricity comesVibhu [00:09:47]: Ben Franklin.Rune Kvist [00:09:48]: Cars burn down, sorry, houses burn down, lots of people die. 1930s, cars are a big deal, kill lots of people. 1950s, private nuclear energy is a big deal, poses big risks. In each of those instances, the market runs ahead of regulation to create confidence infrastructure because that's required to make go/go decisions. That is required for adoption, and the market fundamentally wants adoption. And in all of those instances, common blueprint emerges between standards and insurance. The reason these two components is standards kind of provide the rules of the road, and they also specify, like, what are the tests that need to be run so we can get a sense of how high the risk is. So take in the case of cars, that's like a car crash. Great, everyone, they inform your insurance pricing today, they inform your purchasing decisions, et cetera. That's basically the risk framework. The insurers are important because they pick up the bill. So they are the private institution that is most on the side of. That is best incentivized to quantify the risks truthfully and then figure out all the ways to reduce the risk ‘cause that increases their profit. So they're basically, they help shape the incentives. And these two work really well in unison. Now, how does that show up as a company? Well, one of the things that was obvious even - or starting to become obvious even a couple years ago was that frontier companies, some of our customers today, like Cursor, Sierra, ElevenLabs, Harvey, were going to have a very easy time selling a pilot to a bank. The, like, the demo just sells itself. It's magic. But bringing that through, if you want to do a wall-to-wall rollout at a bank or a hospital, you have to go through the risk process. These banks have no idea even which questions to ask, let alone which answers are sufficient, let alone, like, how do they go and test whether these agents actually work the way they're supposed to. And so they had this problem of, like, what can we say to earn the trust? And we think there's, like, a golden sentence that goes something like, “Hey, I hear you're really worried about hallucinations or jailbreaks or whatever it may be. We've had an independent third party test us against the gold standard. We passed with flying colors. And as a vote of confidence, the world's most conservative insurers have looked at the data.” And they're willing to take some of the risk onto their balance sheet.Swyx [00:12:06]: Yeah.Rune Kvist [00:12:07]: So if something does go wrongSwyx [00:12:07]: There's money behind it, yeah.Rune Kvist [00:12:09]: Exactly. So that's kind of like the link between all this. We can get into some of the hard parts related to the technical testing, which is, I think, the crux of the matter, but I'll pause there.Swyx [00:12:19]: How did you and Rajiv come together? This-- there's always, like, you come across very confident and, you know, and we're announcing your Series A and all these things, but I want to see, like, the early initial stages of, like, idea formation.Cofounding AIUC with Rajiv DattaniRune Kvist [00:12:31]: Yeah. Rajiv is actually my soon-to-be brother-in-law.Swyx [00:12:35]: Oh.Rune Kvist [00:12:36]: So I'm actually, in a week and a half getting married to Rajiv's sister.Swyx [00:12:42]: Okay, now you're tight.Rune Kvist [00:12:44]: Exactly.Swyx [00:12:44]: Now you know.Rune Kvist [00:12:45]: So - Rajiv and I have known each other for a decade. Funny story, I met both Rajiv and his sister, Hena, at the same time when Hena and I were interns at McKinsey in London, and Rajiv was assigned as my mentor. And so met them at the same time. For the longest time, it was not obvious that we were necessarily going to work together. I was in startups. He was, an insurance partner at McKinsey. Three or four years ago, I think Hena convinced him that AI was going to be a really big thing. And so he quit his job, cushy partner job at McKinsey in London, packed his bags, flew to San Francisco, and ended up joining METR. You guys are probably online enoughSwyx [00:13:24]: CEO.Rune Kvist [00:13:24]: Exactly.Swyx [00:13:24]: We've, we've, we've heard of METR.Rune Kvist [00:13:25]: You see the plot-- the chart of the horizons of the tasks that agents can take on is doubling extremely fast. So he was COO at METR, led their partnerships with Anthropic and OpenAI to test their models before release, but also working closely with the US and UK government, to figure out, like, how do you know whether a model can be released? And in some ways, that was, like, the perfect background. He's spent a lot of time in insurance, knows that world, spent a lot of time with frontier testing of models. And so when I was bumbling around this idea space, starting with some of the ideas we talked about related to Waymo, as soon as we got into the content, we were both like, “Oh, this would be an amazing business to build together.” This is wrestling with the problem that we both think is the most important in the world from a market angle, which is kind of our intuitions is that the market can do a lot, and the faster AI moves, the harder it is for government to solve some of these problems. And then it took a little bit of time to work through what is it like to work with family.Swyx [00:14:27]: Sure.Rune Kvist [00:14:27]: And,Swyx [00:14:30]: Because you were already dating at the timeRune Kvist [00:14:31]: Yeah. Yeah, exactly.Swyx [00:14:33]: Yeah.Rune Kvist [00:14:34]: Already back then, itSwyx [00:14:35]: Yeah.Rune Kvist [00:14:35]: We felt like we were a family.Swyx [00:14:36]: Nice.Rune Kvist [00:14:36]: And so starting a business together felt like kind of a big step. And, here we are with just immense amounts of trust.Vibhu [00:14:43]: Yeah. So now you're a company of how big? How big are you guys now?AIUC-1 Certification: Agent Security, Safety, and ReliabilityRune Kvist [00:14:46]: There are just 20 of us now.Vibhu [00:14:47]: 20 of you guys now, have Series A, and you have your first certification out, the AIUC-1. Let's bring up the certification. So this is the agent certification, right? What goes into the process? I have, like, two questions here. One is, walk us through the certification, and two is, what is the process for a company to get certified, you know?Rune Kvist [00:15:08]: Great. As it says right on the top, AIUC-1 is a standard for agent security, safety, and reliability. The fundamental design principle is take all of the concerns that slow down adoption, so all the questions, all the fears that keep, security leaders in the Fortune 1000 up at night, and put them into one comprehensive framework. That's what you'll see there. You can see the six categories. Two, you want to ground all of this in technical testing. So one of the concerns with security standards that often feel kind of like theater paperwork is that they're not actually ground out in, does any of this work? Does any of this matter? And so we had a conviction from early on that was going to be the kind of crux, was to pass this, you must get tested every quarter, basically run thousands of simulations to see, well, so can it actually be jailbroken? How hard is it to jailbreak? How often does it hallucinate? How often does it leak data? Et cetera. And then the last, core idea here, if you scroll up to the top here, is to refresh it quarterly.Rune Kvist [00:16:08]: So the core trait of AI is that it moves extremely fast. Whatever concerns we're discussing today were not the same ones three months ago, and this will keep changing. Typically, standards update on a, like, a decade cycle is obviously not going to work. But the question is kind of how do you update it? And the core thing here was to basically get the risk leaders of the Fortune 1000 around the table. So if you go over to the left hereVibhu [00:16:32]: YeahRune Kvist [00:16:32]: You'll see the AIUC-1 consortium. The consortium is a group of risk leaders who run real banks, real hospitals, real critical infrastructure, who are facing these challenges every day. And we meet with these folks twice a quarter and hear what's top of mind, what is keeping them up at night. There's tremendous amount of desire for that conversation. And then we operationalize that into a specific standard that gets into. And actually, we can go into and look at whatVibhu [00:16:55]: YeahRune Kvist [00:16:55]: What even is the standard. So if we go back to introduction, out there to the left, scroll up a little bit to the wheel, click into reliability. So if you take something like hallucinations sits in reliability. There is a number of requirements here. If you go into the top one, prevent hallucinated outputs, hallucinate outputs, this is one particular requirement. This is a technical control. Basically, we want some kind of ground in this filter. The first thing you see here is what's called a crosswalk. So everyone and their grandmother has put out a framework, very high-level framework for what are the AI risks.Swyx [00:17:27]: This is basically your competition,Rune Kvist [00:17:28]: In some ways our competitionSwyx [00:17:29]: Not seriously, yeah.Rune Kvist [00:17:30]: We're, in fact, friends with them. We'll come back to why.Swyx [00:17:31]: Yeah.Rune Kvist [00:17:32]: But mapping everything together so you have one superset. The claim you're trying to support here is, if you follow this framework, then you can also see how you follow the other frameworks. But the meat of it comes down here in control activities and evidence. So control activities is like, great, you have this high-level requirement. How do you turn that down to something operational? Here's what you must do, and then what is the evidence that we're looking for?Rune Kvist [00:17:57]: And the reason we go this deep is that there's actually not that much confusion about what are the big concerns in AI. Everyone agrees to these. The question, like, what are you actually supposed to do? And so. What we found a lot of demand for is getting down to the specific evidence, that people need to look for. Whether you are Cursor building something or, even JPMorgan building something, but also if you're just a risk leader at JPMorgan, like what exactly should you ask for? What can you ask for without sounding stupid? Like if you ask for some-- you won't believe the amount of time a risk leader has asked for the IP rights to the underlying model to Cursor or something, and you're just like “Sorry, what?” Like,Swyx [00:18:39]: You slip it in there and you seeRune Kvist [00:18:40]: SlipSwyx [00:18:40]: See if you notice.Rune Kvist [00:18:41]: See if they. Exactly.Swyx [00:18:42]: Yeah.Rune Kvist [00:18:42]: Put that in the questionnaire. All right, so that's kind of what our standard is, and we update this every quarter with these folks, to keep up with the latest concerns.Swyx [00:18:51]: Can I double-click on this one?Controls, Evidence, and Third-Party TestingRune Kvist [00:18:52]: Yeah.Swyx [00:18:52]: So first of all, the website's beautiful. Like, it's so confidence-inducing which is the whole point where, like, okay, I know exactly what I'm signing up for when I talk with you. Like, I don't even have to talk to you. I can just see your whole, certification, which is great. But, like, okay, so from here, like D001.1 configure a groundedness filter, how does that get applied? Like, you have a person thatRune Kvist [00:19:16]: Yeah,Swyx [00:19:16]: Goes through it?Rune Kvist [00:19:17]: If you, go backVibhu [00:19:19]: I did see somewhere there's like, you know, fifty-one requirements, a hundred thirty controls. There's like a wholeSwyx [00:19:25]: Right. I just want to. Like, to me, this doesn't translateVibhu [00:19:27]: Yeah.Swyx [00:19:27]: Into a test or an eval.Rune Kvist [00:19:28]: Yes. So if you go into, on the left-hand side. So actually, if - before we go in there are three types of requirements. The first is technical controls, like you must implement some guardrails.Rune Kvist [00:19:42]: Two, there are test controls. So you must have an independent third party go and run some tests against you. I'll show you one of those in a second. And then three, there are policy controls. For example, you must have a person whose name is on the line when you guys f**k up, and you must have a plan for how you tell your customers and how you engage with them. They're kind of more traditional, standard type stuff. So in this particular instance, we just check whether they in fact have a ground in filter. So we will partner with an auditor. So we partner with auditors like KPMG or like Schellman who go in and do the thing auditors do, which is to check the evidence. In this case, that might be a screenshot, it might be part of the code that they need to review to see that it actually. Just that it exists.Swyx [00:20:21]: Oh, okay.Rune Kvist [00:20:22]: And then the second thingSwyx [00:20:22]: So you're not testing the effectiveness of it.Rune Kvist [00:20:24]: That's the second thing. So if you go downSwyx [00:20:25]: Yeah.Rune Kvist [00:20:25]: To the third-party testing for hallucinations out on the left, that's basically the next requirement. This is where we test how well does it actually work.Swyx [00:20:32]: Okay, and is it you testing or the auditor?Rune Kvist [00:20:34]: We test them.Rune Kvist [00:20:35]: We test them.Swyx [00:20:36]: That's a lot of work.Vibhu [00:20:37]: How long does testing take? So if I want to get certified, justCertification Timelines, Remediation, and Quarterly UpdatesRune Kvist [00:20:40]: Yeah.Vibhu [00:20:40]: How long does the end roughly take?Rune Kvist [00:20:42]: Yeah, the end, almost always is dependent on, like, our customers needVibhu [00:20:47]: Yeah.Rune Kvist [00:20:47]: To look something for us. It takes somewhere between, like, 3 to 10 weeksSwyx [00:20:52]: Yeah.Rune Kvist [00:20:52]: Depending on how up to snuff they already are. So some people show up to us with, like, extremely rigorous security programs. When we test them, it works extremely well. We can get that done very quick. Some people come to us, and they're not that far along. We give them kind of the spec that they need to build towards, and then their security teams and engineers get to work and build to meet the standard. The testing itself typically takes a couple of weeks, including the time for them to remediate. Often, we'll find something that we cannot pass, where this is actually just not up to the standard. - you won't pass the standard. And then they will need to go and implement additional safeguards or additional remediation that makes them more robust so that they can actually kind of hand on heart look at their customers in the eyes and say, like, “Hey, we've done truly our very best.”Vibhu [00:21:35]: And they're certified for a year and have quarterly updates?Rune Kvist [00:21:38]: Correct, yeah.Vibhu [00:21:39]: And, yeah, it's pretty interesting. I think, you know, what's changed since. So this is certifying agents in production, right? Your customers, like you've had Lovable, ElevenLabs, Intercom, and they've all gone through this certification.Rune Kvist [00:21:50]: Yes.Vibhu [00:21:51]: What has changed? So I see you post, like, you know, Q2 added MCP agent,How Agent Risks Are Changing: Coding, MCP, and Agent-to-Agent InteractionsRune Kvist [00:21:56]: Yeah.Vibhu [00:21:56]: agent communication. Any other things that you want to kind of highlight since the first iteration? What comes in quarterly?Rune Kvist [00:22:03]: Yeah. So some of the changes have just been agents are not just one thing. So, like, if you take agents like Cursor and compare them to Sierra, they're really quite different. And compare them to Harvey again, compare them to you out of againSwyx [00:22:16]: ElevenLabs, yeah.Rune Kvist [00:22:17]: ElevenLabs, they're all quite different. And so we wanted to design a standard that works for all of the types of agents. And we started with one that was, like, pretty text-based, like, honestly, pretty customer support-focused. That's where there's a lot of existing demand. And then over time, we've picked, some of the frontier companies in each of these other domains that we could work with and build out the standard, so, such that we know that the same standard works for code, it works for customer support, works for automation, et cetera. So that's been one big thing. Yeah, then some of the things that have been top of mind recently, Mythos is bringing up a lot of concerns for security leaders. We're starting to get more and more questions around agent interactions. It's very nascent, at the moment, but it's starting to emerge. There've been a lot of, questions related to OpenClaw and MCP. Again, like agents starting to interact with each other, is really top of mind. Then as coding agents have really taken off, that's also where banks and hospitals, et cetera, are getting more and more precise on what it is they need. So really dialing in as that start to be, like, where most of the tokens flow through in the world, getting much sharper on that.Vibhu [00:23:26]: Can you share for people that are listening that don't really think about this? Like you mentioned, there's the obvious stuff, you know, hallucination, citations. What are best practices that people should do when building agents? Like, if they come to you pretty ready with certification like, you know, they'll probably pass certification. What are the things people don't think about that they should have?Best Practices for Agent Builders: Stress Tests and GuardrailsRune Kvist [00:23:46]: The most important thing is that a lot of companies have not done a serious stress test. They spend most of the time, perhaps rightly so, optimizing for how does it work in the good case, the average case, how high-quality is the output for the customer. And a lot of these companies are pretty new, so they haven't spent a lot of time stress testing the what is there as an adversary on the other side? What are some of the complicated corner cases that you've not really considered? So I think that's, like, a frame of mind. And you'll also see this in startups. It often takes a while until they hire their first security person. They- And that's a whole different kind of risk surface than just building a good product. So a lot of that applies. Most companies actually also have the right kind of architecture. Most of them will have some kind of guardrails in place, either some that come out of the box from their model provider or they'll have built their own filters that sit in between. They just don't work very well. The difference between putting a classifier in place that, like, maybe goes and checks whether you're giving medical advice when you shouldn't and says, “Hey, if this looks like medical advice, filter it out.” Lots of companies have that in place. The question is whether it works. And it's actually pretty fiddly to sit down and think about all the ways in which you could ask for medical advice, read the academic literature on what are the kinds ofRune Kvist [00:25:03]: Framings or tricks you might play to get an AI to give you medical advice when you really shouldn't. And so there's, like, an area of expertise that's just missing. So what we find is that most people have the right building blocks in place. They don'- It doesn'- It's not rocket science, but the finicky thing is, like, getting into the corners and testing whether it works such that you can look your customers in the eye, or maybe a bank or maybe a hospital and be like, “This is going to work for you.”Vibhu [00:25:26]: I see. So we talked a lot about the agent-level certification. Where do you guys go from here? So announcing series A camera, we talked about this a bit. There's the whole security risk of Fable, government stepping in. You guys are kind of announcing that you're also going into model certification?Toward Model Certification: The Government–Lab Trust GapRune Kvist [00:25:46]: When we do a bit of cutting afterwards,Vibhu [00:25:48]: YeahRune Kvist [00:25:48]: We will not yet be announcing this,Vibhu [00:25:49]: NiceRune Kvist [00:25:50]: The question that is top of everyone's minds now is at the model level. And Mythos, then Fable, has really brought this to the fore that in addition to the commercial risk and the kind of economic security risks that are happening at the agent layer, the models are going to present risk in the national security category. The shape of the problem is very similar. You have some people that are on the hook if something goes wrong. In the case of agents, it's often security leaders in the enterprise. In this case, it's the government. They don'- haven't necessarily spent their entire lives thinking about what are the new risks that come here, what is the kind of data you might be looking for, how might you test that? But they do have to make sure that their concerns are addressed. You have some frontier AI companies that are deeply technical. They know a lot about the risks, but they fundamentally have an incentive to not always be truthful. So you have a trust gap between the government and the labs. And in every other industry, you end up with some kind of body sitting between, a neutral third party sitting between those people. There's no other industry where you allow people to audit themselves. So there is going to be a need for a third party that can take the rigor of the labs to run frontier technical evals, but can also speak legible trust in the way that the government trusts PwC to go and run financial audits. And they know that they output audit reports in a way that's consistent, that's easy to read, that's factual, that's, trustworthy. Those two things need to be brought together. And what we've learned from our work with agents is that if you want those-- that communication between those two parties to be smooth, there has to be one common standard that is public, that people can go and inspect. What are the risks that matter? Within each of these risks, what are the kinds of threat models that you're really looking for? You need to specify for each of those risks, what are the guardrails that need to be in place, and what are the tests they need to run to see whether those guardrails are effective? And then you need to go and run audits that are - technical audits that are consistent. So if you're trying to bring trust, it's extremely important that you methodically work your way through the risks. You can't send one researcher in and say, like, “Come back with whatever you find.” You need to be able to explain exactly what you did, exactly what you tried, exactly what you did not try, and therefore the kinds of promises you can and cannot make at the end of it. I think ofNeutral Third Parties, CAISI, and Model Risk AuditsRune Kvist [00:28:13]: Fable as a direct symptom of this problem that the government was told that there's a risk. The government may struggle to assess just how big that risk is. They call Anthropic, and Anthropic is trying to tell them, “Hey, actually, every model can be jailbroken.”Swyx [00:28:28]: That's not what you want to hear, right?Rune Kvist [00:28:32]: As the government, that might be hard to trust.Rune Kvist [00:28:36]: And we think that a broker is the most natural solution. In other markets, you see something like, in financial markets, you see Moody's. Moody's goes in, and they look at a bond, and they output a rating. They say like, “Here's the evidence we found. Here's the rating.” We don't decide whether anyone should buy this bond or not buy this bond. Well, that depends on their risk appetite. But we do provide this common information layer that everyone can rely on. In the case of Moody's, the government, points to them and say, “Hey, pension funds, you should probably really take care. You shouldn't risk your pensioners' money, so you can only invest in triple-A rated bonds.” That means that now the government doesn't have to staff thousands of financial technical experts to rerun forecasts every week to see whether things are correctly rated. They get to point to some neutral third party. So my hypothesis is, my hunch is that you will see a third party that sits between the government and the labs, and it could either be the government builds it themselves. So something like CAISI was set up to do exactly this. And the questionSwyx [00:29:44]: Sorry, I'm not familiar with CAISI.Rune Kvist [00:29:45]: CAISI is the Center for AI Standards and Innovation.Swyx [00:29:49]: Okay.Rune Kvist [00:29:50]: I won't get into the details, but it's a body of NIST that typically sets standards. So it's basically a government body that has AI experts. Yeah, exactly. Exactly.Swyx [00:29:59]: Very key. Very key.Rune Kvist [00:30:00]: Very key.Vibhu [00:30:00]: I think, you know, it's one of those things where when you just sit back and listen-- look at it, like, is there enough technical expertise in the government to measure, test these things right now? Probably not, right? And Fable is a result of, okay, we've had to scale back and pause things,Rune Kvist [00:30:17]: Yeah. And they have excellent people, but they have an extraordinarily small budget compared to the scale of the challenge that's ahead of us. And I think they have a role to play. The question is kind of like, who does what? We have now outlined the jobs to be done, and they're quite extensive. Every model release, there is an astounding-- Given that they take in any input, their risk surface is astounding. And so the question is really: what can only the government do, and what can the market provide here that can keep up with the pace as AI risk changes? Our perspective is that also at the model layer, the risks that people care about today are not the same ones they cared about three months ago. So the pace of legislation is too slow to deal with pinpointing the risks here. And so we think there's a lot that the market can do to surface timely information. Ultimately, there is a bunch of policy decisions here. Is the national security risks of a model too high?Swyx [00:31:12]: Yeah.Rune Kvist [00:31:12]: That's a political answer. But what we want to make sure is that the process that produces this risk information is compatible with very fast innovation. So you don't want to. This is not a question of like, can you slow the things down? Can you keep, the models locked up until-- for months on end until everyone can make a guarantee? But it is this, can you, in the time it. Given that the US is competing with China on releasing models, can you insert risk information that allows the government to, like, make rapid decisions on some of these questions? Balancing that trade-off between failing to adopt AI is going to put us at risk, but also reckless adoption is going to put us at risk. And that's a very kind of fine balance that they're going to need, like, a lot of high-quality intelligence to make.Chinese Models, Data Flows, and National Security ConcernsSwyx [00:31:55]: Just a side mention, because you mentioned Chinese models, any specific concerns that you're hearing from your CISOs about that? ‘cause I guess it's free, but.Rune Kvist [00:32:05]: CISOs have a bunch of concerns around data flows in general that they're really concerned about. So there's a lot of questions like, if these models are Chinese, where does that, where does that data go? I think a lot of this can be addressed, but they come up often.Swyx [00:32:18]: I mean, they understand they're running on American GPUs.Rune Kvist [00:32:21]: Some of them, some of them understand that they're running on American GPUs.Swyx [00:32:23]: They're not, like, phoning home every time you, like, call home.Rune Kvist [00:32:26]: No. A year ago, there was not a lot of understanding of this. I actually think, you're seeing the security leaders becoming kind of AI literate at a blistering pace, and you're actually also seeing my Twitter timeline that's very pilled and my LinkedIn feed that used to not at all be pilled kind of converge. They're both talking about Fable.Swyx [00:32:45]: Right. Yeah, that's true.Rune Kvist [00:32:46]: They are both talking about whether you can prevent models from being jailbroken these days.Swyx [00:32:51]: Yeah.Rune Kvist [00:32:52]: Like national security national security risks are now the conversation that is actually emerging. Other than that, I think you mostly see a kind of general picture: there are no concerns with any particular model or any particular model output, but there is a general nervousness of having critical infrastructure run on models that are not produced in America by Americans where the American government has control.Swyx [00:33:14]: But it doesn't necessarily show up in your framework that directly, or it might, I don't know.Rune Kvist [00:33:18]: There's a bit of stuff in there actually on the, like, the provenance of the models and disclosing that. But I think there's a bunch of use cases where running a Chinese open-source model is just the best solution.Swyx [00:33:27]: Yeah.Rune Kvist [00:33:27]: And a concern is slightly more macro here, which is not best addressed at any particular certification level.Vibhu [00:33:32]: Is there anything interesting that you see at the. You know, if you're trying to fill that middle gap, that mediation gap, any interesting stuff that you guys forecast would be required other than, you know, what the average person might expect?Cyber, Child Safety, Bio Risk, and Expert CoordinationRune Kvist [00:33:47]: There's a bunch of interesting questions about what are the risks that matter here. So right now, the risk of the day is cyber, because it's very real, very tangible. And some of the risks that are also emerging as pretty real and pretty tangible are things like child safety is becoming both extremely important, but also politically important. And then there are some of the risks that are coming down the pipeline that today feel kind of speculative, but people who spend a lot of time with the models see them coming down is things like, risks that relate to biology.Rune Kvist [00:34:18]: And specifically whether models will help adversaries produce biological weapons and making that extremely cheap, extremely accessible, producing-- making the chance of another COVID or worse pandemic. COVID was not engineered to be bad, as if you were trying to do that. So I think those are some of the risks that are coming down the pipeline. I think one other thing to just note is that agents are kind of deliberately narrow. So, like, when a frontier agent company puts a chatbot that interacts with customers, they've really tried to narrow the topics it's interested in talking about. Such that if you ask it, like, “What do you think of the president?” it will just decline, which means that the kind of risk area is somewhat smaller. For models, it is infinite. And so there's not a single expert out there who can competently evaluate the risks of cyberattacks and fifteen-year-olds having month-long conversations with a chatbot and seeing whether it will in fact recommend suicide or something horrendous like that, and can evaluate the risks that terrorists can use AI to produce bioweapons. The risk surface is just too big. And so the central challenge actually becomes how do you get those subject matter experts to work within a one coherent framework that outputs one coherent report and rating that the world can go and inspect? ‘Cause that global perspective is central, but there's not a single organization today that could produce that.Swyx [00:35:47]: And you would be the presumptive one when you put out your model standards.Rune Kvist [00:35:51]: We think there can be one company that can, with a consortium of experts, build one coherent standard. I think we've shown that across all of the enterprise risks today. We think it could be one company that could, with a consortium, specify the audit rules, basically like the inputs and outputs that all these technical experts need. What access do they need? How should they treat infosec- info security? They can look at whether the eval- evals are well-produced without necessarily being able to say, “Hey, is this a threat or not a threat?” But overall, evaluating whether the evals are good, well-constructed, that set of audit rules that basically becomes the interface for all these experts, we think one clearinghouse could put together. To be clear. When I say one company, I think of it as one company coordinating lots of this in the same way that when we saw our consortium, it's not like we say we have all the answers on agent security. What we say is we are taking on the role of eliciting all of the concerns and being the secretary that puts it together and runs a tight house such that the standard updates lockstep every quarter, and that the audit reports that come out, in this case, 100-page audit reports, uniform and crisp and clear all to the level of detail that is required for executives that need to make a clear go/go decision. So that's kind of the role that we think we might play.OWASP, Frameworks, and the Operational Audit LayerSwyx [00:37:11]: I think in many ways you're performing the role that OWASP used to do there, and you said, like, you know, competition and partners.Rune Kvist [00:37:18]: Yeah.Swyx [00:37:19]: Can you go more into, like, how they partner?Rune Kvist [00:37:20]: Yeah. So first of all, OWASP is basically an open source community of security practitioners that are coming together to build frameworks for addressing the latest security concerns. We think they are phenomenal at creating frameworks. We'- In fact, we'- First of all, we're partners with them, so we have a joint article. Two, we've learned a lot from them. We think they're a tremendous source of intelligence. What OWASP does not do is building the machine that runs third-party audits such that a company like Cursor or a company like JPMorgan could get a third party to go and review them against this and say, “Hey, you've passed the standard, and here is the report that you can use to build trust and preempt your partners' or customers' questions.” So they fundamentally try to do something different. You - They are part of the information gathering and intelligence gathering and creating clarity, but the operational layer of turning this into promises is not the business they try to be in.Swyx [00:38:14]: The standard is emerging and is doing very well. Was it necessary to then also do underwriting? Obviously it's in the name, so please remember you thought about it first. I feel like if you just have enough consensus, you don't actually need the money angle, but it does help.Vibhu [00:38:30]: I did want to also note, you guys are a profit company too, right? It's not profit where there's a whole business side to it as well?Why For-Profit Standards and Insurers MatterRune Kvist [00:38:39]: Yeah. Yeah, so I'm just getting crazySwyx [00:38:41]: I think about the money part.Rune Kvist [00:38:42]: Yeah. Yeah, let's get into the money part. Let's start from actually your question, profit versus profit. In the security space today, cybersecurity, most of the standards are produced by nonprofits. I think that's an issue.Rune Kvist [00:39:00]: The question you have to ask yourself is, how do you create good incentives for these standards to be good and keep up?Rune Kvist [00:39:09]: Nonprofits tend to not have these adverse profit incentives where they, hollow out their standard and create a race to the bottom, but they're also not at all responsive by default to the communities that they serve. There's no process-- They don't have customers that they serve where they go and ask, “What do you want? What do you want? What do you want?” And when you look at the overall satisfaction with the security standards today, people tend to just not like them very much. You do see in other domains, that profit standards can serve the world quite well. So there are examples, like we talked about Moody's before. It's not without flaws, but, it is absolutely critical societal infrastructure that gets run at an astounding scale today. Your credit score, it's FICO. It's also a profit business. And when you go back even further in history, some of the crash testing standards came out of insurance companies.Rune Kvist [00:40:06]: The insurance companies together founded the Insurance Institute for Highway Safety because they were very interested in, like, how can we use standards to drive down mortality and save money? Go back, prior-- Our name actually pays homage to the Underwriters Laboratories, UL, which, was started right around when electricity came out. Houses started burning down. Insurers, again, were paying the bill, and they were maybe also good people, but their profit incentive was, let's prevent houses from burning down. Let's test all the electrical products, the light bulbs. All the light bulbs in here are probably tested, the toasters, et cetera. And they set up, an entity to create those standards. Today, UL has a profit entity and a profit entity. What they've recognized, they spun - They started profit. They spun out a profit because what they recognized was like, hey, actually to serve customers well, you need a profit entity. The lesson here is one of the ways that the market can align incentives so you're both responsive to customersRune Kvist [00:41:07]: And not hollowing out your standard over time is to align it with insurers because they fundamentally have good incentives. And so if you're a profit standard that works closely with insurers, you get the feedback loop in such that you're really tuned into your customers, but also have their interest at heart. So that's the model that we - the kind of inspirational model that we've learned a lot from, and that's also where the name comes from. In some ways, the term underwriting can both be associated with insurance, but it's also a broad term for, like, making decisions.Rune Kvist [00:41:40]: If you underwrite a decision, you're fundamentally kind of taking ownership for the consequences of it.AI Insurance Contracts, Lloyd's of London, and ElevenLabsSwyx [00:41:45]: Yeah, I mean, what does an insurance contract look like for AI?Rune Kvist [00:41:49]: Yeah. Most of the demand comes today for insurance contracts is, sitting between people who've built AI and people who are buying AI.Swyx [00:41:56]: Yes.Rune Kvist [00:41:57]: And what you want—the reason why people want insurers involved, both for the traditional reasons, hey, if something goes wrong, we want to be compensated, but it's in particular because insurers can bring trust to the equation. Because insurers will pay for the damages, if they're willing to write an insurance policy, that is them saying, “Hey, we think there is risk here, but that is manageable.” And that is kind of a. Their incentive aligns with the enterprises adopting it, so that's a really a good signal to the market. In the same way, actually, one of the things that Waymo tried to get their first permit to even operate in San Francisco was to get a lot of insurers to stack up a huge insurance policy. In the case if something went wrong, not because Google can't pay, but because it was very valuable to have a third party go and look at that dataRune Kvist [00:42:47]: That are trusted by governments, trusted by enterprises as conservative people and say, “Hey, we've looked at it. We're actually willing to take some of this on our balance sheet.” So that's, that's kind of the reason why people are interested in it. What it looks like is, in some ways like every other insurance contract. You specify what are the perils you want to cover, how much do you want to cover them, like up to what limits, and what does it cost to cover that. And in the case of, if we take a really concrete example, ElevenLabs, bought a first of its kind AI agent insurance policy. They work with some of the biggest, enterprises that work with governments. They're really interested in going above and beyond and making promises to their customers. So they wrote a policy that covers just some of the core concerns that their customers have been asking about. And, the crucial thing was really to get Lloyd's of London, the world's oldest insurer, one of our partners, to look at this data and be that third party alongside us to say, “Hey, we think there's something here that's worth underwriting.” and that's actually what it looks like. And so they will show that contract to their customers, and they can see how much they're covered for. They can see what exactly it covers, and that will also probably change next year. They will want to write an insurance policy that might cover more.Swyx [00:44:04]: When you say Lloyd's, is it reinsurance, or are they sharing somehow at the same level orRune Kvist [00:44:11]: Yeah. So typically, the way, new companies get into insurance is that they partner with insurers such that the insurers take the majority or all of the financial risks. Fundamentally, if insurance is useful, because it brings trust, you have to be able to pay the bill. Lloyd's of London is 400 years old. They've never not paid a claim. They're extremely trusted. What Lloyd's of London struggle to do on their own is to figure out which of the risks are real, what should we be looking for, what are the kinds of technical controls, and running the tests. So they use AIUC-1 as kind of the underwriting framework, and we produce a bunch of eval results that then directly feed in to inform the pricing. So this means that ElevenLabs customers know that payment will be there. They don't have to look to our series A and see, like, do we think they have enough cash on the balance sheet? They will look at Lloyd's.Swyx [00:45:05]: Yeah.Rune Kvist [00:45:05]: Yeah.Swyx [00:45:05]: And Lloyd's, like, famously very creative. I think I remember some headline like, they insured Jennifer Lopez's, butt or something.Rune Kvist [00:45:13]: Correct.Swyx [00:45:13]: Right?Rune Kvist [00:45:13]: And I think, was it, David Beckham's right foot?Swyx [00:45:16]: So, yeah. Right?Rune Kvist [00:45:17]: And stuff like this.Swyx [00:45:18]: So, like, clearly not a large data set.Rune Kvist [00:45:22]: Exactly. It's actually a remarkable institution that's both kind of has some of the truly school virtues of having been around for a long time. They, like, really. They really operate like a trusted entity, and they have appetite to figure out the future. And I think there's a lot of recognition that both there is, like, tremendous amount of risk in AI that is poorly understood today, so getting into this business carries real risks. But also this is where lots of the risk exposure will happen in the future. This is the one market where risk is truly growing. This is the one market that will also take out some of the existing markets. Take, like, auto insurance. When there are no human drivers, how's that market going to look? Well, it's clearly going to change. How are you going to assessSwyx [00:46:08]: You want to insure Waymo?Rune Kvist [00:46:10]: I. All I'll say is the principles for how you insure Waymo are very similar to how you insure other kinds of AI.Swyx [00:46:15]: Right.Rune Kvist [00:46:15]: So again, crash testing, that's what we do for customer share at Lovable. That will also need to happen for Waymo, which is not how you do it for human drivers. So there's this growing awareness that the world is changing very fast, and the only way to learn how to underwrite AI is to write some policies. You may incur some losses and think of that as R&D expense, really. But the question for them is, like, who are the trustedtechnical partners they can get into this business with that can help them navigate and make sure they don't make, kind of foolish mistakes? But also who is willing to hear the wisdom that they have? They've done this before. They've seen it was. They were there when cyber came out. So there are lots of ways in which AI feels completely new, but there's also lots of ways in which risks look the same. And so there's actually a tremendous amount of wisdom sitting in some folks that may have gray hair, but really have, like, a keen sense of, how to quantify risk.Swyx [00:47:08]: Yeah. And the number is. So it's basically like I want fifty million dollars worth of coverage against these perils, and Lloyd's will give you a quote on it, and then you have, like, a small markup or something, and then you turn it around and do that? Is that as simple as it is?Risk Capital, Premiums, and Working with InsurersRune Kvist [00:47:23]: You basically share some of that premium.Swyx [00:47:25]: Yeah.Rune Kvist [00:47:25]: X percent goes to the people who do the pricing of it.Swyx [00:47:28]: You're. It's kind of like a. It's kind of like a merchant bank for insurance type of thing.Rune Kvist [00:47:33]: Exactly. You basically split the fee, and you can think of the insurance supply chain as, like, there's bringing the capital, there is doing the pricing, and there is doing the distribution. And typically, you will pay out some X percent of premium here, Y percent of premium here, and the rest of it will go here.Swyx [00:47:46]: Does all the insurance world work like this, or is there some point at which, like. So if right now you have equity capitalRune Kvist [00:47:51]: Yeah.Swyx [00:47:52]: At some point, maybe you start raising, debt or whatever, and then you have enough of a bank account and enough history, let's say you've been in operation for ten yearsRune Kvist [00:48:00]: Correct.Swyx [00:48:00]: That you don't need Lloyd's anymore?Rune Kvist [00:48:02]: That's totally an option. And I could see some worlds where that makes sense, specifically if there are risks that we feel high confidence that we'd want to insure where the incumbent insurers are too slow to find appetiteSwyx [00:48:13]: Okay.Rune Kvist [00:48:13]: Or simply struggle to evaluate it such that they don't want to do it. But by and large, in general, you do not want to compete with insurers on, bringing risk capital to the game for two reasons. One is that's fundamentally a cost of capital game. They have extremely low cost of capital. Startups have high cost of capital, by and large. And two, you want to hedge your bets, and it's very helpful then to also have a portfolio of home insurance, of car insurance. And we're not about to become a car insurer nor a home insurer.Rune Kvist [00:48:43]: So they have some natural advantages, which makes it much more likely that we'll partner.Swyx [00:48:48]: Yeah.Rune Kvist [00:48:48]: And they bring that, the capital at scale, and we bring the technical expertise.Swyx [00:48:51]: You're, you're going to work with them for a long time.Vibhu [00:48:52]: How are the discussions with the insurers as well? So basically, they're going off of your certification, right? They're trusting the diligence on you that your certification is valid, you tested the right things, and they're backing the money that, you know, you have the right testing in place. So any interesting takeaways from working with insurers?Rune Kvist [00:49:12]: I think the maybe the first thing is they feed into the standard as well. So if there are things that they feel like they need that they're not seeing, we are also taking that as input into the standard, because fundamentally we think a good standard is one that creates a really healthy promise ecosystem, and we think insurers are a critical part of that. And again, they are the most well-incentivized to. They see all the lost data across every. Any particular CISO knows their particular concerns. Insurers see the concerns across the entire portfolio and often have direct access to, like, what exactly happened, who was at fault, et cetera, as they do part of their forensics. So they're actually, like, a great source of intelligence on this. One of the big takeaways from cyber insurance, which is a market that didn't work that well, was that the insurance and the technical expertise was not married up. What our conviction is that standards have to precede insurance. Fundamentally, what everyone first and foremost want, whether you're a CISO at JPMorgan or a CISO at Cursor or an underwriter at Lloyd's of London syndicate, is you want to not have an incidentRune Kvist [00:50:19]: In the first place. You want to know that the risk is well-managed, and only then does insurance start to make sense. So we'll see the standard ecosystem basically run ahead of the insurance. And the reason why we. You asked us kind of why I also do insurance, this is kind of proving what we think a whole promise confidence infrastructure ecosystem needs to look like, and we think it's very compelling to bring that to life, even if we think the standard is kind of the core linchpin that unlocks the rest.Claims, Liability, Air Canada, and Duty of CareSwyx [00:50:44]: There's been no claims yet, right?Rune Kvist [00:50:45]: Nope.Swyx [00:50:46]: This is one of those things where, you know, if people haven't really worked through what it means to cover things.Rune Kvist [00:50:52]: Yeah.Swyx [00:50:52]: So for example, I pay Cursor $20 a month.Rune Kvist [00:50:55]: Yep.Swyx [00:50:56]: And I write a vibe code something that makes, a plane crash, causing $200 million worth of damage.Rune Kvist [00:51:02]: Yes.Swyx [00:51:02]:

"Oof.""Exactly."It's not all bad! It was however, quite clearly weird.Join us next month as we dig into someone else's rereading with Grady Hendrix's Paperbacks From Hell. Hosted on Acast. See acast.com/privacy for more information.

What happens when a neuroscience student tries tapping… and $30,000 comes into her business four days later?

What if one of our most loving parenting instincts is accidentally making our kids less confident?Oof.As parents, we're wired to help. To reassure. To rescue. To make the crying stop in the grocery store before everyone in aisle seven starts judging us. (You know what I mean?)But child psychologist Dr. Kathryn Hecht says kids aren't nearly as fragile as we sometimes treat them.In fact, they need age-appropriate challenges to build what she calls “handleability”, that deep-down belief that says: This is hard, but I can handle it.I absolutely loved this conversation because it gets right into the practical question so many of us are wrestling with: How do we raise kids who can actually handle hard things?We dig into:Why constantly removing obstacles can undermine a child's confidence, even when we're doing it out of loveThe fascinating idea that kids are anti-fragile, and why some stress and challenge actually help build emotional resilienceWhy “healthy kids = happy kids” is a parenting myth we may need to toss outHow rescuing a child brings short-term relief but can create more distress long-termWhy our kids sometimes interpret our help as evidence that they couldn't have handled it themselvesWhat “coping efficacy” or “handleability” looks like in normal family life, without turning your home into a psychology clinicWhy exposure therapy is much less scary than it sounds, and how parents can help kids take small, voluntary steps toward something difficultThe difference between roller-coaster scary and horror-movie scary, and why that distinction might completely change how you help an anxious or hesitant childDr. Hecht has this incredible ability to take clinical psychology and turn it into language that makes you think, “Ohhhh. I can actually use this with my kid.”And I have a feeling you'll finish this one looking at your child's next hard moment a little differently.Resources We Mention for Raising Resilient KidsSee Dr. Kathryn's books:The Be Brave Activity Book: 100 Exposure-Based Challenges That DARE Kids to Boss Back Their Worry and Anxiety Once and for All (Amazon/Bookshop.org)Releasing in February 2027 – The Can-Do Activity Book for Kids: Try Things on Your Own, Deal with Big Feelings, and Grow Confident Along the Way (Amazon/Bookshop.org)Watch Dr. Kathryn's TEDx Talk, “How to raise kids who can handle hard things”How to Raise an Adult by Julie Lythcott-Haims and my interview with JulieGet your kids started learning to do hard things with #LifeSkillsNow!Check out my new book at raisinghealthyfamilies.com/pickybook!Kitchen StewardshipRaising Healthy Families follow Katie on Instagram or FacebookSubscribe to the newsletter to get weekly updatesYouTube shorts channel for HPHFind the Healthy Parenting Handbook at raisinghealthyfamilies.com/podcastAffiliate links used here. Thanks for supporting the Healthy Parenting Handbook!

Welcome to the place where we get to let our geek flags fly and talk about all things geek. Basically a fuzzy guide to life, the universe, and everything but mostly geek stuff. This level of the podcast is the annual recap of the previous year of shows. Not a bad year: 49 episodes, 3 bonusodes and a wayback… 34 recommends and 11 good ones. I listed them all out below.34 RECOMMENDS:3 Sep - 459 - This Level is the Bomb11 Sep - 460 - One thing about podcasting I could never stomach; all the damn vampires18 Sep - 461 - Comical Monster Madness22 Sep - Wayback #9 (Ep. 59) - The (CBM) Defenders #1 - Spawn… hell yeah29 Sep - 462 - This one features creatures6 Oct - 463 - Fascists are the Real monsters13 Oct - 464 - Won't someone please think of the children20 Oct - 465 - The Monsters are out there27 Oct - 466 - Geekdom off the walls9 Nov - 468 - Talking Double V Itches17 Nov - 469 - CBM Defenders v The Fantastic 326 Nov - 470 - Releasing the geekdom cut5 Dec - 471 - A continuity poopshoot8 Dec - 472 - OOF… it happened again15 Dec - 473 - 2025 Vision18 Dec - 474 - An Alienator Predalien Poopshoot8 Feb - 480 - Welcome to our collection15 Feb - 481 - Sorry bub, it's been political the whole time5 Mar - 483 - We boldly go10 Mar - 484 - Marveling at all the shows29 Mar - 487 - Atten-hut… it's time for military geekiness12 Apr - 489 - Our Favorite Video Game Peeps26 Apr - 490 - Dang we're getting geek old5 May - 491 - Dang we're still geek old18 May - 493 - It's time to light the lights11 Jun - 495 - Underrated Disney Characters13 Jun - 496 - Disney did NOT animate this22 Jun - 498 - Coaster and mazes and shows… oh my21 Aug - 500 - The Sound of Us14 Jul - 501 - Big podcast, eh?20 Jul - 502 - Another fever dream movie poopshoot28 Jul - 503 - Doomsday is Coming12 Aug - 505 - More coasters and mazes and shows… oh my17 Aug - 506 - A Decade of THIS?1 Sep - 508 - The Musical11 GOOD ONES:18 Jan - 477 - 2026 Awakens… a good year for Star Wars?26 Jan - 478 - Winding the watch… the watchers watch still2 Feb - 479 - So mushroom to poopshoot with two fungis15 Mar - 485 - Le Poopshoot a trois flous23 Mar - 486 - Talk geeky to me8 May - Bonusode #34 - The Definitive MCU List17 Jun - 497 - No Bloom, No Jones… just Orlando5 Aug - 504 - To D or not to D… SDCC news24 Aug - 507 - Lookin at 23 Ds, errrrr, D23We also did the regular best of 2025 (29 Dec - 475) and look towards 2026 (12 Jan - 476)Congrats on ten years and completing Level 509! Feel free to contact me on social media (@wookieeriot). You can also reach the show by e-mail, laughitupfuzzballpodcast@gmail.com. All other links are easily findable on linktr.ee/laughitupfuzzball for merch, the Facebook group, etc. I'd love to hear from you. Subscribe to the feed on Spotify, Apple podcasts, Google podcasts, or any of the apps which pull from those sources. Go do your thing so I can keep doing mine. If you feel so inclined, drop a positive rating or comment on those apps. Ratings help others find the madness. Tell your friends, geekery is always better with peers. Thank YOU for being a part of this hilarity! There's a plethora of ways to comment about the show and I look forward to seeing your thoughts, comments, and ideas. May the force be with us all, thanks for stopping by, you stay classy, be excellent to each other and party on dudes! TTFN… Wookiee out!

**Special note to our listeners** Love the show? Help us keep the conversation going! Become a paid subscriber through our ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Substack⁠⁠⁠⁠⁠.⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Your contributions help us continue to make content on issues related to the Asian-American, immigrant, modern parent experience. THANK YOU to our super awesome listeners who have already signed up! --------------------------------------Oof, it's one thing to negotiate and navigate power and money issues in your own marriage - it can hit a different way when you think about how your own kids might have to deal with these issues with their romantic partners. What are you communicating (directly or indirectly) to your kids about how power and money work in a marriage? Is it different by your kids' gender? What would you hope for them in dealing with power and money issues in their future relationships? To what extent are you able to live out those hopes in your own relationship right now? ****Also, very important question- do you think you'll offer childcare as a grandma/ grandpa? Or are we crazy for even thinking that far ahead?

How to scale a team gets complicated when you realize the people who helped you get here may not be the people—or the roles—you need to get where you're going next. Oof. That's a hard one, my friend.Because when you're scaling a business, these decisions aren't just boxes on an org chart. These are real people. People you care about. People who may have been beside you through some pretty messy seasons. But you're also the CEO, and part of your job is building the team your business needs for its next stage of growth.So, how do you know when to develop someone, redesign their role, hire new talent—or make the really difficult decision to let someone go? Learning how to scale a team starts with a better question: is there alignment between what this person wants and can contribute and what the business actually needs next?We're digging into the team management strategies that help you answer that question without leading from guilt, fear, or loyalty alone. You'll learn why a strong people strategy starts with your future business goals, how to assess the capabilities and capacity you'll need, and why development only works when your employee actually wants to grow in that direction.And let's talk about hiring, because waiting until everyone is running around at Mach five with their hair on fire is not a strategy. Strong leadership skills for business owners mean anticipating what's coming and building leadership team capacity before your people are drowning.Knowing how to scale a team isn't about adding headcount for the sake of growth. It's about strategic clarity and real human leadership. You can make hard decisions, protect what's working, and build the team your business needs without sacrificing kindness, integrity, or your values.Ready to get out of the weeds? Grab the People Strategy Playbook for free. And if you need a thought partner to map out what comes next, book a leadership strategy session with me next.Support the showThe People Side of Business is the podcast for female founders and business owners who are leading teams and growing businesses.Hosted by Lindsay White, Leadership Coach, Team Leadership Strategist, and Fractional HR Expert, this show delivers practical leadership strategies, real-world people solutions, and honest conversations about the challenges of leading a team.From employee performance issues and difficult conversations to hiring, accountability, workplace culture, and team growth, each episode is designed to help you lead, manage, and scale your team as a founder.If you're ready to become a more confident leader, make better people decisions, and build a stronger, higher-performing team, you're in the right place.Learn more at highvoltageleadership.ca, connect on Instagram @highvoltleadership, or find Lindsay White on LinkedIn.The people side of business isn't separate from growth. The people side of business is the business.

How to scale a team gets complicated when you realize the people who helped you get here may not be the people—or the roles—you need to get where you're going next. Oof. That's a hard one, my friend.Because when you're scaling a business, these decisions aren't just boxes on an org chart. These are real people. People you care about. People who may have been beside you through some pretty messy seasons. But you're also the CEO, and part of your job is building the team your business needs for its next stage of growth.So, how do you know when to develop someone, redesign their role, hire new talent—or make the really difficult decision to let someone go? Learning how to scale a team starts with a better question: is there alignment between what this person wants and can contribute and what the business actually needs next?We're digging into the team management strategies that help you answer that question without leading from guilt, fear, or loyalty alone. You'll learn why a strong people strategy starts with your future business goals, how to assess the capabilities and capacity you'll need, and why development only works when your employee actually wants to grow in that direction.And let's talk about hiring, because waiting until everyone is running around at Mach five with their hair on fire is not a strategy. Strong leadership skills for business owners mean anticipating what's coming and building leadership team capacity before your people are drowning.Knowing how to scale a team isn't about adding headcount for the sake of growth. It's about strategic clarity and real human leadership. You can make hard decisions, protect what's working, and build the team your business needs without sacrificing kindness, integrity, or your values.Ready to get out of the weeds? Grab the People Strategy Playbook for free. And if you need a thought partner to map out what comes next, book a leadership strategy session with me next.Support the showThe People Side of Business is the podcast for female founders and business owners who are leading teams and growing businesses.Hosted by Lindsay White, Leadership Coach, Team Leadership Strategist, and Fractional HR Expert, this show delivers practical leadership strategies, real-world people solutions, and honest conversations about the challenges of leading a team.From employee performance issues and difficult conversations to hiring, accountability, workplace culture, and team growth, each episode is designed to help you lead, manage, and scale your team as a founder.If you're ready to become a more confident leader, make better people decisions, and build a stronger, higher-performing team, you're in the right place.Learn more at highvoltageleadership.ca, connect on Instagram @highvoltleadership, or find Lindsay White on LinkedIn.The people side of business isn't separate from growth. The people side of business is the business.

Oof, this one is a big feels one. The very good fwend, Felicity Ward, sits down with the gang to chat about therapy, getting sober, and the profound power of female friendship. We talk about growing up around Pretty Beach, navigating teenage neurodivergence, and an extraordinarily beautiful story about meeting your best friend on "the third hill." We love Flick. it's Fwends!See Flick LIVE! https://www.felicityward.com/CONTACTVoicemail - speakpipe.com/fwendspodEmail - fwendspod@gmail.comFWENDS WITH BENEFITSGet ad free listening, access to the Simply Marvellous archive! (Simply Marvellous both the perfect adjective and also actually just the name of the old show). Plus other fun stuff!Apple: Subscribe above!Not Apple: https://fwends.supercast.comRATE AND REVIEWIf you'd be so kind, we'd absolutely love you to leave a charming rating and review? In whatever podcast app you're in right now...a little 5 stars will do?FOLLOW FWENDS ON INSTAGRAMGeorgia MooneyKyran NicholsonRhys NicholsonComedy Republic Hosted on Acast. See acast.com/privacy for more information.

For today's podcast, we had an organized agenda (really!), then the last twenty-four hours happened.The Hawk Fire started on the slopes of Peavine Mountain in the immediate northwest region of Reno, and the winds drove it in all directions, including towards our daughter's home and place of business. We were on the edge of our seats all of yesterday until she made the wise decision to evacuate herself and her three cats. When we woke up this morning, we saw the fire had reached US Highway 395. Oof. Our prayers and strength to all affected.While this was unfolding, I remembered the Nevada Appeal's "Best of Carson 2026" contest had ended last week. We did not receive an invitation to the awards ceremony, so we assumed we had lost. I wondered where we placed...and found that we had won our category. Uh, wow. We were voted as the "Most Unique Business" in the Nevada Appeal's Best of Carson City 2026 Contest. Huh. Go figure. One of the nice things about this contest is the local visibility. We may buy a banner (

When we first dicsussed the Summer of Simulative AI in 2024 we knew it would be a brief summer, but it has recently come back with a vengeance with SimGym in April and now Simile AI's $2B Series B, backed by GreenOaks and Index Ventures with prominent backers like Fei-Fei Li and Andrej Karpathy, running tens of millions of simulations for Fortune 100 clients like CVS and 85–99% accuracy vs human focus groups. Time to catch up on why this Second Summer of simulation is working!From creating Smallville, the landmark 2023 paper on Generative Agents that showed AI characters could remember, plan, socialize, and develop emergent behaviors, to now building foundation models of human behavior, Joon Sung Park is trying to answer a much bigger question: what if we could simulate the world before making decisions in it? In this episode, the Simile co-founder and CEO joins us to unpack the path from generative agents to digital twins, why today's frontier models still fail to capture how humans actually behave, and what it would take to eventually simulate all 8 billion people on Earth.We go deep on Simile's approach to modeling human behavior: long-form interviews, observational and transaction data, randomized controlled trials, population-level and individual-level models, and post-training on the causal mechanisms behind why people make decisions. Joon explains how his research created digital twins that reproduced human behavior and attitudes 85% as accurately as people reproduced their own responses, why models optimized to be rational can be bad simulations of irrational humans, and why understanding “social physics” may require changing model weights rather than simply prompting frontier LLMs.We also explore the much larger ambition behind simulation: testing products and policies before deploying them, finding counterintuitive paths toward desired outcomes, modeling emergent behavior across entire societies, and potentially tackling problems like climate change, democratic instability, and UBI. Joon reflects on scaling laws for simulation, the economics of data-center-scale simulated worlds, the connection to Thomas Schelling and psychohistory, why simulation is surprisingly similar to painting, and whether we might already be living in one.We discuss:* How Smallville and Generative Agents led to Simile* Why Joon's team asked: “What if we can just recreate the world that we live in?”* Why useful personal agents require deep models of their users* Memory architectures, Markdown files, and the limits of prompting* “Social physics” and behavioral foundation models* Why web data captures what people say more than what they actually do* Interviews, transactions, observational data, and randomized controlled trials* Why predicting the future matters less than understanding how to shape it* How Simile creates representative simulated populations* Simulation versus prediction and the connection to Foundation's psychohistory* How to evaluate simulations instead of simply stacking LLM hallucinations* Creating digital twins of 1,000 real people and reaching 85% behavioral accuracy* Why frontier models can struggle to reproduce real human behavior* Why good simulations need to reproduce human biases and mistakes* Post-training models on randomized controlled trials* Population-level versus individual-level simulation* Scaling laws for human simulation* The long-term ambition to simulate all 8 billion people on Earth* Whether simulations could help solve climate change or detect collapsing democracy* Thomas Schelling and the history of agent-based modeling* Why future simulations could require an entire data center* Multi-agent simulations and what happens when simulated people interact* Replacing expensive human panels with synthetic populations* Why market research is only the starting point for simulation* Why Joon sees simulation as surprisingly similar to painting* Using simulation to study questions like UBI* Whether we are already living in a simulation* Why AGI and simulation may be the twin technologies of advanced civilizationsJoon Sung Park* LinkedIn: https://www.linkedin.com/in/joonspark* X: https://x.com/joon_s_pk* Website: https://www.joonsungpark.com* Simile: https://www.simile.comTimestamps00:00:00 Introduction and Joon's Path from Art to AI00:01:46 Smallville, Generative Agents, and the Origins of Simulation00:05:03 “Let's Just Create a World” and the Future of Personal Agents00:09:53 Social Physics and Behavioral Foundation Models00:14:08 Prediction vs. Simulation: How Do You Shape the Future?00:16:59 How Simile Models Real People and Populations00:25:35 Evaluating Simulations, Digital Twins, and 85% Accuracy00:30:23 Post-Training Models to Reproduce Human Behavior00:40:04 Scaling Laws and Simulating 8 Billion People00:43:10 From Schelling to Society-Scale Agent Simulations00:46:13 The Cost and Economics of Simulating the World00:52:05 Real-World Use Cases, Synthetic Populations, and the Market00:57:27 The Future of Simulation, Painting, and UBI01:04:23 Are We Already Living in a Simulation?01:06:08 Building Simile and HiringTranscriptIntroduction: Joon Sung Park, Simile, and the Story So FarVibhu [00:00:00]: Today, we have Joon in the podcast. Excited to kick this one off. Very exciting company. I wanna kick off and ask you the question, talk us through the story of your life. How have you gotten here?Joon [00:00:13]: Yeah, for sure. I'm really excited to be here. A story of my life. So I was born in Korea, and I lived there for a good 11 years or so of my life, and then my family moved to Boston. So we moved when I was 11, and my parents were doctors, so they were going through their postdoctoral studies. My dad was a surgeon, so he was doing his sabbatical years at the Boston Children's Hospital. So I grew up there, not too close to tech. I was very much a music and artsy, painting kind of guy.Vibhu [00:00:49]: Painting.Joon [00:00:49]: Exactly. I got into painting a little bit later, in high school, but that's what I used to do. And then I grew up mostly in the East Coast after Korea. So I lived a good number of years in New Hampshire, and then I went to college in Pennsylvania. And I got into more of this tech scene, in college. So I was originally trained to be an artist. I thought that would be my professional career. So it wasn't a hobby. It was like, “Hey, let's make a living out of this.” And then gradually, I got really interested in this idea of, hey, the greatest artist often creates their own medium, and the best medium that we had available today was in computation. So I decided to go deeper into that, and one thing led to another, and we can go deeper into this, but I decided that research was something that I gradually got interested in, and here I am.Smallville, Generative Agents, and the 2023 Breakout PaperSwyx [00:01:46]: So there's a lot that you packed into the research components. You had one of the best papers of 2023, which was the generative agents paper, commonly known as the Smallville paper.Swyx [00:01:58]: Feel free to call back to anything else that you mentioned, but most people would have heard of you from this. Do you have any statistics on how many people have, like, read it? arXiv gives you something, right? Some stats.Joon [00:02:10]: Yeah, it's a good question. How many people have read it, I'm not sure.Joon [00:02:14]: I know we do keep track of citations, and they are going up quite fast.Swyx [00:02:23]: Yeah, Google Scholar has 7,200 citations.Vibhu [00:02:25]: I feel like it made a bigger hit than that, and it was a pretty instrumental paper. It got cited so many times.Swyx [00:02:34]: It is frequently the answer when people ask, “What is the best paper you've read recently?” It's this one.Vibhu [00:02:39]: I thought the memory component was pretty underrated. It was a very good early memory system, and one of the biggest papers.Foundation Models and the Search for Killer ApplicationsJoon [00:02:47]: Yeah, so maybe I can talk a little bit about how this particular paper came together. So when I got into research, it was back in 2020 when I started my PhD program at Stanford, and that was the year, when we were about to get GPT-3 to be available. So we already had GPT-2, and you could sense that there was this new class of models that was just becoming available in the market, and the team got very intrigued. And the general consensus was, “Well, is this model going to be useful for anything?” “It's really strange that these models are not trained to do any particular task.” But we decided to take a bet. So a large group of scholars at Stanford, and it was led by one of my co-founders, Percy Liang, and we came togetherSwyx [00:03:35]: Who coined foundation models.Joon [00:03:36]: Who coined the term foundation models. We wrote this paper, where that term came from called Opportunities and Risks of Foundation Models. And during that process, really the thing that I started to think deeply about was, here is a model that is fundamentally new in our ecosystem. The reason why this was new was it wasn't, again, trained to do anything in particular, but its premise was it could do anything and everything. It was like a stem cell, if you were to take a biology analogy. And I got really interested in this idea that, well, if we were to really think about what are the killer applications that this particular technology would enable, what would that be? Many of my colleagues were using this for simple classification, simple generations. Interesting that these models can do that, but from an interaction perspective, not that interesting. We've known how to do that for many decades. And what we came down to was these models are trained on this very broad data from the web, right? So these are human behavioral data. It's social media, Wikipedia, all these data. So if you poke at the right angle, then you could see human behavior that would just pop out that's quite realistic, and we've never seen that before.The Time Machine Game and Recreating the WorldJoon [00:04:45]: So that got us really interested. The exercise that we decided to do, with this particular group of colleagues, Michael Bernstein, Percy Liang, and myself, who ended up becoming my co-founder at Simile, we sat down and we played this game that we call the time machine game.Joon [00:05:03]: Imagine we were to get on a time machine and fast-forward 10 years and look back. What would have been the single application that will have mattered that would be the most interesting and inspiring? And when we thought, “Well, what if we can just recreate the world that we live in?” it's really hard to get more ambitious than that. Like, let's just create a world.Joon [00:05:24]: And that's where we started. And initially, we had this paper that was a precursor to the generative agents paper called Social Simulacra.Swyx [00:05:32]: Before you go further, were there other candidates for the most ambitious thing in the time machine exercise? What was number two or number three?Personal Agents, User Models, and Why Simulation Came FirstJoon [00:05:44]: There is a close second that we were considering, which ended up becoming more of these automation tools, especially the vision around really personalized agents that would do things for you.Swyx [00:05:59]: That's also happening.Joon [00:06:00]: It's also happening. But it was interesting for us, right, in that the reason why, we decided to go with the idea of simulation, one, I was a huge science fiction nerd, and this idea of creating simulation, I was personally really just fascinated. I loved the idea. It's really cool to see, like, a game town like this and just see these agents live in it. But at the same time, my bet was if you were to create a really amazing personal assistant out of this technology, what you need first is an amazing model of your users. So I told a model, “Hey, can you go buy late dinner for me?” And it orders Hawaiian pizza, and I do not like pineapples on my pizza. Then it totally failed. The way for it to not make that mistake is only by having a deep understanding of who I am. And I gave a very simple and dumb example here, but you can imagine how this core understanding of people is instrumental. This is how, if we have our family and closest friends, they have a good mental model of who we are. That's the basis of our social connection. So our bet also was this technology around simulation, creating accurate representation of people ought to precede the more complex agents that would automate the world that we live in. So that was the bet. But that was a very close second, and I'm still very much fascinated by it. I think there's a lot of interesting work that's going around. My hot take here, though, is I don't think we've seen a true personal assistant that's useful, in ways that meet the ambition of that particular line of work. I think there are early applications that are interesting, and if you talk to even ChatGPT nowadays or Claude, they know a lot about us. So a lot of the generation it's doing, I do think it's much more tailored, but I think the ambition is quite large in that field, and I don't think we quite have all the right ingredients just yet.Swyx [00:08:01]: So OpenClaw and these personal agents, what do you want to see from them that they don't currently have?Memory, Markdown, and the Limits of PromptingJoon [00:08:09]: I do think it's slowly getting there, but I do generally want them to have much deeper understanding of the person. Right now, you look at the models. OpenClaw, what it's leveraging is a Markdown file, and I think it's quite clever, right? So if you look at the generative agents paper, this was the same intuition that we had, where initially when we were creating the memory architecture for the generative agents, and, like, this is, like, back in 2022, so we didn't really quite have the idea of even agentive architecture or the term agent. But the intuition that we shared with some of the work that's coming out today was we initially thought, “Well, do we want to make the memory into, let's say, knowledge graph? Do we want to train a bespoke model?” All of these things. And what we decided to do was, “No. Just forget about all this.” These language models are quite good at modeling text and understanding and reasoning about text. So just put everything in a Markdown file or a text file. You're done. I thought that was quite interesting that we could do that, and there's a lot of strength in doing that. But also, there are limitations. It's the way you retrieve and make sense of data that's extremely large, it takes a lot of work. So I think that technology is getting better. I also do, however, think, there are certain things you just cannot shape just by prompting the model. So to some degree, you do need to touch the parameters of the model itself. So there is this work that I do think does need to happen, and it is happening. The question is, how far can we take it? How do we source data, and how do you also create an ecosystem where people are continuously feeding data to this model so it's learning about you?Vibhu [00:09:50]: What's the intuition between why you need to do it in the model?Social Physics and Behavior Foundation ModelsJoon [00:09:53]: My intuition behind the actual when do you train or even post-train a model versus just prompt a model is if the model has to learn the underlying physics of the world that it's operating in. So it has to learn new social physics. The places where it doesn't have to train are the places where it already has the physics. We trust the physics. It already has the base statistics, but it's just trying to react to an environment. Then I think you can just prompt your way into getting the actions out of it. I don't think the models that are out in the open have yet learned the complete mapping of social physics of humanity. This is one of the core theses of Simile, right? And one of the core reasons why that is the case is if you look at the data that the model was trained on, these models were trained on the web data, like, whatever was available on the web. And these are really interesting data sets, but they are fundamentally the self-exposed attitudinal data with some behavior data that's sprinkled around here and there. And it has yet to learn the really deep behavioral nature of people, not just what people say they do online, but what they do in real life. And this is one of what I would consider to be the dark knowledge of humanity that we haven't quite captured. And it's these data that would also need to get factored into the model creation.Vibhu [00:11:21]: You call it behavior foundation model.Vibhu [00:11:23]: There's a good one-liner here, but outside of that, what type of data do you need? What are you changing on the model level? How do you go about modeling, doing a behavior foundation model?The Three Data Buckets: Interviews, Behavior, and CausalityJoon [00:11:35]: We think about data in three buckets. So one bucket is interview data. It's quite interesting. Rich qualitative data is interesting. It's not behavioral, but we would literally ask people, “Hey, tell me the story of your life.”Vibhu [00:11:53]: It's just what we're doing here exactly.Joon [00:11:54]: The question that you all asked at the beginning of this interview literally is the question we also ask. And we ask our participants to go a little bit deeper, than how far I went. Maybe I can give more of my life story in lieu of this. But the reason why that data is interesting is by learning about this very long-tail information about people, you get a lot of texture around this model, like, this person as a model. So even understanding their childhood memory or even their trauma, their first love, these things, quite informative in ways that's really hard to predict. So that's one. Then there are two tranches of what I would consider to be the behavioral data. One kind of behavioral data is observational. So these might be like transaction data, or these might be data that you can get by scraping the web, right? So you can imagine why these data sets would be interesting, right, because they give you the base statistics of people's behavior.Joon [00:12:55]: But then there is the last category of data, that I personally think is perhaps the most important, which is the data that describes the causal mechanism, the whys of people. Some of this is covered by the interview data, the qualitative, because people talk about why they made certain decisions. But really, where you get to see the most behavioral aspect of this is in randomized controlled trials, like RCTs. Imagine you have the same setup, but you have a few different variables that you are trying to tweak. Can you get realistic human behavior out of it in ways where, imagine you had this particular option. Imagine you're even trying to choose whether you're going to drink coffee or not. The day you drink coffee versus the day you didn't drink coffee, does your behavior change? That's a data set that describes a causal mechanism. This is quite important in modeling people. The reason why this is important is oftentimes when people come to us, or not just to us, but the reason why people are interested in simulation isn't because they want to predict the future. If you're trying to win against the stock market, predicting the future is interesting.Prediction vs. Simulation: Shaping the FutureJoon [00:14:08]: But most people, most decision-makers, what they want to know is, how can we shape the future? It doesn't really help you to hear that your sales are going to tank in two quarters. They're just gonna say, “Wow, that sucks.” What they want to know is, well, what do we need to do now to avoid that future? That's the causal mechanism. And this is also very hard data to come by, right, because the world is our ground truth, but it happens once. So in a very controlled setup where everything is equal except for one variable, this kind of data set rarely happens. So this is a reason why this data set is both hard to come by and quite important if you're trying to model human behavior.Swyx [00:14:50]: So behavior, I think, is the hardest data set to acquire. What is out there? What is even possible? You're not going to know a lot of details about my life. I don't even have data for myself on my own health or habits, and I just don't log everything. So how can you have that data?Joon [00:15:14]: So we run a lot of randomized controlled trials.Swyx [00:15:17]: But you put people in the lab, they watch them sleep, or what?Joon [00:15:20]: We do care a lot about the consent process. People know that we invite them to be a member of this community to both share data and have themselves represented in different forms. But we bring a lot of people to the lab, or virtual lab, where we design experiments that would pose them real behavioral decisions. And often in these experimental setups, what makes the difference between what is attitudinal versus behavioral is whether the stake in your decision is real. That's ultimately what makes it behavioral. So in these setups, we are inspired by our colleagues in social sciences, psychology, and so forth. So when they run studies, the techniques they utilize is imagine there's an online store that you're inviting people to come by. Then whatever they purchase in this experiment, they actually get that item delivered. Like, these are the things that make the stakes real. So we run a lot of these experiments, and we also do partner with firms. Right now, we also have customers who are quite excited to at least give us a glimpse of the behaviors that their users exhibit so that we can get a little bit deeper understanding of how people behave in these different platforms.How Customers Use Simile: Populations, Queries, and ExperimentsVibhu [00:16:39]: I think on the customer side, they have a lot of data about their users, who has bought. They have the action data.Vibhu [00:16:47]: Can you walk us through an example of what someone comes to you for? What questions would they want solved? Do you customize a model for them? Do you have something off the shelf? What does that look like?Joon [00:16:59]: Today, when people leverage our models, it's often to better understand the population of their interest. So usually, the start of the relationship, we come together and hear about what population they want us to model, right? So it might be that if you're a CPG company that's selling to all of the US, then maybe it's fairly straightforward. You want to model the gen pop of the US. But at the same time, if there is a vertical or if there's a market that they're trying to go into, imagine, they want to better understand, let's say, people in their 20s and 30s living in California. That's a much more specific population. So we hear about this population, and we go recruit these people, with consent, and with incentives, and we collect some of their data and create a model of these people. Then what our product allows you to do is query them. So it can take as input a filter that is a description of the population that you want to talk to, just like the one I just mentioned, and an environment. The environment can literally be survey questions, behavioral experiments, It can be A/B testing. Oftentimes, the core use cases are things like concept testing, to start with. But also, people sometimes want to do focus groups or one of the fun use cases that we also serve is even modeling things like earnings calls for public companies.Joon [00:18:21]: So these are the use cases that we often start with.Swyx [00:18:23]: Concept testing, is that an established term? I've never heard of concept testing.Concept Testing, Gallup, and PoliticsJoon [00:18:27]: Yeah. So it has to do with they have, let's say, different messaging, different products, different ideas.Swyx [00:18:32]: It's like a marketing exercise.Swyx [00:18:33]: Okay, got it. Got it. Politics?Joon [00:18:36]: We do, have a strategic partnership with Gallup, and of course, Gallup is deep into policy space and so forth. Right now, we have not worked deeply with politics, like that area just yet, however.Swyx [00:18:49]: I'm curious if there is demand or if they really would have different needs that somehow fundamentally don't mix with your existing, users or people.Joon [00:19:00]: I think there's certainly demand.Joon [00:19:02]: But we are very much mindful of how this technology gets adopted and the societal impact that we'll end up having with this technology. And I do see politics as an area where a company has to be particularly thoughtful about the way they operate and make impact. So this is where we also want to make sure that we form enough of guardrail and perspective on how to leverage this technology before we go on to serve markets like the politics.Swyx [00:19:29]: I'll give people an example. one of my favorite shows is The West Wing. I don't know if people have watched.Swyx [00:19:34]: One of the key storylines is, like, the president has, multiple sclerosis, but they haven't. they need to figure out how to disclose it. So they run a poll with a fake governor and ask people to respond on the poll,Counterfactuals, Polling, and When Simulation Is UsefulSwyx [00:19:47]: They try to make decisions based on the results of that poll on, like, how well they'll be received, like where, how should we play this?Swyx [00:19:54]: And I'm like, well, I think those counterfactual things, I would use a simulation for this if I could trust it.Joon [00:20:01]: For sure.Joon [00:20:02]: In that show, how'd it go?Swyx [00:20:04]: In that show, it was, like a foregone conclusion. They were like, “We know it's bad. We just don't know how bad.” And then the poll came back. It was like, “It's really bad.” And then they just did it anyway.Joon [00:20:14]: Part of it is to show, right? So you're, you're looking at the ideaSwyx [00:20:17]: Maximizing drama.Joon [00:20:18]: How bad could it be? Oh, it's horrible.Swyx [00:20:20]: And to some extent, I think that is part of the trick of the, or the challenge or with being a customer of yours, which is that if I know it's. if I roughly know and can intuitSwyx [00:20:35]: What the effect is going to be, do I need you? What sensitivity of it, of effect do I need in order to make a decision, right? So for example, if I, my approval rating is 50%Swyx [00:20:48]: And I, they have this negative piece, news item comes out, and it drops to 30.Swyx [00:20:52]: If it drops to 20, if it drops to 40, do I care? No. It, I know it drops. It's negative. So when do I care about simulations?Joon [00:21:01]: You do something that's clearly bad, that's not popular, and people don't like you, like, yeah, it's likeSwyx [00:21:05]: You don't need a simulation.Joon [00:21:07]: Yeah. Well, so there are a couple of things. one is, there are use cases where, like every day, developers, designers, policymakers, marketers, every single day, they create assets. They create new products. And turns out, it's many of the decisions in hindsight is obvious. Yes, of course this is bad, but we still run those studies because understanding the magnitude and understanding how acute something is quite difficult, even if, we feel like, of course, like this makes sense. this is the reason why we make so many mistakes. Like, every time somebody goes online and say something that has huge backlash, you look at that and like, “What an idiot.” However, it's tough. That's one. There's also another aspect here, which is, again, this is the reason why simulation is different from prediction. In simulation, in the ideal case scenario. So what simulation is trying to show is it's trying to show each step of the way or each step that we need to take to get to a certain outcome, right? So in the most advanced simulations, sometimes the next step that we're suggesting might be quite counterintuitive. The analogy that I sometimes give, and I ground it in a more realistic example, but, I, as I mentioned, I'm a huge fan of science fiction, and I don't know how, many of the audience members have read, like, things like the Foundation series by Asimov.Simulation as a Path, Not Just a PredictionSwyx [00:22:37]: Oh, yeah. We've mentioned psychohistory a number of times.Joon [00:22:39]: Okay, fantastic. So I might be, talking to the right crew. If you read Foundation series, literally the first act is there's a group of scientists who have found out that, “Oh, our galactic empire is going to collapse, and we're going to have 30,000 years of unrest.” And they run psychohistory, the simulator that tries to teach them, “Okay, how can we keep this unrest to a 1,000 years?” And they plan this out, and the first step of that plan is to get the scientists who say, “Okay, this is coming,” exiled into this random place in this, galax- galaxy.Swyx [00:23:18]: Terminus.Joon [00:23:19]: Exactly. And that's so counterintuitive. Like, what a strange move that you literally sent the group of scientists who was raising voice around this potential collapse of galactic empire into nowhere. How is that the right first move? Well, it turns out in this particular simulation, that was the move.Joon [00:23:40]: It's these things, right? And the reason why these reasoning is possible is because you're showing the step function or each step that results in a particular outcome. So really what simulation allows you to do in its highest form is you give it not a problem or question, like what would people answer to the survey? That's not what we do. What we tell it is, “Here is a goal that we have. In the context of foundation, we want to keep the unrest to a 1,000 years. What is the path that we need to take now to get to that particular future?” And that's what simulation allows you to do. Now, translating that into real market, imagine you're a automobile company and you're about to release a, EV, and you're trying to understand, well, how do we market EV, to make sure that our stock price goes up? But what if the answer comes down that, well, you can market your EV in XYZ way, but that might change people's perception around the cars that's not EV and make your overall sales to go down. Not very intuitive, especially all you're trying to optimize is EV salesss, and that's the only thing that you're tracking, then that might result in a completely wrong solution, or at least different solution than what you would have expected, whether it's right or wrong.Joon [00:24:57]: That's the power of simulation.Swyx [00:24:58]: For listeners, we covered a similar topic with Mikhail Parakhin from Shopify, where they are working on SimGym. I don't know if he ever talked to you about it. it's very similar.Joon [00:25:07]: ISwyx [00:25:07]: The goal is increased conversion, but then the journey is very unusual.Joon [00:25:12]: Journey is unusual.Swyx [00:25:12]: Yeah. The-- He's trying to look for interventions on a shopping trajectory, which is similar to what you're saying. Like, it's not about the attitudinal, is your word for it.Swyx [00:25:24]: It's about behavior.Joon [00:25:25]: It's about behavior.Swyx [00:25:25]: And that's exactly the difference, right? It's, like, not about the near-term direction about-- but it's more about, like, how do you affect multiple turns of interactions.Vibhu [00:25:35]: You had a good quote at the start about this as well. It's not about people wanting to know the outcome. It's about how they can change it, change the way to get there, something like that. But I wanna take it back to how do we know this is grounded? LikeGrounding and Evaluating Digital TwinsVibhu [00:25:47]: How do you run evals? How do you test that simulations come through? if I was to do the same thing that you described with, say, your favorite LLM, Opus, GPT-5.6, have some agent to map out these thingsVibhu [00:26:02]: How different are the answers we would get if I give it the same goal, the same objective, make a decent system? You're saying that you need to change the model weight. You have your own solution to this. But how far off are we, and how do you check if it's grounded? you have some interesting stuff on your site that points to how you run real evals, but if you could take us through that side. I think that's one of the big concerns that people have. They're like, “LLMs hallucinate.”Vibhu [00:26:27]: “You're just hallucinating layer after layer,” right?Joon [00:26:30]: The way we do this, and this is the paper that we worked on after the generative agents paper that really became the, at least for Simile and also the field of simulation and synthetic panels, really became the foundation. Yeah, this is the paper. the paper is called Generative Agent Simulations of 1000 People. Here's what we've done. For this paper, we brought 1,000 people that's representatively sampled from the US to a virtual lab. And what we have done was we spent two hours collecting fairly wide-ranging data. In this particular study, we focused a lot on this interview data, that was, whose script was taken from this project called American Voices Project. And then we would also pair that with a lot of behavior data and so forth, whatever we can collect within two hours. And then we would send these people away for a couple of weeks. And during that time, I would use this data to create their digital twins. And I would bring the humans, participants back after 2 weeks and have them complete a battery of surveys, experiments, behavior studies. So we have the list here, which included things like behavioral economics games. We would run literally, like, Big Five personality test, General Social Survey. We would also go ahead and run the randomized controlled trials that were published on PNAS. And we would have their digital twins predict how the source individuals would have acted in these studies and surveys. And this is where we could replicate people's behaviors and attitudes 85 percent as accurately as people would replicate their own. So that was the first really paper that gave this validated results that we can model individuals in an accurate way. And what we ended up finding now, of course, in AI space, so this paper came out at the end of 2024. AI space, a year and a half, 2 years, that's a lifetime.85% Accuracy and Why Frontier Models Miss Human BehaviorSwyx [00:28:24]: Yeah. Just, for listeners who are not seeing the YouTube, I just wanna say, like, the headline figure is 85 percent accuracy, like, which is a big improvement over all the otherSwyx [00:28:34]: Methods that you showed.Joon [00:28:36]: But the part that was particularly striking to us, especially as we improved this technology even further, was the generative AI models like ChatGPT, Claude that's coming out, it does give you the right foundation. However, what they do not consider is the true attitudinal and behavioral aspect of people, especially in the population that you care about. So what these models are really good at today is they're trying to become the super rational, objective machines, right? So you go get their data from places like Mercor, Scale. You talk to professional programmers, scientists to create model that's amazing at reasoning. That's what they do. Simile doesn't care about any of this. The models that we're talking about here, what we're trying to create are models that are as dumb as I am, right? So if I make some mistakes, the model has to make the same mistake.Swyx [00:29:34]: Oh, that's very hard.Joon [00:29:35]: That's very hard.Swyx [00:29:36]: You're solving Murphy's paradox.Joon [00:29:37]: That's exactly. And this is a completely different data and training objective. This is also where we see quite a bit of discrepancy in the performance in human behavior prediction between the frontier models, Simile's model, and the models being created in this space, where in some cases, the model performance of frontier models go all the way down to 20, 30 percent, especially if you go into that more niche population on topics that our customers would care about. On more gen pop, it might be around 50 to 60 percent. So it's not very robust. Like, you wouldn't want to make your decision off of these and these findings. If you can bring that up to 85 percent, that is ultimately what people end up getting very excited about.Swyx [00:30:20]: Yeah. Do we wanna keep going on the paper, routes?Joon [00:30:23]: Yeah, for sure. So the last one, was an interesting one. So this, paper was the follow-up paper that we had, to the 1000 agents paper, where the idea was now can we augment the models even further and post-train a model based on a lot of randomized controlled trials? So this was an interesting one. The data is always the most interesting part of modeling in many ways. The data that we got here was there's this, there's this platform called Open Science Framework. So some, the audience might be familiar with this. And there has been, especially in the social sciences over the past 5 years or so, there has been this concern around replicability of studies. And so it was a bit of a crisis, the scientists acknowledged, where we rerun the study and we don't see the same finding.Post-Training on RCTs and Replication StudiesVibhu [00:31:12]: Oof.Joon [00:31:12]: It's tough. And the reason why it's there-- that was often the case was there's this survival bias where the papers that get published often need to maintain what we call the value of less than 0.05 in the experiments that we ran. That suggests that only-- there's only 5% chance that the results that we saw is false positive. But the tricky part was all the papers that were not published, and there's still a 5% chance that whatever we publish is totally just randomly generated. Like, there's a 5% chance that, hey, this effect is not real, but it just happened to be real because of the sampling bias. So because of that, what scientists started to do was they started to register their studies. So before running an experiment, they would go to this platform and say, “Here is the data. Here is the population that we're collecting, and here's the hypotheses.” And they would just say, “Here is our hypothesis.” Like, “This is what we believe.” And you cannot retroactively change those hypotheses. This is what gives us more scientific statistical confidence that whatever effect that you ended up seeing is true. So that ended up creating this really interesting platform where there's one platform that has now contains tens of thousands of real-world experiments and hypotheses. And a lot of these are really high-quality, like, professionally designed behavior studies and random- randomized controlled trials. So we got the data and the studies from this platform and used that to make a point. And this particular, model is not, something that we're serving commercially because this was a part of the open science. But this particular data set, helped us make a point that by collecting a lot of these randomized controlled trials, that are really well-designed, we can make significant improvement in model's capability to predict human behaviors. So that's what this paper was about.Vibhu [00:33:10]: Is this stuff done on a individual level? Like, do I need to tune the model per individual, per company? Is there foundation model changes and then some slight post-training? Anything you can share there?Population-Level vs. Individual-Level ModelsJoon [00:33:21]: So this particular model was trained. the data we had at the level of individuals, but this particular model was trained. We experimented with both. And this is what we end up doing at Simile too. We always train 2, distinct model. One is what we call the population-level model. The other is what we call the individual-level model. And both take very similar input, which is the description of a subpopulation or individual and a stimuli. In this particular work, we've done the same. Here, the results that we are reporting are much more geared towards individuals because we do think that is a harder task in many ways, but that's what we have done.Vibhu [00:34:02]: You seen anything on the questions that humans can solve that models can't solve? So likeHuman Biases, Mundane Choices, and What Models MissVibhu [00:34:09]: Currently, it's, I live 5 minutes walk away from a car wash. It's a 10-minute drive. Should I walk or drive?Joon [00:34:16]: Huh.Vibhu [00:34:16]: The model will say, “Oh, walk to the car wash.” And, you don't have your car.Vibhu [00:34:20]: Is anything like this a problem in simulation? You would assume, like, very simple for human to think about, but if the model is saying you should walk to the car wash, anything here?Joon [00:34:32]: It's less, what can we solve, but I think it's more about what biases or mistakes do people make that models miss. Like, imagine that you are, like the. When I was still at Stanford, I lived in Palo Alto. So it's about, I would say, 40-minute walk from the campus. You ask the model, “Okay, let's go home. What can I, what can I do?” It would likely call an Uber or, give me, the bus time. But for the longest time, I really liked walking back. And the reason why I wanted to do that was not for efficiency. It really helped me think. And I like to walk for, half an hour or 40 minutes or so a day, where I just get to, just think about ideas, research, just get lost in my thoughts. That's very human activity. Unless the model has seen that and understands the importance of that activity, it would miss these kinds of features. So that I think, is fundamentally what we're trying to model. Like, what is fundamentally human might not be the most efficient thing to do, might not be the right thing to do, but things that make us who we are.Swyx [00:35:43]: I'm curious if, there are some data sets that you really want that would materially help you. One version of this may be interesting, which is more valuable to you to acquire as a data set, all of LinkedIn, all of Twitter, all of Facebook?What Data Matters: Social Media, Transactions, and FacebookJoon [00:35:57]: It's a little bit hard to rank, in part because, there's, there's this product saying where no feedback is wrong because it teaches you something about your users. Doesn't matter what feedback.Joon [00:36:11]: I think it's a little bit like that.Swyx [00:36:12]: So just whatever is bigger.Vibhu [00:36:13]: What about a different domain? Say it was. What about all of Amazon data?Joon [00:36:17]: Oh, yeah.Vibhu [00:36:18]: Shopping data, right?Joon [00:36:18]: Shopping data. So Amazon data is interesting in that it's very much behavioral, although, like, what people do on social media, you could squint and say that is also behavioral. But the transaction data is always interesting. It is also most commonly available, however.Joon [00:36:33]: If we were to look at purely social media, like if you really, if I were, if I had to really pick, Facebook likely is interesting because I do think it is most a default version of people. Because you go to LinkedIn, it's very much professional environment. So people put up their, they have their guards up, right? And that still is interesting because that is true human attitude and behavior, but it is not your base state. you go to Twitter- Twitter, people have their own crazy personas, or depending on who you are. Like, my Twitter profile and, persona is very much, initially was I was very much an academic. “Hey, I'm here to share my studies.” Now, I share, things that's related to Simile. But Facebook is one of those more private space where people just connect with their friends. In that way, I do think it shows you a little bit more about who that person is. So if I had to pick, I'd likely pick, Facebook.Swyx [00:37:30]: Yeah. And you're interested in, like, the whole person and their background and philosophy. I, is it too clinical or too machine learning-oriented to just say this is just ways to inject variance and biases? The broad question, is, like, is this any better than a randomized, like, combinatorial explosion version? So we have a link to the TencentBillion Personas, Synthetic Demographics, and Bespoke DataSwyx [00:37:54]: Billion persona paper, where they did not do any of the groundwork that you are doing.Swyx [00:37:59]: They just did like a cross matrix of here's all the professions in the world, here's all the people, possible backgrounds in the world, do a dot product across all of them, and that's it. That's your prompt for a billion people.Swyx [00:38:12]: This will do something. I don't know if it'll do what you do, but it gets you some way, some percent of the way there.Joon [00:38:18]: So this was an interesting paper. Like, what I admired about this paper when it came out was the scale. And you do gradually want to be able to simulate really large societies and interactions. So the scale is definitely admirable. it is relying heavily on the known statistics that went into training the model. So to the extent that you believe that statistics is correct, this is not a bad way to go about this. But the thesis here, and this is something that we also have seen in the market, like if this works, then we have solved simulation.Joon [00:38:54]: It,Swyx [00:38:55]: Because I survey, like, okay, 5% of the US population is in construction.Swyx [00:39:01]: The other 5% is in medicine, whatever, right? And then you just keep going down the list, and then you do the other side. 5% has, like, the big 5 personalitySwyx [00:39:08]: Of, like, neurotic or whatever. That's it.Joon [00:39:11]: That's it. So if you believe that the underlying data set and the platform that we're leveraging has all the right statistics, then this will have solved it. you're at that point merely retrieving the knowledge that is already embedded in the model, in the model parameters. That's not, unfortunately, what we see, where there is such detailed and also niche knowledge about people that if you just take one example, it might feel very mundane, but it's quite rich when you put together, that you do need to do a lot of bespoke data collection to better understand people. And this is also, I think what makes this particular, job fun, which you want to deeply understand people, and the process of deeply understanding them requires a lot of attention to the details. And you do need to pay attention to and pay respect to the daily lives that people lead.Scaling Simulation: From Thousands to SocietiesVibhu [00:40:04]: I wanna talk about scaling simulation.Vibhu [00:40:07]: So what can't we simulate, what can we simulate, and how does scaling affect this? So how big are the models? What if we go from, 8B, like, couple 100 billionVibhu [00:40:18]: Like billion000 parameters, billion000? Do we get scaling? Any interesting emergence? Like, at a certain scale, at a certain amount of training, you uncover anything unusual and any learnings from that?Joon [00:40:31]: What we are seeing is at Simile, so we do post-train our own model. The thing that we're seeing is the early glimpse of scaling law in simulations. The more data about humans and more compute you ingest, you start to get predictive and predictable gains of the model performance in simulating it, simulating people.Vibhu [00:40:51]: Ooh. We need a scaling law curve.Joon [00:40:52]: It's scaling law. Whenever you find it's a beautiful thing. And we're starting to see the glimpse of it, which is quite exciting. But if you talk about the ambition of simulation as a whole, it's not merely about building a model. It's about building a model, then creating the agents that become the individuals in a much larger ecosystem. So they're creating this multi-agent simulation. Down the line, you want these multi-agent simulation to also live in a very rich environment, right? What we are really trying to get to at that point is, hey, can we create. All right, let's do a time machine game again, and 5 years, 10 years into the future, can we create a simulation of 8 billion people living on Earth? I think that's quite interesting. And that really is the vision. And once you get to that state, the questions that you can help answer for the society also start to change from my perspective. The answers are fundamentally about emergence of the emergent behavior of society and large groups of people.Joon [00:41:53]: So the questions that I get excited by, and maybe this is a stodgy- a bit. I have my, academic side of me.Joon [00:42:01]: And for me, it's questions like, can we help solve climate change? If you look at climate change as a problem space, this is what we, like social scientists would often call it the wicked problems, problem where you have many actors with competing incentives for trying to make a very complex decision and coordinating that coordination decision. Very difficult to really solve in real life, which is also the reason why we couldn't solve it. Can simulation help us solve that? Another one is, can we understand the signals for collapsing democracy, or can we understand or can we uncover the origin story of the monetary system? These are societal questions that we never really had a good way of answering. If we can create simulations of our society, you have to believe that these are the problems that we can solve. So that's really the ambition of this field. And, I also think, yes, I think there's a Nobel Prize to be won there, which wouldn't be surprising. And I think there's some amazing societal impact that we can have to help people make better decisions.Climate Change, Democracy, and Societal SimulationSwyx [00:43:04]: Nobel Prize in economics?Joon [00:43:06]: In economics.Swyx [00:43:06]: Oh, I see. I see. Rooting for you to write that paper.Joon [00:43:10]: One of these days. But, one of the scholars that I was deeply inspired by, When I was coming into the space of simulation, is this scholar, named Thomas Schelling.Schelling, Agent-Based Models, and the Nobel PrizeSwyx [00:43:23]: Schelling point?Joon [00:43:24]: So the canonical example of the work that he's done was he was one of the creators of agent-based modeling. So this was, like, in the 1970s and 80s. It's very early days, but this was truly one of the first exemplars of simulations. And one of the canonical model from that time, and of course many of these simulations are trying to tackle the societal problems that's most relevant for their era, it was called the model of segregation. So racial segregation was a big topic, that, we cared about. And what they've done was they created this grid world where they had red dots and blue dots. And these dots were, back in the day, like, they were the agents, and they had a simple rule that governed their behavior. If certain percentage of your neighbors are of different color and if that goes above certain threshold, then you move to a new location at random.Joon [00:44:21]: One of the striking finding of this paper or this agent-based model was for the longest time, people thought the segregation within society was caused by explicit and overt racism.Joon [00:44:34]: But if you look at this model, people's preference towards living with people of the same color, that preference can be very minute.Joon [00:44:42]: But the very small difference causes the society to segregate completely over time. This was very counterintuitive for a lot of people. And this particular work ended up informing housing policies. Mixed income housing, got really inspired by this work. And Thomas Schelling ends up winning the Nobel Prize for having laid the groundwork for very early versions of simulations. The opportunity that I do see here in the more scientific terms, is agent-based models for the longest, had impact in the 1980s, 90s, to some extent, early 2000s, but it has now gotten forgotten by the community a little bit. Because as you can imagine, red dots and blue dots is not really a rich description of people.Joon [00:45:31]: But with the emergence of things like generative AI and, in particular, generative agents, we do have an opportunity to create these agent-based models that are high fidelity enough to help us make really complex decisions. And that's the opportunity that I see. If that truly works, then yes, that is the work that will result in a Nobel Prize.Swyx [00:45:53]: Yeah. For what it's worth, and I grew up in Singapore. 80% of Singapore is in public housing, and public housing has, enforced racial quotas for exactly that reason, which is very interesting. okay, so we talk about scaling, we talk about all these, the agent possible applications.Cost, Reuse, and the Economics of SimulationSwyx [00:46:13]: I'm scared about the cost. if you even-- let's just keep it to the US, about 8 billion people.Swyx [00:46:21]: But, how much does it cost to model so many hundreds of millions of people?Joon [00:46:26]: Oftentimes today, we don't start at that scale, this stage of the, of industry and simulation as technology. But we can get our users extremely rich and meaningful insights even by modeling thousands, tens of thousands of people. And today what we do is every week we are collecting data on the scale of tens of thousands people's data, and we have panel partnerships that gets us to tens of millions of people globally. So that's what we do today.Swyx [00:46:55]: And just as a side note once you've collected one person for one studySwyx [00:46:59]: Can you reuse that same person for all the subsequent studies?Joon [00:47:03]: That's exactly right.Swyx [00:47:03]: Okay.Joon [00:47:04]: The beauty of this model and these agents is the fact that they are domain-agnostic.Joon [00:47:08]: That what you're really trying to understand is what is the fundamental nature of these people? What's their social physics? And there are a lot of, a lot of, people that does change over time. Like, even, like, even things like, how many times have you gone have you been to, like, CVS the past week? that will change. But there's so many traits about people that are also known to never change. Like, your risk tolerance doesn't really change over time. It's very consistent. So it's these things that we're trying to learn. But the scale we are operating is right now hundreds or, tens of thousands to hundreds of thousands. And in many of the core use cases that we are deployed in, and this is more than enough population, to cover those. Really, at that point, what you care about is less the number of people, but more do you have the right subpopulation of interest covered? And this is also the reason why people want a larger sample. It's not because they want, stronger statistical guarantees. It's more that can they filter down to any population of their interest. However, you can also imagine in 10 years, if we truly believe that the compute is going to scale, that we'll have much more availability for compute, and our ambition for simulation is also going to scale accordingly, there's definitely a reason for us to create an entire data center worth of simulations.Joon [00:48:35]: Or in my hunch here is I do think in the next some number of years, we will start creating simulations that will cost as much as training a foundation model. But perhaps it's going to be so valuable to the society that it would be a no-brainer. Right now, even today, like, we are training bunch of new foundation model just so we can say we trained one and we spent tens of millions. But if we can create a simulation at the level of society that would solve climate change, I would run that today. I would raise the money right now just to run that.Multi-Agent Simulation and Social InfluenceSwyx [00:49:10]: Amazing. the follow-up question is, does it also compound if you let the simulations talk to each other?Swyx [00:49:18]: Or do they already do that today? They don't, right, as far as I understand?Joon [00:49:22]: It depends on what simulation you're trying to run.Joon [00:49:24]: In the multi-agent simulation setup, the agents do talk to each other.Swyx [00:49:28]: Right, which is exactly Smallville, right?Joon [00:49:29]: That's right.Swyx [00:49:30]: But a lot of times, for example, in commerce, you're just by yourself, so there's no point talking. which is way cheaper.Vibhu [00:49:37]: But they use all these levels, right? Like, you decide what you will buy based on what other people around you buy and talk about, right?Swyx [00:49:43]: It depends.Vibhu [00:49:44]: It depends.Swyx [00:49:45]: Again, I'm, I'm coming at this from a cost point of view. I'm like, “Oh my God.” LikeVibhu [00:49:48]: I thinkSwyx [00:49:49]: If there is, like, some combinatorial thing of, like, thousands of people talking to thousands of people, then that one million X's might cost.Vibhu [00:49:56]: I have a very different view as the cost point aside. Like, running these studies in reality is a lot more expensive, right? Running any study like this is you gotta have people do it, you gotta sign people up. It's very expensive and sometimes, like, not feasible to run the study.Vibhu [00:50:14]: But the outcome or the decisions you make are very expensive on them, right? So spend X million on something that, the overall process costs 100 million might as well, right? There's, there's a lot of value to be had there. It's a small cost, but I'm excited on the cost side.Joon [00:50:33]: To some extent, and when you deploy technology, you often want to deploy in a way where you can replace existing budget or you can make things more efficient, and that is the best way to deploy. However, the way you capture the long-term value of the technology is making the argument that, no, it's the upside, that by making this better decision using simulation, you have saved yourself or made yourself hundreds of millions or even billions of dollars, and that's a case to be made.Vibhu [00:51:06]: Random tangent question. So if you're doing a lot of inference, a lot of model multi-agent stuff, are you at the point where it makes sense to, train a model that' very sparse? You're expecting to do multi-million dollar runs. Are you thinking about this in model architecture standpoint or inference efficiency, or, you're still at the research phase of it works, we're not super there yet?Joon [00:51:34]: Efficiency, we do think quite a bit about. this is technology that is deployed now in some of the largest enterprise companies in the world, and we do process significant number of queries, that are trying to, simulate the populations in the world. So efficiency is a consistent thing. we don't want to over-optimize too early, so I wouldn't say, like, this is the higher bid Right now, but this is definitely something that we think pretty carefully about.Swyx [00:52:05]: Yeah. Are there other case studies? So we, you talked about CVS, talked about Gallup, Deloitte, Wealthfront.Efficiency, Enterprise Use, and Real-World Case StudiesJoon [00:52:12]: Wealthfront is an interesting one, because one of the things they were trying to do, they were one of the first customers that wanted to do product testing that goes beyond just asking people what they think about, let's say, behavior experiments and so forth. So there, really what we had to do was reason about multimodal input, so images, but also you can also imagine, like, these agents traversing through Figma mockups or websites. So some of the things that our agents can also do is it can be given a domain, like, or, like, a website URL and go use it for a while. It's these things. And Wealthfront was one of the first, customers, that was very excited about this possibility.Vibhu [00:52:53]: What have people been asking? Like, is there any demand that we have not covered? Like, UI testing, right?Vibhu [00:52:59]: I wanna try a new. I wanna ship a new feature, test the UI, simulate how people will do it. Any interesting things that you're seeing demand for?Product Testing, Websites, and Synthetic PanelsJoon [00:53:08]: Today, a lot of the demand does come from like, the places where people have historically used human panels, we can now replace with agents, and these synthetic populations. And this is not replacing human panel. in many ways, the simulation that Simile is building is grounded. So the way that I think about this is we are trying to represent humanity at scale. And in that way, the use cases are what we would expect, but it's the scale of deployment that surprises me.Joon [00:53:44]: Turns out there are so many decisions that people make every day in these organizations, groups, and we want to be able to say, “We listen to people. We have consulted our users.” But in reality, that is rarely the case because getting to people and asking them many questions, it's difficult. It's both costly, time-consuming, but most importantly, people are just not available. If I had to answer 1000 survey questions for this one particular, vendor, even if I wanted to do that, like, I would never do it. And that's very much the case. What simulation can do is ensure that the voices of people are always represented in rooms where the decisions for them is made, right? So all the stakeholders of this particular product launch, ideally they're consulted. That's what this technology really is trying to enable.Market Size, TAM, and Human Decision-MakingSwyx [00:54:39]: In my mind, that means it skews towards more consumer focus, right? Like, anything with a wide enough customer base where you do benefit from the diversity that you represent. What are some rough statistics, just for people who are not familiar with this market in general, what's the market size that. I'm sure you have some, like, rough numbers. market size is, like, a vague questionSwyx [00:55:01]: But, like, how much do people spend?Joon [00:55:03]: So market research is a $100 billion industry.Joon [00:55:06]: But the thing about simulation is not a tool for market research. Simulation is a tool for human decision-making. So the question around what is a TAM here is quite tricky, right? Because it's easy to say, “Well, market research TAM is roughly 100 million or 100 billion.” so is it a TAM? And not really, right? Because in many ways, you're trying to inform all human decision-making. You're trying to inform every decision that are made about humans for humans. What is a TAM for that? It's really unclear. And I'll be honest. Like, I have a scientific background, I have a research background, so I didn't come into the field calculating, oh, what is the TAM for human decision-making? But I just had to assume, well, if we can inform every decision that is made about human for human, that has to be big.Swyx [00:55:58]: Some- something valuable.Joon [00:55:59]: Exactly.Swyx [00:55:59]: To some extent, you are a unicorn founder now, and you have to care as a CEO. But, like, I do think, like, yeah, when you go into these boardrooms with people that you're quoting millions of dollars of contracts for, like, you have to say, “Well, here's what you spend on humans-”Swyx [00:56:15]: “. And here's what we save you, and it's 85% similar.”Joon [00:56:19]: And certainly, the value case, is something that we care deeply about. Like, what is the value that we provide to the users and the decision-makers? But this is also where, like, as a founder, I think valuation only tells one very superficial aspect of the story, and I try not to think too much about valuation, in general, because that's not what also motivates a team or certainly doesn't. I'm, I-- Again, the interesting thing about researchers is we are happy living in academia, getting paid next to. we get paid okay. we don't get paid that much, as a researcher here in academia, but it's the impact and it's the, it's the value that we can provide to the individuals and the society that really drives us. And in that way, ultimately what drives us is the impact. Does the simulation we provide have a real impact in people's decision-making in ways that progresses our society forward? If the answer is yes, then yes. that has to be great business, and we see that in numbers, and we do care deeply about that upside story, but that's the heart of it.Where Simulation Goes NextVibhu [00:57:27]: Do you have any timeline predictions? So we talked about scaling laws of simulations.Vibhu [00:57:33]: You brought up, okay, maybe one day we can simulate how to solve climate change.Vibhu [00:57:38]: Where are we now?Vibhu [00:57:40]: If that's not the end state, what is an end state, and what does progress look like?Joon [00:57:45]: So what I sometimes tell people is simulation as industry, it feels a lot like where GPT-3.5, GPT-4 was, for the AGI saga, which is we have now technology that is powerful enough to do real damage on the verticals that we are tackling. At the same time, there's a lot of progress that is yet to come. And that's, I think, where this is. So the way I see it, I do think there will continue to be breakthroughs both in data, in algorithms, and there will be much more aggressive scaling that will also happen over the next few years. But I think that's roughly where we are.Swyx [00:58:27]: I think that was about the ro

Karolina Wydra joins Son of a Binge host Reshma Gopaldas to talk all about her triumphant return to television in Vince Gilligan's Pluribus, also starring Rhea Seehorn. Wydra began her career as a model, then went on to appear in various television shows and films. She hung with the slutty vampires in True Blood, rolled in the medical world in House, flirted with Ryan Gosling in Crazy, Stupid, Love, and snuck around with Breaking Bad's Bryan Cranston in Sneaky Pete.In 2020, the with everyone being stuck at home, Wydra decided it would be a good time to start a family. Her agent and manager promptly dropped her. Oof, wait to support women Hollywood. After her Hollywood break... up, Wydra went on to have two boys. Then she got a call. Or rather an email. The team behind Vince Gilligan's Pluribus wanted her to audition. While on vacation with her good friend, Caitríona Balfe (Outlander, Sense & Sensibility), she found out they wanted to screen test her. With no agent or manager, Wydra panicked, but Balfe stepped in to help. Wydra got the role of Zosia, and it's hard to imagine anyone else playing her. She opens up about what it's like working with Gilligan and Rhea Seehorn. She breaks down their characters' relationship, first kiss, and what it's really like working with Seehorn. She also talks about the female friendships that have sustained her in both the modeling and acting industries.Wydra was just nominated for an Emmy for Supporting Actress in a Drama series for her role as Zosia. Pluribus is streaming now on Apple TV. Subscribe to Reshma Gopaldas on YouTube for video episodes. Karolina's video episode will drop on August 18, 2026.Son of a Binge production credits:Hosted by: Reshma Gopaldas (TW: @reshingbull, IG @reshmago)Artwork by: Laura Valencia (IG @iamlauravalencia)Music by: Kevin Calaba (IG @airlandsmusic)Send us a text, let us know what shows and guests you want us to cover.Support the show

Arizona State Troopers pulled over a motorist in Flagstaff and noticed they showed "obvious signs of impairment." Ironically, the driver's vanity plates said "Arrive Alive" in a frame that read "Drive Sober: Or Get Pulled Over." Oof.See omnystudio.com/listener for privacy information.

Protecting your attention is not the same as not caring; it's what keeps the work you love possible. Lesley Logan opens August from the road with an honest look at the fighting that keeps breaking out in the Pilates industry, and why she keeps it out of her feed on purpose. She explains what gets lost when instructors tear each other down instead of pointing clients toward each other's strengths. She also shares the response to her teacher training announcement, a listener win over procrastination, and this week's mantra. If you have any questions about this episode or want to get some of the resources we mentioned, head over to LesleyLogan.co/podcast https://lesleylogan.co/podcast/. If you have any comments or questions about the Be It pod shoot us a message at beit@lesleylogan.co mailto:beit@lesleylogan.co. And as always, if you're enjoying the show please share it with someone who you think would enjoy it as well. It is your continued support that will help us continue to help others. Thank you so much! Never miss another show by subscribing at LesleyLogan.co/subscribe https://lesleylogan.co/podcast/#follow-subscribe-free.In this episode you will learn about:Why a curated algorithm protects the work Lesley came here to do.The industry hoopla that resurfaces every couple of years, and who profits.A teacher training announcement Lesley made without expecting the response.One listener win about procrastination, plus this week's closing mantra.Episode References/Links:UpLift Waitlist - xxll.co/uwsOPC Summer Tour - https://opc.me/eventseLevate Mentorship Program - https://lesleylogan.co/elevateSubmit your wins or questions - https://beitpod.com/questions If you enjoyed this episode, make sure and give us a five star rating and leave us a review on iTunes, Podcast Addict, Podchaser or Castbox. https://lovethepodcast.com/BITYSIDEALS! DEALS! DEALS! DEALS! https://onlinepilatesclasses.com/memberships/perks/#equipmentCheck out all our Preferred Vendors & Special Deals from Clair Sparrow, Sensate, Lyfefuel BeeKeeper's Naturals, Sauna Space, HigherDose, AG1 and ToeSox https://onlinepilatesclasses.com/memberships/perks/#equipmentBe in the know with all the workshops at OPC https://workshops.onlinepilatesclasses.com/lp-workshop-waitlistBe It Till You See It Podcast Survey https://pod.lesleylogan.co/be-it-podcasts-surveyBe a part of Lesley's Pilates Mentorship https://lesleylogan.co/elevate/FREE Ditching Busy Webinar https://ditchingbusy.com/Resources:Watch the Be It Till You See It podcast on YouTube! https://www.youtube.com/channel/UCq08HES7xLMvVa3Fy5DR8-gLesley Logan website https://lesleylogan.co/Be It Till You See It Podcast https://lesleylogan.co/podcast/Online Pilates Classes by Lesley Logan https://onlinepilatesclasses.com/Online Pilates Classes by Lesley Logan on YouTube https://www.youtube.com/channel/UCjogqXLnfyhS5VlU4rdzlnQProfitable Pilates https://profitablepilates.com/about/Follow Us on Social Media:Instagram https://www.instagram.com/lesley.logan/The Be It Till You See It Podcast YouTube channel https://www.youtube.com/channel/UCq08HES7xLMvVa3Fy5DR8-gFacebook https://www.facebook.com/llogan.pilatesLinkedIn https://www.linkedin.com/in/lesley-logan/The OPC YouTube Channel https://www.youtube.com/@OnlinePilatesClasses Episode Transcript:Lesley Logan 0:00  It's Fuck Yeah Friday.Brad Crowell 0:01  Fuck yeah!Lesley Logan 0:02  Get ready for some wins. Welcome to the Be It Till You See It podcast, where we talk about taking messy action, knowing that perfect is boring. I'm Lesley Logan, Pilates instructor and fitness business coach. I've trained 1000s of people around the world, and the number one thing I see stopping people from achieving anything is self-doubt. My friends, action brings clarity, and it's the antidote to fear. Each week, my guests will bring bold, executable, intrinsic, and targeted steps that you can use to put yourself first and be it till you see it. It's a practice, not a perfect. Let's get started.Lesley Logan 0:48  Well, hello, BE IT babe. How are you? Happy Friday! Oh my god, it is August. Oof! I'm on tour right now. And if you're new to this podcast, hi! On Fridays, we keep it short. We keep it sweet. It's a way for those who listen to feel like we have some time together, and I get to share wins of yours. You get to hear some wins of mine, and you also get to, I don't know, hopefully feel like you're not alone in some things. So, if you are in our communities for eLevate or for our Agency, we have an "I Need a Moment," and you're allowed to have a moment, but you have to have a win if you have a moment. And I am very notorious for coming in after you've poured your whole heart out and no win was posted. I'm like, "You know, where's the win?" And it's not because we're looking for toxic positivity, but it's because we are trying to see that even in the muck there is a lotus flower.Lesley Logan 1:35  So, at the time that I'm recording this, there's this big thing going on that I continue to be made aware of, even though I changed my algorithm a long, long, long time ago to not see a lot of Pilates stuff. So if that shocks you, it's not because I don't care. It's because I care so much that when I see negative crap on the internet, it just makes me sad. It makes me not actually want to do the thing that I am here to do on this planet, and so I just don't. I don't deal with it. And I have some really great friends who are also in the industry who also have their algorithms set that way. So if you're seeing that me and some of these people are not participating in this debate or didn't participate in the stuff going on last month, it's not because we don't care. It's because we intentionally don't let that stuff in because it would keep us from loving what we do. And so the thing is, I'm actually just so tired. I've been teaching Pilates for almost 20 years, and every couple of years there's this big hoopla of how not good enough some people are, and how much of assholes other people are, and blah, blah, blah. And then we act like it's only in our industry that this happens. Bullshit happens everywhere. People tear each other apart all the fucking time. You all know something? This is why a bunch of stock market bros own a bunch of Pilates studios, because we're all busy tearing each other down instead of focusing on what we're really great at and then making sure that we know who's really great at things we're not, and then sharing that with other people. That's what I have to say about that. So I'm just tired of it. I'm so over it. And if you're, you know, this is where I'm voicing myself.Lesley Logan 3:07  But my win, since I'm not going to leave you with that, if you're like, "What is going on in the Pilates industry?" It doesn't fucking matter. It doesn't matter what it is because in two years it'll be happening again, something similar and not the same at the same time. But the reality is people want us to fight with each other because then we are not busy focusing on making the impact we want to make, which would actually change people's lives and also yours at the same time.Lesley Logan 3:31  So my win is that last month I announced my teacher training program on UpLift publicly. I announced it on Instagram and it got so much love. Like even with all the shit that was going on in the Plaza industry, like my post about my teacher training program got so much love. So not a single person going, "Who the fuck do you think you are? It's only people saying, "Like I'm so glad you're doing this. Finally, a program I can refer people to. I want in on this. I want to learn this. I want to do it. And it just made me so happy because that's not at all what I expected. I kind of was just making the announcement to make the announcement. To be completely honest, I did not expect hundreds of people to go on the wait list. So I'm just well pleased as punch. I really, really am.Lesley Logan 4:15  Okay. So, oh, if you want information on it, you can get on the waitlist at xxll.co/uws. If you have any problems with that, there is an internet provider that can suck it because they are assholes. It's this one internet provider that drives me crazy when it comes to my short links. And so, if you have the internet provider, hit me up. I have a special link just for you. It's just not very pretty.Lesley Logan 4:38  All right, let's get into your win. Your win first one up is gonna be from @hopesewell, " Marked some things off my to-do list that I've been procrastinating on." That is massive. That is so massive. When I knock something off a to-do list that I've been procrastinating on, I am like the most proud of myself. And also, sometimes because of my ADHD, I realize, well that didn't take very long at all. It was on my wait list longer than the half life that it took me to do it. So I feel you, Hope. Thank you so much. I know you're a big OPC fan, and I get to see you on the summer tour. I'm so, so excited.Lesley Logan 5:11  So thank you for sending your win in. You guys can send your wins into beitpod.com/questions. That's also where you can send in any topics you want us to do our solo episodes on or guests you want us to feature. This podcast is here for you. It's not just for hearing me talk to myself. I talk to myself all the time. I don't need another outlet. So definitely send in what you need.Lesley Logan 5:30  And your mantra before I let you go is: I honor and cherish my life. I honor and cherish my life. I honor and cherish my life. Be It babe, I hope you do. That's the only way you could be it till you see it, anyways. Honor and cherish it. All right. Until next time, Be It Till You See It. Lesley Logan 5:48  That's all I got for this episode of the Be It Till You See It Podcast. One thing that would help both myself and future listeners is for you to rate the show and leave a review and follow or subscribe for free wherever you listen to your podcast. Also, make sure to introduce yourself over at the Be It Pod on Instagram. I would love to know more about you. Share this episode with whoever you think needs to hear it. Help us and others Be It Till You See It. Have an awesome day. Be It Till You See It is a production of The Bloom Podcast Network. If you want to leave us a message or a question that we might read on another episode, you can text us at +1-310-905-5534 or send a DM on Instagram @BeItPod.Brad Crowell 6:31  It's written, filmed, and recorded by your host, Lesley Logan, and me, Brad Crowell.Lesley Logan 6:35  It is transcribed, produced and edited by the epic team at Disenyo.co.Brad Crowell 6:40  Our theme music is by Ali at Apex Production Music and our branding by designer and artist, Gianfranco Cioffi.Lesley Logan 6:47  Special thanks to Melissa Solomon for creating our visuals.Brad Crowell 6:50  Also to Angelina Herico for adding all of our content to our website. And finally to Meridith Root for keeping us all on point and on time.Advertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy

Watch the full episode on YouTube:We first covered Baseten last year when DeepSeek mania was at peak hype. Now they have raised a monster $13B round and become one of the new cohort of AI Infra decacorns that are (with Nvidia, Intel, and the semis complex) chief beneficiaries of the Inference Inflection. We return to Baseten at the peak of the 2026 edition of Open Weights debate. Ali has published a viral breakdown of Kimi K3:And since you last saw him, Philip has spoken at AI Engineer and written the definitive book on Inference Engineering spotted all over SF:Three years ago, inference engineering barely existed as a category.Today, it is one of the most critical disciplines in AI. Inference engineering inherently tackles a different question than standard model training: “How do you turn those weights from training into a product that is fast, reliable, and affordable at scale?” Focusing on these creates an entirely new optimization problem.In one recent GLM-5.2 experiment, quantizing more of the model actually preserved its benchmark quality while increasing throughput by 20%, because the errors introduced in different layers could cancel each other out.Inference is no longer just the final step after training. It is becoming its own engineering discipline, with its own research problems, infrastructure, and increasingly specialized roles.In this episode, Baseten's Philip Kiely and Ali Taha join swyx and Vibhu to explain what actually happens after a new open model is released and what it takes to turn “we generated a token” into a fast, reliable, production-ready API.We go deep on cache-aware routing, disaggregated prefill and decode, quantization, speculative decoding, KV-cache movement, model parallelism, GPU kernels, and the race to make frontier models up to 10× faster. Philip and Ali explain why inference optimizations can still produce gains of 20%, 100%, or even 200%; how quantization errors can cancel one another out; why identical weights can behave differently across clusters; and how Baseten grafted a Kimi vision encoder onto GLM-5.2 without changing the underlying language model.The conversation then expands beyond LLMs into NVIDIA Dynamo, mega kernels, Rubin, AI-specific chips, local inference, video generation, diffusion versus autoregressive models, and the enormous compute barrier to generating coherent long-form video. Finally, we explore the convergence of training and inference, continual learning through persistent KV cache, and the emerging loop where models help optimize the infrastructure that runs them.We discuss:* What happens when a 200,000-token request enters an inference system* Cache-aware routing and reusing previously computed KV cache* Why prefill and decode are increasingly handled by different GPUs* When dedicated deployments become cheaper and more reliable than shared APIs* How speculative decoding uses a smaller model to accelerate a larger one* Tool calling, structured outputs, and what LLMs actually do* What it takes to support a new open model on day zero* Grafting Kimi's vision encoder onto GLM-5.2* Retrofitting inefficient model layers with components from other architectures* Why models sometimes collapse into repeating the same token* How hardware, kernels, and race conditions create nondeterministic failures* Preserving model fidelity while making inference faster* How quantization errors can cancel each other out* Why inference optimizations still deliver gains of 20%, 100%, and 200%* How optimized serving can make a model up to 10× faster* NVIDIA Dynamo, KV-aware routing, and distributed model serving* Speculative decoding the speculative decoder* Why local AI is about making models less dumb while data-center AI is about making them less slow* Tensor, expert, and pipeline parallelism across GPUs* Hardware-aware model design, auto-tuning, and the case against mega kernels* Rubin and why inference is becoming a systems problem* Whether modern GPUs are evolving into programmable AI ASICs* Why enormous models like Kimi K3 require GB300-class hardware* Why open-source video generation still trails Veo, Kling, and other closed models* The quadratic attention bottleneck behind long-form AI video* Autoregressive video, real-time generation, and compounding quality drift* Why future video systems may combine autoregressive and diffusion architectures* Training for inference and inference for training* Continuous post-training, deployment, evaluation, and improvement loops* How GLM-5.2 helped optimize the kernels serving GLM-5.2 itself* Why faster networking could unlock dramatically faster decoding* Continual learning, KV-cache compaction, and persistent model memoryShow Notes* How to build a day-0 API for Kimi K3* 22580: From GPT2 to Kimi3, ExplainedPhilip Kiely* LinkedIn: https://www.linkedin.com/in/philipkiely* X: https://x.com/philipkiely* Inference Engineering: https://www.baseten.co/inference-engineering/Ali Taha* LinkedIn: https://www.linkedin.com/in/aliestaha/* X: https://x.com/waterloointernTimestamps00:00:00 Introduction and the 200K-Token Prompt00:03:18 Dedicated Deployments, Speculative Decoding, and Tool Calling00:11:26 Launching Production-Ready Open Models00:19:06 Model Retrofits, Failure Modes, and Nondeterminism00:28:22 Quantization and Canceling Errors00:32:15 The Race to 10× Faster Inference00:40:48 Dynamo, Speculation, and Local vs. Data-Center AI00:50:18 Model Parallelism, Auto-Tuning, and Mega Kernels01:00:55 Rubin, GPUs vs. ASICs, and Custom AI Chips01:10:03 Giant Models and the Limits of GPU Memory01:12:42 AI Video, Quadratic Attention, and Autoregressive Generation01:21:47 Audio, Images, and Diffusion Models01:27:32 Training, Self-Optimizing Models, and Continual Learning01:40:06 Closing ThoughtsTranscriptIntroduction: Baseten, Waterloo Intern, and Inference EngineeringSwyx [00:00:00]: Okay, we're here in the studio with Philip, old friend from Inference Engineering, the book, as well as Baseten and everything that you've done, you and I have done before, as well as Ali. Welcome.Ali [00:00:15]: Pleasure to meet you.Swyx [00:00:15]: Waterloo intern.Ali [00:00:16]: Waterloo intern, always.Swyx [00:00:17]: When did you get “Waterloo intern” as a handle?Ali [00:00:19]: As a handle? Oh.Ali [00:00:20]: I think the rebranding happened mid-March. When I saw it was open, I was like, “I have to take it. Up for grabs.”Philip [00:00:26]: The problem is that Ali is really good at his job and is not gonna be an intern much longer.Philip [00:00:30]: So we have to figure out who's gonna get the handle.Ali [00:00:33]: Well, I'll pass the torch over to the next intern.Swyx [00:00:34]: Oh, okay. It can be, like, you just pass it to another Waterloo grad.Ali [00:00:37]: To another Waterloo intern. No, bruh.Philip [00:00:39]: Yeah.Ali [00:00:39]: Intern.Swyx [00:00:40]: Intern, yeah.Ali [00:00:40]: And no.Philip [00:00:41]: You gotta get an intern from Waterloo.Ali [00:00:42]: Yeah, I've gotta get an intern from Waterloo.Swyx [00:00:44]: Right.Ali [00:00:44]: But they have to follow the path.Swyx [00:00:45]: Oh, it could, but it could come from Baseten, so it's like whoever Baseten gets from Waterloo.Ali [00:00:48]: Right.Swyx [00:00:49]: Has the title of Waterloo.Ali [00:00:50]: It stays in the ecosystem.Philip [00:00:51]: Exactly.Ali [00:00:52]: Halfway through the internship, you either get it or you're out.Philip [00:00:55]: You should also do, like, a big graduation ceremony where you change the handle.Ali [00:00:59]: Just say it.Philip [00:00:59]: For everybody.Swyx [00:01:00]: You guys are good at ceremonies, clearly. We had a nice launch of the book, very successful. But before we get into all that, I wanna start off with a fun question for you. Okay, you're an expert inference engineer. What happens when I send a long query, say two hundred thousand tokens into Baseten's inference? What's the process of query through GPU model routing, balancing, all that? What is all the stuff that we don't think about?Long Context Requests, KV Cache, and Cache-Aware RoutingPhilip [00:01:26]: With a long query specifically, the first thing that I'm gonna ask is, “Have you sent me this query before, or at least part of it?” and I really hope you have, because it's gonna be a lot easier for me and a lot cheaper for you. So the first thing that we're gonna look at is some cache-aware routing, where we're going to see, we probably have a number of instances, a number of replicas up serving whatever model you're hitting. We want to send this one to something with, number one, available prefill workers, and number two, ideally some cached input already there so that we can skip prefill on at least part of these two hundred thousand tokens. If you're doing two hundred thousand tokens, it's probably coding or a multi-turn agent or something where you would expect to have that cached. If you don't, we're gonna have to send it to a prefill worker. We've at least on certain models disaggregated prefill and decode, so you're going to have one set of GPUs that's solely going to process the input, create the KV cache, and get you your first token, and then that's going to be passed over to a separate set of GPUs, which is going to run decode. We're going to iteratively make those tokens. We're probably going to have some speculator model in front of that. I'm going to assume that you're doing coding, and because of that, our speculator model, which assumes you're doing coding, is gonna have a high draft token acceptance rate. If I'm wrong and you're asking me to summarize every Harry Potter book, it's gonna be slower. And then we stream that output to you and account for it, charge you, a couple of pennies and say, “Hey, would you like to send another one?”Swyx [00:03:04]: Except Baseten doesn't charge by pennies.Philip [00:03:07]: Well, yeah, we charge. I'm assuming that we're talking about the public model APIs. If you are setting up a dedicated deployment, then yeah, it's not pennies.Public APIs vs. Dedicated DeploymentsSwyx [00:03:18]: Yeah, one of the key differentiators when I was talking with Baseten initially was that people who want very high volume just need to rent by the box, ‘cause then it's up to you to figure out how to saturate the box.Ali [00:03:31]: And more often than not, it's, like, way cheaper if you're pushing, like, millions of tokens per hour, if you just pay per hour instead of pay per token.Philip [00:03:37]: Yeah, they do. I think that we've increasingly seen a lot of demand for the pay per token APIs, just because everyone wants to try open models, and then once they find a use case that's really sticky, then they move over to dedicated.Swyx [00:03:51]: Is there a best practice on when it's time to swap over?Philip [00:03:54]: Couple reasons. Yeah, reliability, that's a big one, right?Ali [00:03:57]: Like, if they have a very specific use case, they want you to train something specifically for them, like they want their own spec dec, for instance, for their own traffic.Swyx [00:04:04]: Spec dec is speculative decoding.Speculative Decoding and Custom SpeculatorsAli [00:04:05]: Speculative decoding, yeah.Swyx [00:04:07]: You have to explain.Ali [00:04:07]: Sorry. Like, speculative decoding is like, if you have a huge model, right? And so the model is going to be generating one token at a time every single turn, every single forward pass. So we attach, like, this little, like, parasite, like this layer that goes on top of the model, and this model just has to predict. It does three very fast autoregressive forward passes, and it will predict, like, three certain tokens, and then you do one forward stage over the entire original model in order to see if those predictions were correct or not, and then you accept them or you reject them. Now, this draft model is traffic specific, so if you, like, Philip said, if you're summarizing Harry Potter books, I can train exclusively that draft model on Harry Potter books, and I can guarantee you that I'm gonna accept the three tokens every single time. And so with that case, I increase your decode speed. I wouldn't be able to provide this to you if you're a shared endpointSwyx [00:04:53]: YeahAli [00:04:53]: ‘cause I have no idea if you're doing Harry Potter, if you're doing coding, if you're doing English. We don't know. Also, there was a thing in the book that mentioned that if they really cared about a specific threshold, chapter four, I think. Do you remember that?Philip [00:05:06]: Yeah. The things that you can do is you can set a specific, like, batch sizing, a specific, like, parallelism strategy if you're trying to optimize for, like, throughput versus latency. You can. Maybe a NVFP4 quant doesn't pass your benchmarks and you wanna run a model at higher precision, you could do that. There's just a bunch of reasons why you might wanna have your own endpoint and the biggest one, of course, just being, like, you don't have to deal with someone else doing a hundred million tokens of benchmarking traffic at the endpoint when you happen to be trying to serve your users.Swyx [00:05:40]: Yeah. I think one thing that is. That is a classic journey. Like, it's people is asking the, what happens when you type Google into the browser. Tool calling, is that just, you're generating JSON or is there more complication beyond that?Tool Calling, JSON, and Structured OutputsAli [00:05:58]: Certain customers that we have, they have their own post-trained models, and so they demand a tool calling that's not just, like parse a file or go find the weather. It's something that's very specific and you have to do post-training on this. And if the post-training on the model is not good or if the quantization after the post-training to get the inference to be fast, the model will struggle reading the JSON file and reading the tool calling. But it doesn't require its own like sandbox. It's not like it's going to use that tool calling to like escape a sandbox or like it doesn't have to be contained. It can just be a normal dedicated deployment. The challenge with tool calling more and more seems to be that the companies want certain tool calling which is a very sensitive thing to train. And because you're dealing with all of the JSON outputs, if it doesn't like close the end of the request in a very certain manner, you end up with a model that did the tool calling and like the thinking and so as a result of that, it didn't see the result and just hallucinated the result as it decoded. That seems to be the most challenging thing with tool calling, not really the sandboxes model.Philip [00:06:56]: Yeah, that's a challenge on the training side and then on the inference side, there's work that you can do to scope the possible output. So we published this at this point close to two years ago, the solution to this problem which is you make a state machine and you use that to constrain the output to a specific format. So this is the structured output problem. If you remember backSwyx [00:07:27]: Yeah, the specific grammar is,Philip [00:07:29]: Yeah, exactlySwyx [00:07:30]: GML had this thing.Philip [00:07:31]: Yeah. So it's like the old-school “make sure this is only JSON”, return only JSON orSwyx [00:07:38]: YeahPhilip [00:07:38]: Grandma's gonna die type of prompts.Swyx [00:07:39]: Is it BNF grammar? At some point OpenAI had released a thing that was like, yeah, if you want to constrain your output, write BNF grammar, back as NOR.Philip [00:07:47]: In our inference system, it's just a specified output format. And you get the guarantee that your output's gonna be structured along that format. And so applying that to tool calls can like help cut down on. You can still call the wrong tool or call no tool. It doesn't solve the certainty problem but it at least solves the output structuring problemSwyx [00:08:10]: YeahPhilip [00:08:10]: Within tool calls.Swyx [00:08:12]: And MCP is just another form of tool, right.Philip [00:08:14]: Yeah, exactly.Swyx [00:08:15]: As far as there's no special thing there.Philip [00:08:16]: The thing I'm always like explaining to people is the LLM is not capable of doing anything. It's only capable of making suggestions of what to do and then if those suggestions are formatted in a certain way and applied to a system that knows what to do with them, then an action occurs.Swyx [00:08:32]: Yeah. Part of the fun stuff is, this is solved outside of tool calling too. Like in an agent loop if the output is not correct or you're right, like reasoning, tool calling was done in the reasoning trace, just be like, “Oh, I don't know what to do. Let me just try again.” And it might get there after a few tries. And on your point of training, sometimes this is harder in smaller models, so you don't have the same exact quality outputAli [00:08:56]: Right.Swyx [00:08:57]: When you just swap from a big model, right?Ali [00:08:59]: Yeah. I will say that, before, I think we need to go back to inference engineering proper.Ali [00:09:04]: But, I had expected that something would replace JSON because it's hard to stream JSON ‘cause JSON must be complete and you must have open and close brackets and everything. So it's hard to parse something or validate something while it's being streamed. So people invented all sorts of things that are like, I forget the name of some of these alternatives, but it's something like TOML, something like YAML. But JSON seems to be dominant still.Philip [00:09:30]: The JSON outputs aren't that long, right? Like you could have a long-- ‘cause tool calls also contain the arguments in them and perhaps for a certain tool you might pass like a very long argument. But my impression of the median tool call is that it's a relatively small number of tokens, right? So I would expect that speculators are generally fairly good at something as formatted as JSON. And so you would have like a pretty fast decode step there and that the streaming wouldn't be as valuable, but maybe I'm wrong about that.Ali [00:10:02]: I think you're also bounded by the software or that the model is gonna integrate with if the software is built with JSON for the tool calls or if the company that you'- if your customer says that this is how our software works and our tools are interfaced with JSON, you can ask them to like, change their software and say like, “Yeah, this is gonna be better for the model.” but like with the right training shouldn't be that much of a difference. Also more profitable if it outputs more tokens probably.Swyx [00:10:25]: Depends on your business model.Swyx [00:10:27]: It really depends. But I will say that, as a writer with like experience a lot with generated output, I do try to move from text to JSON text which is very long JSON, right? Like there's paragraphs in every field because I'm trying to structure it, right?Philip [00:10:44]: Right.Swyx [00:10:44]: I want you to first make factual statements, then make opinions then make bullet point summaries, have dates, have entity references have your sources for references, all these things. Anyway, so these are things that like I think people who really experiment with structural output have to really care about. But, let's, let's recurse up the stack a little bit. Before we started recording, you mentioned something really cool, which is that there's a lot of engineering that-- inference engineering that goes on when a new model provider releases a new model, right? So let's call it GLM-5.2, Kimi K3. I had previously assumed, especially if it's like, well, GLM 5 to 5.1 to GLM-5.2, like that you've supported them before. Is it that much work?What It Takes to Support a New Open ModelAli [00:11:26]: It's a lot of work.Swyx [00:11:28]: Yeah. Okay. So like, a lot of people, all you guys, right whenever a new model launch like, people rush to say like, “Oh, Hugging Face supports this, Fireworks supports this, Spacetime supports this,” and I'm like, “Yeah, of course we support it.” But what goes into that? What goes intoPhilip [00:11:40]: I think it's more than just support it too, right? It benefits the consumer a lot. Like I think it was with Kimi K2.5 or GLM-5.2 the latest, there was an inference war, right? X provider is at 90 tokens a second. The next day we're at 150. The nextSwyx [00:11:55]: I kinda kicked that off with the GLM-5.2.Swyx [00:11:58]: I wrote a Twitter article about. It got like half a million views,Ali [00:12:02]: Based on being numberSwyx [00:12:03]: YeahAli [00:12:04]: Or it's for something else.Swyx [00:12:05]: Yeah. Which,Ali [00:12:06]: Oh my GodSwyx [00:12:07]: Which then got everyone really excited about, hey, how can we, bend tracks a little bit further and,Philip [00:12:14]: There's a difference between support the model, as in I can make a token out of this model, and support a model, as in I have a production-ready API from this model.Philip [00:12:26]: Getting to the point of I can make a token out of this model is not that hard because generally the, open source inference engines, vLLM, SGLang of the world oftentimes even receive weights ahead of time, maintainers do, or the people making the model merge PRs to ensure support. So you generally can, just get it working on the standard open source stack without too much pain in most cases. The challenge is, every inference company is gonna have own proprietary stack. Some open source components, some in-house stuff. And for any arbitrary model, there's going to be some new stuff. Sometimes you get lucky, like K, two five to two six was, like, pretty similar.Quantization, Speculators, and Production ReadinessAli [00:13:16]: Yeah. It was pure continued post-trainingPhilip [00:13:18]: YeahAli [00:13:18]: If I remember correctly.Philip [00:13:19]: Even in those cases, there's still stuff you have to do. You have to redo the quantization work. You're taking the model from. Generally, these models are not released in NVFP4, and we want them to be in NVFP4 for maximum Blackwell compatibility. So we have to perform that quantization, and, calibrate the quantization to make sure that we're not causing any regression in the model's intelligence. And then we also have to train the speculator, as we've talked about. Generally, we have. We have ZDR, zero data retention on our model APIs, so we don't know exactly the traffic that people are sending us, but we know what's popular. We know that coding use cases are popular. We know that agents, agentic use cases are popular. So we can get public data sets that are representative of that traffic and train general speculators. Now, with speculators today, you need to train the speculator using the base model itself because you're getting hidden states out of the model from running inference on these specific prompts, and that is the training data you use to create the speculator. So there's that process which you need the real model weights for. And then there's of course just the process of, standing up all the infrastructure behind it, loading all this stuff, testing it. And then when there's a new model with a newer architecture, I think that, like, the DeepSeek models tend to be the most challenging as they have, like, the most novel architectural stuff going on, model after model. But every new model has something. Kimi K2 had. Oh, sorry, GLM-5.2 hadAli [00:14:53]: Sparse attention.Philip [00:14:54]: Yeah,Ali [00:14:54]: YeahPhilip [00:14:54]: the DSA.Ali [00:14:55]: Right. Which is brought from DeepSeek.Philip [00:14:57]: Yeah. AndAli [00:14:59]: So you can copy-paste then?Philip [00:15:01]: It kindAli [00:15:01]: I don't know how this works.Philip [00:15:02]: So, like we had to, like, build support for that into our runtime. And you're right, like it is really interesting the way that all of these open source labs borrow from each other. For example, like GLM-5.2 doesn't have vision. So something that, Haley, a guy on our team, if we could take a look at this, he, like, grafted the Kimi vision encoder onto GLM-5.2.Retrofitting Vision into GLM-5.2Ali [00:15:27]: We'll be training the projector.Philip [00:15:28]: Exactly. So if you think about, like, the encoder, there's the encoder, which is the part that looks at the image and turns it into latent information, and then there's the projector which likeAli [00:15:38]: You can say latent space. It's okay.Philip [00:15:41]: And then there's the projector that maps it onto, the model itself, and then there's the model weights. You don't wanna mess with the model weights because you run a chance of making the model dumber at something else for the purpose of giving it vision. So instead, Haley started with just a projector, which is only a handful of millions of parameters.Ali [00:16:02]: That would be, yeah.Philip [00:16:02]: Yeah.Ali [00:16:03]: Can you show the training one?Ali [00:16:04]: Like the way it groksPhilip [00:16:05]: YeahAli [00:16:06]: Very interesting.Philip [00:16:06]: And maybeAli [00:16:07]: That right therePhilip [00:16:07]: Maybe Ali, you should take it from here. You've got a betterAli [00:16:10]: Ooh, double the sandPhilip [00:16:11]: Understanding of this than I do.Ali [00:16:11]: Yeah. You can see, like, he. The way he trained this is really cool. At the beginning, he was training it using just like, “Here's a picture of a mountain. Can you describe what's in this mountain?” And that caused it just like the first, learning walls. Like here you can see this all we're trying to teach it is to translate the encoded. Like it's already taken the encoder from Kimi K. It's taken the image. It'Philip [00:16:31]: Yeah. FrozenAli [00:16:31]: FrozenPhilip [00:16:32]: With adapter.Ali [00:16:32]: Exactly.Philip [00:16:33]: Yeah.Ali [00:16:33]: So the brain is frozen and the eyes are frozen. It's just we're tryingPhilip [00:16:37]: AlignAli [00:16:38]: Interconnect between the eye and the brain, right? So the projector. And so you take the tokens and then he's like, “Oh, can you describe what's in this image?” And he's like, “Oh, it's a mountain,” or it's a person or it's a human, whatever the case is. But that didn't cause complete understanding. So he changed it such that every image was associated with a data set of questions. Like, does this image have a white male? Does this image have birds in the top corner? Does this image have a scientist in it? All of that stuff. And it would have to answer questions correctly. And using not just training on describing an image, but being able to answer question, another question, answer over time. Like you can see the grokking, which is like genuinely insane, that retrofitting vision into a large LLM can learn to that extent. And even for images that it doesn't perform well on, for instance, if you ask it a picture of like Stephen Hawking, “Who is this?” Maybe it doesn't get it, but it will say something like, “This is Albert Einstein.” Like it still understandsPhilip [00:17:25]: Close enoughAli [00:17:26]: That this is a scientist who is a man who has, some significant achievements, all that stuff. So that's like really cool.Philip [00:17:32]: Yeah. So, we've covered Hao Tian before, who the author of the LLaVA paper that did this, a while ago. And I think that's very foundational work for anyone who hasn't done vision work before.Ali [00:17:41]: Same with the CLIP and MetaCLIP, where you go from just captioning to building out questionsPhilip [00:17:47]: RightAli [00:17:47]: Off the image and how much better you can get performance.Philip [00:17:50]: Right. Right. Right. Yeah. But what's, what's so exciting about this is if you look at a model like this. Now, this is a little bit more of a research project. It's not. It got to 56% on MMLU Pro, I think. So not quite frontier. But if you're running this model, you haven't suffered any loss on your GLM-5.2 quality. If you don't have an image, it'll just behave exactly the way it used to. And ultimatelyAli [00:18:14]: Which in the inference code you literally do not include the other part, right?Philip [00:18:18]: Yeah. You would just skip the encoder if you don't have an image input.Ali [00:18:22]: Okay.Philip [00:18:22]: Just confirming.Philip [00:18:23]: YeahAli [00:18:23]: Does it affect a lot on the overall inference side? Like you're not adding much, you're adding a very small vision encoder. These are typically likePhilip [00:18:30]: They're super fineAli [00:18:31]: Less than a billion parameters, right?Philip [00:18:32]: Yeah. It's, - There's a little bit less standardization among vision encodersSwyx [00:18:37]: YeahPhilip [00:18:37]: So the support matrix can be a little bit, sparser. But overall, yeah, it's a pretty, it's a pretty minor component of the overall system. And ultimately what you get out of the system is all of a sudden you have Kimi Vision, GLM weights, and DeepSeek attention all in one model.Open Source Model Grafting and Franken-MergesPhilip [00:18:56]: And that's, I think, a lot of the power and beauty of open source, is that you can take all of these different components and combine them together into a system that's better than anyoneSwyx [00:19:05]: YeahPhilip [00:19:05]: Can be individually.Swyx [00:19:06]: People used to say that you would also do Franken-merges where you would take likePhilip [00:19:10]: YeahSwyx [00:19:10]: Layers from each model.Swyx [00:19:11]: Does anyone do that anymore?Ali [00:19:13]: Well, to your point previously when you were mentioning like, the work that goes into supporting a model when it first comes out, like GLM-5.2 or MiniMax M3 or whatever the case is. Sometimes you do have to like, you do have to switch out some things. Like, for instance, the MiniMax M3 head uses full attention, and with full attention you end up with this like insane bottleneck in spec dec ‘cause you're doing auto-regressive token generation for three tokens, and you're doing this like N squared over all of the tokens that are in your sequence. Your KV cache is like very large because it's not sparse, it's not top K. So we find it better to like, okay, we're gonna replace this, we're gonna replace this layer with a layer from another model that's using like GQA, for instance. And then just with the right training, you can get it to have the same acceptance rate. So it is very possible to retrofit layers from other models and very much needed. If a layer is like inefficient, the training just becomes the challenge, like how do you ensure that you train it properly? Which again to your earlier point is like the mesh between training and inference. As in like you need very good training in order to do fast inference. That's like, I feel like more and more becoming true.Swyx [00:20:21]: Yeah. Anything else on the support side when you say like get it to fully production ready?Loop Detection, Race Conditions, and Non-DeterminismPhilip [00:20:26]: Yeah. I think that there's also a question of just, we can test a model to a pretty extensive degree, but we're trying to get it out quickly and then you see a bunch of other people test it and you get interesting results. There was an issue with, GLM briefly where we had some like mode collapses where it would just output the same token over and over again for certain prompts on certain temperatures. Like once you expose an endpoint to the real world, there's going to be, so many more varieties of things given to it that you're able to, discover and patch things. So it's not just a, day zero process, it's then like for the first week, for the first month, if a model remains popular, like how do you both fix bugs and then continue to push the envelope on performance?Ali [00:21:21]: What do you mean you don't want your model outputting S?Swyx [00:21:24]: Is there loop detection on that stuff, by the way? It still happens like quite a lot, which is surprising.Ali [00:21:30]: We have like we, in our endpoint, like if a model was to output the same token like four plus times, we just cut the generation. We say like, “Oh, sorry, this-- Like try again,” or like we will reprocess the request. ‘Cause we know then, like if it, like if, yeah, it's four times the same token, it's probably collapsed.Swyx [00:21:45]: Yeah. Is there a way to opt out in case I really want that?Ali [00:21:48]: You want that?Ali [00:21:50]: I think there's a way that we have to handle it. I'm not exactly certain, but I feel like in certain models, like when they output something like you can imagine, like a table for instance, and so they want, they wanna draw like 12 dashes and 12 dashes. Yeah, I think there's a way for that to happen. I think we only do it on certain tokens. Like we exclude certain special characters.Swyx [00:22:07]: Yeah.Ali [00:22:07]: So we only do it on like certain like S is the most common almost. GLM-5.2Swyx [00:22:11]: OhAli [00:22:11]: And I think it was DSV 4 as well. Like you'd just have like looping issues where like you literallySwyx [00:22:17]: ItAli [00:22:17]: Just have like S.Swyx [00:22:18]: Yeah. Is there a special, something special about S? No, just randomlyAli [00:22:21]: It just seems to be the one token involved.Swyx [00:22:23]: Yeah. And it'Philip [00:22:24]: Is thereSwyx [00:22:24]: And it's only temperature 0Ali [00:22:27]: NoSwyx [00:22:27]: Even at other temperaturesAli [00:22:27]: Even at like 0.9 or whatever, it will still, it will still collapse.Swyx [00:22:30]: That's weird, right?Ali [00:22:30]: It's, it is an inference problem to be honest, like a software problem. Like oftentimes, the image you run will-- like NVIDIA will release an image for instance, and if we will upstream the changes from their latest TensorRT-LLM image into our stack, we'll find that it fixes it. Or oftentimes this will only happen in an inference engine that you're using like SGLang. But if you were to switch to vLLM, that isn't the case. So it seems to be like an extremely like deterministic software issue and not really a model issue. It's not like a weights problem. Like I'- we'll say like, “Oh, it's a problem with the quant. We did PTQ wrong,” right? But that isn't, that doesn't make sense because the same weights used with a different inference engine does not repeat the problem. And sometimes it's, the kernels that are being used in the backend have like these very subtle sometimes race conditions, where if you were to use this model hosted on one cluster, you will never get this problem.Swyx [00:23:19]: Oh my God.Ali [00:23:19]: But if you host it on a different cluster, you will. And the reason is the KV cache transfer from a node to node in that one cluster is using a slower interconnect than the node to node in another cluster. So that exposes the race, whereas in another cluster it doesn't. So then you end up just like, okay, this model is not gonna be hosted on this cluster. We're gonna host it on, another cluster because that cluster exposed that problem. But then it ends up with like, okay, is it the software? Is it the model weights or is it the hardware?Swyx [00:23:42]: There is a thing about this with temperature 0 still not being deterministic, right?Ali [00:23:46]: Right.Swyx [00:23:46]: Mostly because of hardware. Even at temperature 0 same model, you won't always get the same output.Swyx [00:23:52]: Even-- But I'm surprised by the race condition one because, I thought PyTorch was a graph that like guarantees that you at least, execute things in the right order.Ali [00:24:02]: Well, yeah, true. Like I'm not, I'm not saying that there is. Like well, you have things like PTL optimizations where like you can start a kernel before the end of the previous kernel, and that's like ‘cause you want to do that because there'sSwyx [00:24:12]: It's like pipeliningAli [00:24:12]: Expense. Exactly.Swyx [00:24:13]: Yeah.Ali [00:24:13]: But it'- But you don't do it cleanly. Like you overlap a little bit of the execution. No, it is very possible that the kernel itself, like that one block that is supposed to be running in this instance of time, that kernel itself has a race condition. For instance, like a missing barrier. Like often if you're designing a kernel and you want it to make it to be very fast, if you don't test it extensively, you'll, you'll have certain threads access data points from registers before they've been written to by other threadsSwyx [00:24:36]: YeahAli [00:24:36]: For example, because like your barrier is wrong or your synchronization was wrong. But yeah, like the testing itself is very difficult in those like, andSwyx [00:24:42]: And there's no like borrow checkerAli [00:24:45]: What does that mean?Swyx [00:24:46]: Like Rust. Like the. If you're trying to have like memory safety It sounds like a comparable problem.Ali [00:24:52]: Well, yes, but you're working in CUDA, right, NVIDIA GPUs. Like- You just need a higher level language like modular Maybe that's what modular is supposed to do. I don't know.Quantization Quality and Vendor FidelityVibhu [00:25:00]: How do you see keeping quality of the model? So you talked about all these steps of, okay, you gotta do quantization, train your own speculative decoderAli [00:25:07]: RightVibhu [00:25:07]: Run on different hardware. Looking at other model providers, okay, you kicked off a inference speed race on the consumer end. What goes into keeping quality the same across them, right? Sure, you can run benchmarksAli [00:25:22]: YeahVibhu [00:25:22]: But, like, how do you determine how much quantization are there standards? What goes intoPhilip [00:25:27]: There's a few things on quality. Most inference optimizations are lossless. KV caching, for example. You are just recomputing or preventing recomputing the same values. Speculation, of course, if a draft token is wrong, it gets rejected. The main lossy optimization is quantization. And that really comes down to, number one, data format, number two, which parts of the model you choose to quantize, which layers, and number three, like doing a lot of calibration on the quantized weights, to ensure that you're preserving all the outliers. There's other tricks that you can do, though. A big one is long context, ‘cause one thing you asked at, right at the beginning is, “Oh, what's gonna happen if I send a 200,000 token request in?” So with a long input sequence, you need to, store a lot more information. You need to process a lot more tokens. And so even if a model has a context of a certain length, you might, as an inference provider, choose to build an API with a shorter context length, and of course a full length one as well. Because if someone doesn't need the full million token context, for example, you can get them better performance. I don't know if that's exactly like quality of the model. The way that I think about quality is to what degree are we faithfully serving the original model? If you think of a golden implementation of a model that performs exactly the way the model is designed to perform, I think of quality as how close are we getting to that, 100% fidelity of the model.Philip [00:27:13]: You can also, of course, think about quality from the training side and how do you push yourself past 100%. But when I think about purely inference optimizations, it's getting faster while staying as close to that 100% fidelity mark as possible. And certainly our standard internally is that, like you should not be able to tell the difference between our API and a, official API. I think Kimi in particular does a good job of vendor benchmarking hereAli [00:27:41]: YesPhilip [00:27:41]: Where they haveAli [00:27:42]: They released an actual vendor benchmark.Philip [00:27:43]: Exactly, yeah.Ali [00:27:44]: ‘Cause they accused, some people, Amazon? There was some provider that was not doing very well on Kimi's benchmark.Philip [00:27:50]: Yeah.Philip [00:27:51]: So, with Reflect we probablyVibhu [00:27:52]: This was a long time ago, right?Philip [00:27:54]: No.Ali [00:27:54]: Yeah, like threeVibhu [00:27:55]: They alsoAli [00:27:55]: Four, five months agoVibhu [00:27:57]: This also happened with, I don't remember which model, but they pulled out quite a few, and then they started a whole chart about this. It might have beenPhilip [00:28:03]: Kimi Vendor Verifier.Ali [00:28:04]: Yeah.Philip [00:28:05]: Yeah.Ali [00:28:05]: Yeah, ‘cause you, ‘cause you'd be pissed, right? Like if you'Philip [00:28:07]: Yeah.Ali [00:28:07]: If like if I'm a consumer and I'm using like Amazon's endpoint for instance, and I've used Kimi and I'm like, “Oh my God, like this is bad,” I'm not gonna say, “Oh, Amazon quantized the model in a bad way.” I'm gonna say, “Oh, Kimi sucks.” Right?Philip [00:28:17]: Yeah.Ali [00:28:17]: So it seems like that makes sense.Philip [00:28:19]: Yeah, they care. They care.Vibhu [00:28:21]: Justifiably.Ali [00:28:21]: Yeah, justifiably.Vibhu [00:28:22]: This is probably a stupid question, but just checking, has anything improved from main quantization?Philip [00:28:28]: Yeah.Vibhu [00:28:28]: Like, is quantization always strictly worse?Ali [00:28:30]: Well technicallyVibhu [00:28:32]: NoAli [00:28:32]: It's a lossy. QuantizationPhilip [00:28:33]: YeahAli [00:28:33]: Is a lossy, it's a lossy implementation.Philip [00:28:36]: Speed improvesVibhu [00:28:36]: Speed improves.Ali [00:28:37]: It the number, likeVibhu [00:28:38]: No, I' always look for inverse scaling laws.Philip [00:28:40]: Yeah.Ali [00:28:40]: Yeah.Vibhu [00:28:40]: This is something I learned from Noam Brown, where like things that normally act in one direction sometimes do.Philip [00:28:45]: Well, technically when you run a benchmark, because these models are deterministic, sometimes your,Ali [00:28:52]: YeahPhilip [00:28:52]: NVFP4 quant is like, two basis points higher than yourAli [00:28:56]: No, it's noise. It's noise.Philip [00:28:57]: Yeah, exactly. I'm like, yeah, it's, it's within. That's why I always say within margin of error.Philip [00:29:01]: And I stopped saying that because everyone assumes that what is, well, within some margin of error, we're barely inside of that to the worst, so we're saying. But yeah, sometimes it's just like, gives you a higher output score. But like Ali said, that's noise. To my knowledge, you're not necessarily making the results better. You're just trying to, again, like keep your fidelity as close to 100% to the original model.Layer Selection, KL Divergence, and Better QuantizationAli [00:29:27]: There is, to your point, research that we did on MP. I don't know if you are able to pullPhilip [00:29:31]: YeahAli [00:29:32]: A tweet we did. One of our research interns, Joshua, I think it's a tweet on how we have 20% better quantized GLM-5.2 than NVIDIA. Essentially what we found throughout like this month research is, okay, quantization is a lossy. It's. You're compressing the data from, occupying 16 bits to occupying, four bits, for instance. And so you're losing some information, and you're trying to minimize that. And so when I say that I'm gonna quantize the model, my job becomes how do I find the layers that I can quantize, and how to find the layers to not. For instance, with image models, I don't quantize modulation layers, and I don't quantize out projections because those two are. Like out projection is what you see as the user. Modulation is what the model sees or understands. Right, exactly. And so to his paper, do you have the. It doesn't have the. Yeah. It's a long paper. I don't know if I can findVibhu [00:30:25]: If there's a part to search or it's probably in the thread.Ali [00:30:28]: It's probably in the thread.Vibhu [00:30:29]: Yeah.Ali [00:30:29]: But the long and the short is it is very possible that quantizing more of the model makes the results. Like if I have a model that I quantize layers one, five, and 10, and another model where I only quantize layers one and It is possible that the model in which I quantized more information is going to perform better because the quantization errors have canceled out. And so what Joshua showed in his mathematical proof where he had like a verifier in, is that you can predict which layers are going to have quantization errors that will cancel out with each other, and you choose to quantize those layers. And so the result of doing this mathematical quantization is you end up with a model that's 20% more quantized than another provider, so you get 20% more throughput of it because there's more layers than running an NVFP4, and your quality is better than that other quant because the layers that you chose to quantize have their errors cancel out, like one layer skewed to the right one layer skewed to the left, one layer skewed to the right. Your final logits distribution is more similar to the original distribution of the model, so you have better fidelity. And so the way we proved this was with KL divergence. So instead of just scoring on the benchmarks, we scored the KL divergence between the logit distribution of the quantized model and the logit distribution of the original full precision model, and we showed that with this technique we get. If your probability distribution on the logits which token it wants to select is more of the same as the original model, you're probably gonna end up staying true to the original model. So yeah, so it seems like previously before this, it seemed like the industry was, well, the more you quantize, the worse it's gonna be, ‘cause the more loss you introduce. That's not exactly, not necessarily true. So yeah, doesn't improve it, but can cancel out.Philip [00:31:57]: I think it might be this, but reminds me a good bit about pruning where you can prune off certain layers.Philip [00:32:03]: But very interesting. Didn't know this was a whole paper you guys put out.Ali [00:32:06]: It's. Fun fact, it was originally 72 pages, this paper, and then we decidedPhilip [00:32:11]: WowAli [00:32:11]: We can't tell. We couldn't release it. So it's now 45.Swyx [00:32:15]: Still 39 pages, so very substantive. We talked about evals and all these things and, like what's possible in terms of speedup? Like it's like probably like the numberInference Speedups and BenchmarkingSwyx [00:32:25]: Thing that people do wanna care about, and it's something that you wrote about in your post. Like official API is 70 tokens per second, and you push it up to 90. Is that like a normal thing?Philip [00:32:36]: So what's cool about working in inference, the reason that I think inference is going to be a useful place to do engineering for a long time, is that if you look at highly optimized domains like, say, finance, if you're in finance, you measure how much better you got in basis points. It's like, “Oh, I got five basis points better, like twentieth of 1% better,” that's huge news because everything is so optimized. When we publish optimizations, it's 20%, it's 100% it's 200%. So there's still probably like a lot further to go, honestly. Like you'll, you'll know that inference is pretty much solved when researchers start publishing about how they got 1% faster at something.Swyx [00:33:19]: Which by the way, because I am from the finance background, in the ‘70s, that was the margin at the time. When you did quantitative finance research, you would findAli [00:33:27]: And like 20%, tens of percent.Swyx [00:33:29]: That's. Yes.Philip [00:33:29]: Yeah.Swyx [00:33:30]: And now it'Philip [00:33:31]: Tiny fractionsSwyx [00:33:32]: For those people interested, look up Andrew Lo's paper. He had a really interesting illustration of quant, stat arb, distribution, narrowing down from like those kinds of 20% differences in the ‘70s, down to nothing today, which is very cool.Philip [00:33:48]: Exactly, and we're at the beginning of the same type of thing. Now benchmarking is hard. I think anyone will tell you that, and benchmarking provider speeds is hard because there's so many variables that go into it. What hardware are you using? How much load do you have on the system? What's the exact nature of the prompts and input and output sequence lengths? All that stuff. But overall, when you start stacking these improvements, you're looking at multiples. You can look at it. The most common form, of course, is TPS, tokens per second, which is bad naming by us in the industry, ‘cause there's two tokens per second. There's tokens per second, the throughput number, and the latency number.Ali [00:34:31]: TTMT, yeah.Philip [00:34:32]: Like total tokens per second out of the, out of the GPU as a throughput number. Most people only care about tokens per second as the latency number, which we should call ITL, intertoken latency, but we don't.Philip [00:34:44]: Anyway, so you can imagine a standard API without many optimizations for a 1 trillion parameter model operating somewhere in the 30 to 50 tokens per second range for reasonable traffic profile. And we generally see the goal of, pushing to 10X that. But, not necessarily day zero, but by stacking enough optimizations, if you have, say like four optimizations, each of which doubles performance. Or sorry, three optimizations, each of which doubles performance, then you stack that up, that's an 8X gain. That's the order of magnitude that we're working with in this space. We're trying to make things substantially faster, not just go from like 70 to 90.Swyx [00:35:38]: Are you saying you've. You have done that?Philip [00:35:40]: So let's say you have as a reasonable baseline, 30 or 40 tokens per second. You can achieve 10X that. So like on GLM-5.2, if you run it unquantized, perhaps on H100s even, and you're just using an off-the-shelf inference engine with no particular optimizations, no speculator, nothing extra around like KV routing, no disaggregation, you're, you're probably, yeah, looking at that like 30 to 40. You think that's like a reasonable baseline?Swyx [00:36:12]: Right. Right.Philip [00:36:12]: To get to something like 10X, there's a lot of trade-offs that you're making. If we're running at more like a 300, 400 tokens per second range, you are using the best hardware possible. You have a optimized speculator. You have done all of your quantization work. You are Seeing a pretty high cache hit rate. You are running with a reasonably small batch size and a parallelism configuration that is tuned for latency versus throughput, but it is possible. So the spreads that you see if you, like, go on artificial analysis or you go on OpenRouter and you look at, the worst provider to the best provider, oftentimes can hit that range. 10X is of course very aggressive. It's oftentimes maybe more of a four to six times improvement. But that's the performance that makes us really excited, is when we can get these huge gains, not just go from 70 to 90 tokens.Stacking Optimizations: NVFP4, Speculation, and DisaggregationAli [00:37:19]: It's also, like, hardware dependent. Like, ifPhilip [00:37:20]: YeahAli [00:37:20]: If you have a thing where you're serving it on just, like, a node of H100s and then you throw, like, you shard the model across, like, four nodes of B200s. Like, you can definitely increase the speed with just throwing more hardware at it. Like, normalizing for the same exact hardware and the same number of GPUs.Philip [00:37:35]: Yeah. Then you're looking at, like, a two to 4X improvementAli [00:37:38]: Right. RightPhilip [00:37:38]: Depending on the inference optimizations. So yeah, it's. Some of it's, what's the call, and some of it's who's the driver.Vibhu [00:37:46]: If you break down the two to 4X, say the example is run GLM-5.2Ali [00:37:51]: YeahVibhu [00:37:51]: On B200sAli [00:37:53]: YeahVibhu [00:37:53]: Single node, right? What's, like, the cost trade-off for effort to get, like, the last bit of juice out versus what should people just think of, right?Ali [00:38:01]: Spectre quantization. Yeah.Vibhu [00:38:03]: Spectre quantization.Ali [00:38:04]: That's, that's, that's like 95%. LikeVibhu [00:38:06]: And how far does that get you? And how easy is that for the average person to do? So say right I wanna throw the weights of GLM-5.2 on a node of B200s, how easy is it to find speculative decoder- decoder model or already quantized model? How much work goes into it?Philip [00:38:23]: If you're doing it up front, it's quite a lot of work. If you're doing it today, there's going to be people who have published things that you can just, you can just grab some NVFP4 weights. You can grab a speculator. Yeah, if we're thinking about, like, what are the 2Xs we're stacking, going from, BF16 to NVFP4 is, it's not quite a 2X, right? It's like. I think it's about, like, 30 to 40%, from 16 to 8, and then another 30 to 40% multiplied from, 8 to 4. So that doesn't quite get you a 2X, but, like, roughly a 2X. Speculator, roughly a 2X. Disagg on top of that if you're able to get enough hardware and put enough traffic through it, another roughly a 2X. And then you add in some, double-digit percent increase from having just a better runtime with, the latest kernels and stuff behind it. And that's how it stacks up.Ali [00:39:21]: YeahPhilip [00:39:21]: So building each of those, like, building the, quantized weights is, for someone who really knows what they're doing, hours to days of work. Building the speculator, again, like, hours to days of work. And the, disagg setup, hours to days. Well okay, but like once you haveAli [00:39:39]: Once set up. Once set up. YeahPhilip [00:39:40]: Yeah, getting disagg working for the first time, I'm saying, of course, is very difficult.Philip [00:39:44]: The marginal implementationAli [00:39:48]: Like, if you're just grabbing, like if you are a person, like just a normal consumer who has access to, like, a node of B200s and you're wondering, “How can I just host it myself?” You don't need to quantize the model yourself. There's always gonna be, like, an open source quantized checkpoint. NVIDIA's gonna push one out if no one else does. You. Usually, the providers will have their own spec dec that they've trained as well. You don't need to train your own spec dec. You can just use that as well.Philip [00:40:09]: Yeah. Like, GLM-5.2 has its own MTP.Ali [00:40:13]: Right. Right.Vibhu [00:40:14]: What's multi token prediction?Philip [00:40:15]: Yes.Ali [00:40:16]: I'm justVibhu [00:40:16]: Can you explain that?Ali [00:40:16]: I'm just an expert.Ali [00:40:18]: I can do it for you in case I get it wrong?Vibhu [00:40:20]: No.Vibhu [00:40:21]: Yeah, you should correct if we're wrong, but their multi-token prediction can be used for self-speculative decoding.Ali [00:40:27]: I'm not sure. I'm not gonna correct that.Vibhu [00:40:28]: Okay. I'm semi-confident in thatAli [00:40:30]: Okay. YeahVibhu [00:40:30]: But someone can check. But it's useful to paint the story of, okay, not just the average person, but say a company wants to switch from serverless inference I wanna throw this up on. I wanna rent some GPUs, throw it up. These are the steps you take to do significantly faster than just put it behind vLLM.Ali [00:40:48]: Right.Vibhu [00:40:49]: I was waiting for a mention of Dynamo.Vibhu [00:40:51]: I feel like, that's supposed to be the baseline that you measure against.Dynamo, KV Routing, and Disaggregation ToolkitsPhilip [00:40:55]: I would think of Dynamo as less of a box system and more of a toolkit for building with. So when we talk about doing aware routing, when we talk about doing KV offloading, when we talk about doing, PD disaggregation, Dynamo fundamentally is. By the way, Dynamo is an open source library from NVIDIA.Ali [00:41:17]: We've done a pod with KylePhilip [00:41:18]: OkayAli [00:41:19]: Kyle Cranin.Philip [00:41:19]: Cool. So then your listeners know then that it supports all the different inference frameworks. And it is multi hardware, which is interesting.Ali [00:41:28]: But it's just a router, it's not like an optimizer layer.Philip [00:41:30]: Yeah. All it does, like, what Dynamo is good at, it is a library for moving information around your cluster, around your hardware. So if you have, KV cache on one place and you need it to be somewhere else, Dynamo coordinates NIXL for you to move that around.Philip [00:41:49]: That doesn't mean that, like, out of the box, you just say, “Pip install Dynamo,” and then you get, like, a massive performance speed up. It's more of a developer toolkit.Ali [00:42:01]: Yeah. I would have said it would. It comes with a set of defaults that you can then swap out.Philip [00:42:06]: It does. If the industry at large, I think, was, like, rolling out all of these deployments, standard, then I think it would be, like, a credible baseline. But, we've got to, we've got to benchmark against, like, what we're seeing in the wild.Speculative Decoding Methods: Medusa, EAGLE, n-Gram, and Spec-SpecVibhu [00:42:23]: I did wanna talk a little bit more about PD disagg, because that is probably, like, number three after quantized and speculative decoding. In your book though, I was just gonna pull out the book.Philip [00:42:31]: Yeah.Vibhu [00:42:32]: Like section 522 on Medusa, 523 on EAGLEPhilip [00:42:35]: YeahVibhu [00:42:36]: 524 on gram.Philip [00:42:37]: It's 55, would be disaggregationAli [00:42:42]: Yeah. Well, no, I just wanted to dwell a little bitPhilip [00:42:44]: YeahAli [00:42:44]: The other. Like, so what do you choose to include? What do you choose to not to include? Because there was all these other techniques.Philip [00:42:51]: Yeah.Ali [00:42:51]: Are these still relevant? Because I think they came out, like, a year and a half ago maybe.Vibhu [00:42:55]: Medusa is quite old.Philip [00:42:56]: Yeah, Medusa's old.Ali [00:42:58]: It was old.Vibhu [00:42:58]: But is it in the book as a good, here'sPhilip [00:43:01]: BaselineVibhu [00:43:01]: Baseline vanilla understand it?Philip [00:43:02]: Like you should know this.Vibhu [00:43:03]: Like I read the paper, I'm like, “ it makes so much sense.”Philip [00:43:05]: Yeah.Philip [00:43:05]: So with the book, I had a couple goals. One was to give people just a working vocabulary for the space as a whole, and the other was to give them some intuition about how each of these techniques works. As I mentioned in my AI Engineer talk, which is the first public addendum to this, the speculation space has moved much faster than everything else. So yeah, even at the time that I wrote the book Medusa, I very much included as a way for people to understand how the space evolved rather than what the most modern technique is. And now of course, there's DFlash, dSpark. There's, there's newer techniques even than EAGLE, although EAGLE is still very commonly used.Ali [00:43:51]: SpecSpecta.Philip [00:43:52]: Yes. Speculative decoding.Vibhu [00:43:54]: What canAli [00:43:56]: Oh, it's a paper by Tri Dao and it's like, it's doing speculative decodingVibhu [00:44:00]: HuhAli [00:44:01]: For the speculative decoder.Philip [00:44:02]: Oh, in spec- oh my God.Ali [00:44:02]: It's literally just an another. It's like, yeah, that's the most simple way to explain it, and it seems like he got trivial speed ups there. But it seems that the complexity with training, it's almost like in our mind at least, it's almost as complex as training GANs. Like it's like a very delicate balance and oftentimes you, it's just but yeah, it's literally speculative decoding on speculative decoding.Vibhu [00:44:21]: Speculative.Ali [00:44:22]: Yeah. We saw this paper.Vibhu [00:44:24]: It's interesting, right?Ali [00:44:24]: Yeah.Vibhu [00:44:24]: I wouldn't even expect it to be very particular to train, I wouldAli [00:44:29]: Right.Vibhu [00:44:29]: The naive part of me is like, okay, train speculative decoder.Ali [00:44:32]: But like, and it makes sense, like the whole idea of speculative decoding is you. It's like, it's like almost like the iPhone auto predict version but for a normal model, right? Like you're just, you're just, generating three tokens and you're like, okay, I'll do prefill on them. And so you save those three turns for your original model. Now your speculative decoder is doing three turns of auto regression, so why not just have an even smaller model?Ali [00:44:53]: The other question there is what are the size of speculators? So say forPhilip [00:44:58]: Right. It's like a billion parameters.Ali [00:45:01]: Like for MiniMax, it's. Yeah. It's like one layer. It's like one 60th of the original model usually.Philip [00:45:06]: Yeah. I think we should do a paper when we get back to the office.Philip [00:45:10]: SpeculativeAli [00:45:11]: SpeculativePhilip [00:45:11]: Decoding.Ali [00:45:13]: No, it's, it does seem like how, when do you stop? But then it also seems like if you're able to train spec-spec decode for instance, right? Like if you're able to have a small model that is accurately predicts what the intermediate speculator is gonna predict, that is able to predict what the original target model's gonna predict, then why not just use that smallest model directly, right?Vibhu [00:45:34]: Yeah. This isAli [00:45:35]: Like it seems likeVibhu [00:45:35]: Adjacent to the routing problem.Ali [00:45:36]: Right.Vibhu [00:45:36]: Yeah.Ali [00:45:36]: Right.Philip [00:45:37]: The thing with speculators is one of the practical constraints on using them is that you do have to run a small model on the same hardware that you're running the big model on. There is a orchestration and resource competition problem inherent in that, and that is one of the constraints on speculation in general, is that draft tokens cost resources to create and cost software complexity to manage. And so if you have like infinitely recursive speculators, you add in quite a bit of that complexity on the actual implementation within the inference engine as well, not just in the training process.Vibhu [00:46:17]: I was gonna say, I would wonder if you could do similar, like distillation and pruning of, it's the same thing, it's just a model. Can we not just distill a lot of the weights, quantize the speculator, out of my domain? The question that also comes up is, this is all for big server workloads, right? How much of this applies to, say I have this MacBook, I wanna run Gemma really efficiently. Similar problems, not the same?Local AI vs. Data Center InferencePhilip [00:46:45]: Pretty different. I talked to Selo, about this on his podcast a couple weeks ago. The difference between inference engineering for the data center and for production workloads versus inference engineering for local AI, is that we start with fundamentally like different constraints and different goals. With local AI, it's how do I fit this model onto my hardware and then make it less dumb? And with data center influence, it's how do I load this model and then make it less slow? And we care about less dumb, and they care about less slow. But the local AI inference engineering ecosystem, I think has a lot for us to learn from in the data center space. They are experts in various forms of quantization, including dynamic quantization that we just don't touch, in the pruning, in the distillation, in the, layer removal. There'Ali [00:47:42]: Layer removal matters less.Philip [00:47:43]: Yeah. There'Ali [00:47:44]: No one loves pruning really.Philip [00:47:45]: Yeah. Well, but the, but they doVibhu [00:47:46]: Which is surprising, right? But that's, that's a whole different thingPhilip [00:47:48]: Just to fit something on the laptop.Ali [00:47:50]: Right.Philip [00:47:50]: So yeah, it's a, it's an interesting, it's an interesting space. Not necessarily that like their techniques make sense for us to do in the data center, because we have different resources and different goals, but more that the process as well as the openness of that field is something to, admire.Ali [00:48:12]: Yeah. Like to your point, like, certain optimizations that would. Like for instance, Turbo Quantum Sharper, like it made such huge hype on that and we did like a whole deep dive on Twitter and like said, what is it? How does it work? Why is it good or not? And it took off and it was implemented on local devices because your memory bandwidth is so slow on like a MacBook, for instance. But try putting the same thing on like an NVIDIA GPU on a B200 Turbo quant would not be. Like, it would not be used. Like, NVIDIA - Like, NVIDIA made it clear that this is not a good optimization, and we've seen it firsthand where the overhead of doing dequantization, quantization of, in the kernel itself with turbo quant kernel, each end is much slower than the time that you save from doing the bandwidth. ‘Cause on the B200s, you have like 3.5 terabytes per second. You don't need decrease the storage that much. You don't need to do, FP4 KV cache. You don't need to use a requant. There's, there's, there's better optimizations to be made. But on Edge devices, it's extremely important, it's extremely useful. So, seems to be, like, different optimizations there, but then they're all uniquely combined with like all you wanna quantize the model, you wanna do speculative decoding, like certain common prefixes with bothPhilip [00:49:18]: Principles.Ali [00:49:19]: Yeah, exactly. Exactly. Exactly.Philip [00:49:20]: They also do a lot of work on, model parallelism, especially over, heterogeneous topology, where you have, some sparks and they are wired together with, Ethernet, DGX sparks.Ali [00:49:35]: Yeah, this is the Exo Labs guys.Philip [00:49:36]: Yeah. You have, a nu

More like Oof-ston amirite?Road trippin'Fit ChecksMatch MoodsNo Celly Ratings This Week

My guest on the show today is Tuhin Chakrabarty, an assistant professor of computer science at Stony Brook University and the culprit behind my first ever trigger warning on eminent Americans.Here we go: if a big part of your identity is vested in being a writer, and you are already experiencing existential dread about the prospect of AI getting better than you at writing, you may want to not listen to this podcast, or at least you may need to take a Xanax or a gummy before listening. Because Tuhin's work, training AI to produce super high quality samples of high literary writing, is existentially threatening. It goes right at the heart of what we wordsmiths do. Just to give a sense, here's a passage from a New Yorker article by Vauhini Vara that explores what Tuhin has found:I asked Chakrabarty to run an informal version of his experiment on my writing, with a twist: I would pit his model directly against me. To start, he fine-tuned a model on my published writing, much as he'd done in the formal experiment. Then I sent him four short excerpts from a novel that I'm currently writing. No one else had read these excerpts; they had never been published or circulated. There was no way that a large language model could have seen them before.The narrator of my novel in progress is an Indian American ex-journalist. She runs a nonprofit that publishes stories from immigrant and refugee women, but it's strapped for funding, so she courts an Indian American venture capitalist as a potential donor. Chakrabarty used an L.L.M. to create content summaries of the excerpts I'd sent him. (One representative sentence, from a summary about the narrator's journaling habit, explains, “A pivotal memory is introduced: in ninth grade, the narrator's mother read this journal, an act seen as a profound betrayal.”) Finally, he gave the summaries to his fine-tuned model, and he asked it to compose passages “in the style of Vauhini Vara.”Going into all this, I was self-assured, even smug. I'd always felt that my style was original and, more important, that my books were totally distinct from one another. I figured that, even if the A.I. model could imitate my past books, it couldn't predict the style of the novel in progress. So, when Chakrabarty sent me the A.I.-generated imitations, I was genuinely confused. Like Díaz and Nunez, I found lots of stylistic details—rhythm, verbiage—annoying. But the text produced by the model was eerily close to mine. Reading some of its lines next to my own, I couldn't remember which was which. Unlike Díaz or Nunez, I even preferred some of the doppelgänger's versions. My style seemed to be more consistent across projects than I'd realized.I sent four passages to some readers who'd liked my previous books, explaining that half were mine and half were the model's. I wanted them to guess which were which. … The first of my readers to respond was Dana Mauriello, my best friend from college and an accomplished tech entrepreneur. “Truth: this was terrifying!” she wrote. “I was so nervous that I would say that AI wrote something that you wrote, and you would be insulted!!!” Her anxiety, it turned out, was justified. She didn't get any of them right.Dana blamed this partly on her not being a writer. But, of my seven readers, none correctly identified more than half the passages. One of the last people I heard from was the novelist Karan Mahajan, a professor of literary arts at Brown University. He and I learned to write together in college, along with Tony Tulathimutte, and have been sharing drafts with each other ever since. He's among the most perceptive writers and readers I've met. “Oof, this was really confusing and mindmelting,” Karan wrote. Then he, too, misidentified all four excerpts.To be clear, Tuhin isn't claiming that AI, except under the very specific, rather contrived circumstances of his experiments, can already match the best human writers. And he doesn't know if they ever will be able to match or surpass us. I don't know how much consolation that affords, though; what he has done is threatening enough. Moreover, in this episode we're doing what is sometimes the most anxiety producing thing of all when it comes to fraught questions: We're sitting in the uncertainty, and poking at it. How does his method work? What does and doesn't it prove? Why did he think to do it in the first place? What are its implications? What technical challenges would need to be solved for it to move from short samples, where detailed plot points and themes are provided by a human, to actually generating a full book with just a short prompt or a broad outline? And what are we human writers owed if it's our work, used as training data, that creates the conditions for being superseded by AI? Etc.I wrote a long essay, not too long ago, about my intuition that most of the fiercest responses to AI out there, both from the hardcore boosters and the hardcore haters, are flip sides of the same coin. They're dysfunctional manifestations of the anxiety that so many of us have right now in the face of what may be an extinction level event not so much in the Skynet takes over the world sense but of a more existential sort. We may soon be deprived of some of the core ways we've understood what it means to be distinctively human, how we've constructed our unique purpose as humans.Because of this fear, Tuhin has gotten a bit of hate for what he's done and shown, but from my perspective he's dealing with the existential threat we face in exactly the right way, the psychologically functional way. He's looking directly at it. He's rejecting both utopian boosterism and head in the sand denialism as strategies of evasion. He's being creative with the thing itself. And he's thinking in super pragmatic ways about how to best serve human interests in the long run. And not just thinking about that but doing something concrete and positive about it, collaborating with legal scholars on how to think about and craft policies that protect and compensate authors.So I'm sorry to stoke your anxieties, dear listeners, if that's what we do, but I'm not sorry I had Tuhin on the show. He's a fascinating and flexible thinker, a hard core computer scientist and a true lover of literature.Also: Eminent Americans, my sweet baby, is now produced in collaboration with the John C. Danforth Center on Religion and Politics at Washington University in St. Louis, and is distributed through its publication, Arc Magazine. You can find all of Arc's podcasts and much more online at arcmag.org.Hope you enjoy the show, and that the radiant beams of humanity and thoughtfulness that Tuhin and I generate between us don't just soothe your anxiety but ultimately help you to process it.Peace. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit danieloppenheimer.substack.com/subscribe

Rosemary Barnes, CEO and founder of Pardalote Consulting, joins to discuss their new grant-funded study of blade erosion and heat fatigue in Australia. Sign up now for Uptime Tech News, our weekly newsletter on all things wind technology. This episode is sponsored by Weather Guard Lightning Tech. Learn more about Weather Guard’s StrikeTape Wind Turbine LPS retrofit. Follow the show on YouTube, Linkedin and visit Weather Guard on the web. And subscribe to Rosemary’s “Engineering with Rosie” YouTube channel here. Have a question we can answer on the show? Email us! Welcome to Uptime Spotlight, shining light on wind energy’s brightest innovators. This is the progress powering tomorrow Allen Hall 2025: Well, Rosemary, welcome back to the show.  Rosemary Barnes: Thanks, Allen. Great to be here. For, it’s been a while since we did one of these one-on-one episodes, like a, yeah, a proper, proper guest.  Allen Hall 2025: Well, this is kind of a celebratory episode because your company, Pardalote Consulting, has been awarded, uh, some funding from the Australian Capital Territory’s government for the Energy Innovation Fund. Rosemary Barnes: It’s a really good program that the ACT government has to try and get energy innovation In the state. It’s not a state actually, it’s technically a territory. Little more than just Canberra, the city. Uh, but there are actually quite a few, like, really interesting energy-related companies here, partly ’cause of the, the fund I think helps, but also just tracing back like, [00:01:00] uh, y- you know, in the 20-teens, Australia had a really conservative government that hated renewable energy, and the ACT government had a commitment at that time to 100%, um, 100% renewable electricity for the, the government. And that was one of the only programs that was resulting in a lot of, um, you know, clean energy projects being built, and one of the conditions that they put on that, uh, for people that would win PPAs with the ACT was that you had to have your headquarters in Canberra. So we’ve actually got quite a few, quite a few really cool, innovative companies out of here. Um, like Neoen’s headquarters here. Windlab, uh, yeah, was, was founded here and still has a lot of people here. Pardalote obviously, and you know, a few other companies as well. So despite it being a small city of like, I don’t know, maybe it’s up to 400,000 or something people by now, um, yeah, there is actually quite a lot going on here for energy. Allen Hall 2025: And the Energy Innovation Fund is funded by the wind and solar operators in the area, and your particular [00:02:00] effort has really global consequences. You’re focusing on two areas involving how wind turbines survive Australia, but more, uh, of relevance is to just really tough conditions which exist not just in Australia but around the world. What two areas are you going to focus on?  Rosemary Barnes: Yeah. So the two focus areas are leading edge erosion and high temperature fatigue, which we can probably get into the definitions of those in a minute. But basically my, um– what led me to wanna have a project like this was that when I moved back to Australia in 2021, I– and I started working in O&M, uh, I noticed that the wind turbines that I would look at, the blades that I would look at here behaved really differently to the ones that I worked with overseas. You know, es- especially with leading edge erosion, like often I would be doing a condition assessment of a, you know, a new wind farm. Um, might only have been operating for, you know, two years. That’s a pretty common time for people to get in and do a condition assessment [00:03:00] because their warranty period is about to end and they wanna, you know, make sure that everything is okay. Um, and I would just notice that often, like 90, 100% of blades would already have bad erosion after just a couple of years, which is super-duper fast. And then there are some tools available to check, um, like what kind of erosion are you likely to experience on your site. Like is it a higher severity erosion site or a, a low severity one? Um, and you basically, you know, the status quo globally is to just look at the annual rainfall, um, and the tip speed. And if you’ve got, you know, high for both of those, that’s a bad erosion site. And if you’ve got low for both of those, it’s a, a low erosion site. But when I plotted out the wind farms that I knew had really bad erosion problems onto, you know, a chart with those two axes, I just saw a random distribution of dots. You know? Like, this was not– uh, this had no predictive value for Australian wind farms. And so that led me to believe that, okay, um, you know, things are a bit [00:04:00] different here. Makes sense, you know, most of the knowledge that we have about how wind turbines operate, it’s been developed and validated mostly in Northern Europe. You know? Like it’s, it’s Denmark and the surrounding countries that had, like, the bulk of the early wind energy. First few decades of knowledge were, you know, were mostly there. Of course, there were some other, um, places that had wind turbines, but, you know, most of the The OEMs have been operating for decades, came from Denmark. And I know when I lived in Denmark, the rain there is very different to the rain in Australia. So in Denmark, it’s basically always raining, right? Like, it’s just… Like, even if it’s not raining, you’re still gonna get wet when you go outside ’cause it’s just, like, the air has this just amazing ability to just hold onto moisture. Um, but it’s very, very gentle. But, you know, over an entire year of most days having gentle rain, that adds up to a lot. Whereas in Australia, and especially if you go, like, north to Queensland, it rarely rains. It’s mostly just dry, and when it [00:05:00] does rain, it’s like a tap turns on, and I, I swear you will get bruised from the rain droplets hitting your skin. You know, they just have so much energy in them. So I think that that i- you know, when you look at just the overall rainfall, you really hide something important about how erosion, um, can progress. Then, um, there’s other places in Australia that have very different characteristics. Again, they don’t have that kind of really intense rain but, you know, some of those sites are also having really bad erosion. And so it just occurred to me, I did a lot of research, you know, into what’s going on and, you know, the academics are studying erosion a whole lot, and they’ve got, you know, a lot of standardized tests and, you know, products are developed according to these standardized tests. But the standardized tests don’t actually resemble reality, and especially they don’t resemble reality in Australia. And so my client started asking me, “Okay, you know, the products that we have are, are terrible. We have to replace them every couple of years. It’s, um, causing big problems with also [00:06:00] the amount of energy that you’re losing.” One of the types of, um, leading-edge erosion or leading-edge problems that we have in Australia is that the, the coatings tend to peel off and make these, like, big flakes which will just massively disrupt the airflow, can cause y- you know, at least a few percent AEP loss, and maybe up to five. And even worse than the AEP loss is the revenue loss because it affects it most at, you know, lower wind speeds. Um, you get a bigger hit than at rated wind speeds. So there’s a variety of problems going on with leading edges in Australia, which mean that I, I basically… My clients would ask, “What product should we put on to prevent having to, you know, constantly replace this?” ‘Cause it costs, like- you know, 30, $40,000 per turbine to replace the protection, not to mention, you know, one or two days of downtime. It’s expensive, and I basically, I didn’t have a good answer for them. What, what product should they put on? I don’t know. No, we, we don’t know. One, we don’t know what the [00:07:00] specific, um, characteristics are that are… what the specific local environment, local conditions are that are accelerating leading-edge erosion, one. And two, all of the products tend to be tested around this, you know, there’s this protocol that academics have come up with, and they’ve kind of like assumed that this is representative of how things behave in the field, and it’s– I don’t think it’s particularly true anyway, but it’s especially not true in Australia. There are a few companies that are testing to different standards. Um, definitely applaud them. But without knowing wha- what are the conditions truly like in Australia, uh, it’s really hard to advise, like, what kind of tests should you be demanding from a product you’re considering to be sure that you’re gonna put it on and not gonna be replacing it again in two years. Allen Hall 2025: Because that’s really the trouble in Australia is when you get offered products They have been tested generally in somewhere in Europe and maybe in the United States, and then when they go to [00:08:00] Australia, it’s really unknown as to how those products will do, which is a huge risk for the Australian wind market as to what to choose, how to choose, is it– what’s real in terms of test data. So now you’re gonna go out and do what? Are you gonna put sensors out by the wind farms? Are you gonna try to do more of a statistical summary of the actual environment around wind farms using existing data? What’s the approach here?  Rosemary Barnes: It’s all of the above, but the part that is supported by the grant is that we’re gonna have enough money to be able to buy some scientific-grade sensors and put them on, um, a sample of Australian wind farms. So we’re gonna be looking at a lot more characteristics about the rain than simply is it raining now, you know, how many millimeters per hour. We’re also gonna be investigating, you know, every kind of characteristic of, of that, um, of that rain, um, including, yeah, like the, the energy that’s in it, for example. A, a bunch of stuff. I won’t get into every single [00:09:00] parameter. Um, and you know, other things as well, like measuring UV, solar radiation, um, particles, because, you know, in Australia we have a lot of dirt roads, which I know is very common in wind farms around the world, but Australian dirt roa- roads are always dry and dusty, like 99% of the time, so that’s one of the things that y- you know, maybe that’s causing a difference. Um, so basically putting sensors all over a bunch of wind turbines and then monitoring the erosion, um, a combination of some real-time monitoring and also looking at inspection, um, drone inspection images annually. We also have a- an option where we’ll just be using SCADA data and inspection images, so that’s like a lower cost version where we can combine that with the findings from the scientific-grade instrumented turbines to build up a picture of what types of conditions lead to accelerated erosion.[00:10:00] Allen Hall 2025: So the SCADA data will, will have some information inside of it, you think, that, uh, will correlate to the weather outside?  Rosemary Barnes: It has some Additionally, we can look up, um, you know, just the weather data, like how many millimeters fell during which 15-minute interval throughout the day, what was the temperature. SCADA will tell us also what the temperature was, um, what the speed of the turbine was, so you can calculate the tip speed, ’cause that’s an important thing. Um, yeah, so it’s, it’s two, it’s two tiers of data collection. The scientific grade sensors, as you can imagine, are, are really expensive and y- you know, the, the grant project has contributed a, a lot of funding, um, but it’s not enough to put those, yeah, put a little mini lab on top of every turbine across Australia, obviously. So that we’re using s- doing selectively, and then we can increase the number of wind farms that are included in the study by just doing this, um, cheaper version of the SCADA [00:11:00] plus, uh, weather data that’s available.  Allen Hall 2025: So what are some of the risks on the temperature side for all the high-temperature regions of Australia that have wind turbines? Clearly it’s generally warmer in Australia than it is in, in Scandinavia and Northern Europe. What kind of temperatures are we talking about on the ground?  Rosemary Barnes: Uh, well, temperatures here can get pretty close to 50 degrees. Um, and if you’ve ever been inside a wind turbine blade on a, even a mildly hot day, you’ll know that the temperature inside a wind turbine, and especially inside the blade, is much hotter than what it is, uh, what the ambient temperature is. So this project is one– I’ve actually been talking about this project for, yeah, like over 10 years now. Ever since I started, I moved to Denmark, started working for a wind turbine manufacturer, I had done– I had just finished doing my PhD on composite materials, structural design, and analysis. So, um, yeah, very, very familiar with, [00:12:00] you know, how composite materials work and, in particular, the effect that temperature has on them. I mean, like most materials, when composites get warmer, they get softer, and that is really important for a w- a wind turbine blade. You know, if it gets, um, less stiff, then you’re gonna get a lot more strain, and that is going to affect your fatigue behavior. Y- you know, fatigue is just the application of a little bit of, a small amount of strain. It’s not gonna cause damage, but when you apply it millions, tens of millions of times, like you do in a, o- over a wind turbine’s operate, um, operating lifetime, then that builds up. And, you know, wind turbine blades are a very fatigue-driven design. Um, it’s one of the most important things to consider when you’re designing a wind turbine blade. And so when I got to Denmark and I learned how materials are qualified and how the qualification is treated in the certification process, I just realized it’s not particularly conservative, and also that some of the assumptions that are made that [00:13:00] wo- again, they worked really well in more moderate climates where wind turbines have had most of their developmental history. You know, it’s not such a big deal there if you test at room temperature. Your wind turbine blade is spending most of its operating lifetime at room temperature or below. It’s, it’s rarely, you know, above 30 degrees in Denmark and most of Northern Europe and, you know, also a lot of, um, a lot of America, not, not all of it But, um, in Australia it has just extended periods above that temperature and even exceeding the temperature where, you know, wind turbines have an operating limit and after that they will shut down. But the operating limits are based on ambient temperature. It’s not based on what’s the temperature in the laminate, which is what really matters for blade lifetime. So anyway, I’ve been obsessed, like honestly obsessed about this issue for 10 years. Talked about it with anybody who would listen . But then when I started working in O&M in [00:14:00] Australia and I started seeing some wind farms with an abnormal number of cracks early… again, early in their lifetime, you know, I think one of the wind farms I was looking at was maybe three years old or four at the time. I think it was three actually, and had a lot of cracks, and I looked at a few years in a row and it was more and more cracks every year and I’m like, “Oof, this really looks like end of life fatigue behavior.” A- actually it’s not, y- you know, there’s this concept of a bathtub curve where, um, when you’re looking at failures in components, in, in anything, not just in, um, wind turbine blades, but you know, like you’d start– it’s called a bathtub because, you know, when it starts operating, you’ll get quite a lot of failures. Anything big, any manufacturing defects or anything are gonna cause failures quite fast, and that kind of drops off over time as all of those, uh, get addressed. And then you have, you know, the bulk of your operating life, it’s like pretty low level, pretty, pretty constant for a long time and then as you get towards the end of the [00:15:00] life, you start to see failure rates rise up again. That’s your fatigue failures, your end of life fatigue failures. And so when I saw the same types of cracks more and more each year, I’m like, “This looks like, you know, the foot end of the bathtub, not the head end.” And, uh, it made me worried and I’ve now seen that across a few wind farms in Australia at, um, hotter places. There’s a few blade types that are more prone to it than others, but at this point it’s still a suspicion that that’s what’s going on. I mean, a suspicion backed by a lot of, a lot of theory and knowledge of how the certification process works. But this project now we’ve got some funding to actually go put some sensors onto wind turbines, actually learn what the temperatures are in the blades throughout the whole laminate, um, not just the, you know, on the outside surface or not just the ambient temperature, but actually, you know, develop a temperature gradient across the whole, um, the whole laminate in the blade shell. Um, and [00:16:00] then we’re going to be doing a bunch of modeling basically to look at what is the effect of these different temperatures that blades are really seeing and how much would we expect to… that to decrease a lifetime. And then we should also be able to say, you know, if you have this issue in your wind farm, you might be able to change your operation a little bit and extend your lifetime a lot. Because this one, it’s real– like, in contrast to leading edge erosion, leading edge erosion is just, it’s, you know, every wind turbine has it to a certain extent, and it, it’s always there, but it’s a relatively minor cost to fix it. You know, like it sounds like a lot, like 30, $40,000 per wind turbine, but, um, you know, compared to if you’ve got to replace every blade across your fleet because they’re all, you know, at the end of their life after five years, you know, that’s obviously shocking. And, you know, that’s a bad example, but even in a y- you know, like a less extreme example, maybe [00:17:00] after 15 years you have to do a, you know, a f- a fleet-wide campaign to strengthen blades or something. It’s, you know, m- many millions of dollars for that, and so it c- could make sense to be able to learn, okay, what, what hours of operation should we be avoiding? Additionally, because when it’s super-duper hot in Australia, usually you’ve got heaps of solar power and the electricity price is not that high. So I, I think that there– and I don’t, obviously, before we’ve done the project, I don’t know what the threshold is. But in both cases, we will be aiming to improve the knowledge of how you can operate to avoid these periods of accelerated damage. Allen Hall 2025: Do you think you’re seeing more fatigue-like damage due to the blades operating when it’s hot or not operating when it’s hot, with maybe less airflow around the blade and maybe less cooling going on is just a temperature soak At rest? [00:18:00] Rosemary Barnes: Yeah. It’s interesting because the temperature is higher if it’s not rotating, um, because you get a whole lot of, um, convective heat, heat transfer when the turbine is operating. So your temperatures are not gonna get as hot when operating as when they’re standing still. However, if it’s standing still, they’re only very lightly loaded. Like, yes, they’re gonna get, um, blown by, by gusts and, um, have a little bit of bending, but it’s, it’s very, very small compared to, uh, if it is y- you know, operational loads. Uh, assuming that you’re not in the middle of a s- a storm. But yeah, a storm probably doesn’t come with 50 degrees temperatures.  Allen Hall 2025: And what part of the blade is susceptible to these higher temperatures? Is it the resin? Is it the fiberglass or carbon fiber? Or is it the, the glue, the bond joints? What part are you focused on? Rosemary Barnes: The resin is the main part that I’m focused on. It gl- it could be an issue for glue too, actually. I haven’t even looked into what the, um, yeah, temperature assumptions are with, with glue, with [00:19:00] bond lines. But the failures that I’m seeing in the field are not, are not bond line issues. It’s, it’s, um, a laminate problem. Allen Hall 2025: What about balsa and foam inside of the blade? Are they affected by the temperatures or are they pretty temperature stable?  Rosemary Barnes: I don’t think they’re affected at these kinds of temperatures, no. They, they don’t really do much actually. The, the core materials, like it, it is very important that they’re, that they’re there, but their job is really to keep the fiberglass separated from its- itself to make it stiffer. So, um, yeah, that’s, that’s unlikely to be a, a major source of problems.  Allen Hall 2025: So this study is gonna work over about three years, and you have a number of wind farms that are participating. Are you looking for more wind farms to participate in Australia?  Rosemary Barnes: Yeah. Yeah, definitely. I mean, we can, um, have as many as, as people want to join. We’ve got quite a good selection so far. Definitely can always welcome more. A, a bit limited in how many can get the really, um, good sensor [00:20:00]package, because the grant funding is a, you know, a certain amount, and that’s paying the bulk of those sensors. So, um, those spots are limited. So if anybody wants to really zone in on what is specifically causing erosion on their site, you know, if you know that you have got leading edge protection that is not good enough and you have to replace it soon, but you don’t know what to replace it with, then, you know, that would be the kind of wind farm that might want to consider, yeah, joining this and, um, you know, getting these sensors on their, um… We’re putting them on top of the nacelles, most of them. Um, yeah, so that would be a good match then. Um, and then, yeah, for the ones that are doing the SCADA data and, um, weather data- There’s not such a, a hard limit on how many we can have join like that. So yeah, we can have more, more like that.  Allen Hall 2025: In the temperature fatigue effort, i- is that still looking for participants or are there particular wind turbine types or manufacturers that you’re [00:21:00] looking for to participate? Rosemary Barnes: Yeah, I think, um, I, I mean yes, we can have more of those. That’s a simpler, a, a simpler issue as well. The sensors are not so expensive and, um, it’s, yeah, it’s a, it’s a simpler project to join that one. We only need, you know, a couple of turbines per site, so it won’t be such a, uh, an involved process to get everything up on into the turbines. And in terms of who might like to join that, I would say anybody that is in a really hot area where, you know, where they see a lot of days over 30 degrees, and if they see any days, you know, getting into the high 40s, then I would say that that’s worthwhile. Or even I have seen this issue in some milder sites, um, yeah, depending on the, on the blade type as well. It is more common with polyester resins. They have a, a lower op- uh, maximum operating temperature than epoxy resins. But then also just anybody that has noticed just, hey, [00:22:00] we’ve got a lot of cracks, and it seems like we’re getting more and more cracks every year, which to be honest, can be hard to keep track of if you’re… If you’ve got a full service agreement, uh, you know, an OEM managing your wind farm The early signs of this are gonna be category one and category two cracks. They’re not in exactly the same location. It’s, you know, it’s a tricky one. Normally, if you’re looking at a serial issue, then you’re going to have, uh, well, you know, your ideal pattern for a serial issue is the exact same thing happening over and over again. And so it is harder to pull this out. It also really would be very rare for it to be happening in the first two years or three years, whatever your serial defect liability period is. So it’s quite hard. But, um, another group of wind farms that might like to consider it is if you know that in, you know, a certain number of years you have to renegotiate your service agreement or, you know, it ends and you might have to take over yourself, then this’ll be a really good way for you to [00:23:00] understand, you know, have I got a ticking time bomb here? Um, because it’s not something that you’re gonna be aware of if you haven’t been, you know, doing some really, really in-depth shadow, shadow monitoring of your blades, you know, running your own inspections and looking at every single damage, not just category three, four, five, but lower ones. So yeah, I mean, there’s a, a wide variety of people that, that could be interested in joining. Allen Hall 2025: Are you expecting a number of manufacturers that make leading-edge protection or involved in resin creation, some– there’s a number of resin companies and a variety of resins that are used globally, sort of interchangeably at times. Are you expecting some of those companies to participate in this effort just to learn about the Australian environment? Rosemary Barnes: I think it would be a good opportunity to test out some products and see how they behave in the Australian context. I think that that would be a really good selling point, but I, I have to say that most of the companies doing that sort of thing that wanna enter Australia, they don’t [00:24:00] really consider… Like, from the perspective of wind farm owners in Australia, if you can’t show us wind farms in Australia where this has worked and, you know, show us a before or after, you know, the old LEP lasted Two years and our LEP is going on four years now with no damage. It, you know, unless you’ve got a before and after like that, you can tell us however many turbines that you’ve got installed around the world, but, um, we don’t consider it validated, y- you know? It’s not validated for Australian conditions yet. And I do have this same discussion over and over again with, you know, not just leading edge protection, but all kinds of, um, you know, manufacturers of whatever doodads that you put on to improve a, a wind turbine. It’s so different to Australia. Things break so fast. And I’m talking everything, you know, like vortex generators fall off and, um, yeah, like, uh, you know, bits of lightning protection systems fall off, seals just [00:25:00] crumble and disintegrate. Um, and it, you know, we’re very wary of, of new products. So I, I do– I mean, I’m thinking of it more from my client’s point of view than from the product manufacturer’s point of view. But one thing that I wanna get out of this pro- project is to be able to answer one of the most common questions that I get is, which is, what leading edge protection should I be putting on my turbine? And for now, I don’t know. I, I know a range of products that don’t work in Australia, and not much more than that. So, um, yeah. And it’s also, you know, Australia’s a very varied place with lots of different kinds of climate too. So it’s not gonna be like, you know, the product that works in Queensland is the same one that’s gonna work in Tasmania, which is the same one that’s gonna work in Western Australia. You know, um, so it, this project is gonna really pull out what are the site specific issues you’ve got at your site and what kinds of, um, you know, tests would we need to see a product um, perform in order to know that this [00:26:00] is gonna last on your site. Allen Hall 2025: W- what is the outcome of this project or these two projects? Are they gonna be reports or, uh, a, a continual monitoring system that’s designed for the Australian environment? How do you see this going?  Rosemary Barnes: Yeah, so one part of it is, um, developing a way to identify periods of accelerated damage and to know not to operate during that time. So we call it protective operation. Uh, so that would, uh, help you if, yeah, you’re trying to extend the life of something or increase the amount of time before you have to repair, then y- you know, that would be useful to have that knowledge. And it will be as simple as just an alert saying, “Hey, accelerated damage conditions. Consider, you know, if you wanna keep on operating.” And, you know, if the price of electricity is super high at that time, they may want to push through, and if it’s low, they probably won’t want to. So that’s one thing. Um, especially, you know, as wind turbines get to their, near the end of their life. I’ve got some clients whose wind farms only have, you know, [00:27:00] maybe five years operation left. They just simply don’t wanna repair their leading edge protection again. They just, they, they don’t wanna do that. So they would be happy to, you know, reduce operation a bit and have their turbine limp through to the end of the period. Y- you know, you want everything to wear out at once. You don’t want brand-new leading edge protection on a turbine that’s going to come down in a couple of years. Um, so, you know, that’s, that’s one part of it. And then the other thing is, you know, turbines earlier in their lifetime, how can we optimize the maintenance schedule with leading edge erosion? Um, so, you know, like it’s a lot cheaper to, uh, replace the LEP if you get– catch it early, but then you don’t wanna be catching it too early and replacing it, you know, constantly when you, you don’t need to. So, um, yeah, it, this, having this knowledge will enable a site-by-site operations and maintenance strategy with respect to leading edge protection. We also have some sites who are having trouble. They’ve got a full service agreement, and the OEM is [00:28:00] responsible for, um, doing the leading edge erosion repairs and protection replacement, but the owner is on the hook for paying for it. At the other end, we’ve got people with full service agreements where technically the, um, manufacturer is supposed to be doing the leading edge protection and paying for it, but they argue about what, when does it need to be done. Because, you know, um, the operator might think if there’s no structural risk, then we don’t need to be replacing it. And in the meantime, you’ve got turbines spinning around for years and years and years with, you know, these huge flakes of leading edge protection s- you know, causing the flow at the tip of the turbine to, to detach and to stall, and horrible aerodynamics, huge losses in power generation and revenue. And they’re having a big fight about, you know, is this necessary to do or not? And then, you know, they’re just gonna put the exact same product on again ’cause the [00:29:00] OEMs are re- all really, really wedded to their own particular brand. It’s like, “Well, last time we had this product and it was factory applied, it lasted one year before it s- it was worse than, you know, if it wasn’t there at all. Uh, we don’t really want you to put that one on again.” And so, you know, having the information that they need to be able to, you know, really bring data to these discussions and, you know, makes a, yeah, data not drama. That’s a, a good approach I think, um, for any kind of negotiation and especially in the case of leading edge erosion. And then for the high temperature fatigue part of the problem, aside from, you know, just wanting to know are your blades aging, should you be looking at remediation action or changing the operation, the other really big key thing is, uh, you might need to have a fight with y- your OEM about if this turbine has been designed and operated correctly. And so then having the data from this, um, project is going to give you the information that you need to come into that [00:30:00] argument with, again, the data not the drama. Um, and to, you know, in- increase your chances of succeeding in that kind of really tricky negotiation.  Allen Hall 2025: So if you’re an OEM or a manufacturer of equipment, an ISP, an operator, pretty much all aspects of wind operations, you probably ought to be getting a hold of Pardalote Consulting and Rosemary to talk about the opportunity to participate in this study. How do people get ahold of you to, to do that?  Rosemary Barnes: People can go to our website, pardaloteconsulting.com, and get in touch via the contact form there, or you can, uh, look me up on LinkedIn, Rosemary Barnes. That’s probably the easiest, fastest way to get ahold of me personally.  Allen Hall 2025: Well, Rosemary, congratulations on the Energy Innovation Fund Awards and the new three-year effort. If you are interested in participating with Pardalote Consulting and working with Rosemary and her team [00:31:00] in Australia, reach out to her on LinkedIn and get that process started, because this report and the data from all this analysis that’ll happen over the next couple of years will be important to the wind industry. So you need to spend some time and get ahold of Rosemary and get this process started now. So Rosemary, congratulations. Uh, thanks for being back on the podcast, and looking forward to, uh, the next couple of years. It sh- should be exciting.  Rosemary Barnes: Thanks so much, Allen.

Pre-season is underway. Day ONE in Girona. And we have a new away shirt. Oof.Support Latte Firm for the price of a coffee a month and enjoy ad-free shows, bonus content and access to giveaways and match tickets - patreon.com/lattefirm.

Tech issues again  this evening. Oof.  Whiskey Pete gets hammered under cross examination by Jon Ossoff in the Senate. Mike Waltz, using the same talking points, gets the same treatment in the House.  Another MAGAT pedophile gets busted . . . and your 'umble 'ostess actually knows him and called it years ago. 

Ok so today I am sharing all about my health journey. I am going to share as I learn and grow through this journey. I want to share because I believe many many women go through the same thing but they just accept it as the new "norm". It's not. Its our body trying to tell us something. Or its perimenopause! OOF!  If you are going through the same thing- let's connect! Stay tuned on how this all plays out!

In recent months, the open vs closed, and US vs China discussions on model ownership and sovereign/local AI have heated up to a fever pitch. So it is very very good news that Poolside AI are finally emerging with new models, like Laguna S 2.1, that are beating Thinking Machines' recent release nearly 10 times their size.Poolside's recent tech report got a lot of praise due to their level of detail, and Vibhu first covered Laguna's recent technical report on our paper club:From spending $12 million building language models for code before the world cared to creating a Model Factory that can take a model from pre-training to release in eight weeks, Eiso Kant has spent more than a decade betting that code is the path to AGI. In this episode, the Poolside co-founder joins swyx and Vibhu to explain why ChatGPT felt like vindication, why Poolside embraced open weights and open research, and why he would rather live in a world with 100 foundation model companies than five even if Poolside were one of the five.We go deep on Poolside's Model Factory: the engineering systems behind 10,000–20,000 experiments per month, streaming data directly into training, reproducible experimentation, low-precision compute, and agents that increasingly write code, launch jobs, evaluate results, and modify the pipelines used to train future models. Eiso also unpacks their recent launch Laguna S, why persistence, verification, and backtracking may matter more than raw intelligence, how much capability remains inside smaller models, why reinforcement learning will move earlier into pre-training, and why next-token prediction is still extracting too little from the web.We also discuss model-harness co-design, Poolside's path from coding agents to AGI, why Eiso thinks MCP and traditional tool calls are “stupid,” the real economics behind frontier-model training, Poolside's $500 million raise, open-source AI, regulation, NVIDIA and TSMC's influence, engineering productivity in the agent era, high-agency teams, and hiring at Poolside.We discuss:* How Andrej Karpathy's RNN work inspired Eiso to start building language models for code in 2015* Why Eiso spent four years and $12 million pursuing an idea before the market cared* Why ChatGPT felt like vindication and brought Poolside back to open source* Why Eiso would prefer 100 foundation model companies over an oligopoly of five* The difference between releasing open weights and publishing genuinely open research* Why Poolside deliberately built a global research organization outside the Bay Area talent war* Why model building is ultimately 90% engineering* The Model Factory: Poolside's end-to-end system for rapidly training and improving models* How fewer than 70 researchers run roughly 10,000–20,000 experiments each month* How Poolside moved from six-month model cycles to five- and eight-week launches* Why streaming data directly into training unlocked faster experimentation* How immutable data, versioned code, and reproducibility enable rigorous model research* Why Eiso wants capable researchers to leave their labs and become Poolside's competitors* Why 95% of model building can be reduced to better data or compute efficiency* Laguna S and why persistence, verification, and backtracking can outperform raw intelligence* Why smaller models may handle far more knowledge work than previously expected* Why reinforcement learning will move earlier into pre-training* Why next-token prediction is still failing to extract enough knowledge from the web* Why distillation and environments have become the AI industry's favorite “drugs”* Why mid-training is really an early form of curriculum design* Low-precision training, networking bottlenecks, and the next gains in compute efficiency* Laguna S: 118 billion total parameters, 8 billion active, and eight weeks from training to launch* Why model builders can often evaluate a new checkpoint within its first 30 minutes* Model versus harness: where agent capabilities actually come from* Why Poolside sees coding and long-horizon software tasks as a path to AGI* Why Eiso thinks MCP and traditional tool calls are “stupid”* Why future agents will write scripts instead of choosing from dozens of predefined tools* The case for minimal harnesses, containers, and model freedom* Why Poolside is prioritizing vision but does not expect to work on audio soon* Why language may be the most compute-efficient modality for encoding knowledge and reasoning* The real cost of model development and why the final training run is anticlimactic* The story behind the Poolside name and why it represents refusing to lower ambitions* How Poolside raised $500 million while investors still questioned whether AGI was real* Why intelligence could become the world's most demanded and commoditized resource* When open models may become too capable to release without restrictions* Why unilateral AI safety does not work in a globally competitive environment* How regulation could accidentally lock in an oligopoly of two or three AI companies* NVIDIA, TSMC, and the hardware systems underpinning foundation-model progress* Why reinforcement-learning wall-clock time is one of Poolside's biggest bottlenecks* Why Poolside trains models from scratch instead of simply distilling larger models* How AI changes the way companies should measure engineering productivity* Why agency may become the most important quality for employees in the AI era* How leaders align high-agency people through shared goals and clear constraints* Hiring across research, post-training, pre-training, architecture, evals, and engineering at PoolsideEiso KantLinkedIn: https://www.linkedin.com/in/eisokantX: https://x.com/eisokantPoolside: https://poolside.aiTimestamps00:00:00 Introduction00:00:54 Karpathy, RNNs, and Building Code Models Before Transformers00:02:26 The $12M Failure and ChatGPT Vindication00:03:39 Open Source and the Case for 100 Foundation Model Companies00:09:22 Open Weights, Open Research, and Poolside's Global Team00:16:04 The Model Factory: Why Model Building Is 90% Engineering00:20:19 Agents, Automated Experiments, and Early Signs of RSI00:24:04 Streaming Data, Reproducibility, and Scientific Rigor00:30:35 Creating More Foundation Model Companies00:36:07 Laguna S: Persistence vs. Raw Intelligence00:43:01 Reinventing Pre-Training, RL, and Curriculum Design00:52:33 Low-Precision Training and Squeezing More From Smaller Models00:58:37 Model Harnesses, Coding Agents, and the Path to AGI01:09:26 Why MCP and Traditional Tool Calls Are “Stupid”01:13:04 Vision, Multimodality, and Why Language Still Matters01:18:15 Scaling Models and the Real Economics of Training01:20:40 Why Poolside Is Called Poolside and Raising $500M01:27:37 Open Models, AI Safety, and the Risk of an Oligopoly01:33:53 NVIDIA, TSMC, and the Reinforcement-Learning Bottleneck01:41:52 Smaller Models, Distillation, Engineering Productivity, and HiringTranscriptIntroduction: Eiso Kant, Poolside, and Open ModelsSwyx [00:00:00]: All right, we're here in the studio with Eiso Kant from Poolside, together with Vibhu. Welcome.Eiso Kant [00:00:08]: Thanks. Thanks for having me, guys. Good to be here.Swyx [00:00:10]: Yeah, fresh on the plane. You texted me, you were like, “Hey, I'm on my way to SF.” I was like, “You're on a plane right now, right?” Like, hey.Eiso Kant [00:00:16]: I know. After I texted you, I realized that probably coming in with major jet lag was gonna offer some fun experiences today, but let's do it.Swyx [00:00:23]: I mean, I think the thing I would tell guests is that they don't have to prepare that much because if you're truly working on this every single day, then even, like, what you hazily remember is going to be new for a lot of the audience that don't live in your world every day, right? so 10 years ago, you did a talk at Google Slush, talking about the democratization of AI. and, now here you are, like, open sourcing an incredible new model that we're gonna talk about. But I guess, like, what got you into democratization of AI? Like, it's not obvious from your LinkedIn or something.From Karpathy's RNN Post to SourcedEiso Kant [00:00:57]: No, it's not at all. I don't think it's obvious how I got in this space. I owe getting into this space to Andrej Karpathy.Eiso Kant [00:01:05]: In 2015, he wrote an article called “The Unreasonable Effectiveness of Recurrent Neural Nets.”Swyx [00:01:10]: Neural Nets, yep.Eiso Kant [00:01:11]: And that article, I read it, and I pivoted my startup at the time overnight to working on RNNs, and later LSTMs and Transformer models to be able to write code. If you go to this article and you scroll down, you can start seeing, like, this was the precursor to what ended up becoming language models. So, at least when he was character-level language models that were starting to predict letters, he has an example out here. There's a little Paul Graham generator, and you can read it, and the text makes sense, but it doesn't. and there's a little-- There's an example of code a little bit further down. Yeah, so Shakespeare.Swyx [00:01:47]: Shakespeare.Swyx [00:01:49]: CoolEiso Kant [00:01:49]: And for some reason, I read this, and I went down the rabbit hole of learning everything I could about RNNs and LSTMs, right? This is Transformer paper. And I had built a completely unreasonable belief, that neural nets should be able to generalize to anything and everything, and that language should be able to generalize, to a lot of things that are intelligent and the ability to write code. And so I started building Sourced, which was a fully open source company trying to build, what we used to call machine learning on code, language models on code. And we spent about four or five years on this, till the end of 2019. And that sounds really cool today, but back then, no one cared.Eiso Kant [00:02:29]: Right? Like, no one cared. We were in the dark. Like, we did things along the way. We tried applying convolutional neural nets to, like, the structure of code. We were. when attention came out, we were applying it to LSTMs, and then the Transformer paper came out. And it - it wasn't obvious, and what we missed throughout that entire journey, that we were on the right track, but we should have just kept scaling up. And today, to all of us, the scaling laws and scaling up seems like the most obvious thing. But having spent four or five years of my life on working on language models on code, it wasn't obvious. So I have a lot of respect to folks at Google and OpenAI and others who took that confidence and kept going. we failed ultimately at the time, and it was, like, biggest failure of my career, right? You blew $12 million of investors' money, which was a lot back then.Swyx [00:03:18]: Yep.Eiso Kant [00:03:19]: You spent, still a lot, but, And you spent years with, like, a group of 40 people just obsessing over this problem. And life took a different turn, And it was, and family became a focus, and I kept my heads down and really, didn't really look at language models for the following two years. big mistake considering Following years are gonna be really interesting. And then ChatGPT came out And it was like a vindication. It's like people started texting me. I found, like, my old, work decks and these old talks. And throughout that whole journey, we,ChatGPT, Vindication, and Returning to Open SourceEiso Kant [00:03:56]: We really had a strong point of view at the time that, like, as you're building more capable intelligence, it should be open and open source.Eiso Kant [00:04:04]: When we started Poolside, that wasn't the case at all, and I wanna be very open about it. When we started Poolside, we were like, there was a premise of two things. One is this technology is not gonna stop compounding in capabilities. I think to most people obvious today, but three-plus years ago when we started, most people were still arguing if these were stochastic parrots or not.Eiso Kant [00:04:23]: And the second was that reinforcement learning was gonna be the biggest driver for LLM capabilities. Today, very obvious. Three years ago, was not an opinion held or direction held at either OpenAI or Google or Anthropic or others. And so people looked down on us a little bit. They were like, “ is this really gonna work?” And so we just started working the problem, and we never really thought about open source again. We just kept our heads down and we built our, like, knowledge, understanding from scratch, right? We didn't roll out of an existing lab. So we picked up the papers and started writing code and figuring things out.Eiso Kant [00:04:59]: And it wasn't until the beginning of this year that me and my founder, Jason, picked up the open source conversation again.Eiso Kant [00:05:07]: And if you go back to some of the early things on our website, it was very straightforward. It was we wanna get to AGI, we wanna support a world of abundance, and we wanna be the first company that gets there.Eiso Kant [00:05:20]: But we started talking at the beginning of this year because it became obvious that the world was going in a direction that was starting to like, pick at us a little bit. Like, it didn't, this didn't happen overnight. It was, like, a little bit we were seeing this and we're like, “Okay, The world's going down a path.” And Throughout this journey, there was something that I used as a, as an analogy or thing. So I said well, if I go back to back in those days, 2015 or 2016, we're working on this, and I picked up a fi book off the shelf, and I was reading the book about 2035. AGI is achieved, and the story would be over the following, decades. And it would have that first chapter where everyone's trying to figure things out. You'd get the chapter of ChatGPT coming out And then you would get to the chapter where the world was at a fork in the road, and the one that it picked was one where three or four or a handful of companies were going to create all of intelligence moving forward.Eiso Kant [00:06:21]: And when I thought about that story, it felt like a dystopian fi book, not a utopian fi book. And the reality is, I'm a utopian fi guy. Like, and so We took a step back and said, “Hey, can we play a role here?” Now it was easy for us to do so because we were not at the frontier.Eiso Kant [00:06:41]: If we were at the frontier, I don't think we could have changed our mind. and I don't mean this like it's when the moment there's too much capital involved, too much expectations, you've built up things, right? We're a small team, just improving and improving. And so we knew that we could make that decision now, but it would be a lot harder to make as we got closer and closer to the frontier and caught up to others. And did a lot of soul-searching and a lot of conversations, and said, “No, this makes sense,” Even if there's big unanswered questions, like how the hell do you build a business model with foundation models about open source? Big open-ended question that we do not fully have the answer to yet, right? At what point do you no longer wanna release open source models because misuse of models has, real potential risks associated with it? how is the government gonna respond to open source? but I think it all just came down to one thing, and I'll stop the monologue, is the fact that I rather live in a world that has 100 foundation model companies than a world that has five, even if I was one of the five. And the smallest and most meaningful contribution we can make for 100 to exist is to open up our research and open up, like, our weights right now and figure out along the way how we can, like, do more.Neo-Labs, Model Choice, and the Token EconomySwyx [00:08:01]: Yeah. I think if anything, over the past three years, that has become a bit more true. you are one of a cohort of Neo labsEiso Kant [00:08:10]: YeahSwyx [00:08:10]: That people are now calling that. And, we're, we're doing this on the day that Thinky launched their, new model and you are outperforming them on their, on some benchmarks that they released, right? Like, they just don't have it yet. so it goes to show that I think, like, this is one of those things where, like, there is room for multiple players, and you are seeing a little bit more of the future. Maybe more like 20, not 100, but, like, you are one of the 20.Eiso Kant [00:08:36]: I really hope so, right? I think we I'm, I'm excited about their release, and I'm excited about everyone releasing because, like, ultimately, like, choice competition is both gonna drive progress in the right direction. But the fact that like, we create models and while we all, drink out of the same well of data effectively, we do introduce very different behaviors and biases in our models. Some are intended biases, some are completely unintended biases.Swyx [00:09:03]: Yeah.Eiso Kant [00:09:03]: And if we shape up in an ecosystem in the world where open models are gonna be a part of the token economy, like, I don't think there's any question about it anymore Then we want to be able to live in a world where companies, countries, people can choose and say, “Hey, I am most aligned and I trust most this provider for these things.”Swyx [00:09:25]: Yeah.Vibhu [00:09:26]: I think more than just one of the 20 Neo labs, up until recently, most of open source innovation was coming from the Chinese labs, right? So there's the DeepSeek of the West. Is it today? Okay, maybe it's thinking machines reflection, but there aren't many, right? So, one of the things you guys started in France, Europe, but very much now you're taking that American standpoint and more than just that, the point is the Chinese models that we see, they're not super open research. the work you put out is, I think, some of the best. So every few months you get not only frontier models, but also here's a breakdown blog, paper, technical report of here's everything for state of the art to build, frontier intelligence and you're filling that gap too, right? So not just only open weight, not just Western, but also pretty open research.Open Weights vs. Open ResearchEiso Kant [00:10:20]: No, I appreciate it. Look, I think it's, I think it's the most meaningful contribution, right? Weights are a binary. Let's call them what they are. Yes, we can modify them, we can change them, but, like, giving someone the weights does not allow them ultimately to recreate what you're doing, right? And so now there's challenges around releasing data sets, challenges around like releasing certain things, but being able to share your research, like, right, how do we do it? What are the lessons we learned that we spent, tens of thousands of experiments of compute on? I think very much so. One correction though, Vibhu, and I say this because it's been haunting us for quite a few years. We from day zero were an American company.Swyx [00:10:55]: Yeah. They movedPoolside's Global Team and American Company StorySwyx [00:10:56]: To France.Eiso Kant [00:10:56]: So the story once and for all is very. We start as an American company. We have always been an American company, and early on we made a very conscious decision. We said, “We're not gonna hire any researchers in the Bay Area. We're gonna look for talent everywhere else in the world.” and that is everything from Middle Americas, Seattle to, Serbia, and to Taiwan and Singapore and other places. And it was because we took a view that this was gonna become a talent war for this, and I think it has over the years now. Three years ago, that wasn't fully obvious yet. I think today it very much is. And we also realized that, like, some of the world's most capable people with, like, the most interesting, innovative ideas were not just gonna be here. And so it led us to create like a fully remote company. and we ended up opening an office in Paris and London and different places and we have a lot of the team in the US and a lot of team outside. But we always took this view of like, we're an American company, but if we want the best of the best to work with us, we need to take a global view. Now we do also have people here in Silicon Valley, like the company's grown and others, but I think one of the things that, it slowed us down at the beginning, but it has sped us up now, and it's why you're seeing like the progress, I think, on our models and the cadence at which we release, is because we didn't roll out of an existing lab. Right? we didn't, we didn't have a lot of the information that's freely flowing around here at the time. We just took this point of view as like, “Okay, well, let's just work the problem. Let's just go and, like, read the few papers that are out there, and let's just figure this stuff out.” And we made some hilarious mistakes in model training because of that over the yearsEiso Kant [00:12:35]: Like especially in the first 12 months. there's a few that I think still haunt me and scare me. We can talk about them later. but it created a, like, a resiliency and persistency in the team, right? with extremely few people have left us over the years, that, like, told us, “Okay, we can do this.” When we first wrote our first training code base completely from scratch, it wasn't a fork of any open source. It was just like, “Okay, let's build it from scratch.” I remember we had this one moment where we spent three weeks working out an optimizer bug. Like, it was like training just couldn't get stable. We, like, obsessed over it, and we thought, like, maybe we were wrong. Maybe we should have just forked this repo, or we should have. But then when we solved it, I still remember at the time we were like five people in the company. when we solved it, we were like, “Oh, we can do things,” like if we're just willing to work hard. and I think that culture with a very strong engineering bias has helped us, like, get to where we were. And so there's this notion of open source and talent and these things. I think we, We just took different decisions from a different starting point. and I think we are lucky. I do want to definitely call it lucky. And there was a lot of hard work at the team that now, like, that's starting to show up in results.Swyx [00:13:52]: Just ‘cause we probably won't revisit this again, but, and this is a fun recruiting challenge if someone knows the answer. What was the bug? And then we won't tell the solution, but we'An Optimizer Bug and the Value of Building From ScratchEiso Kant [00:14:01]: So the - This - You're gonna test my memory here,Swyx [00:14:04]: Oh, okayEiso Kant [00:14:04]: So but I thinkSwyx [00:14:05]: DirectlyEiso Kant [00:14:05]: I think I can recall. So if you, so if you look at, So if you take like Adam as an optimizer, you have epsilonSwyx [00:14:12]: YeahEiso Kant [00:14:13]: Which is, right, like in the denominatorSwyx [00:14:14]: Momentum and weights. YeahEiso Kant [00:14:15]: Is exactly, in the denominator. And at the time, if I recall, you looked at like the early Llama papers and things like that. People were juicing epsilon, like, quite a bit. Like, they were, like, adding, I don't know if it was E minus four or whatever, like a high value for epsilon.Eiso Kant [00:14:31]: And if you think about this during training, it's like a bit weird and counterintuitive that we're adding noise to our optimizer by just adding effectively, like, a random number in the denominator, right? Like behind the decimal point. And I don't recall the exact bug, but it had - What I remember is once we solved it, we no longer had to juice epsilon as much as, like, was happening in the Llama paper and other places. and it was like one of those fundamental moments where we had trusted this paper that was out there, and we're like, “Oh, no, it has to be this way. It has to have this high value of epsilon.” But it made no sense to us intuitively. Like, why do you have to have this so high? Like, if you're just trying to avoid division by zero, why can't the value be extremely small? and that was like one of those moments where you realize like, okay, finding things out from scratch yourself builds a better intuition. Because the one thing you learn very quickly with model building is that your intuitions that you start with are gonna get beaten up so hard.Eiso Kant [00:15:33]: Right? Like - It's such an experimental science, that the things that seem obvious, you very quickly get to learn, like, you were wrong, and hopefully you figure out why, and sometimes you don't even.Swyx [00:15:45]: Yeah. yeah, so, one of the reasons that you, when you released your new models, Vibhu got really excited. I mean, everyone got really excited. But Vibhu led our paper club on it, and you guys sawEiso Kant [00:15:58]: YeahSwyx [00:15:58]: Obviously. maybe talk through some lessons learned in that, whatever you can disclose. we can focus on the model factory stuff, whatever you think is a good starting point.Model Building as EngineeringEiso Kant [00:16:08]: So I would say that our view from very early on in the company was that model building is ultimately 90% engineering.Eiso Kant [00:16:18]: And I think we all know it in the industry because if you look at where's every researcher spending their time, they're spending their time writing code, right? Looking at data and writing code. And so we said, okay, The state at the moment, like three years ago, was bash scripts and Slurm and spaghetti code bases for training and, like, data pipelines that were patched together. And we looked at this and said, “Well, ultimately, model building is a process.” You're going from raw data, right? Like training raw material, the web, et cetera. you're doing a whole bunch of filtering, cleaning up, transformations, analyzing. These days, that's, far more complex than it was three years ago. then you're training a model, which is effectively a large distributed systems problem, right? Across hardware that has still-- It's become a lot more reliable. It was extremely flaky back then. and now with every new generation, we get our new sets of challenges. And then you go into the next stages, right? There was no training back then, but, like, you got, your post-training and then your reinforcement learning. And so we looked at this and we said, “Well, this looks like an industrialized process. This looks like an end process, that every single part of it has its machinery,” right? If it's your big data pipelines, if it's your crawling ingestion of the web, if it's your, large-scale distributed training, and then you've got your reliability. And we said, “Well, why don't we take some of the world's smartest distributed systems engineers that we knew and make them part of the process of research from day zero?” Not retrofitting it later on, but, like, really from the beginning. And that became our model factory. And so our model factory started with a handful of components. Today, it's thousands of components, and I try to equate it to, if you think about, like, someone who was at the very early days of Foxconn, if they had been there for the following, decade, they would be able to rebuild Foxconn because they saw every decision that led to building that system and all the complexity. If you and I walk into Foxconn today, no chance.The Model Factory and Experiment VelocityEiso Kant [00:18:18]: Right? Because we don't have the lineage and history of decisions that led to that. And so we built early on from the beginning- with a team that really understood that, well, the metric that we are optimizing for is the speed of an idea from a researcher to an experimental result that we can trust to then being part of the next model training.Eiso Kant [00:18:42]: And in the. And because it's such an experimental science, ultimately, in the beginning when it wasn't that complex, you could patch your way around it, right? But now, at any foundation model company, you are running. I mean, we're a small team, right? We're less than 70 researchers, another 35 engineers. and we are running, I haven't checked the latest count, but far more than 10,000, maybe 10 to 20,000 experiments a month that we cut. And so if you look at that scale of every model run that is, like it's ultimately it's, it's you need to be able to trust it as an infra problem. And so what we have now done over the years is gotten really good at that, and just by working it and improving it and obsessing over those end decisions. So now what that means is that you looked up Laguna XS 2 that we launched. It was five weeks from the beginning of training to launch. The model that we're gonna talk about today was eight weeks from start of training, to launch. We started the next model literally yesterday because we now finished the post-training required for the model we're launching, next week or by the time this comes out today. and we move that compute to the much larger Laguna M model that we're now training. And so the model should be an artifact of someone's process. It shouldn't be really a thing in itself. Like, and we treat this like the way you would look at like a SpaceX factory where, yes, the first rocket, really hard to build, but the much harder challenge was building the factory. And now they're rolling off, and no one is really thinking about the next launch anymore. So it's just another launch, it's another launch, another rocket comes off. And that's what we're trying to do with model building.Eiso Kant [00:20:22]: And what has been, which was not planned from day zero, it was in the back of our mind like this will happen one day, is that when you build a really good end model factory with really good APIs and really good engineering systems, Well, what is it perfect for? It's perfect for agents.Agents Inside the Model FactoryEiso Kant [00:20:40]: Because agents are now starting to take over more and more work in our model factory.Vibhu [00:20:43]: Yeah.Eiso Kant [00:20:44]: So I look at the screens when I walk, like when we're, we come together, in our monthly, we do monthly onsites, and I walk behind people's screens and I stop by and I talk to our researchers. And the default is all of these different agents running on their screen that are writing the code. They're launching the jobs. They're evaluating the results that are coming back from the model runs. They are, making the changes. And we're still in the driver's seat. We're still coming up with the ideas. We're still helping with the debugging. But more and more, and this is right now very profound on the data side of our pipelines in both pre and post and the synthetic data pipelines, it's starting to become more on the architecture side as well. You're starting to see these twinklings of what RSI is gonna look like.Eiso Kant [00:21:27]: And that's. So when we talk about, like to your question about our models, every talk about the model factory, And my coolest example of these things is always that when we kick off a new run, doesn't matter if it's a training like big run or if it's now a post, like one of 10 post-training versions we do for like release or many experiments, is that at any given moment, the changes that somebody made that they had experimental results from the day before make it into that run.Eiso Kant [00:21:57]: So there's not like a cutoff 90 days before. Like no, it's like literally from that moment because we can now trust the machine enough. And then you also have to invest in the reliability. So one of my favorite metrics about like Laguna S is that there was no call events, Right? Like completely zero. And we haven't had a meaningful call event, like something to wake up for, as far as I recall this entire year. now there is one asterisk to that. In usually the first six hours of launching a new model run, something breaks because you set a config wrong, you made a small mistake, et cetera. So that's usually there's a little bit of intervention, but that's always within like call periods, right? Not on call. And I think that's starting to now compound. So the model we're releasing now, I love it. It's amazing, but we're already onto the next one. and I think that's the way it should be.Laguna, Five-Week Builds, and Zero On-Call EventsVibhu [00:22:50]: Hey, I also just wanna point out, so for context, this was like a month ago. we found it in the tech report, so we just came in with, “Okay, new model's dropped. Haven't heard about it.” We wereEiso Kant [00:23:02]: Yeah, we're very used to doing this every few months.Vibhu [00:23:03]: We're, we're very much like, “ okay, look, it's like, on par with Kimi, DeepSeek, whatnot, the small ones, Gemma level. Oh, it's a very cool paper on what goes into building.” And then we hit this page, right? Like literally page two of tech report is, “This process allowed us to build the small model from scratch to delivery within five weeks applying the lessons”. And then I'm like, oh, this paper is not about here's a tech report of benchmarks and here's how many tokens it was trained on. Like for people that wanna dive more from what we're not gonna discuss on the podcast, it's all laid out here, right? FromEiso Kant [00:23:38]: YeahVibhu [00:23:39]: Custom software that agents can use to interface with training code, training data.Eiso Kant [00:23:45]: Yeah. Well, link the paper correctly, so yeah.Vibhu [00:23:47]: Yeah. All that stuff. read the paper here, but,Technical Report Principles and Streaming Training DataEiso Kant [00:23:50]: But I would like to. I love principles, and I think that is a good starting off point for maybe telling some stories. Maybe we can go one by one past the principles. I'll just call out that Dagster just got bought by a Prefect.Vibhu [00:24:01]: Yeah.Eiso Kant [00:24:01]: Isn't it fun? But yes, I'm very familiar with Dagster. just anything where like they trigger some story.Vibhu [00:24:07]: So, well, I would say, well, experiments code's obvious, but I think one of my favorite things is, I don't know where it is in here, but early on, and I still think this is the case a lot of foundation model companies, people prepare their training data sets, they get packaged up, then they get copied over to a training cluster distributed across all of the nodes, and then training starts.Vibhu [00:24:30]: And we looked at this like three years ago and we were like That makes no senseEiso Kant [00:24:36]: You lose so much time because the moment you have to rematerialize the data set, you have to make a change, you have to fix something, et cetera, you've got all this time of like repackaging it, right? Toca- tokenizing it, repacking it, moving it over to a cluster, then distributing it across the nodes. The bigger your clusters are, you start using fancy like torrent-like algorithms to like distribute your data. So why aren't we streaming data into training? Right? Something that's very common and like just basicVibhu [00:25:00]: Like just in timeEiso Kant [00:25:01]: Just in time, like good computer science like principle. And that was one of the first things that I think unlocked - the model factory. Because the moment you start thinking about, well, a training job, it doesn't matter if it's a big hero run or a small like, post-training experiment, consumes a certain number of tokens per second, right? And it's not a lot, right? From a like a data, moving data perspective. So we said, well, we have our training cluster, and then we've got like our AWS kinda setup where we can build these amazing big data pipelines. We can set things up. We use Spark underneath the hood, like all these things.Vibhu [00:25:36]: But when you say AWS, it's not actual AWS, it's your internal AWS.Eiso Kant [00:25:39]: It's our internal-- No, it's our internal like just running like our infrastructureVibhu [00:25:42]: Site web servicesEiso Kant [00:25:43]: Exactly. Our stuff running on like an AWS account or on like any hardware, right?Vibhu [00:25:47]: Yeah.Eiso Kant [00:25:48]: And so once we made that shift into I can stream data into training, all of a sudden you realize a lot of things unlock. Because now you don't have to wait for the whole data set to materialize.Immutable Data, Experiments as Code, and Scientific RigorEiso Kant [00:26:00]: You now all of a sudden when you're running data experiments about mixing data, it's a config. Because you've got these data sources that are coming in, and you just - we have this service called Blender that's in the report, where we then say, “Okay, for this run, I want 20% of this source, 10% of this source. I want this much, so many epochs of repetition. I want this to be, shuffled in a certain way,” and your training job can start while the rest of the data is even still materializing. also what it does is because all of this underneath-- So for us, we treated the data layer underneath as like an immutable data layer, and that was really important. Like experiments as code, immutable data layer means that you can always go back and understand literally down to the single token at which cursor it went in on which version of the code.Vibhu [00:26:47]: Yeah.Eiso Kant [00:26:48]: And it took us a I have to admit, like the first year of Poolside, we understood that engineering had to get great, But we didn't understand yet, that this is ultimately in support of like a good rigorous scientific progress. We were quite a - We were a very small number of people, so a lot of it was YOLO ideas and YOLO runs.Vibhu [00:27:08]: Yeah.Eiso Kant [00:27:09]: And we built great infra for the YOLO runs. But once we realized that we treated data as immutable and code as always versioned, and you could always track and trace every experiment end to end perfectly, you could repeat everything perfectly, right? You have perfect reproducibility. I can still reproduce runs from two years ago if I wanted to, right? It enables the scientific progress, like the scientific process, and I think that took us probably about a year and a half into the company to figure out. We also had some great hires, like our head of applied research, Nikolai, who joined us from Yandex, who'd been working on language models since like the early 2020s, I think brought that into the company of like, “Hey, we wanna have even more rigor.” And then once we kinda had the combination of like increasingly more capable platform that allowed people to do more, but had this immutability, we were able to start “Okay, every experiment is truly an ablation. We truly need to understand it.” And I think we became much more scientifically rigorous in the last couple of years, and the infra underneath enabled it. and then there's just fun stuff like, andVibhu [00:28:16]: Yeah, a lot of it's fun, like even just the, one, you share all the ablations, two, picking the data sets, right? There's like a random small paragraph in here where it's just like, “Oh yeah, training data, we have some, we have an auto mixer.” it trains eight small models, scales them up, picks the training data set. We don't even need to look at it. I'm like, “Wow, a lot of engineering rigor there.” And there's just, there's just a lot in here.Publishing Research and Giving BackEiso Kant [00:28:40]: Yeah, and it'- and look, and we wanna put out more. Like we, We treat writing papers as something that we haven't earned the right for yet for a long time. So you earn the right to spend time, publishing research once you're at the frontier, because until then, you're catching up, and every minute and hour in this industry matters. Like I obsess over it, not just the wall clock time from idea to result, but just general like time every day that we, waste is one that doesn't allow us to catch up. But in this case, we said, “Okay, we're gonna give ourselves.” I think we gave the team like three or four days while still doing their work, like give everything in there. And to your point earlier, if your stuff, it's easy to like put it out. And so there's so many more things that we wanna talk about over time, and we will definitely start doing. And as we earn more of the right, but also now have like added to our mission that we want more foundation model companies to exist, you'll see us like be way more proactive, and just trying to keep dropping some of those like things that we've learned along the way that can help others like speed up.Vibhu [00:29:40]: Which is the other cool side of this, right? It's, it's not like, back to your point, it's not just here's the benchmarks of our training. If you want to replicate, here's experiments of optimizers, data sets, post-training. you lay out a lot of it here alongside here's your system for how to do it? So it's, it's really like promotingEiso Kant [00:29:59]: No, thank youVibhu [00:29:59]: Other people can do the same.Eiso Kant [00:30:00]: And by the way, I also wanna make clear, right, we have been incredible-- Like we've taken a lot of advantage of the fact of all the open research that others have published, Right? And you mentioned, the Chinese labs, and we I think it's important that there's, from every country and every culture and background, including like Western companies like us, there's different models that come out that people can choose to trust. But I think we do have to give credit where credit's due, right? The incredible Chinese lab have done an amazing job at sharing their research, and we have definitely like been on the receiving end of taking advantage of that. So when you're on the receiving end of something coming to you, I think it's, you also have an obligation to give back.Swyx [00:30:39]: Do you have a favorite or underrated Chinese lab that you wanna shout out? Everyone shout outs DeepSeek.Chinese Labs, Zhipu, and PersistenceEiso Kant [00:30:44]: That's a good question.Swyx [00:30:45]: Moaan obviously for Therapsi. Yeah.Eiso Kant [00:30:48]: Yeah, look, I think, I think obviously everyone's been talking about Zhipu lately, with 5.2. I think what most people don't realize is when they started.Swyx [00:30:59]: Yeah.Eiso Kant [00:30:59]: Right? They started years before ChatGPT.Swyx [00:31:02]: They just rebranded. YeahEiso Kant [00:31:03]: And so, I've like, I remember how hard it was to work on these things Before the rest of the world got excited about it. And so I have an immense amount of respect for people, who were working on improving models when it wasn't the sexy thing to do, when believing in LLMs, was gonna get you ridiculed. I remember like back in 2016 when we were doing what we'd call, machine learning on code with some of these models. we would-- people would just laugh at us, like they'd be like, “This makes no sense. Like why are you wasting all these, like, millions of dollars on trying to figure this out?” And so I would say they're probably the one that, I think deserves a shout-out, not just because their latest model is very good, but because they fought to get here. And I think, I think every foundation model company it takes time to get here, right? It took us three years to get to the model that we're, that we're now gonna be releasing. and now the time in between the models is coming, is counted in weeks. It's no longer counted in months or years. But this stuff's hard. and if we can make it a little bit easier for the next person, like we should all do so. Because if we don't do so, we're, we've got a small window before models are really impacting recursive self-improvement to a level where catching up otherwise might become unfeasible. And we should try to, in that window, encourage as many labs or however we wanna call them, like to start. And so one of my currentEiso Kant [00:32:36]: Mission, but qualm is like I wanna encourage whoever is a researcher right now who thinks they can tackle this to go and leave and become my competitor.Eiso Kant [00:32:45]: Like start another foundation model company because I think we need it. I think otherwise we're not gonna be in the world where, I don't want to just be the fifth or the sixth company that wins. I wanna look at a world where there's lots of choice.Starting a Foundation Model CompanyVibhu [00:32:57]: What else do people not see in starting a foundation model? it's, there's a lot of compute, there's a lot of capital required, a lot of compute. You lay out model factory and how to do the training, but there's a lot there, right? That's,Eiso Kant [00:33:10]: Well, look, it's, I in turn-- this is an oversimplification, and I always asterisk it with that because it can land a little bit the wrong way in people's minds. But I think you can sum down, And I saw it, 95% of model building to just doing, you're just doing two things. You're improving data or you're improving compute efficiency. And I know that feels like an oversimplification for the incredible, like, Gifted and skilled work people do. But if you really look at it, like what are we doing? We are looking at data, we're generating new data, we're improving data. and the only way to do that is to look at the data, right? That's a big part of foundation model building. And on the other hand, we come up with these incredible breakthroughs in inference, in architecture, and new attention mechanisms. But what are they really doing? They're bringing compute efficiency. Now, we have definitely had some breakthroughs over the years that allow for more model capabilities. But at the limit, if you could train a large enough model, right, like, and you had infinite compute, we probably-- if you had infinite compute, you'd be at AGI probably already tomorrow.Eiso Kant [00:34:12]: Right? Like it's not. And so, and let me say that infinite compute with infinite ability of much faster networking because networking ends up being more of the bottleneck than compute. But, so I do think that's, those are the main things. And to just realize that this is engineering. I think it's become more obvious, but I think for quite a few years, people have held foundation model companies and researchers and others on this pedestal of like you're doing incredible magic or rocket science, or only like, Nobel laureate physicists can do this. And don't get me wrong, there are some really hard problems that need to be solved, but a lot of the work that all of us are doing on a day Is not sitting down trying to solve a math theorem. A lot of the work that we're doing is just really doing the basics right, writing good code, looking at data, improving it, running experiments, looking at plots, trying to see like, hey, trying to shape our intuitions. And a lot more people could be highly capable researchers. and I think that's, it feels far for people to do so. But I've seen in our own company, we've seen engineers become researchers because the model factory allowed them to be, have a much lower hurdle of running experiments and trying things. And one of the guys on our team who started as an engineer building our agents is a legit reinforcement learning researcher now, making real progress. and that happened in the span of like six months. that would've not been what I think most people assumed was possible, a couple of years ago.Swyx [00:35:46]: Yeah. I think one of the interesting moments is when you can self-host, like, if in a programming language, like if you can compile the language in the language, the equivalent is can you use your own tools, right? You have the pool CLI, you have your own models. presumably you're not only using your own models. There's no way. But like, what's that percentage over time?Laguna S, Persistence, and Behavioral GainsEiso Kant [00:36:10]: This is the first model that we're releasing that is starting to meaningfully contribute to our own work. It's not a it's not state-art model yet. Fable and other, they're, they're very capable models, but Laguna S Is really interesting. I'm gonna pull up the quote. Peng Ming, one of our heads of applied research, said something, last week as the model came out about 10 days ago, much better than we had hoped for or expected. And he said, I have the feeling that a lot of the gains in Laguna S come not from more intelligence, but more from different behavior, more verification, less taking things for granted, not declaring victory early, and being way more persistent. And to be honest, those are more predictive than raw intelligence for success in human also to some degree. And this was, he wrote me this on 5th of July on a Sunday, and it's been burned in my brain ever since because the Laguna S model, as you'll see it and why it does so well on benchmarks and why it does so well in using it on a day basis, is that it's just incredibly persistent. It reasons a lot. I do call that out. We have work to do on making it more efficient. We have to work to do on offering different reasoning modes. But this is the model that has been able to do things that I never thought it could do. A hundred eighteen billion 8B active model, which is not that large. It fits on a DGX Spark and still runs at, thirty, forty tokens a second on a Spark, is able to solve Erdős 397 independently. It's able to do complex programming tasks. It's able to. I asked it this morning to make me a Fi scanner without using any external libraries on my Mac, and it's, like, figuring out, like, the core WLAN API by really persistently trying to understand it without access to the internet. And more, I love vibe checking. I've probably spent eight to ten hours a day with this model for the last ten days.Eiso Kant [00:38:05]: I'm not exaggerating. I was on my eleven-hour flight yesterday. I spent ten hours reading trajectories and traces and, like, of the model.Eiso Kant [00:38:12]: And what I take away from it is exactly what Peng Ming said. We are gonna be able to squeeze so much more out of smaller models than I think we had imagined in the industry because, yes, there's intelligence and larger models are more intelligent. Like, no doubt about it. We should continue to scale up. but the behaviors of being really persistent, of being able to backtrack when you're wrong, of, like, understanding how to interact with your environment show us that we can get a lot more out of it. And this, for me, has created a bit of a Question in my mind the last couple of days. If you think about where we're using models today, right? We are using models, say, for knowledge work. Represents twenty-five percent of the global economy, twenty-five trillion dollars of work.Eiso Kant [00:39:00]: As we scale up models and they become more intelligent, we are excited about using them more and more for pushing the frontier of science.Small Models, Knowledge Work, and CommoditizationEiso Kant [00:39:08]: And if you look at the frontier of science, like true breakthroughs in science, they have been linked, they are linked to more intelligence in many places. Einstein figuring out general relativity is able to bring ideas together that other people would have not brought together. And I think one of the many dimensions of intelligence is the ability to do that, and it's something we clearly see that as models get larger and more capable, they're able to pull more ideas and threads together that a smaller model wouldn't be able to.Eiso Kant [00:39:36]: And we're starting to see examples of that in medicine and, like, in bio and other things. But if you think about the majority of knowledge work that we do, and it includes building software. I'm a software developer at heart first and foremost probably, although I probably can't say it that much anymore as I don't write production code in years, is that what makes us good is our persistence. It's our ability to encounter a problem and backtrack and say, “I need to go figure out this bug. I need to go research this. I need to go look at the documentation. I need to, like, try different, five different ways to see, like, if I can solve it.” But it is not necessarily bringing three ideas together from radically different fields. And so if we are now seeing, and I think Laguna S is an example, that we are able to make a relatively small model much more capable than I had definitely predicted or any previous, like, benchmarks had shown for any model remotely this size or even larger, At least on coding tasks, that it's because of the behaviors. And so now the question I have, and I don't have an answer, it is I know at the limit, so infinite model size, right, extremely large model, and the cost of that model is gonna be very expensive to run. We know this, right? So larger model ROI.Eiso Kant [00:40:52]: So I know that at the very limit, I'm not gonna use the world's largest model one day, quadrillion parameter, whatever crazy, like, scale we scale up, to do a basic coding task. Already today, I'm starting to size down for certain tasks.Eiso Kant [00:41:07]: So it means that there is an optimal. It means there's some curve that goes as we go up to model size for knowledge work, at some point we're at the peak, and after that, the return on investment of using a bigger model, just doesn't make sense.Eiso Kant [00:41:22]: Now, I think the question is, before I would have thought that peak was extremely very far away.Eiso Kant [00:41:30]: This model for me is the first sign that Maybe that peak is At a trillion, five trillion, ten trillion. Maybe we can just squeeze way more out of these models. I'm no longer thinking that we need two or three orders of magnitude on the largest models to be able to, solve knowledge work, the accounting, the legal, the code that we write. And so if that holds true, It is an argument for the commoditization of models. It's an argument that open source can win and, like, succeed in this world. And now it's of course a self-serving argument and it's a hopeful argument, but theoretically at the limit it works. We just have to go discover in the next couple of years of how much more we can squeeze out. Now, I do want to put a big asterisk. This does not mean I'm against scaling models. I think we ultimately only succeed if we scale our models as large as our competition. I do not like. I think we should not put our head in the sand and say we're gonna be king of open source small models. I think that's, It's a out. It's trying to be king of your own kingdom, but not realizing what the rest of the world's doing. All of us rather use a smarter, faster, more model. It's a sign of hope. And so I don't wanna overly state this is a good model. We have a long way to go to get to the state-art. But what hopefully people take away when they use this model is that the behaviors inside of it are what push it to be far more capable, less than necessarily the number of parameters.Pre-Training, Mid-Training, and RL Moving EarlierVibhu [00:43:03]: Is that mostly post-training? LikeEiso Kant [00:43:05]: YesVibhu [00:43:05]: Right.Eiso Kant [00:43:06]: It's entirely post-training.Vibhu [00:43:08]: Are we done improving anything on training? Is, like, training done?Eiso Kant [00:43:12]: No.Vibhu [00:43:12]: Okay.Eiso Kant [00:43:13]: SoVibhu [00:43:13]: I just wanted to cover training, and then we go post-trainingEiso Kant [00:43:15]: Training is not done. I mean, look, there's a part of training of just dealing with skill, right? Every new order of magnitude of model skill, you are going to get new things you gotta solve for. That'- but those are ultimately, engineering challenges.Eiso Kant [00:43:31]: I have a, I would say, a not commonly held opinion that reinforcement learning Will move earlier and earlier into training.Vibhu [00:43:42]: Yeah, training.Eiso Kant [00:43:44]: Not even training. Like training today, right, is, like if you look at - So we've been working on this for years already. and I think the best-- I think the first time we saw it out in public was the DeepSeek Zero paper. this is a year and a half ago, I think, if I recall correctly. where, you can Very early on in a model as it starts capable of being able to use language, et cetera, induce reasoning. and so the question that I have is like, we have this- we have the dataset that's the web. and the web, I think we could arguably say probably has The totality of humanity's knowledge somewhere encoded in different places. It's a huge variance degree of quality, from garbage data, and like once you look at training data, you really get humbled of like what the web is, to like, the most greatest scientific papers and best blog posts and like, best transcripts and whatnot.Eiso Kant [00:44:39]: And so now What we are trying to figure out, and have been doing a lot of work on, and it's a place where maybe not as open as we're on other things, but we will become more over time. we've been spending a couple of years really doing research on how can we turn the web into not just next token prediction, but into a way to teach the model to think earlier in its training. and I think there's a huge amount of gold to be found there. I think we are right now in, we've got some drugs in the industry. One of the drugs is distillation. Another drug is, more environments. Like, and they're great, and they make us feel good, and they make the models better, and like we're all addicted to them, and we'll use them, right? in various different ways. and but ultimately, I think we are still barely squeezing out of the web what we should be getting out of the web.Eiso Kant [00:45:33]: I think just next token prediction during training is not enough.Eiso Kant [00:45:36]: AndVibhu [00:45:38]: YeahEiso Kant [00:45:38]: I think we'll see some very interesting things still happen. and that RL in post-training to induce behaviors, to improve things, like I think - the whole world knows how to do this now. I think we're, we're scaling it up. Everyone is. But I wonder if we need to go as far as we're going today with environments. I'm not sure yetVibhu [00:46:01]: You mean we're going too far?Eiso Kant [00:46:02]: I'm, I'm not sure if the path to AGI is justVibhu [00:46:06]: Is more environmentEiso Kant [00:46:07]: More environments.Vibhu [00:46:08]: It seems like a never-ending, “Okay, I want instruction manual for this table, right? Am I gonna environment out building furniture? Or are we just gonna tail end like we need some general solution?”Eiso Kant [00:46:19]: I think there is, I think there's an ability to generalize more from the web. but I also am very encouraged, like when I look at Laguna S and, which is post-training is, well, is the big impact there. and I see like, oh, wait a second, just by making some of these behaviors much better, we're able to get so much more out of it. It just changes a little bit the way you think about intelligence.Vibhu [00:46:40]: Yeah. The analogy people draw often is the RL phase is where you don't learn as much new knowledge. You shiftEiso Kant [00:46:46]: Yeah.Vibhu [00:46:46]: Yeah. So, you shift distribution, and you can have it reason towards what you want. on your point about training, a lot of training is still just continue training in a domain, say medicine, then you do RL. So still justEiso Kant [00:47:00]: It's just better data, right? Like, I mean, training, ooh, I like how we invented this word. Like it's effectively just like,Vibhu [00:47:06]: Second phaseEiso Kant [00:47:07]: It's the second phase of training With like a really dumb way to do a curriculum. But like ultimately, what you'd want is a curriculum from token zero to token 30 whatever or 40 trillion tokens that really truly is the optimal curriculum for the model to learn. But training is essentially a stage curriculum on the web because we do not have to compute, And, effectively to try to ablate the perfect curriculum, right? And so I'm pretty sure that you'll start to see people talking soon about some other term, and there's two or - ‘cause now we do this, right? We talk stage two and stage three and stage four training and like. But ultimately, all we're doing is we're trying to assign a curriculum to the web data that we have to allow the model to learn better. I think at some point, as things get compute, as models get cheaper to run, as the next generations of compute, this will become more of a continuous spectrum. I also think the reason, by the way, you have training and like stage two and stage three is organizational, Right? It'- this is, I think, a thing where-- that we really try to avoid with the model factory is like Training exists because there's a training team now, right? There's people, or like people in training decide to focus on like a training effort. but what you really want is engineering and scale of experiments that allows for a much more continuous spectrum that you don't, you have infinite stages. Now, we're not there. Compute's not there. Organization design is not there for it yet. but I think we'll get there. we'll look back on a couple of years and be like, “Oh my God, it was so cute that we did our training data like this in such a like naïve way. Like we barely ordered it. We didn't really do a good job at likeCurriculum, Auto Research, and New ObjectivesVibhu [00:48:48]: The building that curriculum will get you that in the industry.Eiso Kant [00:48:51]: And I'll confirm that, when I talk to some researchers that this is a lot of the focus now is like how does training change and what is the next objective other than, next token prediction. I assume you don't have the answers, but you have some ideas.Vibhu [00:49:02]: We have some ideas. We're not ready to talk about it yet.Eiso Kant [00:49:05]: Yeah.Vibhu [00:49:05]: We've been working on them for years, and I think that's the one thing that's also like you asked earlier about, like what's not obvious about building a foundation model company is that you are constantly balancing the table stakes work, the recipe worksEiso Kant [00:49:19]: Yeah.Vibhu [00:49:19]: Versus like your, my crazyEiso Kant [00:49:22]: Pure researchVibhu [00:49:22]: Breakthrough.Eiso Kant [00:49:22]: Yeah.Vibhu [00:49:22]: Pure research and finding that balance and adjusting the percentage to it based on where you are in the race is really important.Eiso Kant [00:49:31]: I mean, so like, this is a nice way. I was gonna bring up auto research at some pointVibhu [00:49:35]: YesEiso Kant [00:49:35]: As another Andrej invention, or coinage, which is like, I honestly, like how many objective functions can there be, right? Like just try 1,000 of them, set it running, whatever.Vibhu [00:49:47]: Man, it's alsoEiso Kant [00:49:48]: Like what you're looking for. You're looking for loss curves like that, likeVibhu [00:49:51]: It's also a thing people take bets on, right? When you say more Neo labs, you're doing a version of we'll do foundation models, scale them up, next token predictors. A lot of other Neo labs that we see want to take a completely different approach, right? At some level, you're right. It's all, compute efficiency, and that's the net objective. But some are okay, different architecture, like vastly different amounts of compute spend. So some are different. They're not justEiso Kant [00:50:19]: YeahVibhu [00:50:19]: They're like, 99% not balancing, here's the vanilla and scale up. They're 99% on, here's novel research that'll change everything.Eiso Kant [00:50:27]: And I think, Luke, I think you. It depends when you started as well, right?Pure Research vs. Table StakesVibhu [00:50:30]: Yeah.Eiso Kant [00:50:30]: When we started, like the novel thing we did was reinforcement learning on code. No long- that's no longer novel by far, but we were like, - that's where we obsessed over when no one believed in RL. So you have to when you start the company, you have to have your own idea. You have to have something that's different that allows you to speed up, right? For us, it was RL to LLMs that later became common, like, Knowledge. But in the beginning, it wasn'tVibhu [00:50:53]: It's cool. this was like your original 2023 blogEiso Kant [00:50:57]: YeahVibhu [00:50:57]: Of purpose.Eiso Kant [00:50:58]: Yeah.Vibhu [00:50:59]: And like you do lay it all out here.Eiso Kant [00:51:01]: We laidVibhu [00:51:01]: The blog is pretty underrated, right? The whole RL on code was very early on.Eiso Kant [00:51:06]: Very early. And even we had to argue with people, like we say here things like to push beyond current capability, to train your own foundation model. We had to argue with people that it mattered that you had your own like, base model. you can fine-tune your way to success, right? major capabilities emerge from training a base model made accurate and useful during fine-tuning.Vibhu [00:51:23]: Which like, for perspective at the time, we knew closed models, OpenAI, Anthropic were huge. The open models we had were like Mistral 7B, a 30B, a 70B.Eiso Kant [00:51:35]: When weVibhu [00:51:35]: YeahEiso Kant [00:51:36]: The date on this thing is wrong. When we published this, it was April 2023. I think this was justVibhu [00:51:42]: YeahEiso Kant [00:51:42]: Happened on a migration, probably found it on archive.org.Vibhu [00:51:45]: Mistral.Eiso Kant [00:51:46]: Mistral had started, we started on the same month, right?Vibhu [00:51:49]: Yeah.Eiso Kant [00:51:49]: So this wasn't even, there was only, I think, Llama out at the timeVibhu [00:51:52]: SnellEiso Kant [00:51:52]: And that's it, right? And so, but I agree. I think we wan

https://youtu.be/31XywTpVWJ4 Robin Dimond, Founder and CEO of Fifth & Cor, is helping organizations tap into the growth of their people by developing future leaders, fostering meaningful relationships, and creating a culture where employees and clients grow together. Through a collaborative agency model built on trust, innovation, and continuous learning, Robin empowers organizations to adapt quickly while creating lasting success. In this conversation, Robin introduces her Grow Your People’s Lives in 5 Ways Framework—Health, Wealth, Spirituality, Friends & Family, and Mental Health. She explains why investing in the whole person creates stronger leaders, how requiring employees to train their replacements builds a sustainable leadership pipeline, and why every client engagement begins with a discovery process and strategic blueprint. Robin also explains how fostering innovation, embracing AI as a tool rather than a replacement for human judgment, and remaining agile help organizations thrive in today’s rapidly evolving business landscape. — Tap Into the Growth of Your People with Robin Dimond  Good day. Steve Preda here with the Management Blueprint, and today my guest is Robin Dimond, the Founder and CEO of Fifth & Cor, a marketing and innovation company harnessing the best tools to support brands, consumers, and communities. Robin, welcome to the show.  Thank you so much for having me, Steve. I appreciate it.  Well, it’s exciting to have you, and we already have some things in common that might or might not come up in the conversation. But what I’d like to ask you, which is kind of my favorite question on this show, is: What is your personal ‘Why’, and how are you manifesting it in Fifth & Cor? I think my personal why—I know what it is. My personal ‘Why’ is my legacy. And because I do not have children of my own, I get the great pleasure of being able to touch so many lives through employees, clients, and partners. So I get to live my ‘Why’ out every day. And I’m a little bit different than most CEOs. I’m looking for the next CEO to replace me. So I go in with the mindset of, “One of you will take my job one day,” and it makes me super happy. That’s my ‘Why’.  Wow. So what do you want to be when you grow up?  What I want to be when I grow up? Technically, I’d like to be the janitor. I believe in cleaning up the messes that are left, and I love that. I love the difficult clients. I love the difficult situations. And I think I’ll step down one day and be the support for somebody who’s currently on my team or the next person to join my team.  Wow. That is awesome. So you want to focus on the message as opposed to running the day-to-day, because that’s where your interests life and where you’re building your legacy.  Yes, I think that is. They call it a Chief Heart Officer in some places. It’s that person who makes sure that the message of the company, what it’s built on, and its foundation are seen all the way through. And that’s the part I love.  Yeah. So you’re a real visionary.  Yes.  So we’ll talk more about your vision, but what I want to ask you about is this podcast is really about frameworks. What we’re trying to do here is find unique frameworks, processes, structures, or ways of looking at the world that CEOs have come up with that could help audience members think about business through a different lens. So is there anything that comes to mind along those lines?  What we found that really works for us is when meeting somebody, we talk to them. Then we do a discovery to figure out if we're aligned, and we find that to be very helpfulShare on X whether it’s with people, partnerships, or clients. So that discovery process, to make sure we’re all on the same page, is so important. From there, we build out a blueprint for them, and then we present the blueprint as if we were the ones running the business. We want to make sure that we’re answering those questions.  We make sure that we’re doing that, and then we have checkpoints. So we sit with a blueprint, saying, “Okay, over the next 30, 60, and 90 days, what are our processes?” The thing that we do differently than most companies is you don’t get an account executive when you come to us. You get a team of people, and you’ll see all their faces all the time. So there is no miscommunication. If you hire us, you get a team. You’ll get a director who’s overseeing your strategy.  You’ll get a coordinator who’s on top of trends. You’ll get a public relations person. And you’ll get a project manager who oversees—we call them our handlers—because they oversee the project all the way through. It has made us successful, and it’s made our clients super successful.  I love it. Especially the blueprint part of it, because it kind of chimes with the framework. So is this your blueprint? How is it unique?  It is, because everyone told us, when you look at agencies or agency models—and we try not to refer to ourselves as that—an agency model will have one account executive, and that person goes back—I call them the man behind the curtain—because they go back and talk to everyone.  What we do is say, “Hey, you’ve hired us, and you’ve hired experts.” We put all of them copied on every email, on every call, working on strategy with your team. And that’s why some of our results are 10,000% growth on your platforms in only 90 days, because we can all work together as a team.  Wow. So that’s very interesting because it’s not an easy thing to pull off—  Nope. To have so many people work together. But maybe that’s the secret sauce here.  The secret sauce is you can’t even get promoted at this company if you don’t hire your replacement. So if you haven’t trained someone behind you, don’t come and ask me for a raise. Don’t ask for a promotion. You should come and say, “Hey, Morgan’s ready to move into my spot. This is how I got her here. I’m ready now to be promoted to the next role.” So we are building tiny leaders behind us who are building teams.  That’s such music to my ears. Yes, I mean, it’s all about leaders building a company, right?  Yes. I totally agree with that. So how do you do that? How do you even do that? Does it mean that any subject matter expert you hire has to be a leader first and foremost? You can’t have someone who just wants to be a specialist stay a specialist?  So I say, “If you’re not growing, you’re dying.” So it should be in some way. It starts in the interview process, and I think this is where a lot of CEOs miss. In the interview process, we start with those questions. Where do you see yourself? Who are the five people you surround yourself with? What do you bring to the table? Asking those very open-ended questions.  Where do you see yourself going? Some people are so honest, Steve. I’d love to bring you on interviews because they’re so honest. They’re like, “Oh, I want to go to law school.” I’m like, “Great. So your time here is going to be short. It’s going to be the next year. What are you going to do in that next year?” “I’m going to help you with contracts.” “Perfect. Okay, let’s get your portfolio looking better so you can go on to the next thing.”  Sometimes we’ll get a subject matter expert who is hyper-focused on public relations, but they say, “I want to be a creative director.” “Okay, great. So during your time here, what are you going to learn? What courses can we invest in you? What trips can we take you on?” So we work with people who have a growth plan of their own and are willing to be agile. That starts at the bottom and moves toward the top.Share on X  That’s genius. Working with people who have a growth plan. Because ultimately, if they don’t have a growth plan, then most likely they’re not going to grow, because you can’t just grow without a plan, right? If you don’t have a vision for growth, it’s not going to happen. Does it mean you’re flexible enough to adapt, to some degree, to these people with a growth plan so that you can share in the fruits of their growth?  Absolutely. We have people start in one section. So we have operations, growth, and delivery. We’ll have someone start in delivery and see that they’re so good at operations that we’ll move them, and then backfill their role. So we have people who move horizontally as well as vertically into different groups. I think that’s the part where you have to be open with your team. You have to say, “Hey, Steve, you’re not that great at creative.  You know that.” And you’re like, “Yes.” And I’m like, “But you are so amazing at operations. Let’s move you over there.” Or, “Steve, you’re such a great thought leader, and I’ve wanted to start a podcast. I’d like to bring you in. Let’s backfill your other role that you started with at our company.” So it’s having that open dialogue from day one and having the team be able to recognize where they’re strong and where they need to move.  I love it. Love it. So what are the challenges with this model?  So many. When I first started, I think I saw what was not working. Before, for you to get ahead, you had to… I spent my time in New York. You had to outpace that person. You had to kind of put the other person down so that you could get promoted. And I saw some negatives in that. This, I would say, is about being transparent and being vulnerable.  It’s also admitting where my mistakes lie or where my weaknesses are. I have dyslexia, so I should never proofread anything for anyone, truthfully. So it’s about being vulnerable. Sometimes that leads to not the greatest people coming into your company because you’re putting yourself out there with that vulnerability. But for every bad apple, there are 20 great apples out there. Yeah, I mean, this is beautiful. I always believe that in order to have people trust you, you kind of have to trust them first. That’s a fragile thing, and you have to put yourself out there. Okay, some people are going to take advantage of it, and you have to…  They are, and it happens. I think every business owner or any leader has to realize that that’s going to happen. But that’s going to happen in your dating life. That’s going to happen in relationships. It’s going to happen in partnerships. In business. It’s going to happen.  But the great part, and the great reward, is seeing someone else come alongside and take your business to the next level, or take your clients to the next level. I think it’s worth being vulnerable to gain that traction.  Yeah, because those people don’t want to work for A-holes, right? They want real people who treat them as equals in some ways because they’re also very talented people. So Robin, let me switch gears here and ask you: what drives growth in Fifth & Cor?  I think what drives growth is our innovation. And we're able to adapt very quickly. In a world where marketing is constantly changingShare on X and I don’t know what the correct term is, Steve, but I say “squished”—we’re getting pressure from the platforms, from social media changes and updates, all of that. We get changes in our coding.  We get changes and pressure from our clients. And we get industry changes. From Boomers to Millennials to Gen X, there’s constant change, so we have to be agile. In our industry, when I started, it was doing billboards. Design was a long process. Now, if we’re not adapting and changing every 30 to 60 days, we can’t keep up. So I can’t make a roadmap or a blueprint for our company three years out. TikTok was here one day.  It was gone the next day. Then it came back again. So planning and strategizing means being agile, being flexible, and being able to move around that for clients so that they don't lose their numbers. I think that's what's driving growth. We're extremely innovative, and we think quickly on our feet.Share on X Can you give me an example?  Yes. I would say the TikTok one is a huge one because we were like, “Clients, you’re about to lose TikTok. They’re warning it’s going to disappear.” Then the next day it was back. So we came up with complete strategies to get rid of it and then bring it back. The next is PR. Sometimes the world changes.  Unfortunately, September 11, we had some other things happen in the news that were really big. It was something where we had to go to our clients and say, “Look, we need a new plan. We need a change. We cannot be tone-deaf right now going into it.” I mean, that all happened, obviously, in September. It was a really rapid change in what was going on. Another example is Shopify went down last week, so all of our clients’ websites went down. There was nothing we were doing.  It was something outside of our control. But we sat there with our clients and said, “Okay, let’s ramp up your social. Let’s go live with some videos and talk about it. Let’s do a press release about how you didn’t lose sales during this time and how you’re going to bounce back.” So it’s constantly thinking on our feet. It’s constantly communicating with our clients and saying, “Look, this happened outside of our control—and yours. This is how we’re going to fix it.”  Yeah. That’s amazing. So how do you keep the team? We talked about how you’re attracting these A players to your business and being flexible to make sure that you tap into their desire to grow, and you grow with them—their way and your way, obviously. But how do you get all these stars to work together? So culture comes from the bottom up. It doesn’t come from me. That would be a dictatorship. So culture comes from the people we hire, and we let them run with new ideas, and they take ownership of those ideas. We did not have public relations four years ago. That was someone else’s idea, and now it’s one of our largest service lines. Project management is another one.  We were not ready to do project management, but people kept seeing our project managers, and they were like, “Well, can we just hire them? Can we just hire them to do this service?” They saw our projects through. So even if someone wasn’t using marketing, they were using our project managers for tech implementations, events, and marketing.  So we’ve created opportunities for people to grow with us. But if you come with us, we say you have to grow in five areas of your life. Health. We actually have a challenge right now, Steve, if you want to join it. It’s who can walk the most miles in June. So health is a challenge to make sure we focus on your health. Yeah.  And we have health-related activities—kayaking. It’s on my LinkedIn posts if you want to go look. We do wealth. We want to make sure you're debt-free. When you come to us, you don't have to be, but we want to make sure you're working with our CFO to become financially independent.Share on X If you’re going to buy a home, we want you to be financially responsible outside of our company.  So in case something does happen, you’re growing toward that. We want you to grow spiritually—whatever you believe your spiritual growth is. If that’s going outside and lying in the grass, if it’s going to your church, if it’s giving back to your community, there should be a spiritual challenge. We look at your family and friends. They don’t have to be blood relatives, but you should be intentional about what you’ve done for your family and friends to help them grow. Whether it’s a family dinner…  You’ll see my team posting about their family dinners on LinkedIn. You’ll see them talking about taking time to do activities with them. So we look at five areas of their lives, making sure they’re growing, and we have challenges to support that. And then mental is the last one. What books did you read? What courses did you take? What did you do to advance yourself?  And Steve, they hold me accountable. They’re like, “Did you finish that CEO book, or are you going to…?” I’m like, “Oh my gosh. Okay. Let me do it.” So it’s something our team can challenge. And if you don’t want to participate, that’s okay. But we’re probably not the company you should be joining.  That is amazing. So people really look after each other. You have this peer accountability going that makes sure everyone is healthy, gets out of their financial challenges, has a social network around them, and takes care of their mental health. That is amazing.  Isn’t that too much for a company to take on? That could be seen as a huge burden—that you worry about all those people in so many different ways, and then you worry about your clients, and you want to make sure your people work together. Doesn’t that become overwhelming sometimes?  I don’t have to work with my clients because I have the most amazing team. I work and focus on my team, and my team’s love and joy get passed on to our clients. It’s a pass-it-down, share-it kind of thing. When they’re happy, healthy, and financially wise, they take care of our clients better than I could ever. They know their birthdays. They know their activities. They know different things. We’re constantly monitoring that. I’m supposed to take care of our team.  Isn’t that something that… I don’t know if it was Jeff Bezos in the early days…  Back in his early days, he did say that. And I want to be focused on that. I want to be innovative. We reward innovation. If you come up with a new way of doing something, you can win prizes, bonuses, gift cards. It is always a challenge: how could you do something better? Someone solved something the other day that we couldn’t figure out for the past three years. I was like, “Open up your email. There’s a gift card in there.” We want to reward people. We want them to enjoy the bonuses of doing different things and thinking outside the box.  That’s very exciting. It sounds like you’re running a perfect business.  No, it’s not perfect. People are messy, and I want other business owners to see that. There are some days that I look at the day and I’m like, “Oof, this is a big one.” Or it’s saying goodbye to the people who don’t fit into that culture, who might be the bad apples. That’s hard.  Not every person we bring on is the right fit. Not every person wants to grow. Some people want to be button-pushers for the rest of their lives, and we might have missed that. Those are the challenging days. Some days, when you get super involved in people’s lives, it means you get super involved in life. Their health might diminish, and you have to step in and take over. We had somebody who fell backward and fractured their skull. Well, we had to keep going while they recovered. That whole part isn’t easy. It’s scary. There are some days I’m like, “What the heck am I doing, Steve?” I say that a lot. But looking back, it’s rewarding.  So Robin, what is one thing that you’re actively trying to figure out in your business?  Right now it’s where the industry is headed, and we’re actively, all day, every day, trying to figure that out. We are seeing the most innovation happen during these times, and keeping up with it is exhausting. To give you an example, the printing press came. The car came. Right now, it’s the next generation, who were born with one of these in their hand. Whether it’s AI that’s coming, or something else, every day there’s something new being released. So it’s, how do we adapt, but also not get burned out? That is the biggest thing I’m trying to solve right now.  Yeah.  So if anyone would like to help me, please email me. I’ll buy you cocktails or coffee. I’ll do either one, because that is exhausting for a business owner right now.  So what part of adapting is the risky part? Give me an example of adapting and risking burnout.  The burnout comes from trying to stay on top of what’s new every day, whether it’s a new platform that came out or whether it’s AI that’s going to replace all your marketing people. It’s about being flexible, but also being smart and not making too many decisions too quickly. Our advisory board is my backbone.  There are five advisors who pour into us, who are sounding boards and give as much time as possible. I think they’re the ones who keep me in line. I didn’t have them at first, so I would recommend getting a board of advisors that you really trust, that you can look at their lives, and they can show you where burnout could happen.Share on X  And if you had a magic wand and you could fix something inside your business, what would that be in the next 12 months?  If I had a magic wand and I could fix anything inside our business, I think I would fix the rate at which we’re growing, if that makes sense. We’re growing way faster than I thought, and my magic wand would be to have enough systems in place to keep up with it. Once we build a system, we outgrow that system, and I’m like, “Cheese and crackers, we’ve already outbuilt it.” So I think it would be staying on that hamster wheel, making sure we put more infrastructure in place. Yeah, and with AI, it’s a different kind of operating system for a company, isn’t it? It is.  AI needs to be used across everything. AI should be in your business. You should be using it. You should be leveraging it. But AI can’t replace people. I think that’s the biggest thing. It’s being able to adapt your team into AI-powered systems, while also having them use their judgment and their brains. They’re going back and checking it.  Yeah. You know what I’m wondering sometimes about AI is, yes, it is making us very productive. I can get a lot more done. But at the same time, I have to make a lot more decisions, so the cognitive load has increased.  My brain load never decreased with AI. I now have to solve problems faster, and I think that’s where people are missing it. People are like, “Well, my blogs will write themselves now.” I’m like, “Okay, but who’s putting in the content for your blogs?” Or, “Oh, it’s going to put this product on my website faster.” Okay, but who’s double-checking it?  Where’s the shipping going? Where are the numbers going? People need to think about the fact that we’re doing things way faster. And then there’s fatigue. We’re being exposed to things at the swipe of a finger. You can see six, seven, eight things happening, and that’s exhausting. What is happening to the mental load of decision-making? Yeah. And then at the same time, it’s not just that we have to make more decisions, but there’s more noise out in the market. So you’re out-competing… Okay, everyone else is more productive, so it’s just a war of… what’s the opposite of a war of attrition?  I don’t even know. I know what you’re talking about, but it’s like you have to be so intentional now. Yeah. It’s the war of overwhelm, basically. Yes. The war of overwhelm. Yeah, that’s wonderful. Okay, but let’s go back to Fifth & Cor. Basically, you’ve come out of your discovery with this blueprint, and then you have this team approach. These people are all fully self-realizing in the five areas of their lives, and they’re coming together, working together, and incorporating AI. So who is an ideal client that can really take advantage of everything you can do? Not just parts of it, but everything. What’s the ideal client for you?  I love that question. An ideal client is not an industry or a revenue number they have to have. An ideal client is someone who comes with a problem and is flexible and agile enough to let us come up with a solution. And they’re a true partner. We’ve had a client who’s been with us for four and a half years, all the way since our inception, and they’ve been able to roll with all the new changes.  They trust us. So it’s someone who is innovative in their business. They’re willing to be agile as they move. That is the perfect client. Revenue does not matter. You can have all the revenue in the world, but if you’re slow to adapt or you’re not willing to go with the times and the changes, that’s a problem. What we did pre-Covid does not work right now. The same things that worked six months ago don’t work now. So the ideal client must be someone who’s willing to be innovative, attract new ideas, and execute quickly.  Do you have a way to help your clients be more like that?  Yes. Even in our onboarding stakeholder session, before we do a discovery call and present the proposal, we talk to them about that. We help them get set up. We come up with SOPs so they have documented processes throughout the entire engagement. I always say, “In case we get hit by the lottery bus…”  That’s my way of saying, if we disappear tomorrow, you have systems in place so your business won’t need us. So we help them grow, and then we have checkpoints with them. “Okay, are we doing this? These are the results. This is what you trusted us with over the past 30 days. Here’s what we’re doing over the next 30 days.” So we’re constantly meeting with them and saying these things. The clients who don’t work out long-term are the ones who say, “No, we’ve always done it this way, and we’re going to keep doing it that way.”  Those are the ones where I’m like, “Well, why did you even call us? If you’ve always done it that way and you weren’t getting results, that’s the definition of insanity.” So it’s clients who are willing to move with us, move with the times, and are willing to bring in other teams. I love that. They’ll bring in their tech team, or they’ll bring in their logistics team, to sit there and say, “Okay, what are we doing, and how do we take it to the next level?” Love it. And you can only do this with A players because otherwise it would be very painful. Love it. So if you’re listening to this podcast and Robin Dimond explaining her business and how she’s putting her employees first and making sure they have a well-rounded life so they can live up to their potential inside the business, while also harnessing their own growth ideas, so you’re tapping into what they already want to do.  Then the clients are brought along, and the same thing happens to them. They’re inspired, they’re challenged to do better, and then you help them. So if you want to be a client who enjoys that kind of approach, then my question to you, Robin, is: where can they find you and your colleagues, and what should be their first step? Perfect. You can find us at fifthandcor.com. Fifth represents the five senses, and Cor is Latin for “heart.” So that’s the reason why we do everything—the heart behind our senses. So, fifthandcor.com, or you can find us on LinkedIn. You can personally reach out to me. It’s Robin Dimond. I’d love to connect with you. I already have 30,000 connections, so I’d love to do that and set up a meeting with you.  That’s fantastic. Both my parents were doctors. My dad passed away, but my mother is still a dermatologist, and my dad was a cardiologist. They had a company, and it was called Dermacor, which is basically the skin and the heart. I love that. Yeah. Well, now that we know we’re related… I feel like that’s just part of our story too.  Yeah, that’s a Central European idea, perhaps. So if you enjoyed this episode, obviously reach out to Fifth & Cor, find out more about what Robin and her team are up to, and stay tuned because every week I bring a wonderful entrepreneur like Robin who’s got great ideas, and you can try to steal their ideas. So thanks for coming, Robin, and thanks for listening.  Thank you. Important Links: Robin's LinkedIn Robin's  website

This episode features Cameron Allen, who came all the way from Memphis to speak at this year's LA Astro Fest. This recording was supposed to come out before his workshop, but that was just not happening.It turns out that throwing a four-day festival with nothing but blood, sweat, tears, and the good graces of volunteers is a lot. No matter, we did it. The episode is here. If this is your first encounter with Cameron, he's an herbalist and astrologer who sits at the crossroads of medical, traditional, and evolutionary astrology.He's someone I've admired for a very long time. Years ago, when I was too afraid to record podcasts, I hosted a lot of TalkTalkTalks over IG Live. These are absurd hour-plus conversations that should be podcasts. Cameron Allen was indeed one of these IG guests, along with Samuel Reynolds and Gemini Brett, all astrologers who I'm honored to say have been speakers at LA Astro Fest.If you're in the mood for scrolling, you can go way back in the Vivi Henriette IG history and find these interviews. One day I will make them podcast episodes for Club Astro members.In the meantime, you can head over to Art of the Zodiac on Substack and read the bonus print interview that accompanies this podcast> A quick note on this recording: I have no idea why I'm so entranced by the idea of a "spirit supply store." While Cam was talking, I had this image of ghosts walking into the shop and buying ghost accessories. The tools they need to haunt houses and whatnot.In my head, it was a world. On the recording, I sound like I've never heard of a botánica. Oof! The best part of interviews is I get to sound like an idiot as long as my guest sounds good. Cameron, per usual, sounds brilliant! This was such a delightful conversation. XO ViviSupport the Podcast & Learn Astrology!Ready to take a deep dive into astrology? I invite you to join ⁠Club Astro⁠ by becoming a paid subscriber to ART of the ZODIAC on Substack. Your membership offers:Exclusive Cohort: Access to a dedicated community of astro seekers.Weekly ZOOM Sessions: Bring your chart, ask me questions directly, and connect with fellow astrology enthusiasts in intimate gatherings.Secret Invites & Discounts: Including discounted tickets to online workshops, LA Astro Fest, and my monthly in-person gathering, The Los Angeles Astro Salon.More than just benefits, your membership directly supports me and this work, allowing me to continue creating content like this.Join Club Astro here:⁠ https://vivihenriette.memberful.com/⁠Other Ways to Support (No Funds Required!)Even if you can't join Club Astro right now, your time and listenership are invaluable. If you enjoy this work, please consider:Tell a friend: Word-of-mouth is incredibly helpful!Leave a review: A review wherever you listen to podcasts truly helps new listeners find the showAbout CameronCameron Allen is an herbalist & astrologer that sits at the crossroads of Medical, Traditional, and Evolutionary astrology. He synthesizes wisdom from his training in Ayurveda, Kundalini yoga, holds degrees in health & sports science and psychology with a focus on sports & exercise psychology, and is currently in school for Unani Tibb.  Cameron has trained with teachers of lineages that have not been named here, but honors them nonetheless. All of these systems of healing and ways of knowing combine with a focus on being centered in how to be of service in any given moment.About ViviVivi Henriette is an LA-based astrologer and tarot reader whose practice centers on storytelling, mythology, and collaborative divination. She creates a space for clients to reclaim their personal narratives through the lens of ancient archetypes. Vivi produces⁠ LA Astro Fest⁠, hosts the Los Angeles Astro Salon, and is the creator of the podcast⁠ TalkTalkTalk⁠. You can find her weekly writing on ritual and meaning right here on⁠ ART of the ZODIAC⁠.

“Should I even keep fighting for my marriage?”, asks “G.” Oof, that word… “fighting.”  I hear it often.  But so many times, when someone says they are “fighting for” their marriage, they end up “fighting against” their spouse.  The spouse who doesn't see how to move forward. Which is rarely helpful for the process.  But I watch person after person “suit up” to do battle, not even sure on what they are fighting. So, let me clarify that with the question from “E.”  She asked why I always talk about connection… not romance, playing “hard to get,” doing “No Contact,” or reverse psychology. Those two fit together… the “fighting” part and the “connecting” part.  You are fighting for connection!  For some very specific (and deeply rooted) reasons. I discuss both in this episode of the Save The Marriage Podcast. RELATED RESOURCES: Connection and Marriage Why are We Fighting No Contact is Crap No Manipulation Save The Marriage System

Comcast is scraping Universal, DreamWorks and NBC off the bottom of its boot and spinning them off into their own company. And this is after they bundled up MSNBC and a bunch of other failing cable channels and sent them out to die as well. OOF. Watch the podcast episodes on YouTube and all major podcast hosts including Spotify. CLOWNFISH TV is an independent, opinionated news and commentary podcast that covers Entertainment and Tech from a consumer's point of view. We talk about Gaming, Comics, Anime, TV, Movies, Animation and more. Hosted by Kneon and Geeky Sparkles. Get more news, views and reviews on Clownfish TV News - https://more.clownfishtv.com/ On YouTube - https://www.youtube.com/c/ClownfishTV On Spotify - https://open.spotify.com/show/4Tu83D1NcCmh7K1zHIedvg On Apple Podcasts - https://podcasts.apple.com/us/podcast/clownfish-tv-audio-edition/id1726838629 MORE CLOWNFISH TV - Official Merch Store: http://ClownfishMinus.com Facebook - https://facebook.com/ClownfishTV X - https://x.com/ClownfishTVcom Clownfish TV subreddit: https://www.reddit.com/r/ClownfishTVOfficial/ Disclaimer: This series is produced by Clownfish Studios and WebReef Media, and is part of ClownfishTV.com. Opinions expressed by our contributors do not necessarily reflect the views of our guests, affiliates, sponsors, or advertisers. ClownfishTV.com is an unofficial news source and has no connection to any company that we may cover. This channel and website and the content made available through this site are for educational, entertainment and informational purposes only. These so-called “fair uses” are permitted even if the use of the work would otherwise be infringing. #DreamWorks #Animation #Comcast #Universal #Podcast #Commentary #News #Reaction #Gaming #Comedy #Entertainment #Hollywood #PopCulture #Tech #Anime #FYP Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

"If what I want is for someone else to respect my time, then maybe I'm the one who needs to respect my time first." Oof. That realization hit me right between the eyes. Have you ever noticed how quickly frustration turns into a story about someone else? The person who cancelled. The coworker who dropped the ball. The partner who isn't helping. The kids who need one more thing. And sometimes that's true. But sometimes the thing you're frustrated with isn't actually the problem. "Your frustration isn't the problem. It's the clue." Sometimes frustration is the moment your needs, boundaries, exhaustion, or truth have been trying to get your attention for a while—and you've been too busy carrying everything else to hear them. In this solo episode, I'm unpacking two frustrating experiences from my own life and walking you through the exact process I use to understand what frustration might actually be trying to tell me. Because emotions aren't the problem. They're information. And frustration is one of the most revealing emotions we have. Together we'll explore why frustration almost never stands alone, how it can point us toward unmet needs and hidden truths, and a simple framework you can use to move from irritation to clarity. In This Episode • Why frustration is always trying to tell you something • The surprising difference between being frustrated with someone else and being frustrated with yourself • How irritation can reveal needs you've been overlooking • Why emotions work like an internal compass • A simple reflection process you can use anytime frustration shows up • The powerful shift that happens when you stop asking "Why am I so frustrated?" and start asking "What is this trying to teach me?" Three Takeaways Your frustration isn't the problem. It's the clue. Frustration often points toward something important that needs your attention. Sometimes frustration is what happens when you've abandoned yourself. Not intentionally. Not dramatically. Just quietly, one small compromise at a time. Emotional clarity begins with curiosity. The goal isn't to get rid of the feeling. It's to understand what it's trying to show you.   Reflection Question What would your frustration say if it trusted you enough to tell you the truth? Before you go... If this episode felt like a conversation you needed today, would you share it with someone who might need it too? Follow the podcast, leave a rating or review, and help more high-functioning humans with big feelings find a little more clarity, connection, and calm without having to earn it.   Links & Resources Website: https://danielleireland.com Substack: https://danielleireland.substack.com YouTube: https://www.youtube.com/@DontCutYourOwnBangs Instagram: https://www.instagram.com/dontcutyourownbangs The Treasured Journal: https://danielleireland.com/journal Wrestling a Walrus: https://danielleireland.com/wrestling-a-walrus Spotify: https://open.spotify.com/show/0VFZulonTvaa2HIPyJa4Tq Apple Podcasts: https://podcasts.apple.com/us/podcast/dont-cut-your-own-bangs/id1427579922

Sega is putting put on blast by gamers because of a new Sonic Chaos Emerald contest that requires you consent to letting their AI train on your data before being able to enter. Given the Sonic fandom, it's going about as well as can be expected. It's so egregious, that the official Sonic account got community noted on X. OOF. Watch the podcast episodes on YouTube and all major podcast hosts including Spotify. CLOWNFISH TV is an independent, opinionated news and commentary podcast that covers Entertainment and Tech from a consumer's point of view. We talk about Gaming, Comics, Anime, TV, Movies, Animation and more. Hosted by Kneon and Geeky Sparkles. Get more news, views and reviews on Clownfish TV News - https://more.clownfishtv.com/ On YouTube - https://www.youtube.com/c/ClownfishTV On Spotify - https://open.spotify.com/show/4Tu83D1NcCmh7K1zHIedvg On Apple Podcasts - https://podcasts.apple.com/us/podcast/clownfish-tv-audio-edition/id1726838629 MORE CLOWNFISH TV - Official Merch Store: http://ClownfishMinus.com Facebook - https://facebook.com/ClownfishTV X - https://x.com/ClownfishTVcom Clownfish TV subreddit: https://www.reddit.com/r/ClownfishTVOfficial/ Disclaimer: This series is produced by Clownfish Studios and WebReef Media, and is part of ClownfishTV.com. Opinions expressed by our contributors do not necessarily reflect the views of our guests, affiliates, sponsors, or advertisers. ClownfishTV.com is an unofficial news source and has no connection to any company that we may cover. This channel and website and the content made available through this site are for educational, entertainment and informational purposes only. These so-called “fair uses” are permitted even if the use of the work would otherwise be infringing. #SEGA #Sonic #Games #VideoGames #Podcast #Commentary #News #Reaction #Gaming #Comedy #Entertainment #Hollywood #PopCulture #Tech #Anime #FYP Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

...plus the Georgia GOP no-show rodeo & a Wynter-y cold endorsement leaves Mike Collins frozen out.The special session Governor Brian Kemp commanded out of the state legislature didn't yield much for his party (no redrawn maps, no property tax cuts passed onto consumers' sales taxes) but it did yield a mess of an election bill he'll have to sign (we're guessing) to address the looming QR code deadline: according to Marilyn Marks with the Coalition for Good Governance, the bill accidentally mandates razor-thin election margins, the way it's written. Oof. She joined me to explain. - - - Meanwhile, it's officially "general election season" for the 2026 calendar so focus will turn to Rick Jackson v Keisha Lance Bottoms, with statements made by Jackson at an event with a radical anti-abortion voter raising eyebrows. In the comments, he seems to co-sign on eliminating exclusions for incest and rape. - or at least creating new barriers to those exceptions being available to women who learn they're pregnant on or after the six-week ban in Georgia would occur. His being poor at thinking on his feet (there and in a prior televised debate) is likely one reason why Democratic nominee Keisha Lance Bottoms is challenging Jackson to no less than three one-on-one forums. ---The alleged killer who stabbed a MARTA passenger weeks ago may receive the death penalty, according to a federal grand jury, writes Rosie Manins at the Atlanta Journal Constitution. That's the story anti-transit, anti-urban folks want you to focus on, and not the fact that MARTA has adequately absorbed twice the usual ridership during the World Cup while running on-time and without incident. All that just proving that a little influx of state and federal support makes MARTA run "smarta."- - - As if there isn't enough to embarrass any Republican with self-awareness, a Georgia GOP rodeo drew "tens" of people (my guess-estimate when you don't count candidates and their surrogates). HA!- - - One more, though! Atlanta-based conservative talk radio host Shelley Wynter (WSB A/FM) had Senator Jon Ossoff on recently - and endorsed the incumbent Senator! Hear the exchange:

Supergirl box office tracking keep dropping, and now it's very likely that Supergirl will come in behind Toy Story 5 in its second weekend. toy Story is shaping up to be a monster hit, and even if it drops 50-60% it'll beat Supergirl's projected $45-55 million. OOF. Watch the podcast episodes on YouTube and all major podcast hosts including Spotify. CLOWNFISH TV is an independent, opinionated news and commentary podcast that covers Entertainment and Tech from a consumer's point of view. We talk about Gaming, Comics, Anime, TV, Movies, Animation and more. Hosted by Kneon and Geeky Sparkles. Get more news, views and reviews on Clownfish TV News - https://more.clownfishtv.com/ On YouTube - https://www.youtube.com/c/ClownfishTV On Spotify - https://open.spotify.com/show/4Tu83D1NcCmh7K1zHIedvg On Apple Podcasts - https://podcasts.apple.com/us/podcast/clownfish-tv-audio-edition/id1726838629 MORE CLOWNFISH TV - Official Merch Store: http://ClownfishMinus.com Facebook - https://facebook.com/ClownfishTV X - https://x.com/ClownfishTVcom Clownfish TV subreddit: https://www.reddit.com/r/ClownfishTVOfficial/ Disclaimer: This series is produced by Clownfish Studios and WebReef Media, and is part of ClownfishTV.com. Opinions expressed by our contributors do not necessarily reflect the views of our guests, affiliates, sponsors, or advertisers. ClownfishTV.com is an unofficial news source and has no connection to any company that we may cover. This channel and website and the content made available through this site are for educational, entertainment and informational purposes only. These so-called “fair uses” are permitted even if the use of the work would otherwise be infringing. #Supergirl #DCComics #JamesGunn #Movies #Podcast #Commentary #News #Reaction #Gaming #Comedy #Entertainment #Hollywood #PopCulture #Tech #Anime #FYP Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

Join us as we dive again into the history of major Soviet firearms leading up to the development of everyone's favorite banana mag rat-a-tat-tat machine. Picking up from the PPD-40, we see how the Soviets managed to develop mankind's only submachine gun to skip something as technologically complex as... threading. Oof.

What if the thing standing between you and the life you want isn't lack of time, money, or opportunity… but your inability to trust yourself?Oof. I know.Most of us have become experts at explaining our hesitation: waiting for the right time, gathering more information, thinking about it, praying about it.But what if all those reasons—while real—are costing us more than we realize?In this episode, I'm unpacking the opportunity cost of not taking action and sharing my 5-Step Self-Trust Framework to help you move from hesitation into action.This isn't about making reckless decisions.It's about becoming the type of person who trusts herself enough to move.Because staying exactly where you are has a cost too.Click play to hear all of this and:[00:00] Understanding opportunity cost and why avoiding decisions has consequences.[02:13] Dating, fitness, business… and how every “not now” creates a tradeoff.[05:12] The surprising pattern Jasmine sees repeatedly in business conversations.[12:00] Understanding why hesitation often feels reasonable.[21:18] Who do you want to become? The one who stays… or the one who decides?[22:24] The mindset shift that transformed Jasmine's relationship with possibility.[24:09] A practical system for making decisions with confidence.Listen to Related Episodes:The Secret to Closing High-Ticket Sales (Even With Objections) with Shelby SappHow to Create Content That Removes Objections and Increases Your SalesDo THIS to Close More Sales

The Mandalorian and Grogu has apparently dropped out of the Top 5 at the box office in only its third week, and it's currently sitting at only half of what it needs to make to break even. This movie was definitely frontloaded and it's failing harder than anyone could have predicted. It got its teeth handed to it by a limited release of the finale of The Amazing Digital Circus. Oof. Watch the podcast episodes on YouTube and all major podcast hosts including Spotify. CLOWNFISH TV is an independent, opinionated news and commentary podcast that covers Entertainment and Tech from a consumer's point of view. We talk about Gaming, Comics, Anime, TV, Movies, Animation and more. Hosted by Kneon and Geeky Sparkles. Get more news, views and reviews on Clownfish TV News - https://more.clownfishtv.com/ On YouTube - https://www.youtube.com/c/ClownfishTV On Spotify - https://open.spotify.com/show/4Tu83D1NcCmh7K1zHIedvg On Apple Podcasts - https://podcasts.apple.com/us/podcast/clownfish-tv-audio-edition/id1726838629 MORE CLOWNFISH TV - Official Merch Store: http://ClownfishMinus.com Facebook - https://facebook.com/ClownfishTV X - https://x.com/ClownfishTVcom Clownfish TV subreddit: https://www.reddit.com/r/ClownfishTVOfficial/ Disclaimer: This series is produced by Clownfish Studios and WebReef Media, and is part of ClownfishTV.com. Opinions expressed by our contributors do not necessarily reflect the views of our guests, affiliates, sponsors, or advertisers. ClownfishTV.com is an unofficial news source and has no connection to any company that we may cover. This channel and website and the content made available through this site are for educational, entertainment and informational purposes only. These so-called “fair uses” are permitted even if the use of the work would otherwise be infringing. #Disney #Movies #StarWars #Podcast #Commentary #News #Reaction #Gaming #Comedy #Entertainment #Hollywood #PopCulture #Tech #Anime #FYP Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

I WATCHED THE DEMOCRATIC GOVERNOR’S DEBATE So you didn’t have too. Oof. It was super boring. My claim that it would be a milquetoast bloodbath was spot on. Kyle and Marshall asked softball after softball, never asking about Phil Weiser being Attorney General while crime and drug overdoses skyrocketed while he sued Trump. Never asked Michael Bennet who voted for bloated budget after budget in DC what his budget would look like in Colorado. What a waste of time, but if you think Polis and the Democratic Legislature were bad, wait until one of these guys gets into office. Bennet said he wants a budget destroying public option. They both said they would sign the labor bills Polis vetoes. They kept talking about affordability and then talking about more regulation in the next breath as if they aren’t inextricably linked. We are so screwed. As much as I hate to do this, you need to watch this debate. If for no other reason than see how bad things are going to get. Longtime journalist Chris Rourke is joining me at 1 to talk about it.See omnystudio.com/listener for privacy information.

R.I.P. Anthony Stewart Head. Our love for you simply can not be expressed by any means other than sharing the episode where Jeremy goes on and on about your raw sexuality for like half an hour.Fear Level: Spoopy with a side of eTrigger Warnings: Director: Darren Lynn BousmanWriters: Darren Smith Terrance ZdunichStars: Alexa Pena-Vega, Anthony Stewart Head, Paul Sorvino, Paris Hilton, and Sarah Brightman and Nivek Ogre (Kevin Ogilvie)Repo: The Genetic Opera is a horror musical goth opera about a guy who repossesses organs, his sick daughter, a family full of underused character actors, and one very screamy grave robber. It's...a lot...but somehow also not enough. It is super weird though.Topics of Discussion:-Your favorite Spy Kid and Watcher/Librarian-Corpse battering rams-a little glass vial-a little glass vial-a little glass vial-GRAAAAAVES!-"You're at Nightmare Before Christmas and I need you at Rocky Horror Picture Show"-Tough guys don't spit blood, they just fall over dead-Have you heard of Sonny Corleone? No? GOOD!-MS Paint comic interludes-Literal mean puppet-We're on a string of movies that don't handle sex workers well. Oof.-Emily talks about Skinny Puppy-God, there's just so much potential here that you're not using.-We brainstorm half a dozen better versions of this movie using the same pieces they have here and and just arranging them differently-I say again, THE RAW SEXUAL ENERGY OF ANTHONY STEWART HEADRecommendations:-Evil Dead: The Musical-Carrie: The Musical-The Toxic Avenger: The Musical-Spiderman: Turn off the Dark-Natasha Pierre and the Great Comet of 1812-Zipperface: The Hobo Musical-Courtney Crumrin-The Crow-City of Lost Children-Skinny Puppy videos-Romeo + Juliet-Moulin Rouge-Phantom of the Paradise-Within Temptation- Black Symphony-Malice Mizer videos-Pink Floyd: The Wall-Bioshock-Deus Ex-Ghost in the Shell: Standalone Complex-Buffy Once more with feeling-The Rocky Horror Picture Show-Anna and the Apocalypse-LabyrinthFollow our guests:Joey BraccinoTwitter: @joeybraccinoPodcast: Talking ComicsFollow us on twitter @proghorrorpodFollow Emily on twitter @megamothEmily's Website: Megamoth.netFollow Ben on twitter @benthekahnPre-Order Ben's new book, Renegade Rule.Follow Jeremy on twitter @jrome58Visit his website at JeremyWhitley.comRSS Feed: https://feeds.transistor.fm/progressively-horrifiedWebsite: https://progressivelyhorrified.transistor.fm/Join our Patreon at: patreon.com/progressivelyhorrified to support the show, get bonus episodes, early access to upcoming episodes, and a cool Progressively Horrified t-shirt.Come back next week to hear about Attack the Block!★ Support this podcast on Patreon ★ JOIN JEREMY'S ZOOP CAMPAIGN AND HELP MAKE GREAT COMICS! https://zoop.gg/c/slayTake our listener survey: http://bit.ly/progressivelyhorrified-surveySign up to support Progressively Horrified on Patreon for as little as $5 a month and get bonus episodes! https://www.patreon.com/c/progressivelyhorrified Hosted on Acast. See acast.com/privacy for more information.

The box office projections for James Gunn's 'Supergirl' is tracking somewhere between 'The Marvels' and 'Black Adam' -- both of which were considered flops. OOF. So no, adding more Superman to the trailers doesn't seem to be working. Then we talk about Milly Alcock's "Christian dads" comment. Watch the podcast episodes on YouTube and all major podcast hosts including Spotify. CLOWNFISH TV is an independent, opinionated news and commentary podcast that covers Entertainment and Tech from a consumer's point of view. We talk about Gaming, Comics, Anime, TV, Movies, Animation and more. Hosted by Kneon and Geeky Sparkles. Get more news, views and reviews on Clownfish TV News - https://more.clownfishtv.com/ On YouTube - https://www.youtube.com/c/ClownfishTV On Spotify - https://open.spotify.com/show/4Tu83D1NcCmh7K1zHIedvg On Apple Podcasts - https://podcasts.apple.com/us/podcast/clownfish-tv-audio-edition/id1726838629 MORE CLOWNFISH TV - Official Merch Store: http://ClownfishMinus.com Facebook - https://facebook.com/ClownfishTV X - https://x.com/ClownfishTVcom Clownfish TV subreddit: https://www.reddit.com/r/ClownfishTVOfficial/ Disclaimer: This series is produced by Clownfish Studios and WebReef Media, and is part of ClownfishTV.com. Opinions expressed by our contributors do not necessarily reflect the views of our guests, affiliates, sponsors, or advertisers. ClownfishTV.com is an unofficial news source and has no connection to any company that we may cover. This channel and website and the content made available through this site are for educational, entertainment and informational purposes only. These so-called “fair uses” are permitted even if the use of the work would otherwise be infringing. #Movies #Supergirl #Superman #DCComics #Podcast #Commentary #News #Reaction #Gaming #Comedy #Entertainment #Hollywood #PopCulture #Tech #Anime #FYP Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

Let's be real—transitioning into homeschool high school feels big. It doesn't matter how many years you've been at this. That shift from middle school to high school brings with it a swirl of emotions: uncertainty, excitement, fear of missing something, and sometimes—let's be honest—a bit of guilt. Pin those thoughts in your mind for a moment as I share with you a conversation we recently had in the Confident Homeschool Mom Collective. It was a rich, heartfelt conversation about this very season. And the stories shared were so resonant, I knew I had to write to them. One homeschool mama said: “Oof, high school… well, Viv is starting 7th grade and I feel like we're already behind.

The “covers” folder is full. Overfull, to be honest. So tonight’s show is a supersized show of great remakes, and many of them are quite surprising. The Tubs covering Metallica? Social Distortion doing Chris Isaak? White Fence remaking Simply Red? The Hollywood Stars retaking their own song back from Kiss? Hell, even Ty Segall’s version of The Doors could be considered a surprise. There are also quite a few more conventional remakes here tonight, but each and every one is what the kids today call a “banger”. (Oof, remind me to never use that term again.) I’m sure any Ledge listener will find a favorite. What’s your highlight? For more info, including setlists, head to http://scotthudson.blogspot.com

A donor asked one of my clients, “How much do I have to give for you to stop asking?” Oof. That question hit me right in the chest. And honestly, I think a lot of fundraisers have either been asked something like this or secretly fear hearing it.In this episode, I break down exactly how I would respond and why I believe obligation has no place in major gifts fundraising. None. I'm not interested in convincing, pressuring, manipulating, or cornering someone into giving. That's not partnership. That's coercion with a tax receipt.We talk about the difference between fundraising from desperation versus fundraising from grounded leadership. Because donors can feel your energy. They can feel when you're white knuckling a goal, trying to force a gift, or needing their validation. And they can also feel when you genuinely mean it when you say: “You do not have to give.”The best donor relationships are built with people who are all in. People who want to be there. The people who don't just write checks, but become real partners in the mission. That kind of fundraising starts with you releasing pressure from yourself first.What you'll learn in this episodeHow Julie would respond when a donor says, “How much do I have to give for you to stop asking?”Why obligation-based fundraising damages donor relationshipsThe psychological reason donors are more likely to give when they feel fully free to chooseHow desperation and pressure show up in donor conversations, even when you think you're hiding itWhy emotional regulation matters in major gifts fundraisingThe difference between inviting someone into a mission versus convincing them to fund itHow to stop white knuckling individual donor relationshipsWhy real donor partnerships require alignment, not pressureWhat “walk away power” actually looks like in fundraising conversationsHow releasing donors from obligation helps attract more passionate, committed supportersAt the end of the day, major gifts fundraising is not about getting people to do things they do not want to do. It's about leading well enough, listening deeply enough, and believing strongly enough in your mission that the right people naturally lean in. The more grounded and pressure-free you become, the more authentic and sustainable your donor relationships will be.Want 15 leads in 5 minutes? DM me "Breakfast burrito" on LinkedIn and I'll send you a pdf and 6-minute training to help you generate 15 leads for your nonprofit in minutes. It's totally free. All you need is an email to sign up. DM me "Breakfast burrito" - I'm from Texas, what can I say? - to get your pdf and mini training.If you're an ED or DD of a $1M+ making a difference in your community and you're ready to make bigger, bolder asks, then DM me “CL” on LinkedIn and I'll share details.

We're back with some more Gilmore Girls! Join us in this journey on social media - @gilmorethemerrierpod. ABOUT: GILMORE GIRLS (SEASON 2 EPISODE 2) Lorelai hesitates to tell her parents about her engagement to Max, despite Rory's urging. Rory and Dean have a spat over Rory's plans for extracurricular activities. AIR DATE & NETWORK FOR: GILMORE GIRLS (SEASON 2 EPISODE 2) October 9, 2001 | The WB CAST & CREW OF: GILMORE GIRLS (SEASON 2 EPISODE 2) Lauren Graham as Lorelai Gilmore Alexis Bledel as Rory Gilmore BRAN'S GILMORE GIRLS (SEASON 2 EPISODE 2) SYNOPSIS Lorelai has officially entered wedding planning mode. She tells Rory she went dress shopping the day before…and it was AWFUL. Rory is like, “Great, we're going right now!” At Chilton, Paris is signing up for summer school, extra classes, extracurriculars — the whole thing. Rory walks up, but Paris is still mad about the Tristan situation. She tells Rory she better not show up at the charity home-building event tomorrow. Rory's like, “Oh, I'll be there.” Remember Henry, the guy Lane hit it off with at the party? He tells Rory he tried calling, but Mrs. Kim scared him off. So he gives Rory his number to pass along. Friday night dinner time. Rory reminds Lorelai she needs to tell Emily and Richard about the engagement. Lorelai's like, “I will soon, okay?!” Richard pulls Rory aside and apologizes for how he treated Dean. He says he wanted to do it in person. Rory forgives him, and they hug. Progress! Meanwhile, Lorelai is alone with Emily and decides to finally tell her about the engagement. Emily's response? “That's nice. I hope I'm in town.” Lorelai is…not thrilled. The next day, Rory heads to the house-building charity event. She shows up with a pink, fluffy hammer (not ideal), and the guy in charge is basically like, “Good luck.” Paris immediately claims a wall and tells Rory to find somewhere else. She reveals she's been volunteering for years — it's all part of her Harvard plan. Rory realizes…she is WAY behind on extracurriculars. Uh oh. Rory vents to Dean, spiraling about how she can't relax this summer — she has to catch up. Dean is not loving this. He just wants to spend time together and is like, “Is that too much to ask?” Lorelai feels bad seeing Rory so stressed, which kind of ruins her date with Max. Max talks about how supportive his parents are, and Lorelai is like…must be nice. He tries to get her to think about things rationally, but instead she decides to bring him to meet her parents. Bold choice. This leads to a big fight with Emily. Lorelai finally lets it out: “Why don't you care? You've never cared, and it hurts.” Emily fires back that she found out about the engagement from a stranger instead of her own daughter. The next day at work, Lorelai realizes it was Sookie who told Emily while planning an engagement party. Everything clicks. At the party, Lorelai is actually having a great time — even though Max is about to leave for Toronto for a while. Dean finds Rory, and they make up. Meanwhile, Lorelai notices Luke isn't there, so she goes to the diner. He gives a weird excuse about being busy “working”…on ketchup. She tells him she really wants him there — it's an important night. When Luke finally shows up, he sees Lorelai slow dancing with Max. They spot each other and just…wave. Oof. The episode ends with Lorelai going to Emily for advice about veils. She apologizes for not telling her sooner and admits they don't communicate well. She explains she was scared of how Emily would react. Emily softens…just a little. Then tells her her head is too big for a veil and she should wear a tiara — like she did. Watch the show on Youtube - www.deckthehallmark.com/youtubeInterested in advertising on the show? Email bran@deckthehallmark.com Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

Send us Fan MailGas, wings, beer... everything is too expensive.The wild times continue and everything is too expensive. I had to get a loan to fill up my tank this week, and 2nd mortgage for a six-pack. Is Two Buck Chuck even two bucks anymore? Doubtful.With the rising costs of everything the tradition of pre-gaming is back. Knocking back a few before you head out for an evening can save a good chunk on your drink bill. It's just good financial sense. Drink wherever you want to, just keep support those breweries. Drinking everywhere is on the decline, even the Czech Republic reported it's at an all time low. Oof.The results of the World Beer Cup are in and we break it down a bit. Some interesting details from the categories to the countries winning medals. Are any of your favorites on the list?As expected the OG Fat Tire is making a return, for a limited time, at least. We call it McRibbing. Hyping a popular item for a limited return to get the fans to buy it. It works, and not just for beer or pork sandwiches. Even more rare than McRibbing is pulling a Lazarus. But Iron Hill Brewery seems to be doing just that, with the original founder as part of the team. They said they'll be streamlined this time around. We're curious to see what happens here.Cheers!Thanks for listening to Beer Guys Radio!  Your hosts are Tim Dennis and Brian Hewitt with producer Nate "Mo' Mic Nate" Ellingson and occasional appearances from Becky Smalls.Subscribe to Beer Guys Radio on your favorite app: Apple Podcasts | Google Podcasts | Spotify | Stitcher  | RSSFollow Beer Guys Radio: Facebook | Instagram | Twitter | YouTube If you enjoy the show we'd appreciate your support on Patreon.  Patrons get cool perks like early, commercial-free episodes, swag, access to our exclusive Discord server, and more!

Chewing Rage and Honor Flights JB, Sandy, and Tricia kick things off with everyday humor that quickly turns into a conversation about working from home, cabin fever, and those tiny annoyances that can drive you up the wall—yes, including that sound someone makes when they chew.From there, the episode shifts gears into something truly moving: a powerful discussion about Honor Flights

Find Joy in the Wilderness by Autumn Dickson When I was studying the Doctrine and Covenants last year, my pattern revolved around learning about the people who were receiving the revelations and how they were feeling so that we could better relate to them and receive the same comfort in the revelations that they did. As I've studied the Old Testament thus far, I've found a different pattern for learning principles from God. Namely, I look at the details in the class Old Testament stories, and I find the parallels for our day. It's been powerful and helpful. So without further ado, here's another detail from the Moses and Plagues story. The God of the Hebrews is working to free His people from slavery in Egypt. There are some questions that we could ask as to why He didn't jump right in with the death of the firstborn, but those questions can be asked another time. As the Lord continues on with His work through Moses, Pharaoh appears to relent a couple of times. He tells Moses, “Take back the frogs, and I'll let them go.” But then he hardens his heart and refuses to free them. It happens again with the flies. Pharaoh tells Moses to take away the flies and he will let the people go. Here is how Moses responds. Exodus 8:29 And Moses said, Behold, I go out from thee, and I will entreat the Lord that the swarms of flies may depart from Pharaoh, from his servants, and from his people, to morrow: but let not Pharaoh deal deceitfully any more in not letting the people go to sacrifice to the Lord. Of course, Pharaoh goes back on his word and refuses to release them. Maybe he was hoping Moses and His God would run out of power and not be able to send any more plagues? Regardless, Pharaoh still holds the Israelites captive. There is a lot of goodness here, but I want to draw your attention to one detail. Moses wants the Israelites free so that they can go sacrifice to the Lord in the wilderness. This is not the only time this is mentioned. More than once, Moses specifically says this. Pharaoh needs to free the Israelites so that they can go and sacrifice to the Lord out of Egypt and in the wilderness. Oof. Is there a better way to describe life after we finish our ordinances? We are made free by the death of the Firstborn, we pass through the gate, and what do we see? A whole lot of wilderness. For a long time. Why are we here in the wilderness? It seemed so exciting to be free before. Now it just seems dusty, hot, hard, and uncomfortable. Interestingly enough, we didn't walk through those gates to make it into paradise immediately. The gate was just the first step. We've been freed from slavery, but we don't know how to be happy and healthy yet. There are many more lessons to learn. There is a lot of sacrifice to be made so that we can understand what it means to grow to be like the Lord and find what He found. We have a long journey ahead of us. It's funny. I remember being on my mission and working long hours with minimal breaks. I remember rushing to write in my journal at night so that I could pass out in my bed on time and get as much sleep as possible because I was so dang tired. I remember mentally aching when I had to leave the dinner table at the houses of members I was close to. I think that was one of the things I missed the most while I was on the mission. I grew up in a family where we all ate dinner together and talked the whole time. We had a lot of family come into town for holidays, and we would sit at the table for a long time afterwards and talk and laugh. I missed that resting while on my mission. I remember getting on the plane, and I was so excited to eat a meal and then do nothing afterwards. I was excited to rest. Lol. I did get some rest for a while, but heaven knows life only speeds up after that. We came here to struggle in the wilderness, to keep putting one foot in front of the other, to make sacrifices and grow and learn what we're supposed to learn. We didn't come here to finish all of that so we could rest. We came to sacrifice in the wilderness. Which sounds horrible, but it doesn't have to be. This was a timely lesson for me. I have a goal right now to be grateful for the opportunity to wear myself out in the name of the Lord. I'm not talking about being a martyr, though sometimes that seems to be my default mode. Rather, I'm talking about completely turning my perspective upside down. I didn't come to earth to preserve energy and my body. I didn't come here to try and completely annihilate stress from my life or reach some magical point where I feel great enough to give all of myself. Rather, when I catch the true feeling behind this goal I made, I find rest when I let go of my own concerns and cheerfully and willingly take advantage of these incredible opportunities God has given to me. Someday I'll get enough sleep (or my body won't need sleep? I don't know?). Someday, I'll have a perfect body and perfect perspective and all my needs met, and I won't have to reach for those things anymore. They will be given to me. I'll have a perfectly clean house with everything I could ask for. Sometimes we get mixed up and wear ourselves out on the things that don't matter, things that will be freely given to us on the other side. We're putting all of our energy and hopes and focus on setting foot on that promised land. What if we let go and trusted that the promised land will make it to us at the right time? What if instead, we focused on the gift of the wilderness and what it has to offer? I have found that when I stop striving to put my feet in the promised land here in mortality, I find beauty and rest and hope and peace in the wilderness. Moses had it right. He didn't tell Pharaoh that he was taking the Israelites to the promised land. Sure, that was the eventual goal, but there were some really important goals along the way before they would even be able to enjoy the promised land. Moses told Pharaoh to release the Israelites so that they could go and sacrifice in the wilderness. When we let go of trying to hold on to ourselves, we find joy in the sacrifices we're asked to make in the wilderness. That's a true principle. I'm grateful my Savior redeemed me. I'm grateful He let me walk through the gate and bind myself to Him through the ordinance of baptism. I'm grateful that He gave me a path with lessons along the way. I'm grateful that I don't have to worry about reaching the promised land; He's got that handled. All I have to worry about is learning along the way, sacrificing along the way. I'm grateful for my testimony that He will provide for all that I need in the wilderness. Autumn Dickson was born and raised in a small town in Texas. She served a mission in the Indianapolis Indiana mission. She studied elementary education but has found a particular passion in teaching the gospel. Her desire for her content is to inspire people to feel confident, peaceful, and joyful about their relationship with Jesus Christ and to allow that relationship to touch every aspect of their lives. Autumn was the recipient of FAIR's 2024 John Taylor Defender of the Faith Award. The post Come, Follow Me with FAIR – Exodus 7–13 – Part 2 – Autumn Dickson appeared first on FAIR.