Podcasts about Allen Institute

Earlier this month, the organizations EverythingALS and Vision 2030 announced a partnership with the Allen Institute to accelerate research into the progression of amyotrophic lateral sclerosis, also known as Lou Gehrig's disease or ALS. The disease is a progressive neurodegenerative condition where patients lose the ability to walk, use their hands, talk and eventually breathe. But the new investment will be geared towards finding using AI-powered technology to look for a cure at the cellular level. Tech entrepreneur and founder of EverythingALS, Indu Navar, and ALS patient living in Covington, Louisiana, Tim Fulham, join us for more on their journeys with the disease and ongoing research. Over the last two weeks, we've brought you parts one and two of the latest episode of Sea Change: Losing Paradise. In each episode, we learned about the ongoing battle between fisherman and oil companies over the rights to drowned land. This fight is playing out on land, sea and in the courtroom. But is there a way out of this deadlock that won't take decades of case-by-case lawsuits?LSU Law School professor John Lovett has been studying this issue for years. He spoke with the host of Sea Change, Carlyle Calhoun, about what believes might be a solution to the long simmering battle. ___Today's episode of Louisiana Considered was hosted by Alana Schreiber. Our managing producer is Alana Schreiber. Matt Bloom and Aubry Procell are assistant producers. Our engineer is Garrett Pittman.You can listen to Louisiana Considered Monday through Friday at noon and 7 p.m. It's available on Spotify, Google Play and wherever you get your podcasts. Louisiana Considered wants to hear from you! Please fill out our pitch line to let us know what kinds of story ideas you have for our show. And while you're at it, fill out our listener survey! We want to keep bringing you the kinds of conversations you'd like to listen to.Louisiana Considered is made possible with support from our listeners. Thank you!

Getting into the clinic fast to de-risk assets has become the name of the game in biotech, and at the academia-industry interface, too.From AI to NAMs to starting a Phase I trial in the U.S., BioCentury's 3rd Grand Rounds-U.S. conference brought together academic innovators, biopharma leaders and early-stage investors to debate key bottlenecks in translation and how to make early-stage R&D investible.Sam Blackman, entrepreneur in residence at GV and co-founder of Day One Biopharmaceuticals, and Aaron Coe, managing director of innovation for the Allen Institute, joined BioCentury's analysts on stage last week in Seattle for a podcast recording to wrap up Grand Rounds and discuss key takeaways from the event.Editor's note: We invite you to join BioCentury and Regional Host Chairs Forbion and BGV at our next edition of BioCentury Grand Rounds, scheduled for Sept. 23-25 in Amsterdam.View full story: https://www.biocentury.com/article/659729#TranslationalScience #DrugDevelopment #BiopharmaInnovation #AcademicInnovation #GrandRoundsUS00:53 - World-Class Regional Hosts02:56 - Building Grand Rounds Community05:21 - Two Nobels, One City07:43 - AI Goes End-to-End09:47 - The Data Problem14:12 - AI, Animals, Australia19:53 - Study Startup Bottlenecks26:11 - Early Science InvestabilityTo submit a question to BioCentury's editors, email the BioCentury This Week team at podcasts@biocentury.com.Reach us by sending a text

Welcome back to the communal Schauer, make sure to don your tin foil hat because this week we're wading into the murky waters of interventionism and bungled progress. A word of warning to those of childbearing age: I swear a lot in this and it has been scientifically proven that profanity makes your ovaries grieve.  Tear. Pour. Live More. Go to https://LiquidIV.com and get 20% off your first purchase with code SCHAUER at checkout Download Hily Dating App from the App Store or Google Play, or visit https://hily.go.link/jRMKW And yes, I do apologize for the late upload, I'm trying to get the hang of recording at home. I appreciate everyone's patience, you all are incredible and should be celebrated. I hope you all enjoy this week's episode! The Allen Institute's Collab w/ KEXP  https://alleninstitute.org/kexp  I helped with this! General Resources: Alzheimer's Disease https://emedicine.medscape.com/article/1134817-overview#a2  Anatomy, Abdomen and Pelvis: Celiac Ganglia https://www.ncbi.nlm.nih.gov/books/NBK538129/#article-19097.s6 Federal Food, Drug, and Cosmetic Act of 1938 https://www.ncbi.nlm.nih.gov/books/NBK585046/ Toxic Effects of Mercury on the Cardiovascular and Central Nervous System https://pmc.ncbi.nlm.nih.gov/articles/PMC3395437/  Tampons as a source of exposure to metal(loid)s https://www.sciencedirect.com/science/article/pii/S0160412024004355  Patents on Psychedelics: The Next Legal Battlefront of Drug Development  https://harvardlawreview.org/forum/no-volume/patents-on-psychedelics-the-next-legal-battlefront-of-drug-development/ Caffeine-Induced Psychosis: A Case Report and Review of Literature https://pmc.ncbi.nlm.nih.gov/articles/PMC11376648/  The effect of caffeine and stress on auditory hallucinations in a non-clinical sample https://www.sciencedirect.com/science/article/abs/pii/S019188691000591X  Scientists Stop Pancreatic Cancer Before It Starts in Landmark Preclinical Study https://scitechdaily.com/scientists-stop-pancreatic-cancer-before-it-starts-in-landmark-preclinical-study/  The Brain Waste System Disrupted by Alzheimer's Mapped https://neurosciencenews.com/glymphatic-brain-waste-clearance-30785/  The Resurgence of Hallucinogen Drugs in Clinical Research https://www.sciencedirect.com/science/article/pii/S0034837625001457  Residential psychedelic (LSD) therapy for the narcotic addict. A controlled study https://pubmed.ncbi.nlm.nih.gov/4575166/  This study is from 1973 - if you would do me the favor of scrolling down to the “similar articles” section I'd like you to note the dates of publication for related research. If you're seeing what I'm seeing, psychedelics could've really helped a lot of people. Books Clean: The New Science of Skin and the Beauty of Doing Less - James Hamblin Natural Capitalism: Creating the Next Industrial Revolution - Paul Hawken Undermining Science: Suppression and Distortion in the Bush Administration - Seth Shulman Sweet and Deadly: How Coca-Cola Spreads Disinformation and Makes Us Sick - Murray Carpenter How to Change Your Mind: What the New Science of Psychedelics Teaches Us About Consciousness, Dying, Addiction, Depression, and Transcendence - Michael Pollan Unwell Women: Misdiagnosis and Myth in a Man-Made World - Elinor Cleghorn A History of Transgender Medicine in the United States: From Margins to Mainstream - Carolyn Wolf-Gould, Dallas Denny, Jamison Green, Kyan Lynch, Editors Food & Lobbying Resources: Nutrition Websites & Databases https://libguides.regiscollege.edu/nutrition/intro EWG's Food Scores https://www.ewg.org/foodscores/ Open Secrets https://www.opensecrets.org/  Learn more about your ad choices. Visit podcastchoices.com/adchoices

Christof Koch is a neuroscientist at the Allen Institute for Brain Science and the Chief Scientist of the Tiny Blue Dot Foundation. As a professor for 25 years at Cal Tech, he pioneered research on the neural basis of consciousness, and he has advanced one of the leading theories of consciousness called Integrated Information Theory. Among his many publications, Christof's most recent book is entitled Then I Am Myself the World: What Consciousness Is and How to Expand It. In our mini-series on the nature of intelligence, we have talked with Simon Conway Morris about the evolution of intelligence, with Susan Schneider about the rise of artificial intelligence, and with Laszlo Barabasi about the structure of collective intelligence. Check our Templeton Ideas podcast feed to listen to these episodes. Follow us on social media: Twitter, Facebook, Instagram, LinkedIn, and YouTube.

The Trump administration has removed ocean observation stations from waters off the WA coast, Mayor Wilson asks voters to renew the Seattle Transit Measure, and the Allen Institute makes major investments to treat brain diseases. It’s our daily roundup of top stories from the KUOW newsroom, with host Paige Browning. And make sure to join us this Saturday for a live taping of Seattle Now where Patricia Murphy will interview King County Executive Girmay Zahilay at the Cascade PBS Ideas Festival. Get tickets here. Use promo code SEATTLENOW to access a 20% discount. We can only make Seattle Now because listeners support us. Tap here to make a gift and keep Seattle Now in your feed. Got questions about local news or story ideas to share? We want to hear from you! Email us at seattlenow@kuow.org, leave us a voicemail at (206) 616-6746 or leave us feedback online.See omnystudio.com/listener for privacy information.

Rubén Lozano-Aguilera, Product Lead for Asta at the Allen Institute for AI (Ai2), explores the transformative potential of agentic AI in scientific research with Humanitarian AI Today guest host Lindsey Moore, Founder of DevelopMetrics. Rubén introduces AutoDiscovery, a powerful new tool developed by his team that moves beyond traditional query-based analysis to autonomously generate and test scientific hypotheses. This shift from manual data processing to autonomous discovery offers a powerful force multiplier for researchers, helping them surface blind spots and hidden patterns that traditional methods often overlook in fields ranging from melanoma research to marine ecology.   For humanitarian and development organizations, Ai2's work represents a vital new advancement in what Rubén calls "shared AI infrastructure." Ai2's deep commitment to the open-source movement, providing open models, checkpoints, code, and training data, ensures transparency and accessibility for all. This approach is particularly impactful for organizations operating in resource-constrained environments, as it allows them to leverage state-of-the-art predictive analytics without the high costs or "black box" risks associated with proprietary systems. By democratizing access to high-level research tools, Ai2 enables any researcher or developer to maintain data ownership while utilizing sophisticated AI to solve the world's most pressing problems.   The conversation next turned to the deeper philosophical stakes of automating scientific discovery itself. Drawing on his graduate research in AI ethics at Cambridge, Rubén separates what philosophers of science call the "context of discovery”, how a hypothesis is generated, from the "context of justification," how it is tested and validated. The worry is deskilling: as scientists offload hypothesis generation to AI, will they lose the instincts needed to catch when the machine is wrong? His answer centers on cultivating "meta-AI skills", the practiced ability to evaluate AI outputs critically. That raises its own problems: how do those skills get built, and are they really the same kind of skill as the hypothesis-generating instincts they would replace? Ai2 is actively studying this by examining how tools like AutoDiscovery affect students and early-career researchers. For humanitarian and development professionals navigating an era of shrinking research budgets and growing AI adoption, these added points raise essential questions about keeping human judgment at the center of discovery.

In this episode, Niall speaks with Dr. Christof Koch, Chief Scientist of the MindScope Program at the Allen Institute for Brain Science, former Professor at Caltech, and author of “Then I Am Myself the World”. Dr. Koch is a leading researcher in the science of consciousness and a key proponent of Integrated Information Theory. In this conversation, they explore: — Why consciousness may be fundamental, while physical matter exists only in relation to other things — How an experience on a beach in Brazil changed his understanding of reality — The discovery of “covert consciousness” in patients thought to be in vegetative states — How the perturbational complexity index (PCI) shows a clear boundary between conscious and unconscious states, and why this matters — How Integrated Information Theory approaches the question of free will You can learn more about Dr. Koch's work at https://christofkoch.com. --- Dr. Christof Koch is a Meritorious Investigator at the Allen Institute. Christof received his baccalaureate from the Lycée Descartes in Rabat, Morocco, his B.S. and M.S. in physics from the University of Tübingen in Germany and his Ph.D. from the Max-Planck Institute for biological Cybernetics in 1982. Subsequently, he spent four years as a postdoctoral fellow in the Artificial Intelligence Laboratory and the Brain and Cognitive Sciences Department at the Massachusetts Institute of Technology. From 1987 until 2013, Koch was a professor at the California Institute of Technology (Caltech) in Pasadena, from his initial appointment as Assistant Professor, Division of Biology and Division of Engineering and Applied Sciences in 1986, to his final position as Lois and Victor Troendle Professor of Cognitive & Behavioral Biology. See here for Christof's academic pedigree and his students. Christof joined the Allen Institute for Brain Science as Chief Scientific Officer in 2011 and became President in 2015. Christof writings and interests integrate theoretical, computational and experimental neuroscience with philosophy and contemporary trends, in particular artificial intelligence. His latest book, Then I Am Myself the World: What Consciousness Is and How to Expand It, publish in May 2024. His previous book, Consciousness: Confessions of a Romantic Reductionist, blends science and memoir to explore topics in discovering the roots of consciousness. Stemming in part from a long-standing collaboration with the late Nobel Laureate Francis Crick, Christof authored the book The Quest for Consciousness: A Neurobiological Approach. Koch also authored the technical books Biophysics of Computation: Information Processing in Single Neurons and Methods in Neuronal Modeling: From Ions to Networks, and served as editor for several books on neural modeling and information processing. --- Interview Links: — Dr. Koch's website: https://christofkoch.com — Dr. Koch's book: https://amzn.to/4mIKG9W

Bill Hilf has spent decades enterprise tech, open-source technologies, and AI, from IBM and Microsoft to running Paul Allen's portfolio as the CEO of Vulcan. He now chairs the Allen Institute for AI and American Prairie. His debut sci-fi novel, "The Disruption," imagines AI gone very wrong, and implicitly challenges the industry to think differently about how it's building our real future today. With GeekWire co-founder Todd Bishop. Edited by Curt Milton.See omnystudio.com/listener for privacy information.

[Expertpanelen] Avsnitt 156 med Alexander Öqvist, SEO-specialist på Synlighet, om de viktigaste nyheterna och trenderna inom sök och SEO under första kvartalet 2026. Reder ut buzzword-stormen kring SEO, GEO och AEO samt vad som egentligen skiljer mellan dessa, om något. Vi pratar också om hur agentic commerce har tagit fart med Googles UCP, OpenAIs ACP och Shopifys olika initiativ. Plus en spaning från Alexander om att många företag just nu bygger om eller byter plattform. Du får dessutom höra om: Våra takes kring om GEO och AEO behövs Att få företag har börjat med agentic commerce Varför produktfeeden blir ännu viktigare Drivkrafterna bakom migreringsvågen just nu WebMCP som manual för AI-agenter på sajter Google börjar straffa massproducerat AI-innehåll Microsofts verktyg för att mäta synlighet i AI-sök Om gästen Alexander Öqvist är AI Lead, senior digital strateg och SEO-specialist på Synlighet, en norsk marknadsföringsbyrå. Han har tidigare varit ansvarig för Danske Banks svenska webb och ledde arbetet med SEO i Norden.  Alexander är också en del av min expertpanel och fokuserar på SEO och organiskt sök. Tidsstämplar [00:01:54] Buzzword-stormen kring SEO, GEO och AEO. Alexander och Tony reder ut skillnaderna mellan begreppen, varför de inte är separata discipliner, och vad som faktiskt är nytt jämfört med bara SEO i ny förpackning. [00:20:43] Agentic commerce tar fart på riktigt. Genomgång av Googles UCP, Shopifys Agentic Plan och OpenAIs svängning från Instant Checkout. Plus vad det betyder för svenska e-handlare, logistiken och vem som äger kunden. [00:37:35] Migreringsvågen och WebMCP. Alexander berättar om att 7 av 10 av hans kunder just nu bygger om eller byter plattform, vad som driver det, och varför WebMCP blir en viktig del när sajter framtidssäkras för AI-agenter. [00:50:05] Lightning round om AI-innehåll och mätning. Hur Google börjar straffa massproducerat AI-innehåll, Google-uppdateringar, och nya förstapartsverktyg för att mäta AI-synlighet från Microsoft. Länkar Alexander Öqvist på LinkedIn Synlighet (webbsida) How Jason Barnard introduced AEO to the world in 2018 – Jason Barnard / SEMrush (artikel) GEO: Generative Engine Optimization – Princeton, Georgia Tech, Allen Institute for AI  (studie)  AIs are highly inconsistent when recommending brands – SparkToro (artikel) Agentic commerce, AI tools, protocol, retailers, platforms – Google (artikel)  AI shopping gets simpler with Universal Commerce Protocol updates – Google (artikel) About the Universal Commerce Protocol (UCP) and UCP-powered checkout feature on Google – Google Merchant Center (dokumentation)  The agentic commerce platform: Shopify connects any merchant to every AI conversation  – Shopify (artikel)  Introducing the Shopify Agentic plan – Shopify (webbsida)  Powering product discovery in ChatGPT – OpenAI (artikel)  WebMCP is available for early preview – Google (artikel) Is Google finally cracking down on best-of lists? – Lily Ray / Substack (artikel)  Does AI content rank well in search?  – Semrush (studie) Introducing AI Performance in Bing Webmaster Tools – Bing (artikel)  Understanding Your Influence in AI Answers with Microsoft Clarity (Early Access) – Microsoft (artikel)  Introducing the branded queries filter in Search Console – Google (artikel)

Fresh off raising a monster $15B, Marc Andreessen has lived through multiple computing platform shifts firsthand, from Mosaic and Netscape to cofounding A16z. In this episode, Marc joins swyx and Alessio in a16z's legendary Sand Hill Road office to argue that AI is not just another hype cycle, but the payoff of an “80-year overnight success”: from neural nets and expert systems to transformers, reasoning models, coding, agents, and recursive self-improvement. He lays out why he thinks this moment is different, why AI is finally escaping the old boom-bust pattern, and why the real bottleneck may be less about models than about the messy institutions, incentives, and social systems that struggle to absorb technological change.This episode was a dream come true for us, and many thanks to Erik Torenberg for the assist in setting this up. Full episode on YouTube!We discuss:* Marc's long view on AI: from the 1980s AI boom and expert systems to AlexNet, transformers, and why he sees today's moment as the culmination of decades of compounding technical progress* Why “this time is different”: the jump from LLMs to reasoning, coding, agents, and recursive self-improvement, and why Marc thinks these breakthroughs make AI real in a way prior cycles were not* AI winters vs. “80-year overnight success”: why the field repeatedly swings between utopianism and doom, and why Marc thinks the underlying researchers were mostly right even when the timelines were wrong* Scaling laws, Moore's Law, and what to build: why he believes AI scaling laws will continue, why the outside world is messier than lab purists assume, and how startups can still create durable value on top of rapidly improving models* The dot-com crash and AI infrastructure risk: Marc's comparison between today's AI capex boom and the fiber/data-center overbuild of 2000, plus why he thinks this cycle is different because the buyers are huge cash-rich incumbents and demand is already here* Why old NVIDIA chips may be getting more valuable: the pace of software progress, chronic capacity shortages, and the idea that even current models are “sandbagged” by supply constraints* Open source, edge inference, and the chip bottleneck: why Marc thinks local models, Apple Silicon, privacy, trust, and economics all point toward a major role for edge AI* American vs. Chinese open source AI: DeepSeek as a “gift to the world,” why open models matter not just because they're free but because they teach the world how things work, and how open source strategies may shift as the market consolidates* Why Pi and OpenClaw matter so much: Marc's claim that the combination of LLM + shell + filesystem + markdown + cron loop is one of the biggest software architecture breakthroughs in decades* Agents as the new “Unix”: how agent state living in files allows portability across models and runtimes, and why self-modifying agents that can extend themselves may redefine what software even is* The future of coding and programming languages: why Marc thinks software becomes abundant, why bots may translate freely across languages, and why “programming language” itself may stop being a salient concept* Browsers, protocols, and human readability: lessons from Mosaic and the web, why text protocols and “view source” mattered, and how similar principles may shape AI-native systems* Real-world OpenClaw use: health dashboards, sleep monitoring, smart homes, rewriting firmware on robot dogs, and why the most aggressive users are discovering both the power and danger of agents first* Proof of human vs. proof of bot: why Marc thinks the internet's bot problem is now unsolvable via detection alone, and why biometric + cryptographic proof of human becomes necessaryTimestamps* 00:00 Marc on AI's “80-Year Overnight Success”* 00:01 A Quick Message From swyx* 01:44 Inside a16z With Marc Andreessen* 02:13 The Truth About a16z's AI Pivot* 03:29 Why This AI Boom Is Not Like 2016* 06:33 Marc on AI Winters, Hype Cycles, and What's Different Now* 10:09 Reasoning, Coding, Agents, and the New AI Breakthroughs* 12:13 What Founders Should Build as Models Keep Improving* 16:33 AI Capex, GPU Shortages, and the Dot-Com Crash Analogy* 24:54 Open Source AI, Edge Inference, and Why It Matters* 33:03 Why OpenClaw and PI Could Change Software Forever* 41:37 Agents, the End of Interfaces, and Software for Bots* 46:47 Do Programming Languages Even Have a Future?* 54:19 AI Agents Need Money: Payments, Crypto, and Stablecoins* 56:59 Proof of Human, Internet Bots, and the Drone Problem* 01:06:12 AI, Management, and the Return of Founder-Led Companies* 01:12:23 Why the Real Economy May Resist AI Longer Than Expected* 01:15:53 Closing ThoughtsTranscriptMarc: Something about AI that causes the people in the field, I would say, to become both excessively utopian and excessively apocalyptic. Having said that, I think what's actually happened is an enormous amount of technical progress that built up over time. And like for, for example, we now know that neural network is the correct architecture.And I, I will tell you like there was a 60 year run where that was like a, you know, or even 70 years where that was controversial. And so, so the way I think about what's happening is basically, I think, I think about basically the, the, the period we're in right now is it's, I call it 80 year overnight success, right?Which is like, it's an overnight success ‘cause it's like bam, you know, chat GPT hits and then, and then oh one hits, and then, you know, open claw hits and like, you know, these are open, these are, these are like overnight, like radical, overnight transformative successes, but they're drawing on an 80 year sort of wellspring backlog, you know, of, of, of, of ideas and thinking it's not just that it's all brand new, it's that it's an unlock of all of these decades of like very serious, hardcore research.If I were 18, like this is a hundred, this is what I would be spending all of my time on. This is like such an incredible conceptual breakthrough.swyx: Before we get into today's episode, I just have a small message for listeners. Thank you. We will not be able to bring you the ai, engineering, science, and entertainment contents that you so clearly want if you didn't choose to also click in and tune into our content.We've been approached by sponsors on an almost daily basis, but fortunately enough of you actually subscribed to us to keep all this sustainable without ads, and we wanna keep it that way. But I just have one favor to ask all of you. The single, most powerful, completely free thing you can do is to click that subscribe button.It's the only thing I'll ever ask of you, and it means absolutely everything to me and my team that works so hard to bring the in space to you each and every week. If you do it, I promise you will never stop working to make the show even better. Now, let's get into it.Alessio: Hey everyone, welcome to the Lidian Space Pockets. This is CIO, founder Kernel Labs, and I'm joined by s Swix, editor of Lidian Space.swyx: Hello. And we're in a 16 Z with a, uh, mark G and welcome.Marc: Yes, yes. A and what, half of 16? Something like that. A one. Exactly,swyx: exactly. Uh, apparently this is the, the final few days in your, your current office.You're moving across the road.Marc: Uh, we're, yeah. We have a, we have some, we have some projects underway, but yeah, this is actually, oh, this is the original. We're in actually the original office. We're in the, we're in the, we're, we're in the whole thing.swyx: It's beautiful. Yeah. Great.Marc: Thank you.swyx: So I have to come out, uh, this is a, you know, I wanted to pick a spicy start in October, 2022.I just made friends with Roone and, uh, I wanted to give him something to sort of be spicy about. And I said, uh. Uh, it'll never not be funny. The A 16 Z was constantly going. The future is where the smart people choose to spend their time and then going deep into crypto and not in ai. And that was in October 22nd, 2022.And Ruen says there was an internal meeting in a 16 Z to reorient around Gen ai. Obviously you have, but was there a meeting? What, what was that?Marc: I mean, I don't, look, I've been doing AI since the late eighties.swyx: Yeah.Marc: So I, I don't know, like all that, as far as I'm concerned, this stuff is all Johnny cum lately.Yeah. You, I mean, look, we've been doing ar entire existence. I mean, we've been doing AI machine learning deep, you know, deeply. We've been doing this stuff way from the beginning. Obviously a AI is just core to computer science. I, I, I actually view them as like quite, uh, quite continuous. Um, you know, Ben and I both have computer science degrees.Um, you know, we, we both, Ben, Ben and I actually both are world enough to remember the actual AI boom in the 1980s. Yeah. There was like a, there was a big AI boom at the time. Um, and there was a, was names like expert systems. Um, and they of like lisp and lisp machines. Uh, I, I coded in lisp. I was coding a lisp in 1989.When that was the, the language of the AI future. Um, yeah. So this is something that we're like completely, you completely comfortable with. I've been doing the whole time and are very enthusiastic aboutswyx: is there a strong, like this time is different because, uh, my closest analog was 20 16 17. It was an AI boom.Mm-hmm. And it petered out very, very quickly. Um, we, it just, it just in terms of investingMarc: sort of, sort of,swyx: yeah. Investment, investment excitement.Marc: Although that's really when the, the, the Nvidia phenomenon really, it was, I would say it was in that period when it was very clear that at, at the time it, the vocabulary was more machine learning, but it, it was very clear at that time that machine learning was hitting some sort of takeoff point.Alessio: Yeah.Marc: Well, and as you guys, you guys have talked about this at length on, on your thing, but, you know, if you really track what happened, I think the real story is, it was, it was the Alex net, uh, basically breakthrough in like 2013. That was the, that was the real knee in the curve. Um, and then it was obviously the transformer breakthrough in 17.Alessio: Yeah.Marc: Um, and then everything that followed. But, but, you know, look, machine learning, you know, there were, you know, look, uh, I mean look, I've been working, you know, I've been working with, uh, one of my, you know, kind of projects working with Facebook since 2004. Um, and on the board since 2007, and of course, you know, they, they started using machine learning very early, um, and, you know, have used it basically, you know, for like 20 years for, you know, content, you know, feed optimization and advertising optimization.And obviously many, you know, financial services. You know, many, many, many companies, many different sectors have been doing this. And so it's like one of these things, it's like, it's not a, it's not a single thing. Like it's, it's like, it's like layers, right? Yeah. Um, and, and the layers arrive at different paces and, but they kind of build up.swyx: Yeah.Marc: Uh, they kind of build up over time and then, and then, yeah. And then look, in retrospect, it was 2017 was kind of the, you know, the key, the key point with the trans transformer and then. And then as you guys know, there was this really weird like four year period where it's like the, the transformer existed and then it was just like,swyx: let's go.Yeah.Marc: Well, but, but it was just, but, but between 2020, but between 2017 and 2021, I mean, that was the era of which like companies like Google had internal chat Botts, but they weren't letting anybody use them.swyx: Yeah.Marc: Right. And then, you know, and then OpenAI developed Chat GT or GPT two, and then they told everybody, this is way too dangerous to deploy.Right. Yeah. You know, we can't possibly let normal people, normal people use this thing. And then you, you guys, I'm sure remember AI Dungeon, um mm-hmm. So the o for, there was like a year where like the only way for a normal person to use GP T three was in, in AI dungeon.Alessio: Yeah.Marc: And so you, you, we would do this, you'd go in there and you'd pretend to play Dungeons and Dragons.In reality, you're just trying to talk to talk to GPT. And so there was this, you know, there was this long, you know, and I, you know, the big, big companies, you know, big companies are cautious and, you know, the big companies were cautious. It, it, by the way, it took open ai. You know, they, they, they talk about this, it took open AI time to actually adjust, you know, kind of re redirect their researchswyx: path.I, I think, uh, let say Rosewood, right? Uh, the, the dinner that founded OpenAI was right there.Marc: Right, right. But that, that dinner would've taken place in 20swyx: 18Marc: 19. The formation of OpenAI Uhhuh as late as 2018.swyx: Uh, uh, sorry. Uh, no, I'm, I'm, I'm, I'm wrong. Probably It should be 20. Yeah. They just celebrated a 10 year anniversary, so it it is 2025.Yeah, so, so 2015?Marc: Yeah. 2015. Yeah. 2015. But then, uh, um, Alec Radford did G PT one in what, probablyswyx: mm-hmm. 17, 18,Marc: yeah. 17, 18. So it, yeah. For, and then, and then they didn't really, and then GPT three was what? 2020? 2020.swyx: 2020.Marc: Because that became copilot immediately. Even open ai, which has been, you know, the leader of, of this thing in the last decade, you know, e even they had to adapt and, and, and lean into the new thing.And so. Um, yeah, I, I think it's just this process of basically sort of wave after wave layer after layer, you know, building on itself. And then you kind of get these catalytic moments where, where the whole thing pops and, and obviously that's what's happening now.swyx: Is it useful to think about will there be any ai, winter?‘cause there's always these patterns. Like, is this, in the summer is something I constantly think about because do I get, do I just like. Just get endlessly hyped and just trust that I will only be early and never wrong or right. Well, are we, will there be a winter?Marc: So there's something about, say the following.There's something about AI that has led to this repeated pattern. Um, and, and, and you guys know this,swyx: it's summer, winter, summer,Marc: winter, summer, winter, summer, winter. And it goes back 80 years. Yeah. 80 years. Uh, so the original neural network paper was 1943. Right. Which is, which is amazing. Uh, that it was, it was far back that long.And then there was you, if you guys have ever talked about this on your show, but there was this, uh, there was a big, uh, there was an a GI conference at Dartmouth University in 1950. 55. 55, yeah. And they got a NSF grant to, uh, for the, all the AI experts at the time to spend the summer together. And they figured if they had 10 weeks together, they could get a GI, uh, at the other end.And they got their, by the way, they got the grant, they got the 10 weeks and then, you know, 1955, you know. No, no. A GI. And like I said, I, I lived through the eighties version of this where there was a big, a big boom and a crash. And so, so there is this thing, and there, there is something about AI that causes the people in the field, I would say, to become both excessively utopian and excessively apocalyptic.Um, and, and it's probably on both sides of like the, the, the boom bus cycle. You, you kind of see that play out. Having said that, I think what's actually happened is like just, and you know, and we now know in retrospect like an enormous amount of technical progress that built up over time. And like for, for example, we now know that neural network is the correct architecture.And I, I will tell you like there was a 60 year run where that was like a, you know, or even 70 years or that was controversial. And, and we now know that that's the case. And so we, we now, you know, everything we're building on today just sort of derives from the original idea in 1943. And so, so in retrospect, we, we now know that like, these, these guys are right.They, they, you know, they would get the timing wrong and they thought, you know, capabilities would arrive faster, or they were, it could be turned into businesses sooner or whatever, but like, they were fundamentally, the, the scientists who worked on this over the course of decades were fundamentally correct about what they were doing.And, and the, and the payoff from, from, from all their work is happening now. And so, so the way I think about what's happening is basically, I think, I think about basically the, the, the period we're in right now is it's, I call it 80 year overnight success, right? Which is like, it's an overnight success.‘cause it's like bam, you know, chat, GPT hits and then, and then oh one hits, and then, you know, open claw hits and like, you know, these are open, these are, these are like overnight, like radical, overnight transformative successes, but they're drawing on an 80 year sort of wellspring backlog, you know, of, of, of, of ideas and thinking it's not just that it's all brand new, it's that it's an unlock of all of these decades of like very serious, hardcore research.Um, and thinking, and look, there were AI researchers who spent their entire lives. They got their PhD. They, they worked for, they've researched for 40 years. They retired in a lot of cases, they passed away and they never actually saw it work.swyx: Yeah. It's all sad.Marc: It is. It is sad. It's sad. Knewswyx: Jeff Hinton was like the last guy.Marc: Yeah. Yeah. Well, there were the guys, uh, was a guy, Alan Newell. I mean, there's tons of John McCarthy. You know, John McCarthy was like one of the inventors in the field. He's one of the guys who organized the Dartmouth Conference and you know, he taught at Stanford for 40 years. Wow. And passed, you know, passed away, I don't know, whatever, 10, 10 years ago or something.Never, never actually go. Got to see it happen. But like, it is amazing in retrospect, like, these guys were incredibly smart and they worked really hard and they were correct. So anyway, so then it's like, okay, you know, say history doesn't repeat, but it rhymes. It's like, okay, does that mean that there's gonna be another, like, you know, basically boom buzz cycle.And I, I will tell you, like, let, like in a sense, like yes, everything goes through cycles and, you know, people get overly enthusiastic and overly depressed and there's, there's a time, there's a timelessness to that. Having said that, there's just no question. Um, so the form, the foremost dangerous words in investing this time are, this time is different.Do you know the 12 most dangerous words investing? No. The four most d foremost dangerous words in investing are this time is different. Yeah. Um, the 12 most dangerous words. And so like, I'll tell you what's different. Like now it's working like, like there's just no, I mean, look, there's just no question.And by the way, I, I'll just give you guys my take. Like L LLMs, like from, from basically the Chad G PT moment through to spring of 25. I think you could still, I think well intention, well, and of. Form skeptics could still say, oh, this is just pattern completion. And oh, these things don't really understand what they're doing.And you know, the hall hallucination rates are way too high. And, you know, this is gonna be great for creative writing and creating, you know, Shakespeare and so sonnets and, you know, as, as rap lyrics or whatever, like, it's gonna be great and all that stuff, but we're not gonna be able to harness this to make this relevant in, you know, coding or in medicine or in law or in, you know, you know, kind of feels that, you know, kind of really, really matter.And I think basically it was the reasoning breakthrough. It, it was oh one and then R one that basically answered that question basically said, oh no, we're gonna be able to actually turn this into something that's gonna work in the real world. And, and then obviously the coding breakthrough over the, over basically the coding breakthrough that kind of catalyzed over the holiday break was kind of the third step in that.Mm-hmm. Where you're just like, alright, if, if, you know, if Linus Tova is saying that the AI coding is no better than he is like. Like, that's, that's never happened before. That's theswyx: benchmark.Marc: Yeah. That's never happened before. And so now we know that it's, it's gonna sweep through coding and, and then, and then we, we know, you know, we know that if it's gonna work in coding, it's gonna work in everything else.Right. It's just then, because that's, that's like, that's like, that's like the hardest in many ways. That's the hardest example. And how everything else is gonna be a, a derivative of that. And then on top of that, we just got the agent breakthrough, you know, with Open Claw, which is fantastic. Which is amazing and incredibly powerful.And then we just got the, the, um, the auto research, uh, you know, the, the self-improvement. You know, we're now into the self-improvement breakthrough. And so the, so the way I think about it is we've had four fundamental breakthroughs in functionality, l OMS reasoning, uh, agents, um, and then, uh, and, and then now RSI, um, and, and they're all actually working.Um, and so I'm, I'm just, as you like, you can tell I'm jumping outta my shoes. Like, like this is, like this is it like this, this is the culmination of 80 years worth of worth of work, and this is the time it's becoming real.Alessio: Yeah.Marc: I, I'm completely convinced.Alessio: I think the anxiety that people feel is like during the transistor era, yet Mors law, and it's like, all right, we understand why these things are getting better.We understand the physics of it. Yeah. With ai, it's. It's so jagged in like the jumps where like, like you said, it's like in three months you have like this huge jump like, and people are like, well this can keep happening. Right? But then it keeps happening,Marc: it'll keep happening.Alessio: And so like how do you think about also timelines of like what's we're building?I think we always have this question with guests, which is like, you know, should you spend time building harness for a model versus like the next model just gonna do it one shot in the lead space. Right. And how does that inform, like how you think about the shape of the technology? You know, you talk about how it's a new computing platform.If you have a computing platform, then like every six months it like drastically changes in what it looks like. It's hard to build companies on top of it.Marc: Yeah. So, so a couple things. So one is like, look, the, the Moore's law was what we now call a scaling law. Like Moore's Law was a scaling law and for your younger viewers, more Moore's Law was every chip chip chips either get twice as powerful or twice as cheap every, every 18 months.And that, and that and that, you know, that it's gotten more complicated in the last few years. But like that, that was like the 50 year trajectory of, of, of the computer industry. And then, and then by the way, and that's what took the mainframe computer from a $25 million current dollar thing into, you know, the phone in your pocket being, you know, a million times more powerful than that.Like that, you know, for, for 500 bucks. And so that, that was a scaling law. And then, and then, and then key to any scaling law, including Moore's Law and the AI scaling laws is, you know, they're not really laws, right? They're, they're, they're, they're predictions, but when they work, they become self-fulfilling predictions because they, they, they, they, they set a benchmark and, and then the entire industry, right?All the smart people in the industry kind of work to make sure that, that, that actually happens. And so they, they kind of motivate the breakthroughs that are required to, to keep that going. And, and in and in chips, that was a 50 year, that was a 50 year run. Right. And it, it was amazing. And it's still happening in, in some areas of, of chips.I think the same thing is happening with the, the core scaling laws. The core scaling laws. In, in, in ai, you know, they're, they're not really laws, but like they, they are basically. There are predictions and then they're motivating catalysts for the research work that is required to be. And, and, and, and by the way, also the investment, uh, dollars, um, uh, you know, required to basically keep, you know, keep the curves going and, and look, it, it is, it's gonna be complicated and it's gonna be variable and they're, you know, there're gonna be walls that are gonna look like they're fast approaching, and then they're gonna be, you know, engineers are gonna get to work and they're gonna figure out a way to punch through the walls.And obviously that's, you know, that's been happening a lot, you know, and then look, there's gonna be times when it looks like the walls have, you know, the, the, the laws have petered out and then they're gonna, they're gonna pick up again and surge and then, and then, and then it, it appears what's happening to the eyes is there's not multiple, you know, multiple scaling laws.Um, there's multiple areas of improvement. And, and I think, you know, I don't know how many more there are already yet to be discovered, but there are probably some more that we don't know about yet. You know, they, like, for example, there's probably some scaling law around, um, world models and robotics that we don't fully understand, you know, kind of acquisition of data at scale in the real world that we don't fully understand yet.So that, that, that one will probably kick in at some point here. There's a bunch of really smart people working on that. Um, and so, yeah, I, I think the expectation is that, that, you know, the, the scaling laws generally are gonna continue. Yeah. The, the pace of improvement will continue to move really fast.Um. To your question on like what to build. So, uh, I'm a complete believer the scaling laws are gonna continue. I'm a complete believer the capabilities are gonna keep getting amazing, um, you know, leaps and bounds. Uh, the part where I kind of part ways a little bit with how, what I would describe as the AI purists, um, you know, which is, which I would characterize as like the people who are.In many ways, the smartest people in the field, but also the people who spend their entire life, like at a lab, um, and have, have, I would say, have very little experience in the outside world. Um, the, the, the nuance I would offer is the outside world of 8 billion people and institutions and governments and companies and economic systems and social systems is really complicated.Um, and, um, and doesn't, you know, it it 8 billion people making collective decisions on planet Earth is not a simple process of like, just like you see this happening now. It's like a bunch of AI CEOs have this thing, which is just like, well, there's just this, they just all have this kind of thing when they talk in public where they're just like, well, there's these, these obvious set of things that so society to do.Alessio: Mm-hmm.Marc: And then they're like, society's not doing any of those things. Right. And it's like, how can society not, you know, what, whatever their theory is, how can society not see x, y, Z? Mm-hmm. And the answer is, well, society is number one. There's no single society, it's like 8 billion people. And they like all have a voice, and they all have a vote, like at the end of the day of how they, they react to change.And then, you know, it just like, it's just human reality is just really complicated and messy. Um, and, and, and so the specific answer to your question is like, as usual, it depends. Um, you know, it, it depends. Look, pe there's no question people are gonna, like, there's no question they're gonna be companies.It's already happening. There are companies that think that they're building value on top of the models and then they're just gonna get blissed by the, by the next model. There's no question that's happening. But I think there's no question also that just the process of adaptation of any technology into the real and into the real messy world of humanity is, is just going to be messy and complicated.It's, it's not going to be simple and straightforward. It's gonna be messy and complicated. And there are gonna be a lot of companies and a lot of products, um, uh, and in, in fact entire industries that are gonna get built to, to, to basically actually help all of this technology actually reach real people.Alessio: The amount of capital going into these companies, I mean, Dario talked about it on the Door Cash podcast and Door Cash was like, why don't you just buy 10 x more GPUs? And he is like, because I'm gonna go bankrupt if the model doesn't exactly hit the, the performance level. How do you think about that?Also as a risk on, you know, you guys are investors, open AI and thinking machines and world apps. It seems like we're leveraging the scaling loss at a pretty high rate, right? Like how comfortable, I guess, do you feel with the downside scenario, like, and say like things Peter out, you think you can kind of like restructure uh, these build outs and uh, you know, capital investments.Marc: Yeah. So should start by saying, so I live through the.com crash, um, and I can tell you stories for hours about the.com crash and it was horrible. No, it was awful. It was, it was, it was apocalyptic by the way. The, a lot of the.com crash was actually at the time, it was actually a telecom crash. It was a bandwidth crash.Like the, the thing that actually crashed, that wiped out all the money with the tele, the telecom companies.swyx: GlobalMarc: crossing. Global, global, yeah.swyx: I'm from Singapore and they, they laid so much cable o over over our oceans.Marc: Actually there was a scaling law in the.com. Era. And it was literally the, the US Commerce Department put out a report in 1996 and they said internet traffic was doubling every quarter.Um, and, and actually in 1995 and 1996, internet traffic actually did double every quarter. And so that became the scaling law. And so what all these telecom entrepreneurs did was they went out and they raised money to build fiber, anticipating that the demand for bandwidth is gonna keep doubling every quarter.Doubling every quarter though is like, you know, grains of chess and the chessboard, like at some point the numbers become extremely large. Right. And, and, and it really, and really what happened was the internet. The internet by the way, continuously kept growing basically since inception. And it's, you know, it's, it's continuously grown.It's never shrunk. And it's grown really fast compared to anything else. Mm-hmm. You know, in, in, in human history. But it wasn't doubling every quarter as of 19 98, 19 99. And so there was this gap in the expectation of what they thought was a scaling law versus reality. And that's actually what caused the.com crash, which was the, it they, they way over companies like global crossing way overbuilt fiber, which is sort of the, and by the way, fiber, telecom equipment, you know, so all the, all the networking gear, you know, and then, and then by the way, the actual physical data centers, like that was the beginning of the, of the, of the data center build and then, and the data center overbuild.And so you had that, but it was, it was literally, I think it was like $2 trillion got wiped out, right? It was like Jesus, it was like a big, it was. And by the way, the other, the other subtlety in it was the internet companies themselves never really had any debt. ‘cause tech, tech companies generally don't run on debt, but the telecom companies run on debt.Physical infrastructure companies run on debt. And so the companies like Global Crossing not just raise a lot of equity, they also raise a lot of debt. So they're highly levered. And so then you just do the thing. It's just like, okay, you have a highly levered thing where you're, you're just over, you're overbuilding capacity.Demand is growing, but not as fast as you hoped. And then boom, bankrupt. Right. And, and then it, and then it's like they say about the hotel industry, which is, it's always the third owner of a hotel that makes money. It has to go bankrupt twice, right? You have to wash out all of the over optimistic exuberance before it gets to actually a stable state.And then it makes money. So by the way, all of those data centers and all of those, all the fiber that they're in use, it's all in use today. Yeah. But 25 years later. But it, it, it took, and actually the elapsed time was, it took 15 years. It took 15 years from 2000 to 2015 to actually fill, fill up all that capacity.The cautionary warning is the, the overbuild can happen. Um, and, and, and, and, you know, you, you get into this thing where basically everybody, everybody who basically has any sort of institutional capital, it's like, wow. It's just, I, I don't know how to invest in these crazy software things. For sure I can put build data centers and for sure I can buy GPUs that I can deploy, you know, compute grids and, and all these things.Um, and so, you know, if you're a pessimist, you could look at this and you could say, wow, this is like really set up to be able to basically replicate, you know, what we went through, what we went through in 2000. Obviously that would be bad. The counter argument, which is the one I I agree with, which is the counter on, on the other side is a couple things.One is the companies that are investing all the, the companies that are investing the money are like the bluest chip of companies. And so back, back, back in the, in the do, like Global Crossing was like a, it was like an entrepreneur. It was like a, a new venture, but like the money that's being deployed now at scale is Microsoft, and, you know, and Amazon and Google, Facebook and Facebook and Nvidia and, you know, these, these, these, and, and now you know, by the way, open ai philanthropic, which are now at like, you know, really serious size, um, you know, as companies with, you know, very serious revenue.These are very large scale companies with like, lots, lots of cash, lots of debt capacity that they've, they've never used. And so th this is institutional in a way that, that really wasn't at the time. And then the other is, at least for now, every dollar that's being put into anything that results in a running GPU is being turned into revenue right away.Like so, and you guys know this, like everybody's starved for capacity, everybody's starved for compute capacity and then, you know, all the associated things, memory and, and, and interconnected and everything else. Um, data center space. And so e every dollar right now that's being put into the ground is turning into revenue.And, and it, and in fact, I actually think there's an interesting thing happening, which is because everybody starve for capacity, the models that we actually have that we can use today are inferior versions of what we would have if not for the supply constraints. That's true. Um, if Right pose a hypothetical universe in which GPUs were 10 times cheaper and 10 times more plentiful mm-hmm.The models would be much better. ‘cause you would just allocate a lot more money to training and you'd just build better models and they would be better. Um, and so we're, we're actually getting the sandbag version of the technology.swyx: Yeah. No. Everything we use is quantized because the, the labs have to keep the, the full versions,Marc: right?swyx: LikeMarc: we're not even getting the good stuff.swyx: Yeah.Marc: But, but getting the good stuff, it's, it's just, even if technical progress stops. Once there's like a much bigger build of like GPU manufacturing capacity and memory, you know, all, all the things that have to happen in the course of the next five or 10 years.Once it happens, even the current technology is gonna get, gonna get much better. And then as you know, like there's just like a million ways to use this stuff. Like there's just like a million use cases for this. Mm-hmm. Like, it, it, you know, this isn't just sending packets across a, a thing, whatever, and hoping that people find something to do with it.This is just like, oh, we apply intelligence into every domain of human activity. And then it works like incredibly well. Yeah. Um. Here's what I know, here's what I know. Um, in the next three or four year, it's like somewhere between three or four years out, basically everything is selling out. So like the, the entire supply chain is, is, is, is sold out or, or, or selling out.And so there, there's no, like, we're just gonna have like chronic supply shortage for, you know, for years to come. Um, there's going to be a response from the market that's gonna result in an enormous, you know, it's happening now. An enormous flood of investment in a new fab capacity and ev you know, every, everything else to be able to do that, at some point the supply chain constraints will unlock, you know, at least to some degree that will be another accelerant to industry growth when that happens.‘cause the products will get better and everything will get cheaper. Um, and so, so I know that's gonna happen. I know that, you know, the deployments, you know, the, the actual use cases are like really compelling. And then, like I said, you know, with reasoning and agents and so forth, like, I know they're just gonna get like much, much better from here.And so I, I, I know the capabilities are like really real and serious. I also know that the technical progress is not going to stop. It. It, it is excel. It is, is accelerating. Like the, the breakthroughs are are tremendous. I mean, even just month over month, the breakthroughs are really dramatic. And so, you know, I think if you were a cynic and there, there are cynics, you can look at 2000, you can find echoes.But I can't even imagine betting it that this is gonna like somehow disappoint and, you know, at least for years to come, I think it would be essentially suicidal to make that bet. Yeah. Um, it was that Michael Burry, uh, uh, that'sswyx: anMarc: interesting guy, huh? We'll pick on a guy. We'll pick, let's pick on one guy.We'll pick. Well ‘cause he did, he he came out with, it was, it was the, heswyx: doesn't mind.Marc: It was the Nvidia short. Right. He came with the Nvidia short. And then if you guys probably talked about this, which is the, the analysis now that like the current models are getting better faster at such a rate that if you are running an Nvidia, if you're running an Nvidia inference chip today, that's three years old, you're making more money on it today than you did three years ago because the pace of improvement of the software is, is faster than the, the, the depreciation cycle, the chip.And then my understanding is Google is running. I don't if they've, I don't know exactly what, uh, these are rumors that I've heard or maybe it's public, but, um, I think Google's running very old TPUs, very profitably. Ference. Yeah. And very profit and very profitably. Yeah. Um, and so, so it actually turns out, as far as I can tell, it's actually the opposite of the Beery thesis is actually.He was actually 180 degrees wrong. It's actually the, the, the, the old Nvidia chips are getting more valuable, which is something that's like literally never happened before. Like it's never been the case that you have an older model chip that becomes more valuable, not less valuable. And that, and again, that's an expression of the just ferocious pace of software progress.Ferocious pace of capability payoff. Yeah. Uh, that you're getting on the other side of this. And so I just, the idea of betting against that, like.swyx: Yeah. Yeah. Well, one ofMarc: my, it seems like an invitation to get your face ripped up.swyx: One of my early hits was like modeling the lifespan of the H 100 and h two hundreds and, and going like, you know, usually they advise like four to seven years and it was, you know, maybe you sort of realistically haircut cut it down to two to three.Yeah. But actually it's going up and not down. Yeah. And, and uh, that's, I mean that's, I think that's the dream. Uh, we are finding utilization and I think utilization solves all problems. Like, you can, you can find use, use cases for even like the poor, like even memory, we're having a shortage. Right. And, and even like the, the shittier versions of, of memory that we do have, we are finding use cases for it.So like That's great.Marc: Yeah.Alessio: How, how important is open source AI and kinda like edge inference in a world in which you have three years of supply crunch. Like, do you think in the, like, you know, if you fast forward like five years, like how do you think about inference, uh, in the data center versus at the edge?Marc: Well, so just to start, yeah. So I think, I think open source is very important for a bunch of reasons. I think edge, edge inference is very important for a bunch of reasons. I, I think just practically speaking, if we're just gonna have fundamental construc, supply crunches for the next, I mean, you, you guys know if you just project forward demand over the next three years, right?Yeah. Relative to supply, one of the, its main predictions you can do is what's gonna, what, what's gonna happen to the cost of, of inference in the core, uh, over the next three years? And like, it may rise dramatically, right? Like, so, so what is, and then is, is, you know, like the, the, the big model competition are subsidizing heavily right now.Right? Right. And so, so what's the, what will be the average person's, you know, per day, per month token cost, you know, three years from now to do all the things that they want to do. And I, I don't know, it's gonna. I mean, I have, you guys probably have friends, I have friends today who are paying a thousand dollars a day for open claw, for claw tokens to run open claw.Right? And so, okay. $30,000 a month. Right? And, and by the way, those, those friends have like a thousand more ideas of the things that they want their claw to do, right? Yeah. And so you, you could imagine there, there's like latent demand of up to, I don't know, five or $10,000 a day of, of, of tokens for a fully deployed, you know, per personal agent.Uh, and obviously consumers can't pay that, right? And so, so, but it gives you a sense of the fu of the fu of the future scope of demand, right? And so, so even, even if there's a 10 x improvement in price performance, that still, you know, goes to a hundred dollars a day, which is still way beyond what people can pay.Mm-hmm. So there's just gonna be like. Ferocious to me, by the way. The agent thing, the other interesting thing is I think the agent thing, so up until now, a lot of the constraints of GGPU constraints, I think the agent thing now also translates into CPU constraints. Mm-hmm. Right?swyx: CPU memory.Marc: Yes. CPU memory, right?And so, like the entire chip ecosystem is just gonna get wait,swyx: wait for network constraints, that that will be the killer.Marc: It's all bottleneck potentially for years. And so, so I, I think that Brad, and, and I think it's actually possible, I mean, generally inference costs are gonna keep coming down, but I think the, let's put it this way, the rate of decline, I think may level out here for a bit because of these supply constraints.And then at some point, maybe the lab stops subsidizing so much and that, that, that again, will be, be an issue. And so there's just gonna be so much more demand for inference than, than can be satisfied. Um, you know, kind of with the centralized model. And then, and then, you know, you guys know this, but like all the, just the dramatic, I mean just the dramatic innovations that have happened in the Apple silicon to be able to do, uh, inferences, it's quite amazing the level of effort being put.Like the open source guys are putting incredible effort into getting, you know, this recurring pattern where the big model will never run on a pc, and then six months later mm-hmm. Oh, it runs in a pc, right? It's like amazing. And there's very smart people working on that. So there's all that. And then look, there's also, you know.There's also like other, there's other motivators. There's other motivators which is just like, okay, how much trust are the big centralized model providers? You know, how much trust are they building in the market versus, you know, how much are, you know, at least for, in certain cases with some people, for certain use cases, people being like, well, I'm not willing to just like, turn everything over.So there, there, there's all the trust issues. Um, by the way, there's also just like straight up price optimization. There's many uses of AI where you don't need Einstein in the cloud. You just need like a, a a, a smart local model. There's also performance issues where you want, you know, you want, you know, you're gonna want your doorknob to have an AI model in it.Right. You know, to be able to, you know, do, um, you know, to be able to do access control. Um, obviously like everything with a chip is gonna have an AI model in it. Mm-hmm. And it, a lot of those are gonna be local. Um, and so, yeah. No, like I think, I think you're gonna have ti and then you're gonna, by the way, also wearable devices, you know, you don't wanna do a complete round trip.You want, you know, you, whatever your smart devices are, you want it to be like super low latency. Yeah.swyx: The question, do we care who makes it? Yeah. One of the biggest news this week was the collapse of AI two, the Allen Institute. Mm-hmm. One of the actual American open source model labs. Yeah. Um, and, uh, I'm not that optimistic on, on American open source.Yeah. Like you, you guys invested in MIS trial and MIS trial's doing extremely well outside of China. That's about it.Marc: Yeah. We'll see. We'll see. I look, I, number one, I do think we care. Uh, I do think we, I do think we care who makes it. Um, I would say this, the, the, the, the previous presidential administration wanted to kill it in the us Oh yeah.They wanted to drown in the bathtub. Um, and so they wanted to kill it. So at least we have a government now that actually like, actually wants it wants it to happen. And youswyx: earned to councilMarc: and Yeah. And the new and the P pcast. Yeah. So the, the, you know, this admin for whatever other political issues people have, which are many, you know, this administration has, I think a very enlightened view and in particular an enlightened view on AI and in particular on open source ai.Uh, and so they're very supportive. Um, my read is the Chi. The Chinese have a very, the various Chinese companies have a very specific reason to do open source, which is, they, they, they don't fundamentally, they don't think they can sell commercial, uh, AI outside of China right now. And or at least specifically not, not in the US for a combination of reasons.And so they, they kind of view, I think, open source AI as a bit of a loss leader against basically domestic, uh, you know, paid, paid services. And then kind of an, you know, kind of an ancillary products. You know, they're, they're very excited about it, by the way. I think it's great. I think it's great that they're doing it.Um, you know, I think Deeps seek was like a gift to the world. Um, I think. The great thing about open source, open source, the, the, the impact of open source is felt two ways. One is you, you get the software for free, but the other is you get to learn how it works, right? And so like the paper, the paper, the paper and, and the code, right?And the code. And so, like, for example, I thought this was amazing. So open comes out with L one and it's an amazing technical breakthrough, and it's just like, absolutely fantastic. But of course they don't explain how it works in detail. And then of course they hide the, they hide the reasoning traces, right?And, and then, and then, and then everybody's like, okay, this is great, but like, who's gonna be able to replicate this? Are other people gonna be able to do this? You know, is their secret sauce in there? And then our one comes out and it's just like, there's the code and there's the paper, and now the whole world knows how to do it.And then, you know, three months later, every other AI model is, is adding reasoning. And so, so you get this kind of double, like even if the Chinese models themselves are not the models that get used, the education that's taken place to the rest of the world, the information diffusion, you know, is incredibly powerful.So that happens and then, I don't know. We'll, we'll see. You know, there are a bunch of American, you know, open source, you know, ai, uh, model companies. I mean, look, there's gonna be tremendous, you know, there already is. There's, you know, there's gonna be tre there's tremendous competition, uh, among the primary model companies.You know, there's, depending on how you count, there's like four or five, you know, big co model companies now that are, you know, kind of neck and neck, uh, in different ways. Um, uh, you know, and, and, and, um, you know, and then obviously Bo Bo both X and then MetAware involved are, you know, both have huge, you know, huge attempts to, you know, kind of, to kind of leapfrog underway.And then you've got, you know, a whole fleet of startups, new companies, including a whole bunch that we're backing, that are, you know, trying to come out with different approaches. And then you've got whatever it is. I don't know how, how many, how many, like main line foundation model companies are there in China at this point?It's probably six. It'sswyx: five Tigers is what they call it. Yeah. Uh, Quinn is in questionable because there's change in leadership,Marc: right?swyx: Yeah.Marc: But that, does that include, that includes like Moonshot,swyx: yes. Can deep seek, uh, uh, ZI, um, Quinn oh one is in there.Marc: Right. And then, um, and by dance and, and then you see,swyx: ance would be like the next tier ance.They weren't as prominent. They weren't, didn't haveMarc: a leading. Yeah. But they, you at least, you know, ance is very inspiring and presumably they have more stuff coming and Tencent probably has more stuff coming and, and so forth. And so, so, so like, look, here, here would be a thing you can anticipate, which is there are not these markets, there are not going to be between the US and China right now, there's like a dozen primary foundation model companies that are like at scale, at, at some level of a critical mass.It's not gonna be a dozen in three years, right? Like, it just because these industries don't bear a dozen, it's, it's gonna be three or you know, there's gonna be three or four big winners or maybe one or two big winners. And so there's gonna be like a whole bunch of those guys that are gonna have to figure out alternate strategies.Um, and I think like open source is one of those strategies. And so I, I think you could see like a whole, i, I, I think the questions like, who's gonna do open source? I think that could change really fast. I, I think that, that, that's a very dynamic thing. I think it's very hard to predict what happens. And, and I think it's very important.swyx: NVIDIA's doing a lot.Marc: Well, I was gonna say. Well, exactly. And then you're got Nvidia and then, and then, you know, just to, again, indu, there's an old thing in business strategy, which is called, uh, commoditize Compliments. Commoditize the compliment. That's right. And so if your Jensen is just kind of obvious, of course, you wanna commoditize the software.Yeah. And he's, and to his enormous credit, he's putting enormous resources behind that. And so maybe it, maybe it's literally Nvidia and I think that would be great.Alessio: Yeah. Uh, narrative violation to European projects, uh, in the, uh, damn.swyx: I'm hosting my, uh, Europe, uh, conference soon. And I got both of them.Alessio: They got us.They got us. MarkMarc: finished. They got us, us. Well, wait a minute. Where was Peter? So where was Steinberger when he did? In AustriaAlessio: was, yeah, yeah, yeah.Marc: He was in what? He was in Vienna. Oh, he was in Vienna. And then where is he now?swyx: Uh, he's moving to sf.Marc: Okay. Okay. Alright. Okay, there we go. And then, yeah, the PI guy, right?The PI guys are European.swyx: Yeah, they're also, they're buddies inAlessio: Australia. Mario's also there. Yeah.Marc: Right. And are they, yeah, they haven't announced yet. Any sort of change changed or have theyAlessio: No, they're, they have a company there.Marc: Okay. Got, okay. Good.Alessio: Good, good,good.Alessio: Um,Marc: yeah, good.swyx: Anyways, I think pie and open cloud very important software things and, and I just wanted you to just go off on what you think.Marc: Yeah. So I think in co the, the combination of the two of them I think is one of the 10 most important softwares. Openswyx: Claw got all the attention, but Right. Talk about pie,Marc: pi pie's, kind of the Yeah. PI's, PI's kind of the architectural breakthrough for those of us who are older. There was this whole thing that was very important in the world of software basically from like 1970 to, I don't know, it still is very important, but like 19, from 1973 to like basically the creation of Linux, which is basically this, this thing used to call like the Unix mindset.Like so, so, ‘cause there were all these different, you know, theories. There are all these different operating systems and mainframes and, and then you know, all these windows and Mac and all these things. And then there was this, but kind of behind it all was this idea of kind of the Unix mindset. And the Unix mindset was this thing where basically you don't have these, like, like in the old days, like, like the operating system that like made the computer industry really work, like in the 1960s mm-hmm.Was this thing called o os 360, which was this big operating system that IBM developed that was supposed to basically run everything. And it was this like giant monolithic architecture in the sky. It was like a, you know, it was like a giant castle. Um, of software. And, and by the way, it worked really well and they were very successful with it.But like, it was this huge castle in the sky, but it was this thing, it was almost unapproachable, which is like, you had to be kind of inside IBM or very close to IBM. And you had to really understand every aspect, how the system worked. And then the, the Unix sky is originally out of at and t and then out out of Berkeley, um, you know, came out and they said, no, let's have a completely different architecture.And the way architecture's gonna work is we're gonna have, we're gonna have a, a prompt and, and a, and a shell. And then, and then we're gonna, all, all the functionality is gonna be in the form of these discreet modules, and then you're gonna be able to chain the modules together. Mm-hmm. Yeah. And so like the, the, the op, it's almost like the operating, operating system itself is gonna be a programming language.Um, and then that led led to the, the, the sort of centrality of the shell. Um, and then that led to sort of, uh, you know, basically chaining together Unix tools. And then that led to the emergence of these, these scripting languages like Pearl, where you, you could basically kind of very easily do this, and then the shells got more sophisticated and then, and then, and then look like, you know, that, that, that number one, that worked and that, that was the world I grew up in.Like I was, I was a Unix guy. You know, sort of from, call it 1988 to, you know, kind of all, all the way through my work and it worked really well. It, it's in the background, um, you know, nor normal people don't need to, didn't need to necessarily know about it, but like, if you were doing like system architecture, application development, you, you, you knew all about it.Um, and then, you know, it's been in the background ever since. And, you know, look, your Mac still has a Unix shell, you know, kind of in there, and your iPhone still has a Unix shell kind of buried in there somewhere. So they're kind of in there. And then, you know, the Windows shell is kind of a, you know, sort of a weird derivative of that.But, um, you know, but look, the inter, the internet runs on Unix, um, and that smartphones, actually, both iOS and Android are Unix derivatives. And so, you know, kind of Unix did end up winning. But, but anyway, and then we just started taking that for granted. And then, and then so, so basically the, the way I think about what happened with Pie and then with Open Claw is basically what those guys figured out is, I always say the, the great breakthroughs are obvious in retrospect, right?Which is the best kind, the best kind. They weren't obvious at the time or somebody else would've done them already. Um, and so there is a, like a real conceptual leap, but then you look at it sort of the backwards looking and you're just like, oh, of course. Mm-hmm. Like the, the, to me those are always the best breakthroughs.Well, actually language models themselves are like that. It's just like, oh, next token completion. Oh, of course.swyx: Yeah. What other objective mattered?Marc: Yeah, exactly. But, but like it, right. But she's even saying it wasn't obvious until somebody actually did it. Right. And so the conceptual breakthrough is real and deep and powerful and, and very important.And so the way I think about pie and olaw is it's basically marrying the, the language model mindset to the un to the Unix, basically shell prompt mindset. And so it's, it's basically this idea that what, what, so what is an agent, right? And as, as, and as you know, like many smart people who have been trying to figure out what an agent is for, for, for decades, and they've had many architectures to build agents and the whole thing.And it turns out what is an agent. So it turns out what we now know is an agent is the following. It's, so it's a language model. And then above that, it's a ba, it's a bash shell. Um, so it's a, it's a Unix shell, and then it's, and then the agent has access, uh, has access to, to the shell. And, you know, hopeful, hopefully in a sandbox, maybe in, maybe in a sandbox.So it's, it's the model. Um, it's the shell. Um, and then it's a fi, it's a file system. Um, and then the state is stored in files. And then, you know, there's the markdown format for the, you know, for, for the files themselves. And then, and then there's basically what in Unix is called Aron job. There's a loop and then there's a heartbeat for the, there's heartbeat and, and the thing basically Wake Wakes up.Wakes up. So it's basically LLM plus shell, plus file system, plus markdown, plus kron. And it turns out that's an agent. And, and, and every part of that, other than the model is something that we already completely know and understand. And in fact, it turns out that like the latent power of the Unix shell is like extraordinary because basically like all, like, there's just like an, there's just enormous latent power in the shell.There's enormous numbers of Unix commands, there's enormous number of command line interfaces into all kinds of things already in the, you know, your entire, I mean your entire, just to start with, your computer runs on a shell. If you're running a Mac or a, or, or a phone, your computer, your computer's running on a shell, uh, already.And so like the full power of your computer is available at the command line level. Um, and then it turns out it's really easy to expose other functions as a command line interface. And so like this whole idea where we need like MCP and these like product mm-hmm. Fancy protocols, whatever, it's like, no, we don't, we just need like a command, command line thing.So that's the architecture. And then it turns out what is your agent? Your agent has a bunch of files starting a file system. And then there's the thing that just like completely blew my mind when I write my head around it as a result of this, which is like, okay. This means your agent is now actually independent of the model that it's running on.Because you can actually swap out a different LLM underneath your agent and your, your agent will change personality somewhat. ‘cause the model is different, but all of the state stored in the files will be retained.swyx: Yeah. Different instruction set, but you just compiledit.Marc: Right, exactly. And it's all right.It's like right. Swapping out a ship and recompiling, but it's, it's still, it's still your agent with all of its memories. Um, and with all of its capabilities. And then by the way, you can also swap out the shell, uh, so you can move it to a different execution environment that is also, is also a b shell, by the way, you can also switch out the file system, right.Uh, and you can, and you can, and you can swap out the, the, the heartbeat for the, the crown framework, the, the loop that the agent framework itself. And so your agent basically is ba basically at the end of the day, it's just. It's just, its files. Um, and then, and then there's of course it a openswyx: call.Marc: Yeah, it's, it's basically, it's, it's just the files.Um, and then by the way, as a consequence of that, the agent and then the agent itself, it turns out a couple important things. So one is it, it's, it, it can migrate itself, right? And so you're, you can instruct your agent, migrate yourself to a different, uh, runtime environment, migrate yourself to a different file system, migrate yourself to a different, you know, swap out the language model.Your agent will do all that stuff for you. And then there's the final thing, which is just amazing, which is the agent is the agent actually has full introspection. It actually, it actually knows about its own files and it could rewrite its own files. Right. Which by the way, is basically no widely deployed software system in history where the, the, the thing that you're using actually has full introspective knowledge of how it itself works and is able to modify itself.Like that, that, I mean, there have been toy systems that have had that, but there, there's never been a widely deployed system that has that capability and then that leads you to the capability. That just like completely blew my mind when I wrap my head around it, which is you can tell the agent to add new functions and features to itself and it can do that.Extend yourself. Yeah. Right? Extend, extend yourself. Like extend yourself. Give yourself a new capability. Right? And so, and so literally it's just like you run into somebody at a party and they're like, oh, I have my open claw, do whatever, connect to my eat, sleep bed, and it gives me better advice and sleep.And you go home at night and you tell your claw, or if they're at the party, by the way, you tell your claw, oh, add this capability to yourself. And your claw will say, oh, okay, no problem. And it'll go out on the internet and it'll figure out whatever it needs and then it'll go out to claw code or whatever.It'll write whatever it needs. And then the next thing you know, it has this new capability. And so you don't even have to, like, you can have it upgrade itself without even having to, without having to do anything other than tell it that you want it to do that. And so anyway, so the, the combination of all this is just, I mean, this is just like a massive, incredible, I mean, it's just incredible.Like if I, if I were, if I were 18, like this is a hundred, this is what I would be spending all of my time on. This is like such an incredible conceptual breakthrough. Yeah. And again, pe people are gonna look at it and they already get this response. People are gonna look at it and they're gonna say, oh, well, where's the breakthrough?‘cause these, the, all of these components were already known before. Mm-hmm. But, but this is the key, the key to the breakthrough was by using all these components that were known before, you get all of the underlying capability of that's buried in there. And so all, and so for example, computer use all of a sudden just kind of falls, trivi, trivial.Of course it's gonna be able to use your computer. It has full access to the shell. Right. And then, and then you just, you, you give it access to a browser, and then you've got the computer and the browser and, and often away it goes. And, and then you've got all the abilities of the browser also. Um, yeah.And so, and so the capability unlock here is profound. My friends who are, you know, deepest into this, are having their claw do like a, like, literally like a thousand things in their lives. They have new ideas every day. They're just like constantly throwing new challenges at the thing. And by the way, it's early and, you know, these are, you know, these are prototypes and there are, you know, as you guys know, there's security issues.Yeah. And, and so, you know, there's a bunch of stuff to be ironed out, but the, the unlock of capability is just incredible.swyx: Yeah.Marc: And I, I have absolutely no doubt that everybody in the world is gonna, is gonna have at least, you know, an agent like this, if not an entire family of agents. And w

Upcoming GeekWire Podcast Live Event: Join us from 4 p.m. to 6 p.m. Thursday, Feb 12 at Fremont Brewing for a live recording of the GeekWire Podcast with Todd Bishop and John Cook. Free for Fremont Chamber members, $15 otherwise. Register here. This week on the show: Andy Jassy tells Wall Street that Amazon is planning $200 billion in capital expenses this year, mostly to build out AI infrastructure, and investors give it a thumbs down. Microsoft's financial results beat expectations but the company loses $357 billion in market value in a single day after investors learn the extent of its dependence on OpenAI. Meanwhile, OpenAI leases 10 floors of office space in Bellevue, lawmakers in Olympia propose new taxes impacting startup exits and high-income earners, and the bots get their own social network. In our featured conversation, recorded at a dinner hosted by Accenture in Bellevue, GeekWire co-founder Todd Bishop sits down with computer scientist and entrepreneur Oren Etzioni to talk about AI agents, the startup landscape, the fight against deepfakes, and what good AI leadership looks like. Etzioni is co-founder of AI agent startup Vercept, founder of the AI2 Incubator, a venture partner at Madrona, and the former founding CEO of the Allen Institute for AI. With GeekWire co-founder Todd Bishop. Edited by Curt Milton. Music by Daniel L.K. Caldwell.See omnystudio.com/listener for privacy information.

Nathan Lambert and Sebastian Raschka are machine learning researchers, engineers, and educators. Nathan is the post-training lead at the Allen Institute for AI (Ai2) and the author of The RLHF Book. Sebastian Raschka is the author of Build a Large Language Model (From Scratch) and Build a Reasoning Model (From Scratch). Thank you for listening ❤ Check out our sponsors: https://lexfridman.com/sponsors/ep490-sc See below for timestamps, transcript, and to give feedback, submit questions, contact Lex, etc. Transcript: https://lexfridman.com/ai-sota-2026-transcript CONTACT LEX: Feedback – give feedback to Lex: https://lexfridman.com/survey AMA – submit questions, videos or call-in: https://lexfridman.com/ama Hiring – join our team: https://lexfridman.com/hiring Other – other ways to get in touch: https://lexfridman.com/contact SPONSORS: To support this podcast, check out our sponsors & get discounts: Box: Intelligent content management platform. Go to https://box.com/ai Quo: Phone system (calls, texts, contacts) for businesses. Go to https://quo.com/lex UPLIFT Desk: Standing desks and office ergonomics. Go to https://upliftdesk.com/lex Fin: AI agent for customer service. Go to https://fin.ai/lex Shopify: Sell stuff online. Go to https://shopify.com/lex CodeRabbit: AI-powered code reviews. Go to https://coderabbit.ai/lex LMNT: Zero-sugar electrolyte drink mix. Go to https://drinkLMNT.com/lex Perplexity: AI-powered answer engine. Go to https://perplexity.ai/ OUTLINE: (00:00) – Introduction (01:39) – Sponsors, Comments, and Reflections (16:29) – China vs US: Who wins the AI race? (25:11) – ChatGPT vs Claude vs Gemini vs Grok: Who is winning? (36:11) – Best AI for coding (43:02) – Open Source vs Closed Source LLMs (54:41) – Transformers: Evolution of LLMs since 2019 (1:02:38) – AI Scaling Laws: Are they dead or still holding? (1:18:45) – How AI is trained: Pre-training, Mid-training, and Post-training (1:51:51) – Post-training explained: Exciting new research directions in LLMs (2:12:43) – Advice for beginners on how to get into AI development & research (2:35:36) – Work culture in AI (72+ hour weeks) (2:39:22) – Silicon Valley bubble (2:43:19) – Text diffusion models and other new research directions (2:49:01) – Tool use (2:53:17) – Continual learning (2:58:39) – Long context (3:04:54) – Robotics (3:14:04) – Timeline to AGI (3:21:20) – Will AI replace programmers? (3:39:51) – Is the dream of AGI dying? (3:46:40) – How AI will make money? (3:51:02) – Big acquisitions in 2026 (3:55:34) – Future of OpenAI, Anthropic, Google DeepMind, xAI, Meta (4:08:08) – Manhattan Project for AI (4:14:42) – Future of NVIDIA, GPUs, and AI compute clusters (4:22:48) – Future of human civilization

Robots aren't just software. They're AI in the physical world. And that changes everything.In this episode of TechFirst, host John Koetsier sits down with Ali Farhadi, CEO of Allen Institute for AI, to unpack one of the biggest debates in robotics today: Is data enough, or do robots need structured reasoning to truly understand the world?Ali explains why physical AI demands more than massive datasets, how concepts like reasoning in space and time differ from language-based chain-of-thought, and why transparency is essential for safety, trust, and human–robot collaboration. We dive deep into MOMO Act, an open model designed to make robot decision-making visible, steerable, and auditable, and talk about why open research may be the fastest path to scalable robotics.This conversation also explores:• Why reasoning looks different in the physical world• How robots can project intent before acting• The limits of “data-only” approaches• Trust, safety, and transparency in real-world robotics• Edge vs cloud AI for physical systems• Why open-source models matter for global AI progressIf you're interested in robotics, embodied AI, or the future of intelligent machines operating alongside humans, this episode is a must-watch.

Will AGI happen soon - or are we running into a wall?In this episode, I'm joined by Tim Dettmers (Assistant Professor at CMU; Research Scientist at the Allen Institute for AI) and Dan Fu (Assistant Professor at UC San Diego; VP of Kernels at Together AI) to unpack two opposing frameworks from their essays: “Why AGI Will Not Happen” versus “Yes, AGI Will Happen.” Tim argues progress is constrained by physical realities like memory movement and the von Neumann bottleneck; Dan argues we're still leaving massive performance on the table through utilization, kernels, and systems—and that today's models are lagging indicators of the newest hardware and clusters.Then we get practical: agents and the “software singularity.” Dan says agents have already crossed a threshold even for “final boss” work like writing GPU kernels. Tim's message is blunt: use agents or be left behind. Both emphasize that the leverage comes from how you use them—Dan compares it to managing interns: clear context, task decomposition, and domain judgment, not blind trust.We close with what to watch in 2026: hardware diversification, the shift toward efficient, specialized small models, and architecture evolution beyond classic Transformers—including state-space approaches already showing up in real systems.Sources:Why AGI Will Not Happen - https://timdettmers.com/2025/12/10/why-agi-will-not-happen/Use Agents or Be Left Behind? A Personal Guide to Automating Your Own Work - https://timdettmers.com/2026/01/13/use-agents-or-be-left-behind/Yes, AGI Can Happen – A Computational Perspective - https://danfu.org/notes/agi/The Allen Institute for Artificial IntelligenceWebsite - https://allenai.orgX/Twitter - https://x.com/allen_aiTogether AIWebsite - https://www.together.aiX/Twitter - https://x.com/togethercomputeTim DettmersBlog - https://timdettmers.comLinkedIn - https://www.linkedin.com/in/timdettmers/X/Twitter - https://x.com/Tim_DettmersDan FuBlog - https://danfu.orgLinkedIn - https://www.linkedin.com/in/danfu09/X/Twitter - https://x.com/realDanFuFIRSTMARKWebsite - https://firstmark.comX/Twitter - https://twitter.com/FirstMarkCapMatt Turck (Managing Director)Blog - https://mattturck.comLinkedIn - https://www.linkedin.com/in/turck/X/Twitter - https://twitter.com/mattturck(00:00) - Intro(01:06) – Two essays, two frameworks on AGI(01:34) – Tim's background: quantization, QLoRA, efficient deep learning(02:25) – Dan's background: FlashAttention, kernels, alternative architectures(03:38) – Defining AGI: what does it mean in practice?(08:20) – Tim's case: computation is physical, diminishing returns, memory movement(11:29) – “GPUs won't improve meaningfully”: the core claim and why(16:16) – Dan's response: utilization headroom (MFU) + “models are lagging indicators”(22:50) – Pre-training vs post-training (and why product feedback matters)(25:30) – Convergence: usefulness + diffusion (where impact actually comes from)(29:50) – Multi-hardware future: NVIDIA, AMD, TPUs, Cerebras, inference chips(32:16) – Agents: did the “switch flip” yet?(33:19) – Dan: agents crossed the threshold (kernels as the “final boss”)(34:51) – Tim: “use agents or be left behind” + beyond coding(36:58) – “90% of code and text should be written by agents” (how to do it responsibly)(39:11) – Practical automation for non-coders: what to build and how to start(43:52) – Dan: managing agents like junior teammates (tools, guardrails, leverage)(48:14) – Education and training: learning in an agent world(52:44) – What Tim is building next (open-source coding agent; private repo specialization)(54:44) – What Dan is building next (inference efficiency, cost, performance)(55:58) – Mega-kernels + Together Atlas (speculative decoding + adaptive speedups)(58:19) – Predictions for 2026: small models, open-source, hardware, modalities(1:02:02) – Beyond transformers: state-space and architecture diversity(1:03:34) – Wrap

Support the show to get full episodes, full archive, and join the Discord community. The Transmitter is an online publication that aims to deliver useful information, insights and tools to build bridges across neuroscience and advance research. Visit thetransmitter.org to explore the latest neuroscience news and perspectives, written by journalists and scientists. Read more about our partnership. Sign up for Brain Inspired email alerts to be notified every time a new Brain Inspired episode is released. To explore more neuroscience news and perspectives, visit thetransmitter.org. Can you look at all the synaptic connections of a brain, and tell me one nontrivial memory from the organism that has that brain? If so, you shall win the $100,000 prize from the Aspirational Neuroscience group. I was recently invited for the second time to chair a panel of experts to discuss that question and all the issues around that question - how to decode a non-trivial memory from a static map of synaptic connectivity. Before I play that recording, let me set the stage a bit more. Aspirational Neuroscience is a community of neuroscientists run by Kenneth Hayworth, with the goal, from their website, to "balance aspirational thinking with respect to the long-term implications of a successful neuroscience with practical realism about our current state of ignorance and knowledge." One of those aspirations is to decoding things - memories, learned behaviors, and so on - from static connectomes. They hold satellite events at the SfN conference, and invite experts in connectomics from academia and from industry to share their thoughts and progress that might advance that goal. In this panel discussion, we touch on multiple relevant topics. One question is what is the right experimental design or designs that would answer whether we are decoding memory - what is a benchmark in various model organisms, and for various theoretical frameworks? We discuss some of the obstacles in the way, both technologically and conceptually. Like the fact that proofreading connectome connections - manually verifying and editing them - is a giant bottleneck, or like the very definition of memory, what counts as a memory, let alone a "nontrivial" memory, and so on. And they take lots of questions from the audience as well. I apologize the audio is not crystal clear in this recording. I did my best to clean it up, and I take full blame for not setting up my audio recorder to capture the best sound. So, if you are a listener, I'd encourage you to check out the video version, which also has subtitles throughout for when the language isn't clear. Anyway, this is a fun and smart group of people, and I look forward to another one next year I hope. The last time I did this was episode 180, BI 180, which I link to in the show notes. Before that I had on Ken Hayworth, whom I mentioned runs Aspirational Neuroscience, and Randal Koene, who is on the panel this time. They were on to talk about the future possibility of uploading minds to computers based on connectomes. That was episode 103. Aspirational Neuroscience Panel Michał Januszewski@michalwj.bsky.social Research scientist (connectomics) with Google Research, automated neural tracing expert Sven Dorkenwald @sdorkenw.bsky.social Research fellow at the Allen Institute, first-author on first full Drosophila connectome paper Helene Schmidt@helenelab.bsky.social Group leader at Ernst Strungmann Institute, hippocampus connectome & EM expert Andrew Payne @andrewcpayne.bsky.social Founder of E11 Bio, expansion microscopy & viral tracing expert Randal Koene Founder of the Carboncopies Foundation, computational neuroscientist dedicated to the problem of brain emulation. Related episodes: BI 103 Randal Koene and Ken Hayworth: The Road to Mind Uploading BI 180 Panel Discussion: Long-term Memory Encoding and Connectome Decoding

On this special episode of the GeekWire Podcast, recorded backstage at the GeekWire Gala at the Showbox Sodo, we sit down with five of the inventors, scientists, and entrepreneurs selected as the Seattle region's 2025 Uncommon Thinkers, in partnership with Greater Seattle Partners. Jeff Thornburg spent years building rocket engines for Elon Musk at SpaceX and Paul Allen at Stratolaunch. Now, as CEO of Portal Space Systems, he's moved past chemical rockets to revive a concept NASA studied decades ago but never pursued — a spacecraft powered by focused sunlight. He calls it a "steam engine for space." Read the profile. Anindya Roy grew up in rural India without electricity, came to the U.S. with two suitcases and $2,000, and earned a spot in the lab of a Nobel Prize winner. Now, as co-founder of Lila Biologics, he's using AI to design proteins from scratch (molecules that have never existed in nature) to treat cancer. Read the profile. Jay Graber runs Bluesky, the decentralized social network that's become a leading alternative to X and other centralized platforms. But while most tech CEOs build moats to lock users in, Jay and the Bluesky team are building a protocol designed to let them leave. She sees the network as a "collective organism," and she's creating a tech foundation meant to outlive her own company. Read the profile. Read the profile. Kiana Ehsani came to Seattle from Iran for her PhD and spent four years at the Allen Institute for AI before becoming CEO of Vercept. She and the Vercept team are competing directly with OpenAI, Google and others in AI agents, building efficient agents that handle mundane digital tasks on computers so humans can spend less time on screens. Read the profile. Brian Pinkard spent six months after college flipping rocks and building trails because he wanted to do work that mattered. That instinct led him to Aquagga, where he's proving that the industry standard of filtering and burying "forever chemicals" is obsolete. Instead, he's using technology originally designed to destroy chemical weapons to annihilate PFAS under extreme heat and pressure. Read the profile. Eagle-eyed readers may have noticed we're missing one honoree — Chet Kittleson, co-founder and CEO of Tin Can, the startup making WiFi-enabled landline phones to help kids connect without screens. Chet wasn't able to join us, but we plan to speak with him on a future episode. With GeekWire co-founder Todd Bishop. Edited by Curt Milton.See omnystudio.com/listener for privacy information.

Watch on YouTube → https://www.youtube.com/watch?v=VU65vnqkBmE When two of the world's leading ALS researchers meet for the first time on a podcast, something remarkable happens. Dr. Merit Cudkowicz - Tim's close friend and arguably the world's foremost ALS expert, returns to Nothing Left Unsaid for her third appearance, this time joined by Dr. Tanya Daigle from the Allen Institute, who's pioneering genetic tools that could transform how we treat neurodegenerative diseases. Together, they reveal a future for ALS research that's closer than most people realize. This isn't your typical medical conversation. Merit and Tanya speak with unguarded hope about gene therapies that have stopped disease progression in 40% of patients, AI-driven drug discovery that's accelerating at unprecedented rates, and precision targeting tools that could deliver treatments directly to dying motor neurons without invasive brain surgery. They discuss the real bottlenecks between laboratory breakthroughs and patient impact, why most ALS drug trials aren't actually failures, and how the field has evolved from a handful of researchers to a global collaborative effort spanning continents. CHAPTERS: 00:00 Trailer01:09 Intro02:55 Current State of ALS Research05:19 Misconceptions About ALS09:32 Progress and Future of ALS Treatments21:17 Innovations in Genetic Tools for ALS31:59 A New Frontier in ALS Research35:03 Efficiency of Platform Trials in ALS Research36:59 The Importance of Funding and Science Collaboration37:55 Advancements in Treating ALS Smarter38:52 The Role of Biomarkers and Individualized Therapies41:31 Dream Trials and Future Prospects48:01 The Impact of AI in ALS Research54:08 The Next Big Pivot in ALS Research58:18 Hope and Collaboration in ALS Research01:01:04 How to Be Part of ALS Community RESOURCES & LINKS: Allen Institute: https://alleninstitute.orgHealey & AMG Center for ALS at Mass General: https://www.massgeneral.org/neurology/als SPONSORS:ElevenLabs: Thanks to ElevenLabs (https://elevenlabs.io) for supporting this episode and powering Tim's voice. SOCIAL:Website: https://nlupod.com/X: https://x.com/nlutimgreenFacebook: https://www.facebook.com/NLUpodInstagram: https://www.instagram.com/nlupod LISTEN ON OTHER PLATFORMS: Apple Podcasts: https://podcasts.apple.com/us/podcast/nothing-left-unsaid/id1734094890Audible: https://www.audible.com/podcast/Nothing-Left-Unsaid/B0CWTCRKGZCastbox: https://castbox.fm/channel/id6405921?country=usOvercast: https://overcast.fm/itunes1734094890iHeartRadio: https://www.iheart.com/podcast/263-nothing-left-unsaid-155769998/ PERSONAL: Tackle ALS: https://www.tackleals.comTim Green Books: https://authortimgreen.comTim's New Book - ROCKET ARM: https://www.amazon.com/dp/0062796895/ Learn more about your ad choices. Visit megaphone.fm/adchoices

Prepare to get meta, I'm going to walk you through my creative, divergent thinking process while explaining the neuroscience of creativity and divergent thinking! We do need our brains and bodies to be in sync for this to be meaningful, illuminating, give you (and I) an “aha!” moment. Please note, divergent thinking means you think in a way that is not typical or standard, so I apologize if I'm hard to follow, however it is necessary to illustrate the point I'm making. I want you to know that you don't have to understand every single detail, however you should focus on the actual “route” my mind is taking - the “figure 8.” Thank you so much to The Allen Institute for inviting me to Neuroscience 2025 in San Diego, I am beyond grateful and appreciative for the experience. I encourage everyone to check out their website, as well as their mission, because science (and creativity) truly are for everyone. The Allen Institute: https://alleninstitute.org/  New Book Club Information: https://www.patreon.com/posts/new-book-for-143088045  Resources: This Is What It Sounds Like - Susan Rogers and Ogi Ogas  Your Brain on Art: How the Arts Transform Us - Susan Magsamen & Ivy Ross Horror in Architecture: The Reanimated Edition - Joshua Comaroff + One Ker-Shing Future Tense: Why Anxiety Is Good for You (Even Though It Feels Bad) - Tracy Dennis-Tiwary, PhD This is the book I recommended on arousal state splitting off into excitement or anxiety. A neurocomputational model of creative process https://www.sciencedirect.com/science/article/pii/S0149763422001452 Functional Fixedness: When We Stick to What We Know https://nesslabs.com/functional-fixedness This is not the Time Magazine article but it also covers functional fixedness and how it impacts creativity Sensorimotor experience and verb-category mapping in human sensory, motor and parietal neurons https://www.sciencedirect.com/science/article/abs/pii/S0010945217301491 Mental time travel, language, and evolution https://www.sciencedirect.com/science/article/abs/pii/S0028393219302441 Isometric Handgrip Exercise Speeds Working Memory Responses in Younger and Older Adults https://pmc.ncbi.nlm.nih.gov/articles/PMC10238670/ This article does include hand exercises for younger adults, most however are focused on improving working memory for older populations Analogy: Definition, Examples, and Usage https://www.grammarly.com/blog/literary-devices/analogy/ Learning from the Double Diamond: How Divergent and Convergent Thinking Can Improve Collaboration and Problem-Solving in Museums https://www.aam-us.org/2024/04/05/learning-from-the-double-diamond-how-divergent-and-convergent-thinking-can-improve-collaboration-and-problem-solving-in-museums/ On the emergence of interdisciplinary scientific fields: (how) does it relate to science convergence? https://www.sciencedirect.com/science/article/pii/S0048733324000751 A Global Map of Science Based on the ISI Subject Categories https://www.leydesdorff.net/map06/texts/index.htm The Brain Science of Elusive ‘Aha! Moments' https://www.scientificamerican.com/article/the-elusive-brain-science-of-aha-moments/ Recommended Books: The Geometry of Grief - Michael Frame The Tao of Physics - Fritjof Capra  The Gentrification of the Mind - Sarah Schulman  On the Art and Craft of Doing Science - Kenneth Catania  The Meaning of Proofs: Mathematics as Storytelling - Gabriele Lolli The Botany of Desire - Michael Pollan  The Story Grid: What Good Editors Know - Shawn Coyne  When Narcissism Comes to Church - Chuck DeGroat Humour - Terry Eagleton The Knowledge Illusion: Why We Never Think Alone - Philip Fernbach & Steven A. Sloman Learn more about your ad choices. Visit podcastchoices.com/adchoices

Scientific Sense ® by Gill Eapen: Dr. Christof Koch is a Meritorious Investigator at the Allen Institute. His writings and interests integrate theoretical, computational and experimental neuroscience with philosophy and contemporary trends, in particular artificial intelligence. His latest book is then I am myself the world: WHAT CONSCIOUSNESS IS AND HOW TO EXPAND IT.Please subscribe to this channel:https://www.youtube.com/c/ScientificSense?sub_confirmation=1

In this special release episode, Matt sits down with Nathan Lambert and Luca Soldaini from Ai2 (the Allen Institute for AI) to break down one of the biggest open-source AI drops of the year: OLMo 3. At a moment when most labs are offering “open weights” and calling it a day, AI2 is doing the opposite — publishing the models, the data, the recipes, and every intermediate checkpoint that shows how the system was built. It's an unusually transparent look into the inner machinery of a modern frontier-class model.Nathan and Luca walk us through the full pipeline — from pre-training and mid-training to long-context extension, SFT, preference tuning, and RLVR. They also explain what a thinking model actually is, why reasoning models have exploded in 2025, and how distillation from DeepSeek and Qwen reasoning models works in practice. If you've been trying to truly understand the “RL + reasoning” era of LLMs, this is the clearest explanation you'll hear.We widen the lens to the global picture: why Meta's retreat from open source created a “vacuum of influence,” how Chinese labs like Qwen, DeepSeek, Kimi, and Moonshot surged into that gap, and why so many U.S. companies are quietly building on Chinese open models today. Nathan and Luca offer a grounded, insider view of whether America can mount an effective open-source response — and what that response needs to look like.Finally, we talk about where AI is actually heading. Not the hype, not the doom — but the messy engineering reality behind modern model training, the complexity tax that slows progress, and why the transformation between now and 2030 may be dramatic without ever delivering a single “AGI moment.” If you care about the future of open models and the global AI landscape, this is an essential conversation.Allen Institute for AI (AI2)Website - https://allenai.orgX/Twitter - https://x.com/allen_aiNathan LambertBlog - https://www.interconnects.aiLinkedIn - https://www.linkedin.com/in/natolambert/X/Twitter - https://x.com/natolambertLuca SoldainiBlog - https://soldaini.netLinkedIn - https://www.linkedin.com/in/soldni/X/Twitter - https://x.com/soldniFIRSTMARKWebsite - https://firstmark.comX/Twitter - https://twitter.com/FirstMarkCapMatt Turck (Managing Director)Blog - https://mattturck.comLinkedIn - https://www.linkedin.com/in/turck/X/Twitter - https://twitter.com/mattturck(00:00) – Cold Open(00:39) – Welcome & today's big announcement(01:18) – Introducing the Olmo 3 model family(02:07) – What “base models” really are (and why they matter)(05:51) – Dolma 3: the data behind Olmo 3(08:06) – Performance vs Qwen, Gemma, DeepSeek(10:28) – What true open source means (and why it's rare)(12:51) – Intermediate checkpoints, transparency, and why AI2 publishes everything(16:37) – Why Qwen is everywhere (including U.S. startups)(18:31) – Why Chinese labs go open source (and why U.S. labs don't)(20:28) – Inside ATOM: the U.S. response to China's model surge(22:13) – The rise of “thinking models” and inference-time scaling(35:58) – The full Olmo pipeline, explained simply(46:52) – Pre-training: data, scale, and avoiding catastrophic spikes(50:27) – Mid-training (tail patching) and avoiding test leakage(52:06) – Why long-context training matters(55:28) – SFT: building the foundation for reasoning(1:04:53) – Preference tuning & why DPO still works(1:10:51) – The hard part: RLVR, long reasoning chains, and infrastructure pain(1:13:59) – Why RL is so technically brutal(1:18:17) – Complexity tax vs AGI hype(1:21:58) – How everyone can contribute to the future of AI(1:27:26) – Closing thoughts

自然科学研究機構・生理学研究所にて独立する萩原賢太さんゲスト回。アメリカの現状と脱出に至る過程、立ち上げるラボでのプロジェクト、五十嵐ラボ@東北大、など。 (9/16 収録)Show Notes (番組HP):Position inquiry用連絡先: kenta.m.hagihara+recruit@gmail.com着任を待たずに准教授・助教公募を出せることになりました。学振PD、大学院進学を希望する方も是非早めに声かけてください。萩原ラボHP (beta) (新規ドメインとかいってハネられた場合こちらから)Allen Institute for Neural Dynamics生理研五十嵐さん五十嵐さん過去回 1 2,3 4 5藤島さんカレル北西さん回神経科学学会所長招聘セミナー直で教授になった萩原さんの先生生理研現所長・伊佐先生前所長・鍋倉先生生理研技術課自然科学研究機構の事務局NIPSリサーチフェローという独自フェローシップがある生理研大学院総研大大学院に興味のある方は体験入学制度もあるので是非FMIAllen/FMIでつかってるIVC(テクニプラスト社)奥山さん奥山さん回 1 2北村ラボ東大医学部教育研究棟GiocomoラボLeutgebラボMoserラボ部屋を11個作った研究Albert Leeの48mのでかい迷路デルタとかイプシロンとかiGluSnFR4Mini2pニューロモジュレーターを全脳で見るPerforated patch (pdf)レコーディングしてpost hocに染め物をするような研究RPEじぇれまいあIllana WittenのVTA深部イメージング論文スイスの元ボスAmygdalaのGRCGRC Basal GangliaVijay内田さんAMED五十嵐さんポスターさきがけ-生命力帰国発展研究Takaki Komiyama生理研トレーニングコースぶさきらぼSvobodaラボに行った五十嵐さん弟子の論文Buzsakiラボに行った五十嵐さん弟子の論文LongラボThomas McHughラボ金子先生くねりうむラボ: 正確には、新ラボメンバーにはまずジムがインプラントの手本を見せる。タイムリミットがあったプロジェクトでは結果的にジムが全部インプラントしたこともあった。とのことでした。ろーら・こるぎんpayline:上から何パーセントの申請書に予算が付くか年６回までしかNIH予算出せない河西先生R21R35東北大・国際卓越研究大学認定五十嵐ラボ＠東北大中川達貴さん大隅先生東北大学NeuroGlobal筒井先生松井先生佐々木先生坂野先生安田さんモーザーラボと東北大の合同セミナーMenno Witterアスコナギャツビー財団ピコカローニ狩野先生高橋先生インタビュー(pdf)横溝さん国家公務員の給料は国会議員を上回れないStanfordから東大経済が引き抜いた森郁恵先生Josh Johansen最近アナウンスした これにTomがBooってコメントつけてるのがウケる(萩)佐々木さんこれまでの過去ゲストがさんざん”参考になる”ジョブハント記録をシェアしてくれていたのに対して、あまり参考にならない体験談になってしまいました。ただ、日本の学会に顔出すとか人と会うとか、いらねーと思っていたMDが結果的に生きてくるとか「いい研究をすること」に加えて有効であったファクターのエッセンスは滲み出ているのではと思います。(1)給料水準 (2)給料着任までわからない文化 (3)着任まで異動を公表しづらい文化等、変わっていけばいいなと思います。大学医学部及び研究所の面接で何を聞かれたとか、スタートアップの額とかはシェアできるのでこっそり声かけてください。そしてラボ参加に興味ある方は是非お声がけください！ (萩)本当におめでとうございます！ラボの小部屋一つ収録スタジオにしましょう！(脇)記念すべき第100回エピソードが素晴らしいニュースで嬉しいです。第200回では何が起きるでしょうか。改めておめでとうございます！（藤）萩原君、あらためておめでとうございます！NRというコミュニティが研究者育成道場の役割を果たしているのだと思います。これから日本の神経科学を盛り立てて行ってください。日本の大学の給料や定年のメカニズムがよくならないのは、かつて教員は国家公務員であり、当時は国家公務員関連の法律に縛られていたのをいまだに引きずっているのが原因でしょうね。でも研究者獲得は世界レベルでの待ったなしの競争なので、世界水準の条件にどんどん変えていく必要があると思うのです。日本の国会議員は海外からオファーはもらえないけど、僕たち研究者は世界中からオファーがもらえますからね！日本の研究システムが少しでもよくなるよう、みんなで環境を良くしていきましょう！(五十嵐)

Broadcast from KSQD, Santa Cruz on 10-23-2025: Dr. Dawn opens with a passionate plea about E-bike safety after observing riders ignoring stop signs and wearing inadequate helmets in Santa Cruz. She explains the physics of collisions, noting that force equals mass times acceleration, and a car hitting an E-bike rider at 20 mph delivers impact equivalent to falling from a two-story building. She emphasizes that 97% of bike fatalities in New York involved helmetless riders, and brain injuries result from the brain striking the skull twice during impact - once on the impact side and again on the opposite side during deceleration. She urges drivers to honk at helmetless riders and calls for stricter helmet law enforcement. An emailer asks about hydroxyapatite in toothpaste. Dr. Dawn traces its origins to NASA research in the 1960s by Dr. Bernard Rubin studying crystal growth for preventing bone and tooth mineral loss in astronauts. Japanese company Sangi acquired the patent and created the first hydroxyapatite toothpaste by 1980, receiving official anti-cavity recognition in 1993. Studies show it matches fluoride's cavity prevention effectiveness by filling microscopic cracks where bacteria take root. It also relieves temperature sensitivity by sealing micro-fractures in enamel that expose the dentin layer, making it especially helpful for people who clench their jaws. Researchers from UC Berkeley and the Allen Institute used electrodes and lasers to study how mouse brains process optical illusions like the Kanizsa triangle. They discovered specialized IC encoder neurons in the visual cortex that fill in missing information, creating complete shapes from partial cues. When these pattern-completing circuits activate inappropriately, they may trigger hallucinations in conditions like schizophrenia. Dr. Dawn explains that illusions occur when the brain perceives something different from actual visual input, while hallucinations create perceptions with no external stimulus. She discusses frontotemporal dementia where visual hallucinations result from protein deposits in the occipital cortex, and notes that a 2021 British Journal of Psychiatry study found hallucination rates varying from 7% in young people to 3% in those over 70. An emailer describes unbearable chronic lumbar pain with degenerative disc disease shown on MRI. Dr. Dawn emphasizes that MRI findings don't necessarily correlate with pain levels, citing shopping mall studies showing equal degenerative changes in people with and without back pain. She stresses checking for sciatica symptoms including leg pain below the knee, sensory differences between sides, calf size asymmetry, and ability to walk on tiptoes and heels. Without these red flags, the degenerative disease likely isn't causing the pain. She warns against unnecessary surgery citing frequent "failed back" syndrome when surgery for a disk image doesn't "fix" the pain. She recommends water jogging with a ski vest, McKenzie exercises, abdominal strengthening, ergonomics, removing wallets from back pockets, and alternating heat and ice therapy. She discusses mindfulness meditation and cognitive behavioral therapy for pain management. A caller references Daniel Levitin's book "Your Brain on Music," discussing research using functional MRIs showing distinct brain activation patterns in musicians versus non-musicians due to integrated auditory, visual, and kinesthetic training. Dr. Dawn describes how infant brains develop from three to six layers with increasingly complex synaptic connections resembling circuit boards. She highlights a blindfold study where college students' visual cortices began responding to sound within two weeks as the auditory cortex expanded. She shares her husband's remarkable recovery demonstrating adult brain plasticity through intensive rehabilitation. Learning new musical instruments helps dementia patients by activating multiple brain regions simultaneously and improving standard cognitive test performance. A caller describes an eight-day chest cold with thick white phlegm. Dr. Dawn recommends guaifenesin as a mucus-thinning expectorant to prevent bacterial growth in respiratory secretions that serve as "bacteria chow." She emphasizes the importance of current flu, COVID, and RSV vaccinations. Secondary bacterial infections develop when bacteria colonize viral-induced mucus in the lungs and invade tissues. She advises aggressive hydration and chicken soup, which research shows helps clear mucus. Another caller provides additional information about Daniel Levitin as a musicologist, neurologist, and musician who runs the Music Perception, Cognition and Expertise laboratory at McGill University.

Andrew Humberman BioSnap a weekly updated Biography.Andrew Huberman has been especially visible over the past week with several key developments making headlines in science, health, and public speaking circles. He just released new episodes of the Huberman Lab podcast, including a widely discussed conversation with Dr. Konstantina Stankovic, chair of otolaryngology at Stanford, about the science of hearing loss, cognitive decline, and the best protocols to protect hearing. Huberman framed the growing link between hearing loss and dementia as one of the most pressing public health challenges, and the episode is rapidly climbing streaming charts. At the same time, he continued the podcast's focus on neurological essentials with another episode highlighting major advances in how the brain interprets vision and color in partnership with Brown University's Dr. David Berson. These have been spotlighted in recent roundups from academic health sites and cited on social media by science journalists and wellness influencers.Another highly anticipated story is the pre-release buzz around his book Protocols, an actionable guide to optimizing brain performance, mood, and physical health, which has now opened for pre-order according to his official website. Given the early attention and relevance to wide health-conscious audiences, some outlets predict Protocols could become a reference on science-driven lifestyle strategies. The Huberman Lab's membership continues to soar, with newsletter engagement and YouTube subscriber counts both hitting new records.In person, Huberman is generating excitement for a live tour with an upcoming appearance scheduled for October 22 in Atlanta, alongside filmmaker Casey Neistat at The Eastern. Ticketing partners like Vivid Seats confirm this is expected to be a sold-out event, and fan speculation is mounting about whether Huberman will tease additional initiatives or collaborations on stage. Meanwhile, he stays active in the academic environment, notably joining discussions hosted by the Allen Institute, where he recently interviewed Dr. Christof Koch, adding to his ongoing engagement with leading thinkers in neuroscience.Social media chatter is up, with several viral clips from his recent podcasts trending on platforms like Instagram and X. Huberman's commentary on hearing health and lifestyle optimization has been featured in micro-influencer reels, and #hubermanlab remains a top-trending topic in the wellness community. There is occasional fan speculation about possible commercial partnerships due to repeat mentions of supplement and cookware brands during his podcast ads, but these remain standard sponsor reads and not confirmed equity deals. No controversies or negative stories have emerged, and the upcoming book release and Atlanta event are widely considered his next major moments.Get the best deals https://amzn.to/3ODvOtaThis content was created in partnership and with the help of Artificial Intelligence AI

In honor of OCD Awareness Week, this episode features two deeply personal stories about living with obsessive compulsive disorder.Part 1: For Hannah Hedelius, a classmate's hiccups trigger an overwhelming reaction she can't hold back. Part 2: As a graduate student, Rachel Hostetler begins to realize that her intrusive thoughts may be more than just regular stress. Hannah Hedelius was born and raised in Idaho. She received her Bachelors of Psychology from Boise State University and is currently working on her Masters in Biomolecular Sciences. Hannah plans to attend medical school where she will work towards a dual doctorate for a career as a medical scientist. Hannah is a graduate assistant for the Dean of Students Office where she works in substance misuse prevention. She focuses on creating alcohol and other drugs education and awareness. Hannah is also doing research at Boise State where she studies cholera toxin and its potential in treatment of inflammatory bowel diseases. In her free time, Hannah enjoys reading and spending time with her family. Rachel Hostetler is a scientist at the Allen Institute for Brain Science and her role focuses on providing scientific training to users of the institute's tools and datasets. Prior to working at the institute, she completed a Ph.D. in Neuroscience at West Virginia University, where she used many Allen Institute datasets to guide her research on somatostatin inhibitory interneuron diversity. She completed her B.S. at the University of Minnesota, double-majoring in Neuroscience and German Studies. Now living in Seattle, she yells out in excitement whenever she sees a mountain (not just Mt. Rainier but literally any mountain) after growing up in the Midwest. When not yelling at mountains, she spends her free time trying new seafood restaurants with her partner, snuggling with their very needy cat, and exploring the beauty of the PNW.See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

My guest is Dr. Christof Koch, PhD, a pioneering researcher on the topic of consciousness, an investigator at the Allen Institute for Brain Science and the chief scientist at the Tiny Blue Dot Foundation. We discuss the neuroscience of consciousness—how it arises in our brain, how it shapes our identity and how we can modify and expand it. Dr. Koch explains how we all experience life through a unique “perception box,” which holds our beliefs, our memories and thus our biases about reality. We discuss how human consciousness is changed by meditation, non-sleep deep rest, psychedelics, dreams and virtual reality. We also discuss neuroplasticity (rewiring the brain), flow states and the ever-changing but also persistent aspect of the “collective consciousness” of humanity. Read the episode show notes at hubermanlab.com. Thank you to our sponsors AGZ by AG1: https://drinkagz.com/huberman BetterHelp: https://betterhelp.com/huberman Our Place: https://fromourplace.com/huberman Helix: https://helixsleep.com/huberman LMNT: https://drinklmnt.com/huberman Timestamps (0:00) Christof Koch (2:31) Consciousness; Self, Flow States (8:02) NSDR, Yoga Nidra, Liminal States; State of Being, Intelligence vs Consciousness (13:14) Sponsors: BetterHelp & Our Place (15:53) Self, Derealization, Psychedelics; Selflessness & Flow States (19:53) Transformative Experience, VR, Racism & Self; Perception Box, Bayesian Model (28:29) Oliver Sacks, Empathy & Animals (34:01) Changing Outlook on Life, Tool: Belief & Agency (37:48) Sponsors: AGZ by AG1 & Helix Sleep (40:23) Alcoholics Anonymous (AA) & Higher Power (42:09) Neurobiology of Consciousness; Accidents, Covert Consciousness (51:09) Non-Responsive State; Disability Bias, Will to Live, Resilience (55:34) Will to Live, Akinetic Mutism, Neural Correlates of Consciousness (57:43) Conflicting Perception Boxes, Meta Prior, Religion, AI (1:06:47) AI, Violence, Swapping Perception Boxes, Video (1:12:19) 5-MeO-DMT, Psychedelics, Light, Consciousness & Awe; Loss of Self (1:20:54) Death, Mystical Experience, Ocean Analogy; Physicalism & Observer (1:27:57) Sponsor: LMNT (1:29:29) Meditation, Tool: Spacetime Bridging; Ball-bearing Analogy; Digital Twin (1:36:16) Mental Health Decline, Social Media, Pandemic, Family & Play, Tool: Body-Awareness Exercises (1:41:34) Dog Breeds; Movement, Cognitive Flexibility & Longevity (1:47:17) Cynicism, Ketamine, Tool: Belief Effect; Heroes & Finding Flaws (1:52:46) Cynicism vs Curiosity, Compassion; Deaths of Despair, Mental Health Crisis (1:57:26) Jennifer Aniston, Recognition & Neurons; Grandmother Hypothesis (2:03:20) Book Recommendation; Meaning of Life (2:09:10) Zero-Cost Support, YouTube, Spotify & Apple Follow & Reviews, Sponsors, YouTube Feedback, Protocols Book, Social Media, Neural Network Newsletter Disclaimer & Disclosures Learn more about your ad choices. Visit megaphone.fm/adchoices

This and all episodes at: https://aiandyou.net/ . I am talking with neuroscientist Christof Koch, and as he says, "How is it that we, a piece of furniture of the universe like a rock or a star or a tree, can love or hate or see or hear?" What, in other words, makes us conscious, and what does that mean? He is known for his work exploring the substrate of consciousness in humans, animals, and machines and is the author of more than 350 peer-reviewed publications and five books, the latest of which is Then I Am Myself the World: What Consciousness Is and How to Expand It. A physicist and neurobiologist, he was for more than a quarter of a century a professor of biology and engineering at the California Institute of Technology in Pasadena. In 2011, he became the Chief Scientist at the Allen Institute for Brain Science in Seattle and in 2015, its president; now a Meritorious Investigator. He is also the Chief Scientist of the Tiny Blue Dot Foundation in Santa Monica, seeking to understand consciousness, its place in nature, and how this knowledge can benefit all of humanity. In part 2, we talk about a theory of consciousness that Christof is a primary researcher of: Integrated Information Theory, and tools for detecting and measuring consciousness, the magic number φ, the possibility of consciousness transfer, philosophical zombies, and neural correlates of consciousness. All this plus our usual look at today's AI headlines. Transcript and URLs referenced at HumanCusp Blog.        

The CEO of Whitepages enlightens us about the importance of sharing information in order to have a source of truth, the rise and fall of ReachNow, and we connect over our shared passion for a Ponzi scheme story.Top Stories:1. Allen Institute for AI lands $152M from Nvidia and NSF2. More tech layoffs: Oracle, F5, T-Mobile3. Everett man indicted for Ponzi scheme (and this article)4 Washington grapes save BC wineries after freezeAbout guest Leigh McMillan - CEO, Whitepages:Leigh joined Whitepages in 2017 to lead its consumer division. Two years later, she became CEO, steering the company through a new phase of growth. Before Whitepages, Leigh led marketing and growth at fast-moving startups like Avvo, which she helped grew one of the internet's largest legal marketplaces, and ReachNow, BMW's U.S. mobility service. Earlier, she led communications for the Seattle Mariners during the high-stakes campaign for a new ballpark and served as National Finance Director for U.S. Senator Maria Cantwell.Outside of the office, Leigh is an investor in Seattle-area funds, including the Allen Institute's AI2 incubator, and serves on the board of the Transparency Coalition, a non-profit advocating for responsible development of AI technology. She is also co-owner and winemaker at her family's winery, Welcome Road, and is an active supporter and former board member of Seattle's Woodland Park Zoo.About host Rachel Horgan:Rachel is an independent event producer, emcee and entrepreneur. She worked for the Business Journal for 5 years as their Director of Events interviewing business leaders on stage before launching the weekly podcast. She earned her communication degree from the University of San Diego. Contact:Email: info@theweeklyseattle.comInstagram: @theweeklyseattleWebsite: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠www.theweeklyseattle.com

Keegan McBride, Senior Policy Advisor in Emerging Technology and Geopolitics at the Tony Blair Institute, and Nathan Lambert, a post-training lead at the Allen Institute for AI, join Alan Rozenshein, Associate Professor at Minnesota Law and Research Director at Lawfare, and Kevin Frazier, the AI Innovation and Law Fellow at the University of Texas School of Law and a Senior Editor at Lawfare, to explore the current state of open source AI model development and associated policy questions.The pivot to open source has been swift following initial concerns that the security risks posed by such models outweighed their benefits. What this transition means for the US AI ecosystem and the global AI competition is a topic worthy of analysis by these two experts. Hosted on Acast. See acast.com/privacy for more information.

This and all episodes at: https://aiandyou.net/ . As my guest today says, "How is it that we, a piece of furniture of the universe like a rock or a star or a tree, can love or hate or see or hear?" What, in other words, makes us conscious, and what does that mean? He is the cognitive scientist Christof Koch, known for his work exploring the substrate of consciousness in humans, animals, and machines. He is the author of more than 350 peer-reviewed publications and five books, the latest of which is Then I Am Myself the World: What Consciousness Is and How to Expand It. A physicist and neurobiologist, he was for more than a quarter of a century a professor of biology and engineering at the California Institute of Technology in Pasadena. In 2011, he became the Chief Scientist at the Allen Institute for Brain Science in Seattle and in 2015, its president; now a Meritorious Investigator. He is also the Chief Scientist of the Tiny Blue Dot Foundation in Santa Monica, seeking to understand consciousness, its place in nature, and how this knowledge can benefit all of humanity. Why is an AI show interested in consciousness? Because the questions constantly arise, is AI conscious? How will we know when it is? How can or should we make it conscious? And if we can't answer those questions for human beings, how will we answer them for anything else? We talk about the relationships between existence, identity, quantum mechanics, language, and consciousness, and cosmic consciousness, how conscious parts of your body might be, connecting brains to each other, including an example that's already happened, and… opera. It is possibly literally mind blowing. All this plus our usual look at today's AI headlines. Transcript and URLs referenced at HumanCusp Blog.        

The Allen Institute for AI launched MolmoAct, an open-source robotics system that enables robots to convert 2D images into 3D visualizations, preview and adjust movements in real time, and interpret natural language commands. All code, data, and training methods are publicly available. MolmoAct is designed for use in unstructured environments such as homes, warehouses, and disaster response, and builds on Ai2's existing multimodal AI model by adding 3D reasoning and robotic control. The initiative supports transparency and collaboration in AI development and is part of Ai2's broader mission to provide open-source AI tools.Learn more on this news by visiting us at: https://greyjournal.net/news/ Hosted on Acast. See acast.com/privacy for more information.

Allen Institute, D3の伊藤慎也 (@shixnya)さんゲスト回、後編。釣り生活、SC (superior colliculus)での音表象の仕事、Allenに移ってからの仕事、LLMの発展による日々の仕事への影響、など  (6/20 収録)Show Notes(番組HP):Walmart なんでも売ってる全国チェーンのスーパー。銃も売ってる。Rockfish メバル(?)SCの音のspatial map論文 (Ito et al., 2020)頭部輸送(伝達)関数、Head-Related Transfer Function, HRTF音のタイミングの差で音源定位、トリは脳幹のディレイラインでやっているが、マウスはどうやら違う、というコメンタリ論文。 解説:バットなどの小さな哺乳類でInteraural Tining Difference (ITD)が定位に機能しないのはわかっていて、その外挿としてマウスもITDを使用しないだろうということは言われていました。2020の論文はITDがSCの Spatial RF および空間マップに寄与しないことを言っていて、behaviorの実験はしていないので、他の脳部位の活動にITDが寄与する可能性は排除しません。SCのマップに関してはマウスでの最初の論文になるのでそれに対してどの要素(ILD, ITD, Spectral)が影響しているかについても最初の報告となります。(伊藤)Tucker-Davis Technologies (TDT) マルチセンサリー論文SCのvision mapを崩すEph genetic manipulationモデルとその応用Camplanola, Seeman et al. 2022 大規模にマルチパッチしてconnectivityを調べたAIBSのin vitro electrophysiology部隊のflagship論文去年出た伊藤さんのvisual-behaviorのcell-typeモデル解析論文Change detection taskMarinaのplatform paper、VIPのsurprise表象が主なfinding (気づけば最初のプレプリントから既に2年経っている)Stabilized supralinear network by Ken MillerAlexの同データセットをHMM-GLMで解析した論文MarinaのeLife (Visual Behavior部隊の1st Gen Flagship論文)Shohei Furutachiの論文、上記のplatform paperとコンセプトがやや被ったSimuationでマルチパッチと対をなす元データの一つ:EMでconnectivityを大規模に調べた仕事 (MICrONS)Billeh et al., 2020 元になったAntonのグループの大規模シュミレーション仕事の前作Scanziani-Lienモデル: Lien&Scanziani 2013, 2018Mrsic-Flogelのlike-to-like connectivityを実験的に示した一連の仕事 1,2,3Wolfgang MaassのTensorflowトレーニング論文NVIDIA A100ADAMオプティマイザExponentiated ADAM Exponentialと言い間違えていましたが、正しくはExponentiatedですね(伊藤)Transformerの初出は2017年Matthew Botvinick 気づいたらなぜかYaleのLaw SchoolにいるAntonは小説家A. BrainAge of Cindyユヴァル・ノア・ハラリ21 Lessons for the 21st CenturyNexus: A Brief History of Information Networks from the Stone Age to AIサピエンス全史銃・病原菌・(鉄) ジャレド・ダイアモンド伊藤さんの新潟での登壇予定:Advances in Brain Modeling and AI applications (7/25に本学会)Connecting Digital Brains Across the World (7/23のサテライト)Editorial Notes:収録した話を聞いてるとこの人NeuroscienceそのものよりNeuroscience周辺の技術の方が好きなんじゃないの、と思いました。Neuromorphic chipに触ってみたいです。(伊藤)スマブラ強化学習とNVIDIAの株価の話をするのをすっかり忘れてました。あと”北大卒で音源定位”というとどうやってもマークKonishiなわけですが、その話をするのも忘れました。また次回のお楽しみということで (萩)Blue Brain Projectみたいなのを想像していましたが、それとは違ってプロジェクト毎に目的が設定されていく感じが新鮮でした。シンポジウム楽しそう (脇)

Allen Institute, D3の伊藤慎也 (@shixnya)さんゲスト回、前編。Allenの理論・計算部隊の実態、札幌での高校〜大学学部時代、インディアナ〜サンタクルズでの理論素粒子物理から実験神経科学への転身、科学哲学、など(6/20 収録)Show Notes(番組HP):SfN@SanDiegoは2013年でしたMindScopeChristoph KochCamplanola, Seeman et al. 2022 大規模にマルチパッチしてconnectivityを調べたAIBSのin vitro electrophysiology部隊のflagship論文D3: Center for Data-Driven DiscoveryD3のDirector公募中Uygar Sümbül (はD3ではなくてAIBSであることに初めて気がついた)Stefan MihalasMichael BuiceAnton ArkhipovLearning mFISHMarina GarrettPeter Groblewski札幌南高校響け！ユーフォニアムソクラテスの弁明ソフィーの世界能をつかんとする人 from 徒然草(吉田兼好)英語は絶対勉強するな！Peter Higgsユーイング装置によるヤング率測定John BeggsAlan LitkeSotris MasmanidisUCLA Silicon probe 萩原が昔使っていたのは128”D”Nでした。Nano-ZIntanDavid FeldheimNiel and Stryker 2008Jianhua CangMarcus MeisterFu/StrykerのVIPの論文(2014)Michael StyrkerKilosort: spikesortingのアルゴリズム+softwareパッケージ 最新版の4の論文Neuropixels1.0の最初の論文(2017)伊藤さんのUCSC時代の最初の論文(2017)Editorial Notes:人生を大きく左右したキーワードの一覧を見ると、なんとなく心をつっつかれるような気持になります。他の人には何でもないものなのかもしれませんが。(伊藤)東京支部・海外支部があるぐらい高校のOBOG会(六華同窓会)が強力なのですが、卒業後に登録等一切してないので行方不明者扱いだと思います。大学の同窓会もそうですが。(萩)ローテーションで別分野の研究や様々なラボの文化を体験する、というのはやってみかったことの一つ。やる側は大変だとは思いますが。(脇)

A debate on the mind, soul, consciousness, and the afterlife. Michael Egnor, MD, is Professor of Neurosurgery and Pediatrics at the Renaissance School of Medicine at Stony Brook University. He received his medical degree from the College of Physicians and Surgeons at Columbia University and trained in neurosurgery at the University of Miami. He has been on faculty at Stony Brook since 1991. He is the neurosurgery residency director and has served as the director of pediatric neurosurgery and as vice-chairman of neurosurgery at Stony Brook Medicine. He has a strong interest in Thomistic philosophy, philosophy of mind, neuroscience, evolution and intelligent design, and bioethics and has published and lectured extensively on these topics. His new book is The Immortal Mind: A Neurosurgeon's Case for the Existence of the Soul. Christof Koch is a neuroscientist at the Allen Institute and at the Tiny Blue Dot Foundation, the former president of the Allen Institute for Brain Science, and a former professor at the California Institute of Technology. Author of four previous titles—The Feeling of Life Itself: Why Consciousness Is Widespread but Can't Be Computed, Consciousness: Confessions of a Romantic Reductionist, and The Quest for Consciousness: A Neurobiological Approach—Koch writes regularly for a range of media, including Scientific American. His latest book is Then I Am Myself the World.

Tricks to Fine Tuning // MLOps Podcast #318 with Prithviraj Ammanabrolu, Research Scientist at Databricks. Join the Community: https://go.mlops.community/YTJoinIn Get the newsletter: https://go.mlops.community/YTNewsletter // AbstractPrithviraj Ammanabrolu drops by to break down Tao fine-tuning—a clever way to train models without labeled data. Using reinforcement learning and synthetic data, Tao teaches models to evaluate and improve themselves. Raj explains how this works, where it shines (think small models punching above their weight), and why it could be a game-changer for efficient deployment.// BioRaj is an Assistant Professor of Computer Science at the University of California, San Diego, leading the PEARLS Lab in the Department of Computer Science and Engineering (CSE). He is also a Research Scientist at Mosaic AI, Databricks, where his team is actively recruiting research scientists and engineers with expertise in reinforcement learning and distributed systems.Previously, he was part of the Mosaic team at the Allen Institute for AI. He earned his PhD in Computer Science from the School of Interactive Computing at Georgia Tech, advised by Professor Mark Riedl in the Entertainment Intelligence Lab.// Related LinksWebsite: https://www.databricks.com/~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExploreJoin our Slack community [https://go.mlops.community/slack]Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)] Sign up for the next meetup: [https://go.mlops.community/register]MLOps Swag/Merch: [https://shop.mlops.community/]Connect with Demetrios on LinkedIn: /dpbrinkmConnect with Raj on LinkedIn: /rajammanabroluTimestamps:[00:00] Raj's preferred coffee[00:36] Takeaways[01:02] Tao Naming Decision[04:19] No Labels Machine Learning[08:09] Tao and TAO breakdown[13:20] Reward Model Fine-Tuning[18:15] Training vs Inference Compute[22:32] Retraining and Model Drift[29:06] Prompt Tuning vs Fine-Tuning[34:32] Small Model Optimization Strategies[37:10] Small Model Potential[43:08] Fine-tuning Model Differences[46:02] Mistral Model Freedom[53:46] Wrap up

Tricks to Fine Tuning // MLOps Podcast #318 with Prithviraj Ammanabrolu, Research Scientist at Databricks.Join the Community: https://go.mlops.community/YTJoinIn Get the newsletter: https://go.mlops.community/YTNewsletter // AbstractPrithviraj Ammanabrolu drops by to break down Tao fine-tuning—a clever way to train models without labeled data. Using reinforcement learning and synthetic data, Tao teaches models to evaluate and improve themselves. Raj explains how this works, where it shines (think small models punching above their weight), and why it could be a game-changer for efficient deployment.// BioRaj is an Assistant Professor of Computer Science at the University of California, San Diego, leading the PEARLS Lab in the Department of Computer Science and Engineering (CSE). He is also a Research Scientist at Mosaic AI, Databricks, where his team is actively recruiting research scientists and engineers with expertise in reinforcement learning and distributed systems.Previously, he was part of the Mosaic team at the Allen Institute for AI. He earned his PhD in Computer Science from the School of Interactive Computing at Georgia Tech, advised by Professor Mark Riedl in the Entertainment Intelligence Lab.// Related LinksWebsite: https://www.databricks.com/~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExploreJoin our Slack community [https://go.mlops.community/slack]Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)] Sign up for the next meetup: [https://go.mlops.community/register]MLOps Swag/Merch: [https://shop.mlops.community/]Connect with Demetrios on LinkedIn: /dpbrinkmConnect with Raj on LinkedIn: /rajammanabroluTimestamps:[00:00] Raj's preferred coffee[00:36] Takeaways[01:02] Tao Naming Decision[04:19] No Labels Machine Learning[08:09] Tao and TAO breakdown[13:20] Reward Model Fine-Tuning[18:15] Training vs Inference Compute[22:32] Retraining and Model Drift[29:06] Prompt Tuning vs Fine-Tuning[34:32] Small Model Optimization Strategies[37:10] Small Model Potential[43:08] Fine-tuning Model Differences[46:02] Mistral Model Freedom[53:46] Wrap up

Just how weird will the AI-powered future be? To discuss, ChinaTalk interviewed Nathan Lambert, who writes the Interconnects newsletter and researches AI at the Allen Institute. We get into… Why OpenAI is trending toward engagement farming and sycophancy, The state of Chinese AI innovation six months post-DeepSeek, and the factors influencing diffusion of Chinese vs American models, Meta's organizational culture and how it influences the quality of the Llama models, Unconventional career advice for the AI age. Nathan's book recommendation: Careless People: A Cautionary Tale of Power, Greed, and Lost Idealism by Sarah Wynn-Williams Learn more about your ad choices. Visit megaphone.fm/adchoices

Just how weird will the AI-powered future be? To discuss, ChinaTalk interviewed Nathan Lambert, who writes the Interconnects newsletter and researches AI at the Allen Institute. We get into… Why OpenAI is trending toward engagement farming and sycophancy, The state of Chinese AI innovation six months post-DeepSeek, and the factors influencing diffusion of Chinese vs American models, Meta's organizational culture and how it influences the quality of the Llama models, Unconventional career advice for the AI age. Nathan's book recommendation: Careless People: A Cautionary Tale of Power, Greed, and Lost Idealism by Sarah Wynn-Williams Learn more about your ad choices. Visit megaphone.fm/adchoices

"Vous avez quoi entre les mains ?" "De l'or !"Et ça, les trois milliardaires les plus en vogue l'ont bien compris.Xavier Niel (Free), Rodolphe Saadé (CMA-CGM) et Eric Schmidt (Google) ont financé à hauteur de 300 millions d'euros le laboratoire de recherche ouverte (open source) à but non lucratif dirigé par Patrick Perez, chercheur en IA appliquée.Patrick est à la tête de Kyutai, fondé en 2023, qui est déjà l'un des leaders français en IA, avec plusieurs outils disponibles : Moshi, leur IA vocale conversationnelle ; Hibiki, pour la traduction en live ; et MoshiVis, pour l'analyse d'images.Au programme de cet épisode : taxis autonomes, erreurs inhérentes à l'IA, entraînement des modèles par les humains, problème des contenus synthétiques… et là où l'IA est la plus lucrative.Avant de fonder Kuytai, Patrick a navigué entre recherche académique et industrie. Il a dirigé la stratégie IA chez Valeo, travaillé sur le traitement d'images chez Technicolor, et il a aussi mené des travaux chez Microsoft et à l'INRIA, deux références en innovation technologique.Ce parcours lui permet aujourd'hui de s'attaquer à l'un des sujets les plus prometteurs du moment : la multimodalité en IA — une approche qui combine texte, image et audio pour créer des outils plus puissants et plus intuitifs.Et bonne nouvelle, c'est la nouvelle vague de recherche qui sera à l'origine des prochaines grandes percées dans le domaine.Cet épisode est un point d'étape pour vraiment comprendre où en est la recherche en IA et comment se positionne la France.Entre fantasmes et réalités, Patrick explique comment fonctionne l'IA et comment elle capte peu à peu les signaux du monde réel — et pourquoi c'est une révolution.TIMELINE:00:00:00 : La beauté des mathématiques appliquées rendue accessible grâce à l'IA00:11:17 : Vers une IA vraiment multimodale : comprendre sans passer par le texte00:21:20 : Donner des yeux et des oreilles à l'IA00:30:17 : La rencontre entre IA et robotique : des robotaxis à Paris ?00:48:09 : Les prochaines avancées de l'IA vont TOUT changer00:55:20 : GPT se trompe encore… et c'est une bonne chose !01:00:51 : Quand la machine devient professeur pour d'autres machines01:08:33 : L'intervention des humains dans l'entraînement des IAs est encore nécessaire01:21:33 : Le problème des contenus synthétiques qui ne se présentent pas comme tels01:34:07 : Deviendrons-nous débiles en déléguant trop à l'IA ?01:42:40 : Là où l'IA est la plus lucrative01:53:09 : Convaincre des géants : Xavier Niel, Rodolphe Saadé, Eric Schmidt02:07:36 : L'IA pour coder : où en est-on ?02:15:59 : Ce qu'on peut faire avec l'IA et le coût des GPULes anciens épisodes de GDIY mentionnés : #450 - Karim Beguir - InstaDeep - L'IA Générale ? C'est pour 2025#397 - Yann Le Cun - Chief AI Scientist chez Meta - L'Intelligence Artificielle Générale ne viendra pas de Chat GPT#267 - Andréa Bensaïd - Eskimoz - Refuser 30 millions pour viser le milliard#418 - Clément Delangue - Hugging Face - 4,5 milliards de valo avec un produit gratuit à 99%#414 - Florian Douetteau - Dataiku - La prochaine grande vague de l'IA : l'adopter ou périr ?Nous avons parlé de :KYUTAIMoshi (l'IA de Kyuntai)Inria : Institut national de recherche en sciences et technologies du numériqueStéphane MallardTest des taxis autonomes Weymo : vidéo InstaDocumentaire aux USHibiki (outil de traduction)Allen Institute for Artificial IntelligenceVous pouvez contacter Patrick sur Linkedin et sur Bluesky.Vous souhaitez sponsoriser Génération Do It Yourself ou nous proposer un partenariat ?Contactez mon label Orso Media via ce formulaire.Distribué par Audiomeans. Visitez audiomeans.fr/politique-de-confidentialite pour plus d'informations.

Consciousness is easier to possess than to define. One thing we can do is to look into the brain and see what lights up when conscious awareness is taking place. A complete understanding of this would be known as the "neural correlates of consciousness." Once we have that, we could hopefully make progress on developing a theoretical picture of what consciousness is and why it happens. Today's guest, Christof Koch, is a leader in the search for neural correlates and an advocate of a particular approach to consciousness, Integrated Information Theory.Blog post with transcript: https://www.preposterousuniverse.com/podcast/2025/03/24/309-christof-koch-on-consciousness-and-integrated-information/Support Mindscape on Patreon.Christof Koch was awarded a Ph.D. from the Max Planck Institute for Biological Cybernetics. He is currently a Meritorious Investigator at the Allen Institute for Brain Science, where he was formerly president and chief scientist, and Chief Scientist at the Tiny Blue Dot Foundation. He is the author of several books, most recently Then I Am Myself the World - What Consciousness Is and How to Expand It.Web siteAllen Center web pageGoogle Scholar publicationsAmazon author pageWikipediaSee Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

In this episode, Jeff Dance, Host and founder of Fresh Consulting, is joined by Jason Thane, Co-founder and CEO of GenUI, and Elisha Terada, Technical Innovation Director at Fresh Consulting and Co-founder of Brancher AI, to discuss the evolution and future of AI agents. They highlight the shift from traditional AI bots to agentic AI, which involves more autonomous decision-making. The conversation covers the implications of decentralized AI technology and its potential to enhance human creativity and productivity. Episode Breakdown: 0:00 - Introduction 0:40 - About the Guests 2:45 - Jason's Background in AI 5:22 - Elisha's Journey into AI 10:23 - Defining AI Agents vs. AI Bots 18:11 - Examples of AI Applications 25:10 - Future of AI Agents and AGI 29:35 -Industries using AI Agents 41:39 - Human-AI Collaboration 45:17 - Reflections on the Future

It has been a wild few weeks and eventful few months in AI: DeepSeek, OpenAI, Stargate, Microsoft, Meta, Amazon, Salesforce, Google, Elon Musk, and more. In all of this, there's a heightened focus on what it takes to train AI models and the importance of open-source AI. This week on the GeekWire Podcast, we get insights from Ali Farhadi, CEO of the Allen Institute for AI (Ai2), the Seattle-based nonprofit that has been innovating in open-source AI since long before it was popular. "If the U.S. wants to maintain its edge ... we have only one way, and that is to promote open approaches, promote open-source solutions," Farhadi says, reflecting on the past few months. "Because no matter how many dollars you're investing in an ecosystem, without communal, global efforts, you're not going to be as fast." Related Coverage and Links: Allen Institute for AI's new open-source iOS AI app runs on-device for secure, private, offline use Allen Institute for AI challenges DeepSeek on key benchmarks with big new open-source AI model Allen Institute for AI’s new model points to items in images, aims to make bigger point in industry New York Times: An Industry Insider Drives an Open Alternative to Big Tech’s A.I. Ken Yeung, "The AI Economy" newsletter: Ai2: The AI House That Paul Allen Built Ai2 Blog: OLMoE, meet iOS Hosted by GeekWire co-founder Todd BishopSee omnystudio.com/listener for privacy information.

Our 199th episode with a summary and discussion of last week's big AI news! Recorded on 02/09/2025 Join our brand new Discord here! https://discord.gg/nTyezGSKwP Hosted by Andrey Kurenkov and Jeremie Harris. Feel free to email us your questions and feedback at contact@lastweekinai.com and/or hello@gladstone.ai Read out our text newsletter and comment on the podcast at https://lastweekin.ai/. In this episode: - OpenAI's deep research feature capability launched, allowing models to generate detailed reports after prolonged inference periods, competing directly with Google's Gemini 2.0 reasoning models.  - France and UAE jointly announce plans to build a massive AI data center in France, aiming to become a competitive player within the AI infrastructure landscape.  - Mistral introduces a mobile app, broadening its consumer AI lineup amidst market skepticism about its ability to compete against larger firms like OpenAI and Google.  - Anthropic unveils 'Constitutional Classifiers,' a method showing strong defenses against universal jailbreaks; they also launched a $20K challenge to find weaknesses. Timestamps + Links: (00:00:00) Intro / Banter (00:02:27) News Preview (00:03:28) Response to listener comments Tools & Apps (00:08:01) OpenAI now reveals more of its o3-mini model's thought process (00:16:03) Google's Gemini app adds access to ‘thinking' AI models (00:21:04) OpenAI Unveils A.I. Tool That Can Do Research Online (00:31:09) Mistral releases its AI assistant on iOS and Android (00:36:17) AI music startup Riffusion launches its service in public beta (00:39:11) Pikadditions by Pika Labs lets users seamlessly insert objects into videos Applications & Business (00:41:19) Softbank set to invest $40 billion in OpenAI at $260 billion valuation, sources say (00:47:36) UAE to invest billions in France AI data centre (00:50:34) Report: Ilya Sutskever's startup in talks to fundraise at roughly $20B valuation (00:52:03) ASML to Ship First Second-Gen High-NA EUV Machine in the Coming Months, Aiming for 2026 Production (00:54:38) NVIDIA's GB200 NVL 72 Shipments Not Under Threat From DeepSeek As Hyperscalers Maintain CapEx; Meanwhile, Trump Tariffs Play Havoc With TSMC's Pricing Strategy Projects & Open Source (00:56:49) The Allen Institute for AI (AI2) Releases Tülu 3 405B: Scaling Open-Weight... (01:00:06) SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model (01:03:56) PhD Knowledge Not Required: A Reasoning Challenge for Large Language Models (01:08:26) OpenEuroLLM: Europe's New Initiative for Open-Source AI Development Research & Advancements (01:10:34) LIMO: Less is More for Reasoning (01:16:39) s1: Simple test-time scaling (01:19:17) ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning (01:23:55) Streaming DiLoCo with overlapping communication: Towards a Distributed Free Lunch Policy & Safety (01:26:50) US sets AI safety aside in favor of 'AI dominance' (01:29:39) Almost Surely Safe Alignment of Large Language Models at Inference-Time (01:32:02) Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming (01:33:16) Anthropic offers $20,000 to whoever can jailbreak its new AI safety system

Dylan Patel is the founder of SemiAnalysis, a research & analysis company specializing in semiconductors, GPUs, CPUs, and AI hardware. Nathan Lambert is a research scientist at the Allen Institute for AI (Ai2) and the author of a blog on AI called Interconnects. Thank you for listening ❤ Check out our sponsors: https://lexfridman.com/sponsors/ep459-sc See below for timestamps, and to give feedback, submit questions, contact Lex, etc. CONTACT LEX: Feedback - give feedback to Lex: https://lexfridman.com/survey AMA - submit questions, videos or call-in: https://lexfridman.com/ama Hiring - join our team: https://lexfridman.com/hiring Other - other ways to get in touch: https://lexfridman.com/contact EPISODE LINKS: Dylan's X: https://x.com/dylan522p SemiAnalysis: https://semianalysis.com/ Nathan's X: https://x.com/natolambert Nathan's Blog: https://www.interconnects.ai/ Nathan's Podcast: https://www.interconnects.ai/podcast Nathan's Website: https://www.natolambert.com/ Nathan's YouTube: https://youtube.com/@natolambert Nathan's Book: https://rlhfbook.com/ SPONSORS: To support this podcast, check out our sponsors & get discounts: Invideo AI: AI video generator. Go to https://invideo.io/i/lexpod GitHub: Developer platform and AI code editor. Go to https://gh.io/copilot Shopify: Sell stuff online. Go to https://shopify.com/lex NetSuite: Business management software. Go to http://netsuite.com/lex AG1: All-in-one daily nutrition drinks. Go to https://drinkag1.com/lex OUTLINE: (00:00) - Introduction (13:28) - DeepSeek-R1 and DeepSeek-V3 (35:02) - Low cost of training (1:01:19) - DeepSeek compute cluster (1:08:52) - Export controls on GPUs to China (1:19:10) - AGI timeline (1:28:35) - China's manufacturing capacity (1:36:30) - Cold war with China (1:41:00) - TSMC and Taiwan (2:04:38) - Best GPUs for AI (2:19:30) - Why DeepSeek is so cheap (2:32:49) - Espionage (2:41:52) - Censorship (2:54:46) - Andrej Karpathy and magic of RL (3:05:17) - OpenAI o3-mini vs DeepSeek r1 (3:24:25) - NVIDIA (3:28:53) - GPU smuggling (3:35:30) - DeepSeek training on OpenAI data (3:45:59) - AI megaclusters (4:21:21) - Who wins the race to AGI? (4:31:34) - AI agents (4:40:16) - Programming and AI (4:47:43) - Open source (4:56:55) - Stargate (5:04:24) - Future of AI PODCAST LINKS: - Podcast Website: https://lexfridman.com/podcast - Apple Podcasts: https://apple.co/2lwqZIr - Spotify: https://spoti.fi/2nEwCF8 - RSS: https://lexfridman.com/feed/podcast/ - Podcast Playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4 - Clips Channel: https://www.youtube.com/lexclips

This week on the GeekWire Podcast, we dive deep into DeepSeek, the AI project that shaking up the tech world, to better understand the underlying technical advances and the long-term implications for the industry. Joining us is Bill Howe, an associate professor at the University of Washington's Information School and the co-founding director of the UW Center for Responsible AI Systems and Experiences, among other UW roles. Related stories: DeepSeek’s new model shows that AI expertise might matter more than compute in 2025 Allen Institute for AI challenges DeepSeek on key benchmarks with big new open-source AI model Microsoft CEO says AI use will ‘skyrocket’ with more efficiency amid craze over DeepSeek Who will win in AI? DeepSeek’s breakthrough stirs questions around value capture We open the show from the Microsoft campus in Redmond, after getting an inside look at the company's history for an upcoming installment in our Microsoft @ 50 series. John marvels at the size of new campus project, which is still under way, and we experience first-hand the company's vast parking garage when we try to leave. Also on our agenda this week: Amazon's lawsuit against Washington state over a Washington Post public records request, and what it says about the conflicts inherent to Amazon founder Jeff Bezos' ownership of the newspaper. Related story: Bezos vs. Bezos: Amazon sues WA state over Washington Post request for Kuiper records With GeekWire's Todd Bishop and John Cook. Edited by Curt Milton. See omnystudio.com/listener for privacy information.

Being able to change to meet one's circumstances is essential to survival. As HG Wells famously wrote: “adapt or perish.” In this week's episode, both of our storytellers find themselves in unfamiliar territory and need to change course.Part 1: As the only American, microbiologist Chris Robinson struggles to make friends with the other researchers in Chernobyl.Part 2: In his quest to study the adaptability of stickleback fish, neuroscientist Ashwin Bhandiwad keeps needing to adjust his experiment with each new hurdle.Chris Robinson is a published writer and PAm-Costco USA Scholar in the midst of his PhD at Indiana University. His research uses the honey bee as a model to study the ecology and evolution of the gut microbiome and how evolutionary adaptations, such as antibiotic resistance, are transmitted by mobile genetic elements. Originally from the Lowcountry of South Carolina, Chris has harvested watermelon with the USDA, spent a few years as a line cook in Charleston kitchens, and was formally a Fulbright Research Fellow in Ukraine. When not staring at a computer screen, Chris can be found deep into a bicycle ride, playing in the garden, or lamenting the failure of some baking experiment.Ashwin Bhandiwad has spent a remarkable amount of time trying to understand how the brain is organized. Once called "the most handsome boy in the world" by his mom, Ashwin is now a scientist at the Allen Institute for Brain Science working on developing tools to create maps of the brain. Ashwin received his PhD in Psychology from the University of Washington where he investigated how loud noise causes damage in the inner ear. Simultaneously, he disregarded that research by attending many loud concerts. Ashwin also loves swimming, starting projects that he'll never finish, and talking in silly voices to his young children. Learn more about your ad choices. Visit megaphone.fm/adchoicesSee Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

Being able to change to meet one's circumstances is essential to survival. As HG Wells famously wrote: “adapt or perish.” In this week's episode, both of our storytellers find themselves in unfamiliar territory and need to change course. Part 1: As the only American, microbiologist Chris Robinson struggles to make friends with the other researchers in Chernobyl. Part 2: In his quest to study the adaptability of stickleback fish, neuroscientist Ashwin Bhandiwad keeps needing to adjust his experiment with each new hurdle. Chris Robinson is a published writer and PAm-Costco USA Scholar in the midst of his PhD at Indiana University. His research uses the honey bee as a model to study the ecology and evolution of the gut microbiome and how evolutionary adaptations, such as antibiotic resistance, are transmitted by mobile genetic elements. Originally from the Lowcountry of South Carolina, Chris has harvested watermelon with the USDA, spent a few years as a line cook in Charleston kitchens, and was formally a Fulbright Research Fellow in Ukraine. When not staring at a computer screen, Chris can be found deep into a bicycle ride, playing in the garden, or lamenting the failure of some baking experiment. Ashwin Bhandiwad has spent a remarkable amount of time trying to understand how the brain is organized. Once called "the most handsome boy in the world" by his mom, Ashwin is now a scientist at the Allen Institute for Brain Science working on developing tools to create maps of the brain. Ashwin received his PhD in Psychology from the University of Washington where he investigated how loud noise causes damage in the inner ear. Simultaneously, he disregarded that research by attending many loud concerts. Ashwin also loves swimming, starting projects that he'll never finish, and talking in silly voices to his young children. Learn more about your ad choices. Visit megaphone.fm/adchoices

Nathan Lambert of the excellent https://www.interconnects.ai/ newsletter and the Allen Institute joins the pod for a rundown of the biggest AI stories of this year and next. We also talk about what he's learned training advanced AI models at the Allen Institute. Outtro Music: Young and Holtful by Young-Holt Unlimited, 1969. https://open.spotify.com/track/5am0dV7aB91Q6sWqIAuurA?autoplay=true Learn more about your ad choices. Visit megaphone.fm/adchoices

Near death experiences can be profound and even life changing. People describe seeing bright lights, staring into the abyss, or meeting dead relatives. Many believe these experiences to be proof of an afterlife. But now, scientists are studying these strange events and gaining insights into the brain and consciousness itself. Will we uncover the scientific underpinning of these near-death events?  Guests: Steve Paulson - executive producer of To the Best of Our Knowledge for Wisconsin Public Radio Sebastian Junger - journalist, filmmaker and author of “The Perfect Storm: A True Story of Men Against the Sea”  Christoph Koch - neuroscientist at the Allen Institute in Seattle and chief scientist of the Tiny Blue Dot Foundation in Santa Monica California  Daniel Kondziella - neuroscientist in the Department of Clinical Medicine at the University of Copenhagen Featuring music by Dewey Dellay and Jun Miyake Originally aired September 25, 2023 Big Picture Science is part of the Airwave Media podcast network. Please contact advertising@airwavemedia.com to inquire about advertising on Big Picture Science. You can get early access to ad-free versions of every episode by joining us on Patreon. Thanks for your support!   Learn more about your ad choices. Visit megaphone.fm/adchoices

Science can sometimes feel like an exclusive club that only certain people are allowed into. In this week's episode, produced in partnership with the Allen Institute, both of our storytellers try to find their place in science.Part 1: After getting accepted to a PhD program, Max Departee can't help but feel like he's not good enough to be there.Part 2: Han Arbach is worried coming out as non-binary will ruin their scientific career.Max Departee is a research scientist from the Pacific Northwest who has always had a fascination with the natural world. A curious nature and outdoor spirt led him to attend Montana State University where, between fly-fishing on local rivers and skiing the Rockies, he earned his Bachelors Degree in Biotechnology. Max's career and training as a scientist have taken him many places, from a PhD program in North Carolina, to a small Biotech Start-up in Washington, and back to his home town of Seattle where he now works at the Allen Institute for Brain Science. Han Arbach grew up dreaming of becoming an astronaut after watching the space shuttle land at the military base their family was stationed at. As they continued to grow up and became a “frequent flyer” in the orthopedics department for various injuries, their aspirations shifted towards medical training. Encouraged by fantastic AP Biology and Chemistry teachers in high school they pursued a biochemistry major at Mount Holyoke College. Here they were encouraged by a chemistry professor to try out research. This fostered a newfound love for discovery and research, and with it a new dream career path of becoming a scientist. Han completed their Ph.D. in Biochemistry at the University of Washington studying tail regeneration and nuclear structure in tadpoles. They then did Postdoctoral work at the Fred Hutchinson Cancer Center using viruses as a tool to probe facets of cell biology. Now, they are a Program Officer at the Paul G. Allen Frontiers Group. Outside of work, you will find them raising two dogs with their partner, attempting to befriend crows, and being a poor but enthusiastic gardener.  Learn more about your ad choices. Visit megaphone.fm/adchoicesSee Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

Science can sometimes feel like an exclusive club that only certain people are allowed into. In this week's episode, produced in partnership with the Allen Institute, both of our storytellers try to find their place in science. Part 1: After getting accepted to a PhD program, Max Departee can't help but feel like he's not good enough to be there. Part 2: Han Arbach is worried coming out as non-binary will ruin their scientific career. Max Departee is a research scientist from the Pacific Northwest who has always had a fascination with the natural world. A curious nature and outdoor spirt led him to attend Montana State University where, between fly-fishing on local rivers and skiing the Rockies, he earned his Bachelors Degree in Biotechnology. Max's career and training as a scientist have taken him many places, from a PhD program in North Carolina, to a small Biotech Start-up in Washington, and back to his home town of Seattle where he now works at the Allen Institute for Brain Science. Han Arbach grew up dreaming of becoming an astronaut after watching the space shuttle land at the military base their family was stationed at. As they continued to grow up and became a “frequent flyer” in the orthopedics department for various injuries, their aspirations shifted towards medical training. Encouraged by fantastic AP Biology and Chemistry teachers in high school they pursued a biochemistry major at Mount Holyoke College. Here they were encouraged by a chemistry professor to try out research. This fostered a newfound love for discovery and research, and with it a new dream career path of becoming a scientist. Han completed their Ph.D. in Biochemistry at the University of Washington studying tail regeneration and nuclear structure in tadpoles. They then did Postdoctoral work at the Fred Hutchinson Cancer Center using viruses as a tool to probe facets of cell biology. Now, they are a Program Officer at the Paul G. Allen Frontiers Group. Outside of work, you will find them raising two dogs with their partner, attempting to befriend crows, and being a poor but enthusiastic gardener.  Learn more about your ad choices. Visit megaphone.fm/adchoices