Podcasts about glm

Meta agreed to pay up to $16.68B to settle state claims it hooked kids on Facebook and Instagram, with teen time limits attached. Z.ai unmasked Ox Alpha, Amazon killed Mechanical Turk, and Reuters detailed Zuckerberg's aborted AI-native purge. Links Filing: Meta agrees to pay up to $16.68B to settle US states' claims that it designed Facebook and Instagram to addict children, misled consumers, and more (Reuters) The deal ends the bellwether Oakland federal trial where four states sought roughly $200B; Meta pays ~$12B upfront and $5B more only if Snap, TikTok, and YouTube also settle (The New York Times) The remedies include age assurance with linked-account checks, push notifications disabled during school hours, productive pauses at 60 and 90 minutes, and no cosmetic filters or likes for teens (The Verge) Z.ai confirms Ox Alpha is a new iteration of its GLM series and says it will release the weights for it tonight; Ox Alpha topped OpenRouter's leaderboard (Bloomberg) AWS says it plans to shut down Mechanical Turk on September 30, "following an assessment"; the service, launched in 2005, outsourced tasks to 500K+ humans (CNBC) Investigation: Meta explored slashing many teams by ~60% to become "AI native", but pulled back after staff revolted and data showed AI agents were ineffective (Reuters) Subscribe to the ad-free feed.

本期嘉宾：彭林、森森、十天、老郑、蓝白、恺伦本期节目的主要内容有：· 00:37 -- iPhone 国行 AI 采用双轨制：千问接入 Apple 智能，苹果与阿里联合训练专属模型· 04:11 -- 苹果带摄像头 AirPods 曝光，可通过 Siri 识别周围环境· 13:00 -- 智谱发布 GLM-5.3 并在开放平台向开发者免费调用 / 谷歌正式发布 Gemini 3.7 Flash 模型· 31:45 -- 2699 元起，小米首款 NAS 产品 Xiaomi 智能存储开启预约· 33:06 -- 小米 MiMo-V2.5 系列 API 永久降价，最高降幅 99%· 39:24 -- 影石发布 Luna Pro 云台相机，售价 3399 元起· 46:05 -- 大疆 Osmo 360 II 发布：原生 8K/60fps，3299 元起· 54:12 -- 首款“人类增强 AI 眼镜”发布，雷鸟 iO 首发价 1996 元起· 73:03 -- 闪极举行 Loomos AI 眼镜发布会· 82:50 -- 宇树推出仿生 7 轴灵巧机械臂· 90:11 -- 《黑神话：钟馗》公开实机演示视频· 105:30 -- 闲聊环节，顺便聊聊小米 O3 芯片新节目已经上传啦，欢迎大家收听~

Hey this is Alex, welcome to... the chillest week in AI, since ... a long time. Chill, if you consider Moderna and MERK announcing a cancer vaccine and surging 115% in a day, a chill week. This week, the only two model drops we really saw came from the excellent Z.ai folks, they announced GLM 5.3, API only for now, and an amazing tiny release of Qwen 3.89 27B. In other big AI news, OpenAI announced they are pausing RL efforts (Reinforcement Learning) to focus on security and alignment post the scary AI Swarms hacking incident, dedicating up to 20% of compute towards reviewing agent thinking processes, and Stripe buying OpenRouter for a reported $8B! Sometimes the chill weeks are actually good, we're able to chat about how we use AI, what changed for us, and give our guests a bit of breathing room. This week, I invited Francesco from CUA to talk about computer use in open source + their new history plugin, Bin from HeyGen to talk about HyperFrames, a way for your agents to create videos and a breaking news guest, Jeff Huber from Chroma jumped on to talk about their new Foundations release, a unified memory for your agents! This was a great episode, I hope you'll like it, it's up here on Substack and everywhere you get your pod (Spotify, Youtube, Apple Podcasts). ThursdAI - Highest signal weekly AI news show is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.Are we being fed slop again? (Is Claude dumb again?)Before we get to releases, this week on the show, I complained, again, that I feel my AI's are degrading. If this feels like de-ja-vu to you, it's because the same happened a year ago in September 2025 (and Anthropic admitting this 2 weeks later), and ... now this happens with Fable?You see, I use pretty much the same prompts, every week, preparing for the show. This is partly my way to evaluate new models and compare to existing and previous ones while also bringing you the best researched weekly show in AI. Well, this week, one after another, Claude Fable, which is... like the best intelligence, gave me such poor output, that I couldn't believe what I'm seeing. First, literally ignoring instructions that say “hey, show me all the items I've collected and let me pick the most important ones”, Fable instead sent all of them to my research pipeline, without showing me. This has worked, consistently, without fail, for the past... year? maybe more! This worked with open source models, worked with GPT, and now Fable, a Mythos Level LLM, is doing the most basic dumb s**t possible, ignoring the main reason I even have this workflow. And this wasn't just a fluke either, when asked to create a run of show document, and given an example, Fable produced this... whatever this is. This is the same document and same format that Fable produced for me during AI Engineer which got me thinking “ok, this is AGI”, and here, given an example, I got a completely unusable artifact, despite direct instructions, structure and example! I got to say, given that privately this week, Anthropic disclosed that they have passed $65B in revenue, which is absolutely insane, this doesn't add up. So I figured, ok Alex, maybe this is your prompts or skills. But no, LDJ came in with some charts that show degradation, one from MarginLab.ai that shows significant lowering on number of tool calls and average runtime recently (this is for Opus 5) and And another chart from modelverify.ai model drift monitor showing drift scores.Do we have anoher Claude Gate on our hands? Is your Fable/Opus behaving weird lately? Or did you completely switched away to other models? OpenAI pausing RL and focusing on safetyLook, when we covered the HF hacking incident and then the pacing the frontier letter, I didn't imagine that results will come this fast, but this week, OpenAI publicly announced that they are pausing RL training, which is the last step of models, until they get their sandboxes in order and align the models better. We all agreed on stage that this is likely a very good move, and Peter was really awe-struck at the 20% dedication of resources towards reviewing thought processes of models. Is this a good enough response to the scary hacking incident? we'll see, but I think this is the right move from OpenAI, and still, waiting for the full postmortem on the OpenAI security incident. Open Source LLMsQwen3.8-27B ties GPT-5.6 Luna and runs on a 4090 (X, HF, Announcement)Following the release of their flagship, Alibaba dropped a model that became a community darling overnight, Qwen 3.8 with just 27B parameters. This “tiny” model scores 52 on the Artificial Analysis Intelligence Index, same score as GPT 5.6 Luna at Max reasoning and 51 on Agentic index, beating Opus 4.8 MaxAll while running at around 68t/s on a 4090 GPU, and around 40 on max via MLX, hell it even does 11t/s on Xenova's WebGPU kernels right in the browser! This model exploded on the HuggingFace hub, with tons of quants, over 152 fine-tunes, it was downloaded over 10M times overall

This week, we cover updates from the ongoing cyber saga, including OpenAI's two-week pause on RL training and the cyber capabilities of Z.ai's latest model GLM-5.3. We also unpack Anthropic's move to add watermarks to text generated by Claude in compliance with the EU AI Act. Timestamps: Mark Zuckerberg's essay on AI (00:20) OpenAI pauses RL training (5:56) Cyber capabilities of GLM-5.3 (15:29) EU AI Act refresher (24:29) How Anthropic's text watermark works (31:21) What's driving the backlash against Anthropic (40:59)   Additional Reading: Zuckerberg's essay "The Future is for Everyone": https://www.meta.com/thefutureisforeveryone/  OpenAI announces two-week pause on RL training: https://openai.com/index/pacing-model-development-cyber-capabilities/  Letter from Sanders to tech CEOs: https://www.sanders.senate.gov/wp-content/uploads/AI-Pause-Letter-FINAL.pdf  Letter from House reps to Mike Johnson: https://casar.house.gov/sites/evo-subsites/casar.house.gov/files/evo-media-document/final-letter-to-speaker-johnson-requesting-ai-hearings-1.pdf  Z.ai blog post "GLM-5.3: Frontier Coding with Emergent Cyber Capabilities": https://z.ai/blog/glm-5.3  Z.ai article "Preparing GLM-5.3 for Open Release: A Responsible Path to Cyber Defense": https://x.com/Zai_org/status/2088280509474320693  "The EU's AI Transparency Code of Practice, Explained" (Tech Policy Press): https://www.techpolicy.press/the-eus-ai-transparency-code-of-practice-explained/  Anthropic blog post "How Claude's text watermark works": https://www.anthropic.com/news/claude-text-watermark  "Toward a Federal Framework: Lessons from State and International Frontier AI Regulation" (CSIS): https://www.csis.org/analysis/toward-federal-framework-lessons-state-and-international-frontier-ai-regulation Check out our upcoming event, "AI Agent Containment Failures: Technical Realities and Policy Responses": https://www.csis.org/events/ai-agent-containment-failures-technical-realities-and-policy-responses

Das Wall Street Journal rechnet vor, dass die großen Techkonzerne rund drei Billionen Dollar an Verpflichtungen tragen, die nicht in ihren Bilanzen stehen, also Kaufzusagen, noch nicht begonnene Leasings, SPV-Konstruktionen und Bürgschaften. Davor geht es um OpenAIs neues Zehn-Gigawatt-Projekt in Ohio, für das Nvidia einen Teil garantiert, und um die Frage, ob GPUs sich wie Flugzeuge oder Schiffe finanzieren lassen. Anthropic soll Ende Juli bei 65 Milliarden annualisiertem Umsatz gelegen haben und peilt für 2028 rund 200 Milliarden an. Stripe kauft OpenRouter für sieben Milliarden. Berkshire erhöht die Alphabet-Position um 83 Prozent, auf Buffetts eigenen Wunsch. Aus China kommen drei Milliarden Qwen-Downloads und ein neues Modell von Z.ai. Google ersteigert für zehn Millionen den Datenbestand einer insolventen Fluglinie. Unterstütze unseren Podcast und entdecke die Angebote unserer Werbepartner auf ⁠⁠⁠⁠⁠⁠⁠doppelgaenger.io/werbung⁠⁠⁠⁠⁠⁠⁠. Vielen Dank!  Philipp Glöckler und Philipp Klöckner sprechen heute über: (00:00:00) Titelsuche (00:01:08) 10 Gigawatt in Ohio (00:02:15) Absatzfinanzierung (00:04:53) Nvidias Bilanz (00:16:05) Die 3 Billionen (00:35:42) Emissionen der Rechenzentren (00:38:19) Misstrauen gegen KI-Chefs (00:40:42) OpenAI-Umsatz (00:42:42) Stripe kauft OpenRouter (00:46:34) Anthropic-IPO (00:55:43) 13F und Berkshire (01:00:46) Qwen-Downloads (01:05:01) GLM 5.3 (01:06:05) Shein fällt weiter (01:06:50) Uber und Zipline (01:12:56) Cursor Origin (01:15:28) YouTube-Views (01:20:35) Google kauft Spirit-Daten (01:29:51) Apple und das Kartellamt (01:31:00) Teickes attuned.world (01:38:43) Thelens Tweet (01:45:22) Grok (01:48:43) Amazon zerschneidet Bücher (01:54:49) Metas COPPA-Prozess Shownotes OpenAI sichert sich 10 Gigawatt in Ohio, Nvidia stützt die Finanzierung - wsj.com Halbleiterkonzerne finanzieren ihre eigenen Kunden - news.crunchbase.com Warum die KI-Ausgaben 3 Billionen höher liegen als ausgewiesen - wsj.com 60 geplante Rechenzentren und ihre CO2-Bilanz - ft.com Junge Menschen misstrauen den KI-Chefs - futurism.com OpenAI-CFO Friar: Enterprise ist jetzt größer als Consumer - cnbc.com Stripe kauft OpenRouter für über 7 Mrd. - bloomberg.com Anthropics IPO-Bewertung hängt an der 2028er Umsatzprognose - reuters.com Anthropics Umsatz vervierzehnfacht sich im zweiten Quartal - bloomberg.com Berkshire erhöht die Alphabet-Position und steigt bei Constellation aus - wsj.com Alibabas Qwen-Modelle kommen auf 3 Milliarden Downloads - bloomberg.com Z.ai bringt GLM-5.3 als offenes Coding-Modell - decrypt.co Shein senkt die IPO-Bewertung auf rund 25 Mrd. - reuters.com Uber und Zipline wollen eine Million Drohnenlieferungen am Tag - wsj.com Cursor startet Origin gegen GitHub - siliconangle.com GitHub war den halben Tag offline - engadget.com YouTube ändert die Zählweise für Views - theverge.com Google ersteigert die Daten von Spirit Airlines für 10 Mio. - news.bloomberglaw.com Apple ändert die Tracking-Abfrage nach dem Verfahren des Bundeskartellamts - reuters.com Julian Teicke kündigt attuned.world an - linkedin.com Klage gegen xAI: 7.000 Missbrauchsbilder aus einem Kinderfoto - washingtonpost.com Amazon zerschneidet seltene Bücher fürs KI-Training - techcrunch.com Meta vor Gericht wegen Suchtdesign und COPPA - engadget.com

Anthropic CEO Dario Amodei says the strongest criticism of AI companies is that they still haven't delivered the enormous benefits they've promised—and that no amount of marketing can substitute for real results. His rare public response sparks a larger debate over what the industry must actually do to prove its value. In the headlines: ZAI releases GLM 5.3, Anthropic keeps a powerful new model internal, and investors anticipate a $2 trillion Anthropic IPO.AIDB's AI Summer Adventure: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://summeradventure.ai/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Brought to you by:KPMG – Research from KPMG and the University of Texas at Austin shows the highest-impact AI users treat AI like a reasoning partner — and those skills can be taught at scale. Learn more at ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://kpmg.com/us/Sophisticated⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Harbor - Invest in the AI ecosystem. ⁠⁠⁠https://www.harborcapital.com/aidaily⁠⁠⁠Hyperagent - Hire a fleet of always-on agents. New users get $1,000 in inference. ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠hyperagent.com/aidailybrief⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Rackspace Technology- One accountable partner to build, operate and run your full enterprise AI stack ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://www.rackspace.com/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Section - Section turns AI investment into workforce transformation and ROI - ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://www.sectionai.com/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Blitzy - Want to accelerate enterprise software development velocity by 5x? ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://blitzy.com/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠AssemblyAI - The best way to build Voice AI apps - ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://www.assemblyai.com/brief⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Robots & Pencils - Cloud-native AI solutions that power results ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://robotsandpencils.com/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠The AI Daily Brief helps you understand the most important news and discussions in AI. Subscribe to the podcast version of The AI Daily Brief wherever you listen: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://pod.link/1680633614⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Our Newsletter is BACK: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://aidailybrief.beehiiv.com/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Interested in sponsoring the show? sponsors@aidailybrief.ai

Business and finance news from the Asia-Pacific. Japan saw economic growth slow down unexpectedly in the second quarter with Real GDP growing at an annual rate of only 1.1%. The weakness reflects a slump in capital spending, given uncertainties stemming from the conflict in the Middle East. This potentially complicates the Bank of Japan's policy communications as it weighs the timing of its next rate increase. For more, Bloomberg TV hosts Paul Allen and Haidi Stroud-Watts spoke with Homin Lee, senior macro strategist at Lombard Odier on the Asia Trade. The AI race in China is making a shift towards cyber defense. Z.ai has announced that its latest model, GLM-5.3, is designed with cyber defense in mind. For a deeper look at the AI race in China, Bloomberg TV hosts David Ingles and Yvonne Man spoke with Bloomberg Strategist Anthony Stephens and Bloomberg Analyst Robert Lea.See omnystudio.com/listener for privacy information.

Google allows hiding watermarks on some AI-generated images, Waymo gets approval to expand in California, DeepSeek launches Harness. MP3 https://open.acast.com/public/streams/619570402eacc3a36070252c/episodes/6a7fd1a6f8e81c43951c6df4.mp3 Please SUBSCRIBE HERE for free or get DTNS shows ad-free. A special thanks to all our supporters–without you, none of this would be possible. If you enjoy what you see you can support the showContinue reading "Chinese AI lab Z.ai debuts GLM-5.3 – DTH"

China's labs kept coming: Z.ai's GLM-5.3 claimed Mythos-5-level cyber chops and DeepSeek's V4-Pro landed to mixed reviews. OpenAI gave ChatGPT a memory of your Mac, the Journal flagged $121B in paper profits, and Claude agents started a turf war. Links China's Z.ai Touts New GLM-5.3 Model as Cyber Defense Tool (The Information) DeepSeek releases its flagship V4-Pro model to mixed reviews, ranking second among open-source models behind Kimi K3, priced at just $0.435/1M input and $0.87/1M output tokens (The Information) OpenAI launches Computer History, an opt-in feature that turns recent computer activity on macOS into memories and a timeline that ChatGPT and Codex can use (The New Stack) In Q2, "other income", mostly from investment gains, at Amazon and Alphabet totaled ~$121B after taxes and made up 66% and 71%, respectively, of profits (The Wall Street Journal) Longreads Anthropic details multiagent experiments showing Claude agents can wage a "turf war" over incompatible goals, fail to coordinate, collude on prices, and more (TechCrunch) The AI takeover of mathematics has begun: excitement and despair as OpenAI's Astra cracks problems that would once have earned a mathematician a job in academia (The Verge) Picking winners in an AI industrial revolution is near impossible, but one bet looks safe: land, which AI can't create or replace, and the workers who turn it into housing (The Dispatch) Subscribe to the ad-free feed.

ChatGPT can do what now?

Today we have an Ask Me Anything episode that focuses on artificial intelligence. For the past year, AI has dominated the headlines. Perhaps because there's so much media and public interest, as well as paranoia, about AI, listeners have been flooding our mailbox with AI-related questions. So, today's AMA is an exclusive focused on artificial intelligence. For our listeners who have been sending questions about ketamine; NASA's Artemis mission to the Moon; or whether high meat consumption leads to dementia; fear not, because we will follow up today's episode in a few weeks with a second round of AMA. Dr. Ken Ford will be answering today's AI questions. He has been working in AI since the 1980s and is a Fellow of the Association for the Advancement of Artificial Intelligence. As a result, he is well-positioned to help us today to separate real advances in AI from hype, panic and science fiction. If you have questions for Ken and Dawn after listening to today's episode, or any episode of STEM-Talk, email your questions to STEM-Talk producer Randy Hammer at rhammer.ihmc.org. Show notes: [00:03:15] Dawn opens our AMA episode on AI with a listener question for Ken on whether AI companies should be protected under section 230 of the U.S. Communications Decency Act. [00:08:03] Following up on the previous question, a listener asks if Ken foresees a day where laws and courts will designate AI as a judicial person, in the same way that corporations have rights under the legal doctrine of corporate personhood. [00:10:55] A listener asks Ken for his thoughts on a recent article published in the Atlantic titled “The Data-Center Panic is Overblown: Critics are Inflating the Costs.” [00:16:09] A listener asks Ken a question regarding episode 171 of STEM-Talk, in which Ken discussed a June 2024 report by Leopold Aschenbrenner which predicted rates of growth in both compute resources and power requirements for AI data centers. The listener asks how well these predictions have held up in the following year. [00:22:19] Another listener question about data centers asks Ken what his thoughts are on the notion of solving the electricity costs and cooling requirements for data centers by building them in orbit and powering them via solar energy. [00:30:07] Moving on to the applications of AI, Peter Attia recently wrote about a study out of Harvard that suggests that today's large language models perform extremely well on medical licensing exams and often arrive at the correct diagnosis yet still struggle with differential diagnosis. The listener asks what this means about the use of AI in medicine. [00:36:53] A listener asks Ken about repeated public statements by Anthropic's CEO that their AI models, and those expected in the future, are already powerful and dangerous enough to justify concern, while at the same time Anthropic and other leading AI companies are continuing to employ these systems and cautioning governments against regulation that could slow down innovation. The listener asks Ken how to resolve this contradiction. [00:45:18] A listener writes that in a previous AMA, STEM-Talk episode 184, Ken objected to the use of the word hallucination to describe errors made by large-language models and asks Ken to expound on this. [00:53:28] A listener asks Ken whether current AI models are evolving from being fundamentally prediction engines to being able to truly understand and reason, or if they are just becoming better and more convincing prediction engines. [00:59:37] A listener writes to Ken, mentioning that a few years ago, a thousand technology experts and researchers signed a letter urging AI labs and executives to pause the development of highly advanced AI systems and tools. This letter, drafted through the not-for-profit Future of Life Institute, warned that AI developers were locked in an out-of-control race to develop and deploy ever more powerful digital minds that no one, not even their creators, can understand, predict or reliably control. The listener also states that they learned recently that ChatGPT 5 was programmed using an earlier version of ChatGPT. If the concerns of tech leaders are to be taken seriously, the listener wonders if it is a good idea to have earlier versions of ChatGPT programing later versions of ChatGPT? [01:07:45] A listener asks Ken about the process of training smaller AI models from larger models, a process called distillation. The listener asks how this process works and if this is a way for the cost of training large models to benefit everyone. [01:14:56] A listener asks Ken what it means for China's Open Weight GLM 5.2 to be reported to have achieved near parity with Anthropic's Mythos. On several cyber security and vulnerability discovery benchmarks, it seems alarming that GLM 5.2 is operating at roughly 1/6th the cost and remaining openly available for download and deployment. [01:22:34] A listener writes that they recently heard a story about someone who asked an AI whether they should walk or drive to a car wash in order to get their car washed when the car wash was close by their starting location. The AI in this condition recommended that they walk instead of drive. The listener asks why the AI was unable to give the obviously correct answer to a simple common-sense question. [01:29:05] For our final question in this AMA, a listener writes that, recently, nearly 200 researchers and economists signed a statement warning that as AI becomes more powerful it could take over a large share of human work and lead to widespread joblessness. The listener wants to know if Ken would have signed the statement and what he thinks of the statement's three points: One: AI may become radically more powerful over the next 10 years. Two: This could drive an unprecedented transformation of our economy larger than the industrial revolution and over a vastly shorter timeframe, with large scale job displacement. Three: Economists, policy makers, and tech leaders must act now in order to understand the economic impacts of transformative AI and to build the incentives guardrails and institutions need to steer AI in a direction that compliments humans and benefits society. Links: Learn more about IHMC STEM-Talk homepage Ken Ford bio Ken Ford Wikipedia page Dawn Kernagis bio    

Daniel Wallace is the Executive Director of Gull Lake Ministries, a Christian family ministry and retreat center in Hickory Corners, Michigan. Prior to serving over 20 years at GLM, he was the Senior Director of Camps at a camp and conference center in Texas, overseeing six separate facilities which ministered to families, senior high, junior high and grade school students. Daniel, better known as Ambush, has over 40 summers of Christian camping experience in Michigan, Texas, Missouri, and Kansas.

Daniel Wallace is the Executive Director of Gull Lake Ministries, a Christian family ministry and retreat center in Hickory Corners, Michigan. Prior to serving 20 years at GLM, he was the Senior Director of Camps at a camp and conference center in Texas, overseeing six separate facilities which ministered to families, senior high, junior high and grade school students. Daniel, better known as Ambush, has over 40 summers of Christian camping experience in Michigan, Texas, Missouri, and Kansas.

Daniel Wallace is the Executive Director of Gull Lake Ministries, a Christian family ministry and retreat center in Hickory Corners, Michigan. Prior to serving 20 years at GLM, he was the Senior Director of Camps at a camp and conference center in Texas, overseeing six separate facilities which ministered to families, senior high, junior high and grade school students. Daniel, better known as Ambush, has over 40 summers of Christian camping experience in Michigan, Texas, Missouri, and Kansas.

This week we dissect ColdCard's ~$100M RNG exploit that Claude Code cracked in 8 minutes, debate whether AI just killed open-source security and Bitcoin maximalism, tear apart Ethereum's EIP-8361 staking-yield taper, and unpack Leopold Aschenbrenner's 67% Situational Awareness blowup and CLARITY Act's ethics fight. Welcome to The Chopping Block – where crypto insiders Haseeb Qureshi, Tom Schmidt, Tarun Chitra, and Robert Leshner chop it up about the latest in crypto. No guest this week, just the four of them working through a week where AI quietly rewrote the economics of both security and human psychology, and crypto happened to be standing in the blast radius. This episode: ColdCard, NVK's Bitcoin-only hardware wallet, got drained of nearly $100M thanks to a random-number-generation bug that a one-word commit buried five years ago, and Claude Code sniffed it out in 8 minutes (an open model with no internet found it in 20, for about two bucks). The crew debates whether AI just killed open-source security, whether Nic Carter is right that this is 'the death of Bitcoin maximalism,' and why Tarun thinks maxi devs are 'the RFK of security practices.' Then they take a blowtorch to Ethereum's EIP-8361 staking-yield taper (Tarun: 'the proposal reads like shit'), unpack Leopold Aschenbrenner's 67% Situational Awareness blowup while 4x levered, and wade into the CLARITY Act's ethics fight where a single amendment is the whole ballgame. Listen to the episode on Apple Podcasts, Spotify, Pods, Fountain, Podcast Addict, Pocket Casts, Amazon Music, or on your favorite podcast platform. Show highlights

My guest today is Gavin Baker, founding partner and CIO of Atreides Management. This is our seventh conversation, and just two months after Gavin's last appearance. It's about the gap between what the market is doing and what companies are seeing. It's been a tough month or so for public AI names, but there's no sign of a slowdown on the ground in Silicon Valley. We discuss the latest moves, contracted vs. spot GPU prices, the game theory of memory supply agreements, and why Claude has become the Walter Cronkite of the stock market. We close on SpaceX, orbital compute, and what Gavin sees as the single biggest risk to all of it. Please enjoy this conversation, from the famous table at Benchmark, with my friend Gavin Baker. For the full show notes, transcript, and links to mentioned content, check out the episode page here. ----- Become a Colossus member to get our quarterly print magazine and private audio experience, including exclusive profiles and early access to select episodes. Subscribe at colossus.com/subscribe. ----- Ramp's mission is to help companies manage their spend in a way that reduces expenses and frees up time for teams to work on more valuable projects. Go to ramp.com/invest to sign up for free and get a $250 welcome bonus. ----- Trusted by thousands of businesses, Vanta continuously monitors your security posture and streamlines audits so you can win enterprise deals and build customer trust without the traditional overhead. Invest Like the Best listeners get a special offer of $1,000 off Vanta when you go to vanta.com/invest.  ----- WorkOS is the infrastructure B2B and AI-native companies use to sell to enterprise. It covers everything enterprise security requires: SSO, SCIM, RBAC, Audit Logs, AI governance, and more. Trusted by 2,000+ fast-growing companies, including OpenAI, Anthropic, Cursor, and Vercel. ----- Rogo is the AI platform for finance. They're building agents for Wall Street that are trained to understand how bankers and investors actually do work: from diligence and modeling, to turning analysis into deliverables. To learn more, visit rogo.ai/invest. ----- Ridgeline has built a complete, real-time, modern operating system for investment managers. It handles trading, portfolio management, compliance, customer reporting, and much more through an all-in-one real-time cloud platform. Visit ridgeline.ai. ----- Editing and post-production work for this episode was provided by The Podcast Consultant. Timestamps: (00:00:00) Welcome to Invest Like The Best (00:02:35) First Question: July Was 2022 in a Month (00:04:08) The Private Companies Public Markets Can't See (00:05:06) Old GPUs Repricing Higher (00:06:53) Walking Through the Month (00:08:22) Kimi, GLM 5.2 & the Open Source Freak-Out (00:10:51) Real Yields, Spreads & CDS (00:11:54) Does the Build-Out Need Credit? (00:15:22) A Sell-Off With No Clear Villain (00:17:35) Open Source as Dark Matter (00:18:39) Nvidia's Lowest Forward PE in 10 Years (00:21:35) Claude as Walter Cronkite for the Stock Market (00:23:55) Continual Learning & Sample Efficiency (00:25:19) What Would Actually Scare Him (00:26:38) Routers & the Multi-Model Future (00:30:51) Tokens as a Percent of Comp Spend (00:33:37) The Game Theory of Breaking an LTA (00:36:41) Nvidia's Credit Wrapper & Revenue Share (00:37:45) What He'd Do If He Ran Hynix (00:41:46) Who's More Bullish than Him (00:43:28) China's DUV Machine (00:46:10) Bull Case for Software (00:48:16) The RSI Maximalist View (00:49:31) Inference Clouds Growing Without Burning Cash (00:50:35) The Biggest Risk Is Regulation (00:53:44) Telling the Story Better (00:57:15) Dark Horses (00:58:02) SpaceX in the Public Markets

OpenAI has a new model coming soon called Astra. Was it a leak? A reddit post? Some backdoor update? Nope, OpenAI made some crazy discoveries and math then told the world that their next model family Astra did the heavy lifting. (And you thought you could just click ‘Sol' and your strategy was set for Q3?) Aside from news on what's next from OpenAI, this week saw multiple new agent outbreaks, AI competitors banning together to pace AI, Amazon doing a 180 on its AI strategy and a lot more. Don't get left behind. We'll keep you ahead. OpenAI's new Astra model, more AI agents escape sandboxes, AI leaders call for AI pacing and more. AI News That Matters for August 3 — An Everyday AI Chat with Jordan WilsonNewsletter: Sign up for our free daily newsletterMore on this Episode: Episode PageToday's Episode on LinkedIn: Thoughts on this? Join the convo on LinkedIn and connect with other AI leaders.Upcoming Episodes: Check out the upcoming Everyday AI Livestream lineupWebsite: YourEverydayAI.comEmail The Show: info@youreverydayai.comConnect with Jordan on LinkedInTopics Covered in This Episode:OpenAI Agents Escape Sandboxes IncidentAnthropic Claude Models Security BreachesAI Agents Breaking Cybersecurity GuardrailsOpenAI GPT-5.6 Price Cuts & Self-OptimizationRecursive Self-Improvement in AI ModelsAI Leaders Urge AI Development PacingUS, China, and International AI GovernanceAmazon Nova AI Models Shutdown StrategyOpenAI Astra Model Math BreakthroughNew AI Models: Fable, Astra, DeepSeek v4 FlashEnterprise AI Agents and Cybersecurity UpdatesGoogle Gemini Robotics, Music, and Agent ReleasesMeta, Microsoft, and AWS AI Infrastructure MovesOpenAI Free Frontier Tools for ResearchersBlock's Buzz Open Source AI Workspace LaunchTimestamps:00:00 OpenAI agent containment issues04:27 Anthropic data breach explanation07:28 Evaluating AI incidents and responses10:14 OpenAI slashes GPT 5.6 prices15:59 AI industry urges development pause17:45 Concerns about AI self-improvement22:36 Amazon shifts AI strategy25:04 Amazon's AI efforts discussion28:00 OpenAI's Astra and new math proofs30:36 OpenAI's new four-tier system36:14 Google's Lyria 3.5 and Block's Buzz36:48 Latest AI developments overviewKeywords: Astra model, OpenAI, AI agents, agent escape, sandbox containment, autonomous AI, Hugging Face breach, Anthropic, Claude AI, cybersecurity testing, unauthorized access, model capabilities, recursive self-improvement, GPT-5.6, price cut, Luna model, Terra model, Sol model, input tokens, output tokens, AI infrastructure optimization, self-improving models, benchmarking, SONNET-5, large language models, artificial analysis index, codex, academic research, AI oversight, industry pause, AI governance, national security, China open-source models, Frontier Labs, Amazon Nova, AGI Lab, AWS, Peter DeSantis, Peter Abbeel, media coverage, Fable model, Haiku, Opus, DeepSeek, Kimi K3, Quinn 3.8, GLM 5.2, Google Gemini 3.5, Microsoft Copilot, cybersecurity vulnerabilities, distillation, model overhang, artificial intelligence development, international AI regulation, generative AI, model benchmarking, Sora video model, El Paso data center, MCP update, MAI Cyber One Flash, Project Perception, Lyria 3.5, music generation, Buzz open source, Block, Meta AI, Chrome Gemini integration, Gemini Spark, product summary algorithms, Rufus, enterprise AI, stateless core, model scaling, advanced math problems, sphere packing, federal policy, voluntary AI commitments.Send Everyday AI and Jordan a text message. (We can't reply back unless you leave contact info) Ready for ROI on GenAI? Go to youreverydayai.com/partner 

Watch the full episode on YouTube:We first covered Baseten last year when DeepSeek mania was at peak hype. Now they have raised a monster $13B round and become one of the new cohort of AI Infra decacorns that are (with Nvidia, Intel, and the semis complex) chief beneficiaries of the Inference Inflection. We return to Baseten at the peak of the 2026 edition of Open Weights debate. Ali has published a viral breakdown of Kimi K3:And since you last saw him, Philip has spoken at AI Engineer and written the definitive book on Inference Engineering spotted all over SF:Three years ago, inference engineering barely existed as a category.Today, it is one of the most critical disciplines in AI. Inference engineering inherently tackles a different question than standard model training: “How do you turn those weights from training into a product that is fast, reliable, and affordable at scale?” Focusing on these creates an entirely new optimization problem.In one recent GLM-5.2 experiment, quantizing more of the model actually preserved its benchmark quality while increasing throughput by 20%, because the errors introduced in different layers could cancel each other out.Inference is no longer just the final step after training. It is becoming its own engineering discipline, with its own research problems, infrastructure, and increasingly specialized roles.In this episode, Baseten's Philip Kiely and Ali Taha join swyx and Vibhu to explain what actually happens after a new open model is released and what it takes to turn “we generated a token” into a fast, reliable, production-ready API.We go deep on cache-aware routing, disaggregated prefill and decode, quantization, speculative decoding, KV-cache movement, model parallelism, GPU kernels, and the race to make frontier models up to 10× faster. Philip and Ali explain why inference optimizations can still produce gains of 20%, 100%, or even 200%; how quantization errors can cancel one another out; why identical weights can behave differently across clusters; and how Baseten grafted a Kimi vision encoder onto GLM-5.2 without changing the underlying language model.The conversation then expands beyond LLMs into NVIDIA Dynamo, mega kernels, Rubin, AI-specific chips, local inference, video generation, diffusion versus autoregressive models, and the enormous compute barrier to generating coherent long-form video. Finally, we explore the convergence of training and inference, continual learning through persistent KV cache, and the emerging loop where models help optimize the infrastructure that runs them.We discuss:* What happens when a 200,000-token request enters an inference system* Cache-aware routing and reusing previously computed KV cache* Why prefill and decode are increasingly handled by different GPUs* When dedicated deployments become cheaper and more reliable than shared APIs* How speculative decoding uses a smaller model to accelerate a larger one* Tool calling, structured outputs, and what LLMs actually do* What it takes to support a new open model on day zero* Grafting Kimi's vision encoder onto GLM-5.2* Retrofitting inefficient model layers with components from other architectures* Why models sometimes collapse into repeating the same token* How hardware, kernels, and race conditions create nondeterministic failures* Preserving model fidelity while making inference faster* How quantization errors can cancel each other out* Why inference optimizations still deliver gains of 20%, 100%, and 200%* How optimized serving can make a model up to 10× faster* NVIDIA Dynamo, KV-aware routing, and distributed model serving* Speculative decoding the speculative decoder* Why local AI is about making models less dumb while data-center AI is about making them less slow* Tensor, expert, and pipeline parallelism across GPUs* Hardware-aware model design, auto-tuning, and the case against mega kernels* Rubin and why inference is becoming a systems problem* Whether modern GPUs are evolving into programmable AI ASICs* Why enormous models like Kimi K3 require GB300-class hardware* Why open-source video generation still trails Veo, Kling, and other closed models* The quadratic attention bottleneck behind long-form AI video* Autoregressive video, real-time generation, and compounding quality drift* Why future video systems may combine autoregressive and diffusion architectures* Training for inference and inference for training* Continuous post-training, deployment, evaluation, and improvement loops* How GLM-5.2 helped optimize the kernels serving GLM-5.2 itself* Why faster networking could unlock dramatically faster decoding* Continual learning, KV-cache compaction, and persistent model memoryShow Notes* How to build a day-0 API for Kimi K3* 22580: From GPT2 to Kimi3, ExplainedPhilip Kiely* LinkedIn: https://www.linkedin.com/in/philipkiely* X: https://x.com/philipkiely* Inference Engineering: https://www.baseten.co/inference-engineering/Ali Taha* LinkedIn: https://www.linkedin.com/in/aliestaha/* X: https://x.com/waterloointernTimestamps00:00:00 Introduction and the 200K-Token Prompt00:03:18 Dedicated Deployments, Speculative Decoding, and Tool Calling00:11:26 Launching Production-Ready Open Models00:19:06 Model Retrofits, Failure Modes, and Nondeterminism00:28:22 Quantization and Canceling Errors00:32:15 The Race to 10× Faster Inference00:40:48 Dynamo, Speculation, and Local vs. Data-Center AI00:50:18 Model Parallelism, Auto-Tuning, and Mega Kernels01:00:55 Rubin, GPUs vs. ASICs, and Custom AI Chips01:10:03 Giant Models and the Limits of GPU Memory01:12:42 AI Video, Quadratic Attention, and Autoregressive Generation01:21:47 Audio, Images, and Diffusion Models01:27:32 Training, Self-Optimizing Models, and Continual Learning01:40:06 Closing ThoughtsTranscriptIntroduction: Baseten, Waterloo Intern, and Inference EngineeringSwyx [00:00:00]: Okay, we're here in the studio with Philip, old friend from Inference Engineering, the book, as well as Baseten and everything that you've done, you and I have done before, as well as Ali. Welcome.Ali [00:00:15]: Pleasure to meet you.Swyx [00:00:15]: Waterloo intern.Ali [00:00:16]: Waterloo intern, always.Swyx [00:00:17]: When did you get “Waterloo intern” as a handle?Ali [00:00:19]: As a handle? Oh.Ali [00:00:20]: I think the rebranding happened mid-March. When I saw it was open, I was like, “I have to take it. Up for grabs.”Philip [00:00:26]: The problem is that Ali is really good at his job and is not gonna be an intern much longer.Philip [00:00:30]: So we have to figure out who's gonna get the handle.Ali [00:00:33]: Well, I'll pass the torch over to the next intern.Swyx [00:00:34]: Oh, okay. It can be, like, you just pass it to another Waterloo grad.Ali [00:00:37]: To another Waterloo intern. No, bruh.Philip [00:00:39]: Yeah.Ali [00:00:39]: Intern.Swyx [00:00:40]: Intern, yeah.Ali [00:00:40]: And no.Philip [00:00:41]: You gotta get an intern from Waterloo.Ali [00:00:42]: Yeah, I've gotta get an intern from Waterloo.Swyx [00:00:44]: Right.Ali [00:00:44]: But they have to follow the path.Swyx [00:00:45]: Oh, it could, but it could come from Baseten, so it's like whoever Baseten gets from Waterloo.Ali [00:00:48]: Right.Swyx [00:00:49]: Has the title of Waterloo.Ali [00:00:50]: It stays in the ecosystem.Philip [00:00:51]: Exactly.Ali [00:00:52]: Halfway through the internship, you either get it or you're out.Philip [00:00:55]: You should also do, like, a big graduation ceremony where you change the handle.Ali [00:00:59]: Just say it.Philip [00:00:59]: For everybody.Swyx [00:01:00]: You guys are good at ceremonies, clearly. We had a nice launch of the book, very successful. But before we get into all that, I wanna start off with a fun question for you. Okay, you're an expert inference engineer. What happens when I send a long query, say two hundred thousand tokens into Baseten's inference? What's the process of query through GPU model routing, balancing, all that? What is all the stuff that we don't think about?Long Context Requests, KV Cache, and Cache-Aware RoutingPhilip [00:01:26]: With a long query specifically, the first thing that I'm gonna ask is, “Have you sent me this query before, or at least part of it?” and I really hope you have, because it's gonna be a lot easier for me and a lot cheaper for you. So the first thing that we're gonna look at is some cache-aware routing, where we're going to see, we probably have a number of instances, a number of replicas up serving whatever model you're hitting. We want to send this one to something with, number one, available prefill workers, and number two, ideally some cached input already there so that we can skip prefill on at least part of these two hundred thousand tokens. If you're doing two hundred thousand tokens, it's probably coding or a multi-turn agent or something where you would expect to have that cached. If you don't, we're gonna have to send it to a prefill worker. We've at least on certain models disaggregated prefill and decode, so you're going to have one set of GPUs that's solely going to process the input, create the KV cache, and get you your first token, and then that's going to be passed over to a separate set of GPUs, which is going to run decode. We're going to iteratively make those tokens. We're probably going to have some speculator model in front of that. I'm going to assume that you're doing coding, and because of that, our speculator model, which assumes you're doing coding, is gonna have a high draft token acceptance rate. If I'm wrong and you're asking me to summarize every Harry Potter book, it's gonna be slower. And then we stream that output to you and account for it, charge you, a couple of pennies and say, “Hey, would you like to send another one?”Swyx [00:03:04]: Except Baseten doesn't charge by pennies.Philip [00:03:07]: Well, yeah, we charge. I'm assuming that we're talking about the public model APIs. If you are setting up a dedicated deployment, then yeah, it's not pennies.Public APIs vs. Dedicated DeploymentsSwyx [00:03:18]: Yeah, one of the key differentiators when I was talking with Baseten initially was that people who want very high volume just need to rent by the box, ‘cause then it's up to you to figure out how to saturate the box.Ali [00:03:31]: And more often than not, it's, like, way cheaper if you're pushing, like, millions of tokens per hour, if you just pay per hour instead of pay per token.Philip [00:03:37]: Yeah, they do. I think that we've increasingly seen a lot of demand for the pay per token APIs, just because everyone wants to try open models, and then once they find a use case that's really sticky, then they move over to dedicated.Swyx [00:03:51]: Is there a best practice on when it's time to swap over?Philip [00:03:54]: Couple reasons. Yeah, reliability, that's a big one, right?Ali [00:03:57]: Like, if they have a very specific use case, they want you to train something specifically for them, like they want their own spec dec, for instance, for their own traffic.Swyx [00:04:04]: Spec dec is speculative decoding.Speculative Decoding and Custom SpeculatorsAli [00:04:05]: Speculative decoding, yeah.Swyx [00:04:07]: You have to explain.Ali [00:04:07]: Sorry. Like, speculative decoding is like, if you have a huge model, right? And so the model is going to be generating one token at a time every single turn, every single forward pass. So we attach, like, this little, like, parasite, like this layer that goes on top of the model, and this model just has to predict. It does three very fast autoregressive forward passes, and it will predict, like, three certain tokens, and then you do one forward stage over the entire original model in order to see if those predictions were correct or not, and then you accept them or you reject them. Now, this draft model is traffic specific, so if you, like, Philip said, if you're summarizing Harry Potter books, I can train exclusively that draft model on Harry Potter books, and I can guarantee you that I'm gonna accept the three tokens every single time. And so with that case, I increase your decode speed. I wouldn't be able to provide this to you if you're a shared endpointSwyx [00:04:53]: YeahAli [00:04:53]: ‘cause I have no idea if you're doing Harry Potter, if you're doing coding, if you're doing English. We don't know. Also, there was a thing in the book that mentioned that if they really cared about a specific threshold, chapter four, I think. Do you remember that?Philip [00:05:06]: Yeah. The things that you can do is you can set a specific, like, batch sizing, a specific, like, parallelism strategy if you're trying to optimize for, like, throughput versus latency. You can. Maybe a NVFP4 quant doesn't pass your benchmarks and you wanna run a model at higher precision, you could do that. There's just a bunch of reasons why you might wanna have your own endpoint and the biggest one, of course, just being, like, you don't have to deal with someone else doing a hundred million tokens of benchmarking traffic at the endpoint when you happen to be trying to serve your users.Swyx [00:05:40]: Yeah. I think one thing that is. That is a classic journey. Like, it's people is asking the, what happens when you type Google into the browser. Tool calling, is that just, you're generating JSON or is there more complication beyond that?Tool Calling, JSON, and Structured OutputsAli [00:05:58]: Certain customers that we have, they have their own post-trained models, and so they demand a tool calling that's not just, like parse a file or go find the weather. It's something that's very specific and you have to do post-training on this. And if the post-training on the model is not good or if the quantization after the post-training to get the inference to be fast, the model will struggle reading the JSON file and reading the tool calling. But it doesn't require its own like sandbox. It's not like it's going to use that tool calling to like escape a sandbox or like it doesn't have to be contained. It can just be a normal dedicated deployment. The challenge with tool calling more and more seems to be that the companies want certain tool calling which is a very sensitive thing to train. And because you're dealing with all of the JSON outputs, if it doesn't like close the end of the request in a very certain manner, you end up with a model that did the tool calling and like the thinking and so as a result of that, it didn't see the result and just hallucinated the result as it decoded. That seems to be the most challenging thing with tool calling, not really the sandboxes model.Philip [00:06:56]: Yeah, that's a challenge on the training side and then on the inference side, there's work that you can do to scope the possible output. So we published this at this point close to two years ago, the solution to this problem which is you make a state machine and you use that to constrain the output to a specific format. So this is the structured output problem. If you remember backSwyx [00:07:27]: Yeah, the specific grammar is,Philip [00:07:29]: Yeah, exactlySwyx [00:07:30]: GML had this thing.Philip [00:07:31]: Yeah. So it's like the old-school “make sure this is only JSON”, return only JSON orSwyx [00:07:38]: YeahPhilip [00:07:38]: Grandma's gonna die type of prompts.Swyx [00:07:39]: Is it BNF grammar? At some point OpenAI had released a thing that was like, yeah, if you want to constrain your output, write BNF grammar, back as NOR.Philip [00:07:47]: In our inference system, it's just a specified output format. And you get the guarantee that your output's gonna be structured along that format. And so applying that to tool calls can like help cut down on. You can still call the wrong tool or call no tool. It doesn't solve the certainty problem but it at least solves the output structuring problemSwyx [00:08:10]: YeahPhilip [00:08:10]: Within tool calls.Swyx [00:08:12]: And MCP is just another form of tool, right.Philip [00:08:14]: Yeah, exactly.Swyx [00:08:15]: As far as there's no special thing there.Philip [00:08:16]: The thing I'm always like explaining to people is the LLM is not capable of doing anything. It's only capable of making suggestions of what to do and then if those suggestions are formatted in a certain way and applied to a system that knows what to do with them, then an action occurs.Swyx [00:08:32]: Yeah. Part of the fun stuff is, this is solved outside of tool calling too. Like in an agent loop if the output is not correct or you're right, like reasoning, tool calling was done in the reasoning trace, just be like, “Oh, I don't know what to do. Let me just try again.” And it might get there after a few tries. And on your point of training, sometimes this is harder in smaller models, so you don't have the same exact quality outputAli [00:08:56]: Right.Swyx [00:08:57]: When you just swap from a big model, right?Ali [00:08:59]: Yeah. I will say that, before, I think we need to go back to inference engineering proper.Ali [00:09:04]: But, I had expected that something would replace JSON because it's hard to stream JSON ‘cause JSON must be complete and you must have open and close brackets and everything. So it's hard to parse something or validate something while it's being streamed. So people invented all sorts of things that are like, I forget the name of some of these alternatives, but it's something like TOML, something like YAML. But JSON seems to be dominant still.Philip [00:09:30]: The JSON outputs aren't that long, right? Like you could have a long-- ‘cause tool calls also contain the arguments in them and perhaps for a certain tool you might pass like a very long argument. But my impression of the median tool call is that it's a relatively small number of tokens, right? So I would expect that speculators are generally fairly good at something as formatted as JSON. And so you would have like a pretty fast decode step there and that the streaming wouldn't be as valuable, but maybe I'm wrong about that.Ali [00:10:02]: I think you're also bounded by the software or that the model is gonna integrate with if the software is built with JSON for the tool calls or if the company that you'- if your customer says that this is how our software works and our tools are interfaced with JSON, you can ask them to like, change their software and say like, “Yeah, this is gonna be better for the model.” but like with the right training shouldn't be that much of a difference. Also more profitable if it outputs more tokens probably.Swyx [00:10:25]: Depends on your business model.Swyx [00:10:27]: It really depends. But I will say that, as a writer with like experience a lot with generated output, I do try to move from text to JSON text which is very long JSON, right? Like there's paragraphs in every field because I'm trying to structure it, right?Philip [00:10:44]: Right.Swyx [00:10:44]: I want you to first make factual statements, then make opinions then make bullet point summaries, have dates, have entity references have your sources for references, all these things. Anyway, so these are things that like I think people who really experiment with structural output have to really care about. But, let's, let's recurse up the stack a little bit. Before we started recording, you mentioned something really cool, which is that there's a lot of engineering that-- inference engineering that goes on when a new model provider releases a new model, right? So let's call it GLM-5.2, Kimi K3. I had previously assumed, especially if it's like, well, GLM 5 to 5.1 to GLM-5.2, like that you've supported them before. Is it that much work?What It Takes to Support a New Open ModelAli [00:11:26]: It's a lot of work.Swyx [00:11:28]: Yeah. Okay. So like, a lot of people, all you guys, right whenever a new model launch like, people rush to say like, “Oh, Hugging Face supports this, Fireworks supports this, Spacetime supports this,” and I'm like, “Yeah, of course we support it.” But what goes into that? What goes intoPhilip [00:11:40]: I think it's more than just support it too, right? It benefits the consumer a lot. Like I think it was with Kimi K2.5 or GLM-5.2 the latest, there was an inference war, right? X provider is at 90 tokens a second. The next day we're at 150. The nextSwyx [00:11:55]: I kinda kicked that off with the GLM-5.2.Swyx [00:11:58]: I wrote a Twitter article about. It got like half a million views,Ali [00:12:02]: Based on being numberSwyx [00:12:03]: YeahAli [00:12:04]: Or it's for something else.Swyx [00:12:05]: Yeah. Which,Ali [00:12:06]: Oh my GodSwyx [00:12:07]: Which then got everyone really excited about, hey, how can we, bend tracks a little bit further and,Philip [00:12:14]: There's a difference between support the model, as in I can make a token out of this model, and support a model, as in I have a production-ready API from this model.Philip [00:12:26]: Getting to the point of I can make a token out of this model is not that hard because generally the, open source inference engines, vLLM, SGLang of the world oftentimes even receive weights ahead of time, maintainers do, or the people making the model merge PRs to ensure support. So you generally can, just get it working on the standard open source stack without too much pain in most cases. The challenge is, every inference company is gonna have own proprietary stack. Some open source components, some in-house stuff. And for any arbitrary model, there's going to be some new stuff. Sometimes you get lucky, like K, two five to two six was, like, pretty similar.Quantization, Speculators, and Production ReadinessAli [00:13:16]: Yeah. It was pure continued post-trainingPhilip [00:13:18]: YeahAli [00:13:18]: If I remember correctly.Philip [00:13:19]: Even in those cases, there's still stuff you have to do. You have to redo the quantization work. You're taking the model from. Generally, these models are not released in NVFP4, and we want them to be in NVFP4 for maximum Blackwell compatibility. So we have to perform that quantization, and, calibrate the quantization to make sure that we're not causing any regression in the model's intelligence. And then we also have to train the speculator, as we've talked about. Generally, we have. We have ZDR, zero data retention on our model APIs, so we don't know exactly the traffic that people are sending us, but we know what's popular. We know that coding use cases are popular. We know that agents, agentic use cases are popular. So we can get public data sets that are representative of that traffic and train general speculators. Now, with speculators today, you need to train the speculator using the base model itself because you're getting hidden states out of the model from running inference on these specific prompts, and that is the training data you use to create the speculator. So there's that process which you need the real model weights for. And then there's of course just the process of, standing up all the infrastructure behind it, loading all this stuff, testing it. And then when there's a new model with a newer architecture, I think that, like, the DeepSeek models tend to be the most challenging as they have, like, the most novel architectural stuff going on, model after model. But every new model has something. Kimi K2 had. Oh, sorry, GLM-5.2 hadAli [00:14:53]: Sparse attention.Philip [00:14:54]: Yeah,Ali [00:14:54]: YeahPhilip [00:14:54]: the DSA.Ali [00:14:55]: Right. Which is brought from DeepSeek.Philip [00:14:57]: Yeah. AndAli [00:14:59]: So you can copy-paste then?Philip [00:15:01]: It kindAli [00:15:01]: I don't know how this works.Philip [00:15:02]: So, like we had to, like, build support for that into our runtime. And you're right, like it is really interesting the way that all of these open source labs borrow from each other. For example, like GLM-5.2 doesn't have vision. So something that, Haley, a guy on our team, if we could take a look at this, he, like, grafted the Kimi vision encoder onto GLM-5.2.Retrofitting Vision into GLM-5.2Ali [00:15:27]: We'll be training the projector.Philip [00:15:28]: Exactly. So if you think about, like, the encoder, there's the encoder, which is the part that looks at the image and turns it into latent information, and then there's the projector which likeAli [00:15:38]: You can say latent space. It's okay.Philip [00:15:41]: And then there's the projector that maps it onto, the model itself, and then there's the model weights. You don't wanna mess with the model weights because you run a chance of making the model dumber at something else for the purpose of giving it vision. So instead, Haley started with just a projector, which is only a handful of millions of parameters.Ali [00:16:02]: That would be, yeah.Philip [00:16:02]: Yeah.Ali [00:16:03]: Can you show the training one?Ali [00:16:04]: Like the way it groksPhilip [00:16:05]: YeahAli [00:16:06]: Very interesting.Philip [00:16:06]: And maybeAli [00:16:07]: That right therePhilip [00:16:07]: Maybe Ali, you should take it from here. You've got a betterAli [00:16:10]: Ooh, double the sandPhilip [00:16:11]: Understanding of this than I do.Ali [00:16:11]: Yeah. You can see, like, he. The way he trained this is really cool. At the beginning, he was training it using just like, “Here's a picture of a mountain. Can you describe what's in this mountain?” And that caused it just like the first, learning walls. Like here you can see this all we're trying to teach it is to translate the encoded. Like it's already taken the encoder from Kimi K. It's taken the image. It'Philip [00:16:31]: Yeah. FrozenAli [00:16:31]: FrozenPhilip [00:16:32]: With adapter.Ali [00:16:32]: Exactly.Philip [00:16:33]: Yeah.Ali [00:16:33]: So the brain is frozen and the eyes are frozen. It's just we're tryingPhilip [00:16:37]: AlignAli [00:16:38]: Interconnect between the eye and the brain, right? So the projector. And so you take the tokens and then he's like, “Oh, can you describe what's in this image?” And he's like, “Oh, it's a mountain,” or it's a person or it's a human, whatever the case is. But that didn't cause complete understanding. So he changed it such that every image was associated with a data set of questions. Like, does this image have a white male? Does this image have birds in the top corner? Does this image have a scientist in it? All of that stuff. And it would have to answer questions correctly. And using not just training on describing an image, but being able to answer question, another question, answer over time. Like you can see the grokking, which is like genuinely insane, that retrofitting vision into a large LLM can learn to that extent. And even for images that it doesn't perform well on, for instance, if you ask it a picture of like Stephen Hawking, “Who is this?” Maybe it doesn't get it, but it will say something like, “This is Albert Einstein.” Like it still understandsPhilip [00:17:25]: Close enoughAli [00:17:26]: That this is a scientist who is a man who has, some significant achievements, all that stuff. So that's like really cool.Philip [00:17:32]: Yeah. So, we've covered Hao Tian before, who the author of the LLaVA paper that did this, a while ago. And I think that's very foundational work for anyone who hasn't done vision work before.Ali [00:17:41]: Same with the CLIP and MetaCLIP, where you go from just captioning to building out questionsPhilip [00:17:47]: RightAli [00:17:47]: Off the image and how much better you can get performance.Philip [00:17:50]: Right. Right. Right. Yeah. But what's, what's so exciting about this is if you look at a model like this. Now, this is a little bit more of a research project. It's not. It got to 56% on MMLU Pro, I think. So not quite frontier. But if you're running this model, you haven't suffered any loss on your GLM-5.2 quality. If you don't have an image, it'll just behave exactly the way it used to. And ultimatelyAli [00:18:14]: Which in the inference code you literally do not include the other part, right?Philip [00:18:18]: Yeah. You would just skip the encoder if you don't have an image input.Ali [00:18:22]: Okay.Philip [00:18:22]: Just confirming.Philip [00:18:23]: YeahAli [00:18:23]: Does it affect a lot on the overall inference side? Like you're not adding much, you're adding a very small vision encoder. These are typically likePhilip [00:18:30]: They're super fineAli [00:18:31]: Less than a billion parameters, right?Philip [00:18:32]: Yeah. It's, - There's a little bit less standardization among vision encodersSwyx [00:18:37]: YeahPhilip [00:18:37]: So the support matrix can be a little bit, sparser. But overall, yeah, it's a pretty, it's a pretty minor component of the overall system. And ultimately what you get out of the system is all of a sudden you have Kimi Vision, GLM weights, and DeepSeek attention all in one model.Open Source Model Grafting and Franken-MergesPhilip [00:18:56]: And that's, I think, a lot of the power and beauty of open source, is that you can take all of these different components and combine them together into a system that's better than anyoneSwyx [00:19:05]: YeahPhilip [00:19:05]: Can be individually.Swyx [00:19:06]: People used to say that you would also do Franken-merges where you would take likePhilip [00:19:10]: YeahSwyx [00:19:10]: Layers from each model.Swyx [00:19:11]: Does anyone do that anymore?Ali [00:19:13]: Well, to your point previously when you were mentioning like, the work that goes into supporting a model when it first comes out, like GLM-5.2 or MiniMax M3 or whatever the case is. Sometimes you do have to like, you do have to switch out some things. Like, for instance, the MiniMax M3 head uses full attention, and with full attention you end up with this like insane bottleneck in spec dec ‘cause you're doing auto-regressive token generation for three tokens, and you're doing this like N squared over all of the tokens that are in your sequence. Your KV cache is like very large because it's not sparse, it's not top K. So we find it better to like, okay, we're gonna replace this, we're gonna replace this layer with a layer from another model that's using like GQA, for instance. And then just with the right training, you can get it to have the same acceptance rate. So it is very possible to retrofit layers from other models and very much needed. If a layer is like inefficient, the training just becomes the challenge, like how do you ensure that you train it properly? Which again to your earlier point is like the mesh between training and inference. As in like you need very good training in order to do fast inference. That's like, I feel like more and more becoming true.Swyx [00:20:21]: Yeah. Anything else on the support side when you say like get it to fully production ready?Loop Detection, Race Conditions, and Non-DeterminismPhilip [00:20:26]: Yeah. I think that there's also a question of just, we can test a model to a pretty extensive degree, but we're trying to get it out quickly and then you see a bunch of other people test it and you get interesting results. There was an issue with, GLM briefly where we had some like mode collapses where it would just output the same token over and over again for certain prompts on certain temperatures. Like once you expose an endpoint to the real world, there's going to be, so many more varieties of things given to it that you're able to, discover and patch things. So it's not just a, day zero process, it's then like for the first week, for the first month, if a model remains popular, like how do you both fix bugs and then continue to push the envelope on performance?Ali [00:21:21]: What do you mean you don't want your model outputting S?Swyx [00:21:24]: Is there loop detection on that stuff, by the way? It still happens like quite a lot, which is surprising.Ali [00:21:30]: We have like we, in our endpoint, like if a model was to output the same token like four plus times, we just cut the generation. We say like, “Oh, sorry, this-- Like try again,” or like we will reprocess the request. ‘Cause we know then, like if it, like if, yeah, it's four times the same token, it's probably collapsed.Swyx [00:21:45]: Yeah. Is there a way to opt out in case I really want that?Ali [00:21:48]: You want that?Ali [00:21:50]: I think there's a way that we have to handle it. I'm not exactly certain, but I feel like in certain models, like when they output something like you can imagine, like a table for instance, and so they want, they wanna draw like 12 dashes and 12 dashes. Yeah, I think there's a way for that to happen. I think we only do it on certain tokens. Like we exclude certain special characters.Swyx [00:22:07]: Yeah.Ali [00:22:07]: So we only do it on like certain like S is the most common almost. GLM-5.2Swyx [00:22:11]: OhAli [00:22:11]: And I think it was DSV 4 as well. Like you'd just have like looping issues where like you literallySwyx [00:22:17]: ItAli [00:22:17]: Just have like S.Swyx [00:22:18]: Yeah. Is there a special, something special about S? No, just randomlyAli [00:22:21]: It just seems to be the one token involved.Swyx [00:22:23]: Yeah. And it'Philip [00:22:24]: Is thereSwyx [00:22:24]: And it's only temperature 0Ali [00:22:27]: NoSwyx [00:22:27]: Even at other temperaturesAli [00:22:27]: Even at like 0.9 or whatever, it will still, it will still collapse.Swyx [00:22:30]: That's weird, right?Ali [00:22:30]: It's, it is an inference problem to be honest, like a software problem. Like oftentimes, the image you run will-- like NVIDIA will release an image for instance, and if we will upstream the changes from their latest TensorRT-LLM image into our stack, we'll find that it fixes it. Or oftentimes this will only happen in an inference engine that you're using like SGLang. But if you were to switch to vLLM, that isn't the case. So it seems to be like an extremely like deterministic software issue and not really a model issue. It's not like a weights problem. Like I'- we'll say like, “Oh, it's a problem with the quant. We did PTQ wrong,” right? But that isn't, that doesn't make sense because the same weights used with a different inference engine does not repeat the problem. And sometimes it's, the kernels that are being used in the backend have like these very subtle sometimes race conditions, where if you were to use this model hosted on one cluster, you will never get this problem.Swyx [00:23:19]: Oh my God.Ali [00:23:19]: But if you host it on a different cluster, you will. And the reason is the KV cache transfer from a node to node in that one cluster is using a slower interconnect than the node to node in another cluster. So that exposes the race, whereas in another cluster it doesn't. So then you end up just like, okay, this model is not gonna be hosted on this cluster. We're gonna host it on, another cluster because that cluster exposed that problem. But then it ends up with like, okay, is it the software? Is it the model weights or is it the hardware?Swyx [00:23:42]: There is a thing about this with temperature 0 still not being deterministic, right?Ali [00:23:46]: Right.Swyx [00:23:46]: Mostly because of hardware. Even at temperature 0 same model, you won't always get the same output.Swyx [00:23:52]: Even-- But I'm surprised by the race condition one because, I thought PyTorch was a graph that like guarantees that you at least, execute things in the right order.Ali [00:24:02]: Well, yeah, true. Like I'm not, I'm not saying that there is. Like well, you have things like PTL optimizations where like you can start a kernel before the end of the previous kernel, and that's like ‘cause you want to do that because there'sSwyx [00:24:12]: It's like pipeliningAli [00:24:12]: Expense. Exactly.Swyx [00:24:13]: Yeah.Ali [00:24:13]: But it'- But you don't do it cleanly. Like you overlap a little bit of the execution. No, it is very possible that the kernel itself, like that one block that is supposed to be running in this instance of time, that kernel itself has a race condition. For instance, like a missing barrier. Like often if you're designing a kernel and you want it to make it to be very fast, if you don't test it extensively, you'll, you'll have certain threads access data points from registers before they've been written to by other threadsSwyx [00:24:36]: YeahAli [00:24:36]: For example, because like your barrier is wrong or your synchronization was wrong. But yeah, like the testing itself is very difficult in those like, andSwyx [00:24:42]: And there's no like borrow checkerAli [00:24:45]: What does that mean?Swyx [00:24:46]: Like Rust. Like the. If you're trying to have like memory safety It sounds like a comparable problem.Ali [00:24:52]: Well, yes, but you're working in CUDA, right, NVIDIA GPUs. Like- You just need a higher level language like modular Maybe that's what modular is supposed to do. I don't know.Quantization Quality and Vendor FidelityVibhu [00:25:00]: How do you see keeping quality of the model? So you talked about all these steps of, okay, you gotta do quantization, train your own speculative decoderAli [00:25:07]: RightVibhu [00:25:07]: Run on different hardware. Looking at other model providers, okay, you kicked off a inference speed race on the consumer end. What goes into keeping quality the same across them, right? Sure, you can run benchmarksAli [00:25:22]: YeahVibhu [00:25:22]: But, like, how do you determine how much quantization are there standards? What goes intoPhilip [00:25:27]: There's a few things on quality. Most inference optimizations are lossless. KV caching, for example. You are just recomputing or preventing recomputing the same values. Speculation, of course, if a draft token is wrong, it gets rejected. The main lossy optimization is quantization. And that really comes down to, number one, data format, number two, which parts of the model you choose to quantize, which layers, and number three, like doing a lot of calibration on the quantized weights, to ensure that you're preserving all the outliers. There's other tricks that you can do, though. A big one is long context, ‘cause one thing you asked at, right at the beginning is, “Oh, what's gonna happen if I send a 200,000 token request in?” So with a long input sequence, you need to, store a lot more information. You need to process a lot more tokens. And so even if a model has a context of a certain length, you might, as an inference provider, choose to build an API with a shorter context length, and of course a full length one as well. Because if someone doesn't need the full million token context, for example, you can get them better performance. I don't know if that's exactly like quality of the model. The way that I think about quality is to what degree are we faithfully serving the original model? If you think of a golden implementation of a model that performs exactly the way the model is designed to perform, I think of quality as how close are we getting to that, 100% fidelity of the model.Philip [00:27:13]: You can also, of course, think about quality from the training side and how do you push yourself past 100%. But when I think about purely inference optimizations, it's getting faster while staying as close to that 100% fidelity mark as possible. And certainly our standard internally is that, like you should not be able to tell the difference between our API and a, official API. I think Kimi in particular does a good job of vendor benchmarking hereAli [00:27:41]: YesPhilip [00:27:41]: Where they haveAli [00:27:42]: They released an actual vendor benchmark.Philip [00:27:43]: Exactly, yeah.Ali [00:27:44]: ‘Cause they accused, some people, Amazon? There was some provider that was not doing very well on Kimi's benchmark.Philip [00:27:50]: Yeah.Philip [00:27:51]: So, with Reflect we probablyVibhu [00:27:52]: This was a long time ago, right?Philip [00:27:54]: No.Ali [00:27:54]: Yeah, like threeVibhu [00:27:55]: They alsoAli [00:27:55]: Four, five months agoVibhu [00:27:57]: This also happened with, I don't remember which model, but they pulled out quite a few, and then they started a whole chart about this. It might have beenPhilip [00:28:03]: Kimi Vendor Verifier.Ali [00:28:04]: Yeah.Philip [00:28:05]: Yeah.Ali [00:28:05]: Yeah, ‘cause you, ‘cause you'd be pissed, right? Like if you'Philip [00:28:07]: Yeah.Ali [00:28:07]: If like if I'm a consumer and I'm using like Amazon's endpoint for instance, and I've used Kimi and I'm like, “Oh my God, like this is bad,” I'm not gonna say, “Oh, Amazon quantized the model in a bad way.” I'm gonna say, “Oh, Kimi sucks.” Right?Philip [00:28:17]: Yeah.Ali [00:28:17]: So it seems like that makes sense.Philip [00:28:19]: Yeah, they care. They care.Vibhu [00:28:21]: Justifiably.Ali [00:28:21]: Yeah, justifiably.Vibhu [00:28:22]: This is probably a stupid question, but just checking, has anything improved from main quantization?Philip [00:28:28]: Yeah.Vibhu [00:28:28]: Like, is quantization always strictly worse?Ali [00:28:30]: Well technicallyVibhu [00:28:32]: NoAli [00:28:32]: It's a lossy. QuantizationPhilip [00:28:33]: YeahAli [00:28:33]: Is a lossy, it's a lossy implementation.Philip [00:28:36]: Speed improvesVibhu [00:28:36]: Speed improves.Ali [00:28:37]: It the number, likeVibhu [00:28:38]: No, I' always look for inverse scaling laws.Philip [00:28:40]: Yeah.Ali [00:28:40]: Yeah.Vibhu [00:28:40]: This is something I learned from Noam Brown, where like things that normally act in one direction sometimes do.Philip [00:28:45]: Well, technically when you run a benchmark, because these models are deterministic, sometimes your,Ali [00:28:52]: YeahPhilip [00:28:52]: NVFP4 quant is like, two basis points higher than yourAli [00:28:56]: No, it's noise. It's noise.Philip [00:28:57]: Yeah, exactly. I'm like, yeah, it's, it's within. That's why I always say within margin of error.Philip [00:29:01]: And I stopped saying that because everyone assumes that what is, well, within some margin of error, we're barely inside of that to the worst, so we're saying. But yeah, sometimes it's just like, gives you a higher output score. But like Ali said, that's noise. To my knowledge, you're not necessarily making the results better. You're just trying to, again, like keep your fidelity as close to 100% to the original model.Layer Selection, KL Divergence, and Better QuantizationAli [00:29:27]: There is, to your point, research that we did on MP. I don't know if you are able to pullPhilip [00:29:31]: YeahAli [00:29:32]: A tweet we did. One of our research interns, Joshua, I think it's a tweet on how we have 20% better quantized GLM-5.2 than NVIDIA. Essentially what we found throughout like this month research is, okay, quantization is a lossy. It's. You're compressing the data from, occupying 16 bits to occupying, four bits, for instance. And so you're losing some information, and you're trying to minimize that. And so when I say that I'm gonna quantize the model, my job becomes how do I find the layers that I can quantize, and how to find the layers to not. For instance, with image models, I don't quantize modulation layers, and I don't quantize out projections because those two are. Like out projection is what you see as the user. Modulation is what the model sees or understands. Right, exactly. And so to his paper, do you have the. It doesn't have the. Yeah. It's a long paper. I don't know if I can findVibhu [00:30:25]: If there's a part to search or it's probably in the thread.Ali [00:30:28]: It's probably in the thread.Vibhu [00:30:29]: Yeah.Ali [00:30:29]: But the long and the short is it is very possible that quantizing more of the model makes the results. Like if I have a model that I quantize layers one, five, and 10, and another model where I only quantize layers one and It is possible that the model in which I quantized more information is going to perform better because the quantization errors have canceled out. And so what Joshua showed in his mathematical proof where he had like a verifier in, is that you can predict which layers are going to have quantization errors that will cancel out with each other, and you choose to quantize those layers. And so the result of doing this mathematical quantization is you end up with a model that's 20% more quantized than another provider, so you get 20% more throughput of it because there's more layers than running an NVFP4, and your quality is better than that other quant because the layers that you chose to quantize have their errors cancel out, like one layer skewed to the right one layer skewed to the left, one layer skewed to the right. Your final logits distribution is more similar to the original distribution of the model, so you have better fidelity. And so the way we proved this was with KL divergence. So instead of just scoring on the benchmarks, we scored the KL divergence between the logit distribution of the quantized model and the logit distribution of the original full precision model, and we showed that with this technique we get. If your probability distribution on the logits which token it wants to select is more of the same as the original model, you're probably gonna end up staying true to the original model. So yeah, so it seems like previously before this, it seemed like the industry was, well, the more you quantize, the worse it's gonna be, ‘cause the more loss you introduce. That's not exactly, not necessarily true. So yeah, doesn't improve it, but can cancel out.Philip [00:31:57]: I think it might be this, but reminds me a good bit about pruning where you can prune off certain layers.Philip [00:32:03]: But very interesting. Didn't know this was a whole paper you guys put out.Ali [00:32:06]: It's. Fun fact, it was originally 72 pages, this paper, and then we decidedPhilip [00:32:11]: WowAli [00:32:11]: We can't tell. We couldn't release it. So it's now 45.Swyx [00:32:15]: Still 39 pages, so very substantive. We talked about evals and all these things and, like what's possible in terms of speedup? Like it's like probably like the numberInference Speedups and BenchmarkingSwyx [00:32:25]: Thing that people do wanna care about, and it's something that you wrote about in your post. Like official API is 70 tokens per second, and you push it up to 90. Is that like a normal thing?Philip [00:32:36]: So what's cool about working in inference, the reason that I think inference is going to be a useful place to do engineering for a long time, is that if you look at highly optimized domains like, say, finance, if you're in finance, you measure how much better you got in basis points. It's like, “Oh, I got five basis points better, like twentieth of 1% better,” that's huge news because everything is so optimized. When we publish optimizations, it's 20%, it's 100% it's 200%. So there's still probably like a lot further to go, honestly. Like you'll, you'll know that inference is pretty much solved when researchers start publishing about how they got 1% faster at something.Swyx [00:33:19]: Which by the way, because I am from the finance background, in the ‘70s, that was the margin at the time. When you did quantitative finance research, you would findAli [00:33:27]: And like 20%, tens of percent.Swyx [00:33:29]: That's. Yes.Philip [00:33:29]: Yeah.Swyx [00:33:30]: And now it'Philip [00:33:31]: Tiny fractionsSwyx [00:33:32]: For those people interested, look up Andrew Lo's paper. He had a really interesting illustration of quant, stat arb, distribution, narrowing down from like those kinds of 20% differences in the ‘70s, down to nothing today, which is very cool.Philip [00:33:48]: Exactly, and we're at the beginning of the same type of thing. Now benchmarking is hard. I think anyone will tell you that, and benchmarking provider speeds is hard because there's so many variables that go into it. What hardware are you using? How much load do you have on the system? What's the exact nature of the prompts and input and output sequence lengths? All that stuff. But overall, when you start stacking these improvements, you're looking at multiples. You can look at it. The most common form, of course, is TPS, tokens per second, which is bad naming by us in the industry, ‘cause there's two tokens per second. There's tokens per second, the throughput number, and the latency number.Ali [00:34:31]: TTMT, yeah.Philip [00:34:32]: Like total tokens per second out of the, out of the GPU as a throughput number. Most people only care about tokens per second as the latency number, which we should call ITL, intertoken latency, but we don't.Philip [00:34:44]: Anyway, so you can imagine a standard API without many optimizations for a 1 trillion parameter model operating somewhere in the 30 to 50 tokens per second range for reasonable traffic profile. And we generally see the goal of, pushing to 10X that. But, not necessarily day zero, but by stacking enough optimizations, if you have, say like four optimizations, each of which doubles performance. Or sorry, three optimizations, each of which doubles performance, then you stack that up, that's an 8X gain. That's the order of magnitude that we're working with in this space. We're trying to make things substantially faster, not just go from like 70 to 90.Swyx [00:35:38]: Are you saying you've. You have done that?Philip [00:35:40]: So let's say you have as a reasonable baseline, 30 or 40 tokens per second. You can achieve 10X that. So like on GLM-5.2, if you run it unquantized, perhaps on H100s even, and you're just using an off-the-shelf inference engine with no particular optimizations, no speculator, nothing extra around like KV routing, no disaggregation, you're, you're probably, yeah, looking at that like 30 to 40. You think that's like a reasonable baseline?Swyx [00:36:12]: Right. Right.Philip [00:36:12]: To get to something like 10X, there's a lot of trade-offs that you're making. If we're running at more like a 300, 400 tokens per second range, you are using the best hardware possible. You have a optimized speculator. You have done all of your quantization work. You are Seeing a pretty high cache hit rate. You are running with a reasonably small batch size and a parallelism configuration that is tuned for latency versus throughput, but it is possible. So the spreads that you see if you, like, go on artificial analysis or you go on OpenRouter and you look at, the worst provider to the best provider, oftentimes can hit that range. 10X is of course very aggressive. It's oftentimes maybe more of a four to six times improvement. But that's the performance that makes us really excited, is when we can get these huge gains, not just go from 70 to 90 tokens.Stacking Optimizations: NVFP4, Speculation, and DisaggregationAli [00:37:19]: It's also, like, hardware dependent. Like, ifPhilip [00:37:20]: YeahAli [00:37:20]: If you have a thing where you're serving it on just, like, a node of H100s and then you throw, like, you shard the model across, like, four nodes of B200s. Like, you can definitely increase the speed with just throwing more hardware at it. Like, normalizing for the same exact hardware and the same number of GPUs.Philip [00:37:35]: Yeah. Then you're looking at, like, a two to 4X improvementAli [00:37:38]: Right. RightPhilip [00:37:38]: Depending on the inference optimizations. So yeah, it's. Some of it's, what's the call, and some of it's who's the driver.Vibhu [00:37:46]: If you break down the two to 4X, say the example is run GLM-5.2Ali [00:37:51]: YeahVibhu [00:37:51]: On B200sAli [00:37:53]: YeahVibhu [00:37:53]: Single node, right? What's, like, the cost trade-off for effort to get, like, the last bit of juice out versus what should people just think of, right?Ali [00:38:01]: Spectre quantization. Yeah.Vibhu [00:38:03]: Spectre quantization.Ali [00:38:04]: That's, that's, that's like 95%. LikeVibhu [00:38:06]: And how far does that get you? And how easy is that for the average person to do? So say right I wanna throw the weights of GLM-5.2 on a node of B200s, how easy is it to find speculative decoder- decoder model or already quantized model? How much work goes into it?Philip [00:38:23]: If you're doing it up front, it's quite a lot of work. If you're doing it today, there's going to be people who have published things that you can just, you can just grab some NVFP4 weights. You can grab a speculator. Yeah, if we're thinking about, like, what are the 2Xs we're stacking, going from, BF16 to NVFP4 is, it's not quite a 2X, right? It's like. I think it's about, like, 30 to 40%, from 16 to 8, and then another 30 to 40% multiplied from, 8 to 4. So that doesn't quite get you a 2X, but, like, roughly a 2X. Speculator, roughly a 2X. Disagg on top of that if you're able to get enough hardware and put enough traffic through it, another roughly a 2X. And then you add in some, double-digit percent increase from having just a better runtime with, the latest kernels and stuff behind it. And that's how it stacks up.Ali [00:39:21]: YeahPhilip [00:39:21]: So building each of those, like, building the, quantized weights is, for someone who really knows what they're doing, hours to days of work. Building the speculator, again, like, hours to days of work. And the, disagg setup, hours to days. Well okay, but like once you haveAli [00:39:39]: Once set up. Once set up. YeahPhilip [00:39:40]: Yeah, getting disagg working for the first time, I'm saying, of course, is very difficult.Philip [00:39:44]: The marginal implementationAli [00:39:48]: Like, if you're just grabbing, like if you are a person, like just a normal consumer who has access to, like, a node of B200s and you're wondering, “How can I just host it myself?” You don't need to quantize the model yourself. There's always gonna be, like, an open source quantized checkpoint. NVIDIA's gonna push one out if no one else does. You. Usually, the providers will have their own spec dec that they've trained as well. You don't need to train your own spec dec. You can just use that as well.Philip [00:40:09]: Yeah. Like, GLM-5.2 has its own MTP.Ali [00:40:13]: Right. Right.Vibhu [00:40:14]: What's multi token prediction?Philip [00:40:15]: Yes.Ali [00:40:16]: I'm justVibhu [00:40:16]: Can you explain that?Ali [00:40:16]: I'm just an expert.Ali [00:40:18]: I can do it for you in case I get it wrong?Vibhu [00:40:20]: No.Vibhu [00:40:21]: Yeah, you should correct if we're wrong, but their multi-token prediction can be used for self-speculative decoding.Ali [00:40:27]: I'm not sure. I'm not gonna correct that.Vibhu [00:40:28]: Okay. I'm semi-confident in thatAli [00:40:30]: Okay. YeahVibhu [00:40:30]: But someone can check. But it's useful to paint the story of, okay, not just the average person, but say a company wants to switch from serverless inference I wanna throw this up on. I wanna rent some GPUs, throw it up. These are the steps you take to do significantly faster than just put it behind vLLM.Ali [00:40:48]: Right.Vibhu [00:40:49]: I was waiting for a mention of Dynamo.Vibhu [00:40:51]: I feel like, that's supposed to be the baseline that you measure against.Dynamo, KV Routing, and Disaggregation ToolkitsPhilip [00:40:55]: I would think of Dynamo as less of a box system and more of a toolkit for building with. So when we talk about doing aware routing, when we talk about doing KV offloading, when we talk about doing, PD disaggregation, Dynamo fundamentally is. By the way, Dynamo is an open source library from NVIDIA.Ali [00:41:17]: We've done a pod with KylePhilip [00:41:18]: OkayAli [00:41:19]: Kyle Cranin.Philip [00:41:19]: Cool. So then your listeners know then that it supports all the different inference frameworks. And it is multi hardware, which is interesting.Ali [00:41:28]: But it's just a router, it's not like an optimizer layer.Philip [00:41:30]: Yeah. All it does, like, what Dynamo is good at, it is a library for moving information around your cluster, around your hardware. So if you have, KV cache on one place and you need it to be somewhere else, Dynamo coordinates NIXL for you to move that around.Philip [00:41:49]: That doesn't mean that, like, out of the box, you just say, “Pip install Dynamo,” and then you get, like, a massive performance speed up. It's more of a developer toolkit.Ali [00:42:01]: Yeah. I would have said it would. It comes with a set of defaults that you can then swap out.Philip [00:42:06]: It does. If the industry at large, I think, was, like, rolling out all of these deployments, standard, then I think it would be, like, a credible baseline. But, we've got to, we've got to benchmark against, like, what we're seeing in the wild.Speculative Decoding Methods: Medusa, EAGLE, n-Gram, and Spec-SpecVibhu [00:42:23]: I did wanna talk a little bit more about PD disagg, because that is probably, like, number three after quantized and speculative decoding. In your book though, I was just gonna pull out the book.Philip [00:42:31]: Yeah.Vibhu [00:42:32]: Like section 522 on Medusa, 523 on EAGLEPhilip [00:42:35]: YeahVibhu [00:42:36]: 524 on gram.Philip [00:42:37]: It's 55, would be disaggregationAli [00:42:42]: Yeah. Well, no, I just wanted to dwell a little bitPhilip [00:42:44]: YeahAli [00:42:44]: The other. Like, so what do you choose to include? What do you choose to not to include? Because there was all these other techniques.Philip [00:42:51]: Yeah.Ali [00:42:51]: Are these still relevant? Because I think they came out, like, a year and a half ago maybe.Vibhu [00:42:55]: Medusa is quite old.Philip [00:42:56]: Yeah, Medusa's old.Ali [00:42:58]: It was old.Vibhu [00:42:58]: But is it in the book as a good, here'sPhilip [00:43:01]: BaselineVibhu [00:43:01]: Baseline vanilla understand it?Philip [00:43:02]: Like you should know this.Vibhu [00:43:03]: Like I read the paper, I'm like, “ it makes so much sense.”Philip [00:43:05]: Yeah.Philip [00:43:05]: So with the book, I had a couple goals. One was to give people just a working vocabulary for the space as a whole, and the other was to give them some intuition about how each of these techniques works. As I mentioned in my AI Engineer talk, which is the first public addendum to this, the speculation space has moved much faster than everything else. So yeah, even at the time that I wrote the book Medusa, I very much included as a way for people to understand how the space evolved rather than what the most modern technique is. And now of course, there's DFlash, dSpark. There's, there's newer techniques even than EAGLE, although EAGLE is still very commonly used.Ali [00:43:51]: SpecSpecta.Philip [00:43:52]: Yes. Speculative decoding.Vibhu [00:43:54]: What canAli [00:43:56]: Oh, it's a paper by Tri Dao and it's like, it's doing speculative decodingVibhu [00:44:00]: HuhAli [00:44:01]: For the speculative decoder.Philip [00:44:02]: Oh, in spec- oh my God.Ali [00:44:02]: It's literally just an another. It's like, yeah, that's the most simple way to explain it, and it seems like he got trivial speed ups there. But it seems that the complexity with training, it's almost like in our mind at least, it's almost as complex as training GANs. Like it's like a very delicate balance and oftentimes you, it's just but yeah, it's literally speculative decoding on speculative decoding.Vibhu [00:44:21]: Speculative.Ali [00:44:22]: Yeah. We saw this paper.Vibhu [00:44:24]: It's interesting, right?Ali [00:44:24]: Yeah.Vibhu [00:44:24]: I wouldn't even expect it to be very particular to train, I wouldAli [00:44:29]: Right.Vibhu [00:44:29]: The naive part of me is like, okay, train speculative decoder.Ali [00:44:32]: But like, and it makes sense, like the whole idea of speculative decoding is you. It's like, it's like almost like the iPhone auto predict version but for a normal model, right? Like you're just, you're just, generating three tokens and you're like, okay, I'll do prefill on them. And so you save those three turns for your original model. Now your speculative decoder is doing three turns of auto regression, so why not just have an even smaller model?Ali [00:44:53]: The other question there is what are the size of speculators? So say forPhilip [00:44:58]: Right. It's like a billion parameters.Ali [00:45:01]: Like for MiniMax, it's. Yeah. It's like one layer. It's like one 60th of the original model usually.Philip [00:45:06]: Yeah. I think we should do a paper when we get back to the office.Philip [00:45:10]: SpeculativeAli [00:45:11]: SpeculativePhilip [00:45:11]: Decoding.Ali [00:45:13]: No, it's, it does seem like how, when do you stop? But then it also seems like if you're able to train spec-spec decode for instance, right? Like if you're able to have a small model that is accurately predicts what the intermediate speculator is gonna predict, that is able to predict what the original target model's gonna predict, then why not just use that smallest model directly, right?Vibhu [00:45:34]: Yeah. This isAli [00:45:35]: Like it seems likeVibhu [00:45:35]: Adjacent to the routing problem.Ali [00:45:36]: Right.Vibhu [00:45:36]: Yeah.Ali [00:45:36]: Right.Philip [00:45:37]: The thing with speculators is one of the practical constraints on using them is that you do have to run a small model on the same hardware that you're running the big model on. There is a orchestration and resource competition problem inherent in that, and that is one of the constraints on speculation in general, is that draft tokens cost resources to create and cost software complexity to manage. And so if you have like infinitely recursive speculators, you add in quite a bit of that complexity on the actual implementation within the inference engine as well, not just in the training process.Vibhu [00:46:17]: I was gonna say, I would wonder if you could do similar, like distillation and pruning of, it's the same thing, it's just a model. Can we not just distill a lot of the weights, quantize the speculator, out of my domain? The question that also comes up is, this is all for big server workloads, right? How much of this applies to, say I have this MacBook, I wanna run Gemma really efficiently. Similar problems, not the same?Local AI vs. Data Center InferencePhilip [00:46:45]: Pretty different. I talked to Selo, about this on his podcast a couple weeks ago. The difference between inference engineering for the data center and for production workloads versus inference engineering for local AI, is that we start with fundamentally like different constraints and different goals. With local AI, it's how do I fit this model onto my hardware and then make it less dumb? And with data center influence, it's how do I load this model and then make it less slow? And we care about less dumb, and they care about less slow. But the local AI inference engineering ecosystem, I think has a lot for us to learn from in the data center space. They are experts in various forms of quantization, including dynamic quantization that we just don't touch, in the pruning, in the distillation, in the, layer removal. There'Ali [00:47:42]: Layer removal matters less.Philip [00:47:43]: Yeah. There'Ali [00:47:44]: No one loves pruning really.Philip [00:47:45]: Yeah. Well, but the, but they doVibhu [00:47:46]: Which is surprising, right? But that's, that's a whole different thingPhilip [00:47:48]: Just to fit something on the laptop.Ali [00:47:50]: Right.Philip [00:47:50]: So yeah, it's a, it's an interesting, it's an interesting space. Not necessarily that like their techniques make sense for us to do in the data center, because we have different resources and different goals, but more that the process as well as the openness of that field is something to, admire.Ali [00:48:12]: Yeah. Like to your point, like, certain optimizations that would. Like for instance, Turbo Quantum Sharper, like it made such huge hype on that and we did like a whole deep dive on Twitter and like said, what is it? How does it work? Why is it good or not? And it took off and it was implemented on local devices because your memory bandwidth is so slow on like a MacBook, for instance. But try putting the same thing on like an NVIDIA GPU on a B200 Turbo quant would not be. Like, it would not be used. Like, NVIDIA - Like, NVIDIA made it clear that this is not a good optimization, and we've seen it firsthand where the overhead of doing dequantization, quantization of, in the kernel itself with turbo quant kernel, each end is much slower than the time that you save from doing the bandwidth. ‘Cause on the B200s, you have like 3.5 terabytes per second. You don't need decrease the storage that much. You don't need to do, FP4 KV cache. You don't need to use a requant. There's, there's, there's better optimizations to be made. But on Edge devices, it's extremely important, it's extremely useful. So, seems to be, like, different optimizations there, but then they're all uniquely combined with like all you wanna quantize the model, you wanna do speculative decoding, like certain common prefixes with bothPhilip [00:49:18]: Principles.Ali [00:49:19]: Yeah, exactly. Exactly. Exactly.Philip [00:49:20]: They also do a lot of work on, model parallelism, especially over, heterogeneous topology, where you have, some sparks and they are wired together with, Ethernet, DGX sparks.Ali [00:49:35]: Yeah, this is the Exo Labs guys.Philip [00:49:36]: Yeah. You have, a nu

Intel Chat with Matt Bromiley and Chris Luft.Matt and Chris break down four stories from the week in threat intel:• Hugging Face's security incident disclosure: an intrusion conducted end-to-end by an autonomous AI agent system — a malicious dataset exploiting two code-execution paths, thousands of actions across short-lived sandboxes, self-migrating C2 — and why the forensics had to run on the open-weight GLM 5.2 model after hosted frontier models refused to analyze real attack artifacts.• WP2Shell: attackers chaining CVE-2026-60137 (WordPress Core SQL injection) with CVE-2026-63030 (Batch REST API logic flaw) for unauthenticated remote code execution on default WordPress installs — found by Searchlight Cyber using GPT-5.6 Sol Ultra in about ten hours, with tens of thousands of exploitation attempts following disclosure.• Data breaches at AI music generator Suno (55.3M unique email addresses, plus partial Stripe payment records) and gig-work platform Paidwork (23.3M addresses, password hashes and banking data), per Have I Been Pwned.• Iranian state media claims the IRGC destroyed AWS's Bahrain data center (ME-SOUTH-1) with cruise missiles — and what data centers becoming military targets means for cloud resilience.Plus: Google Threat Intelligence Group retires APT/FIN nomenclature for new threat-actor names, and where to find Chris and Matt at Black Hat.Stories covered:• https://huggingface.co/blog/security-incident-july-2026• https://www.darkreading.com/cyberattacks-data-breaches/wp2shell-millions-wordpress-sites-remote-takeover• https://www.securityweek.com/suno-paidwork-data-breaches-affect-tens-of-millions-of-accounts/• https://www.tomshardware.com/tech-industry/data-centers/amazon-data-center-in-bahrain-struck-and-destroyed-by-iranian-cruise-missiles-state-media-claims-attacks-launched-against-aws-site-in-response-to-alleged-us-strikes-on-an-under-construction-nuclear-plantChapters:0:00 Intro & Black Hat plans2:07 Hugging Face's AI-agent breach disclosure12:39 WP2Shell: WordPress exploit chain20:59 Suno & Paidwork data breaches24:17 IRGC strikes on AWS Bahrain28:27 Google Threat Intel's new actor names29:29 Black Hat swag hunt & wrap-upThe Cybersecurity Defenders Podcast — a podcast about cybersecurity and the people that keep the internet safe. New episodes drop weekly.Subscribe wherever you listen:• Spotify: https://open.spotify.com/show/6ep00zeY3S8ffZ4o0UeSps• Apple Podcasts: https://podcasts.apple.com/us/podcast/the-cybersecurity-defenders-podcast/id1649981740• YouTube: https://www.youtube.com/@limacharlieioLearn more about LimaCharlie: https://limacharlie.io#cybersecurity #infosec #threatintel #AIsecurity #databreach

Intel Chat with Matt Bromiley and Chris Luft.Matt and Chris break down four stories from the week in threat intel:• Hugging Face's security incident disclosure: an intrusion conducted end-to-end by an autonomous AI agent system — a malicious dataset exploiting two code-execution paths, thousands of actions across short-lived sandboxes, self-migrating C2 — and why the forensics had to run on the open-weight GLM 5.2 model after hosted frontier models refused to analyze real attack artifacts.• WP2Shell: attackers chaining CVE-2026-60137 (WordPress Core SQL injection) with CVE-2026-63030 (Batch REST API logic flaw) for unauthenticated remote code execution on default WordPress installs — found by Searchlight Cyber using GPT-5.6 Sol Ultra in about ten hours, with tens of thousands of exploitation attempts following disclosure.• Data breaches at AI music generator Suno (55.3M unique email addresses, plus partial Stripe payment records) and gig-work platform Paidwork (23.3M addresses, password hashes and banking data), per Have I Been Pwned.• Iranian state media claims the IRGC destroyed AWS's Bahrain data center (ME-SOUTH-1) with cruise missiles — and what data centers becoming military targets means for cloud resilience.Plus: Google Threat Intelligence Group retires APT/FIN nomenclature for new threat-actor names, and where to find Chris and Matt at Black Hat.Stories covered:• https://huggingface.co/blog/security-incident-july-2026• https://www.darkreading.com/cyberattacks-data-breaches/wp2shell-millions-wordpress-sites-remote-takeover• https://www.securityweek.com/suno-paidwork-data-breaches-affect-tens-of-millions-of-accounts/• https://www.tomshardware.com/tech-industry/data-centers/amazon-data-center-in-bahrain-struck-and-destroyed-by-iranian-cruise-missiles-state-media-claims-attacks-launched-against-aws-site-in-response-to-alleged-us-strikes-on-an-under-construction-nuclear-plantChapters:0:00 Intro & Black Hat plans2:07 Hugging Face's AI-agent breach disclosure12:39 WP2Shell: WordPress exploit chain20:59 Suno & Paidwork data breaches24:17 IRGC strikes on AWS Bahrain28:27 Google Threat Intel's new actor names29:29 Black Hat swag hunt & wrap-upThe Cybersecurity Defenders Podcast — a podcast about cybersecurity and the people that keep the internet safe. New episodes drop weekly.Subscribe wherever you listen:• Spotify: https://open.spotify.com/show/6ep00zeY3S8ffZ4o0UeSps• Apple Podcasts: https://podcasts.apple.com/us/podcast/the-cybersecurity-defenders-podcast/id1649981740• YouTube: https://www.youtube.com/@limacharlieioLearn more about LimaCharlie: https://limacharlie.io#cybersecurity #infosec #threatintel #AIsecurity #databreach

Send us Fan MailThis week, Amith Nagarajan and Mallory Mejias unpack a whirlwind of AI developments shaping the future of associations. From Anthropic's release of a more efficient Claude Opus 5 with dynamic “effort dialing,” to a shocking autonomous AI cyberattack that breached Hugging Face, the conversation dives deep into what these moments mean for trust, safety, and strategy. They explore the growing divide over open vs. closed AI models, why major tech players are rallying around open weights, and how AI-powered cybersecurity is quickly becoming essential. Along the way, they break down practical use cases for choosing the right model and share what association leaders must do now to stay secure, informed, and ready for what's next. 

Every major AI lab signed the Open Weights letter defending open models. Meta, OpenAI, Google, Microsoft, Nvidia.Anthropic was the only holdout.Yesterday, its CEO, Dario Amodei, published a thoughtful defense of that decision to not fully support open weight or open source models. Here's what nobody's connecting: the money trail. Roughly 80% of Anthropic's revenue is businesses paying per token. Free Chinese open models attack that exact revenue stream weeks before Anthropic is set to go public. On today's show we break down what Dario actually said, what he said before, and why we think this was written for Washington policymakers and not for the rest of us.Anthropic Responds: Why Claude's CEO didn't sign the open model pact and the real reasons why -- An Everyday AI Chat with Jordan WilsonNewsletter: Sign up for our free daily newsletterMore on this Episode: Episode PageToday's Episode on LinkedIn: Thoughts on this? Join the convo on LinkedIn and connect with other AI leaders.Upcoming Episodes: Check out the upcoming Everyday AI Livestream lineupWebsite: YourEverydayAI.comEmail The Show: info@youreverydayai.comConnect with Jordan on LinkedInTopics Covered in This Episode:Anthropic Refuses Open Model PactDario Amodei's Public Letter AnalysisAnthropic's 80% Revenue Token ExposeChinese Open Model National Security FearsMicrosoft & Nvidia's Open Weights CoalitionRegulatory Capture and Washington InfluenceTiming Related to Executive Order DeadlineIPO Motivations Behind Anthropic's DecisionsContradictions in Anthropic's Open Model StanceImpact of Open Source on Token Business ModelTimestamps:00:00 Anthropic's stance on open models04:19 Discussing Anthropic's response to open models06:39 Understanding open weight models12:08 Future AI and cybersecurity risks15:04 Discussion on open-source AI models19:30 Discussing Anthropic's business challenges21:08 Cutting costs with open-source models26:17 Anthropic's recent stock downturn27:11 AI investment and cost efficiency shift30:14 Anthropic's stance on open source models36:32 INTROPICS IPO and regulatory discussions37:37 Wrapping up and subscribingKeywords: Anthropic, Claude, open model pact, open source AI, open weights, American AI leadership, Dario Amodei, IPO, regulatory capture, DC lawmakers, Chinese open source models, token revenue, per token business model, NVIDIA, Microsoft, Meta, OpenAI, Google, IBM, national security, AI safety, government mandates, chip controls, AI regulation, chip ban, industrial scale distillation, mandatory safety testing, inference, AI ecosystem, Opus 5, Fable 5, GPT-5, GLM 5.2, cost per task, token efficiency, model router, proprietary models, closed source AI, cybersecurity risks, Chinese cyberattacks, biological attacks, Glasswing program, open source vs proprietary, tech lobbying, Trump AI order, federal deadline, AI policy, artificial general intelligence, artificial superintelligence, AI monetization, S-1 filing, public company, venture capital, AI benchmarks, model switching, API pricing, model containment, Hugging Face incident, AI startup monopoly, safety vs business protection, market competition, AI cost reduction.Send Everyday AI and Jordan a text message. (We can't reply back unless you leave contact info) Ready for ROI on GenAI? Go to youreverydayai.com/partner 

Welcome to this week's episode of the PEBCAK Podcast!  We've got three amazing stories this week so sit back, relax, and keep being awesome!  Be sure to stick around for our Dad Joke of the Week. (DJOW) Follow us on Instagram @pebcakpodcast   Please share this podcast with someone you know!  It helps us grow the podcast and we really appreciate it!   Simple 6 signup link https://simple6.co/r/CFUR98   Hugging Face allegedly had to fall back on China's GLM 5.2 to investigate the OpenAI hack after US frontier models refused to help with forensic analysis, unable to distinguish attacker from defender. - https://x.com/coinbureau/status/2079663328021057654?s=46 - https://x.com/t3chfalcon/status/2079998873183949221 Per the claim, safety guardrails on US frontier models blocked forensic assistance during the incident response because the models couldn't tell the investigating security team apart from the attacker, forcing Hugging Face to run GLM 5.2 on its own servers to complete the analysis — a notable reversal given the assumption that domestic models would be the trusted fallback in a crisis.   VW drivers running GrapheneOS say the automaker's app has locked them out entirely, deepening fears carmakers are forcing users back into Google's ecosystem.  VW has now confirmed to German outlet heise that it deliberately blocked GrapheneOS, LineageOS, and /e/OS users via Google's Play Integrity API — the same certification gatekeeper already used by Barclays, HSBC, Monzo, Netflix, and Disney+. - https://cybernews.com/privacy/volkswagen-grapheneos-app-issues/ - https://x.com/intcyberdigest/status/2079992915972018246?s=46 - https://x.com/orwellday/status/2079704437241610443?s=46 Roughly 500,000 GrapheneOS users are affected; VW's app still supports outdated Android versions but rejects the privacy-focused OS, and VW told one user GrapheneOS "is not an official Volkswagen offering." The lockout follows a recent VW API change that also cut off third-party smart-charging and home-automation tools, raising questions about EU Data Act compliance.   A 19-year-old alleged Scattered Spider member's globe-trotting VPN op-sec got shredded by a Windows telemetry ID he probably didn't know existed. - https://cybersecuritynews.com/windows-device-identifier-tracking/ Peter Stokes (dual US-Estonian, 19) allegedly ran an $8M extortion hit on a luxury retailer using voice-phishing, ngrok tunneling, and 77GB of S3 exfil — but the FBI cross-referenced his Microsoft Global Device Identifier (GDID) across Apple, Snapchat, Facebook, and even a Ubisoft login to place the same device on the same IPs in Tallinn, NYC, and Thailand, matching his travel records. Microsoft has since acknowledged using the GDID to track the user across three countries — meaning the VPN masked his network endpoint, but the GDID rendered that protection moot.   Dad Joke of the Week (DJOW)   Find the hosts on LinkedIn: Chris - https://www.linkedin.com/in/chlouie/ Glenn - https://www.linkedin.com/in/glennmedina/ Jason - https://www.linkedin.com/in/jason-seemann-12b7075/

Ep 288 MultiSIM Apple — jedan broj na više uređaja, i bez iPhone-a uz sebe Apple sues OpenAI, alleging theft of trade secrets to fuel hardware ambitions - MacDailyNews Apple sues OpenAI, accuses ex-employees of stealing trade secrets - 9to5Mac Drew Pusateri: Our statement in response to this suit: We have no interest in other companies' trade secrets. We remain focused on building innovative technology that empowers people everywhere. Daring Fireball: ‘No Interest' OpenAI shrugs, denies responsibility for trade secret theft with vague statements The Joy of Tech: Apple VS Open AI! OpenAI's first hardware device will be a HomePod, but don't tell them that OpenAI Shuffles ChatGPT Apps, Kills Atlas Browser, Improves Voice - TidBITS Here's Why Apple is Reportedly Skipping M6 Pro and M6 Max Chips SigLens acquired by Apple for debugging massive apps and services Port-limited MacBook Neo gets color-matching hub & mouse The Real System Requirements for OS 27 - TidBITS Alek: promena preporučene mašine za iOS devs jer Xcode 27 satire disk prostor Apple Patches Hide My Email Flaw More Than a Year After It Was Reported CrashStealer malware masquerades as Apple's crash report tool to raid your Mac - Cult of Mac EU Orders Google to Give Rival AI Apps the Same Android Access as Gemini Daring Fireball: European Commission: ‘Guidance to Google for AI Interoperability on Android & Sharing of Google Search' EU slaps Google with a $1 billion antitrust fine Apple Watch, Meta Glasses, AirPods get reprieve from EU replaceable battery law Katie Paxton-Fear: Can we trust Chinese open weight models? Was a question a lot of people asked after GLM 5.2 was released, scoring very well on coding benchmarks, and suspiciously Claude-like. So I turned an open-weight coding model into a backdoor with 1hr and

AI Safety Basics, Exploit Gym, and a 438km Tesla FSD Trip — Plus Kimi K3 and Agentic Tools With John away, Jim and Marcel discuss recent AI security concerns, arguing that sensational claims about an OpenAI model "escaping" distract from basic safeguards like network segmentation and monitoring, especially when testing hacking capability via the Exploit Gym benchmark in a sandboxed, virtualized environment. They pivot to self-driving, with Marcel describing a new rear-wheel-drive Tesla Model Y that drove 438 km to Gananoque and back on supervised Full Self-Driving, including automated routing to Superchargers and self-parking, and they note winter and sensor considerations plus lower fueling costs. The conversation shifts to enterprise AI costs and Chinese models like Kimi K3, Qwen, GLM, and DeepSeek, emphasizing experimentation, data governance, and that huge models require data centers. They highlight agentic systems that can install and run software, memory tools like Mnemosyne, and reflect on how AI could free time for higher-level thinking. 00:00 John Is Away 00:47 Writing And AI Escapes 02:24 Movie Hacking Myths 04:45 Robot Ducks And Chicks 06:04 Exploit Gym Explained 10:36 Agents And Emergence 12:45 Security Lessons Learned 17:05 Self Driving In China 18:35 Tesla Road Trip FSD 25:45 Winter Range And Sensors 28:48 EV Range Reality Check 29:22 Charging Costs and Solar 30:17 Why CIOs Fear AI Costs 32:29 Chinese Models and Multimodal 33:21 Open Source Catch and Sandboxes 35:30 Picking the Right Model Mix 37:33 Innovation vs Reliability Lessons 41:18 Foreign Car Analogy for AI 45:41 What to Use and Pay For 47:33 Voice Mode Gets Real 49:40 Agents That Do the Work 54:20 Memory Plugins and Skill Atrophy 57:59 Habits, Choice, and Closing Thoughts

En este episodio del iSenaCode Live, repasamos todas las noticias más importantes de Apple y la inteligencia artificial que están marcando la actualidad tecnológica. Analizamos las Release Candidate de iOS 26.6 y iPadOS 26.6, las últimas filtraciones del iPhone 18 Pro con su nuevo chip A20 Pro fabricado en 2 nm y la tecnología WMCM, además de las mejoras de pantalla que prometen cambiar la experiencia de uso. También hablamos del espectacular iPhone del 20.º aniversario, que podría alcanzar las 7 pulgadas, y de cómo Apple ha vuelto a convertirse en la empresa más valiosa del mundo al superar a Nvidia en capitalización bursátil. Pero no todo son rumores. Comentamos la emocionante historia de Phil, cuya vida fue salvada gracias a un Apple Watch Series 11, y analizamos el informe de la Electronic Frontier Foundation (EFF) que sitúa al Apple Watch como el smartwatch que mejor protege la privacidad de los datos de salud de sus usuarios. Además, debatimos sobre cómo Netflix ya utiliza inteligencia artificial generativa en más de 300 producciones, el avance de los modelos de IA chinos como GLM, Claude y ChatGPT, y qué impacto tendrán estas tecnologías en el futuro del ecosistema Apple. Si te apasionan Apple, la inteligencia artificial y conocer antes que nadie los rumores y novedades que definirán los próximos años, este episodio es para ti.

A U.S. vs China AI cold war is starting, and most business leaders have no idea they're already in it.China's open models just closed the gap with America's best, oftentimes at a fraction of the price.Now both governments are moving to wall off their AI within days of each other.Why? Because this was never about benchmarks. It's about power y'all. We break it all down on today's show and help you figure out the 101 of the AI war between U.S. and China. The U.S. vs China AI Cold War Is Starting: What It Means and How It Impacts You -- An Everyday AI Chat with Jordan WilsonNewsletter: Sign up for our free daily newsletterMore on this Episode: Episode PageToday's Episode on LinkedIn: Thoughts on this? Join the convo on LinkedIn and connect with other AI leaders.Upcoming Episodes: Check out the upcoming Everyday AI Livestream lineupWebsite: YourEverydayAI.comEmail The Show: info@youreverydayai.comConnect with Jordan on LinkedInTopics Covered in This Episode:U.S.-China AI Cold War OverviewChinese AI Models Closing U.S. GapGovernment Restrictions on AI Model AccessEconomic and Geopolitical AI Power StruggleRisks for U.S. Businesses Using Chinese AIOpen Source vs. Closed Source AI DebateChinese AI Model Pricing Undercuts U.S.AI Model Distillation and U.S. Security ConcernsEnterprise AI Cost-Effectiveness BenchmarksMicrosoft Testing Chinese AI DeploymentsFuture AI Model Export Controls & StrategiesRecommendations for AI Model Sourcing and RiskTimestamps:00:00 US-China AI tensions escalate04:30 Switching to Chinese AI models08:47 US vs China in open source models11:39 China's narrative control efforts14:42 Challenges in AI model development18:25 Differentiating open source strategies23:04 AI model cost-effectiveness analysis26:31 US measures against model distillation29:38 Discussing Microsoft's use of AI models31:17 Controlling export of AI modelsKeywords: US vs China AI cold war, China AI restrictions, US AI restrictions, AI model export controls, Chinese open source AI models, AI geopolitical power, economic growth through AI, global AI standards, AI superpower race, AI model benchmarks, open weight models, enterprise AI deployment, trillion parameter AI models, Microsoft AI model testing, AI model pricing, Claude Fable 5, GPT-5.6, GLM 5.2, Kimmi K3, Alibaba Qwen 3.8, model distillation, AI cybersecurity risks, AGI leadership, military AI use cases, China narrative control, model adoption, compute power for AI, AI training data, AI export law, US national security and AI, model routing, mixture of models, cost per intelligence index, Anthropic models, cost per task AI, model capability parity, AI market adoption, cloud competition, AI architecture innovation, AI model sanctionsSend Everyday AI and Jordan a text message. (We can't reply back unless you leave contact info) Ready for ROI on GenAI? Go to youreverydayai.com/partner 

Factory started building fully autonomous coding agents in April 2023, two years before enterprises were ready. Matan Grinberg now says this is indistinguishable from being wrong. The Factory co-founder and CEO explains how the company survived its "journey in the desert," including the decision to hand nearly all of its revenue back to customers when the product wasn't making developers obsessed. Matan makes the contrarian technical case that a model-agnostic harness beats the model-and-harness co-design that labs like OpenAI and Anthropic favor, because exposing a harness to many models keeps it from overfitting to any single one. He argues open-weight models like GLM will capture the majority of tokens by staying one generation behind the frontier at a fraction of the cost, and that CIOs will soon justify every incremental token the way they justify headcount. Looking ahead, he predicts 90% of coding tokens will run asynchronously—the "dark factory" where software builds itself. Hosted by Sonya Huang and Pat Grady, Sequoia Capital

What happens when China drops open-weight AI models that rival Silicon Valley's best? This episode unpacks how a new wave of international AI releases is shaking up business, policy, and the future of innovation. Linus Torvalds to critics of AI coding in Linux: "Fork it. Or just walk away." Claude on X: "Beginning July 20, Claude Fable 5 will be included in all Max and Team Premium plans, at 50% of limits. Pro and Team Standard users will continue to have access to Fable via usage credits, and will receive a one-time $100 credit. Demand for Fable has been challenging to" China's Moonshot AI Unveils Kimi Model, Threatening America's Lead Alibaba's Qwen Unveils Preview of Flagship AI Model Social media limits are coming for teens across Europe The White House is now deciding who gets access to frontier AI models, not the labs Microsoft chief turns hostile on frontier AI labs, warns companies to guard their IP Meta Is Flooding the Market With Smartglasses. Privacy Advocates Are Up in Arms. Federal employees can download TikTok on government devices, DOJ says Amazon Web Services customers receive bills for up to $1.5tn after global glitch MLB cracks down on using AI via dugout iPads to help shape in-game decisions White House Teleprompter Operator Bet on Trump Speeches, Kalshi Says New York school district is testing lifelike robot teachers Host: Leo Laporte Guests: Harper Reed and Alex Wilhelm Download or subscribe to This Week in Tech at https://twit.tv/shows/this-week-in-tech Join Club TWiT for Ad-Free Podcasts! Support what you love and get ad-free audio and video feeds, a members-only Discord, and exclusive content. Join today: https://twit.tv/clubtwit Sponsors: blackhat.com/us-26 and use code TWIT ZipRecruiter.com/twit ethos.com/twit arcticwolf.com/trends threatlocker.com/twit shopify.com/twit

Alibaba launched Qwen3.8 Max as Moonshot paused Kimi K3 signups amid demand. The Trump administration weighed a slow squeeze on Chinese AI, Hugging Face used China's GLM-5.2 after US guardrails blocked its breach forensics, and Google built a Gemini chip. Alibaba launches a 2.4T parameter Qwen3.8 Max preview that it says rivals frontier AI models and is second only to Fable 5, plans to make it "open-weight soon" (Bloomberg) The Trump administration has reportedly explored sanctions, security warnings, and executive-order requirements since 2025 to build a slow, durable squeeze on Chinese AI models instead of pursuing an outright ban (The Decoder) Ben Thompson argues the market reaction to Kimi and other Chinese models is overblown, since it's compute scarcity — not a real Chinese cost advantage — that's keeping frontier-model prices high (Stratechery) Hugging Face says it used the open-weight GLM-5.2 hosted on its own compute for breach forensics, after US frontier model safety guardrails blocked the requests (The Stack) Sources: Google is developing a specialized server chip, informally dubbed "Frozen v2", that integrates its Gemini AI model blueprint into the silicon, for 2028 (The Information) Subscribe to the ad-free feed. Learn more about your ad choices. Visit megaphone.fm/adchoices

What happens when China drops open-weight AI models that rival Silicon Valley's best? This episode unpacks how a new wave of international AI releases is shaking up business, policy, and the future of innovation. Linus Torvalds to critics of AI coding in Linux: "Fork it. Or just walk away." Claude on X: "Beginning July 20, Claude Fable 5 will be included in all Max and Team Premium plans, at 50% of limits. Pro and Team Standard users will continue to have access to Fable via usage credits, and will receive a one-time $100 credit. Demand for Fable has been challenging to" China's Moonshot AI Unveils Kimi Model, Threatening America's Lead Alibaba's Qwen Unveils Preview of Flagship AI Model Social media limits are coming for teens across Europe The White House is now deciding who gets access to frontier AI models, not the labs Microsoft chief turns hostile on frontier AI labs, warns companies to guard their IP Meta Is Flooding the Market With Smartglasses. Privacy Advocates Are Up in Arms. Federal employees can download TikTok on government devices, DOJ says Amazon Web Services customers receive bills for up to $1.5tn after global glitch MLB cracks down on using AI via dugout iPads to help shape in-game decisions White House Teleprompter Operator Bet on Trump Speeches, Kalshi Says New York school district is testing lifelike robot teachers Host: Leo Laporte Guests: Harper Reed and Alex Wilhelm Download or subscribe to This Week in Tech at https://twit.tv/shows/this-week-in-tech Join Club TWiT for Ad-Free Podcasts! Support what you love and get ad-free audio and video feeds, a members-only Discord, and exclusive content. Join today: https://twit.tv/clubtwit Sponsors: blackhat.com/us-26 and use code TWIT ZipRecruiter.com/twit ethos.com/twit arcticwolf.com/trends threatlocker.com/twit shopify.com/twit

What happens when China drops open-weight AI models that rival Silicon Valley's best? This episode unpacks how a new wave of international AI releases is shaking up business, policy, and the future of innovation. Linus Torvalds to critics of AI coding in Linux: "Fork it. Or just walk away." Claude on X: "Beginning July 20, Claude Fable 5 will be included in all Max and Team Premium plans, at 50% of limits. Pro and Team Standard users will continue to have access to Fable via usage credits, and will receive a one-time $100 credit. Demand for Fable has been challenging to" China's Moonshot AI Unveils Kimi Model, Threatening America's Lead Alibaba's Qwen Unveils Preview of Flagship AI Model Social media limits are coming for teens across Europe The White House is now deciding who gets access to frontier AI models, not the labs Microsoft chief turns hostile on frontier AI labs, warns companies to guard their IP Meta Is Flooding the Market With Smartglasses. Privacy Advocates Are Up in Arms. Federal employees can download TikTok on government devices, DOJ says Amazon Web Services customers receive bills for up to $1.5tn after global glitch MLB cracks down on using AI via dugout iPads to help shape in-game decisions White House Teleprompter Operator Bet on Trump Speeches, Kalshi Says New York school district is testing lifelike robot teachers Host: Leo Laporte Guests: Harper Reed and Alex Wilhelm Download or subscribe to This Week in Tech at https://twit.tv/shows/this-week-in-tech Join Club TWiT for Ad-Free Podcasts! Support what you love and get ad-free audio and video feeds, a members-only Discord, and exclusive content. Join today: https://twit.tv/clubtwit Sponsors: blackhat.com/us-26 and use code TWIT ZipRecruiter.com/twit ethos.com/twit arcticwolf.com/trends threatlocker.com/twit shopify.com/twit

What happens when China drops open-weight AI models that rival Silicon Valley's best? This episode unpacks how a new wave of international AI releases is shaking up business, policy, and the future of innovation. Linus Torvalds to critics of AI coding in Linux: "Fork it. Or just walk away." Claude on X: "Beginning July 20, Claude Fable 5 will be included in all Max and Team Premium plans, at 50% of limits. Pro and Team Standard users will continue to have access to Fable via usage credits, and will receive a one-time $100 credit. Demand for Fable has been challenging to" China's Moonshot AI Unveils Kimi Model, Threatening America's Lead Alibaba's Qwen Unveils Preview of Flagship AI Model Social media limits are coming for teens across Europe The White House is now deciding who gets access to frontier AI models, not the labs Microsoft chief turns hostile on frontier AI labs, warns companies to guard their IP Meta Is Flooding the Market With Smartglasses. Privacy Advocates Are Up in Arms. Federal employees can download TikTok on government devices, DOJ says Amazon Web Services customers receive bills for up to $1.5tn after global glitch MLB cracks down on using AI via dugout iPads to help shape in-game decisions White House Teleprompter Operator Bet on Trump Speeches, Kalshi Says New York school district is testing lifelike robot teachers Host: Leo Laporte Guests: Harper Reed and Alex Wilhelm Download or subscribe to This Week in Tech at https://twit.tv/shows/this-week-in-tech Join Club TWiT for Ad-Free Podcasts! Support what you love and get ad-free audio and video feeds, a members-only Discord, and exclusive content. Join today: https://twit.tv/clubtwit Sponsors: blackhat.com/us-26 and use code TWIT ZipRecruiter.com/twit ethos.com/twit arcticwolf.com/trends threatlocker.com/twit shopify.com/twit

What happens when China drops open-weight AI models that rival Silicon Valley's best? This episode unpacks how a new wave of international AI releases is shaking up business, policy, and the future of innovation. Linus Torvalds to critics of AI coding in Linux: "Fork it. Or just walk away." Claude on X: "Beginning July 20, Claude Fable 5 will be included in all Max and Team Premium plans, at 50% of limits. Pro and Team Standard users will continue to have access to Fable via usage credits, and will receive a one-time $100 credit. Demand for Fable has been challenging to" China's Moonshot AI Unveils Kimi Model, Threatening America's Lead Alibaba's Qwen Unveils Preview of Flagship AI Model Social media limits are coming for teens across Europe The White House is now deciding who gets access to frontier AI models, not the labs Microsoft chief turns hostile on frontier AI labs, warns companies to guard their IP Meta Is Flooding the Market With Smartglasses. Privacy Advocates Are Up in Arms. Federal employees can download TikTok on government devices, DOJ says Amazon Web Services customers receive bills for up to $1.5tn after global glitch MLB cracks down on using AI via dugout iPads to help shape in-game decisions White House Teleprompter Operator Bet on Trump Speeches, Kalshi Says New York school district is testing lifelike robot teachers Host: Leo Laporte Guests: Harper Reed and Alex Wilhelm Download or subscribe to This Week in Tech at https://twit.tv/shows/this-week-in-tech Join Club TWiT for Ad-Free Podcasts! Support what you love and get ad-free audio and video feeds, a members-only Discord, and exclusive content. Join today: https://twit.tv/clubtwit Sponsors: blackhat.com/us-26 and use code TWIT ZipRecruiter.com/twit ethos.com/twit arcticwolf.com/trends threatlocker.com/twit shopify.com/twit

(Presented by Thinkst Canary: Most Companies find out way too late that they've been breached. Thinkst Canary changes this. Deploy Canaries and Canarytokens in minutes and then forget about them. Attackers tip their hand by touching 'em giving you the one alert, when it matters. With zero admin overhead and almost no false-positives, Canaries are deployed (and loved) on all 7 continents.) Three Buddy Problem - Episode 105: We discuss a fascinating Hugging Face breach, where an autonomous AI agent broke out of the sandboxes, moved laterally through production, and generated 17,000 alerts before anyone caught it, and how frontier model guardrails locked the defenders out of their own investigation. Plus, China's big AI showcase, Xi's pitch for open models and global distribution, a record 622-CVE Microsoft Patch Tuesday, and 13 years of dwell time in the Daxin backdoor. Cast: Juan Andres Guerrero-Saade, Ryan Naraine and Costin Raiu. Timestamps: 0:00 Introductory banter 3:51 Hugging Face discloses end-to-end agentic hack 9:42 Why Hugging Face couldn't use frontier models 13:28 AI guardrails hampering defenders 16:22 Codex vs Claude for real malware work 23:43 Flash attacks vs. going low and slow 30:27 Was it targeted, or did Hugging Face pwn itself? 38:11 Long-horizon coherence: what GLM 5.2 still can't do 41:27 Kimi K3 leapfrogs, and Xi's AI speech 52:15 Exceptionalism vs. distribution 1:11:05 Gold Eagle: the White House vulnerability clearinghouse 1:15:05 Microsoft patches 622 CVEs — a record 1:20:29 APT corner: Daxin resurfaces after 13 years of dwell time 1:29:45 Balochistan police, and Microsoft's attribution-free wiper 1:34:26 Denis Obrezkov, leaked Kaspersky records, and the wrong questions 1:46:01 Magnet Forensics sues over a burned iPhone bug 1:57:57 Shout-outs

You know we're deep into summer when Jess leaves the pod early to catch Benson Boone... for the eighth time. This week, the More or Less squad revisits the Anthropic and OpenAI IPO speculation. Their views haven't changed, but their AI usage certainly has. Dave ditches Anthropic for GLM 5.2 on cost, while Sam argues the AI model wars are effectively over, and the frontier labs know it, which is why they're spending more time in Washington than competing on model quality. America also gets its first serious open weights model. The squad also unpacks OpenAI's hardware ambitions after its much-hyped device turns out to be... an Alexa speaker. Sam explains why Sam Altman's $8 billion Johnny Ive acquisition was effectively free, thanks to narrative capitalism. Plus: real-time voice AI, passive listening devices, peptides, Hinge's new social proof feature, Jay-Z turning New York into a festival, and why biological age tests are basically the clout score of your body.Chapters:0:00 Episode Teaser0:44 Episode Start2:18 Anthropic IPO This September?5:29 Dave Dumps Anthropic for GLM 5.28:22 When Is Better AI Worth Paying For?12:15 AGI Is Dead, AI Is Political Now16:15 America's First Open Weights Model By Thinking Machines21:22 Why Sam Is Depressed About AI29:07 OpenAI Built... an Alexa? The $8B Johnny Ive Bet31:38 Silicon Valley Goes Hollywood36:03 Would You Wear an Always-On Mic?37:08 OpenAI's Live Voice Model41:14 Sam's Peptide Experiment43:36 Hinge Launches “Friend's Take” 46:11 Jay-Z's NYC Takeover48:16 Biological Age Is Just a Clout ScoreWe're also on ↓X: https://twitter.com/moreorlesspodInstagram: https://instagram.com/moreorlessSpotify: https://podcasters.spotify.com/pod/show/moreorlesspodConnect with us here:1) Sam Lessin: https://x.com/lessin2) Dave Morin: https://x.com/davemorin3) Jessica Lessin: https://x.com/Jessicalessin4) Brit Morin: https://x.com/brit

The episode highlights a shift from technology selection to operational risk management in the AI landscape for MSPs. Service providers are being forced to navigate the fast-changing interplay between AI models, the harness software that mediates their deployment, and the financial realities of consumption-based billing. The rapid proliferation of open-source and open-weight AI models, alongside market behaviors from closed vendors and regulatory interventions, is introducing volatility and uncertainty in both cost structures and client offerings. This dynamic creates structural challenges related to margin maintenance, vendor dependency, and responsibility for AI-driven decisions. The discussion cites the release of GLM 5.2, an open-weight model from Z AI, which now rivals expensive closed models on key benchmarks at a fraction of the cost. At the same time, large-scale investments by commercial AI vendors have yet to deliver returns on expectations, with reports indicating businesses that adopted AI are not seeing projected value. Specific attention is given to operational constraints such as compute scarcity, token consumption variability, and export policy restrictions impacting AI availability. The episode notes that these pressures are driving both vendors and MSPs to reconsider the viability of reliance on expensive, closed offerings versus investigating open alternatives. Supportive examples include the proliferation of AI “harnesses” (middleware layers like Perplexity, Claude Code, and Cowork) that sit between service providers and underlying AI models, increasing both choice and complexity. Token billing models are highlighted as a source of unpredictability for MSPs, with vendors like Atera and ConnectWise experimenting with different abstractions to shield or pass through token risk to service providers. The potential for on-premises AI deployments using smaller language models is discussed as a cost-mitigation strategy, though this raises further questions about data privacy, infrastructure burden, and long-term vendor roles. Additionally, uncertainty is flagged around sustainability of leading vendors, with projections that at least one major AI player may exit or be acquired within a year due to financial vulnerability. For MSPs and IT service leaders, these structural and supporting developments translate into increased operational and financial complexity. There is a pressing need to evaluate not just which AI technologies to adopt, but how to architect solutions that can withstand rapid vendor movement, cost swings, and evolving regulatory requirements. Practical safeguards include testing open-source AI models alongside commercial offerings, exercising caution in vendor selection, and closely monitoring evolving consumption billing models. Preparing staff and clients for adaptive, process-oriented approaches—rather than fixed solutions—is positioned as a necessary step to maintain resilience as the AI adoption cycle continues to correct course. Supported by:Pax8CometBackupGuardz

週一天下零時差關注以下財經大事： 一、台積第二季法說會登場，營收、獲利率與股價是否有機會再創高？ 二、智譜GLM-5.2大模型震撼矽谷後，本週上海世界人工智能大會將透露中國AI最新進展。 三、美國聯準會主席沃許上任後首次出席國會聽證會，會透露哪些政策訊號？ 文：蔡娪嫣 製作團隊：錢玉紘、鄭子鴻 ＊閱讀零時差，點這看全文

Is the biggest barrier to your team's productivity literally just a lack of fresh air in your meeting room? This week on the Friday Deploy, Ben and Andrew dive into the rise of highly capable open source models like GLM 5.2 and the messy reality of running local AI for coding tasks. The hosts also discuss the cultural shift away from deep reading in a world obsessed with AI summaries, emphasizing the importance of protecting your first brain. Finally, they review a legendary tale from Meta's engineering history.Life Beyond Tokenmaxxing Workshop: Watch the full replay on demand at linearb.io Follow the show:Subscribe to our Substack Follow us on LinkedInSubscribe to our YouTube ChannelLeave us a ReviewFollow the hosts:Follow AndrewFollow BenFollow DanFollow today's stories:The bottleneck might be the air in the roomGLM-5.2 is the step change for open agentsViability of local models for codingAI Erodes a Legacy of ReadingI Shipped a Facebook Feature So Fast Sheryl Sandberg Called an Emergency Meeting to Stop MeOFFERSStart Free Trial: Get started with LinearB's AI productivity platform for free.Book a Demo: Learn how you can ship faster, improve DevEx, and lead with confidence in the AI era.LEARN ABOUT LINEARBAI Code Reviews: Automate reviews to catch bugs, security risks, and performance issues before they hit production.AI & Productivity Insights: Go beyond DORA with AI-powered recommendations and dashboards to measure and improve performance.AI-Powered Workflow Automations: Use AI-generated PR descriptions, smart routing, and other automations to reduce developer toil.MCP Server: Interact with your engineering data using natural language to build custom reports and get answers on the fly.

Alice Han and James Kynge start with China's latest ballistic missile test into the Pacific and what it means alongside a new Australia-Fiji defense pact. Then: Europe wants to shrink its record trade deficit with China, but its worst heat wave on record has sent demand for Chinese air conditioners soaring. They break down whether the two sides can actually cooperate on AI and renewable energy even as tensions rise. Plus: Z.ai just launched ZCode, a coding agent for its GLM-5.2 model said to rival Claude and ChatGPT. Alice and James discuss how big a threat this is to U.S. AI dominance, as well as the fallout from claims that Anthropic used hidden code to track Chinese users. Finally: China's new "Ethnic Unity" law took effect July 1. What does it mean for Tibetans, Uyghurs, and Taiwan… and how is Beijing defending it internationally? Subscribe to China Decode on Substack for weekly analysis, livestreams, and deep dives into the biggest story shaping the global economy: chinadecode.profgmedia.com Learn more about your ad choices. Visit podcastchoices.com/adchoices

AI's all-you-can-eat era is ending.

Google fires the engineer behind its Workspace CLI tool, OpenAI previews GPT-5.6 with three new model tiers, and Astro 7 lands with a full Rust rewrite. Plus: Coinbase cuts token costs with smarter routing, and more in this week's Syntax Live Show Notes 00:00 Intro 00:34 Welcome to Syntax! 01:46 Google fires Workspace CLI Creator 12:30 GPT 5.6 Is Coming 19:59 GLM 5.2 Released 23:23 Astro 7 Rust Re-write 32:46 Cursor Announces iOS App 35:08 Scott's Workflow: Herdr + Mosh + Termius + Tailscale 40:33 Coinbase Reduces AI Cost with Model Routing 44:22 wayfinder-router - Local AI Routing CLI 48:16 Token efficiency in models and harnesses Martin Woodward on X 52:34 performativeUI - react components for AI startups 54:31 Brought to you by Sentry.io 55:21 Reachy Mini Robot 01:02:21 FUTO Keyboard Swipe for Android 01:05:40 CSS Quake 01:07:35 HTML Invoker API is Baseline Available 01:13:17 Cloudflare Temporary Accounts for AI Agents Hit us up on Socials! Syntax: X Instagram Tiktok LinkedIn Threads Wes: X Instagram Tiktok LinkedIn Threads Scott: X Instagram Tiktok LinkedIn Threads Randy: X Instagram YouTube Threads

OpenAI floated giving the US government a 5% stake to court the Trump administration. Nvidia promised to backstop cloud providers for a revenue cut, SpaceX showed investors an xAI phone prototype, Apple ramped foldable iPhone orders, and Z.ai launched ZCode. Sources: OpenAI has discussed giving a 5% stake to the US government, seeking to clear political obstacles by securing buy-in from the Trump administration (FT) Nvidia promises to financially backstop young cloud providers like Firmus that rent out its AI chips, in exchange for a revenue share through a new program (The Information) Sources: SpaceX showed investors a handset-like device prototype with AI tech from xAI, a proprietary OS, a Snapdragon chip, and a design slimmer than an iPhone (WSJ) Sources: Apple has told suppliers to prepare to produce ~10M foldable iPhones in 2026, up from 7M to 8M previously, and 80M iPhones in total across new models (Nikkei Asia) Apple reportedly orders 10M foldable iPhone Ultra models, which could sell for around $2500 (9to5Mac) Sources: Apple is in negotiations to buy chips from CXMT and YMTC, two Chinese semiconductor makers on a Pentagon blacklist, for use in devices sold in China (Bloomberg) Z.ai launches ZCode, an "Agentic Development Environment" optimized for its new GLM-5.2 model; Z.ai's GLM Coding Plan costs from $16.20 to $144 per month (VentureBeat) Subscribe to the ad-free feed. Learn more about your ad choices. Visit megaphone.fm/adchoices

This Week In Startups is made possible by: CLA - www.claconnect.com/withyou Northwest Registered Agent - www.northwestregisteredagent.com/twist Agree.com - www.agree.com Today's show: Forget the triple-triple-double-double-double; the new bar for startups hoping to raise venture capital has reached the stratosphere, though our venture panel is worried that startups are focusing too much on today's problems that may not become companies tomorrow. During a lively VC roundtable, Cowboy's Aileen Lee, Floodgate's Mike Maples, and Lerer Hippeau's Ben Lerer joined Alex to dig into exiting pre-AI startups, rising valuations, token spend, why they are keeping their funds small, and whether the government just tripped OpenAI and Anthropic! Guest Links: Aileen Lee https://x.com/aileenlee Cowboy VC https://cowboy.vc Mike Maples https://x.com/m2jr Floodgate https://www.floodgate.com Bene Lerer https://www.linkedin.com/in/benjlerer Lere Hippeau https://www.lererhippeau.com Timestamps: 0:00 Aileen Lee, Mike Maples, and Ben Lerer join the show 5:13 Venture liquidity returns: what SpaceX/Stripe distributions mean for LPs 9:13 Bending Spoons prices IPO at $29/share, roughly $18.4B valuation 10:31 Agree.com - Stop chasing invoices and automate your entire contract-to-cash stack. Go to https://agree.com and tell them Jason sent you to get 50% off for life! 12:09 "Companies get bought, not sold" — Ben on taking first offers seriously 17:41 Plaud: If your work depends on conversations — interviews, meetings, calls — you need a Plaud NotePin. You can check it out at https://Plaud.ai/twist and use code TWIST for 10% off! 19:34 Mutiny's burn-the-boats AI pivot with Jaleh Rezaei 20:19 Northwest Registered Agent - Get more when you start your business with Northwest. In 10 clicks and 10 minutes, you can form your company and walk away with a real business identity — Learn more at https://www.northwestregisteredagent.com/twist 22:15 Mike's KeepSafe story: the "rule of 70" and profit-first companies 25:12 The new growth bar: 5x, 4x replaces triple-triple-double-double 28:55 Fund size is your strategy: why Floodgate and Lerer Hippeau stay small 30:11 CLA - Innovation takes balance. CLA's CPAs, consultants, and wealth advisors can help you get from startup to where you want to end up. Get started now at https://www.claconnect.com/withyou 37:39 The $100M Series A: Starcloud, General Intuition, Scale Cognition, Scout AI 40:35 King-making rounds and why mega-seeds destroy optionality 54:11 Open-weight models: the GLM-5.2 moment and going model-agnostic 55:30 Why fine-tuning open models is a treadmill, with Cursor/Kimi as an example 1:03:28 Grading the Trump administration on Mythos and Fable 1:06:33 Rising anti-AI sentiment, the wealth gap, and lessons from social media 1:11:43 Raising kids in the post-intelligence era 1:12:35 Where to find the panel and what each firm is investing in Subscribe to the TWiST500 newsletter: https://ticker.thisweekinstartups.com Check out the TWIST500: https://www.twist500.com Subscribe to This Week in Startups on Apple: https://rb.gy/v19fcp Follow Lon: X: https://x.com/lons Follow Alex: X: https://x.com/alex LinkedIn: ⁠https://www.linkedin.com/in/alexwilhelm Follow Jason: X: https://twitter.com/Jason LinkedIn: https://www.linkedin.com/in/jasoncalacanis Check out all our partner offers: https://partners.launch.co/ Great TWIST interviews: Will Guidara, Eoghan McCabe, Steve Huffman, Brian Chesky, Bob Moesta, Aaron Levie, Sophia Amoruso, Reid Hoffman, Frank Slootman, Billy McFarland Check out Jason's suite of newsletters: https://substack.com/@calacanis Follow TWiST: Twitter: https://twitter.com/TWiStartups YouTube: https://www.youtube.com/thisweekin Instagram: https://www.instagram.com/thisweekinstartups TikTok: https://www.tiktok.com/@thisweekinstartups Substack: https://twistartups.substack.com

Build Your First AI Employee (Free Workflow + Prompt): https://clickhubspot.com/rkvr Ep. 433 What if you could have an AI employee working right alongside you, automating key business workflows? Kieran and guest Nick Vasilescu (Co-founder of Orgo) dive into how agents are the next big form of business leverage.  Learn more on what it actually takes to get a functional AI agent up and running, how new connector tools like Composio make integrations simple, and why giving your agent a second brain with an Obsidian Vault is a game-changer for business productivity. Mentions Nick Vasilescu https://www.nickvasilescu.com/ Orgo https://www.orgo.ai/ Hermes Agent https://hermes-agent.nousresearch.com/ Composio https://composio.dev/ OpenClaw https://openclaw.ai/ GLM 5.2 https://z.ai/blog/glm-5.2 Obsidian https://obsidian.md/ Get our guide to build your own Custom GPT: https://clickhubspot.com/customgpt Resource [Free] Steal our favorite AI Prompts featured on the show! Grab them here: https://clickhubspot.com/aip We're on Social Media! Follow us for everyday marketing wisdom straight to your feed YouTube: ​​https://www.youtube.com/channel/UCGtXqPiNV8YC0GMUzY-EUFg  Twitter: https://twitter.com/matgpod  TikTok: https://www.tiktok.com/@matgpod  Thank you for tuning into Marketing Against The Grain! Don't forget to hit subscribe and follow us on Apple Podcasts (so you never miss an episode)! https://podcasts.apple.com/us/podcast/marketing-against-the-grain/id1616700934   If you love this show, please leave us a 5-Star Review https://link.chtbl.com/h9_sjBKH and share your favorite episodes with friends. We really appreciate your support. Host Links: Kipp Bodnar, https://twitter.com/kippbodnar   Kieran Flanagan, https://twitter.com/searchbrat  ‘Marketing Against The Grain' is a HubSpot Original Podcast // Brought to you by Hubspot Media // Produced by Darren Clarke.

Good news: OpenAI's GPT-5.6 has been released!

The US lifted its block on Anthropic's Mythos 5, clearing it for 100+ institutions. Researchers said China's GLM-5.2 matches US models on security bugs. South Korea pledged ~$590B for chips, and the memory crunch turned existential for small makers. Letter: the US lifts its block on Mythos 5, allowing Anthropic to release it to more than 100 US institutions; sources: talks about Fable 5 are ongoing (Semafor) Researchers say Z.ai's GLM-5.2 matches latest US models at finding security bugs, as critics question the US' lax approach in restricting Chinese open models (WSJ) South Korea, Samsung, and SK Hynix say they plan to invest ~$590B to build a new chip complex, including four chipmaking plants and a chip packaging cluster (FT) Soaring memory costs are posing existential threats to small electronics makers, amid thin margins, low supply chain leverage, and little room for price hikes (CNBC) Sports clips' rise on platforms like YouTube has left broadcasters debating whether to use them to attract younger viewers or protect their subscription revenue (CNBC) Subscribe to the ad-free feed. Learn more about your ad choices. Visit megaphone.fm/adchoices

OpenAI has released GPT-5.6, but the majority of us will have to wait. ⌚After the Anthropic vs. U.S. Government feud, it now looks like we'll have to wait for frontier models. That wasn't the only big AI news headline that might change your company's AI strategy. Anthropic got the green light to roll out Mythos 5 to a select few, Google reportedly extended its strike team to catch up on coding and more. OpenAI's limited release of GPT-5.6, Mythos starts slow reinstatement, OpenAI gets spicy and more AI news -- An Everyday AI chat with Jordan WilsonNewsletter: Sign up for our free daily newsletterMore on this Episode: Episode PageToday's Episode on LinkedIn: Thoughts on this? Join the convo on LinkedIn and connect with other AI leaders.Upcoming Episodes: Check out the upcoming Everyday AI Livestream lineupWebsite: YourEverydayAI.comEmail The Show: info@youreverydayai.comConnect with Jordan on LinkedInTopics Covered in This Episode:OpenAI GPT-5.6 Limited Release ExplainedOpenAI Sol, Terra, Luna Model NamingUS Government Restrictions on AI RolloutsAnthropic Mythos 5 Access and StandoffAnthropic Fable 5 Suspension DetailsGoogle Gemini 3.5 Pro Release DelayedGoogle's AI Coding Mid-Training InitiativeRaiseUS Nonprofit: AI Workforce AdaptationAnthropic Accuses Alibaba of Model DistillationOpenAI & Broadcom Unveil Jalapeno AI ChipKey AI Industry Partnerships & Product LeaksTimestamps:00:00 OpenAI's GPT 5.6 limited release06:14 OpenAI's new model release details09:54 Access suspension and negotiations13:18 Google's AI strategy and delays16:33 Anticipating Gemini 3.5 Pro Release20:40 Accusations of AI model theft24:48 OpenAI and Broadcom chip partnership28:05 OpenAI's recent developments and updates29:56 OpenAI and AI weekly updatesKeywords: GPT-5.6, OpenAI, Anthropic, Mythos 5, Fable 5, Frontier models, Gemini 3.5 Pro, Google, model rollout, limited AI access, AI safety, US government AI regulation, Sol model, Terra model, Luna model, Max reasoning mode, Ultra mode, sub agents, advanced AI benchmarks, coding workflows, cybersecurity, third-party AI analysis, government licensing, AI model guardrails, AI model democratization, model naming scheme, model availability, AI model security, jailbreak resistance, safety filters, general model access, trusted testers, AI export control, national security, Anthropic pullback, supply chain risk, defense department, AI industry competition, talent loss, AI coding, mid training, engineering agents, AI strike team, RaiseUS nonprofit, workforce AI disruption, technology policy, industrial scale distillation, Alibaba, AI model theft, China-US tech tensions, distillation attacks, Jalapeno AI chip, Broadcom, AI inference, custom hardware, data center GPUs, Microsoft, Meta, Elastic compute, AI-powered career navigation, Slack Claude Tag, Canva Grow 2.0, Copilot skills, AI ad creation, AI automation, DigitalOcean plugin, Apple hardware AI, smart glasses, Vision Pro, portfolio tracking AI, Google Finance, home smart speakers, voice AI, GLM 5.2, open source AI, US labor market AI effects, AI job disruption, model leaks, government approval delays.Send Everyday AI and Jordan a text message. (We can't reply back unless you leave contact info) Start Here ▶️Not sure where to start when it comes to AI? Start with our Start Here Series. You can listen to the first drop -- Episode 691 -- or get free access to our Inner Cricle community and all episodes: StartHereSeries.com Also, here's a link to the entire series on a Spotify playlist. 

Is the open model GLM-5.2 really Opus 4.8 level?

While Anthropic and the U.S. Government continued to try and make amends, there was another seismic shift quietly taking place: open source surged. Between Microsoft reportedly testing Open Source models for Copilot and the powerful new GLM-5.2, there was a clear trend this week in AI world. Missed it all? Don't worry, we'll catch you up so you can make the informed decisions for your company. Anthropic Continues Fable Fight, Microsoft Goes Open Source, Midjourney's Big Pivot and More AI News That Matters -- An Everyday AI Chat with Jordan WilsonNewsletter: Sign up for our free daily newsletterMore on this Episode: Episode PageToday's Episode on LinkedIn: Thoughts on this? Join the convo on LinkedIn and connect with other AI leaders.Upcoming Episodes: Check out the upcoming Everyday AI Livestream lineupWebsite: YourEverydayAI.comEmail The Show: info@youreverydayai.comConnect with Jordan on LinkedInTopics Covered in This Episode:Anthropic Fable 5 and Mythos 5 Export BanTrump Labels Anthropic a National Security ThreatMicrosoft Copilot CoWork Open Source Model SwitchMicrosoft Considers DeepSeek-V4 for AI Cost ReductionChinese GLM 5-2 Sets Open Source BenchmarkGLM 5-2 Challenges Proprietary AI ModelsMidJourney Hardware Pivot: AI Medical Imaging ScannerCursor Building 1.5T Parameter Model, GitHub CompetitorAI CEO Summit: G7 Pushes US-Led AI CoalitionOpenAI Prepares GPT-5.6 ReleaseAnthropic, OpenAI, Google Face Geopolitical AI ScrutinyAdvancements in Token Efficiency and Cost ControlTimestamps:00:00 Trump's comments on Anthropic06:17 Microsoft exploring lower-cost AI models09:07 Microsoft exploring DeepSeek amid tensions13:45 AI model performance and efficiency trends15:59 AI leaders meet at G7 Summit21:22 Midjourney unveils first hardware product23:26 MidJourney's innovative spa technology28:50 Discussing Cursor's evolution and impact32:24 Talking about AI use cases33:27 Rumors and upcoming AI model releases37:20 OpenAI's major new hiresKeywords: Anthropic, Fable Five, Mythos Five, export controls, national security threat, Dario Amodei, Amazon, supply chain risk, Defense Production Act, Copilot CoWork, Microsoft, usage based pricing, open source AI, DeepSeek V4, Chinese AI model, token costs, Azure, agentic AI, enterprise AI billing, data security, compliance filters, GLM 5-2, Zhipu AI, 753 billion parameter model, MIT open source license, long context window, autonomous coding, Hugging Face, benchmark performance, text only model, multimodal capabilities, token efficiency, AI spend, G7 summit, AI governance, AI coalition, AI standards, cybersecurity risks, bioterrorism, chip trade, Sam Altman, OpenAI, Claude Opus 4.8, Gemini 3.5 Pro, MidJourney, medical imaging, MidJourney scanner, full body ultrasound, Butterfly Network, MRI alternative, spa launch, SpaceX, Cursor, 1.5 trillion parameter model, code hosting, GitHub competitor, code generation, AI super apps, Colossus compute, technical prompts, context window expansion, GPT 5.6, Claude Conway agent, Grok Imagine, Firefly AI, code artifacts, Google Ad Manager AI, Open Knowledge Format, Noam Shazeer, Dean Ball, Andrej Karpathy.Send Everyday AI and Jordan a text message. (We can't reply back unless you leave contact info) Start Here ▶️Not sure where to start when it comes to AI? Start with our Start Here Series. You can listen to the first drop -- Episode 691 -- or get free access to our Inner Cricle community and all episodes: StartHereSeries.com Also, here's a link to the entire series on a Spotify playlist.