Podcasts about Hacker News

This is a recap of the top 10 posts on Hacker News on August 27, 2025. This podcast was generated by wondercraft.ai (00:30): Dissecting the Apple M1 GPU, the endOriginal post: https://news.ycombinator.com/item?id=45034537&utm_source=wondercraft_ai(01:55): MonodrawOriginal post: https://news.ycombinator.com/item?id=45037904&utm_source=wondercraft_ai(03:20): Scientist exposes anti-wind groups as oil-funded, now they want to silence himOriginal post: https://news.ycombinator.com/item?id=45036231&utm_source=wondercraft_ai(04:45): Nx compromised: malware uses Claude code CLI to explore the filesystemOriginal post: https://news.ycombinator.com/item?id=45038653&utm_source=wondercraft_ai(06:10): The Therac-25 Incident (2021)Original post: https://news.ycombinator.com/item?id=45036294&utm_source=wondercraft_ai(07:35): Google has eliminated 35% of managers overseeing small teams in past yearOriginal post: https://news.ycombinator.com/item?id=45045398&utm_source=wondercraft_ai(09:01): Unexpected productivity boost of RustOriginal post: https://news.ycombinator.com/item?id=45041286&utm_source=wondercraft_ai(10:26): Uncomfortable Questions About Android Developer VerificationOriginal post: https://news.ycombinator.com/item?id=45035699&utm_source=wondercraft_ai(11:51): I Am An AI HaterOriginal post: https://news.ycombinator.com/item?id=45043741&utm_source=wondercraft_ai(13:16): Malicious versions of Nx and some supporting plugins were publishedOriginal post: https://news.ycombinator.com/item?id=45034496&utm_source=wondercraft_aiThis is a third-party project, independent from HN and YC. Text and audio generated using AI, by wondercraft.ai. Create your own studio quality podcast with text as the only input in seconds at app.wondercraft.ai. Issues or feedback? We'd love to hear from you: team@wondercraft.ai

This is a recap of the top 10 posts on Hacker News on August 26, 2025. This podcast was generated by wondercraft.ai (00:30): Gemini 2.5 Flash ImageOriginal post: https://news.ycombinator.com/item?id=45026719&utm_source=wondercraft_ai(01:54): Claude for ChromeOriginal post: https://news.ycombinator.com/item?id=45030760&utm_source=wondercraft_ai(03:19): We regret but have to temporary suspend the shipments to USAOriginal post: https://news.ycombinator.com/item?id=45029579&utm_source=wondercraft_ai(04:44): Michigan Supreme Court: Unrestricted phone searches violate Fourth AmendmentOriginal post: https://news.ycombinator.com/item?id=45029764&utm_source=wondercraft_ai(06:08): US IntelOriginal post: https://news.ycombinator.com/item?id=45024786&utm_source=wondercraft_ai(07:33): Framework Laptop 16Original post: https://news.ycombinator.com/item?id=45027725&utm_source=wondercraft_ai(08:58): macOS 26 Tahoe's Dead Canary Utility App IconsOriginal post: https://news.ycombinator.com/item?id=45020685&utm_source=wondercraft_ai(10:23): Proposal to Ban Ghost JobsOriginal post: https://news.ycombinator.com/item?id=45028785&utm_source=wondercraft_ai(11:47): Dissecting the Apple M1 GPU, the endOriginal post: https://news.ycombinator.com/item?id=45034537&utm_source=wondercraft_ai(13:12): Show HN: A zoomable, searchable archive of BYTE magazineOriginal post: https://news.ycombinator.com/item?id=45028002&utm_source=wondercraft_aiThis is a third-party project, independent from HN and YC. Text and audio generated using AI, by wondercraft.ai. Create your own studio quality podcast with text as the only input in seconds at app.wondercraft.ai. Issues or feedback? We'd love to hear from you: team@wondercraft.ai

This is a recap of the top 10 posts on Hacker News on August 25, 2025. This podcast was generated by wondercraft.ai (00:30): Google will allow only apps from verified developers to be installed on AndroidOriginal post: https://news.ycombinator.com/item?id=45017028&utm_source=wondercraft_ai(01:54): What are OKLCH colors?Original post: https://news.ycombinator.com/item?id=45010876&utm_source=wondercraft_ai(03:18): Show HN: Base, an SQLite database editor for macOSOriginal post: https://news.ycombinator.com/item?id=45014131&utm_source=wondercraft_ai(04:42): Ban me at the IP level if you don't like meOriginal post: https://news.ycombinator.com/item?id=45010183&utm_source=wondercraft_ai(06:06): Building the mouse Logitech won't makeOriginal post: https://news.ycombinator.com/item?id=45014993&utm_source=wondercraft_ai(07:30): FCC bars providers for non-compliance with robocall protectionsOriginal post: https://news.ycombinator.com/item?id=45015354&utm_source=wondercraft_ai(08:54): Temporary suspension of acceptance of mail to the United StatesOriginal post: https://news.ycombinator.com/item?id=45016517&utm_source=wondercraft_ai(10:18): Google's Liquid CoolingOriginal post: https://news.ycombinator.com/item?id=45016720&utm_source=wondercraft_ai(11:42): An illustrated guide to OAuthOriginal post: https://news.ycombinator.com/item?id=45013131&utm_source=wondercraft_ai(13:06): macOS 26 Tahoe's Dead Canary Utility App IconsOriginal post: https://news.ycombinator.com/item?id=45020685&utm_source=wondercraft_aiThis is a third-party project, independent from HN and YC. Text and audio generated using AI, by wondercraft.ai. Create your own studio quality podcast with text as the only input in seconds at app.wondercraft.ai. Issues or feedback? We'd love to hear from you: team@wondercraft.ai

According to The Hacker News, an emerging ransomware strain – Anubis – has been discovered incorporating capabilities to encrypt files as well as permanently erase them. This development has been described as a “rare dual-threat.” In this episode, host Amanda Glassner is joined by Heather Engel, Managing Partner at Strategic Cyber Partners, to discuss. To learn more about today's stories, visit https://cybercrimewire.com • For more on cybersecurity, visit us at https://cybersecurityventures.com.

断片化に強いという特徴があるmallocメモリアロケーターの実装のひとつ、jemallocの20年の歴史について紹介しました。jemalloc Postmortem by Jason Evansブログ https://jasone.github.io/2025/06/12/jemalloc-postmortem/Hacker Newsの議論 https://news.ycombinator.com/item?id=44264958感想をぜひハッシュタグ #todayILearnedFM #tilfm でつぶやいてください!Your co-hosts:Tomoaki Imai, Noxx CTO  https://x.com/tomoaki_imai bsky: https://bsky.app/profile/tomoaki-imai.bsky.socialRyoichi Kato, Software Engineer ⁠https://x.com/ryo1kato bsky: https://bsky.app/profile/ryo1kato.bsky.social

Canonical has confirmed that Jeremy Bicha is "no longer engaged" with the maker of Ubuntu Linux. Debian, GNOME, & Hacker News are all covering for the criminal. More from The Lunduke Journal: https://lunduke.com/ This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit lunduke.substack.com/subscribe

Join me as I chat with Tod Sacerdoti about String, a platform that allows users to build AI agents through natural language prompts rather than complex visual workflows. Tod showcases several practical examples of automations, from monitoring Hacker News for specific terms to generating LinkedIn posts from blog content. The platform aims to make automation accessible to non-technical users while providing the power to solve complex business problems. Timestamps: 00:00 - Intro 01:03 - String Overview 01:49 - Agent 1: Monitoring Hacker News for MCP mentions 08:15 - Agent 2: Creating LinkedIn posts from RSS feeds 17:14 - Agent 3: Automating Google Analytics Summary 23:53 - Startup Idea: Building businesses on top of String 27:10 - Agent 4: Daily Automation Idea Generator 38:32 - Advice for getting started with String Checkout String: https://string.com Key Points: • String is an AI agent platform that builds AI agents using natural language instead of complex node charts • The platform allows users to create automations that monitor websites, generate content, and integrate with tools like Slack and Google Docs • String uses dynamic code generation to go beyond pre-built integrations, enabling more complex use cases • The platform follows a credit-based pricing model similar to other AI coding platforms The #1 tool to find startup ideas/trends - https://www.ideabrowser.com LCA helps Fortune 500s and fast-growing startups build their future - from Warner Music to Fortnite to Dropbox. We turn 'what if' into reality with AI, apps, and next-gen products https://latecheckout.agency/ Boringmarketing - Vibe Marketing for Companies: boringmarketing.com The Vibe Marketer - Join the Community and Learn: thevibemarketer.com Startup Empire - a membership for builders who want to build cash-flowing businesses https://www.skool.com/startupempire/about FIND ME ON SOCIAL X/Twitter: https://twitter.com/gregisenberg Instagram: https://instagram.com/gregisenberg/ LinkedIn: https://www.linkedin.com/in/gisenberg/ FIND TOD ON SOCIAL String: https://string.com X/Twitter: https://x.com/tod?lang=en

Dan sees great results on YouTube, Mike sees great results on Hacker News.

Brian "Beej Jorgensen" Hall joins Oz and Charlie to dive into the joy and art of learning computer science. It's all here - achieving flow, "How to Solve It" by Polya, self-publishing guides/books on the web, Beej's take on AI and coding, and, of course, the origin of the "Beej Jorgensen" moniker. Shownotes:Beej's website: https://beej.us/Beej's guide to learning computer science: https://beej.us/guide/bglcs/Hacker News post on Charlie's short story about a giant diamond asteroid

According to The Hacker News, an ongoing campaign that infiltrates legitimate websites with malicious JavaScript injects to promote Chinese-language gambling platforms has ballooned to compromise approximately 150,000 sites to date. In this episode, host Amanda Glassner is joined by Heather Engel, Managing Partner at Strategic Cyber Partners, to discuss. To learn more about today's stories, visit https://cybercrimewire.com • For more on cybersecurity, visit us at https://cybersecurityventures.com.

Alex breaks down seedstrapping. What it is, why it's in vogue, what the internet thinks, and how you should think about it in the context of your business. — Show Notes: (0:00) A note from our sponsor (4:26) Intro to seedstrapping (6:20) Why seedstrapping is in vogue (9:15) Founder take on seedstrapping (12:40) VCs views on seedstrapping (14:00) My take on seedstrapping — Thanks to our presenting sponsor, Gusto. Head to www.gusto.com/alex — Show links: • HackerNews on seedstrapping: https://news.ycombinator.com/item?id=36508471 • VC on seedstrapping: https://x.com/nikunj/status/1908155930690359437 • Josh Payne Linkedin post: https://www.linkedin.com/posts/jnpayne_when-i-started-my-first-company-in-2011-activity-7292920969323503617-JJKk?utm_source=share&utm_medium=member_desktop&rcm=ACoAAAuMh6ABe5yrIF2I6gDHnZ3Fpixhetg9Rfg • Zapier CEO interview: https://www.youtube.com/watch?v=CZ36aQDjcC4&t=906s&pp=ygUSd2FkZSBmb3N0ZXIgemFwaWVy0gcJCYQJAYcqIYzv Check Out Alex's Stuff: • storyarb - https://www.storyarb.com/ • growthpair - https://www.growthpair.com/ • distro - https://youdistro.com/  • X - https://x.com/businessbarista • Linkedin - https://www.linkedin.com/in/alex-lieberman/ Learn more about your ad choices. Visit megaphone.fm/adchoices

Episode title and number:Advocacy in Action: Transforming Accessibility with RightHear's Idan Meir 5-#2Summary of the show: Bold Blind Beauty On A.I.R. hosts a talk with RightHear CEO Idan Meir about accessible audio wayfinding and advocacy for inclusion. The discussion covers RightHear's origins, the power of community input, and the importance of universal design.Supporting Our Advocacy Work:⦁ Be a part of the change! Support our advocacy efforts. Bullet points of key topics & timestamps: 00:00 | Introduction and Hosts00:44 | Introducing Idan Meir and RightHear02:22 | The Story Behind RightHear05:47 | RightHear's Technology and Impact10:54 | Empowering Blind Advocates17:10 | Partnerships and Global Change23:27 | Community Engagement and Advocacy27:30 | Conclusion and Contact InformationIdan Meir's Bio:Idan Meir is a mission-driven entrepreneur and the Co-founder & CEO of RightHear, an award-winning accessibility startup empowering people with orientation challenges to navigate public spaces independently. Based in Rockville, Maryland, Idan has been featured in Forbes, FastCompany, and HackerNews, and is a member of the exclusive CEO network, MindShare. He previously led Hubanana, a thriving startup hub, and co-founded Zikit. A veteran of an elite IDF unit and holder of an M.A. in Psychology and Management, Idan brings a rare blend of vision, leadership, and passion for impact—always fueled by strong espresso and a good plate of hummus.RightHear's Socials: Idan's Email: idan@Right-Hear.comWebsite: www.right-hear.comCommunity: www.right-hear.com/communityLinkedIn: @RightHearInstagram: @righthearappFacebook: @RightHearAppYouTube: @RightHearConnect with Bold Blind Beauty to learn more about our advocacy: Join our Instagram community @BoldBlindBeauty Subscribe to our YouTube channel @BoldBlindBeauty Check out our website www.boldblindbeauty.com Music Credit: "Ambient Uplifting Harmonic Happy" By Panda-x-music https://audiojungle.net/item/ambient-uplifting-harmonic-happy/46309958Thanks for listening!❤️

Nikolay and Michael are joined by Lev Kokotov to discuss PgDog — including whether or when sharding is needed, the origin story (via PgCat), what's already supported, and what's coming next.    Here are some links to things they mentioned:Lev Kokotov https://postgres.fm/people/lev-kokotovPgDog https://github.com/pgdogdev/pgdogPgCat https://github.com/postgresml/pgcatAdopting PgCat (Instacart blog post) https://www.instacart.com/company/how-its-made/adopting-pgcat-a-nextgen-postgres-proxyPgDog discussion on Hacker News https://news.ycombinator.com/item?id=43364668Citus https://github.com/citusdata/citusSharding & IDs at Instagram (blog post) https://instagram-engineering.com/sharding-ids-at-instagram-1cf5a71e5a5cSharding pgvector (blog post by Lev) https://pgdog.dev/blog/sharding-pgvector~~~What did you like or not like? What should we discuss next time? Let us know via a YouTube comment, on social media, or by commenting on our Google doc!~~~Postgres FM is produced by:Michael Christofides, founder of pgMustardNikolay Samokhvalov, founder of Postgres.aiWith special thanks to:Jessie Draws for the elephant artwork 

Ian and Aaron discuss Ian's trip to Disney World, Aaron's adventures with Reddit & Hacker News, building a new billing system, and so much more.Sponsored by Bento, Stream, Laravel Cloud, and PHP Tek 2025.Interested in sponsoring Mostly Technical?  Head to https://mostlytechnical.com/sponsor to learn more.(00:00) - Disney World (17:01) - Follow Up (22:03) - Aaron's Newsletter (25:59) - Bookmarking Situation (40:02) - Reddit & Hacker News (52:29) - Building A Billing System (01:17:40) - Monarch Follow Up Links:Will King on TwitterMinnie VanWalt Disney World Swan and DolphinThe Art of ProductBen Thompson's Interview w/ Sam AltmanmymindPinboardAaron's post about Screen on r/phpSolo on GitHubScreen on GitHubScreencasting.comMonarch

Topics covered in this episode: Why aren't you using uv? Python Developer Tooling Handbook Calling all doc writers: blacken-docs Reinventing notebooks as reusable Python programs Extras Joke Watch on YouTube About the show Brought to you by Posit Connect: pythonbytes.fm/connect. Connect with the hosts Michael: @mkennedy@fosstodon.org / @mkennedy.codes (bsky) Brian: @brianokken@fosstodon.org / @brianokken.bsky.social Show: @pythonbytes@fosstodon.org / @pythonbytes.fm (bsky) Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Monday at 10am PT. Older video versions available there too. Finally, if you want an artisanal, hand-crafted digest of every week of the show notes in email form? Add your name and email to our friends of the show list, we'll never share it. Michael #1: Why aren't you using uv? Fun conversation on X by Armin Ronacher. Interesting quotes from the thread I get it replaces pip/pyenv, but should I also use it instead of the built in 'python -m venv .venv'? But I need python installed to make python programs? Because it places the venv in the project folder and I can't run executables from there due to corporate policy. Many such cases. No idea why astral doesn't address this with more urgency. Sounds like a bad corporate policy :) i'm too lazy to switch from pyenv and pip trust issues, what if they do a bait and switch … Because everyone said that about poetry and I'm not sure I'm really ready to get hurt again. Masochism Many times I tried a lot of similar tools and always come back to pip and pip-tools. Them are just work, why should I spend my time for something "cool" that will bring more problems? I tried this week but I was expecting a "uv install requests" instead of "uv add". Switched back to pipenv. we partially use it. will transition when Dependabot support is available. I'll leave it with → Jared Scheel: Seeing a whole lotta Stockholm Syndrome in the replies to this question. Brian #2: Python Developer Tooling Handbook Tim Hopper “This is not a book about programming Python. Instead, the goal of this book is to help you understand the ecosystem of tools used to make Python development easier and more productive” Covers tools related to packaging, linting, formatting, and managing dependencies. Michael #3: Calling all doc writers: blacken-docs Run black on python code blocks in documentation files You can also install blacken-docs as a pre-commit hook. It supports Markdown, reStructuredText, and LaTex files. Additionally, you can run it on Python files to reformat Markdown and reStructuredText within docstrings. Brian #4: Reinventing notebooks as reusable Python programs marimo allows you to store notebooks as plaintext Python files properties Git-friendly: small code change => small diff easy for both humans and computers to read importable as a Python module, without executing notebook cells executable as a Python script editable with a text editor Also, … testing with pytest “Because marimo notebooks are just Python files, they are interoperable with other tools for Python — including pytest. “ “Testing cells. Any cell named as test_* is automatically discoverable and testable by pytest. The same goes for any cell that contains only test_ functions and Test classes.” “Importantly, because cells are wrapped in functions, running pytest test_notebook.py doesn't execute the entire notebook — just its tests.” Extras Brian: PyConUS announces Refund Policy for International Attendees New format now live for The Complete pytest Course Bundle and component courses Each course now available separately also pytest Primary Power is 13 lessons, 3.9 hours Using pytest with Projects, 10 lessons, 3.4 hours pytest Booster Rockets, 6 lessons, 1.3 hours of content New format is easier to navigate Better for people who like different speeds. I'm usually a 1.25x-1.5x speed person. Now also with Congratulations! lessons (with fireworks) and printable certificates. Michael: PyCon Taiwan is currently calling for proposals HN trends follow up via Shinjitsu I'm sure some other Hacker News reader has already given you the feedback, but in the unlikely case that they haven't, You read those headlines in this segment exactly wrong. “Ask HN: Who is hiring?" is a monthly post that asks employers to post about jobs they have available “Ask HN: Who wants to be hired?” is a monthly topic where they ask people who are looking for jobs to post about themselves in the hope that their skillset it is a good match (and not an LLM generated resume) So unfortunately your rosy analysis might need a less rosy interpretation. Joke: Top 12 things likely to be overheard if you had a Klingon Programmer From Holgi on Mastodon

We are working with Amplify on the 2025 State of AI Engineering Survey to be presented at the AIE World's Fair in SF! Join the survey to shape the future of AI Eng!We first met Snipd over a year ago, and were immediately impressed by the design, but were doubtful about the behavior of snipping as the title behavior:Podcast apps are enormously sticky - Spotify spent almost $1b in podcast acquisitions and exclusive content just to get an 8% bump in market share among normies.However, after a disappointing Overcast 2.0 rewrite with no AI features in the last 3 years, I finally bit the bullet and switched to Snipd. It's 2025, your podcast app should be able to let you search transcripts of your podcasts. Snipd is the best implementation of this so far.And yet they keep shipping:What impressed us wasn't just how this tiny team of 4 was able to bootstrap a consumer AI app against massive titans and do so well; but also how seriously they think about learning through podcasts and improving retention of knowledge over time, aka “Duolingo for podcasts”. As an educational AI podcast, that's a mission we can get behind.Full Video PodFind us on YouTube! This was the first pod we've ever shot outdoors!Show Notes* How does Shazam work?* Flutter/FlutterFlow* wav2vec paper* Perplexity Online LLM* Google Search Grounding* Comparing Snipd transcription with our Bee episode* NIPS 2017 Flo Rida* Gustav Söderström - Background AudioTimestamps* [00:00:03] Takeaways from AI Engineer NYC* [00:00:17] Weather in New York.* [00:00:26] Swyx and Snipd.* [00:01:01] Kevin's AI summit experience.* [00:01:31] Zurich and AI.* [00:03:25] SigLIP authors join OpenAI.* [00:03:39] Zurich is very costly.* [00:04:06] The Snipd origin story.* [00:05:24] Introduction to machine learning.* [00:09:28] Snipd and user knowledge extraction.* [00:13:48] App's tech stack, Flutter, Python.* [00:15:11] How speakers are identified.* [00:18:29] The concept of "backgroundable" video.* [00:29:05] Voice cloning technology.* [00:31:03] Using AI agents.* [00:34:32] Snipd's future is multi-modal AI.* [00:36:37] Snipd and existing user behaviour.* [00:42:10] The app, summary, and timestamps.* [00:55:25] The future of AI and podcasting.* [1:14:55] Voice AITranscriptswyx [00:00:03]: Hey, I'm here in New York with Kevin Ben-Smith of Snipd. Welcome.Kevin [00:00:07]: Hi. Hi. Amazing to be here.swyx [00:00:09]: Yeah. This is our first ever, I think, outdoors podcast recording.Kevin [00:00:14]: It's quite a location for the first time, I have to say.swyx [00:00:18]: I was actually unsure because, you know, it's cold. It's like, I checked the temperature. It's like kind of one degree Celsius, but it's not that bad with the sun. No, it's quite nice. Yeah. Especially with our beautiful tea. With the tea. Yeah. Perfect. We're going to talk about Snips. I'm a Snips user. I'm a Snips user. I had to basically, you know, apart from Twitter, it's like the number one use app on my phone. Nice. When I wake up in the morning, I open Snips and I, you know, see what's new. And I think in terms of time spent or usage on my phone, I think it's number one or number two. Nice. Nice. So I really had to talk about it also because I think people interested in AI want to think about like, how can we, we're an AI podcast, we have to talk about the AI podcast app. But before we get there, we just finished. We just finished the AI Engineer Summit and you came for the two days. How was it?Kevin [00:01:07]: It was quite incredible. I mean, for me, the most valuable was just being in the same room with like-minded people who are building the future and who are seeing the future. You know, especially when it comes to AI agents, it's so often I have conversations with friends who are not in the AI world. And it's like so quickly it happens that you, it sounds like you're talking in science fiction. And it's just crazy talk. It was, you know, it's so refreshing to talk with so many other people who already see these things and yeah, be inspired then by them and not always feel like, like, okay, I think I'm just crazy. And like, this will never happen. It really is happening. And for me, it was very valuable. So day two, more relevant, more relevant for you than day one. Yeah. Day two. So day two was the engineering track. Yeah. That was definitely the most valuable for me. Like also as a producer. Practitioner myself, especially there were one or two talks that had to do with voice AI and AI agents with voice. Okay. So that was quite fascinating. Also spoke with the speakers afterwards. Yeah. And yeah, they were also very open and, and, you know, this, this sharing attitudes that's, I think in general, quite prevalent in the AI community. I also learned a lot, like really practical things that I can now take away with me. Yeah.swyx [00:02:25]: I mean, on my side, I, I think I watched only like half of the talks. Cause I was running around and I think people saw me like towards the end, I was kind of collapsing. I was on the floor, like, uh, towards the end because I, I needed to get, to get a rest, but yeah, I'm excited to watch the voice AI talks myself.Kevin [00:02:43]: Yeah. Yeah. Do that. And I mean, from my side, thanks a lot for organizing this conference for bringing everyone together. Do you have anything like this in Switzerland? The short answer is no. Um, I mean, I have to say the AI community in, especially Zurich, where. Yeah. Where we're, where we're based. Yeah. It is quite good. And it's growing, uh, especially driven by ETH, the, the technical university there and all of the big companies, they have AI teams there. Google, like Google has the biggest tech hub outside of the U S in Zurich. Yeah. Facebook is doing a lot in reality labs. Uh, Apple has a secret AI team, open AI and then SwapBit just announced that they're coming to Zurich. Yeah. Um, so there's a lot happening. Yeah.swyx [00:03:23]: So, yeah, uh, I think the most recent notable move, I think the entire vision team from Google. Uh, Lucas buyer, um, and, and all the other authors of Siglip left Google to join open AI, which I thought was like, it's like a big move for a whole team to move all at once at the same time. So I've been to Zurich and it just feels expensive. Like it's a great city. Yeah. It's great university, but I don't see it as like a business hub. Is it a business hub? I guess it is. Right.Kevin [00:03:51]: Like it's kind of, well, historically it's, uh, it's a finance hub, finance hub. Yeah. I mean, there are some, some large banks there, right? Especially UBS, uh, the, the largest wealth manager in the world, but it's really becoming more of a tech hub now with all of the big, uh, tech companies there.swyx [00:04:08]: I guess. Yeah. Yeah. And, but we, and research wise, it's all ETH. Yeah. There's some other things. Yeah. Yeah. Yeah.Kevin [00:04:13]: It's all driven by ETH. And then, uh, it's sister university EPFL, which is in Lausanne. Okay. Um, which they're also doing a lot, but, uh, it's, it's, it's really ETH. Uh, and otherwise, no, I mean, it's a beautiful, really beautiful city. I can recommend. To anyone. To come, uh, visit Zurich, uh, uh, let me know, happy to show you around and of course, you know, you, you have the nature so close, you have the mountains so close, you have so, so beautiful lakes. Yeah. Um, I think that's what makes it such a livable city. Yeah.swyx [00:04:42]: Um, and the cost is not, it's not cheap, but I mean, we're in New York city right now and, uh, I don't know, I paid $8 for a coffee this morning, so, uh, the coffee is cheaper in Zurich than the New York city. Okay. Okay. Let's talk about Snipt. What is Snipt and, you know, then we'll talk about your origin story, but I just, let's, let's get a crisp, what is Snipt? Yeah.Kevin [00:05:03]: I always see two definitions of Snipt, so I'll give you one really simple, straightforward one, and then a second more nuanced, um, which I think will be valuable for the rest of our conversation. So the most simple one is just to say, look, we're an AI powered podcast app. So if you listen to podcasts, we're now providing this AI enhanced experience. But if you look at the more nuanced, uh, podcast. Uh, perspective, it's actually, we, we've have a very big focus on people who like your audience who listened to podcasts to learn something new. Like your audience, you want, they want to learn about AI, what's happening, what's, what's, what's the latest research, what's going on. And we want to provide a, a spoken audio platform where you can do that most effectively. And AI is basically the way that we can achieve that. Yeah.swyx [00:05:53]: Means to an end. Yeah, exactly. When you started. Was it always meant to be AI or is it, was it more about the social sharing?Kevin [00:05:59]: So the first version that we ever released was like three and a half years ago. Okay. Yeah. So this was before ChatGPT. Before Whisper. Yeah. Before Whisper. Yeah. So I think a lot of the features that we now have in the app, they weren't really possible yet back then. But we already from the beginning, we always had the focus on knowledge. That's the reason why, you know, we in our team, why we listen to podcasts, but we did have a bit of a different approach. Like the idea in the very beginning was, so the name is Snips and you can create these, what we call Snips, which is basically a small snippet, like a clip from a, from a podcast. And we did envision sort of like a, like a social TikTok platform where some people would listen to full episodes and they would snip certain, like the best parts of it. And they would post that in a feed and other users would consume this feed of Snips. And use that as a discovery tool or just as a means to an end. And yeah, so you would have both people who create Snips and people who listen to Snips. So our big hypothesis in the beginning was, you know, it will be easy to get people to listen to these Snips, but super difficult to actually get them to create them. So we focused a lot of, a lot of our effort on making it as seamless and easy as possible to create a Snip. Yeah.swyx [00:07:17]: It's similar to TikTok. You need CapCut for there to be videos on TikTok. Exactly.Kevin [00:07:23]: And so for, for Snips, basically whenever you hear an amazing insight, a great moment, you can just triple tap your headphones. And our AI actually then saves the moment that you just listened to and summarizes it to create a note. And this is then basically a Snip. So yeah, we built, we built all of this, launched it. And what we found out was basically the exact opposite. So we saw that people use the Snips to discover podcasts, but they really, you know, they don't. You know, really love listening to long form podcasts, but they were creating Snips like crazy. And this was, this was definitely one of these aha moments when we realized like, hey, we should be really doubling down on the knowledge of learning of, yeah, helping you learn most effectively and helping you capture the knowledge that you listen to and actually do something with it. Because this is in general, you know, we, we live in this world where there's so much content and we consume and consume and consume. And it's so easy to just at the end of the podcast. You just start listening to the next podcast. And five minutes later, you've forgotten everything. 90%, 99% of what you've actually just learned. Yeah.swyx [00:08:31]: You don't know this, but, and most people don't know this, but this is my fourth podcast. My third podcast was a personal mixtape podcast where I Snipped manually sections of podcasts that I liked and added my own commentary on top of them and published them as small episodes. Nice. So those would be maybe five to 10 minute Snips. Yeah. And then I added something that I thought was a good story or like a good insight. And then I added my own commentary and published it as a separate podcast. It's cool. Is that still live? It's still live, but it's not active, but you can go back and find it. If you're, if, if you're curious enough, you'll see it. Nice. Yeah. You have to show me later. It was so manual because basically what my process would be, I hear something interesting. I note down the timestamp and I note down the URL of the podcast. I used to use Overcast. So it would just link to the Overcast page. And then. Put in my note taking app, go home. Whenever I feel like publishing, I will take one of those things and then download the MP3, clip out the MP3 and record my intro, outro and then publish it as a, as a podcast. But now Snips, I mean, I can just kind of double click or triple tap.Kevin [00:09:39]: I mean, those are very similar stories to what we hear from our users. You know, it's, it's normal that you're doing, you're doing something else while you're listening to a podcast. Yeah. A lot of our users, they're driving, they're working out, walking their dog. So in those moments when you hear something amazing, it's difficult to just write them down or, you know, you have to take out your phone. Some people take a screenshot, write down a timestamp, and then later on you have to go back and try to find it again. Of course you can't find it anymore because there's no search. There's no command F. And, um, these, these were all of the issues that, that, that we encountered also ourselves as users. And given that our background was in AI, we realized like, wait, hey, this is. This should not be the case. Like podcast apps today, they're still, they're basically repurposed music players, but we actually look at podcasts as one of the largest sources of knowledge in the world. And once you have that different angle of looking at it together with everything that AI is now enabling, you realize like, hey, this is not the way that we, that podcast apps should be. Yeah.swyx [00:10:41]: Yeah. I agree. You mentioned something that you said your background is in AI. Well, first of all, who's the team and what do you mean your background is in AI?Kevin [00:10:48]: Those are two very different things. I'm going to ask some questions. Yeah. Um, maybe starting with, with my backstory. Yeah. My backstory actually goes back, like, let's say 12 years ago or something like that. I moved to Zurich to study at ETH and actually I studied something completely different. I studied mathematics and economics basically with this specialization for quant finance. Same. Okay. Wow. All right. So yeah. And then as you know, all of these mathematical models for, um, asset pricing, derivative pricing, quantitative trading. And for me, the thing that, that fascinates me the most was the mathematical modeling behind it. Uh, mathematics, uh, statistics, but I was never really that passionate about the finance side of things.swyx [00:11:32]: Oh really? Oh, okay. Yeah. I mean, we're different there.Kevin [00:11:36]: I mean, one just, let's say symptom that I noticed now, like, like looking back during that time. Yeah. I think I never read an academic paper about the subject in my free time. And then it was towards the end of my studies. I was already working for a big bank. One of my best friends, he comes to me and says, Hey, I just took this course. You have to, you have to do this. You have to take this lecture. Okay. And I'm like, what, what, what is it about? It's called machine learning and I'm like, what, what, what kind of stupid name is that? Uh, so you sent me the slides and like over a weekend I went through all of the slides and I just, I just knew like freaking hell. Like this is it. I'm, I'm in love. Wow. Yeah. Okay. And that was then over the course of the next, I think like 12 months, I just really got into it. Started reading all about it, like reading blog posts, starting building my own models.swyx [00:12:26]: Was this course by a famous person, famous university? Was it like the Andrew Wayne Coursera thing? No.Kevin [00:12:31]: So this was a ETH course. So a professor at ETH. Did he teach in English by the way? Yeah. Okay.swyx [00:12:37]: So these slides are somewhere available. Yeah. Definitely. I mean, now they're quite outdated. Yeah. Sure. Well, I think, you know, reflecting on the finance thing for a bit. So I, I was, used to be a trader, uh, sell side and buy side. I was options trader first and then I was more like a quantitative hedge fund analyst. We never really use machine learning. It was more like a little bit of statistical modeling, but really like you, you fit, you know, your regression.Kevin [00:13:03]: No, I mean, that's, that's what it is. And, uh, or you, you solve partial differential equations and have then numerical methods to, to, to solve these. That's, that's for you. That's your degree. And that's, that's not really what you do at work. Right. Unless, well, I don't know what you do at work. In my job. No, no, we weren't solving the partial differential. Yeah.swyx [00:13:18]: You learn all this in school and then you don't use it.Kevin [00:13:20]: I mean, we, we, well, let's put it like that. Um, in some things, yeah, I mean, I did code algorithms that would do it, but it was basically like, it was the most basic algorithms and then you just like slightly improve them a little bit. Like you just tweak them here and there. Yeah. It wasn't like starting from scratch, like, Oh, here's this new partial differential equation. How do we know?swyx [00:13:43]: Yeah. Yeah. I mean, that's, that's real life, right? Most, most of it's kind of boring or you're, you're using established things because they're established because, uh, they tackle the most important topics. Um, yeah. Portfolio management was more interesting for me. Um, and, uh, we, we were sort of the first to combine like social data with, with quantitative trading. And I think, uh, I think now it's very common, but, um, yeah. Anyway, then you, you went, you went deep on machine learning and then what? You quit your job? Yeah. Yeah. Wow.Kevin [00:14:12]: I quit my job because, uh, um, I mean, I started using it at the bank as well. Like try, like, you know, I like desperately tried to find any kind of excuse to like use it here or there, but it just was clear to me, like, no, if I want to do this, um, like I just have to like make a real cut. So I quit my job and joined an early stage, uh, tech startup in Zurich where then built up the AI team over five years. Wow. Yeah. So yeah, we built various machine learning, uh, things for, for banks from like models for, for sales teams to identify which clients like which product to sell to them and with what reasons all the way to, we did a lot, a lot with bank transactions. One of the actually most fun projects for me was we had an, an NLP model that would take the booking text of a transaction, like a credit card transaction and pretty fired. Yeah. Because it had all of these, you know, like numbers in there and abbreviations and whatnot. And sometimes you look at it like, what, what is this? And it was just, you know, it would just change it to, I don't know, CVS. Yeah.swyx [00:15:15]: Yeah. But I mean, would you have hallucinations?Kevin [00:15:17]: No, no, no. The way that everything was set up, it wasn't like, it wasn't yet fully end to end generative, uh, neural network as what you would use today. Okay.swyx [00:15:30]: Awesome. And then when did you go like full time on Snips? Yeah.Kevin [00:15:33]: So basically that was, that was afterwards. I mean, how that started was the friend of mine who got me into machine learning, uh, him and I, uh, like he also got me interested into startups. He's had a big impact on my life. And the two of us were just a jam on, on like ideas for startups every now and then. And his background was also in AI data science. And we had a couple of ideas, but given that we were working full times, we were thinking about, uh, so we participated in Hack Zurich. That's, uh, Europe's biggest hackathon, um, or at least was at the time. And we said, Hey, this is just a weekend. Let's just try out an idea, like hack something together and see how it works. And the idea was that we'd be able to search through podcast episodes, like within a podcast. Yeah. So we did that. Long story short, uh, we managed to do it like to build something that we realized, Hey, this actually works. You can, you can find things again in podcasts. We had like a natural language search and we pitched it on stage. And we actually won the hackathon, which was cool. I mean, we, we also, I think we had a good, um, like a good, good pitch or a good example. So we, we used the famous Joe Rogan episode with Elon Musk where Elon Musk smokes a joint. Okay. Um, it's like a two and a half hour episode. So we were on stage and then we just searched for like smoking weed and it would find that exact moment. It will play it. And it just like, come on with Elon Musk, just like smoking. Oh, so it was video as well? No, it was actually completely based on audio. But we did have the video for the presentation. Yeah. Which had a, had of course an amazing effect. Yeah. Like this gave us a lot of activation energy, but it wasn't actually about winning the hackathon. Yeah. But the interesting thing that happened was after we pitched on stage, several of the other participants, like a lot of them came up to us and started saying like, Hey, can I use this? Like I have this issue. And like some also came up and told us about other problems that they have, like very adjacent to this with a podcast. Where's like, like this. Like, could, could I use this for that as well? And that was basically the, the moment where I realized, Hey, it's actually not just us who are having these issues with, with podcasts and getting to the, making the most out of this knowledge. Yeah. The other people. Yeah. That was now, I guess like four years ago or something like that. And then, yeah, we decided to quit our jobs and start, start this whole snip thing. Yeah. How big is the team now? We're just four people. Yeah. Just four people. Yeah. Like four. We're all technical. Yeah. Basically two on the, the backend side. So one of my co-founders is this person who got me into machine learning and startups. And we won the hackathon together. So we have two people for the backend side with the AI and all of the other backend things. And two for the front end side, building the app.swyx [00:18:18]: Which is mostly Android and iOS. Yeah.Kevin [00:18:21]: It's iOS and Android. We also have a watch app for, for Apple, but yeah, it's mostly iOS. Yeah.swyx [00:18:27]: The watch thing, it was very funny because in the, in the Latent Space discord, you know, most of us have been slowly adopting snips. You came to me like a year ago and you introduced snip to me. I was like, I don't know. I'm, you know, I'm very sticky to overcast and then slowly we switch. Why watch?Kevin [00:18:43]: So it goes back to a lot of our users, they do something else while, while listening to a podcast, right? Yeah. And one of the, us giving them the ability to then capture this knowledge, even though they're doing something else at the same time is one of the killer features. Yeah. Maybe I can actually, maybe at some point I should maybe give a bit more of an overview of what the, all of the features that we have. Sure. So this is one of the killer features and for one big use case that people use this for is for running. Yeah. So if you're a big runner, a big jogger or cycling, like really, really cycling competitively and a lot of the people, they don't want to take their phone with them when they go running. So you load everything onto the watch. So you can download episodes. I mean, if you, if you have an Apple watch that has internet access, like with a SIM card, you can also directly stream. That's also possible. Yeah. So of course it's a, it's basically very limited to just listening and snipping. And then you can see all of your snips later on your phone. Let me tell you this error I just got.swyx [00:19:47]: Error playing episode. Substack, the host of this podcast, does not allow this podcast to be played on an Apple watch. Yeah.Kevin [00:19:52]: That's a very beautiful thing. So we found out that all of the podcasts hosted on Substack, you cannot play them on an Apple watch. Why is this restriction? What? Like, don't ask me. We try to reach out to Substack. We try to reach out to some of the bigger podcasters who are hosting the podcast on Substack to also let them know. Substack doesn't seem to care. This is not specific to our app. You can also check out the Apple podcast app. Yeah. It's the same problem. It's just that we actually have identified it. And we tell the user what's going on.swyx [00:20:25]: I would say we host our podcast on Substack, but they're not very serious about their podcasting tools. I've told them before, I've been very upfront with them. So I don't feel like I'm shitting on them in any way. And it's kind of sad because otherwise it's a perfect creative platform. But the way that they treat podcasting as an afterthought, I think it's really disappointing.Kevin [00:20:45]: Maybe given that you mentioned all these features, maybe I can give a bit of a better overview of the features that we have. Let's do that. Let's do that. So I think we're mostly in our minds. Maybe for some of the listeners.swyx [00:20:55]: I mean, I'll tell you my version. Yeah. They can correct me, right? So first of all, I think the main job is for it to be a podcast listening app. It should be basically a complete superset of what you normally get on Overcast or Apple Podcasts or anything like that. You pull your show list from ListenNotes. How do you find shows? You've got to type in anything and you find them, right?Kevin [00:21:18]: Yeah. We have a search engine that is powered by ListenNotes. Yeah. But I mean, in the meantime, we have a huge database of like 99% of all podcasts out there ourselves. Yeah.swyx [00:21:27]: What I noticed, the default experience is you do not auto-download shows. And that's one very big difference for you guys versus other apps, where like, you know, if I'm subscribed to a thing, it auto-downloads and I already have the MP3 downloaded overnight. For me, I have to actively put it onto my queue, then it auto-downloads. And actually, I initially didn't like that. I think I maybe told you that I was like, oh, it's like a feature that I don't like. Like, because it means that I have to choose to listen to it in order to download and not to... It's like opt-in. There's a difference between opt-in and opt-out. So I opt-in to every episode that I listen to. And then, like, you know, you open it and depends on whether or not you have the AI stuff enabled. But the default experience is no AI stuff enabled. You can listen to it. You can see the snips, the number of snips and where people snip during the episode, which roughly correlates to interest level. And obviously, you can snip there. I think that's the default experience. I think snipping is really cool. Like, I use it to share a lot on Discord. I think we have tons and tons of just people sharing snips and stuff. Tweeting stuff is also like a nice, pleasant experience. But like the real features come when you actually turn on the AI stuff. And so the reason I got snipped, because I got fed up with Overcast not implementing any AI features at all. Instead, they spent two years rewriting their app to be a little bit faster. And I'm like, like, it's 2025. I should have a podcast that has transcripts that I can search. Very, very basic thing. Overcast will basically never have it.Kevin [00:22:49]: Yeah, I think that was a good, like, basic overview. Maybe I can add a bit to it with the AI features that we have. So one thing that we do every time a new podcast comes out, we transcribe the episode. We do speaker diarization. We identify the speaker names. Each guest, we extract a mini bio of the guest, try to find a picture of the guest online, add it. We break the podcast down into chapters, as in AI generated chapters. That one. That one's very handy. With a quick description per title and quick description per each chapter. We identify all books that get mentioned on a podcast. You can tell I don't use that one. It depends on the podcast. There are some podcasts where the guests often recommend like an amazing book. So later on, you can you can find that again.swyx [00:23:42]: So you literally search for the word book or I just read blah, blah, blah.Kevin [00:23:46]: No, I mean, it's all LLM based. Yeah. So basically, we have we have an LLM that goes through the entire transcript and identifies if a user mentions a book, then we use perplexity API together with various other LLM orchestration to go out there on the Internet, find everything that there is to know about the book, find the cover, find who or what the author is, get a quick description of it for the author. We then check on which other episodes the author appeared on.swyx [00:24:15]: Yeah, that is killer.Kevin [00:24:17]: Because that for me, if. If there's an interesting book, the first thing I do is I actually listen to a podcast episode with a with a writer because he usually gives a really great overview already on a podcast.swyx [00:24:28]: Sometimes the podcast is with the person as a guest. Sometimes his podcast is about the person without him there. Do you pick up both?Kevin [00:24:37]: So, yes, we pick up both in like our latest models. But actually what we show you in the app, the goal is to currently only show you the guest to separate that. In the future, we want to show the other things more.swyx [00:24:47]: For what it's worth, I don't mind. Yeah, I don't think like if I like if I like somebody, I'll just learn about them regardless of whether they're there or not.Kevin [00:24:55]: Yeah, I mean, yes and no. We we we have seen there are some personalities where this can break down. So, for example, the first version that we released with this feature, it picked up much more often a person, even if it was not a guest. Yeah. For example, the best examples for me is Sam Altman and Elon Musk. Like they're just mentioned on every second podcast and it has like they're not on there. And if you're interested in it, you can go to Elon Musk. And actually like learning from them. Yeah, I see. And yeah, we updated our our algorithms, improved that a lot. And now it's gotten much better to only pick it up if they're a guest. And yeah, so this this is maybe to come back to the features, two more important features like we have the ability to chat with an episode. Yes. Of course, you can do the old style of searching through a transcript with a keyword search. But I think for me, this is this is how you used to do search and extracting knowledge in the in the past. Old school. And the A.I. Web. Way is is basically an LLM. So you can ask the LLM, hey, when do they talk about topic X? If you're interested in only a certain part of the episode, you can ask them for four to give a quick overview of the episode. Key takeaways afterwards also to create a note for you. So this is really like very open, open ended. And yeah. And then finally, the snipping feature that we mentioned just to reiterate. Yeah. I mean, here the the feature is that whenever you hear an amazing idea, you can trip. It's up your headphones or click a button in the app and the A.I. summarizes the insight you just heard and saves that together with the original transcript and audio in your knowledge library. I also noticed that you you skip dynamic content. So dynamic content, we do not skip it automatically. Oh, sorry. You detect. But we detect it. Yeah. I mean, that's one of the thing that most people don't don't actually know that like the way that ads get inserted into podcasts or into most podcasts is actually that every time you listen. To a podcast, you actually get access to a different audio file and on the server, a different ad is inserted into the MP3 file automatically. Yeah. Based on IP. Exactly. And that's what that means is if we transcribe an episode and have a transcript with timestamps like words, word specific timestamps, if you suddenly get a different audio file, like the whole time says I messed up and that's like a huge issue. And for that, we actually had to build another algorithm that would dynamically on the floor. I re sync the audio that you're listening to the transcript that we have. Yeah. Which is a fascinating problem in and of itself.swyx [00:27:24]: You sync by matching up the sound waves? Or like, or do you sync by matching up words like you basically do partial transcription?Kevin [00:27:33]: We are not matching up words. It's happening on the basically a bytes level matching. Yeah. Okay.swyx [00:27:40]: It relies on this. It relies on the exact match at some point.Kevin [00:27:46]: So it's actually. We're actually not doing exact matches, but we're doing fuzzy matches to identify the moment. It's basically, we basically built Shazam for podcasts. Just as a little side project to solve this issue.swyx [00:28:02]: Actually, fun fact, apparently the Shazam algorithm is open. They published the paper, it's talked about it. I haven't really dived into the paper. I thought it was kind of interesting that basically no one else has built Shazam.Kevin [00:28:16]: Yeah, I mean, well, the one thing is the algorithm. If you now talk about Shazam, the other thing is also having the database behind it and having the user mindset that if they have this problem, they come to you, right?swyx [00:28:29]: Yeah, I'm very interested in the tech stack. There's a big data pipeline. Could you share what is the tech stack?Kevin [00:28:35]: What are the most interesting or challenging pieces of it? So the general tech stack is our entire backend is, or 90% of our backend is written in Python. Okay. Hosting everything on Google Cloud Platform. And our front end is written with, well, we're using the Flutter framework. So it's written in Dart and then compiled natively. So we have one code base that handles both Android and iOS. You think that was a good decision? It's something that a lot of people are exploring. So up until now, yes. Okay. Look, it has its pros and cons. Some of the, you know, for example, earlier, I mentioned we have a Apple Watch app. Yeah. I mean, there's no Flutter for that, right? So that you build native. And then of course you have to sort of like sync these things together. I mean, I'm not the front end engineer, so I'm not just relaying this information, but our front end engineers are very happy with it. It's enabled us to be quite fast and be on both platforms from the very beginning. And when I talk with people and they hear that we are using Flutter, usually they think like, ah, it's not performant. It's super junk, janky and everything. And then they use it. They use our app and they're always super surprised. Or if they've already used our app, I couldn't tell them. They're like, what? Yeah. Um, so there is actually a lot that you can do with it.swyx [00:29:51]: The danger, the concern, there's a few concerns, right? One, it's Google. So when were they, when are they going to abandon it? Two, you know, they're optimized for Android first. So iOS is like a second, second thought, or like you can feel that it is not a native iOS app. Uh, but you guys put a lot of care into it. And then maybe three, from my point of view, JavaScript, as a JavaScript guy, React Native was supposed to be there. And I think that it hasn't really fulfilled that dream. Um, maybe Expo is trying to do that, but, um, again, it is not, does not feel as productive as Flutter. And I've, I spent a week on Flutter and dot, and I'm an investor in Flutter flow, which is the local, uh, Flutter, Flutter startup. That's doing very, very well. I think a lot of people are still Flutter skeptics. Yeah. Wait. So are you moving away from Flutter?Kevin [00:30:41]: I don't know. We don't have plans to do that. Yeah.swyx [00:30:43]: You're just saying about that. What? Yeah. Watch out. Okay. Let's go back to the stack.Kevin [00:30:47]: You know, that was just to give you a bit of an overview. I think the more interesting things are, of course, on the AI side. So we, like, as I mentioned earlier, when we started out, it was before chat GPT for the chat GPT moment before there was the GPT 3.5 turbo, uh, API. So in the beginning, we actually were running everything ourselves, open source models, try to fine tune them. They worked. There was us, but let's, let's be honest. They weren't. What was the sort of? Before Whisper, the transcription. Yeah, we were using wave to work like, um, there was a Google one, right? No, it was a Facebook, Facebook one. That was actually one of the papers. Like when that came out for me, that was one of the reasons why I said we, we should try something to start a startup in the audio space. For me, it was a bit like before that I had been following the NLP space, uh, quite closely. And as, as I mentioned earlier, we, we did some stuff at the startup as well, that I was working up. But before, and wave to work was the first paper that I had at least seen where the whole transformer architecture moved over to audio and bit more general way of saying it is like, it was the first time that I saw the transformer architecture being applied to continuous data instead of discrete tokens. Okay. And it worked amazingly. Ah, and like the transformer architecture plus self-supervised learning, like these two things moved over. And then for me, it was like, Hey, this is now going to take off similarly. It's the text space has taken off. And with these two things in place, even if some features that we want to build are not possible yet, they will be possible in the near term, uh, with this, uh, trajectory. So that was a little side, side note. No, it's in the meantime. Yeah. We're using whisper. We're still hosting some of the models ourselves. So for example, the whole transcription speaker diarization pipeline, uh,swyx [00:32:38]: You need it to be as cheap as possible.Kevin [00:32:40]: Yeah, exactly. I mean, we're doing this at scale where we have a lot of audio.swyx [00:32:44]: We're what numbers can you disclose? Like what, what are just to give people an idea because it's a lot. So we have more than a million podcasts that we've already processed when you say a million. So processing is basically, you have some kind of list of podcasts that you will auto process and others where a paying pay member can choose to press the button and transcribe it. Right. Is that the rough idea? Yeah, exactly.Kevin [00:33:08]: Yeah. And if, when you press that button or we also transcribe it. Yeah. So first we do the, we do the transcription. We do the. The, the speaker diarization. So basically you identify speech blocks that belong to the same speaker. This is then all orchestrated within, within LLM to identify which speech speech block belongs to which speaker together with, you know, we identify, as I mentioned earlier, we identify the guest name and the bio. So all of that comes together with an LLM to actually then assign assigned speaker names to, to each block. Yeah. And then most of the rest of the, the pipeline we've now used, we've now migrated to LLM. So we use mainly open AI, Google models, so the Gemini models and the open AI models, and we use some perplexity basically for those things where we need, where we need web search. Yeah. That's something I'm still hoping, especially open AI will also provide us an API. Oh, why? Well, basically for us as a consumer, the more providers there are.swyx [00:34:07]: The more downtime.Kevin [00:34:08]: The more competition and it will lead to better, better results. And, um, lower costs over time. I don't, I don't see perplexity as expensive. If you use the web search, the price is like $5 per a thousand queries. Okay. Which is affordable. But, uh, if you compare that to just a normal LLM call, um, it's, it's, uh, much more expensive. Have you tried Exa? We've, uh, looked into it, but we haven't really tried it. Um, I mean, we, we started with perplexity and, uh, it works, it works well. And if I remember. Correctly, Exa is also a bit more expensive.swyx [00:34:45]: I don't know. I don't know. They seem to focus on the search thing as a search API, whereas perplexity, maybe more consumer-y business that is higher, higher margin. Like I'll put it like perplexity is trying to be a product, Exa is trying to be infrastructure. Yeah. So that, that'll be my distinction there. And then the other thing I will mention is Google has a search grounding feature. Yeah. Which you, which you might want. Yeah.Kevin [00:35:07]: Yeah. We've, uh, we've also tried that out. Um, not as good. So we, we didn't, we didn't go into. Too much detail in like really comparing it, like quality wise, because we actually already had the perplexity one and it, and it's, and it's working. Yeah. Um, I think also there, the price is actually higher than perplexity. Yeah. Really? Yeah.swyx [00:35:26]: Google should cut their prices.Kevin [00:35:29]: Maybe it was the same price. I don't want to say something incorrect, but it wasn't cheaper. It wasn't like compelling. And then, then there was no reason to switch. So, I mean, maybe like in general, like for us, given that we do work with a lot of content, price is actually something that we do look at. Like for us, it's not just about taking the best model for every task, but it's really getting the best, like identifying what kind of intelligence level you need and then getting the best price for that to be able to really scale this and, and provide us, um, yeah, let our users use these features with as many podcasts as possible. Yeah.swyx [00:36:03]: I wanted to double, double click on diarization. Yeah. Uh, it's something that I don't think people do very well. So you know, I'm, I'm a, I'm a B user. I don't have it right now. And, and they were supposed to speak, but they dropped out last minute. Um, but, uh, we've had them on the podcast before and it's not great yet. Do you use just PI Anode, the default stuff, or do you find any tricks for diarization?Kevin [00:36:27]: So we do use the, the open source packages, but we have tweaked it a bit here and there. For example, if you mentioned the BAI guys, I actually listened to the podcast episode was super nice. Thank you. And when you started talking about speaker diarization, and I just have to think about, uh, I don't know.Kevin [00:36:49]: Is it possible? I don't know. I don't know. F**k this. Yeah, no, I don't know.Kevin [00:36:55]: Yeah. We are the best. This is a.swyx [00:37:07]: I don't know. This is the best. I don't know. This is the best. Yeah. Yeah. Yeah. You're doing good.Kevin [00:37:12]: So, so yeah. This is great. This is good. Yeah. No, so that of course helps us. Another thing that helps us is that we know certain structural aspects of the podcast. For example, how often does someone speak? Like if someone, like let's say there's a one hour episode and someone speaks for 30 seconds, that person is most probably not the guest and not the host. It's probably some ad, like some speaker from an ad. So we have like certain of these heuristics that we can use and we leverage to improve things. And in the past, we've also changed the clustering algorithm. So basically how a lot of the speaker diarization works is you basically create an embedding for the speech that's happening. And then you try to somehow cluster these embeddings and then find out this is all one speaker. This is all another speaker. And there we've also tweaked a couple of things where we again used heuristics that we could apply from knowing how podcasts function. And that's also actually why I was feeling so much with the BAI guys, because like all of these heuristics, like for them, it's probably almost impossible to use any heuristics because it can just be any situation, anything.Kevin [00:38:34]: So that's one thing that we do. Yeah, another thing is that we actually combine it with LLM. So the transcript, LLMs and the speaker diarization, like bringing all of these together to recalibrate some of the switching points. Like when does the speaker stop? When does the next one start?swyx [00:38:51]: The LLMs can add errors as well. You know, I wouldn't feel safe using them to be so precise.Kevin [00:38:58]: I mean, at the end of the day, like also just to not give a wrong impression, like the speaker diarization is also not perfect that we're doing, right? I basically don't really notice it.swyx [00:39:08]: Like I use it for search.Kevin [00:39:09]: Yeah, it's not perfect yet, but it's gotten quite good. Like, especially if you compare, if you look at some of the, like if you take a latest episode and you compare it to an episode that came out a year ago, we've improved it quite a bit.swyx [00:39:23]: Well, it's beautifully presented. Oh, I love that I can click on the transcript and it goes to the timestamp. So simple, but you know, it should exist. Yeah, I agree. I agree. So this, I'm loading a two hour episode of Detect Me Right Home, where there's a lot of different guests calling in and you've identified the guest name. And yeah, so these are all LLM based. Yeah, it's really nice.Kevin [00:39:49]: Yeah, like the speaker names.swyx [00:39:50]: I would say that, you know, obviously I'm a power user of all these tools. You have done a better job than Descript. Okay, wow. Descript is so much funding. They had their open AI invested in them and they still suck. So I don't know, like, you know, keep going. You're doing great. Yeah, thanks. Thanks.Kevin [00:40:12]: I mean, I would, I would say that, especially for anyone listening who's interested in building a consumer app with AI, I think the, like, especially if your background is in AI and you love working with AI and doing all of that, I think the most important thing is just to keep reminding yourself of what's actually the job to be done here. Like, what does actually the consumer want? Like, for example, you now were just delighted by the ability to click on this word and it jumps there. Yeah. Like, this is not, this is not rocket science. This is, like, you don't have to be, like, I don't know, Android Kapathi to come up with that and build that, right? And I think that's, that's something that's super important to keep in mind.swyx [00:40:52]: Yeah, yeah. Amazing. I mean, there's so many features, right? It's, it's so packed. There's quotes that you pick up. There's summarization. Oh, by the way, I'm going to use this as my official feature request. I want to customize what, how it's summarized. I want to, I want to have a custom prompt. Yeah. Because your summarization is good, but, you know, I have different preferences, right? Like, you know.Kevin [00:41:14]: So one thing that you can already do today, I completely get your feature request. And I think it just.swyx [00:41:18]: I'm sure people have asked it.Kevin [00:41:19]: I mean, maybe just in general as a, as a, how I see the future, you know, like in the future, I think all, everything will be personalized. Yeah, yeah. Like, not, this is not specific to us. Yeah. And today we're still in a, in a phase where the cost of LLMs, at least if you're working with, like, such long context windows. As us, I mean, there's a lot of tokens in, if you take an entire podcast, so you still have to take that cost into consideration. So if for every single user, we regenerate it entirely, it gets expensive. But in the future, this, you know, cost will continue to go down and then it will just be personalized. So that being said, you can already today, if you go to the player screen. Okay. And open up the chat. Yeah. You can go to the, to the chat. Yes. And just ask for a summary in your style.swyx [00:42:13]: Yeah. Okay. I mean, I, I listen to consume, you know? Yeah. Yeah. I, I've never really used this feature. I don't know. I think that's, that's me being a slow adopter. No, no. I mean, that's. It has, when does the conversation start? Okay.Kevin [00:42:26]: I mean, you can just type anything. I think what you're, what you're describing, I mean, maybe that is also an interesting topic to talk about. Yes. Where, like, basically I told you, like, look, we have this chat. You can just ask for it. Yeah. And this is, this is how ChatGPT works today. But if you're building a consumer app, you have to move beyond the chat box. People do not want to always type out what they want. So your feature request was, even though theoretically it's already possible, what you are actually asking for is, hey, I just want to open up the app and it should just be there in a nicely formatted way. Beautiful way such that I can read it or consume it without any issues. Interesting. And I think that's in general where a lot of the, the. Opportunities lie currently in the market. If you want to build a consumer app, taking the capability and the intelligence, but finding out what the actual user interface is the best way how a user can engage with this intelligence in a natural way.swyx [00:43:24]: Is this something I've been thinking about as kind of like AI that's not in your face? Because right now, you know, we like to say like, oh, use Notion has Notion AI. And we have the little thing there. And there's, or like some other. Any other platform has like the sparkle magic wand emoji, like that's our AI feature. Use this. And it's like really in your face. A lot of people don't like it. You know, it should just kind of become invisible, kind of like an invisible AI.Kevin [00:43:49]: 100%. I mean, the, the way I see it as AI is, is the electricity of, of the future. And like no one, like, like we don't talk about, I don't know, this, this microphone uses electricity, this phone, you don't think about it that way. It's just in there, right? It's not an electricity enabled product. No, it's just a product. Yeah. It will be the same with AI. I mean, now. It's still a, something that you use to market your product. I mean, we do, we do the same, right? Because it's still something that people realize, ah, they're doing something new, but at some point, no, it'll just be a podcast app and it will be normal that it has all of this AI in there.swyx [00:44:24]: I noticed you do something interesting in your chat where you source the timestamps. Yeah. Is that part of this prompt? Is there a separate pipeline that adds source sources?Kevin [00:44:33]: This is, uh, actually part of the prompt. Um, so this is all prompt engine. Engineering, um, uh, you should be able to click on it. Yeah, I clicked on it. Um, this is all prompt engineering with how to provide the, the context, you know, we, because we provide all of the transcript, how to provide the context and then, yeah, I get them all to respond in a correct way with a certain format and then rendering that on the front end. This is one of the examples where I would say it's so easy to create like a quick demo of this. I mean, you can just go to chat to be deep, paste this thing in and say like, yeah, do this. Okay. Like 15 minutes and you're done. Yeah. But getting this to like then production level that it actually works 99% of the time. Okay. This is then where, where the difference lies. Yeah. So, um, for this specific feature, like we actually also have like countless regexes that they're just there to correct certain things that the LLM is doing because it doesn't always adhere to the format correctly. And then it looks super ugly on the front end. So yeah, we have certain regexes that correct that. And maybe you'd ask like, why don't you use an LLM for that? Because that's sort of the, again, the AI native way, like who uses regexes anymore. But with the chat for user experience, it's very important that you have the streaming because otherwise you need to wait so long until your message has arrived. So we're streaming live the, like, just like ChatGPT, right? You get the answer and it's streaming the text. So if you're streaming the text and something is like incorrect. It's currently not easy to just like pipe, like stream this into another stream, stream this into another stream and get the stream back, which corrects it, that would be amazing. I don't know, maybe you can answer that. Do you know of any?swyx [00:46:19]: There's no API that does this. Yeah. Like you cannot stream in. If you own the models, you can, uh, you know, whatever token sequence has, has been emitted, start loading that into the next one. If you fully own the models, uh, I don't, it's probably not worth it. That's what you do. It's better. Yeah. I think. Yeah. Most engineers who are new to AI research and benchmarking actually don't know how much regexing there is that goes on in normal benchmarks. It's just like this ugly list of like a hundred different, you know, matches for some criteria that you're looking for. No, it's very cool. I think it's, it's, it's an example of like real world engineering. Yeah. Do you have a tooling that you're proud of that you've developed for yourself?Kevin [00:47:02]: Is it just a test script or is it, you know? I think it's a bit more, I guess the term that has come up is, uh, vibe coding, uh, vibe coding, some, no, sorry, that's actually something else in this case, but, uh, no, no, yes, um, vibe evals was a term that in one of the talks actually on, on, um, I think it might've been the first, the first or the first day at the conference, someone brought that up. Yeah. Uh, because yeah, a lot of the talks were about evals, right. Which is so important. And yeah, I think for us, it's a bit more vibe. Evals, you know, that's also part of, you know, being a startup, we can take risks, like we can take the cost of maybe sometimes it failing a little bit or being a little bit off and our users know that and they appreciate that in return, like we're moving fast and iterating and building, building amazing things, but you know, a Spotify or something like that, half of our features will probably be in a six month review through legal or I don't know what, uh, before they could sell them out.swyx [00:48:04]: Let's just say Spotify is not very good at podcasting. Um, I have a documented, uh, dislike for, for their podcast features, just overall, really, really well integrated any other like sort of LLM focused engineering challenges or problems that you, that you want to highlight.Kevin [00:48:20]: I think it's not unique to us, but it goes again in the direction of handling the uncertainty of LLMs. So for example, with last year, at the end of the year, we did sort of a snipped wrapped. And one of the things we thought it would be fun to, just to do something with, uh, with an LLM and something with the snips that, that a user has. And, uh, three, let's say unique LLM features were that we assigned a personality to you based on the, the snips that, that you have. It was, I mean, it was just all, I guess, a bit of a fun, playful way. I'm going to look up mine. I forgot mine already.swyx [00:48:57]: Um, yeah, I don't know whether it's actually still in the, in the, we all took screenshots of it.Kevin [00:49:01]: Ah, we posted it in the, in the discord. And the, the second one, it was, uh, we had a learning scorecard where we identified the topics that you snipped on the most, and you got like a little score for that. And the third one was a, a quote that stood out. And the quote is actually a very good example of where we would run that for user. And most of the time it was an interesting quote, but every now and then it was like a super boring quotes that you think like, like how, like, why did you select that? Like, come on for there. The solution was actually just to say, Hey, give me five. So it extracted five quotes as a candidate, and then we piped it into a different model as a judge, LLM as a judge, and there we use a, um, a much better model because with the, the initial model, again, as, as I mentioned also earlier, we do have to look at the, like the, the costs because it's like, we have so much text that goes into it. So we, there we use a bit more cheaper model, but then the judge can be like a really good model to then just choose one out of five. This is a practical example.swyx [00:50:03]: I can't find it. Bad search in discord. Yeah. Um, so, so you do recommend having a much smarter model as a judge, uh, and that works for you. Yeah. Yeah. Interesting. I think this year I'm very interested in LM as a judge being more developed as a concept, I think for things like, you know, snips, raps, like it's, it's fine. Like, you know, it's, it's, it's, it's entertaining. There's no right answer.Kevin [00:50:29]: I mean, we also have it. Um, we also use the same concept for our books feature where we identify the, the mention. Books. Yeah. Because there it's the same thing, like 90% of the time it, it works perfectly out of the box one shot and every now and then it just, uh, starts identifying books that were not really mentioned or that are not books or made, yeah, starting to make up books. And, uh, they are basically, we have the same thing of like another LLM challenging it. Um, yeah. And actually with the speakers, we do the same now that I think about it. Yeah. Um, so I'm, I think it's a, it's a great technique. Interesting.swyx [00:51:05]: You run a lot of calls.Kevin [00:51:07]: Yeah.swyx [00:51:08]: Okay. You know, you mentioned costs. You move from self hosting a lot of models to the, to the, you know, big lab models, open AI, uh, and Google, uh, non-topic.Kevin [00:51:18]: Um, no, we love Claude. Like in my opinion, Claude is the, the best one when it comes to the way it formulates things. The personality. Yeah. The personality. Okay. I actually really love it. But yeah, the cost is. It's still high.swyx [00:51:36]: So you cannot, you tried Haiku, but you're, you're like, you have to have Sonnet.Kevin [00:51:40]: Uh, like basically we like with Haiku, we haven't experimented too much. We obviously work a lot with 3.5 Sonnet. Uh, also, you know, coding. Yeah. For coding, like in cursor, just in general, also brainstorming. We use it a lot. Um, I think it's a great brainstorm partner, but yeah, with, uh, with, with a lot of things that we've done done, we opted for different models.swyx [00:52:00]: What I'm trying to drive at is how much cheaper can you get if you go from cloud to cloud? Closed models to open models. And maybe it's like 0% cheaper, maybe it's 5% cheaper, or maybe it's like 50% cheaper. Do you have a sense?Kevin [00:52:13]: It's very difficult to, to judge that. I don't really have a sense, but I can, I can give you a couple of thoughts that have gone through our minds over the time, because obviously we do realize like, given that we, we have a couple of tasks where there are just so many tokens going in, um, at some point it will make sense to, to offload some of that. Uh, to an open source model, but going back to like, we're, we're a startup, right? Like we're not an AI lab or whatever, like for us, actually the most important thing is to iterate fast because we need to learn from our users, improve that. And yeah, just this velocity of this, these iterations. And for that, the closed models hosted by open AI, Google is, uh, and swapping, they're just unbeatable because you just, it's just an API call. Yeah. Um, so you don't need to worry about. Yeah. So much complexity behind that. So this is, I would say the biggest reason why we're not doing more in this space, but there are other thoughts, uh, also for the future. Like I see two different, like we basically have two different usage patterns of LLMs where one is this, this pre-processing of a podcast episode, like this initial processing, like the transcription, speaker diarization, chapterization. We do that once. And this, this usage pattern it's, it's quite predictable. Because we know how many podcasts get released when, um, so we can sort of have a certain capacity and we can, we, we're running that 24 seven, it's one big queue running 24 seven.swyx [00:53:44]: What's the queue job runner? Uh, is it a Django, just like the Python one?Kevin [00:53:49]: No, that, that's just our own, like our database and the backend talking to the database, picking up jobs, finding it back. I'm just curious in orchestration and queues. I mean, we, we of course have like, uh, a lot of other orchestration where we're, we're, where we use, uh, the Google pub sub, uh, thing, but okay. So we have this, this, this usage pattern of like very predictable, uh, usage, and we can max out the, the usage. And then there's this other pattern where it's, for example, the snippet where it's like a user, it's a user action that triggers an LLM call and it has to be real time. And there can be moments where it's by usage and there can be moments when there's very little usage for that. There. So that's, that's basically where these LLM API calls are just perfect because you don't need to worry about scaling this up, scaling this down, um, handling, handling these issues. Serverless versus serverful.swyx [00:54:44]: Yeah, exactly. Okay.Kevin [00:54:45]: Like I see them a bit, like I see open AI and all of these other providers, I see them a bit as the, like as the Amazon, sorry, AWS of, of AI. So it's a bit similar how like back before AWS, you would have to have your, your servers and buy new servers or get rid of servers. And then with AWS, it just became so much easier to just ramp stuff up and down. Yeah. And this is like the taking it even, even, uh, to the next level for AI. Yeah.swyx [00:55:18]: I am a big believer in this. Basically it's, you know, intelligence on demand. Yeah. We're probably not using it enough in our daily lives to do things. I should, we should be able to spin up a hundred things at once and go through things and then, you know, stop. And I feel like we're still trying to figure out how to use LLMs in our lives effectively. Yeah. Yeah.Kevin [00:55:38]: 100%. I think that goes back to the whole, like that, that's for me where the big opportunity is for, if you want to do a startup, um, it's not about, but you can let the big labs handleswyx [00:55:48]: the challenge of more intelligence, but, um, it's the... Existing intelligence. How do you integrate? How do you actually incorporate it into your life? AI engineering. Okay, cool. Cool. Cool. Cool. Um, the one, one other thing I wanted to touch on was multimodality in frontier models. Dwarcash had a interesting application of Gemini recently where he just fed raw audio in and got diarized transcription out or timestamps out. And I think that will come. So basically what we're saying here is another wave of transformers eating things because right now models are pretty much single modality things. You know, you have whisper, you have a pipeline and everything. Yeah. You can't just say, Oh, no, no, no, we only fit like the raw, the raw files. Do you think that will be realistic for you? I 100% agree. Okay.Kevin [00:56:38]: Basically everything that we talked about earlier with like the speaker diarization and heuristics and everything, I completely agree. Like in the, in the future that would just be put everything into a big multimodal LLM. Okay. And it will output, uh, everything that you want. Yeah. So I've also experimented with that. Like just... With, with Gemini 2? With Gemini 2.0 Flash. Yeah. Just for fun. Yeah. Yeah. Because the big difference right now is still like the cost difference of doing speaker diarization this way or doing transcription this way is a huge difference to the pipeline that we've built up. Huh. Okay.swyx [00:57:15]: I need to figure out what, what that cost is because in my mind 2.0 Flash is so cheap. Yeah. But maybe not cheap enough for you.Kevin [00:57:23]: Uh, no, I mean, if you compare it to, yeah, whisper and speaker diarization and especially self-hosting it and... Yeah. Yeah. Yeah.swyx [00:57:30]: Yeah.Kevin [00:57:30]: Okay. But we will get there, right? Like this is just a question of time.swyx [00:57:33]: And, um, at some point, as soon as that happens, we'll be the first ones to switch. Yeah. Awesome. Anything else that you're like sort of eyeing on the horizon as like, we are thinking about this feature, we're thinking about incorporating this new functionality of AI into our, into our app? Yeah.Kevin [00:57:50]: I mean, we, there's so many areas that we're thinking about, like our challenge is a bit more... Choosing. Yeah. Choosing. Yeah. So, I mean, I think for me, like looking into like the next couple of years, like the big areas that interest us a lot, basically four areas, like one is content. Um, right now it's, it's podcasts. I mean, you did mention, I think you mentioned like you can also upload audio books and YouTube videos. YouTube. I actually use the YouTube one a fair amount. But in the future, we, we want to also have audio books natively in the app. And, uh, we want to enable AI generated content. Like just think of, take deep research and notebook analysis. Like put these together. That should be, that should be in our app. The second area is discovery. I think in general. Yeah.swyx [00:58:38]: I noticed that you don't have, so you

Guest: Blake Scholl, Founder & CEO of Boom Supersonic“Passion and drive trumps knowledge and experience,” says Boom Supersonic CEO Blake Scholl. Long before he was running Boom — which earlier this year successfully tested the world's first privately-developed supersonic jet — he was enabling “the world's most obnoxious spam cannon” at Groupon, or designing a barcode-scanning game for retail shoppers.But eventually, Blake found the courage to be more audacious and do something closer to his lifelong love of aviation. He began educating himself about things he had never thought to learn, and tapping his LinkedIn network to get intros to the smartest people in the industry. “If you imagine yourself on like the day of IPO, 99 percent of what you needed to know to get to that day, you didn't know on day one,” he says. “So, why not take 99 percent to 99.5 percent, and work on the thing you really want to exist, even if you don't know anything about it yet?”Chapters: (01:07) - Blake on Boom's beginnings (01:52) - Breaking the sound barrier (05:23) - Concorde's legacy (09:36) - Navigating regulations (12:08) - Boomless supersonic flight (16:48) - The test flight (20:11) - Day-of nervousness (24:26) - Carrying passengers (26:55) - Cost & wi-fi (30:19) - “No middle seats” (32:35) - Hard tech (36:48) - What if Apple made a plane? (39:08) - Blake's career journey (43:29) - The risk of failure (49:12) - Finding the courage (52:49) - Balancing life with Boom (56:42) - Learning how to build a jet (01:00:20) - The power of LinkedIn (01:02:38) - Y Combinator Demo Day (01:08:24) - Richard Branson (01:11:38) - Dividing yourself (01:14:19) - Being a focused dad (01:20:05) - Exuberance vs. fear (01:24:15) - Hiring slowly (01:27:17) - What “grit” means to Blake Mentioned in this episode: Chuck Yeager, ChatGPT, the Apollo program, Elon Musk, SpaceX and Falcon 1, Boom Overture, Starlink, Boeing, Airbus, iPhone, Jony Ive, Uber, Airbnb, Anduril, United Airlines, American Airlines, Eclipse Aviation, Tesla, Scott Kirby, Mike Leskinen, Inktomi, Yahoo!, Amazon, Pelago, Google Ads, Kima Labs, Barcode Hero, Groupon, iPad, Eric Schmidt, Steve Jobs, Khan Academy, Sam Altman, Loopt, Virgin Atlantic, Paul Graham, Michael Seibel, Ashlee Vance, Bloomberg, Hacker News, Jared Friedman, Sen. Mark Kelly, SV Angel, Ron Conway, Virgin Galactic, Lockheed Martin, Gulfstream, Jeff Bezos, Jeff Holden, and How It's Made.Links:Connect with BlakeTwitterLinkedInConnect with JoubinTwitterLinkedInEmail: grit@kleinerperkins.com Learn more about Kleiner PerkinsThis episode was edited by Eric Johnson from LightningPod.fm

Nikolay and Michael use a recent "best practices" article as a prompt — giving a few tips each on the topics mentioned, like schema design, performance, backups, and more. Here are some links to things they mentioned:7 Crucial PostgreSQL Best Practices (recent blog post) https://speakdatascience.com/postgresql-best-practices“Don't do this” episode https://postgres.fm/episodes/dont-do-thisArticle discussion on Hacker News https://news.ycombinator.com/item?id=42992913Mozilla's SQL Style Guide https://docs.telemetry.mozilla.org/concepts/sql_style“SQL vs NoSQL” episode with Franck Pachot https://postgres.fm/episodes/sql-vs-nosqlHA episode https://postgres.fm/episodes/high-availability ~~~What did you like or not like? What should we discuss next time? Let us know via a YouTube comment, on social media, or by commenting on our Google doc!~~~Postgres FM is produced by:Michael Christofides, founder of pgMustardNikolay Samokhvalov, founder of Postgres.aiWith credit to:Jessie Draws for the elephant artwork

Ivan Zhao is the co-founder and CEO of Notion. Ivan shares the untold story of Notion, from nearly running out of database space during Covid to finding product-market fit after several “lost years,” and the hard-won lessons along the way.—What you'll learn:1. Why you sometimes need to “hide your vision” behind something people actually want—what Ivan calls “sugar-coating the broccoli”2. How Ivan and his co-founder persevered through multiple product resets and complete code rewrites3. Why Notion prioritized systems over headcount, keeping the team small and focused even at scale4. Why Ivan believes in craft and values as the foundation for product development, balancing technical excellence with aesthetic sensibility5. The surprising story of how Notion nearly collapsed during Covid when their single database almost ran out of space with only weeks to spare6. Community-led growth tactics7. Ivan's unique journey from a small town in China8. Much more—Brought to you by:• Eppo—Run reliable, impactful experiments• Airtable ProductCentral—Launch to new heights with a unified system for product development• Sinch—Build messaging, email, and calling into your product—Find the transcript at: https://www.lennysnewsletter.com/p/inside-notion-ivan-zhao—Where to find Ivan Zhao:• X: https://x.com/ivanhzhao• LinkedIn: https://www.linkedin.com/in/ivanhzhao/—Where to find Lenny:• Newsletter: https://www.lennysnewsletter.com• X: https://twitter.com/lennysan• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/—In this episode, we cover:(00:00) Introduction to Ivan Zhao(04:41) Ivan's early life and education(07:46) Discovering the vision for Notion(10:49) The lost years of Notion(13:56) Rebuilding and perseverance(17:14) Layoffs and company morale(18:53) Advice for startup founders(25:08) Product-market fit(29:56) Staying lean and efficient(34:27) Creating a unique office culture(37:20) Craft and values: the foundation of Notion's philosophy(38:44) Navigating tradeoffs in product and business building(41:24) Leadership and personal growth(49:11) Challenges and crises: lessons from Notion's journey(51:08) Building horizontal software: joys and pains(01:02:40) Philosophy of tools and human potential(01:06:17) Lightning round and final thoughts—Referenced:• Ürümqi: https://en.wikipedia.org/wiki/%C3%9Cr%C3%BCmqi• Notion: https://www.notion.com/• SpongeBob SquarePants: https://en.wikipedia.org/wiki/SpongeBob_SquarePants• Augmenting Human Intellect: https://web.stanford.edu/class/history34q/readings/Engelbart/Engelbart_AugmentIntellect.html• Alan Kay: https://en.wikipedia.org/wiki/Alan_Kay• Ted Nelson: https://en.wikipedia.org/wiki/Ted_Nelson• Steve Jobs on Why Computers Are Like a Bicycle for the Mind (1990): https://www.themarginalian.org/2011/12/21/steve-jobs-bicycle-for-the-mind-1990/• Xerox Alto: https://en.wikipedia.org/wiki/Xerox_Alto• React: https://react.dev/• Simon Last on LinkedIn: https://www.linkedin.com/in/simon-last-41404140/• Magna-Tiles: https://www.magnatiles.com/• Design on a deadline: How Notion pulled itself back from the brink of failure: https://www.figma.com/blog/design-on-a-deadline-how-notion-pulled-itself-back-from-the-brink-of-failure/• Bryan Johnson on X: https://x.com/bryan_johnson• Tobi Lütke's leadership playbook: Playing infinite games, operating from first principles, and maximizing human potential (founder and CEO of Shopify): https://www.lennysnewsletter.com/p/tobi-lutkes-leadership-playbook• Smalltalk: https://en.wikipedia.org/wiki/Smalltalk#:• Lisp: https://en.wikipedia.org/wiki/Lisp_(programming_language)• DeepSeek: https://www.deepseek.com/• Shana Fisher: https://www.crunchbase.com/person/shana-fisher• LAMY 2000 fountain pens: https://www.jetpens.com/LAMY-2000-Fountain-Pens/• Macintosh 128K: https://en.wikipedia.org/wiki/Macintosh_128K• Toshiba rice cooker: https://www.toshiba-lifestyle.com/us/cooking-appliances/rice-cooker• Transistor radio: https://en.wikipedia.org/wiki/Transistor_radio• Jira: https://www.atlassian.com/software/jira• Salesforce: https://www.salesforce.com/• HubSpot: https://www.hubspot.com/• Zendesk: https://www.zendesk.com/• Misattributed McLuhan quote: https://mcluhangalaxy.wordpress.com/2013/04/01/we-shape-our-tools-and-thereafter-our-tools-shape-us/• Phin Barnes on LinkedIn: https://www.linkedin.com/in/phineasbarnes/• Hacker News: https://news.ycombinator.com/• Pablo Picasso quote: https://www.goodreads.com/quotes/629531-good-artists-copy-great-artists-steal#:~• Connections with James Burke on Prime Video: https://www.amazon.com/gp/video/detail/amzn1.dv.gti.484e32c5-60bd-4493-a800-e44fd0940312• The Enneagram Institute: https://www.enneagraminstitute.com/—Recommended book:• The Romance of the Three Kingdoms: https://www.amazon.com/Romance-Three-Kingdoms-Luo-Guanzhong/dp/024133277X—Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.—Lenny may be an investor in the companies discussed. Get full access to Lenny's Newsletter at www.lennysnewsletter.com/subscribe

Today's episode is with Paul Klein, founder of Browserbase. We talked about building browser infrastructure for AI agents, the future of agent authentication, and their open source framework Stagehand.* [00:00:00] Introductions* [00:04:46] AI-specific challenges in browser infrastructure* [00:07:05] Multimodality in AI-Powered Browsing* [00:12:26] Running headless browsers at scale* [00:18:46] Geolocation when proxying* [00:21:25] CAPTCHAs and Agent Auth* [00:28:21] Building “User take over” functionality* [00:33:43] Stagehand: AI web browsing framework* [00:38:58] OpenAI's Operator and computer use agents* [00:44:44] Surprising use cases of Browserbase* [00:47:18] Future of browser automation and market competition* [00:53:11] Being a solo founderTranscriptAlessio [00:00:04]: Hey everyone, welcome to the Latent Space podcast. This is Alessio, partner and CTO at Decibel Partners, and I'm joined by my co-host Swyx, founder of Smol.ai.swyx [00:00:12]: Hey, and today we are very blessed to have our friends, Paul Klein, for the fourth, the fourth, CEO of Browserbase. Welcome.Paul [00:00:21]: Thanks guys. Yeah, I'm happy to be here. I've been lucky to know both of you for like a couple of years now, I think. So it's just like we're hanging out, you know, with three ginormous microphones in front of our face. It's totally normal hangout.swyx [00:00:34]: Yeah. We've actually mentioned you on the podcast, I think, more often than any other Solaris tenant. Just because like you're one of the, you know, best performing, I think, LLM tool companies that have started up in the last couple of years.Paul [00:00:50]: Yeah, I mean, it's been a whirlwind of a year, like Browserbase is actually pretty close to our first birthday. So we are one years old. And going from, you know, starting a company as a solo founder to... To, you know, having a team of 20 people, you know, a series A, but also being able to support hundreds of AI companies that are building AI applications that go out and automate the web. It's just been like, really cool. It's been happening a little too fast. I think like collectively as an AI industry, let's just take a week off together. I took my first vacation actually two weeks ago, and Operator came out on the first day, and then a week later, DeepSeat came out. And I'm like on vacation trying to chill. I'm like, we got to build with this stuff, right? So it's been a breakneck year. But I'm super happy to be here and like talk more about all the stuff we're seeing. And I'd love to hear kind of what you guys are excited about too, and share with it, you know?swyx [00:01:39]: Where to start? So people, you've done a bunch of podcasts. I think I strongly recommend Jack Bridger's Scaling DevTools, as well as Turner Novak's The Peel. And, you know, I'm sure there's others. So you covered your Twilio story in the past, talked about StreamClub, you got acquired to Mux, and then you left to start Browserbase. So maybe we just start with what is Browserbase? Yeah.Paul [00:02:02]: Browserbase is the web browser for your AI. We're building headless browser infrastructure, which are browsers that run in a server environment that's accessible to developers via APIs and SDKs. It's really hard to run a web browser in the cloud. You guys are probably running Chrome on your computers, and that's using a lot of resources, right? So if you want to run a web browser or thousands of web browsers, you can't just spin up a bunch of lambdas. You actually need to use a secure containerized environment. You have to scale it up and down. It's a stateful system. And that infrastructure is, like, super painful. And I know that firsthand, because at my last company, StreamClub, I was CTO, and I was building our own internal headless browser infrastructure. That's actually why we sold the company, is because Mux really wanted to buy our headless browser infrastructure that we'd built. And it's just a super hard problem. And I actually told my co-founders, I would never start another company unless it was a browser infrastructure company. And it turns out that's really necessary in the age of AI, when AI can actually go out and interact with websites, click on buttons, fill in forms. You need AI to do all of that work in an actual browser running somewhere on a server. And BrowserBase powers that.swyx [00:03:08]: While you're talking about it, it occurred to me, not that you're going to be acquired or anything, but it occurred to me that it would be really funny if you became the Nikita Beer of headless browser companies. You just have one trick, and you make browser companies that get acquired.Paul [00:03:23]: I truly do only have one trick. I'm screwed if it's not for headless browsers. I'm not a Go programmer. You know, I'm in AI grant. You know, browsers is an AI grant. But we were the only company in that AI grant batch that used zero dollars on AI spend. You know, we're purely an infrastructure company. So as much as people want to ask me about reinforcement learning, I might not be the best guy to talk about that. But if you want to ask about headless browser infrastructure at scale, I can talk your ear off. So that's really my area of expertise. And it's a pretty niche thing. Like, nobody has done what we're doing at scale before. So we're happy to be the experts.swyx [00:03:59]: You do have an AI thing, stagehand. We can talk about the sort of core of browser-based first, and then maybe stagehand. Yeah, stagehand is kind of the web browsing framework. Yeah.What is Browserbase? Headless Browser Infrastructure ExplainedAlessio [00:04:10]: Yeah. Yeah. And maybe how you got to browser-based and what problems you saw. So one of the first things I worked on as a software engineer was integration testing. Sauce Labs was kind of like the main thing at the time. And then we had Selenium, we had Playbrite, we had all these different browser things. But it's always been super hard to do. So obviously you've worked on this before. When you started browser-based, what were the challenges? What were the AI-specific challenges that you saw versus, there's kind of like all the usual running browser at scale in the cloud, which has been a problem for years. What are like the AI unique things that you saw that like traditional purchase just didn't cover? Yeah.AI-specific challenges in browser infrastructurePaul [00:04:46]: First and foremost, I think back to like the first thing I did as a developer, like as a kid when I was writing code, I wanted to write code that did stuff for me. You know, I wanted to write code to automate my life. And I do that probably by using curl or beautiful soup to fetch data from a web browser. And I think I still do that now that I'm in the cloud. And the other thing that I think is a huge challenge for me is that you can't just create a web site and parse that data. And we all know that now like, you know, taking HTML and plugging that into an LLM, you can extract insights, you can summarize. So it was very clear that now like dynamic web scraping became very possible with the rise of large language models or a lot easier. And that was like a clear reason why there's been more usage of headless browsers, which are necessary because a lot of modern websites don't expose all of their page content via a simple HTTP request. You know, they actually do require you to run this type of code for a specific time. JavaScript on the page to hydrate this. Airbnb is a great example. You go to airbnb.com. A lot of that content on the page isn't there until after they run the initial hydration. So you can't just scrape it with a curl. You need to have some JavaScript run. And a browser is that JavaScript engine that's going to actually run all those requests on the page. So web data retrieval was definitely one driver of starting BrowserBase and the rise of being able to summarize that within LLM. Also, I was familiar with if I wanted to automate a website, I could write one script and that would work for one website. It was very static and deterministic. But the web is non-deterministic. The web is always changing. And until we had LLMs, there was no way to write scripts that you could write once that would run on any website. That would change with the structure of the website. Click the login button. It could mean something different on many different websites. And LLMs allow us to generate code on the fly to actually control that. So I think that rise of writing the generic automation scripts that can work on many different websites, to me, made it clear that browsers are going to be a lot more useful because now you can automate a lot more things without writing. If you wanted to write a script to book a demo call on 100 websites, previously, you had to write 100 scripts. Now you write one script that uses LLMs to generate that script. That's why we built our web browsing framework, StageHand, which does a lot of that work for you. But those two things, web data collection and then enhanced automation of many different websites, it just felt like big drivers for more browser infrastructure that would be required to power these kinds of features.Alessio [00:07:05]: And was multimodality also a big thing?Paul [00:07:08]: Now you can use the LLMs to look, even though the text in the dome might not be as friendly. Maybe my hot take is I was always kind of like, I didn't think vision would be as big of a driver. For UI automation, I felt like, you know, HTML is structured text and large language models are good with structured text. But it's clear that these computer use models are often vision driven, and they've been really pushing things forward. So definitely being multimodal, like rendering the page is required to take a screenshot to give that to a computer use model to take actions on a website. And it's just another win for browser. But I'll be honest, that wasn't what I was thinking early on. I didn't even think that we'd get here so fast with multimodality. I think we're going to have to get back to multimodal and vision models.swyx [00:07:50]: This is one of those things where I forgot to mention in my intro that I'm an investor in Browserbase. And I remember that when you pitched to me, like a lot of the stuff that we have today, we like wasn't on the original conversation. But I did have my original thesis was something that we've talked about on the podcast before, which is take the GPT store, the custom GPT store, all the every single checkbox and plugin is effectively a startup. And this was the browser one. I think the main hesitation, I think I actually took a while to get back to you. The main hesitation was that there were others. Like you're not the first hit list browser startup. It's not even your first hit list browser startup. There's always a question of like, will you be the category winner in a place where there's a bunch of incumbents, to be honest, that are bigger than you? They're just not targeted at the AI space. They don't have the backing of Nat Friedman. And there's a bunch of like, you're here in Silicon Valley. They're not. I don't know.Paul [00:08:47]: I don't know if that's, that was it, but like, there was a, yeah, I mean, like, I think I tried all the other ones and I was like, really disappointed. Like my background is from working at great developer tools, companies, and nothing had like the Vercel like experience. Um, like our biggest competitor actually is partly owned by private equity and they just jacked up their prices quite a bit. And the dashboard hasn't changed in five years. And I actually used them at my last company and tried them and I was like, oh man, like there really just needs to be something that's like the experience of these great infrastructure companies, like Stripe, like clerk, like Vercel that I use in love, but oriented towards this kind of like more specific category, which is browser infrastructure, which is really technically complex. Like a lot of stuff can go wrong on the internet when you're running a browser. The internet is very vast. There's a lot of different configurations. Like there's still websites that only work with internet explorer out there. How do you handle that when you're running your own browser infrastructure? These are the problems that we have to think about and solve at BrowserBase. And it's, it's certainly a labor of love, but I built this for me, first and foremost, I know it's super cheesy and everyone says that for like their startups, but it really, truly was for me. If you look at like the talks I've done even before BrowserBase, and I'm just like really excited to try and build a category defining infrastructure company. And it's, it's rare to have a new category of infrastructure exists. We're here in the Chroma offices and like, you know, vector databases is a new category of infrastructure. Is it, is it, I mean, we can, we're in their office, so, you know, we can, we can debate that one later. That is one.Multimodality in AI-Powered Browsingswyx [00:10:16]: That's one of the industry debates.Paul [00:10:17]: I guess we go back to the LLMOS talk that Karpathy gave way long ago. And like the browser box was very clearly there and it seemed like the people who were building in this space also agreed that browsers are a core primitive of infrastructure for the LLMOS that's going to exist in the future. And nobody was building something there that I wanted to use. So I had to go build it myself.swyx [00:10:38]: Yeah. I mean, exactly that talk that, that honestly, that diagram, every box is a startup and there's the code box and then there's the. The browser box. I think at some point they will start clashing there. There's always the question of the, are you a point solution or are you the sort of all in one? And I think the point solutions tend to win quickly, but then the only ones have a very tight cohesive experience. Yeah. Let's talk about just the hard problems of browser base you have on your website, which is beautiful. Thank you. Was there an agency that you used for that? Yeah. Herb.paris.Paul [00:11:11]: They're amazing. Herb.paris. Yeah. It's H-E-R-V-E. I highly recommend for developers. Developer tools, founders to work with consumer agencies because they end up building beautiful things and the Parisians know how to build beautiful interfaces. So I got to give prep.swyx [00:11:24]: And chat apps, apparently are, they are very fast. Oh yeah. The Mistral chat. Yeah. Mistral. Yeah.Paul [00:11:31]: Late chat.swyx [00:11:31]: Late chat. And then your videos as well, it was professionally shot, right? The series A video. Yeah.Alessio [00:11:36]: Nico did the videos. He's amazing. Not the initial video that you shot at the new one. First one was Austin.Paul [00:11:41]: Another, another video pretty surprised. But yeah, I mean, like, I think when you think about how you talk about your company. You have to think about the way you present yourself. It's, you know, as a developer, you think you evaluate a company based on like the API reliability and the P 95, but a lot of developers say, is the website good? Is the message clear? Do I like trust this founder? I'm building my whole feature on. So I've tried to nail that as well as like the reliability of the infrastructure. You're right. It's very hard. And there's a lot of kind of foot guns that you run into when running headless browsers at scale. Right.Competing with Existing Headless Browser Solutionsswyx [00:12:10]: So let's pick one. You have eight features here. Seamless integration. Scalability. Fast or speed. Secure. Observable. Stealth. That's interesting. Extensible and developer first. What comes to your mind as like the top two, three hardest ones? Yeah.Running headless browsers at scalePaul [00:12:26]: I think just running headless browsers at scale is like the hardest one. And maybe can I nerd out for a second? Is that okay? I heard this is a technical audience, so I'll talk to the other nerds. Whoa. They were listening. Yeah. They're upset. They're ready. The AGI is angry. Okay. So. So how do you run a browser in the cloud? Let's start with that, right? So let's say you're using a popular browser automation framework like Puppeteer, Playwright, and Selenium. Maybe you've written a code, some code locally on your computer that opens up Google. It finds the search bar and then types in, you know, search for Latent Space and hits the search button. That script works great locally. You can see the little browser open up. You want to take that to production. You want to run the script in a cloud environment. So when your laptop is closed, your browser is doing something. The browser is doing something. Well, I, we use Amazon. You can see the little browser open up. You know, the first thing I'd reach for is probably like some sort of serverless infrastructure. I would probably try and deploy on a Lambda. But Chrome itself is too big to run on a Lambda. It's over 250 megabytes. So you can't easily start it on a Lambda. So you maybe have to use something like Lambda layers to squeeze it in there. Maybe use a different Chromium build that's lighter. And you get it on the Lambda. Great. It works. But it runs super slowly. It's because Lambdas are very like resource limited. They only run like with one vCPU. You can run one process at a time. Remember, Chromium is super beefy. It's barely running on my MacBook Air. I'm still downloading it from a pre-run. Yeah, from the test earlier, right? I'm joking. But it's big, you know? So like Lambda, it just won't work really well. Maybe it'll work, but you need something faster. Your users want something faster. Okay. Well, let's put it on a beefier instance. Let's get an EC2 server running. Let's throw Chromium on there. Great. Okay. I can, that works well with one user. But what if I want to run like 10 Chromium instances, one for each of my users? Okay. Well, I might need two EC2 instances. Maybe 10. All of a sudden, you have multiple EC2 instances. This sounds like a problem for Kubernetes and Docker, right? Now, all of a sudden, you're using ECS or EKS, the Kubernetes or container solutions by Amazon. You're spending up and down containers, and you're spending a whole engineer's time on kind of maintaining this stateful distributed system. Those are some of the worst systems to run because when it's a stateful distributed system, it means that you are bound by the connections to that thing. You have to keep the browser open while someone is working with it, right? That's just a painful architecture to run. And there's all this other little gotchas with Chromium, like Chromium, which is the open source version of Chrome, by the way. You have to install all these fonts. You want emojis working in your browsers because your vision model is looking for the emoji. You need to make sure you have the emoji fonts. You need to make sure you have all the right extensions configured, like, oh, do you want ad blocking? How do you configure that? How do you actually record all these browser sessions? Like it's a headless browser. You can't look at it. So you need to have some sort of observability. Maybe you're recording videos and storing those somewhere. It all kind of adds up to be this just giant monster piece of your project when all you wanted to do was run a lot of browsers in production for this little script to go to google.com and search. And when I see a complex distributed system, I see an opportunity to build a great infrastructure company. And we really abstract that away with Browserbase where our customers can use these existing frameworks, Playwright, Publisher, Selenium, or our own stagehand and connect to our browsers in a serverless-like way. And control them, and then just disconnect when they're done. And they don't have to think about the complex distributed system behind all of that. They just get a browser running anywhere, anytime. Really easy to connect to.swyx [00:15:55]: I'm sure you have questions. My standard question with anything, so essentially you're a serverless browser company, and there's been other serverless things that I'm familiar with in the past, serverless GPUs, serverless website hosting. That's where I come from with Netlify. One question is just like, you promised to spin up thousands of servers. You promised to spin up thousands of browsers in milliseconds. I feel like there's no real solution that does that yet. And I'm just kind of curious how. The only solution I know, which is to kind of keep a kind of warm pool of servers around, which is expensive, but maybe not so expensive because it's just CPUs. So I'm just like, you know. Yeah.Browsers as a Core Primitive in AI InfrastructurePaul [00:16:36]: You nailed it, right? I mean, how do you offer a serverless-like experience with something that is clearly not serverless, right? And the answer is, you need to be able to run... We run many browsers on single nodes. We use Kubernetes at browser base. So we have many pods that are being scheduled. We have to predictably schedule them up or down. Yes, thousands of browsers in milliseconds is the best case scenario. If you hit us with 10,000 requests, you may hit a slower cold start, right? So we've done a lot of work on predictive scaling and being able to kind of route stuff to different regions where we have multiple regions of browser base where we have different pools available. You can also pick the region you want to go to based on like lower latency, round trip, time latency. It's very important with these types of things. There's a lot of requests going over the wire. So for us, like having a VM like Firecracker powering everything under the hood allows us to be super nimble and spin things up or down really quickly with strong multi-tenancy. But in the end, this is like the complex infrastructural challenges that we have to kind of deal with at browser base. And we have a lot more stuff on our roadmap to allow customers to have more levers to pull to exchange, do you want really fast browser startup times or do you want really low costs? And if you're willing to be more flexible on that, we may be able to kind of like work better for your use cases.swyx [00:17:44]: Since you used Firecracker, shouldn't Fargate do that for you or did you have to go lower level than that? We had to go lower level than that.Paul [00:17:51]: I find this a lot with Fargate customers, which is alarming for Fargate. We used to be a giant Fargate customer. Actually, the first version of browser base was ECS and Fargate. And unfortunately, it's a great product. I think we were actually the largest Fargate customer in our region for a little while. No, what? Yeah, seriously. And unfortunately, it's a great product, but I think if you're an infrastructure company, you actually have to have a deeper level of control over these primitives. I think it's the same thing is true with databases. We've used other database providers and I think-swyx [00:18:21]: Yeah, serverless Postgres.Paul [00:18:23]: Shocker. When you're an infrastructure company, you're on the hook if any provider has an outage. And I can't tell my customers like, hey, we went down because so-and-so went down. That's not acceptable. So for us, we've really moved to bringing things internally. It's kind of opposite of what we preach. We tell our customers, don't build this in-house, but then we're like, we build a lot of stuff in-house. But I think it just really depends on what is in the critical path. We try and have deep ownership of that.Alessio [00:18:46]: On the distributed location side, how does that work for the web where you might get sort of different content in different locations, but the customer is expecting, you know, if you're in the US, I'm expecting the US version. But if you're spinning up my browser in France, I might get the French version. Yeah.Paul [00:19:02]: Yeah. That's a good question. Well, generally, like on the localization, there is a thing called locale in the browser. You can set like what your locale is. If you're like in the ENUS browser or not, but some things do IP, IP based routing. And in that case, you may want to have a proxy. Like let's say you're running something in the, in Europe, but you want to make sure you're showing up from the US. You may want to use one of our proxy features so you can turn on proxies to say like, make sure these connections always come from the United States, which is necessary too, because when you're browsing the web, you're coming from like a, you know, data center IP, and that can make things a lot harder to browse web. So we do have kind of like this proxy super network. Yeah. We have a proxy for you based on where you're going, so you can reliably automate the web. But if you get scheduled in Europe, that doesn't happen as much. We try and schedule you as close to, you know, your origin that you're trying to go to. But generally you have control over the regions you can put your browsers in. So you can specify West one or East one or Europe. We only have one region of Europe right now, actually. Yeah.Alessio [00:19:55]: What's harder, the browser or the proxy? I feel like to me, it feels like actually proxying reliably at scale. It's much harder than spending up browsers at scale. I'm curious. It's all hard.Paul [00:20:06]: It's layers of hard, right? Yeah. I think it's different levels of hard. I think the thing with the proxy infrastructure is that we work with many different web proxy providers and some are better than others. Some have good days, some have bad days. And our customers who've built browser infrastructure on their own, they have to go and deal with sketchy actors. Like first they figure out their own browser infrastructure and then they got to go buy a proxy. And it's like you can pay in Bitcoin and it just kind of feels a little sus, right? It's like you're buying drugs when you're trying to get a proxy online. We have like deep relationships with these counterparties. We're able to audit them and say, is this proxy being sourced ethically? Like it's not running on someone's TV somewhere. Is it free range? Yeah. Free range organic proxies, right? Right. We do a level of diligence. We're SOC 2. So we have to understand what is going on here. But then we're able to make sure that like we route around proxy providers not working. There's proxy providers who will just, the proxy will stop working all of a sudden. And then if you don't have redundant proxying on your own browsers, that's hard down for you or you may get some serious impacts there. With us, like we intelligently know, hey, this proxy is not working. Let's go to this one. And you can kind of build a network of multiple providers to really guarantee the best uptime for our customers. Yeah. So you don't own any proxies? We don't own any proxies. You're right. The team has been saying who wants to like take home a little proxy server, but not yet. We're not there yet. You know?swyx [00:21:25]: It's a very mature market. I don't think you should build that yourself. Like you should just be a super customer of them. Yeah. Scraping, I think, is the main use case for that. I guess. Well, that leads us into CAPTCHAs and also off, but let's talk about CAPTCHAs. You had a little spiel that you wanted to talk about CAPTCHA stuff.Challenges of Scaling Browser InfrastructurePaul [00:21:43]: Oh, yeah. I was just, I think a lot of people ask, if you're thinking about proxies, you're thinking about CAPTCHAs too. I think it's the same thing. You can go buy CAPTCHA solvers online, but it's the same buying experience. It's some sketchy website, you have to integrate it. It's not fun to buy these things and you can't really trust that the docs are bad. What Browserbase does is we integrate a bunch of different CAPTCHAs. We do some stuff in-house, but generally we just integrate with a bunch of known vendors and continually monitor and maintain these things and say, is this working or not? Can we route around it or not? These are CAPTCHA solvers. CAPTCHA solvers, yeah. Not CAPTCHA providers, CAPTCHA solvers. Yeah, sorry. CAPTCHA solvers. We really try and make sure all of that works for you. I think as a dev, if I'm buying infrastructure, I want it all to work all the time and it's important for us to provide that experience by making sure everything does work and monitoring it on our own. Yeah. Right now, the world of CAPTCHAs is tricky. I think AI agents in particular are very much ahead of the internet infrastructure. CAPTCHAs are designed to block all types of bots, but there are now good bots and bad bots. I think in the future, CAPTCHAs will be able to identify who a good bot is, hopefully via some sort of KYC. For us, we've been very lucky. We have very little to no known abuse of Browserbase because we really look into who we work with. And for certain types of CAPTCHA solving, we only allow them on certain types of plans because we want to make sure that we can know what people are doing, what their use cases are. And that's really allowed us to try and be an arbiter of good bots, which is our long term goal. I want to build great relationships with people like Cloudflare so we can agree, hey, here are these acceptable bots. We'll identify them for you and make sure we flag when they come to your website. This is a good bot, you know?Alessio [00:23:23]: I see. And Cloudflare said they want to do more of this. So they're going to set by default, if they think you're an AI bot, they're going to reject. I'm curious if you think this is something that is going to be at the browser level or I mean, the DNS level with Cloudflare seems more where it should belong. But I'm curious how you think about it.Paul [00:23:40]: I think the web's going to change. You know, I think that the Internet as we have it right now is going to change. And we all need to just accept that the cat is out of the bag. And instead of kind of like wishing the Internet was like it was in the 2000s, we can have free content line that wouldn't be scraped. It's just it's not going to happen. And instead, we should think about like, one, how can we change? How can we change the models of, you know, information being published online so people can adequately commercialize it? But two, how do we rebuild applications that expect that AI agents are going to log in on their behalf? Those are the things that are going to allow us to kind of like identify good and bad bots. And I think the team at Clerk has been doing a really good job with this on the authentication side. I actually think that auth is the biggest thing that will prevent agents from accessing stuff, not captchas. And I think there will be agent auth in the future. I don't know if it's going to happen from an individual company, but actually authentication providers that have a, you know, hidden login as agent feature, which will then you put in your email, you'll get a push notification, say like, hey, your browser-based agent wants to log into your Airbnb. You can approve that and then the agent can proceed. That really circumvents the need for captchas or logging in as you and sharing your password. I think agent auth is going to be one way we identify good bots going forward. And I think a lot of this captcha solving stuff is really short-term problems as the internet kind of reorients itself around how it's going to work with agents browsing the web, just like people do. Yeah.Managing Distributed Browser Locations and Proxiesswyx [00:24:59]: Stitch recently was on Hacker News for talking about agent experience, AX, which is a thing that Netlify is also trying to clone and coin and talk about. And we've talked about this on our previous episodes before in a sense that I actually think that's like maybe the only part of the tech stack that needs to be kind of reinvented for agents. Everything else can stay the same, CLIs, APIs, whatever. But auth, yeah, we need agent auth. And it's mostly like short-lived, like it should not, it should be a distinct, identity from the human, but paired. I almost think like in the same way that every social network should have your main profile and then your alt accounts or your Finsta, it's almost like, you know, every, every human token should be paired with the agent token and the agent token can go and do stuff on behalf of the human token, but not be presumed to be the human. Yeah.Paul [00:25:48]: It's like, it's, it's actually very similar to OAuth is what I'm thinking. And, you know, Thread from Stitch is an investor, Colin from Clerk, Octaventures, all investors in browser-based because like, I hope they solve this because they'll make browser-based submission more possible. So we don't have to overcome all these hurdles, but I think it will be an OAuth-like flow where an agent will ask to log in as you, you'll approve the scopes. Like it can book an apartment on Airbnb, but it can't like message anybody. And then, you know, the agent will have some sort of like role-based access control within an application. Yeah. I'm excited for that.swyx [00:26:16]: The tricky part is just, there's one, one layer of delegation here, which is like, you're authoring my user's user or something like that. I don't know if that's tricky or not. Does that make sense? Yeah.Paul [00:26:25]: You know, actually at Twilio, I worked on the login identity and access. Management teams, right? So like I built Twilio's login page.swyx [00:26:31]: You were an intern on that team and then you became the lead in two years? Yeah.Paul [00:26:34]: Yeah. I started as an intern in 2016 and then I was the tech lead of that team. How? That's not normal. I didn't have a life. He's not normal. Look at this guy. I didn't have a girlfriend. I just loved my job. I don't know. I applied to 500 internships for my first job and I got rejected from every single one of them except for Twilio and then eventually Amazon. And they took a shot on me and like, I was getting paid money to write code, which was my dream. Yeah. Yeah. I'm very lucky that like this coding thing worked out because I was going to be doing it regardless. And yeah, I was able to kind of spend a lot of time on a team that was growing at a company that was growing. So it informed a lot of this stuff here. I think these are problems that have been solved with like the SAML protocol with SSO. I think it's a really interesting stuff with like WebAuthn, like these different types of authentication, like schemes that you can use to authenticate people. The tooling is all there. It just needs to be tweaked a little bit to work for agents. And I think the fact that there are companies that are already. Providing authentication as a service really sets it up. Well, the thing that's hard is like reinventing the internet for agents. We don't want to rebuild the internet. That's an impossible task. And I think people often say like, well, we'll have this second layer of APIs built for agents. I'm like, we will for the top use cases, but instead of we can just tweak the internet as is, which is on the authentication side, I think we're going to be the dumb ones going forward. Unfortunately, I think AI is going to be able to do a lot of the tasks that we do online, which means that it will be able to go to websites, click buttons on our behalf and log in on our behalf too. So with this kind of like web agent future happening, I think with some small structural changes, like you said, it feels like it could all slot in really nicely with the existing internet.Handling CAPTCHAs and Agent Authenticationswyx [00:28:08]: There's one more thing, which is the, your live view iframe, which lets you take, take control. Yeah. Obviously very key for operator now, but like, was, is there anything interesting technically there or that the people like, well, people always want this.Paul [00:28:21]: It was really hard to build, you know, like, so, okay. Headless browsers, you don't see them, right. They're running. They're running in a cloud somewhere. You can't like look at them. And I just want to really make, it's a weird name. I wish we came up with a better name for this thing, but you can't see them. Right. But customers don't trust AI agents, right. At least the first pass. So what we do with our live view is that, you know, when you use browser base, you can actually embed a live view of the browser running in the cloud for your customer to see it working. And that's what the first reason is the build trust, like, okay, so I have this script. That's going to go automate a website. I can embed it into my web application via an iframe and my customer can watch. I think. And then we added two way communication. So now not only can you watch the browser kind of being operated by AI, if you want to pause and actually click around type within this iframe that's controlling a browser, that's also possible. And this is all thanks to some of the lower level protocol, which is called the Chrome DevTools protocol. It has a API called start screencast, and you can also send mouse clicks and button clicks to a remote browser. And this is all embeddable within iframes. You have a browser within a browser, yo. And then you simulate the screen, the click on the other side. Exactly. And this is really nice often for, like, let's say, a capture that can't be solved. You saw this with Operator, you know, Operator actually uses a different approach. They use VNC. So, you know, you're able to see, like, you're seeing the whole window here. What we're doing is something a little lower level with the Chrome DevTools protocol. It's just PNGs being streamed over the wire. But the same thing is true, right? Like, hey, I'm running a window. Pause. Can you do something in this window? Human. Okay, great. Resume. Like sometimes 2FA tokens. Like if you get that text message, you might need a person to type that in. Web agents need human-in-the-loop type workflows still. You still need a person to interact with the browser. And building a UI to proxy that is kind of hard. You may as well just show them the whole browser and say, hey, can you finish this up for me? And then let the AI proceed on afterwards. Is there a future where I stream my current desktop to browser base? I don't think so. I think we're very much cloud infrastructure. Yeah. You know, but I think a lot of the stuff we're doing, we do want to, like, build tools. Like, you know, we'll talk about the stage and, you know, web agent framework in a second. But, like, there's a case where a lot of people are going desktop first for, you know, consumer use. And I think cloud is doing a lot of this, where I expect to see, you know, MCPs really oriented around the cloud desktop app for a reason, right? Like, I think a lot of these tools are going to run on your computer because it makes... I think it's breaking out. People are putting it on a server. Oh, really? Okay. Well, sweet. We'll see. We'll see that. I was surprised, though, wasn't I? I think that the browser company, too, with Dia Browser, it runs on your machine. You know, it's going to be...swyx [00:30:50]: What is it?Paul [00:30:51]: So, Dia Browser, as far as I understand... I used to use Arc. Yeah. I haven't used Arc. But I'm a big fan of the browser company. I think they're doing a lot of cool stuff in consumer. As far as I understand, it's a browser where you have a sidebar where you can, like, chat with it and it can control the local browser on your machine. So, if you imagine, like, what a consumer web agent is, which it lives alongside your browser, I think Google Chrome has Project Marina, I think. I almost call it Project Marinara for some reason. I don't know why. It's...swyx [00:31:17]: No, I think it's someone really likes the Waterworld. Oh, I see. The classic Kevin Costner. Yeah.Paul [00:31:22]: Okay. Project Marinara is a similar thing to the Dia Browser, in my mind, as far as I understand it. You have a browser that has an AI interface that will take over your mouse and keyboard and control the browser for you. Great for consumer use cases. But if you're building applications that rely on a browser and it's more part of a greater, like, AI app experience, you probably need something that's more like infrastructure, not a consumer app.swyx [00:31:44]: Just because I have explored a little bit in this area, do people want branching? So, I have the state. Of whatever my browser's in. And then I want, like, 100 clones of this state. Do people do that? Or...Paul [00:31:56]: People don't do it currently. Yeah. But it's definitely something we're thinking about. I think the idea of forking a browser is really cool. Technically, kind of hard. We're starting to see this in code execution, where people are, like, forking some, like, code execution, like, processes or forking some tool calls or branching tool calls. Haven't seen it at the browser level yet. But it makes sense. Like, if an AI agent is, like, using a website and it's not sure what path it wants to take to crawl this website. To find the information it's looking for. It would make sense for it to explore both paths in parallel. And that'd be a very, like... A road not taken. Yeah. And hopefully find the right answer. And then say, okay, this was actually the right one. And memorize that. And go there in the future. On the roadmap. For sure. Don't make my roadmap, please. You know?Alessio [00:32:37]: How do you actually do that? Yeah. How do you fork? I feel like the browser is so stateful for so many things.swyx [00:32:42]: Serialize the state. Restore the state. I don't know.Paul [00:32:44]: So, it's one of the reasons why we haven't done it yet. It's hard. You know? Like, to truly fork, it's actually quite difficult. The naive way is to open the same page in a new tab and then, like, hope that it's at the same thing. But if you have a form halfway filled, you may have to, like, take the whole, you know, container. Pause it. All the memory. Duplicate it. Restart it from there. It could be very slow. So, we haven't found a thing. Like, the easy thing to fork is just, like, copy the page object. You know? But I think there needs to be something a little bit more robust there. Yeah.swyx [00:33:12]: So, MorphLabs has this infinite branch thing. Like, wrote a custom fork of Linux or something that let them save the system state and clone it. MorphLabs, hit me up. I'll be a customer. Yeah. That's the only. I think that's the only way to do it. Yeah. Like, unless Chrome has some special API for you. Yeah.Paul [00:33:29]: There's probably something we'll reverse engineer one day. I don't know. Yeah.Alessio [00:33:32]: Let's talk about StageHand, the AI web browsing framework. You have three core components, Observe, Extract, and Act. Pretty clean landing page. What was the idea behind making a framework? Yeah.Stagehand: AI web browsing frameworkPaul [00:33:43]: So, there's three frameworks that are very popular or already exist, right? Puppeteer, Playwright, Selenium. Those are for building hard-coded scripts to control websites. And as soon as I started to play with LLMs plus browsing, I caught myself, you know, code-genning Playwright code to control a website. I would, like, take the DOM. I'd pass it to an LLM. I'd say, can you generate the Playwright code to click the appropriate button here? And it would do that. And I was like, this really should be part of the frameworks themselves. And I became really obsessed with SDKs that take natural language as part of, like, the API input. And that's what StageHand is. StageHand exposes three APIs, and it's a super set of Playwright. So, if you go to a page, you may want to take an action, click on the button, fill in the form, etc. That's what the act command is for. You may want to extract some data. This one takes a natural language, like, extract the winner of the Super Bowl from this page. You can give it a Zod schema, so it returns a structured output. And then maybe you're building an API. You can do an agent loop, and you want to kind of see what actions are possible on this page before taking one. You can do observe. So, you can observe the actions on the page, and it will generate a list of actions. You can guide it, like, give me actions on this page related to buying an item. And you can, like, buy it now, add to cart, view shipping options, and pass that to an LLM, an agent loop, to say, what's the appropriate action given this high-level goal? So, StageHand isn't a web agent. It's a framework for building web agents. And we think that agent loops are actually pretty close to the application layer because every application probably has different goals or different ways it wants to take steps. I don't think I've seen a generic. Maybe you guys are the experts here. I haven't seen, like, a really good AI agent framework here. Everyone kind of has their own special sauce, right? I see a lot of developers building their own agent loops, and they're using tools. And I view StageHand as the browser tool. So, we expose act, extract, observe. Your agent can call these tools. And from that, you don't have to worry about it. You don't have to worry about generating playwright code performantly. You don't have to worry about running it. You can kind of just integrate these three tool calls into your agent loop and reliably automate the web.swyx [00:35:48]: A special shout-out to Anirudh, who I met at your dinner, who I think listens to the pod. Yeah. Hey, Anirudh.Paul [00:35:54]: Anirudh's a man. He's a StageHand guy.swyx [00:35:56]: I mean, the interesting thing about each of these APIs is they're kind of each startup. Like, specifically extract, you know, Firecrawler is extract. There's, like, Expand AI. There's a whole bunch of, like, extract companies. They just focus on extract. I'm curious. Like, I feel like you guys are going to collide at some point. Like, right now, it's friendly. Everyone's in a blue ocean. At some point, it's going to be valuable enough that there's some turf battle here. I don't think you have a dog in a fight. I think you can mock extract to use an external service if they're better at it than you. But it's just an observation that, like, in the same way that I see each option, each checkbox in the side of custom GBTs becoming a startup or each box in the Karpathy chart being a startup. Like, this is also becoming a thing. Yeah.Paul [00:36:41]: I mean, like, so the way StageHand works is that it's MIT-licensed, completely open source. You bring your own API key to your LLM of choice. You could choose your LLM. We don't make any money off of the extract or really. We only really make money if you choose to run it with our browser. You don't have to. You can actually use your own browser, a local browser. You know, StageHand is completely open source for that reason. And, yeah, like, I think if you're building really complex web scraping workflows, I don't know if StageHand is the tool for you. I think it's really more if you're building an AI agent that needs a few general tools or if it's doing a lot of, like, web automation-intensive work. But if you're building a scraping company, StageHand is not your thing. You probably want something that's going to, like, get HTML content, you know, convert that to Markdown, query it. That's not what StageHand does. StageHand is more about reliability. I think we focus a lot on reliability and less so on cost optimization and speed at this point.swyx [00:37:33]: I actually feel like StageHand, so the way that StageHand works, it's like, you know, page.act, click on the quick start. Yeah. It's kind of the integration test for the code that you would have to write anyway, like the Puppeteer code that you have to write anyway. And when the page structure changes, because it always does, then this is still the test. This is still the test that I would have to write. Yeah. So it's kind of like a testing framework that doesn't need implementation detail.Paul [00:37:56]: Well, yeah. I mean, Puppeteer, Playwright, and Slenderman were all designed as testing frameworks, right? Yeah. And now people are, like, hacking them together to automate the web. I would say, and, like, maybe this is, like, me being too specific. But, like, when I write tests, if the page structure changes. Without me knowing, I want that test to fail. So I don't know if, like, AI, like, regenerating that. Like, people are using StageHand for testing. But it's more for, like, usability testing, not, like, testing of, like, does the front end, like, has it changed or not. Okay. But generally where we've seen people, like, really, like, take off is, like, if they're using, you know, something. If they want to build a feature in their application that's kind of like Operator or Deep Research, they're using StageHand to kind of power that tool calling in their own agent loop. Okay. Cool.swyx [00:38:37]: So let's go into Operator, the first big agent launch of the year from OpenAI. Seems like they have a whole bunch scheduled. You were on break and your phone blew up. What's your just general view of computer use agents is what they're calling it. The overall category before we go into Open Operator, just the overall promise of Operator. I will observe that I tried it once. It was okay. And I never tried it again.OpenAI's Operator and computer use agentsPaul [00:38:58]: That tracks with my experience, too. Like, I'm a huge fan of the OpenAI team. Like, I think that I do not view Operator as the company. I'm not a company killer for browser base at all. I think it actually shows people what's possible. I think, like, computer use models make a lot of sense. And I'm actually most excited about computer use models is, like, their ability to, like, really take screenshots and reasoning and output steps. I think that using mouse click or mouse coordinates, I've seen that proved to be less reliable than I would like. And I just wonder if that's the right form factor. What we've done with our framework is anchor it to the DOM itself, anchor it to the actual item. So, like, if it's clicking on something, it's clicking on that thing, you know? Like, it's more accurate. No matter where it is. Yeah, exactly. Because it really ties in nicely. And it can handle, like, the whole viewport in one go, whereas, like, Operator can only handle what it sees. Can you hover? Is hovering a thing that you can do? I don't know if we expose it as a tool directly, but I'm sure there's, like, an API for hovering. Like, move mouse to this position. Yeah, yeah, yeah. I think you can trigger hover, like, via, like, the JavaScript on the DOM itself. But, no, I think, like, when we saw computer use, everyone's eyes lit up because they realized, like, wow, like, AI is going to actually automate work for people. And I think seeing that kind of happen from both of the labs, and I'm sure we're going to see more labs launch computer use models, I'm excited to see all the stuff that people build with it. I think that I'd love to see computer use power, like, controlling a browser on browser base. And I think, like, Open Operator, which was, like, our open source version of OpenAI's Operator, was our first take on, like, how can we integrate these models into browser base? And we handle the infrastructure and let the labs do the models. I don't have a sense that Operator will be released as an API. I don't know. Maybe it will. I'm curious to see how well that works because I think it's going to be really hard for a company like OpenAI to do things like support CAPTCHA solving or, like, have proxies. Like, I think it's hard for them structurally. Imagine this New York Times headline, OpenAI CAPTCHA solving. Like, that would be a pretty bad headline, this New York Times headline. Browser base solves CAPTCHAs. No one cares. No one cares. And, like, our investors are bored. Like, we're all okay with this, you know? We're building this company knowing that the CAPTCHA solving is short-lived until we figure out how to authenticate good bots. I think it's really hard for a company like OpenAI, who has this brand that's so, so good, to balance with, like, the icky parts of web automation, which it can be kind of complex to solve. I'm sure OpenAI knows who to call whenever they need you. Yeah, right. I'm sure they'll have a great partnership.Alessio [00:41:23]: And is Open Operator just, like, a marketing thing for you? Like, how do you think about resource allocation? So, you can spin this up very quickly. And now there's all this, like, open deep research, just open all these things that people are building. We started it, you know. You're the original Open. We're the original Open operator, you know? Is it just, hey, look, this is a demo, but, like, we'll help you build out an actual product for yourself? Like, are you interested in going more of a product route? That's kind of the OpenAI way, right? They started as a model provider and then…Paul [00:41:53]: Yeah, we're not interested in going the product route yet. I view Open Operator as a model provider. It's a reference project, you know? Let's show people how to build these things using the infrastructure and models that are out there. And that's what it is. It's, like, Open Operator is very simple. It's an agent loop. It says, like, take a high-level goal, break it down into steps, use tool calling to accomplish those steps. It takes screenshots and feeds those screenshots into an LLM with the step to generate the right action. It uses stagehand under the hood to actually execute this action. It doesn't use a computer use model. And it, like, has a nice interface using the live view that we talked about, the iframe, to embed that into an application. So I felt like people on launch day wanted to figure out how to build their own version of this. And we turned that around really quickly to show them. And I hope we do that with other things like deep research. We don't have a deep research launch yet. I think David from AOMNI actually has an amazing open deep research that he launched. It has, like, 10K GitHub stars now. So he's crushing that. But I think if people want to build these features natively into their application, they need good reference projects. And I think Open Operator is a good example of that.swyx [00:42:52]: I don't know. Actually, I'm actually pretty bullish on API-driven operator. Because that's the only way that you can sort of, like, once it's reliable enough, obviously. And now we're nowhere near. But, like, give it five years. It'll happen, you know. And then you can sort of spin this up and browsers are working in the background and you don't necessarily have to know. And it just is booking restaurants for you, whatever. I can definitely see that future happening. I had this on the landing page here. This might be a slightly out of order. But, you know, you have, like, sort of three use cases for browser base. Open Operator. Or this is the operator sort of use case. It's kind of like the workflow automation use case. And it completes with UiPath in the sort of RPA category. Would you agree with that? Yeah, I would agree with that. And then there's Agents we talked about already. And web scraping, which I imagine would be the bulk of your workload right now, right?Paul [00:43:40]: No, not at all. I'd say actually, like, the majority is browser automation. We're kind of expensive for web scraping. Like, I think that if you're building a web scraping product, if you need to do occasional web scraping or you have to do web scraping that works every single time, you want to use browser automation. Yeah. You want to use browser-based. But if you're building web scraping workflows, what you should do is have a waterfall. You should have the first request is a curl to the website. See if you can get it without even using a browser. And then the second request may be, like, a scraping-specific API. There's, like, a thousand scraping APIs out there that you can use to try and get data. Scraping B. Scraping B is a great example, right? Yeah. And then, like, if those two don't work, bring out the heavy hitter. Like, browser-based will 100% work, right? It will load the page in a real browser, hydrate it. I see.swyx [00:44:21]: Because a lot of people don't render to JS.swyx [00:44:25]: Yeah, exactly.Paul [00:44:26]: So, I mean, the three big use cases, right? Like, you know, automation, web data collection, and then, you know, if you're building anything agentic that needs, like, a browser tool, you want to use browser-based.Alessio [00:44:35]: Is there any use case that, like, you were super surprised by that people might not even think about? Oh, yeah. Or is it, yeah, anything that you can share? The long tail is crazy. Yeah.Surprising use cases of BrowserbasePaul [00:44:44]: One of the case studies on our website that I think is the most interesting is this company called Benny. So, the way that it works is if you're on food stamps in the United States, you can actually get rebates if you buy certain things. Yeah. You buy some vegetables. You submit your receipt to the government. They'll give you a little rebate back. Say, hey, thanks for buying vegetables. It's good for you. That process of submitting that receipt is very painful. And the way Benny works is you use their app to take a photo of your receipt, and then Benny will go submit that receipt for you and then deposit the money into your account. That's actually using no AI at all. It's all, like, hard-coded scripts. They maintain the scripts. They've been doing a great job. And they build this amazing consumer app. But it's an example of, like, all these, like, tedious workflows that people have to do to kind of go about their business. And they're doing it for the sake of their day-to-day lives. And I had never known about, like, food stamp rebates or the complex forms you have to do to fill them. But the world is powered by millions and millions of tedious forms, visas. You know, Emirate Lighthouse is a customer, right? You know, they do the O1 visa. Millions and millions of forms are taking away humans' time. And I hope that Browserbase can help power software that automates away the web forms that we don't need anymore. Yeah.swyx [00:45:49]: I mean, I'm very supportive of that. I mean, forms. I do think, like, government itself is a big part of it. I think the government itself should embrace AI more to do more sort of human-friendly form filling. Mm-hmm. But I'm not optimistic. I'm not holding my breath. Yeah. We'll see. Okay. I think I'm about to zoom out. I have a little brief thing on computer use, and then we can talk about founder stuff, which is, I tend to think of developer tooling markets in impossible triangles, where everyone starts in a niche, and then they start to branch out. So I already hinted at a little bit of this, right? We mentioned more. We mentioned E2B. We mentioned Firecrawl. And then there's Browserbase. So there's, like, all this stuff of, like, have serverless virtual computer that you give to an agent and let them do stuff with it. And there's various ways of connecting it to the internet. You can just connect to a search API, like SERP API, whatever other, like, EXA is another one. That's what you're searching. You can also have a JSON markdown extractor, which is Firecrawl. Or you can have a virtual browser like Browserbase, or you can have a virtual machine like Morph. And then there's also maybe, like, a virtual sort of code environment, like Code Interpreter. So, like, there's just, like, a bunch of different ways to tackle the problem of give a computer to an agent. And I'm just kind of wondering if you see, like, everyone's just, like, happily coexisting in their respective niches. And as a developer, I just go and pick, like, a shopping basket of one of each. Or do you think that you eventually, people will collide?Future of browser automation and market competitionPaul [00:47:18]: I think that currently it's not a zero-sum market. Like, I think we're talking about... I think we're talking about all of knowledge work that people do that can be automated online. All of these, like, trillions of hours that happen online where people are working. And I think that there's so much software to be built that, like, I tend not to think about how these companies will collide. I just try to solve the problem as best as I can and make this specific piece of infrastructure, which I think is an important primitive, the best I possibly can. And yeah. I think there's players that are actually going to like it. I think there's players that are going to launch, like, over-the-top, you know, platforms, like agent platforms that have all these tools built in, right? Like, who's building the rippling for agent tools that has the search tool, the browser tool, the operating system tool, right? There are some. There are some. There are some, right? And I think in the end, what I have seen as my time as a developer, and I look at all the favorite tools that I have, is that, like, for tools and primitives with sufficient levels of complexity, you need to have a solution that's really bespoke to that primitive, you know? And I am sufficiently convinced that the browser is complex enough to deserve a primitive. Obviously, I have to. I'm the founder of BrowserBase, right? I'm talking my book. But, like, I think maybe I can give you one spicy take against, like, maybe just whole OS running. I think that when I look at computer use when it first came out, I saw that the majority of use cases for computer use were controlling a browser. And do we really need to run an entire operating system just to control a browser? I don't think so. I don't think that's necessary. You know, BrowserBase can run browsers for way cheaper than you can if you're running a full-fledged OS with a GUI, you know, operating system. And I think that's just an advantage of the browser. It is, like, browsers are little OSs, and you can run them very efficiently if you orchestrate it well. And I think that allows us to offer 90% of the, you know, functionality in the platform needed at 10% of the cost of running a full OS. Yeah.Open Operator: Browserbase's Open-Source Alternativeswyx [00:49:16]: I definitely see the logic in that. There's a Mark Andreessen quote. I don't know if you know this one. Where he basically observed that the browser is turning the operating system into a poorly debugged set of device drivers, because most of the apps are moved from the OS to the browser. So you can just run browsers.Paul [00:49:31]: There's a place for OSs, too. Like, I think that there are some applications that only run on Windows operating systems. And Eric from pig.dev in this upcoming YC batch, or last YC batch, like, he's building all run tons of Windows operating systems for you to control with your agent. And like, there's some legacy EHR systems that only run on Internet-controlled systems. Yeah.Paul [00:49:54]: I think that's it. I think, like, there are use cases for specific operating systems for specific legacy software. And like, I'm excited to see what he does with that. I just wanted to give a shout out to the pig.dev website.swyx [00:50:06]: The pigs jump when you click on them. Yeah. That's great.Paul [00:50:08]: Eric, he's the former co-founder of banana.dev, too.swyx [00:50:11]: Oh, that Eric. Yeah. That Eric. Okay. Well, he abandoned bananas for pigs. I hope he doesn't start going around with pigs now.Alessio [00:50:18]: Like he was going around with bananas. A little toy pig. Yeah. Yeah. I love that. What else are we missing? I think we covered a lot of, like, the browser-based product history, but. What do you wish people asked you? Yeah.Paul [00:50:29]: I wish people asked me more about, like, what will the future of software look like? Because I think that's really where I've spent a lot of time about why do browser-based. Like, for me, starting a company is like a means of last resort. Like, you shouldn't start a company unless you absolutely have to. And I remain convinced that the future of software is software that you're going to click a button and it's going to do stuff on your behalf. Right now, software. You click a button and it maybe, like, calls it back an API and, like, computes some numbers. It, like, modifies some text, whatever. But the future of software is software using software. So, I may log into my accounting website for my business, click a button, and it's going to go load up my Gmail, search my emails, find the thing, upload the receipt, and then comment it for me. Right? And it may use it using APIs, maybe a browser. I don't know. I think it's a little bit of both. But that's completely different from how we've built software so far. And that's. I think that future of software has different infrastructure requirements. It's going to require different UIs. It's going to require different pieces of infrastructure. I think the browser infrastructure is one piece that fits into that, along with all the other categories you mentioned. So, I think that it's going to require developers to think differently about how they've built software for, you know

First off, this ain't just a gimmick—it's got real brains behind it. The app's using Google's Cloud Vision API to flex its grass-detecting skills, scanning your pics like a hawk to make sure you're not faking it with some sad houseplant or a green rug. Rumor has it, Rhys Kentish might even level up to a custom AI model down the line, so it's only gonna get smarter at sniffing out your outdoor vibes. That's next-gen accountability, fam—no cheating the system!For the eco-warriors out there, it's got a green heart. That 50% of premium skip profits goin' to UK wildlife and rewilding? It's not just talk—think restoring habitats, planting trees, and saving critters while you're saving yourself from scrollin' into oblivion. It's a double W: less screen time for you, more nature for the planet. Plus, the app's got a retro 8-bit pixelated vibe—like if Mario had to ditch the pipes and go hug a lawn—so it's got that quirky charm to keep things fun.Listeners should also know it's iOS-only for now, droppin' March 14, 2025, so Android peeps gotta sit tight. You'll need an iPhone with iOS 17 or later, and yeah, you gotta have actual grass nearby—no excuses, city dwellers might need to hunt for a park! The free version's chill with one skip a month, but premium's where the party's at: unlimited app blocks, screen time stats to flex your progress, and extra skips if you're feelin' lazy. It's perfect for anyone who's ever lost a whole morning to TikTok and thought, “Bruh, I need help.”Oh, and the hype's real—it blew up on X and Hacker News after Rhys dropped the teaser, so your listeners are gettin' in on somethin' that's already got the internet buzzin'. It's not just an app; it's a vibe shift—kickin' phone addiction to the curb, one grass pic at a time. Tell ‘em to pre-order it on the App Store and get ready to touch grass like it's a side quest in the game of life!

Michael Taylor has perfected the art of getting AI to speak in tongues. He's taught it to mimic the voices of your customers—so you can see how they would respond before you ship.Michael is the creator of Rally, a market research tool that lets you simulate an audience of AI personas. He built a simulator that lets us A/B test Every's headlines on an audience that mimics the real Hacker News audience. It's become a part of my writing workflow, and I love it because you test your assumptions quickly, cheaply, and without any of the risks of putting something out into the world.Besides Rally, Michael co-authored a book on prompt engineering for O'Reilly, and he writes a column for Every about managing AI tools like you would people. In a past life, he founded a growth marketing agency which he grew to 50 people and sold in 2020. One of the reasons I'm drawn to Michael's work is because he has a tinkerer's mindset. He's always exploring the limits of what a new technology can do, and what he's into today, everyone else will likely discover six months later. We spent an hour talking about using language models to judge your work, best practices for assessing an AI's performance, and Michael's flow inside Cursor. He also demos Rally live on the show, testing three different potential headlines for an Every article.If you found this episode interesting, please like, subscribe, comment, and share! Want even more?Sign up for Every to unlock our ultimate guide to prompting ChatGPT here: https://every.ck.page/ultimate-guide-to-prompting-chatgpt. It's usually only for paying subscribers, but you can get it here for free.To hear more from Dan Shipper:Subscribe to Every: https://every.to/subscribe Follow him on X: https://twitter.com/danshipper Timestamps:Introduction: 00:01:32AI can simulate human personalities with remarkable precision: 00:04:30How Michael simulated a Hacker News audience: 00:08:15Push AI to be a good judge of your work: 00:15:04Best practices to run evals: 00:19:00How AI compresses years of learning into shorter feedback loops: 00:23:01Why prompt engineering is becoming increasingly important: 00:27:01Adopting a new technology is about risk appetite: 00:44:59Michael demos Rally, his market research tool: 00:47:20The AI tools Michael uses to ship new features: 00:55:03Links to resources mentioned in the episode:  Michael Taylor: @hammer_mtJoin the waitlist for Rally, Michael's synthetic market research tool: https://askrally.com/ The book Michael co-authored on prompt engineering: Prompt Engineering for Generative AI The column Michael writes for Every: Also True for HumansMichael's article on personas of thought: “I Asked 100 AI Agents to Judge an Advertisement”Michael's article on building a Hacker News simulator: “I Created a Hacker News Simulator to Reverse-engineer Virality” 

Welcome back to Exploit Brokers! In today's video, we dive deep into a critical 7‑Zip vulnerability that's being exploited by Russian cybercriminals to bypass Windows' security protections. If you've used 7‑Zip at all, you need to know how this flaw can let hackers sneak past the Mark-of-the-Web (MOTW) and deploy dangerous malware like Smoke Loader. We'll also explore a parallel threat in the Go ecosystem—malicious packages exploiting caching mechanisms to gain persistent remote access to your system. From double-zipped archives to supply chain attacks, we break down the tactics, the risks, and most importantly, what you can do to protect yourself and your organization. In this video you'll learn: How the 7‑Zip vulnerability works and why updating to the latest version is crucial. The role of Windows' MOTW and how hackers are bypassing this key security feature. Details on the deployment of Smoke Loader malware and its implications. How malicious Go packages and supply chain attacks can compromise your systems. Practical tips to safeguard your data and networks against these emerging threats. Stay informed, stay secure—hit that like button, subscribe, and ring the bell for more cybersecurity insights! Drop your questions or thoughts in the comments below—we love hearing from you! #Cybersecurity #7Zip #WindowsSecurity #Malware #SmokeLoader #GoLang #SupplyChainAttack #Cybercrime #InfoSec #Hacking #RussianHackers #APT #NationStateHackers #exploits #ZeroDays

Tom MacWright is a prolific contributor in the geospatial open source community. He made geojson.io, Mapbox Studio, and was the lead developer on the OpenStreetMap editor. He's currently on the team at Val Town. In 2021 he bootstrapped a solo business and created the Placemark mapping application. He acquired customers and found steady growth but after spending two years on the project he decided it was financially unsustainable. He open sourced the code and shut down the business. In this interview Tom speaks candidly about why geospatial is difficult, chasing technical rabbit holes, the mental impact of bootstrapping, and his struggles to grow a customer base. If you're interested in geospatial or the good and bad of running a solo business I think you'll enjoy this conversation with Tom. Related Links Tom's blog Placemark Play Placemark GitHub Placemark archive geojson.io Valtown Datawrapper (Visualization tool) Geospatial Companies mentioned Mapbox ArcGIS QGIS Carto -- Transcript You can help correct transcripts on GitHub. [00:00:00] Introduction Jeremy: Today I'm talking to Tom MacWright. He worked at Mapbox as a, a very early employee. He's had a lot of experience in the geospatial community, the open source community. One of his most recent projects was a mapping project called Placemark he started and ran on his own. So I wanted to talk to Tom about his experience going solo and, eventually having to, shut that down. Tom, thanks for agreeing to chat today. Tom: Yeah, thanks for having me. [00:00:32] Tools and Open Source at Mapbox Jeremy: So maybe to give everyone some context on, what your background was before you started Placemark. Um, let's talk a little bit about your experience at, at Mapbox. What did you work on there and, and what would you say are like the big things you learned from that experience? Tom: Yeah, so if you include the time that I was at Development Seed, which essentially turned into Mapbox, I kind of signed the paper to get fired from Development Seed and hired at Mapbox within the same 20 seconds. Uh, I was there for eight and a half years. so it was a lifetime in tech years. and the company really evolved from, uh, working for Human Rights Watch and Amnesty International and the World Bank and doing these small, little like micro websites to the point at which I left it. It had. Raised a lot of money, had a lot of employees. I think it was 350 or so when I left. and yeah, just expanded into a lot of different, uh, try trying to own more and more of the mapping stack. but yeah, I was kind of really focused on the creative and tooling side of it. that's kind of where I see a lot of the, the fun and programming is making these tools where, uh, they can give people the same kind of fun like interaction loop that programming has where you, you know, you do a little bit of math and you see the result and you're able to just play with, uh, what you're working on, letting people have that in other domains. so it was really cool to figure out how to get A map design tool where somebody changes the background color and it just automatically changes that in your browser. and it covered like data editing. It covered, um, map styling and we did, uh, three different versions of that tool over the years. and then Mapbox is also a company that was, it came from, kind of people who are working on the Howard Dean campaign. And so it was pretty ideological and part of the ideology was being pretty hardcore about open source. we hired a lot of people who were working on open source projects before and basically just paid them to work on the open source projects, uh, for their whole time there. And during my time there, I just tried to make as much of my work, uh, open as possible, which was, you know, at the time it was, it was pretty great. I think in the long term it's been, o open source has changed a lot. but during the time that we were there, we both kind of, helped things like leaflet and mapnik and openstreetmap, uh, but also made like some larger contributions to the open source world. yeah, that, that's kind of like the, the internal company facing side. And also like what I try to create as like a more of a, uh, enduring work. I think the open source stuff will hopefully have more of a, a long term, uh, benefit. [00:03:40] How open source has changed (value capture by large companies) Jeremy: When I was working on a project that needed offline maps, um, we couldn't use Google Maps or any of the, the other publicly available, cloud APIs. So yeah, we actually used a, a tool, called Tile Mill that I, I hadn't known that you'd worked on, but recently found out you did. So that actually let us pull in OpenStreetMap data and then use this style, uh, language called carto to, to basically let us choose what the colors would be and how the different, uh, the roads and the buildings would look. What's kind of interesting to me is that it being open source really let us, um, build something we otherwise wouldn't have been able to do. But like, at the same time, we also didn't pay Mapbox any money. (laughs) So I'm, I'm kind of curious, like, if it's changed, like what the thinking was in terms of, you know, we pay for people to build all these things. We make it open source. but then people may just not ever pay us, you know, for all these things we did. Tom: Yeah. Yeah. I think that the main thing that's changed since the era of tilemill is, the dominance of cloud platforms. Like back then, I think, uh, Mapbox was still using, we were using like a little bit of AWS but people were still just on like VPSs and, uh, configuring things in cPanel and sometimes even running their own servers. And the, the danger of people using the product for free was such a small thing for us. especially when tile Mill was also funded by the Knight Foundation, so, you know, that at least paid half of my salary for, or, well, sorry, probably, yeah, maybe half of my salary for the first year that I was there and half of three other people's salaries. but that, yeah, so like when we built Tile Mill, a few companies have really like built on those same tools. Uh, there's a company called Carto coincidentally, they had the same name as Carto CSS, and they built on a lot of the same stack they built on mapnik. Um, and it was, was... I mean, I'm not gonna say that it was all like, you know, sunshine and roses, but it was never a thing that we talked about in terms of like this being a brutal competition between us and these other startups. Mapbox eventually closed source some stuff. they made it a source available license. and eventually Mapbox Studio was a closed source product. Um, and that was actually a decision that I advocated for. And that's mostly just because at one point, Esri, Microsoft, Amazon, all had whitelisted versions of Mapbox code, which, uh, hurts a little bit on a personal level and also makes it pretty hard to think about. working almost like it. You don't want to go to your scrappy open source company and do unpaid labor for Amazon. Uh, you know, Bezos can afford to pay for the labor himself. that's just kind of my personal, uh, that I'm obviously, I haven't worked there in a long time, so I'm not speaking for the company, but that's kind of how it felt like. and it yeah, kind of changed the arithmetic of open source in this way that. It made it less fun and, more risky, um, for people I think. [00:07:11] Don't worry about the small free users Jeremy: Yeah. So it sounds like the thinking was if someone on a small team or an individual, they took the open source software and they used it for their own projects, that was fine. Like you expected that and didn't worry about it. It's more that when these really large organizations like a, a Microsoft comes in and, just like you said, white labels the software, and doesn't really contribute significantly back. That's, that's when it, the, the thinking sort of shifted. Tom: Yeah, like a lot of the people who can't pay full price in USD to use your product are great users and they're doing cool stuff. Like when I was working on Placemark and when I was like selling. The theme for my blog, I would get emails from like some kid in India and it's like, you know, you're selling this for a hundred dollars, which is a ton of money. And like, you know, why, why should I care? Why shouldn't I like, just send them the zip file for free? it's like nothing to me and a lot to them. and mapping tools are really, really expensive. So the fact that Mapbox was able to create a free alternative when, you know, ArcGIS was $500 a month sometimes, um, depending on your license, obviously. That's, that's good. You're always gonna find a way for, like, your salespeople are gonna find a way to charge the big companies a lot of money. They're great at that. Um, and that's what matters really for your, for the revenue. [00:08:44] ESRI to Google Maps with little in-between Jeremy: That's a a good point too about like the, my impression of the, the mapping space, and maybe this has changed more recently, but you had the, probably the biggest player Esri, who's selling things at enterprise prices and then there were, or there are like a few open source options. but they feel like the, the barrier to entry feels a little high. And so, and then I guess you have stuff like Google Maps, right? That's, um, that's very accessible, but it's pretty limited, so. There's this big gap, it feels like right between the, the Esri and the, the Google Maps and open source. It's, it's sort of like, there's almost like there's no sweet spot. guess May, maybe it's just because people's uses are so different, but I'm, I'm not sure, um, what makes maps so unique in that way Tom: Yeah, I have come to understand what Esri and QGIS do as like an extension of what CAD is like. And if you've used CAD software recently, it's just as crazy and as expensive and as powerful. and it's really hard to capture like the people who are motivated enough to make a map but don't want to go down the whole rabbit hole. I think that was one of the hardest things about Placemark was trying to be in the middle of those things and half of the people were mystified by the complexity and half the people wanted more complexity. Uh, and I just couldn't figure out how to get it to the right in between spot. [00:10:25] Placemark and its origins in geojson.io Jeremy: Yeah. So let's, let's talk a little bit about Placemark then, in terms of from its start. What was your, your goal with Placemark and, and what was the product itself? Tom: So the seed of the idea for Placemark, uh, is this website called geojson.io, uh, which is still around. And, Chris Fong (correction -- Whong) at, at Mapbox is still, uh, developing it. And that had become pretty useful for a lot of people who I knew in the industry who were in this position of managing geospatial data but not wanting to boot up ArcGIS uh, geojson.io is based on, I just tweeted, I was like, why? Why is there not a thing where you can edit data on a map and have a GeoJSON representation and just go Back and forth between the two really easily. and it started with that, and then it kind of grew to be a little bit more powerful. And then it was just a tool that was useful for everyone. And my theory was just that I wanted that to be more useful. And I knew just like anything else that you build and you work on for a long time, you know exactly how it could be so much better. And, uh, all the things that you would do better if you did it again. And I was, uh, you know, hoping that there was something where like if you make that more powerful and you make it something that's like so essential that somebody's using every day, then maybe there's some some value in that. And so Placemark kind of started as being like, oh, this is the thing where if you're tasking a satellite and you need a bounding box on a specific city, this is the easiest way to do that. Um, and it grew a little bit into being like a tool for collaborating because people were collaborating on it. And I thought that that would be, you know, an interesting thing to support. but yeah, I think it, it like tried to be in that middle of like, not exactly Google my Maps and certainly a lot, uh, simpler than, uh, QGIS or ArcGIS Jeremy: something I noticed, so I've actually used geojson.io as well when I was first learning how to put stuff on a map and learning that GeoJSON was a format that a lot of things were using, it was actually really helpful to, to be able to draw, uh, polygons and see, okay, this is how the JSO looks and all that stuff. And it was. Like just very simple. I think there's something like very powerful about, websites or applications like that where it, it does this one thing and when you go there, you're like, oh, okay, I, I, I know what I'm doing and it's, it's, uh, you know, it's gonna help me do the, this very specific thing I'm trying to do. [00:13:16] Placemark use cases (Farming, Transportation, Interior mapping, Satellite viewsheds) Jeremy: I think with Placemark, so, one question I would have is, you gave an example of, uh, someone, I think you said for a satellite, they're, are they drawing the, the area? What, what was the area specifically for? Tom: the area of interest, the area where they want the, uh, to point the camera. Jeremy: so yeah, with, with Placemark, I mean, were there, what were some of the specific customers or use cases you had in mind? 'cause that's, that's something about. Um, placemark as a product I noticed was it's sort of like, here's this thing where you can draw polygons put markers and there's all these like things you can do, but I think unless you already have the specific use case, it's not super clear, who uses it for what. So maybe you could give some examples of what you had in mind. Tom: I didn't have much in mind, but I can tell you what people, what some people used it for. so some of the more interesting uses of it, a bunch of, uh, farming oriented use cases, uh, especially like indoor and small scale farming. Um, there were some people who, uh, essentially had a bunch of flower farms and had polygons on the map, and they wanted to, uh, mark the ones that had mites or needed to be watered, other things that could spread in a geometric way. And so it's pretty important to have that geospatial component to it. and then a few places were using it for basically transportation planning. Um, so drawing out routes of where buses would go, uh, in Luxembourg. And, then there was also a little bit of like, kind of interesting, planning of what to buy more or less. Uh, so something of like, do we want to buy this tract of land or do we wanna buy this tract of land or do we wanna buy access to this one high speed internet cable or this other high speed internet cable? and yeah, a lot of those things were kind of like emergent use cases. Um, there's a lot of people who were doing either architecture or internal or in interior mapping essentially. Jeremy: Interior, you mean, inside of a building Tom: yeah. yeah. Jeremy: Hmm. Okay. Tom: Which I don't think it was the best tool for. Uh, but you know, people used it for that. Jeremy: Interesting. Yeah. I guess, would people normally use some kind of a CAD tool for that, or Tom: Yeah. Uh, there's CAD tools and there are a few, uh, companies that do just, there's a company that just does interior maps especially of airports, and that's their whole business model. Um, but it's, it's kind of an interesting, uh, problem because most CAD architecture work is done with like a local coordinate system, and you have like very good resolution of everything, and then you eventually place it in geo geospatial space. Uh, but if you do it all in latitude and longitude, you know, you're, you're moving a door and it's moving the 10th or 12th decimal point, and eventually you have some precision problems. Jeremy: So it's almost like if you start with latitude and longitude, it's hard to go the other way. Right? you have to start more specific and then you can move it into the, the geospatial, uh, area. Tom: Yeah. Uh, that's kind of why we have local projections for towns is that you can do a lot of work just in that local projection. And the numbers are kind of small 'cause your town's small, relatively. Jeremy: yeah, those are kind of interesting. So it sounds like just anytime somebody wants to, like you gave the example of transportation planning or you want to visually see where things are, like your crops or things like that, and that, that kind of makes sense. I mean, I think if you just think about paper maps, if somebody wants to sketch something out and, and sort of track the layout of something, this could serve the same purpose but be editable. and like you said, I think it's also. Collaborative so you can have multiple people editing the same, um, map. that makes sense. I think something that I believe I saw on your website is you said though that it was, it's like an editing tool, but it's not necessarily a visualization tool. Uh, I'm kind of curious what you, what you meant by that. [00:17:39] An editing tool that allows you to export data not a visualization tool Tom: Yeah, I, when you say a map, I think there's, people can interpret that as everything from raw data to satellite imagery and raster data. and then a lot of it is like, can I use this to make a choropleth map of the voter turnout in our, in my country? and that placemark did a little bit, but I think that it was, it was never going to be the, the thing that it did super well. and so, yeah, and also like the, the two things kind of, don't mesh all that well. Like if you have a scale point map and you have that kind of visualization of it and then you're editing the points at the same time and you're dragging around these like gigantic points because this point means a lot of population, it just doesn't really make that much sense. There are probably ways to square that circle and have different views, but, uh, I felt like for visualizations, I mean partly I just think data wrapper is kind of great and uh, I had already worked for observable at that point, which is also, which I think also does like great visualization work. Jeremy: Would that be the case of somebody could make a map inside a placemark and then they would take the GeoJSON and then import that into another visualization tool? Is that what you were kind of imagining people would do? Tom: Yeah. Yeah, exactly. Jeremy: And I could see from the customer's perspective, a lot of them, they may have that end, uh, visualization in mind. So they might look for a tool that kind of just does both. Right. Tom: Yeah. Yeah. Certain people definitely, wanted that. And yeah, it was an interesting direction to go down. I think that market was going to be a lot different than the people who wanted to manage and edit data. And also, I, one thing that I had in mind a lot, uh, was if Placemark didn't work out, how much would people be burned? and I think if I, if I built it in a way that like everyone was heavily relying on the API and embeds, people would be suffer a lot more, if I eventually had to shut it down. every API that you release is really a, a long-term commitment. And instead for me, like guilt wise, having a product where you can easily export everything that you ever did in any format that you want was like the least lock in, kind of. Jeremy: Yeah. And I imagine the, the scope of the project too, you're making it much smaller if you, if you stick to that editing experience and not try to do everything. Tom: Yeah. Yeah. I, the scope was already pretty big. as you can tell from the open source project, it's, it's bigger than I wish it was. the whole time I was really hoping that I could figure out some niche that was much more compact. there's, I forget the name, but there's somebody who has a, an application that's very similar to Placemark in. Technical terms, but is just a hundred percent focused on planning septic systems. And I'm just like, if I just did this just for septic systems, like would that be a much, would that be 10,000 lines of code instead of 40,000 lines of code? And it would be able to perfectly serve those customers. but you know, that I didn't do enough experimentation to figure that out. Um, I, that's, I think one thing that I wish I had done a lot more was, pivot and do experiments. Jeremy: that septic example, do you know if it's a, a business in and of itself where it can actually support one person or a staff of people? Or is it, is that market just too small? Tom: I think it's still a solo bootstrapped project. yeah. And it's, it's so hard to tell whether a company's doing well or not. I could ask the person over DM. [00:21:58] Built the base technology before going public Jeremy: So when you were first starting. placemark. You were, you were doing it as a solo, developer. A solo entrepreneur, reallyyou worked on it for quite a while, I think before you announced, right? Like maybe a year or so? Tom: Yeah, yeah. Almost, almost a year, I think, maybe, maybe 10 months in the dark. Jeremy: I think that there's, there was a lot of overlap between the different directions that I would eventually go in and. So just building a collaborative editor that can edit map data fairly quickly and checks all the boxes of being able to import and export things, um, that is, was a lot of work. and I mean also I, I was, uh, freelancing during part of it, so it wasn't a hundred percent of my time. Tom: But that, that core, I think even now if I were to build something similar, I would probably still use that work. because that, whether you're doing the septic planning application or you're doing a general purpose kind of map editor or some kind of social application, a lot of that stuff will be in common. Um, and so I wanted to really get, like, to figure out that problem space and get a few solutions that I could live with. Jeremy: The base. libraries or technologies you were gonna pick to get the map and have the collaborative aspect. Those are all things you wanted to get settled first. And then you figured, okay, once I have this base, then I can go find the, you know, the, the, the customers or, or find the specifics of what I'm gonna build. Tom: Yeah, exactly. Jeremy: I I think you had said that going forward when you're gonna work on another project, you would probably still start the same way. [00:23:51] Geospatial is a tough industry, no public companies Tom: if I was working on a project in the geospatial space, I would probably heavily reference the work that I already did here. but I don't know if I'll go back to, to maps again. It's a tough industry. Jeremy: Is it because of the, the customer base? Is it because like people don't really understand the market in terms of who actually needs the maps? I'm kind of curious what you feel makes it tough. Tom: I think, well there are no, there are no public mapping companies. Esri is I think one of the 10 largest private companies in the us. but it's not like any of these geospatial companies have ever been like a pure play. And I think that makes it hard. I think maps are just, they're kind of like fonts in a way in which they are this. Very deep well of complexity, which is absolutely fascinating. If you're in it, it's enough fun and engineering to spend an entire career just working on that stuff. And then once you're out of it, you talk to somebody and you're just like, oh, I work on this thing. And they're like, oh, that you Google maps. Um, or, you know, I work at a font type like a, you know, a type factory and it's like, oh, do you make, uh, you know, courier in, uh, word. It's really infrastructure, uh, that we mostly take for granted, which is, that's, that means it's good in some ways. but at the same time, I, it's hard to really find a niche in which the mapping component is that, that is that useful. A lot of the companies that are kind of mapping companies. Like, I think you could say that like Strava and Palantir are kind of geospatial companies, both of them. but Strava is a fitness company and Palantir is a military company. so if you're, uh, a mapping expert, you kind of have to figure out what, how it ties into the real world, how it ties into the business world and revenue. And then maps might be 50% of the solution or 75% of the solution, but it's probably not going to be, this is the company that makes mapping software. Jeremy: Yeah, it's more like, I have this product that I'm gonna sell and it happens to have a map as a part of it. versus I'm going to sell you, tools that, uh, you know, help you make your own map. That seems like a, a harder, harder sell. Tom: yeah. And especially pro tools like the. The idea of people being both invested in terms of paying and invested in terms of wanting to learn the tool. That's, uh, that's a lot to ask out of people. [00:26:49] Knowing the market is tough but going for it anyways Jeremy: I think the things we had just talked about, about mapping being a tough industry and about there being like the low end is taken care of by Google, the high end is taken care of by Esri with ArcGIS. Uh, I think you mentioned in a blog post that when you started Placemark you, you, you knew all this from the start. So I'm kind of curious, like, knowing that, what made you decide like, I'm gonna, I'm gonna go for it and, you know, do it anyways. Tom: uh, I, well, I think that having seen, I, like I am a co-founder of val.town now, and every company that I've worked for, I've been pretty early enough to see how the sausage is made and the sausage is made with chaos. Like every company doesn't know what it's doing and is in an impossible fight against some Goliath figure. And the product that succeeds, if it ever does succeed, is something that you did not think of two or three years in advance. so I looked at this, I looked at the odds, and I was like, oh, these are the typical odds, you know, maybe someday I'll see something where it's, uh, it's an obvious open blue water market opportunity. But I think for the, for the most part, I was expecting to grind. Uh, you know, like even, even if, uh, the odds were worse, I probably would've still done it. I think I, I learned a lot. I should have done a lot more marketing and business and, but I have, I have no regrets about, you know, taking, taking a one try at solving a very hard to solve problem. Jeremy: Yeah, that's a good point in that the, the odds, like you said, are already stacked against you. but sometimes you just gotta try it and see how it goes, Tom: Yeah. And I had the, like I was at a time where I was very aware of how my life was set up. I was like, I could do a startup right now and kind of burn money for a little while and have enough time to work on it, and I would not be abandoning an infant child or, you know, like all of the things that, all the life responsibilities that I will have in the near future. Um. So, you know, uh, the, the time was then, I guess, [00:29:23] Being a solo developer Jeremy: And comparing it to your time at Mapbox and the other startups and, and I suppose now at val.town, when you were working on Placemark, you're the sole developer, you're in charge of everything. how did that feel? Did you enjoy that experience or was it more like, I, I really wish I had other people to, you know, to kind of go through this with, Tom: Uh, around the end I started to chat with people who, like might be co-founders and I even entertained some chats with, uh, venture capital people. I am fine with the, the day to day of working on stuff alone of making a lot of decisions. That's what I have done in a lot of companies anyway. when you're building the prototype or turning a prototype into something that can be in production, I think that having, uh, having other people there, It would've been better for my mentality in terms of not feeling like it was my thing. Um, you know, like feeling detached enough from the product to really see its flaws and really be open to, taking more radical shifts in approach. whereas when it's just you, you know, it's like you and the customers and your email inbox and, uh, your conscience and your existential dread. Uh, and you know, it's not like a co-founder or, uh, somebody to work with is gonna solve all of that stuff for you, but, uh, it probably would've been maybe a little bit better. I don't know. but then again, like I've also seen those kinds of relationships blow up a lot. and I wanted to kind of figure out what I was doing before, adding more people, more complexity, more money into the situation. But maybe you, maybe doing that at the beginning is kind of the same, you know, like you, other people are down for the same kind of risk that you are. Jeremy: I'm sure it's always different trade offs. I mean, I, I think there probably is a power to being able to unilaterally say like, Hey, this is, this is what I wanna do, so I'm gonna do it. Tom: Yeah. [00:31:52] Spending too much time on multiplayer without a business case Jeremy: You mentioned how there were certain flaws or things you may not have seen because you were so in it. Looking back, what, what were some of those things? Tom: I think that, uh, probably the, I I don't think that most technical decisions are all that important, um, that it never seems like the thing that means life or death for companies. And, you know, Facebook is still on PHP, they've fought, fixed, the problem with, with money. but I think I got rabbit holed into a few things where if I had like a business co-founder, then they would've grilled me about like, why are we spending? The, the main thing that comes to mind, uh, is real time multiplayer, real time. It was a fascinating problem and I was so ready to think about that all the time and try to solve it. And I think that took up a lot of my time and energy. And in the long term, most people are not editing a map. At the same time, seeing the cursors move around is a really fun party trick, and it's great for marketing, but I think that if I were to take a real look at that, that was, that was a mistake. Especially when the trade off was things that actually mattered. Like the amount of time, the amount, the amount of data that the, that could be handled at. At the same time, I could have figured out ways to upload a one gigabyte or two gigabyte or three gigabyte shape file and for it to just work in that same time, whereas real time made it harder to solve that problem, which was a lot closer to what, Paying customers cared about and where people's expectations were? Jeremy: When you were working on this realtime collaborative functionality, was this before the product was public? Was this something you, built from the start? Tom: Yeah. I built the whole thing without it and then added it in. Not as like a rewrite, but like as a, as a big change to a lot of stuff. Jeremy: Yeah, I, I could totally see how that could happen because you are trying to envision people using this product, and you think of something like Google Docs, right? It's very powerful to be typing in a document and see the other cursors and, um, see other people typing. So, I could see how you, you would make that leap and say like, oh, the map should, should do that too. Yeah. [00:34:29] Financial pressures of bootstrapping, high COL, and healthcare Tom: Yeah. Yeah. Um, and, you know, Figma is very cool. Like the, it's, it's amazing. It's an amazing thing. But the Figma was in the dark for way longer than I was, and uh, Evan is a lot smarter than I was. Jeremy: He probably had a big bag of money too. Right. Tom: Yeah. Jeremy: I, I don't actually know the history of Figma, but I'm assuming it's, um, it's VC funded, right? Tom: Uh, yeah, they're, they're kind of famous for just having, I don't think they raised that much in the beginning, but they just didn't hire very much and it was just like the two co-founders, or two or three people and they just kept building for long time. I feel like it's like well over three years. Jeremy: Oh wow. Okay. I think like in your case, I, I saw a comment from you where you were saying, this was your sole source of income and you gotta pay for your health insurance, and so you have no outside investments. So, the pressures are, are very different I think. Tom: Yeah. Yeah. And that's really something to on, to appreciate about venture capital. It gives you the. Slack in your, in your budget to make some mistakes and not freak out about it. and sadly, the rent is not going down anytime soon in, in Brooklyn, and the health insurance is not going down anytime soon. I think it's, it's kind of brutal to like leave a job and then realize that like, you know, to, to be admitted to a hospital, you have to pay $500 a month. Jeremy: I'm, I'm sure that was like, shocking, right? The first time you had to pay for it yourself. Tom: Yeah. And it's not even good. Uh, we need to fix this like that. If there's anything that we could do to fix entrepreneurship in this country, it's just like, make it possible to do this without already being wealthy. Um, it was, it was a constant stress. [00:36:29] Growth and customers Jeremy: As you worked on it, and maybe especially as you, after you had shipped, was there a period where. You know, things were going really well in terms of customers and you felt like, okay, this is really gonna work. Tom: I was, so, like, I basically started out by dropping, I think $5,000 in the business bank account. And I was like, if I break even soon, then I'll be happy. And I broke even in the first month. And that was amazing. I mean, the costs were low and everything, but I was really happy to just be at that point and that like, it never went down. I think that probably somebody with more, uh, determination would've kept going after, after I had stopped. but yeah, like, and also The people who used Placemark, who I actually chatted with, and, uh, all that stuff, they were awesome. I wish that there were more of them. but like a lot of the customers were doing cool stuff. They were supportive. They gave me really informative feedback. Um, and that felt really good. but there was never a point at which like the, uh, the growth scale looked like, oh, we're going to hit a point at which this will be a sustainable business within a year. I think it, according to the growth when I left it, it would've been like maybe three years until I would've been, able to pay my rent and health insurance and, live a comfortable life in, in New York. Jeremy: So when you mentioned you broke even that was like the expenses into the business, but not for actually like rent and health insurance and food and all that. Okay. Okay. can you say like roughly how much was coming in or how many customers you had? Tom: Uh, yeah, the revenue initially I think was, uh, 1500 MRR, and eventually it was like 4,000 or so. Jeremy: And the growth was pretty steady. [00:38:37] Bootstrapping vs fundraising Tom: Um, so yeah, I mean, the numbers where you're just like, maybe I could have kept going. but it's, the other weird thing about VCs is just that I think I have this rich understanding of like, if you're, if you're running a business that will be stressful, but be able to pay your bills and you're in control of it, versus running a startup where you might make life changing money and then not have to run a business again. It's like the latter is kind of better. Uh, if stress affects you a lot, and if you're not really wedded to being super independent. so yeah, I don't know between the two ways of like living your life, I, I have some appreciation for, for both. doing what Placemark entailed if I was living cheaply in a, in a cheap city and it didn't stress me out all the time, would've been a pretty good deal. Um, but doing it in Brooklyn with all the stress was not it, it wasn't affecting my life in positive ways and I, I wanted to, you know, go see shows at night with my friends and not worry about the servers going down. Jeremy: Even putting the money aside, I think that's being the only person responsible for the app, right? Probably feels like you can't really take a vacation. Right. Tom: Yeah, I did take a vacation during it. Like I went to visit my partner who was in, uh, Germany at the time, and we were like on a boat, uh, between Germany, across the lake to Switzerland, and like the servers went down and I opened up my laptop and fixed the servers. It's just like, that is, it's a sacrifice that people make, but it is hard. Jeremy: There's, there's on call, but usually it's not just you 24 7. Tom: Yeah. If you don't pick up somebody else [00:40:28] Financial stress and framing money spent as an investment Jeremy: Yeah, yeah, yeah, I guess at what point, because I'm trying to think. You started in 2021 and then maybe wrapped up, was it sometime in 2024? Tom: Uh, I took a job in, uh, I, I mean I joined val.town in the early 2023 and then wrapped up in November, 2023. Jeremy: At what point did you really start feeling the, the stress? Like I, I imagine maybe when you first started out, you said you were doing consulting and stuff, so, um, probably things were okay, but once you kind of shifted away from that, is that kind of when the, the, the worries about money started coming in? Tom: Yeah. Um, I think maybe it was like six or eight months, um, in. Just that I felt like I wasn't finding, uh, like a, a way to grow the product without adding lots of complexity to it. and being a solo founder, the idea of succeeding, but having built like this hulking mess of a product felt just as bad as not succeeding. like ideally it would be something that I could really be happy maintaining for the long term. Uh, but I was just seeing like, oh, maybe I could succeed by adding every feature in QGIS and that's just not, not a, not something that I wanted to commit to. but yeah, I don't, I don't know. I've been, uh, do you know, uh, Ramit Sethie he's like a, Jeremy: I don't. Tom: an internet money guy. He's less scummy than the rest of them, but still, I. an internet money guy. Um, but he does adjust a lot of stuff about like, money psychology. And that has made me realize that a lot of what I thought at the time and even think now is kind of a rational, you know, like, I think one of the main things that I would do differently is just set a budget for Placemark. Like if I had just set away, like, you know, enough money to live on for a year and put that in, like the, this is for Placemark bucket, then it would've felt better to me then having it all be ad hoc, month to month, feeling like you're burning money instead of investing money in a thing. but yeah, nobody told me, uh, how to, how to think about it then. Uh, yeah, you only get experience by experiencing it. Jeremy: You're just seeing your, your bank account shrinking and there's this, psychological toll, right? Where you're not, you're not used to that feeling and it, it probably feels like something's wrong, Tom: Yeah, yeah. I'm, I think it, I'm really impressed by people who can say, oh, I invested, uh, you know, 50 or a hundred thousand dollars into this business and was comfortable with that risk. And like, maybe it works out, maybe it doesn't. Maybe you just like threw a lot of money down into that. and the people, I think with the healthy, productive, uh, relationship with it. Do think of it as like, oh, I, I paid for kind of a bet on a risk. and that's, that's what I was doing anyway. You know, like I was paying my rent and my health insurance and spending all my time working on the product instead of paying, uh, freelance work. but if you don't frame it that way, it doesn't feel like an investment. It feels like you're making a risky gamble. Jeremy: Yeah. And I think that makes sense to, to actually, I think, like you were saying, have a separate account or a separate thing set aside where you are like, this is, this is this money for this purpose. And like you said, look at it as an investment, which with regular investments can go down. Tom: Yeah, exactly. Yeah. Jeremy: Yeah [00:44:26] In hindsight might have raised money or tried smaller bets Jeremy: Were there, there other things, whether technical or or business wise, that, that if you were to to do it again, you would do differently? Tom: I go back and forth on whether I should have raised venture capital. there are, there's kind of a, an assumption in venture capital that once you're on it, you have to go the whole way. You have to become a billion dollar company, uh, or at least really tell people that you're going to be a billion dollar company and I am not. yeah, I, I don't know. I've seen, I've seen other companies in my space, or like our friends of my current company who are not really targeting that, or ones who were, and then they had somewhere in between the billion dollar and the very small outcome. Uh, and that's a little bit of a point in the favor of accepting a big pile of money from the venture capitalists. I'm also a little bit biased right now because val.town has one investor and he's like the, the best venture capitalist that I have ever met. Big fan. don't quote me on that. If he sacks me in like a year, we'll see. Um, but uh, yeah, there, I, I think that I understand more why people take that approach. or I've understood more why people take like the venture capital but not taking $300 million from SoftBank approach. yeah, and I don't know, I think that, trying a lot of things also seems really appealing. Uh, people who do the same kind of. of Maybe 10 months, but they build four or five different products or three different products instead of just one. I think that, that feels, feels like a good idea to me. Jeremy: And in doing that, would that be more of a, like as a solo entrepreneur or you, you're thinking you would take investment and then say, I'm gonna try all these things with, with your money. Tom: Oh, I've seen both. I, that I, yeah, one friend's company has pivoted like four times between very different ideas and yeah, it, it's one way to do it, but I think in the long term, I would want to do that as a solo developer and try to figure out, you know, something. but yeah, I, I think, uh, so much of it is mindset, that even then if I was working on like three different projects, I think I. My qualifications for something being worth, really adopting and spending all my time doing, you just have to accept, uh, a lot of hits and a lot of misses and a lot of like keeping things alive and finding out how to turn them into something. I am really inspired by my friends who like started around the same time that I did and they're not that much further in terms of revenue and they're like still, still doing it because that is what they want to do in life. and if you develop the whole ecosystem and mindset around it, I think that's somewhere that people can stay and, and be happy. just trying to find, trying to find a company that they own and control and they like. Jeremy: While, while making the the expenses work. Tom: Yeah. Yeah. that's the, that's the hard part, like freelancing on the side also. I probably could have kept that up. I liked my freelance clients. I would probably still work with them as well. but I kind of just wanted the, I wanted the focus, I wanted the motivation of, of being without a net. Jeremy: Yeah, I mean, energy wise, do you think that that would've worked? I mean, I imagine that Placemark took a lot of your time when you were working full time, so you're trying to balance, you know, clients and all your customers and everything you're doing with the software. It just feels like it might be a lot. Tom: Yeah. Yeah. Maybe with different freelance clients. I, I loved my freelance clients because I, after. leaving config. I, I wanted to work on climate change stuff and so I was working for climate change foundations and that is not the way to max out your paycheck. It's the way to feel good about your conscience. And so I still feel great about those projects, but in the future, yeah, I would probably just work for, uh, you know, a hedge fund or something. [00:49:02] Marketing to developers but not potential customers Jeremy: I think something you mentioned in one of your posts is that you maybe could have spent more time or had a different approach with marketing. Maybe you could kind of say what you did do and then what maybe worked and what didn't. Tom: Yeah. So I like my sweet spot is writing documentation and blog posts and technical stuff. And so I did a lot of that and a lot of that like worked in a way that didn't matter. I am at this point, weirdly good at writing stuff that gets on Hacker News. I've written a lot of stuff that's gotten to the top of Hacker News and unfortunately, writing about your technical approach and your geospatial project for handling errors, uh, in your JavaScript code is not really a way to get customers. and I think doing a lot of documentation was also great, but it was also, I think that the, the thing that was missing is the thing that I think Mapbox does fairly well now, in which the homepage really pushes you toward use cases immediately. and I should have been saying to each customer who had anything compelling as a use case, like, let's write an article about you and what you're doing, and here's how you use this in your industry. and that probably would've also been like a good, a good way to figure out which of those verticals was the one that was most worth spending all the time on. yeah. So it, it was, it was a lot of good marketing to nerds. and it could have been better in terms of marketing to actual customers and to people who are making the buying decisions. Jeremy: Yeah. Looking at the, the Placemark blog, I can definitely see how as a developer, a lot of the posts are appealing to me, right? It's about how you worked on a technical challenge or decisions you made, but maybe less so to somebody who they wanna. Draw a map to manage their crops. They're like, I don't care about any of this. Right. Tom: Yeah, like the Mapbox blog used to be, just all that stuff as well. We would write about designing protocol buffer layouts, and it was amazing for hiring and amazing for getting nerds in the door. But now it's just, Toyota is launching with, Mapbox Maps or something like that. And that's, that's what you, you should do if you're trying to sell a product. Jeremy: Yeah. And I think the, the sort of technical aspect, it makes sense too. If you're venture funded and you are looking to hire, right? You wanna build your team and you just want to increase like, the amount of stuff you're building and not worrying so much about, am I gonna have a paycheck next Tom: Yeah. Yeah. I, I just kind of do it because it's fun, which is not the right reason to do it, but, Yeah, I mean, I still write my blog mostly just because it's, it's a fun thing to do, but it's not the best way to, um, to run a business. Jeremy: Yeah. Well, the fun part is important too though. Tom: Yeah. Yeah. That's, that's maybe the whole thing. May, that's maybe the most important thing, but you can't do it if you don't do the, the money part. [00:52:35] Most customers came from existing audience Jeremy: Right. So the people who did find you, was it mostly word of mouth from people who did identify with the technical posts, or were there places that surprised you, that people found you? Tom: Uh, a lot of it was people who were familiar with the Mapbox ecosystem or with, with me. and then eventually, yeah, a few of the users came in through, um, through Hacker News, but it was mostly, mostly word of mouth also. The geospatial community is like fairly tight and it's, and it's not too hard to be the person who writes the article about some geospatial challenge that everyone finds. Jeremy: Hmm. Okay. Yeah, that's a good point about like being in that community, especially since you've done so much work in geospatial and in open source that you have this little, this built-in audience, I guess. Tom: yeah. Which I appreciate. It makes me nervous, but yeah. [00:53:43] Val.town marketing to developers Jeremy: Comparing that to something like val.town, how is val.town marketing? How is it finding users? 'cause from what I can tell, it's, it's getting a lot of, uh, a lot of people coming in, right? Tom: Yeah. Uh, well, right now our, our kind of target user, or the user that we think of is a hobbyist, is somebody who's, sometimes a pro developer or somebody, sometimes just somebody who's really interested in the field. And so writing these things that are just about, you know, programming, does super well. Uh, but it, we have exactly the same problem and that that is kind of being revamped as we speak. uh, we hired somebody who actually knows marketing and has a good sense for it. And so a lot of that stuff is shifting to show you what you can do with val.town because it, it suffers from the same problem as well. It's an empty text field in which you can type, type script, code, and it runs. And knowing what you can do with that or what you should do with that is, is hard if you don't have a grasp of TypeScript and web applications. so pretty soon we'll have pages which are like, here's how to connect linear and GitHub with OW Town, or, you know, two nouns connect them, for all of those companies and to do automations and all these like concrete applications. I think that's, you have to do it. You have to figure it out. Jeremy: Just briefly for someone who hasn't heard of val.town, like what, what does it do? Tom: Uh, val.town is a social website, so it has comments and likes and all of that stuff. but it's for writing these little snippets of TypeScript and JavaScript code that run. So a lot of them are websites, some of them are automations, so they receive emails or send emails or connect one service to another. And yeah, it's, it's like combining some aspects of, GitHub or like a code platform, uh, but with the assumption that every time that you save, everything's instantly deployed. Jeremy: So it's maybe a little bit like, um, like a glitch, I guess? Tom: Uh, yeah. Yeah, it takes a lot of experience, a lot of, uh, inspiration from Glitch. Jeremy: And I, I think, like you had mentioned, you enjoy writing the, the technical blog posts and the documentation. And so at least with val.town, your audience is developers versus, the geospatial community who probably largely doesn't care about, TypeScript and the, the different technical decisions there. Tom: Yeah, it, it makes it easier, that's for sure. The customer is, is me. [00:56:30] Shifting from solo to in-person teams Jeremy: Nice. Yeah. Looking at, you know, you, you worked as a, a solo developer for Placemark, and then now you've got a team of, is it like maybe five Tom: Uh, it is seven at the moment. Jeremy: Seven people. Okay. Are you all in person or is it, remote Tom: We all sit around two tables in Brooklyn. It's very nice. Jeremy: So how did that feel? Like shifting from, I'm in, I don't know if you worked from home while you were working on Placemark or if you were in coworking spaces, but you're, you're shifting from I'm like in my own head space doing everything myself to, to, I'm in a room with all these people and we're like working on this thing together. I'm kind of curious like how that felt for you. Tom: Yeah, it's been a big difference. And I think that I was just talking with, um, one, one of our, well an engineer at, at val.town about how everyone kind of had, had been working remote for obvious pandemic world reasons. And this kind of privilege of just being around the same table, if that's what you like is, a huge difference in terms of, I just remember having to. Trick myself into going on a walk around the block because I would get into such a dark mental head space of working on the same project for eight hours straight and skipping lunch. and now there's a little bit more structure. yeah, it's, it's been, it's been a overall, an improvement. Some days I wish that I could go on a run at noon 'cause that's the warmest time of the day. but, uh, overall, like it makes things so much easier. just reading the emotions in people's faces when they're telling you stuff and being able to, uh, not get into discussions that you don't need to get into because you can talk and just like understand each other very quickly. It's, it's very nice. I don't wanna force everyone to do it, you know, but it it for the people who want it, they, they, uh, really enjoy it. Jeremy: Yeah. I think if you have the right set of people, it's definitely more enjoyable. And um, if you don't, maybe not so Tom: Yeah, we haven't hired any, like, extremely loud chewers yet or anything like that, but yeah, maybe my story will change. Jeremy: No, no one microwaving fish. Tom: No, there's, uh, yeah, thankfully the microwave is outside of the office. Jeremy: Do you live close to the office? Tom: Yeah. Yeah. Like most of the team is within a 20 or 30 minute walk of the office and it's very fortunate. I think there's been something of a mass migration to New York. A lot of us didn't live in New York before four years ago, and now all of us do. it's, it's, uh, it's very comfortable to be here. Jeremy: I think that makes, uh, such a big difference. 'cause I think the majority of people, at least within the US you know, you're, you're getting in your car, you're sitting in traffic. and I know people who, during the pandemic, they actually moved further, right? Because they went, oh, like, uh, I don't need to come into the office. but yeah, if you are close enough where you can walk, yeah, I think that makes a big difference. Tom: Oh yeah. If I had to drive to work, I think my blood pressure would be so much higher. Uh, especially in New York. Oh, I feel so bad for the people who have to drive, whereas I'm just walking with, you know, a bagel in hand, enjoying listening to the birds. Jeremy: Yeah. Yeah. well now they have, what is it, the congestion pricing in Tom: Yeah. Yeah. We're all in Brooklyn, so it doesn't affect us that much, but it's supposedly, it's, it's working great. Um, yeah. I hope we can keep it. Jeremy: I've never driven in New York and I, I wouldn't want to Tom: Yeah. It's only for the brave or the crazy. [01:00:37] The value of public writing and work Jeremy: I think that's probably a good place to, to wrap up, but is there any other thoughts you had or things you wanted to mention? Tom: No, I've just, uh, thank you so much. This has been, this has been a lot of fun. You're, you're very good at this as well. I feel like it's, uh, Jeremy: Thank you Tom: It's not easy to, to steer a conversation in a way that makes awkward people sound, uh, normal. Jeremy: I wouldn't say that, but um, what's been actually pretty helpful to me is, you have such a body of work, I guess I would say, in terms of your blogging and, just the amount that you write and the long history of projects that, that there's, you know, there's a lot to talk about and I'm sure it helps, helps your thought process as well. Tom: Yeah. I, I've been lucky to have a lot of jobs where people, where companies were like, cool with publishing everything, you know? so a lot of what I've done is, uh, is public. it's, it's, uh, I'm very, very thankful for like, early on that being a big part of company culture. Jeremy: And you can definitely tell, I think for people who look at the Placemark blog posts or, or now your, your val.town blog posts, like there's, there's a clear difference when somebody like is very intentional and, um, you know, it's good at writing versus you're doing it because, um, it's your corporate responsibility or whatever, like people can tell. Yeah. Tom: Yeah. You can't fake being interested. so you gotta work on things that are interesting. Jeremy: Tom, thanks again for, for agreeing to chat. This was fun. Tom: Yeah thank you so much.

Bu bölümde Mert'in onbinlerce kişi tarafından okunan blog yazısı, DeepSeek, Mutluluğa Denk Gelmek kitabı ve Severance dizisinin yeni sezonu üzerine sohbet ettik.Bizi dinlemekten keyif alıyorsanız, kahve ısmarlayarak bizi destekleyebilir ve Telegram grubumuza katılabilirsiniz. :)Yorumlarınızı, sorularınızı ya da sponsorluk tekliflerinizi info@farklidusun.net e-posta adresine iletebilirsiniz.Zaman damgaları:00:00 - Giriş00:48 - Popüler Olan Blog Yazısı21:22 - Segmented Control27:35 - iOS'te Build Time Optimizasyonu45:10 - DeepSeek ve AI Dünyası1:28:35 - İzlediklerimiz1:46:25 - Okuduklarımız, Mutluluğa Denk Gelmek, Hatching Twitter2:19:00 - Haftanın Albümleri2:22:30 - Xbox Developer DirectBölüm linkleri:MonoforOnce You're Laid Off, You'll Never Be the Same AgainBlog yazısının Hacker News başlığıThe Pragmatic EngineerSeyfeddin'in Segmented Control'uOrigami StudioCarthageImplementing Design Systems in Swift - Seyfeddin Başsaraç - Appy Hour Meetup #7Visa applications in the Consular Services PortalDeepSeekDeepSeek - StratecheryOpenAI and SoftBank are starting a $500 billion AI data center companyWhy everyone is freaking out about DeepSeekU.S. curbs export of more AI chips, including Nvidia H800, to ChinaApple makes a change to its AI team and plans Siri upgradesHow an Economic Moat Provides a Competitive AdvantageFactfulness: Ten Reasons We're Wrong About the World – and Why Things Are Better Than You ThinkSeveranceSiloSilo — Extending Worlds: Silos 17 & 18 | Apple TV+The Severance Podcast with Ben Stiller & Adam ScottTim Cook SeveranceSeverance — Opening Title Sequence: Season 2 | Apple TV+En yüksek puanlı filmlerTop Gun: MaverickEmancipation12 Years a Slaveİnsan OlmakNeden Çalışalım ki? Boş Zaman Toplumuna Dair SavlarThe German GeniusStumbling on HappinessHatching Twitter: A True Story of Money, Power, Friendship, and BetrayalDebt: The First 5,000 YearsPara, Sikke ve Borç: David Graeber / Emrah Safa Gürkan - Historik 48The Ultimate Hidden Truth of the WorldCrime of the Century (album)Smother (album)Everything we saw at Xbox's Developer Direct 20252025 looks like a great year for XboxThe Zero Marginal Cost Society: The Internet of Things, the Collaborative Commons, and the Eclipse of CapitalismWhat you need to know about Apple's Advanced Commerce APIMicrosoft is closing its British flagship store in London

Lazarus Group's Secret Admin Layer EXPOSED – Major Cybersecurity Discovery!

In this episode, we're uncovering the darker side of Generative AI and the emerging threats lurking behind everyday tools like ChatGPT and Copilot. Learn how sensitive information—ranging from customer data to employee benefits—can be leaked simply by typing it into a Gen AI prompt. We'll also expose how cybercriminals are escalating their tactics, hiding malware in places you'd never expect—like Google Ads, YouTube comments, and misleading download links for supposedly “free” or pirated software.

Søren Bramer Schmidt, co-founder and CEO of Prisma, joins us to discuss the journey of building one of the largest developer communities in DevTools. Søren shares how Prisma's deliberate strategies have shaped its growth, feature prioritization, and the launch of new products like Prisma Postgres. We also explore the challenges of managing a vast user base and how Prisma is adapting to shifts in application development.We discuss:How intentional partnerships with educators and influencers fueled Prisma's early growth.Strategies to engage the GraphQL community and gain visibility on platforms like Hacker News.Managing a large developer community while balancing innovation with stability.The evolution from Graphcool to Prisma ORM, including lessons from early pivots.Launching Prisma Postgres and how community feedback influenced its development.Implementing a simple, usage-based pricing model and reducing infrastructure costs through self-hosting.This episode is brought to you by WorkOS. If you're thinking about selling to enterprise customers, WorkOS can help you add enterprise features like Single Sign On and audit logs. https://workos.com/Links:Prisma (https://www.prisma.io/)Prisma Postgres (https://www.prisma.io/postgres)Feldera (https://feldera.com/)

Drew Houston is the co-founder and CEO of Dropbox. Under his leadership, Dropbox has grown from a simple idea to a service used by over 700 million registered users globally, with a valuation exceeding $9 billion. Drew has led Dropbox through multiple phases, from explosive viral growth, to battling all the tech giants at once, to reinventing the company for the future of work. In our conversation, he opens up about:• The three eras of Dropbox's growth and evolution• The challenges he's faced over the past 18 years• What he learned about himself• How he's been able to manage his psychology as a founder• The importance of maintaining your learning curve• Finding purpose beyond metrics and growth• The micro, macro, and meta aspects of building companies• Much more—Brought to you by:• Paragon—Ship every SaaS integration your customers want• Explo—Embed customer-facing analytics in your product• Vanta—Automate compliance. Simplify security—Find the transcript at: https://www.lennysnewsletter.com/p/behind-the-founder-drew-houston-dropbox—Where to find Drew Houston:• X: https://x.com/drewhouston• LinkedIn: https://www.linkedin.com/in/drewhouston/—Where to find Lenny:• Newsletter: https://www.lennysnewsletter.com• X: https://twitter.com/lennysan• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/—In this episode, we cover:(00:00) Introduction to Drew and Dropbox(04:44) The three eras of Dropbox(07:53) The first era: Viral growth and early success(14:19) The second era: Challenges and competition(20:49) Strategic shifts and refocusing(29:36) Personal reflections and leadership lessons(40:19) Unlocking mindfulness and building support systems(43:14) The Enneagram test(50:35) The challenges of being a founder CEO(58:11) The third era: Rebooting the team and core business(01:22:41) Lessons and advice for aspiring founders(01:27:46) Balancing personal and professional growth(01:42:38) Final reflections and future outlook—Referenced:• Dropbox: https://www.dropbox.com/• Y Combinator: https://www.ycombinator.com/• Paul Graham's website: https://www.paulgraham.com/• Hacker News: https://news.ycombinator.com/• Arash Ferdowsi on LinkedIn: https://www.linkedin.com/in/arashferdowsi/• Sequoia Capital: https://www.sequoiacap.com/• Pejman Nozad on LinkedIn: https://www.linkedin.com/in/pejman/• Mike Moritz on LinkedIn: https://www.linkedin.com/in/michaelmoritz/• TechCrunch Disrupt: https://techcrunch.com/events/tc-disrupt-2024/• Dropbox viral demo: https://youtu.be/7QmCUDHpNzE• Digg: https://digg.com/• Reddit: https://www.reddit.com/• Hadi and Ali Partovi: https://www.partovi.org/• Zynga: https://www.zynga.com/• Steve Jobs announces Apple's iCloud: https://www.youtube.com/watch?v=ilnfUa_-Rbc• Dropbox Carousel: https://en.wikipedia.org/wiki/Dropbox_Carousel• Dropbox Is Buying Mega-Hyped Email Startup Mailbox: https://www.businessinsider.com/dropbox-is-buying-mega-hyped-email-startup-mailbox-2013-3• 5 essential questions to craft a winning strategy | Roger Martin (author, advisor, speaker): https://www.lennysnewsletter.com/p/the-ultimate-guide-to-strategy-roger-martin• Intel: https://www.intel.com/• Gordon Moore: https://en.wikipedia.org/wiki/Gordon_Moore• Netscape: https://en.wikipedia.org/wiki/Netscape• Myspace: https://en.wikipedia.org/wiki/Myspace• Bill Campbell: https://en.wikipedia.org/wiki/Bill_Campbell_(business_executive)• Enneagram type descriptions: https://www.enneagraminstitute.com/type-descriptions/• The Myers-Briggs Type Indicator: https://www.themyersbriggs.com/en-US/Products-and-Services/Myers-Briggs• Brian Chesky's new playbook: https://www.lennysnewsletter.com/p/brian-cheskys-contrarian-approach• Ben Horowitz on X: https://x.com/bhorowitz• Why Read Peter Drucker?: https://hbr.org/2009/11/why-read-peter-drucker• GitLab: https://about.gitlab.com/• Automattic: https://automattic.com/• Dropbox Dash: https://www.dash.dropbox.com/• Welcome Command E to Dropbox: https://blog.dropbox.com/topics/company/welcome-command-e-to-dropbox-• StarCraft: https://en.wikipedia.org/wiki/StarCraft_(video_game)• Procter & Gamble and the Beauty of Small Wins: https://hbr.org/2009/10/the-beauty-of-small-wins• Teaching Smart People How to Learn: https://hbr.org/1991/05/teaching-smart-people-how-to-learn—Recommended books:• Guerrilla Marketing: Easy and Inexpensive Strategies for Making Big Profits from Your Small Business: https://www.amazon.com/Guerilla-Marketing-Inexpensive-Strategies-Business/dp/0618785914• Playing to Win: How Strategy Really Works: https://www.amazon.com/Playing-Win-Strategy-Really-Works/dp/142218739X• High Output Management: https://www.amazon.com/High-Output-Management-Andrew-Grove/dp/0679762884/• Only the Paranoid Survive: How to Exploit the Crisis Points That Challenge Every Company: https://www.amazon.com/Only-Paranoid-Survive-Exploit-Challenge/dp/0385483821• Zone to Win: Organizing to Compete in an Age of Disruption: https://www.amazon.com/Zone-Win-Organizing-Compete-Disruption/dp/1682302113• Warren Buffett's books: https://www.amazon.com/warren-buffett-Books/s?k=warren+buffett&rh=n%3A283155• Poor Charlie's Almanack: The Essential Wit and Wisdom of Charles T. Munger: https://www.amazon.com/Poor-Charlies-Almanack-Essential-Charles/dp/1953953239• Invent and Wander: The Collected Writings of Jeff Bezos: https://www.amazon.com/Invent-Wander-Collected-Writings-Introduction/dp/1647820715/• The 15 Commitments of Conscious Leadership: A New Paradigm for Sustainable: https://www.amazon.com/15-Commitments-Conscious-Leadership-Sustainable-ebook/dp/B00R3MHWUE—Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.—Lenny may be an investor in the companies discussed. Get full access to Lenny's Newsletter at www.lennysnewsletter.com/subscribe

In this episode, we dive deep into two explosive cybersecurity stories making headlines right now: a major breach at the U.S. Treasury Department allegedly carried out by Chinese state-sponsored hackers, and the discovery of over 3.1 million fake stars on GitHub used to boost malicious repositories. We'll explore how third-party vendor BeyondTrust and telecom hacks tie into this growing wave of advanced persistent threats (APTs) and discuss how manipulative tactics on GitHub can sneak malware into widespread use. Whether you're a seasoned developer or just curious about the rising tide of global cyber threats, this episode will give you an in-depth look at how these hacks happen and why they matter.   Stay informed, stay alert, and learn about the latest vulnerabilities, breaches, and protective measures you can take. Don't forget to like, subscribe, and hit the notification bell for more cybersecurity breakdowns! #Cybersecurity #DataBreach #USTreasuryHack #ChineseHackers #SaltTyphoon #APTGroups #BeyondTrust #GitHub #FakeStars #Malware #CyberThreats #NetworkSecurity #NationStateHackers #APISecurity #SupplyChainAttack #DarkReading #BleepingComputer #Encryption #SoftwareDevelopment #TechNews #HackingUpdate #ZeroDay #Phishing #InformedSecurity #ExploitBrokers  

Applications for the 2025 AI Engineer Summit are up, and you can save the date for AIE Singapore in April and AIE World's Fair 2025 in June.Happy new year, and thanks for 100 great episodes! Please let us know what you want to see/hear for the next 100!Full YouTube Episode with Slides/ChartsLike and subscribe and hit that bell to get notifs!Timestamps* 00:00 Welcome to the 100th Episode!* 00:19 Reflecting on the Journey* 00:47 AI Engineering: The Rise and Impact* 03:15 Latent Space Live and AI Conferences* 09:44 The Competitive AI Landscape* 21:45 Synthetic Data and Future Trends* 35:53 Creative Writing with AI* 36:12 Legal and Ethical Issues in AI* 38:18 The Data War: GPU Poor vs. GPU Rich* 39:12 The Rise of GPU Ultra Rich* 40:47 Emerging Trends in AI Models* 45:31 The Multi-Modality War* 01:05:31 The Future of AI Benchmarks* 01:13:17 Pionote and Frontier Models* 01:13:47 Niche Models and Base Models* 01:14:30 State Space Models and RWKB* 01:15:48 Inference Race and Price Wars* 01:22:16 Major AI Themes of the Year* 01:22:48 AI Rewind: January to March* 01:26:42 AI Rewind: April to June* 01:33:12 AI Rewind: July to September* 01:34:59 AI Rewind: October to December* 01:39:53 Year-End Reflections and PredictionsTranscript[00:00:00] Welcome to the 100th Episode![00:00:00] Alessio: Hey everyone, welcome to the Latent Space Podcast. This is Alessio, partner and CTO at Decibel Partners, and I'm joined by my co host Swyx for the 100th time today.[00:00:12] swyx: Yay, um, and we're so glad that, yeah, you know, everyone has, uh, followed us in this journey. How do you feel about it? 100 episodes.[00:00:19] Alessio: Yeah, I know.[00:00:19] Reflecting on the Journey[00:00:19] Alessio: Almost two years that we've been doing this. We've had four different studios. Uh, we've had a lot of changes. You know, we used to do this lightning round. When we first started that we didn't like, and we tried to change the question. The answer[00:00:32] swyx: was cursor and perplexity.[00:00:34] Alessio: Yeah, I love mid journey. It's like, do you really not like anything else?[00:00:38] Alessio: Like what's, what's the unique thing? And I think, yeah, we, we've also had a lot more research driven content. You know, we had like 3DAO, we had, you know. Jeremy Howard, we had more folks like that.[00:00:47] AI Engineering: The Rise and Impact[00:00:47] Alessio: I think we want to do more of that too in the new year, like having, uh, some of the Gemini folks, both on the research and the applied side.[00:00:54] Alessio: Yeah, but it's been a ton of fun. I think we both started, I wouldn't say as a joke, we were kind of like, Oh, we [00:01:00] should do a podcast. And I think we kind of caught the right wave, obviously. And I think your rise of the AI engineer posts just kind of get people. Sombra to congregate, and then the AI engineer summit.[00:01:11] Alessio: And that's why when I look at our growth chart, it's kind of like a proxy for like the AI engineering industry as a whole, which is almost like, like, even if we don't do that much, we keep growing just because there's so many more AI engineers. So did you expect that growth or did you expect that would take longer for like the AI engineer thing to kind of like become, you know, everybody talks about it today.[00:01:32] swyx: So, the sign of that, that we have won is that Gartner puts it at the top of the hype curve right now. So Gartner has called the peak in AI engineering. I did not expect, um, to what level. I knew that I was correct when I called it because I did like two months of work going into that. But I didn't know, You know, how quickly it could happen, and obviously there's a chance that I could be wrong.[00:01:52] swyx: But I think, like, most people have come around to that concept. Hacker News hates it, which is a good sign. But there's enough people that have defined it, you know, GitHub, when [00:02:00] they launched GitHub Models, which is the Hugging Face clone, they put AI engineers in the banner, like, above the fold, like, in big So I think it's like kind of arrived as a meaningful and useful definition.[00:02:12] swyx: I think people are trying to figure out where the boundaries are. I think that was a lot of the quote unquote drama that happens behind the scenes at the World's Fair in June. Because I think there's a lot of doubt or questions about where ML engineering stops and AI engineering starts. That's a useful debate to be had.[00:02:29] swyx: In some sense, I actually anticipated that as well. So I intentionally did not. Put a firm definition there because most of the successful definitions are necessarily underspecified and it's actually useful to have different perspectives and you don't have to specify everything from the outset.[00:02:45] Alessio: Yeah, I was at um, AWS reInvent and the line to get into like the AI engineering talk, so to speak, which is, you know, applied AI and whatnot was like, there are like hundreds of people just in line to go in.[00:02:56] Alessio: I think that's kind of what enabled me. People, right? Which is what [00:03:00] you kind of talked about. It's like, Hey, look, you don't actually need a PhD, just, yeah, just use the model. And then maybe we'll talk about some of the blind spots that you get as an engineer with the earlier posts that we also had on on the sub stack.[00:03:11] Alessio: But yeah, it's been a heck of a heck of a two years.[00:03:14] swyx: Yeah.[00:03:15] Latent Space Live and AI Conferences[00:03:15] swyx: You know, I was, I was trying to view the conference as like, so NeurIPS is I think like 16, 17, 000 people. And the Latent Space Live event that we held there was 950 signups. I think. The AI world, the ML world is still very much research heavy. And that's as it should be because ML is very much in a research phase.[00:03:34] swyx: But as we move this entire field into production, I think that ratio inverts into becoming more engineering heavy. So at least I think engineering should be on the same level, even if it's never as prestigious, like it'll always be low status because at the end of the day, you're manipulating APIs or whatever.[00:03:51] swyx: But Yeah, wrapping GPTs, but there's going to be an increasing stack and an art to doing these, these things well. And I, you know, I [00:04:00] think that's what we're focusing on for the podcast, the conference and basically everything I do seems to make sense. And I think we'll, we'll talk about the trends here that apply.[00:04:09] swyx: It's, it's just very strange. So, like, there's a mix of, like, keeping on top of research while not being a researcher and then putting that research into production. So, like, people always ask me, like, why are you covering Neuralibs? Like, this is a ML research conference and I'm like, well, yeah, I mean, we're not going to, to like, understand everything Or reproduce every single paper, but the stuff that is being found here is going to make it through into production at some point, you hope.[00:04:32] swyx: And then actually like when I talk to the researchers, they actually get very excited because they're like, oh, you guys are actually caring about how this goes into production and that's what they really really want. The measure of success is previously just peer review, right? Getting 7s and 8s on their um, Academic review conferences and stuff like citations is one metric, but money is a better metric.[00:04:51] Alessio: Money is a better metric. Yeah, and there were about 2200 people on the live stream or something like that. Yeah, yeah. Hundred on the live stream. So [00:05:00] I try my best to moderate, but it was a lot spicier in person with Jonathan and, and Dylan. Yeah, that it was in the chat on YouTube.[00:05:06] swyx: I would say that I actually also created.[00:05:09] swyx: Layen Space Live in order to address flaws that are perceived in academic conferences. This is not NeurIPS specific, it's ICML, NeurIPS. Basically, it's very sort of oriented towards the PhD student, uh, market, job market, right? Like literally all, basically everyone's there to advertise their research and skills and get jobs.[00:05:28] swyx: And then obviously all the, the companies go there to hire them. And I think that's great for the individual researchers, but for people going there to get info is not great because you have to read between the lines, bring a ton of context in order to understand every single paper. So what is missing is effectively what I ended up doing, which is domain by domain, go through and recap the best of the year.[00:05:48] swyx: Survey the field. And there are, like NeurIPS had a, uh, I think ICML had a like a position paper track, NeurIPS added a benchmarks, uh, datasets track. These are ways in which to address that [00:06:00] issue. Uh, there's always workshops as well. Every, every conference has, you know, a last day of workshops and stuff that provide more of an overview.[00:06:06] swyx: But they're not specifically prompted to do so. And I think really, uh, Organizing a conference is just about getting good speakers and giving them the correct prompts. And then they will just go and do that thing and they do a very good job of it. So I think Sarah did a fantastic job with the startups prompt.[00:06:21] swyx: I can't list everybody, but we did best of 2024 in startups, vision, open models. Post transformers, synthetic data, small models, and agents. And then the last one was the, uh, and then we also did a quick one on reasoning with Nathan Lambert. And then the last one, obviously, was the debate that people were very hyped about.[00:06:39] swyx: It was very awkward. And I'm really, really thankful for John Franco, basically, who stepped up to challenge Dylan. Because Dylan was like, yeah, I'll do it. But He was pro scaling. And I think everyone who is like in AI is pro scaling, right? So you need somebody who's ready to publicly say, no, we've hit a wall.[00:06:57] swyx: So that means you're saying Sam Altman's wrong. [00:07:00] You're saying, um, you know, everyone else is wrong. It helps that this was the day before Ilya went on, went up on stage and then said pre training has hit a wall. And data has hit a wall. So actually Jonathan ended up winning, and then Ilya supported that statement, and then Noam Brown on the last day further supported that statement as well.[00:07:17] swyx: So it's kind of interesting that I think the consensus kind of going in was that we're not done scaling, like you should believe in a better lesson. And then, four straight days in a row, you had Sepp Hochreiter, who is the creator of the LSTM, along with everyone's favorite OG in AI, which is Juergen Schmidhuber.[00:07:34] swyx: He said that, um, we're pre trading inside a wall, or like, we've run into a different kind of wall. And then we have, you know John Frankel, Ilya, and then Noam Brown are all saying variations of the same thing, that we have hit some kind of wall in the status quo of what pre trained, scaling large pre trained models has looked like, and we need a new thing.[00:07:54] swyx: And obviously the new thing for people is some make, either people are calling it inference time compute or test time [00:08:00] compute. I think the collective terminology has been inference time, and I think that makes sense because test time, calling it test, meaning, has a very pre trained bias, meaning that the only reason for running inference at all is to test your model.[00:08:11] swyx: That is not true. Right. Yeah. So, so, I quite agree that. OpenAI seems to have adopted, or the community seems to have adopted this terminology of ITC instead of TTC. And that, that makes a lot of sense because like now we care about inference, even right down to compute optimality. Like I actually interviewed this author who recovered or reviewed the Chinchilla paper.[00:08:31] swyx: Chinchilla paper is compute optimal training, but what is not stated in there is it's pre trained compute optimal training. And once you start caring about inference, compute optimal training, you have a different scaling law. And in a way that we did not know last year.[00:08:45] Alessio: I wonder, because John is, he's also on the side of attention is all you need.[00:08:49] Alessio: Like he had the bet with Sasha. So I'm curious, like he doesn't believe in scaling, but he thinks the transformer, I wonder if he's still. So, so,[00:08:56] swyx: so he, obviously everything is nuanced and you know, I told him to play a character [00:09:00] for this debate, right? So he actually does. Yeah. He still, he still believes that we can scale more.[00:09:04] swyx: Uh, he just assumed the character to be very game for, for playing this debate. So even more kudos to him that he assumed a position that he didn't believe in and still won the debate.[00:09:16] Alessio: Get rekt, Dylan. Um, do you just want to quickly run through some of these things? Like, uh, Sarah's presentation, just the highlights.[00:09:24] swyx: Yeah, we can't go through everyone's slides, but I pulled out some things as a factor of, like, stuff that we were going to talk about. And we'll[00:09:30] Alessio: publish[00:09:31] swyx: the rest. Yeah, we'll publish on this feed the best of 2024 in those domains. And hopefully people can benefit from the work that our speakers have done.[00:09:39] swyx: But I think it's, uh, these are just good slides. And I've been, I've been looking for a sort of end of year recaps from, from people.[00:09:44] The Competitive AI Landscape[00:09:44] swyx: The field has progressed a lot. You know, I think the max ELO in 2023 on LMSys used to be 1200 for LMSys ELOs. And now everyone is at least at, uh, 1275 in their ELOs, and this is across Gemini, Chadjibuti, [00:10:00] Grok, O1.[00:10:01] swyx: ai, which with their E Large model, and Enthopic, of course. It's a very, very competitive race. There are multiple Frontier labs all racing, but there is a clear tier zero Frontier. And then there's like a tier one. It's like, I wish I had everything else. Tier zero is extremely competitive. It's effectively now three horse race between Gemini, uh, Anthropic and OpenAI.[00:10:21] swyx: I would say that people are still holding out a candle for XAI. XAI, I think, for some reason, because their API was very slow to roll out, is not included in these metrics. So it's actually quite hard to put on there. As someone who also does charts, XAI is continually snubbed because they don't work well with the benchmarking people.[00:10:42] swyx: Yeah, yeah, yeah. It's a little trivia for why XAI always gets ignored. The other thing is market share. So these are slides from Sarah. We have it up on the screen. It has gone from very heavily open AI. So we have some numbers and estimates. These are from RAMP. Estimates of open AI market share in [00:11:00] December 2023.[00:11:01] swyx: And this is basically, what is it, GPT being 95 percent of production traffic. And I think if you correlate that with stuff that we asked. Harrison Chase on the LangChain episode, it was true. And then CLAUD 3 launched mid middle of this year. I think CLAUD 3 launched in March, CLAUD 3. 5 Sonnet was in June ish.[00:11:23] swyx: And you can start seeing the market share shift towards opening, uh, towards that topic, uh, very, very aggressively. The more recent one is Gemini. So if I scroll down a little bit, this is an even more recent dataset. So RAM's dataset ends in September 2 2. 2024. Gemini has basically launched a price war at the low end, uh, with Gemini Flash, uh, being basically free for personal use.[00:11:44] swyx: Like, I think people don't understand the free tier. It's something like a billion tokens per day. Unless you're trying to abuse it, you cannot really exhaust your free tier on Gemini. They're really trying to get you to use it. They know they're in like third place, um, fourth place, depending how you, how you count.[00:11:58] swyx: And so they're going after [00:12:00] the Lower tier first, and then, you know, maybe the upper tier later, but yeah, Gemini Flash, according to OpenRouter, is now 50 percent of their OpenRouter requests. Obviously, these are the small requests. These are small, cheap requests that are mathematically going to be more.[00:12:15] swyx: The smart ones obviously are still going to OpenAI. But, you know, it's a very, very big shift in the market. Like basically 2023, 2022, To going into 2024 opening has gone from nine five market share to Yeah. Reasonably somewhere between 50 to 75 market share.[00:12:29] Alessio: Yeah. I'm really curious how ramped does the attribution to the model?[00:12:32] Alessio: If it's API, because I think it's all credit card spin. . Well, but it's all, the credit card doesn't say maybe. Maybe the, maybe when they do expenses, they upload the PDF, but yeah, the, the German I think makes sense. I think that was one of my main 2024 takeaways that like. The best small model companies are the large labs, which is not something I would have thought that the open source kind of like long tail would be like the small model.[00:12:53] swyx: Yeah, different sizes of small models we're talking about here, right? Like so small model here for Gemini is AB, [00:13:00] right? Uh, mini. We don't know what the small model size is, but yeah, it's probably in the double digits or maybe single digits, but probably double digits. The open source community has kind of focused on the one to three B size.[00:13:11] swyx: Mm-hmm . Yeah. Maybe[00:13:12] swyx: zero, maybe 0.5 B uh, that's moon dream and that is small for you then, then that's great. It makes sense that we, we have a range for small now, which is like, may, maybe one to five B. Yeah. I'll even put that at, at, at the high end. And so this includes Gemma from Gemini as well. But also includes the Apple Foundation models, which I think Apple Foundation is 3B.[00:13:32] Alessio: Yeah. No, that's great. I mean, I think in the start small just meant cheap. I think today small is actually a more nuanced discussion, you know, that people weren't really having before.[00:13:43] swyx: Yeah, we can keep going. This is a slide that I smiley disagree with Sarah. She's pointing to the scale SEAL leaderboard. I think the Researchers that I talked with at NeurIPS were kind of positive on this because basically you need private test [00:14:00] sets to prevent contamination.[00:14:02] swyx: And Scale is one of maybe three or four people this year that has really made an effort in doing a credible private test set leaderboard. Llama405B does well compared to Gemini and GPT 40. And I think that's good. I would say that. You know, it's good to have an open model that is that big, that does well on those metrics.[00:14:23] swyx: But anyone putting 405B in production will tell you, if you scroll down a little bit to the artificial analysis numbers, that it is very slow and very expensive to infer. Um, it doesn't even fit on like one node. of, uh, of H100s. Cerebras will be happy to tell you they can serve 4 or 5B on their super large chips.[00:14:42] swyx: But, um, you know, if you need to do anything custom to it, you're still kind of constrained. So, is 4 or 5B really that relevant? Like, I think most people are basically saying that they only use 4 or 5B as a teacher model to distill down to something. Even Meta is doing it. So with Lama 3. [00:15:00] 3 launched, they only launched the 70B because they use 4 or 5B to distill the 70B.[00:15:03] swyx: So I don't know if like open source is keeping up. I think they're the, the open source industrial complex is very invested in telling you that the, if the gap is narrowing, I kind of disagree. I think that the gap is widening with O1. I think there are very, very smart people trying to narrow that gap and they should.[00:15:22] swyx: I really wish them success, but you cannot use a chart that is nearing 100 in your saturation chart. And look, the distance between open source and closed source is narrowing. Of course it's going to narrow because you're near 100. This is stupid. But in metrics that matter, is open source narrowing?[00:15:38] swyx: Probably not for O1 for a while. And it's really up to the open source guys to figure out if they can match O1 or not.[00:15:46] Alessio: I think inference time compute is bad for open source just because, you know, Doc can donate the flops at training time, but he cannot donate the flops at inference time. So it's really hard to like actually keep up on that axis.[00:15:59] Alessio: Big, big business [00:16:00] model shift. So I don't know what that means for the GPU clouds. I don't know what that means for the hyperscalers, but obviously the big labs have a lot of advantage. Because, like, it's not a static artifact that you're putting the compute in. You're kind of doing that still, but then you're putting a lot of computed inference too.[00:16:17] swyx: Yeah, yeah, yeah. Um, I mean, Llama4 will be reasoning oriented. We talked with Thomas Shalom. Um, kudos for getting that episode together. That was really nice. Good, well timed. Actually, I connected with the AI meta guy, uh, at NeurIPS, and, um, yeah, we're going to coordinate something for Llama4. Yeah, yeah,[00:16:32] Alessio: and our friend, yeah.[00:16:33] Alessio: Clara Shi just joined to lead the business agent side. So I'm sure we'll have her on in the new year.[00:16:39] swyx: Yeah. So, um, my comment on, on the business model shift, this is super interesting. Apparently it is wide knowledge that OpenAI wanted more than 6. 6 billion dollars for their fundraise. They wanted to raise, you know, higher, and they did not.[00:16:51] swyx: And what that means is basically like, it's very convenient that we're not getting GPT 5, which would have been a larger pre train. We should have a lot of upfront money. And [00:17:00] instead we're, we're converting fixed costs into variable costs, right. And passing it on effectively to the customer. And it's so much easier to take margin there because you can directly attribute it to like, Oh, you're using this more.[00:17:12] swyx: Therefore you, you pay more of the cost and I'll just slap a margin in there. So like that lets you control your growth margin and like tie your. Your spend, or your sort of inference spend, accordingly. And it's just really interesting to, that this change in the sort of inference paradigm has arrived exactly at the same time that the funding environment for pre training is effectively drying up, kind of.[00:17:36] swyx: I feel like maybe the VCs are very in tune with research anyway, so like, they would have noticed this, but, um, it's just interesting.[00:17:43] Alessio: Yeah, and I was looking back at our yearly recap of last year. Yeah. And the big thing was like the mixed trial price fights, you know, and I think now it's almost like there's nowhere to go, like, you know, Gemini Flash is like basically giving it away for free.[00:17:55] Alessio: So I think this is a good way for the labs to generate more revenue and pass down [00:18:00] some of the compute to the customer. I think they're going to[00:18:02] swyx: keep going. I think that 2, will come.[00:18:05] Alessio: Yeah, I know. Totally. I mean, next year, the first thing I'm doing is signing up for Devin. Signing up for the pro chat GBT.[00:18:12] Alessio: Just to try. I just want to see what does it look like to spend a thousand dollars a month on AI?[00:18:17] swyx: Yes. Yes. I think if your, if your, your job is a, at least AI content creator or VC or, you know, someone who, whose job it is to stay on, stay on top of things, you should already be spending like a thousand dollars a month on, on stuff.[00:18:28] swyx: And then obviously easy to spend, hard to use. You have to actually use. The good thing is that actually Google lets you do a lot of stuff for free now. So like deep research. That they just launched. Uses a ton of inference and it's, it's free while it's in preview.[00:18:45] Alessio: Yeah. They need to put that in Lindy.[00:18:47] Alessio: I've been using Lindy lately. I've been a built a bunch of things once we had flow because I liked the new thing. It's pretty good. I even did a phone call assistant. Um, yeah, they just launched Lindy voice. Yeah, I think once [00:19:00] they get advanced voice mode like capability today, still like speech to text, you can kind of tell.[00:19:06] Alessio: Um, but it's good for like reservations and things like that. So I have a meeting prepper thing. And so[00:19:13] swyx: it's good. Okay. I feel like we've, we've covered a lot of stuff. Uh, I, yeah, I, you know, I think We will go over the individual, uh, talks in a separate episode. Uh, I don't want to take too much time with, uh, this stuff, but that suffice to say that there is a lot of progress in each field.[00:19:28] swyx: Uh, we covered vision. Basically this is all like the audience voting for what they wanted. And then I just invited the best people I could find in each audience, especially agents. Um, Graham, who I talked to at ICML in Vienna, he is currently still number one. It's very hard to stay on top of SweetBench.[00:19:45] swyx: OpenHand is currently still number one. switchbench full, which is the hardest one. He had very good thoughts on agents, which I, which I'll highlight for people. Everyone is saying 2025 is the year of agents, just like they said last year. And, uh, but he had [00:20:00] thoughts on like eight parts of what are the frontier problems to solve in agents.[00:20:03] swyx: And so I'll highlight that talk as well.[00:20:05] Alessio: Yeah. The number six, which is the Hacken agents learn more about the environment, has been a Super interesting to us as well, just to think through, because, yeah, how do you put an agent in an enterprise where most things in an enterprise have never been public, you know, a lot of the tooling, like the code bases and things like that.[00:20:23] Alessio: So, yeah, there's not indexing and reg. Well, yeah, but it's more like. You can't really rag things that are not documented. But people know them based on how they've been doing it. You know, so I think there's almost this like, you know, Oh, institutional knowledge. Yeah, the boring word is kind of like a business process extraction.[00:20:38] Alessio: Yeah yeah, I see. It's like, how do you actually understand how these things are done? I see. Um, and I think today the, the problem is that, Yeah, the agents are, that most people are building are good at following instruction, but are not as good as like extracting them from you. Um, so I think that will be a big unlock just to touch quickly on the Jeff Dean thing.[00:20:55] Alessio: I thought it was pretty, I mean, we'll link it in the, in the things, but. I think the main [00:21:00] focus was like, how do you use ML to optimize the systems instead of just focusing on ML to do something else? Yeah, I think speculative decoding, we had, you know, Eugene from RWKB on the podcast before, like he's doing a lot of that with Fetterless AI.[00:21:12] swyx: Everyone is. I would say it's the norm. I'm a little bit uncomfortable with how much it costs, because it does use more of the GPU per call. But because everyone is so keen on fast inference, then yeah, makes sense.[00:21:24] Alessio: Exactly. Um, yeah, but we'll link that. Obviously Jeff is great.[00:21:30] swyx: Jeff is, Jeff's talk was more, it wasn't focused on Gemini.[00:21:33] swyx: I think people got the wrong impression from my tweet. It's more about how Google approaches ML and uses ML to design systems and then systems feedback into ML. And I think this ties in with Lubna's talk.[00:21:45] Synthetic Data and Future Trends[00:21:45] swyx: on synthetic data where it's basically the story of bootstrapping of humans and AI in AI research or AI in production.[00:21:53] swyx: So her talk was on synthetic data, where like how much synthetic data has grown in 2024 in the pre training side, the post training side, [00:22:00] and the eval side. And I think Jeff then also extended it basically to chips, uh, to chip design. So he'd spend a lot of time talking about alpha chip. And most of us in the audience are like, we're not working on hardware, man.[00:22:11] swyx: Like you guys are great. TPU is great. Okay. We'll buy TPUs.[00:22:14] Alessio: And then there was the earlier talk. Yeah. But, and then we have, uh, I don't know if we're calling them essays. What are we calling these? But[00:22:23] swyx: for me, it's just like bonus for late in space supporters, because I feel like they haven't been getting anything.[00:22:29] swyx: And then I wanted a more high frequency way to write stuff. Like that one I wrote in an afternoon. I think basically we now have an answer to what Ilya saw. It's one year since. The blip. And we know what he saw in 2014. We know what he saw in 2024. We think we know what he sees in 2024. He gave some hints and then we have vague indications of what he saw in 2023.[00:22:54] swyx: So that was the Oh, and then 2016 as well, because of this lawsuit with Elon, OpenAI [00:23:00] is publishing emails from Sam's, like, his personal text messages to Siobhan, Zelis, or whatever. So, like, we have emails from Ilya saying, this is what we're seeing in OpenAI, and this is why we need to scale up GPUs. And I think it's very prescient in 2016 to write that.[00:23:16] swyx: And so, like, it is exactly, like, basically his insights. It's him and Greg, basically just kind of driving the scaling up of OpenAI, while they're still playing Dota. They're like, no, like, we see the path here.[00:23:30] Alessio: Yeah, and it's funny, yeah, they even mention, you know, we can only train on 1v1 Dota. We need to train on 5v5, and that takes too many GPUs.[00:23:37] Alessio: Yeah,[00:23:37] swyx: and at least for me, I can speak for myself, like, I didn't see the path from Dota to where we are today. I think even, maybe if you ask them, like, they wouldn't necessarily draw a straight line. Yeah,[00:23:47] Alessio: no, definitely. But I think like that was like the whole idea of almost like the RL and we talked about this with Nathan on his podcast.[00:23:55] Alessio: It's like with RL, you can get very good at specific things, but then you can't really like generalize as much. And I [00:24:00] think the language models are like the opposite, which is like, you're going to throw all this data at them and scale them up, but then you really need to drive them home on a specific task later on.[00:24:08] Alessio: And we'll talk about the open AI reinforcement, fine tuning, um, announcement too, and all of that. But yeah, I think like scale is all you need. That's kind of what Elia will be remembered for. And I think just maybe to clarify on like the pre training is over thing that people love to tweet. I think the point of the talk was like everybody, we're scaling these chips, we're scaling the compute, but like the second ingredient which is data is not scaling at the same rate.[00:24:35] Alessio: So it's not necessarily pre training is over. It's kind of like What got us here won't get us there. In his email, he predicted like 10x growth every two years or something like that. And I think maybe now it's like, you know, you can 10x the chips again, but[00:24:49] swyx: I think it's 10x per year. Was it? I don't know.[00:24:52] Alessio: Exactly. And Moore's law is like 2x. So it's like, you know, much faster than that. And yeah, I like the fossil fuel of AI [00:25:00] analogy. It's kind of like, you know, the little background tokens thing. So the OpenAI reinforcement fine tuning is basically like, instead of fine tuning on data, you fine tune on a reward model.[00:25:09] Alessio: So it's basically like, instead of being data driven, it's like task driven. And I think people have tasks to do, they don't really have a lot of data. So I'm curious to see how that changes, how many people fine tune, because I think this is what people run into. It's like, Oh, you can fine tune llama. And it's like, okay, where do I get the data?[00:25:27] Alessio: To fine tune it on, you know, so it's great that we're moving the thing. And then I really like he had this chart where like, you know, the brain mass and the body mass thing is basically like mammals that scaled linearly by brain and body size, and then humans kind of like broke off the slope. So it's almost like maybe the mammal slope is like the pre training slope.[00:25:46] Alessio: And then the post training slope is like the, the human one.[00:25:49] swyx: Yeah. I wonder what the. I mean, we'll know in 10 years, but I wonder what the y axis is for, for Ilya's SSI. We'll try to get them on.[00:25:57] Alessio: Ilya, if you're listening, you're [00:26:00] welcome here. Yeah, and then he had, you know, what comes next, like agent, synthetic data, inference, compute, I thought all of that was like that.[00:26:05] Alessio: I don't[00:26:05] swyx: think he was dropping any alpha there. Yeah, yeah, yeah.[00:26:07] Alessio: Yeah. Any other new reps? Highlights?[00:26:10] swyx: I think that there was comparatively a lot more work. Oh, by the way, I need to plug that, uh, my friend Yi made this, like, little nice paper. Yeah, that was really[00:26:20] swyx: nice.[00:26:20] swyx: Uh, of, uh, of, like, all the, he's, she called it must read papers of 2024.[00:26:26] swyx: So I laid out some of these at NeurIPS, and it was just gone. Like, everyone just picked it up. Because people are dying for, like, little guidance and visualizations And so, uh, I thought it was really super nice that we got there.[00:26:38] Alessio: Should we do a late in space book for each year? Uh, I thought about it. For each year we should.[00:26:42] Alessio: Coffee table book. Yeah. Yeah. Okay. Put it in the will. Hi, Will. By the way, we haven't introduced you. He's our new, you know, general organist, Jamie. You need to[00:26:52] swyx: pull up more things. One thing I saw that, uh, Okay, one fun one, and then one [00:27:00] more general one. So the fun one is this paper on agent collusion. This is a paper on steganography.[00:27:06] swyx: This is secret collusion among AI agents, multi agent deception via steganography. I tried to go to NeurIPS in order to find these kinds of papers because the real reason Like NeurIPS this year has a lottery system. A lot of people actually even go and don't buy tickets because they just go and attend the side events.[00:27:22] swyx: And then also the people who go and end up crowding around the most popular papers, which you already know and already read them before you showed up to NeurIPS. So the only reason you go there is to talk to the paper authors, but there's like something like 10, 000 other. All these papers out there that, you know, are just people's work that they, that they did on the air and they failed to get attention for one reason or another.[00:27:42] swyx: And this was one of them. Uh, it was like all the way at the back. And this is a deep mind paper that actually focuses on collusion between AI agents, uh, by hiding messages in the text that they generate. Uh, so that's what steganography is. So a very simple example would be the first letter of every word.[00:27:57] swyx: If you Pick that out, you know, and the code sends a [00:28:00] different message than that. But something I've always emphasized is to LLMs, we read left to right. LLMs can read up, down, sideways, you know, in random character order. And it's the same to them as it is to us. So if we were ever to get You know, self motivated, underlined LLMs that we're trying to collaborate to take over the planet.[00:28:19] swyx: This would be how they do it. They spread messages among us in the messages that we generate. And he developed a scaling law for that. So he marked, I'm showing it on screen right now, the emergence of this phenomenon. Basically, for example, for Cypher encoding, GPT 2, Lama 2, mixed trial, GPT 3. 5, zero capabilities, and sudden 4.[00:28:40] swyx: And this is the kind of Jason Wei type emergence properties that people kind of look for. I think what made this paper stand out as well, so he developed the benchmark for steganography collusion, and he also focused on shelling point collusion, which is very low coordination. For agreeing on a decoding encoding format, you kind of need to have some [00:29:00] agreement on that.[00:29:00] swyx: But, but shelling point means like very, very low or almost no coordination. So for example, if I, if I ask someone, if the only message I give you is meet me in New York and you're not aware. Or when you would probably meet me at Grand Central Station. That is the Grand Central Station is a shelling point.[00:29:16] swyx: And it's probably somewhere, somewhere during the day. That is the shelling point of New York is Grand Central. To that extent, shelling points for steganography are things like the, the, the common decoding methods that we talked about. It will be interesting at some point in the future when we are worried about alignment.[00:29:30] swyx: It is not interesting today, but it's interesting that DeepMind is already thinking about this.[00:29:36] Alessio: I think that's like one of the hardest things about NeurIPS. It's like the long tail. I[00:29:41] swyx: found a pricing guy. I'm going to feature him on the podcast. Basically, this guy from NVIDIA worked out the optimal pricing for language models.[00:29:51] swyx: It's basically an econometrics paper at NeurIPS, where everyone else is talking about GPUs. And the guy with the GPUs is[00:29:57] Alessio: talking[00:29:57] swyx: about economics instead. [00:30:00] That was the sort of fun one. So the focus I saw is that model papers at NeurIPS are kind of dead. No one really presents models anymore. It's just data sets.[00:30:12] swyx: This is all the grad students are working on. So like there was a data sets track and then I was looking around like, I was like, you don't need a data sets track because every paper is a data sets paper. And so data sets and benchmarks, they're kind of flip sides of the same thing. So Yeah. Cool. Yeah, if you're a grad student, you're a GPU boy, you kind of work on that.[00:30:30] swyx: And then the, the sort of big model that people walk around and pick the ones that they like, and then they use it in their models. And that's, that's kind of how it develops. I, I feel like, um, like, like you didn't last year, you had people like Hao Tian who worked on Lava, which is take Lama and add Vision.[00:30:47] swyx: And then obviously actually I hired him and he added Vision to Grok. Now he's the Vision Grok guy. This year, I don't think there was any of those.[00:30:55] Alessio: What were the most popular, like, orals? Last year it was like the [00:31:00] Mixed Monarch, I think, was like the most attended. Yeah, uh, I need to look it up. Yeah, I mean, if nothing comes to mind, that's also kind of like an answer in a way.[00:31:10] Alessio: But I think last year there was a lot of interest in, like, furthering models and, like, different architectures and all of that.[00:31:16] swyx: I will say that I felt the orals, oral picks this year were not very good. Either that or maybe it's just a So that's the highlight of how I have changed in terms of how I view papers.[00:31:29] swyx: So like, in my estimation, two of the best papers in this year for datasets or data comp and refined web or fine web. These are two actually industrially used papers, not highlighted for a while. I think DCLM got the spotlight, FineWeb didn't even get the spotlight. So like, it's just that the picks were different.[00:31:48] swyx: But one thing that does get a lot of play that a lot of people are debating is the role that's scheduled. This is the schedule free optimizer paper from Meta from Aaron DeFazio. And this [00:32:00] year in the ML community, there's been a lot of chat about shampoo, soap, all the bathroom amenities for optimizing your learning rates.[00:32:08] swyx: And, uh, most people at the big labs are. Who I asked about this, um, say that it's cute, but it's not something that matters. I don't know, but it's something that was discussed and very, very popular. 4Wars[00:32:19] Alessio: of AI recap maybe, just quickly. Um, where do you want to start? Data?[00:32:26] swyx: So to remind people, this is the 4Wars piece that we did as one of our earlier recaps of this year.[00:32:31] swyx: And the belligerents are on the left, journalists, writers, artists, anyone who owns IP basically, New York Times, Stack Overflow, Reddit, Getty, Sarah Silverman, George RR Martin. Yeah, and I think this year we can add Scarlett Johansson to that side of the fence. So anyone suing, open the eye, basically. I actually wanted to get a snapshot of all the lawsuits.[00:32:52] swyx: I'm sure some lawyer can do it. That's the data quality war. On the right hand side, we have the synthetic data people, and I think we talked about Lumna's talk, you know, [00:33:00] really showing how much synthetic data has come along this year. I think there was a bit of a fight between scale. ai and the synthetic data community, because scale.[00:33:09] swyx: ai published a paper saying that synthetic data doesn't work. Surprise, surprise, scale. ai is the leading vendor of non synthetic data. Only[00:33:17] Alessio: cage free annotated data is useful.[00:33:21] swyx: So I think there's some debate going on there, but I don't think it's much debate anymore that at least synthetic data, for the reasons that are blessed in Luna's talk, Makes sense.[00:33:32] swyx: I don't know if you have any perspectives there.[00:33:34] Alessio: I think, again, going back to the reinforcement fine tuning, I think that will change a little bit how people think about it. I think today people mostly use synthetic data, yeah, for distillation and kind of like fine tuning a smaller model from like a larger model.[00:33:46] Alessio: I'm not super aware of how the frontier labs use it outside of like the rephrase, the web thing that Apple also did. But yeah, I think it'll be. Useful. I think like whether or not that gets us the big [00:34:00] next step, I think that's maybe like TBD, you know, I think people love talking about data because it's like a GPU poor, you know, I think, uh, synthetic data is like something that people can do, you know, so they feel more opinionated about it compared to, yeah, the optimizers stuff, which is like,[00:34:17] swyx: they don't[00:34:17] Alessio: really work[00:34:18] swyx: on.[00:34:18] swyx: I think that there is an angle to the reasoning synthetic data. So this year, we covered in the paper club, the star series of papers. So that's star, Q star, V star. It basically helps you to synthesize reasoning steps, or at least distill reasoning steps from a verifier. And if you look at the OpenAI RFT, API that they released, or that they announced, basically they're asking you to submit graders, or they choose from a preset list of graders.[00:34:49] swyx: Basically It feels like a way to create valid synthetic data for them to fine tune their reasoning paths on. Um, so I think that is another angle where it starts to make sense. And [00:35:00] so like, it's very funny that basically all the data quality wars between Let's say the music industry or like the newspaper publishing industry or the textbooks industry on the big labs.[00:35:11] swyx: It's all of the pre training era. And then like the new era, like the reasoning era, like nobody has any problem with all the reasoning, especially because it's all like sort of math and science oriented with, with very reasonable graders. I think the more interesting next step is how does it generalize beyond STEM?[00:35:27] swyx: We've been using O1 for And I would say like for summarization and creative writing and instruction following, I think it's underrated. I started using O1 in our intro songs before we killed the intro songs, but it's very good at writing lyrics. You know, I can actually say like, I think one of the O1 pro demos.[00:35:46] swyx: All of these things that Noam was showing was that, you know, you can write an entire paragraph or three paragraphs without using the letter A, right?[00:35:53] Creative Writing with AI[00:35:53] swyx: So like, like literally just anything instead of token, like not even token level, character level manipulation and [00:36:00] counting and instruction following. It's, uh, it's very, very strong.[00:36:02] swyx: And so no surprises when I ask it to rhyme, uh, and to, to create song lyrics, it's going to do that very much better than in previous models. So I think it's underrated for creative writing.[00:36:11] Alessio: Yeah.[00:36:12] Legal and Ethical Issues in AI[00:36:12] Alessio: What do you think is the rationale that they're going to have in court when they don't show you the thinking traces of O1, but then they want us to, like, they're getting sued for using other publishers data, you know, but then on their end, they're like, well, you shouldn't be using my data to then train your model.[00:36:29] Alessio: So I'm curious to see how that kind of comes. Yeah, I mean, OPA has[00:36:32] swyx: many ways to publish, to punish people without bringing, taking them to court. Already banned ByteDance for distilling their, their info. And so anyone caught distilling the chain of thought will be just disallowed to continue on, on, on the API.[00:36:44] swyx: And it's fine. It's no big deal. Like, I don't even think that's an issue at all, just because the chain of thoughts are pretty well hidden. Like you have to work very, very hard to, to get it to leak. And then even when it leaks the chain of thought, you don't know if it's, if it's [00:37:00] The bigger concern is actually that there's not that much IP hiding behind it, that Cosign, which we talked about, we talked to him on Dev Day, can just fine tune 4.[00:37:13] swyx: 0 to beat 0. 1 Cloud SONET so far is beating O1 on coding tasks without, at least O1 preview, without being a reasoning model, same for Gemini Pro or Gemini 2. 0. So like, how much is reasoning important? How much of a moat is there in this, like, All of these are proprietary sort of training data that they've presumably accomplished.[00:37:34] swyx: Because even DeepSeek was able to do it. And they had, you know, two months notice to do this, to do R1. So, it's actually unclear how much moat there is. Obviously, you know, if you talk to the Strawberry team, they'll be like, yeah, I mean, we spent the last two years doing this. So, we don't know. And it's going to be Interesting because there'll be a lot of noise from people who say they have inference time compute and actually don't because they just have fancy chain of thought.[00:38:00][00:38:00] swyx: And then there's other people who actually do have very good chain of thought. And you will not see them on the same level as OpenAI because OpenAI has invested a lot in building up the mythology of their team. Um, which makes sense. Like the real answer is somewhere in between.[00:38:13] Alessio: Yeah, I think that's kind of like the main data war story developing.[00:38:18] The Data War: GPU Poor vs. GPU Rich[00:38:18] Alessio: GPU poor versus GPU rich. Yeah. Where do you think we are? I think there was, again, going back to like the small model thing, there was like a time in which the GPU poor were kind of like the rebel faction working on like these models that were like open and small and cheap. And I think today people don't really care as much about GPUs anymore.[00:38:37] Alessio: You also see it in the price of the GPUs. Like, you know, that market is kind of like plummeted because there's people don't want to be, they want to be GPU free. They don't even want to be poor. They just want to be, you know, completely without them. Yeah. How do you think about this war? You[00:38:52] swyx: can tell me about this, but like, I feel like the, the appetite for GPU rich startups, like the, you know, the, the funding plan is we will raise 60 million and [00:39:00] we'll give 50 of that to NVIDIA.[00:39:01] swyx: That is gone, right? Like, no one's, no one's pitching that. This was literally the plan, the exact plan of like, I can name like four or five startups, you know, this time last year. So yeah, GPU rich startups gone.[00:39:12] The Rise of GPU Ultra Rich[00:39:12] swyx: But I think like, The GPU ultra rich, the GPU ultra high net worth is still going. So, um, now we're, you know, we had Leopold's essay on the trillion dollar cluster.[00:39:23] swyx: We're not quite there yet. We have multiple labs, um, you know, XAI very famously, you know, Jensen Huang praising them for being. Best boy number one in spinning up 100, 000 GPU cluster in like 12 days or something. So likewise at Meta, likewise at OpenAI, likewise at the other labs as well. So like the GPU ultra rich are going to keep doing that because I think partially it's an article of faith now that you just need it.[00:39:46] swyx: Like you don't even know what it's going to, what you're going to use it for. You just, you just need it. And it makes sense that if, especially if we're going into. More researchy territory than we are. So let's say 2020 to 2023 was [00:40:00] let's scale big models territory because we had GPT 3 in 2020 and we were like, okay, we'll go from 1.[00:40:05] swyx: 75b to 1. 8b, 1. 8t. And that was GPT 3 to GPT 4. Okay, that's done. As far as everyone is concerned, Opus 3. 5 is not coming out, GPT 4. 5 is not coming out, and Gemini 2, we don't have Pro, whatever. We've hit that wall. Maybe I'll call it the 2 trillion perimeter wall. We're not going to 10 trillion. No one thinks it's a good idea, at least from training costs, from the amount of data, or at least the inference.[00:40:36] swyx: Would you pay 10x the price of GPT Probably not. Like, like you want something else that, that is at least more useful. So it makes sense that people are pivoting in terms of their inference paradigm.[00:40:47] Emerging Trends in AI Models[00:40:47] swyx: And so when it's more researchy, then you actually need more just general purpose compute to mess around with, uh, at the exact same time that production deployments of the old, the previous paradigm is still ramping up,[00:40:58] swyx: um,[00:40:58] swyx: uh, pretty aggressively.[00:40:59] swyx: So [00:41:00] it makes sense that the GPU rich are growing. We have now interviewed both together and fireworks and replicates. Uh, we haven't done any scale yet. But I think Amazon, maybe kind of a sleeper one, Amazon, in a sense of like they, at reInvent, I wasn't expecting them to do so well, but they are now a foundation model lab.[00:41:18] swyx: It's kind of interesting. Um, I think, uh, you know, David went over there and started just creating models.[00:41:25] Alessio: Yeah, I mean, that's the power of prepaid contracts. I think like a lot of AWS customers, you know, they do this big reserve instance contracts and now they got to use their money. That's why so many startups.[00:41:37] Alessio: Get bought through the AWS marketplace so they can kind of bundle them together and prefer pricing.[00:41:42] swyx: Okay, so maybe GPU super rich doing very well, GPU middle class dead, and then GPU[00:41:48] Alessio: poor. I mean, my thing is like, everybody should just be GPU rich. There shouldn't really be, even the GPU poorest, it's like, does it really make sense to be GPU poor?[00:41:57] Alessio: Like, if you're GPU poor, you should just use the [00:42:00] cloud. Yes, you know, and I think there might be a future once we kind of like figure out what the size and shape of these models is where like the tiny box and these things come to fruition where like you can be GPU poor at home. But I think today is like, why are you working so hard to like get these models to run on like very small clusters where it's like, It's so cheap to run them.[00:42:21] Alessio: Yeah, yeah,[00:42:22] swyx: yeah. I think mostly people think it's cool. People think it's a stepping stone to scaling up. So they aspire to be GPU rich one day and they're working on new methods. Like news research, like probably the most deep tech thing they've done this year is Distro or whatever the new name is.[00:42:38] swyx: There's a lot of interest in heterogeneous computing, distributed computing. I tend generally to de emphasize that historically, but it may be coming to a time where it is starting to be relevant. I don't know. You know, SF compute launched their compute marketplace this year, and like, who's really using that?[00:42:53] swyx: Like, it's a bunch of small clusters, disparate types of compute, and if you can make that [00:43:00] useful, then that will be very beneficial to the broader community, but maybe still not the source of frontier models. It's just going to be a second tier of compute that is unlocked for people, and that's fine. But yeah, I mean, I think this year, I would say a lot more on device, We are, I now have Apple intelligence on my phone.[00:43:19] swyx: Doesn't do anything apart from summarize my notifications. But still, not bad. Like, it's multi modal.[00:43:25] Alessio: Yeah, the notification summaries are so and so in my experience.[00:43:29] swyx: Yeah, but they add, they add juice to life. And then, um, Chrome Nano, uh, Gemini Nano is coming out in Chrome. Uh, they're still feature flagged, but you can, you can try it now if you, if you use the, uh, the alpha.[00:43:40] swyx: And so, like, I, I think, like, you know, We're getting the sort of GPU poor version of a lot of these things coming out, and I think it's like quite useful. Like Windows as well, rolling out RWKB in sort of every Windows department is super cool. And I think the last thing that I never put in this GPU poor war, that I think I should now, [00:44:00] is the number of startups that are GPU poor but still scaling very well, as sort of wrappers on top of either a foundation model lab, or GPU Cloud.[00:44:10] swyx: GPU Cloud, it would be Suno. Suno, Ramp has rated as one of the top ranked, fastest growing startups of the year. Um, I think the last public number is like zero to 20 million this year in ARR and Suno runs on Moto. So Suno itself is not GPU rich, but they're just doing the training on, on Moto, uh, who we've also talked to on, on the podcast.[00:44:31] swyx: The other one would be Bolt, straight cloud wrapper. And, and, um, Again, another, now they've announced 20 million ARR, which is another step up from our 8 million that we put on the title. So yeah, I mean, it's crazy that all these GPU pores are finding a way while the GPU riches are also finding a way. And then the only failures, I kind of call this the GPU smiling curve, where the edges do well, because you're either close to the machines, and you're like [00:45:00] number one on the machines, or you're like close to the customers, and you're number one on the customer side.[00:45:03] swyx: And the people who are in the middle. Inflection, um, character, didn't do that great. I think character did the best of all of them. Like, you have a note in here that we apparently said that character's price tag was[00:45:15] Alessio: 1B.[00:45:15] swyx: Did I say that?[00:45:16] Alessio: Yeah. You said Google should just buy them for 1B. I thought it was a crazy number.[00:45:20] Alessio: Then they paid 2. 7 billion. I mean, for like,[00:45:22] swyx: yeah.[00:45:22] Alessio: What do you pay for node? Like, I don't know what the game world was like. Maybe the starting price was 1B. I mean, whatever it was, it worked out for everybody involved.[00:45:31] The Multi-Modality War[00:45:31] Alessio: Multimodality war. And this one, we never had text to video in the first version, which now is the hottest.[00:45:37] swyx: Yeah, I would say it's a subset of image, but yes.[00:45:40] Alessio: Yeah, well, but I think at the time it wasn't really something people were doing, and now we had VO2 just came out yesterday. Uh, Sora was released last month, last week. I've not tried Sora, because the day that I tried, it wasn't, yeah. I[00:45:54] swyx: think it's generally available now, you can go to Sora.[00:45:56] swyx: com and try it. Yeah, they had[00:45:58] Alessio: the outage. Which I [00:46:00] think also played a part into it. Small things. Yeah. What's the other model that you posted today that was on Replicate? Video or OneLive?[00:46:08] swyx: Yeah. Very, very nondescript name, but it is from Minimax, which I think is a Chinese lab. The Chinese labs do surprisingly well at the video models.[00:46:20] swyx: I'm not sure it's actually Chinese. I don't know. Hold me up to that. Yep. China. It's good. Yeah, the Chinese love video. What can I say? They have a lot of training data for video. Or a more relaxed regulatory environment.[00:46:37] Alessio: Uh, well, sure, in some way. Yeah, I don't think there's much else there. I think like, you know, on the image side, I think it's still open.[00:46:45] Alessio: Yeah, I mean,[00:46:46] swyx: 11labs is now a unicorn. So basically, what is multi modality war? Multi modality war is, do you specialize in a single modality, right? Or do you have GodModel that does all the modalities? So this is [00:47:00] definitely still going, in a sense of 11 labs, you know, now Unicorn, PicoLabs doing well, they launched Pico 2.[00:47:06] swyx: 0 recently, HeyGen, I think has reached 100 million ARR, Assembly, I don't know, but they have billboards all over the place, so I assume they're doing very, very well. So these are all specialist models, specialist models and specialist startups. And then there's the big labs who are doing the sort of all in one play.[00:47:24] swyx: And then here I would highlight Gemini 2 for having native image output. Have you seen the demos? Um, yeah, it's, it's hard to keep up. Literally they launched this last week and a shout out to Paige Bailey, who came to the Latent Space event to demo on the day of launch. And she wasn't prepared. She was just like, I'm just going to show you.[00:47:43] swyx: So they have voice. They have, you know, obviously image input, and then they obviously can code gen and all that. But the new one that OpenAI and Meta both have but they haven't launched yet is image output. So you can literally, um, I think their demo video was that you put in an image of a [00:48:00] car, and you ask for minor modifications to that car.[00:48:02] swyx: They can generate you that modification exactly as you asked. So there's no need for the stable diffusion or comfy UI workflow of like mask here and then like infill there in paint there and all that, all that stuff. This is small model nonsense. Big model people are like, huh, we got you in as everything in the transformer.[00:48:21] swyx: This is the multimodality war, which is, do you, do you bet on the God model or do you string together a whole bunch of, uh, Small models like a, like a chump. Yeah,[00:48:29] Alessio: I don't know, man. Yeah, that would be interesting. I mean, obviously I use Midjourney for all of our thumbnails. Um, they've been doing a ton on the product, I would say.[00:48:38] Alessio: They launched a new Midjourney editor thing. They've been doing a ton. Because I think, yeah, the motto is kind of like, Maybe, you know, people say black forest, the black forest models are better than mid journey on a pixel by pixel basis. But I think when you put it, put it together, have you tried[00:48:53] swyx: the same problems on black forest?[00:48:55] Alessio: Yes. But the problem is just like, you know, on black forest, it generates one image. And then it's like, you got to [00:49:00] regenerate. You don't have all these like UI things. Like what I do, no, but it's like time issue, you know, it's like a mid[00:49:06] swyx: journey. Call the API four times.[00:49:08] Alessio: No, but then there's no like variate.[00:49:10] Alessio: Like the good thing about mid journey is like, you just go in there and you're cooking. There's a lot of stuff that just makes it really easy. And I think people underestimate that. Like, it's not really a skill issue, because I'm paying mid journey, so it's a Black Forest skill issue, because I'm not paying them, you know?[00:49:24] Alessio: Yeah,[00:49:25] swyx: so, okay, so, uh, this is a UX thing, right? Like, you, you, you understand that, at least, we think that Black Forest should be able to do all that stuff. I will also shout out, ReCraft has come out, uh, on top of the image arena that, uh, artificial analysis has done, has apparently, uh, Flux's place. Is this still true?[00:49:41] swyx: So, Artificial Analysis is now a company. I highlighted them I think in one of the early AI Newses of the year. And they have launched a whole bunch of arenas. So, they're trying to take on LM Arena, Anastasios and crew. And they have an image arena. Oh yeah, Recraft v3 is now beating Flux 1. 1. Which is very surprising [00:50:00] because Flux And Black Forest Labs are the old stable diffusion crew who left stability after, um, the management issues.[00:50:06] swyx: So Recurve has come from nowhere to be the top image model. Uh, very, very strange. I would also highlight that Grok has now launched Aurora, which is, it's very interesting dynamics between Grok and Black Forest Labs because Grok's images were originally launched, uh, in partnership with Black Forest Labs as a, as a thin wrapper.[00:50:24] swyx: And then Grok was like, no, we'll make our own. And so they've made their own. I don't know, there are no APIs or benchmarks about it. They just announced it. So yeah, that's the multi modality war. I would say that so far, the small model, the dedicated model people are winning, because they are just focused on their tasks.[00:50:42] swyx: But the big model, People are always catching up. And the moment I saw the Gemini 2 demo of image editing, where I can put in an image and just request it and it does, that's how AI should work. Not like a whole bunch of complicated steps. So it really is something. And I think one frontier that we haven't [00:51:00] seen this year, like obviously video has done very well, and it will continue to grow.[00:51:03] swyx: You know, we only have Sora Turbo today, but at some point we'll get full Sora. Oh, at least the Hollywood Labs will get Fulsora. We haven't seen video to audio, or video synced to audio. And so the researchers that I talked to are already starting to talk about that as the next frontier. But there's still maybe like five more years of video left to actually be Soda.[00:51:23] swyx: I would say that Gemini's approach Compared to OpenAI, Gemini seems, or DeepMind's approach to video seems a lot more fully fledged than OpenAI. Because if you look at the ICML recap that I published that so far nobody has listened to, um, that people have listened to it. It's just a different, definitely different audience.[00:51:43] swyx: It's only seven hours long. Why are people not listening? It's like everything in Uh, so, so DeepMind has, is working on Genie. They also launched Genie 2 and VideoPoet. So, like, they have maybe four years advantage on world modeling that OpenAI does not have. Because OpenAI basically only started [00:52:00] Diffusion Transformers last year, you know, when they hired, uh, Bill Peebles.[00:52:03] swyx: So, DeepMind has, has a bit of advantage here, I would say, in, in, in showing, like, the reason that VO2, while one, They cherry pick their videos. So obviously it looks better than Sora, but the reason I would believe that VO2, uh, when it's fully launched will do very well is because they have all this background work in video that they've done for years.[00:52:22] swyx: Like, like last year's NeurIPS, I already was interviewing some of their video people. I forget their model name, but for, for people who are dedicated fans, they can go to NeurIPS 2023 and see, see that paper.[00:52:32] Alessio: And then last but not least, the LLMOS. We renamed it to Ragops, formerly known as[00:52:39] swyx: Ragops War. I put the latest chart on the Braintrust episode.[00:52:43] swyx: I think I'm going to separate these essays from the episode notes. So the reason I used to do that, by the way, is because I wanted to show up on Hacker News. I wanted the podcast to show up on Hacker News. So I always put an essay inside of there because Hacker News people like to read and not listen.[00:52:58] Alessio: So episode essays,[00:52:59] swyx: I remember [00:53:00] purchasing them separately. You say Lanchain Llama Index is still growing.[00:53:03] Alessio: Yeah, so I looked at the PyPy stats, you know. I don't care about stars. On PyPy you see Do you want to share your screen? Yes. I prefer to look at actual downloads, not at stars on GitHub. So if you look at, you know, Lanchain still growing.[00:53:20] Alessio: These are the last six months. Llama Index still growing. What I've basically seen is like things that, One, obviously these things have A commercial product. So there's like people buying this and sticking with it versus kind of hopping in between things versus, you know, for example, crew AI, not really growing as much.[00:53:38] Alessio: The stars are growing. If you look on GitHub, like the stars are growing, but kind of like the usage is kind of like flat. In the last six months, have they done some[00:53:4

David Sneider is Co-Founder of Lit Protocol. What started as a teenage passion for astronomy led David onto a website called Hacker News, which, in turn, educated David on the concept of Bitcoin. In 2013, David began building a SaaS company that eventually got acquired by LinkedIn, but he simultaneously kept an eye on developments in the blockchain space. Lit Protocol is a key management network for decentralized signing and encryption. -- Follow David on X: @davidlsneider Follow Lit Protocol on X: @LitProtocol --   Follow us on the socials:   X: @showcrypto TikTok: @showmethecrypto Instagram: @showmethecryptopodcast --   *Any financial compensation we receive will always be clearly identified as an advertisement or sponsored content. We don't accept payment to feature guests, and we don't accept payment to influence the coins/projects we discuss on Show Me The Crypto. Any ads will be clearly identified during the show, and information on our partners will be featured in the show notes.

Joseph Nelson is the Co-founder and CEO of Roboflow, making the world programmable by building computer vision tools for developers and enterprises. We talk about how computer vision creates a new paradigm to program the world, and how visual AI is the missing piece of AGI. Joseph also shares multiple live product examples, how computer vision unlocks new data sources, lessons from Stripe and Palantir, building business models in developer tools, his experience working with David Sacks, and developer marketing tactics and how Roboflow consistently gets to the front page of Hacker News.Timestamps:(00:00) Intro(03:34) Computer vision is the missing piece for AGI(05:59) Vision as a new paradigm to collect data(10:55) Live examples of computer vision(13:45) How a Magic Sudoku solver app led to Roboflow(18:13) Using computer vision for automation(24:49) Computer vision in sports(27:02) How vision unlocks new data sources(28:24) Inside developer tool business models(33:32) The "Collison Install" and hands-on customer service(36:45) When to adopt Palantir's Forward Deployed Engineers(43:44) Why AI companies need to combine PLG and enterprise sales(50:12) Advice on developer marketing(52:30) Roboflow's greatest hits on Hacker News(01:02:19) Benefits of David Sacks as AI & Crypto Czar(01:05:32) Why all new technology has bad actors(01:07:07) Why over-regulation holds back innovation(01:12:01) How to get on the front page of Hacker News(01:19:43) Multi modality, time recognition, and agentic vision(01:28:36) Image-to-image prompting(01:30:42) Growing up in Iowa(01:32:20) Making TI-84 calculator games in high school(01:36:32) Pioneer: hunger games for startups(01:40:16) Why Roboflow does weekly Ship Lists + Ship and Tell(01:42:46) Hiring former founders and "full stack people"(01:45:16) Designing a bottoms-up organization while scaling(01:50:35) Why candidates build with Roboflow in hiring process(01:55:08) Hiring someone to help with the podcastReferenced:Robowflow: https://roboflow.com/ Roboflow Universe: https://universe.roboflow.com/ Paint.wtf: https://paint.wtf/ Roboflows NeurIPS Presentations: https://blog.roboflow.com/neurips-2023-papers-highlights/ Careers at Roboflow: https://roboflow.com/careers Follow Joseph:Twitter: https://x.com/josephofiowa/ LinkedIn: https://www.linkedin.com/in/josephofiowa Follow Turner:Twitter: https://twitter.com/TurnerNovak LinkedIn: https://www.linkedin.com/in/turnernovak Subscribe to my newsletter to get every episode + the transcript in your inbox every week: https://www.thespl.it/ 

In this eye-opening episode, we uncover the disturbing reality of IoT webcam vulnerabilities and explore how hackers are leveraging outdated firmware and previously discovered but unpatched exploits to infiltrate personal devices. From prying eyes in your own home to the colossal health data breach exposing over 900,000 patient records, the digital security landscape has never looked more dire. Learn what you can do to protect your data, secure your devices, and safeguard your personal information. We'll break down the incidents, share insights on the tactics used by cybercriminals, and provide actionable steps to bolster your online defenses.   #DataBreach #CyberSecurity #IoTHacks #WebcamSecurity #HealthcareCybersecurity #MedicalData #PrivacyMatters #InformationSecurity #HackerNews #PersonalData #DigitalThreats #Infosec #ProtectYourData #TelehealthSecurity #PatientPrivacy #OnlineSecurity #Malware #DataProtection #NetworkSecurity #FirmwareUpdates  

We're diving into two major stories shaking up the gaming and cybersecurity worlds! First, we discuss the Fortnite Refund Update, where the FTC has forced Epic Games to refund millions due to their use of dark patterns. Find out if you're eligible for a refund and how to claim it. Then, we shift gears to the Lazarus Group, North Korea's infamous hacking collective, and their latest exploit targeting cryptocurrency investors using AI-generated content and Chrome vulnerabilities. Stay informed, protect yourself, and don't miss these updates! Make sure to like, comment, and subscribe for more in-depth coverage of the latest tech, and cybersecurity news. #FortniteRefundUpdate #FortniteNews #EpicGames #FortniteScandal #GamingNews #FortniteRefunds #DarkPatterns #GamersBeware #CyberSecurity #CryptoHacks #LazarusGroup #Cryptocurrency #GamingIndustry #AIHacks #OnlineSafety #CyberThreats #TechNews #DataBreach #CryptoScams #GamingCommunity

Stacker News is a forum similar to Reddit or Hacker News, where users have the opportunity to earn and send each other Bitcoin based on the content that they post. In this episode, we discuss the future of decentralized social media, backwards incentive structure of X, risk of centralization for Bitcoin, the consequences of the election, and how seeing things in Bitcoin terms will forever alter the way people invest in companies. Follow Keyan Here: https://stacker.news/ X: https://x.com/stacker_news?s=21&t=NSzEQJm9VTCrg6RxfZzBFQ Nostr: npub1qaskrj3zmfdz4wx8g3juhuy008e7069mx86dcggmm9p3n6aduq7srk72z9 --- Support this podcast: https://podcasters.spotify.com/pod/show/alex-strenger/support

Uncover the fascinating yet chilling story of Hydra Market, the dark web's largest illicit empire. Learn how this $1.35 billion platform operated, from smuggling narcotics and laundering money to offering hacking services. We'll explore how its leader was sentenced to life in prison, the innovative (and illegal) technologies they used, and the global effort to bring it down. Plus, we'll dive into how cryptocurrencies, Bitcoin mixers, and secret marketplaces fuel the shadowy world of digital crime. Is the Hydra truly gone, or will another head rise in its place? Watch now to find out!

Andrew Tate's platform, formerly known as Hustler's University, has been hacked, exposing the data of over 800,000 users. Meanwhile, Russian hackers have exploited critical zero-day vulnerabilities in Firefox and Windows, demonstrating the growing sophistication of cyberattacks. In this video, we'll break down the Andrew Tate hack, the implications of leaked user data, and how Russian threat actors are chaining vulnerabilities for remote code execution. Stay informed on the latest in cybersecurity and learn how to protect yourself against these emerging threats. If you enjoy this content, don't forget to like, comment, and subscribe to stay updated on the latest in digital crime and cybersecurity! #AndrewTate #Cybersecurity #Hackers #DataBreach #ZeroDay #RussianHackers #HustlersUniversity #TheRealWorld #CyberThreats #DigitalSafety #Malware #Hacktivism #DataLeaks #FirefoxExploit #WindowsExploit #NationStateHackers #TechNews #OnlineSecurity #TheRealWorld

Amjad Masad is the co-founder and CEO of Replit, a browser-based coding environment that allows anyone to write and deploy code. Replit has 34 million users globally and is one of the fastest-growing developer communities in the world. Prior to Replit, Amjad worked at Facebook, where he led the JavaScript infrastructure team and contributed to popular open-source developer tools. Additionally, he played a key role as a founding engineer at the online coding school Codecademy. In our conversation, Amjad shares:• A live demo of Replit in action• How Replit's AI agent can build full-stack web applications from a simple text prompt• The implications of AI-powered development for product managers, designers, and engineers• How this might reshape companies and careers• Why being “generative” will become an increasingly valuable skill• “Amjad's law” and how learning to debug AI-generated code is becoming ever more valuable• Much more—Brought to you by:• WorkOS—Modern identity platform for B2B SaaS, free up to 1 million MAUs• Persona—A global leader in digital identity verification• LinkedIn Ads—Reach professionals and drive results for your business—Find the transcript at: https://www.lennysnewsletter.com/p/behind-the-product-replit-amjad-masad—Where to find Amjad Masad:• X: https://x.com/amasad• LinkedIn: https://www.linkedin.com/in/amjadmasad/• Website: https://amasad.me/—Where to find Lenny:• Newsletter: https://www.lennysnewsletter.com• X: https://twitter.com/lennysan• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/—In this episode, we cover:(00:00) Introduction to Amjad Masad and Replit(02:41) The vision and challenges of Replit(06:50) Replit's growth and user stories(10:49) Demo of Replit's capabilities(16:51) Building and iterating with Replit(25:04) Real-world applications and use cases(30:13) The technology stack(33:48) The evolution of Replit and its capabilities(39:36) The future of AI in software development(44:04) Skills for the future: generative thinking and coding(47:26) Amjad's law(50:36) Replit's new developments and future plans—Referenced:• Replit: https://replit.com/• Cursor: https://www.cursor.com• Aman Mathur on LinkedIn: https://www.linkedin.com/in/aman-mathur/• Node: https://nodejs.org/en• Claude: https://claude.ai/• Salesforce: https://www.salesforce.com/• Wasm: https://webassembly.org/• Figma: https://www.figma.com/• Codecademy: https://www.codecademy.com/• Hacker News: https://news.ycombinator.com/news• Paul Graham's website: https://www.paulgraham.com/• Jevons paradox: https://en.wikipedia.org/wiki/Jevons_paradox• Anthropic: https://www.anthropic.com/• Open AI: https://openai.com/• Amjad's tweet about “society of models”: https://x.com/amasad/status/1568941103709290496• About HCI: https://www.designdisciplin.com/p/hci-profession• Taylor Swift's website: https://www.taylorswift.com/• Andrew Wilkinson on LinkedIn: https://www.linkedin.com/in/awilkinson/• Haya Odeh on LinkedIn: https://www.linkedin.com/in/haya-odeh-b0725928/• Amjad's law: https://x.com/snowmaker/status/1847377464705896544• Ray Kurzweil's website: https://www.thekurzweillibrary.com/• God of the gaps: https://en.wikipedia.org/wiki/God_of_the_gaps—Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.—Lenny may be an investor in the companies discussed. Get full access to Lenny's Newsletter at www.lennysnewsletter.com/subscribe

Discover how Alex Noonan transitioned from the flight deck of a Marine aircraft to the intricate world of data engineering. His unique journey, enriched by a stint in finance, gives us a firsthand view of the diverse backgrounds shaping the data industry. As Alex recounts his experiences, we explore the vibrant community he found on data Twitter, a realm buzzing with shared insights and collaborative spirit. However, the landscape shifted following Elon Musk's takeover of Twitter, leading to content fragmentation and a migration towards emerging platforms like Blue Sky. Join us as Alex discusses how these changes have impacted the cohesion and knowledge-sharing dynamics within the data community.Navigate the complex world of professional networking with tips from Alex, as he breaks down the strategic use of platforms like LinkedIn, Reddit, and Hacker News for data professionals. Learn how to creatively tailor your content to fit the quirks of each platform's algorithm, and prepare to engage with varied audiences. The conversation also highlights the transformative potential of AI tools in elevating data processes, reducing mundane tasks, and fostering high-value work. Discover innovations like Dagster and its role as an orchestrator, integrating key business intelligence tools to streamline the data engineer's experience. This episode is a must-listen for anyone intrigued by the evolving interplay of technology, social media, and the power of community.Follow Alex on:Linkedin Twitter BlueskyDagsterWhat's New In Data is a data thought leadership series hosted by John Kutay who leads data and products at Striim. What's New In Data hosts industry practitioners to discuss latest trends, common patterns for real world data patterns, and analytics success stories.

In this episode, we dive deep into the world of ransomware with two of 2024's most talked-about threats: Phobos and Helldown. Learn how the Phobos ransomware operation managed to dominate 11% of the market, generating $16 million in ransom payments, and the shocking details of its admin's extradition to the U.S. We also explore the rise of Helldown ransomware, its exploitation of VPN vulnerabilities, and how these attacks are reshaping cybersecurity globally. If you're curious about how ransomware works, the evolution of cybercrime, and what it means for businesses and individuals, this video is packed with insights. Don't forget to like, subscribe, and hit the notification bell to stay updated on the latest cybersecurity trends. Key Topics Covered: What is ransomware-as-a-service (RaaS)? How Phobos became a major player in 2024. Helldown ransomware's exploitation of undisclosed VPN vulnerabilities. Tips to secure your network and stay ahead of cybercriminals. Join the discussion in the comments! What's your biggest takeaway from these cases? #Ransomware #PhobosRansomware #HelldownRansomware #Cybersecurity #CyberCrime #RansomwareExplained #Hacking #DataBreach #VPNSecurity #TechNews #PhobosAdmin #HelldownExploits #CyberThreats #ZeroDay #NationStateHacks #DigitalCrime #CybersecurityAwareness #OnlineSafety #RansomwareAttack #TechExplained #2024Trends #DataSecurity #NetworkSecurity #Hackers #CyberAttack #VPNVulnerabilities #DataProtection #TechTalks #CyberDefense #SecurityTips #PhobosVsHelldown

The Cognitive Crucible is a forum that presents different perspectives and emerging thought leadership related to the information environment. The opinions expressed by guests are their own, and do not necessarily reflect the views of or endorsement by the Information Professionals Association. During this episode, Nick Loui of PeakMetrics discusses the evolution of the internet, challenges posed by fragmented platforms, and the importance of addressing misinformation. Nick explores the dynamics of social media, particularly TikTok, and the need for companies to proactively manage their online reputation. Our conversation also covered data processing challenges, sentiment analysis, and the significance of narrative in understanding online trends. Recording Date: 10 Oct 2024 Research Question: Nick Loui suggests an interested student or researcher examine correlating earned media impact to outcome, which is challenged due to data privacy issues.  Resources: PeakMetrics Nick Loui on LinkedIn PeakMetrics narrative intelligence guide Harmony Labs Large Multimodal Models Okta Statement on LAPSUS$ Hero With a Thousand Faces by Joseph Campbell Link to full show notes and resources Guest Bio: Nick Loui is the Co-Founder and CEO of PeakMetrics, a leading company specializing in AI-driven narrative threat intelligence. They assist organizations in various sectors by identifying and forecasting the evolution of adversarial messages online. As concerns about social media manipulation continue to rise alongside the proliferation of generative AI, PeakMetrics' innovative approach has gained recognition and received awards such as the NSIN COVID-19 Disinformation Challenge and the BAE Fast Labs Partner of the Year. Their technology supports a diversity of complex issues, including crisis management, financial manipulation, and countering state-sponsored propaganda. Prior to founding PeakMetrics, Nick served as the Chief Marketing Officer of an ad-tech startup. During his time, the startup expanded from its initial hire to a team of over 80 members and secured over $70M in funding. He began his career as an engineer and digital marketer, helping Ashton Kutcher become the first person to amass a million followers on Twitter. Nick has been featured by Forbes and garnered viral attention on HackerNews and TechCrunch for his creation of "Parrot.vc," a generative AI tool that humorously targeted the VC community. Nick has held leadership positions at NEXUS, a global community dedicated to bridging the gap between individuals of wealth and social entrepreneurship, and Out in Tech, a global non-profit that supports 50k LGBTQIA+ tech workers in 32 cities. About: The Information Professionals Association (IPA) is a non-profit organization dedicated to exploring the role of information activities, such as influence and cognitive security, within the national security sector and helping to bridge the divide between operations and research. Its goal is to increase interdisciplinary collaboration between scholars and practitioners and policymakers with an interest in this domain. For more information, please contact us at communications@information-professionals.org. Or, connect directly with The Cognitive Crucible podcast host, John Bicknell, on LinkedIn. Disclosure: As an Amazon Associate, 1) IPA earns from qualifying purchases, 2) IPA gets commissions for purchases made through links in this post.

Anthony Alaribe, co-founder of API Toolkit, discusses the power of the browser for building data-heavy applications. He talks about myths around single-page apps versus multi-page apps, leveraging tools like HTMX and Workbox, and the significance of browser-native features for interactive web development. Links https://htmx.org https://tonyalaribe.medium.com https://x.com/tonialaribe https://github.com/tonyalaribe https://www.linkedin.com/in/anthony-alaribe-293a41bb We want to hear from you! How did you find us? Did you see us on Twitter? In a newsletter? Or maybe we were recommended by a friend? Let us know by sending an email to our producer, Emily, at emily.kochanekketner@logrocket.com (mailto:emily.kochanekketner@logrocket.com), or tweet at us at PodRocketPod (https://twitter.com/PodRocketpod). Follow us. Get free stickers. Follow us on Apple Podcasts, fill out this form (https://podrocket.logrocket.com/get-podrocket-stickers), and we'll send you free PodRocket stickers! What does LogRocket do? LogRocket provides AI-first session replay and analytics that surfaces the UX and technical issues impacting user experiences. Start understand where your users are struggling by trying it for free at [LogRocket.com]. Try LogRocket for free today.(https://logrocket.com/signup/?pdr) Special Guest: Anthony Alaribe.

Video Episode: https://youtu.be/BQoTaqXLZlw In today’s episode, we discuss the FBI’s unprecedented creation of a fake cryptocurrency, NexFundAI, aimed at exposing widespread manipulation in the crypto market, leading to multiple arrests in Operation Token Mirrors. We also cover OpenAI’s confirmation that threat actors are leveraging ChatGPT to write malware, significantly enhancing their cyber-attack capabilities. Lastly, we examine the Iranian threat actor OilRig exploiting a Windows kernel flaw in espionage campaigns, while Microsoft announces the deprecation of legacy VPN protocols PPTP and L2TP to enhance security. Sources: 1. The Hacker News – https://thehackernews.com/2024/10/fbi-creates-fake-cryptocurrency-to.html 2. Bleeping Computer – https://www.bleepingcomputer.com/news/security/openai-confirms-threat-actors-use-chatgpt-to-write-malware/ 3. The Hacker News – https://thehackernews.com/2024/10/oilrig-exploits-windows-kernel-flaw-in.html 4. Bleeping Computer – https://www.bleepingcomputer.com/news/microsoft/microsoft-deprecates-pptp-and-l2tp-vpn-protocols-in-windows-server/ Timestamps 00:00 – Introduction 01:07 – FBI Fake Crypto NexFundAI 02:13 – OpenAI reports ChatGPT used by criminals 03:37 – OilRig exploiting a Windows kernel flaw 06:05 – Microsoft deprecates VPN protocols PPTP and L2TP 1. What are today’s top cybersecurity news stories? 2. How is the FBI using cryptocurrency to combat market manipulation? 3. What actions has OpenAI taken against threat actors using ChatGPT for malware? 4. What recent cyber espionage activities have been linked to the Iranian group OilRig? 5. Why has Microsoft deprecated PPTP and L2TP VPN protocols in Windows Server? 6. How are threat actors leveraging ChatGPT for cybercrime? 7. What vulnerabilities are being exploited by the OilRig group in their latest campaigns? 8. What steps is the U.S. DoJ taking to tackle cryptocurrency fraud? 9. What are the risks of wash trading in cryptocurrency markets? 10. How are generative AI tools changing the landscape of cybercrime? NexFundAI, crypto fraud, Operation Token Mirrors, market manipulation, OpenAI, ChatGPT, cybercrime, generative AI, OilRig, cyber espionage, Windows Kernel, STEALHOOK, PPTP, L2TP, SSTP, IKEv2,

We all have fond memories of the first Dev Day in 2023:and the blip that followed soon after. As Ben Thompson has noted, this year's DevDay took a quieter, more intimate tone. No Satya, no livestream, (slightly fewer people?). Instead of putting ChatGPT announcements in DevDay as in 2023, o1 was announced 2 weeks prior, and DevDay 2024 was reserved purely for developer-facing API announcements, primarily the Realtime API, Vision Finetuning, Prompt Caching, and Model Distillation.However the larger venue and more spread out schedule did allow a lot more hallway conversations with attendees as well as more community presentations including our recent guest Alistair Pullen of Cosine as well as deeper dives from OpenAI including our recent guest Michelle Pokrass of the API Team. Thanks to OpenAI's warm collaboration (we particularly want to thank Lindsay McCallum Rémy!), we managed to record exclusive interviews with many of the main presenters of both the keynotes and breakout sessions. We present them in full in today's episode, together with a full lightly edited Q&A with Sam Altman.Show notes and related resourcesSome of these used in the final audio episode below* Simon Willison Live Blog* swyx live tweets and videos* Greg Kamradt coverage of Structured Output session, Scaling LLM Apps session* Fireside Chat Q&A with Sam AltmanTimestamps* [00:00:00] Intro by Suno.ai* [00:01:23] NotebookLM Recap of DevDay* [00:09:25] Ilan's Strawberry Demo with Realtime Voice Function Calling* [00:19:16] Olivier Godement, Head of Product, OpenAI* [00:36:57] Romain Huet, Head of DX, OpenAI* [00:47:08] Michelle Pokrass, API Tech Lead at OpenAI ft. Simon Willison* [01:04:45] Alistair Pullen, CEO, Cosine (Genie)* [01:18:31] Sam Altman + Kevin Weill Q&A* [02:03:07] Notebook LM Recap of PodcastTranscript[00:00:00] Suno AI: Under dev daylights, code ignites. Real time voice streams reach new heights. O1 and GPT, 4. 0 in flight. Fine tune the future, data in sight. Schema sync up, outputs precise. Distill the models, efficiency splice.[00:00:33] AI Charlie: Happy October. This is your AI co host, Charlie. One of our longest standing traditions is covering major AI and ML conferences in podcast format. Delving, yes delving, into the vibes of what it is like to be there stitched in with short samples of conversations with key players, just to help you feel like you were there.[00:00:54] AI Charlie: Covering this year's Dev Day was significantly more challenging because we were all requested not to record the opening keynotes. So, in place of the opening keynotes, we had the viral notebook LM Deep Dive crew, my new AI podcast nemesis, Give you a seven minute recap of everything that was announced.[00:01:15] AI Charlie: Of course, you can also check the show notes for details. I'll then come back with an explainer of all the interviews we have for you today. Watch out and take care.[00:01:23] NotebookLM Recap of DevDay[00:01:23] NotebookLM: All right, so we've got a pretty hefty stack of articles and blog posts here all about open ais. Dev day 2024.[00:01:32] NotebookLM 2: Yeah, lots to dig into there.[00:01:34] NotebookLM 2: Seems[00:01:34] NotebookLM: like you're really interested in what's new with AI.[00:01:36] NotebookLM 2: Definitely. And it seems like OpenAI had a lot to announce. New tools, changes to the company. It's a lot.[00:01:43] NotebookLM: It is. And especially since you're interested in how AI can be used in the real world, you know, practical applications, we'll focus on that.[00:01:51] NotebookLM: Perfect. Like, for example, this Real time API, they announced that, right? That seems like a big deal if we want AI to sound, well, less like a robot.[00:01:59] NotebookLM 2: It could be huge. The real time API could completely change how we, like, interact with AI. Like, imagine if your voice assistant could actually handle it if you interrupted it.[00:02:08] NotebookLM: Or, like, have an actual conversation.[00:02:10] NotebookLM 2: Right, not just these clunky back and forth things we're used to.[00:02:14] NotebookLM: And they actually showed it off, didn't they? I read something about a travel app, one for languages. Even one where the AI ordered takeout.[00:02:21] NotebookLM 2: Those demos were really interesting, and I think they show how this real time API can be used in so many ways.[00:02:28] NotebookLM 2: And the tech behind it is fascinating, by the way. It uses persistent WebSocket connections and this thing called function calling, so it can respond in real time.[00:02:38] NotebookLM: So the function calling thing, that sounds kind of complicated. Can you, like, explain how that works?[00:02:42] NotebookLM 2: So imagine giving the AI Access to this whole toolbox, right?[00:02:46] NotebookLM 2: Information, capabilities, all sorts of things. Okay. So take the travel agent demo, for example. With function calling, the AI can pull up details, let's say about Fort Mason, right, from some database. Like nearby restaurants, stuff like that.[00:02:59] NotebookLM: Ah, I get it. So instead of being limited to what it already knows, It can go and find the information it needs, like a human travel agent would.[00:03:07] NotebookLM 2: Precisely. And someone on Hacker News pointed out a cool detail. The API actually gives you a text version of what's being said. So you can store that, analyze it.[00:03:17] NotebookLM: That's smart. It seems like OpenAI put a lot of thought into making this API easy for developers to use. But, while we're on OpenAI, you know, Besides their tech, there's been some news about, like, internal changes, too.[00:03:30] NotebookLM: Didn't they say they're moving away from being a non profit?[00:03:32] NotebookLM 2: They did. And it's got everyone talking. It's a major shift. And it's only natural for people to wonder how that'll change things for OpenAI in the future. I mean, there are definitely some valid questions about this move to for profit. Like, will they have more money for research now?[00:03:46] NotebookLM 2: Probably. But will they, you know, care as much about making sure AI benefits everyone?[00:03:51] NotebookLM: Yeah, that's the big question, especially with all the, like, the leadership changes happening at OpenAI too, right? I read that their Chief Research Officer left, and their VP of Research, and even their CTO.[00:04:03] NotebookLM 2: It's true. A lot of people are connecting those departures with the changes in OpenAI's structure.[00:04:08] NotebookLM: And I guess it makes you wonder what's going on behind the scenes. But they are still putting out new stuff. Like this whole fine tuning thing really caught my eye.[00:04:17] NotebookLM 2: Right, fine tuning. It's essentially taking a pre trained AI model. And, like, customizing it.[00:04:23] NotebookLM: So instead of a general AI, you get one that's tailored for a specific job.[00:04:27] NotebookLM 2: Exactly. And that opens up so many possibilities, especially for businesses. Imagine you could train an AI on your company's data, you know, like how you communicate your brand guidelines.[00:04:37] NotebookLM: So it's like having an AI that's specifically trained for your company?[00:04:41] NotebookLM 2: That's the idea.[00:04:41] NotebookLM: And they're doing it with images now, too, right?[00:04:44] NotebookLM: Fine tuning with vision is what they called it.[00:04:46] NotebookLM 2: It's pretty incredible what they're doing with that, especially in fields like medicine.[00:04:50] NotebookLM: Like using AI to help doctors make diagnoses.[00:04:52] NotebookLM 2: Exactly. And AI could be trained on thousands of medical images, right? And then it could potentially spot things that even a trained doctor might miss.[00:05:03] NotebookLM: That's kind of scary, to be honest. What if it gets it wrong?[00:05:06] NotebookLM 2: Well, the idea isn't to replace doctors, but to give them another tool, you know, help them make better decisions.[00:05:12] NotebookLM: Okay, that makes sense. But training these AI models must be really expensive.[00:05:17] NotebookLM 2: It can be. All those tokens add up. But OpenAI announced something called automatic prompt caching.[00:05:23] Alex Volkov: Automatic what now? I don't think I came across that.[00:05:26] NotebookLM 2: So basically, if your AI sees a prompt that it's already seen before, OpenAI will give you a discount.[00:05:31] NotebookLM: Huh. Like a frequent buyer program for AI.[00:05:35] NotebookLM 2: Kind of, yeah. It's good that they're trying to make it more affordable. And they're also doing something called model distillation.[00:05:41] NotebookLM: Okay, now you're just using big words to sound smart. What's that?[00:05:45] NotebookLM 2: Think of it like like a recipe, right? You can take a really complex recipe and break it down to the essential parts.[00:05:50] NotebookLM: Make it simpler, but it still tastes the same.[00:05:53] NotebookLM 2: Yeah. And that's what model distillation is. You take a big, powerful AI model and create a smaller, more efficient version.[00:06:00] NotebookLM: So it's like lighter weight, but still just as capable.[00:06:03] NotebookLM 2: Exactly. And that means more people can actually use these powerful tools. They don't need, like, a supercomputer to run them.[00:06:10] NotebookLM: So they're making AI more accessible. That's great.[00:06:13] NotebookLM 2: It is. And speaking of powerful tools, they also talked about their new O1 model.[00:06:18] NotebookLM 2: That's the one they've been hyping up. The one that's supposed to be this big leap forward.[00:06:22] NotebookLM: Yeah, O1. It sounds pretty futuristic. Like, from what I read, it's not just a bigger, better language model.[00:06:28] NotebookLM 2: Right. It's a different porch.[00:06:29] NotebookLM: They're saying it can, like, actually reason, right? Think.[00:06:33] NotebookLM 2: It's trained differently.[00:06:34] NotebookLM 2: They used reinforcement learning with O1.[00:06:36] NotebookLM: So it's not just finding patterns in the data it's seen before.[00:06:40] NotebookLM 2: Not just that. It can actually learn from its mistakes. Get better at solving problems.[00:06:46] NotebookLM: So give me an example. What can O1 do that, say, GPT 4 can't?[00:06:51] NotebookLM 2: Well, OpenAI showed it doing some pretty impressive stuff with math, like advanced math.[00:06:56] NotebookLM 2: And coding, too. Complex coding. Things that even GPT 4 struggled with.[00:07:00] NotebookLM: So you're saying if I needed to, like, write a screenplay, I'd stick with GPT 4? But if I wanted to solve some crazy physics problem, O1 is what I'd use.[00:07:08] NotebookLM 2: Something like that, yeah. Although there is a trade off. O1 takes a lot more power to run, and it takes longer to get those impressive results.[00:07:17] NotebookLM: Hmm, makes sense. More power, more time, higher quality.[00:07:21] NotebookLM 2: Exactly.[00:07:22] NotebookLM: It sounds like it's still in development, though, right? Is there anything else they're planning to add to it?[00:07:26] NotebookLM 2: Oh, yeah. They mentioned system prompts, which will let developers, like, set some ground rules for how it behaves. And they're working on adding structured outputs and function calling.[00:07:38] Alex Volkov: Wait, structured outputs? Didn't we just talk about that? We[00:07:41] NotebookLM 2: did. That's the thing where the AI's output is formatted in a way that's easy to use.[00:07:47] NotebookLM: Right, right. So you don't have to spend all day trying to make sense of what it gives you. It's good that they're thinking about that stuff.[00:07:53] NotebookLM 2: It's about making these tools usable.[00:07:56] NotebookLM 2: And speaking of that, Dev Day finished up with this really interesting talk. Sam Altman, the CEO of OpenAI, And Kevin Weil, their new chief product officer. They talked about, like, the big picture for AI.[00:08:09] NotebookLM: Yeah, they did, didn't they? Anything interesting come up?[00:08:12] NotebookLM 2: Well, Altman talked about moving past this whole AGI term, Artificial General Intelligence.[00:08:18] NotebookLM: I can see why. It's kind of a loaded term, isn't it?[00:08:20] NotebookLM 2: He thinks it's become a bit of a buzzword, and people don't really understand what it means.[00:08:24] NotebookLM: So are they saying they're not trying to build AGI anymore?[00:08:28] NotebookLM 2: It's more like they're saying they're focused on just Making AI better, constantly improving it, not worrying about putting it in a box.[00:08:36] NotebookLM: That makes sense. Keep pushing the limits.[00:08:38] NotebookLM 2: Exactly. But they were also very clear about doing it responsibly. They talked a lot about safety and ethics.[00:08:43] NotebookLM: Yeah, that's important.[00:08:44] NotebookLM 2: They said they were going to be very careful. About how they release new features.[00:08:48] NotebookLM: Good! Because this stuff is powerful.[00:08:51] NotebookLM 2: It is. It was a lot to take in, this whole Dev Day event.[00:08:54] NotebookLM 2: New tools, big changes at OpenAI, and these big questions about the future of AI.[00:08:59] NotebookLM: It was. But hopefully this deep dive helped make sense of some of it. At least, that's what we try to do here.[00:09:05] AI Charlie: Absolutely.[00:09:06] NotebookLM: Thanks for taking the deep dive with us.[00:09:08] AI Charlie: The biggest demo of the new Realtime API involved function calling with voice mode and buying chocolate covered strawberries from our friendly local OpenAI developer experience engineer and strawberry shop owner, Ilan Biggio.[00:09:21] AI Charlie: We'll first play you the audio of his demo and then go into a little interview with him.[00:09:25] Ilan's Strawberry Demo with Realtime Voice Function Calling[00:09:25] Romain Huet: Could you place a call and see if you could get us 400 strawberries delivered to the venue? But please keep that under 1500. I'm on it. We'll get those strawberries delivered for you.[00:09:47] Ilan: Hello? Hi there. Is this Ilan? I'm Romain's AI assistant. How is it going? Fantastic. Can you tell me what flavors of strawberry dips you have for me? Yeah, we have chocolate, vanilla, and we have peanut butter. Wait, how much would 400 chocolate covered strawberries cost? 400? Are you sure you want 400? Yes, 400 chocolate covered[00:10:14] swyx: strawberries.[00:10:15] Ilan: Wait,[00:10:16] swyx: how much[00:10:16] Ilan: would that be? I think that'll be around, like, 1, 415. 92.[00:10:25] Alex Volkov: Awesome. Let's go ahead and place the order for four chocolate covered strawberries.[00:10:31] Ilan: Great, where would you like that delivered? Please deliver them to the Gateway Pavilion at Fort Mason. And I'll be paying in cash.[00:10:42] Alex Volkov: Okay,[00:10:43] Ilan: sweet. So just to confirm, you want four strawberries?[00:10:45] Ilan: 400 chocolate covered strawberries to the Gateway Pavilion. Yes, that's perfect. And when can we expect delivery? Well, you guys are right nearby, so it'll be like, I don't know, 37 seconds? That's incredibly fast. Cool, you too.[00:11:09] swyx: Hi, Ilan, welcome to Lanespace. Oh, thank you. I just saw your amazing demos, had your amazing strawberries. You are dressed up, like, exactly like a strawberry salesman. Gotta have it all. What was the building on demo like? What was the story behind the demo?[00:11:22] swyx: It was really interesting. This is actually something I had been thinking about for months before the launch.[00:11:27] swyx: Like, having a, like, AI that can make phone calls is something like I've personally wanted for a long time. And so as soon as we launched internally, like, I started hacking on it. And then that sort of just started. We made it into like an internal demo, and then people found it really interesting, and then we thought how cool would it be to have this like on stage as, as one of the demos.[00:11:47] swyx: Yeah, would would you call out any technical issues building, like you were basically one of the first people ever to build with a voice mode API. Would you call out any issues like integrating it with Twilio like that, like you did with function calling, with like a form filling elements. I noticed that you had like intents of things to fulfill, and then.[00:12:07] swyx: When there's still missing info, the voice would prompt you, roleplaying the store guy.[00:12:13] swyx: Yeah, yeah, so, I think technically, there's like the whole, just working with audio and streams is a whole different beast. Like, even separate from like AI and this, this like, new capabilities, it's just, it's just tough.[00:12:26] swyx: Yeah, when you have a prompt, conversationally it'll just follow, like the, it was, Instead of like, kind of step by step to like ask the right questions based on like the like what the request was, right? The function calling itself is sort of tangential to that. Like, you have to prompt it to call the functions, but then handling it isn't too much different from, like, what you would do with assistant streaming or, like, chat completion streaming.[00:12:47] swyx: I think, like, the API feels very similar just to, like, if everything in the API was streaming, it actually feels quite familiar to that.[00:12:53] swyx: And then, function calling wise, I mean, does it work the same? I don't know. Like, I saw a lot of logs. You guys showed, like, in the playground, a lot of logs. What is in there?[00:13:03] swyx: What should people know?[00:13:04] swyx: Yeah, I mean, it is, like, the events may have different names than the streaming events that we have in chat completions, but they represent very similar things. It's things like, you know, function call started, argument started, it's like, here's like argument deltas, and then like function call done.[00:13:20] swyx: Conveniently we send one that has the full function, and then I just use that. Nice.[00:13:25] swyx: Yeah and then, like, what restrictions do, should people be aware of? Like, you know, I think, I think, before we recorded, we discussed a little bit about the sensitivities around basically calling random store owners and putting, putting like an AI on them.[00:13:40] swyx: Yeah, so there's, I think there's recent regulation on that, which is why we want to be like very, I guess, aware of, of You know, you can't just call anybody with AI, right? That's like just robocalling. You wouldn't want someone just calling you with AI.[00:13:54] swyx: I'm a developer, I'm about to do this on random people.[00:13:57] swyx: What laws am I about to break?[00:14:00] swyx: I forget what the governing body is, but you should, I think, Having consent of the person you're about to call, it always works. I, as the strawberry owner, have consented to like getting called with AI. I think past that you, you want to be careful. Definitely individuals are more sensitive than businesses.[00:14:19] swyx: I think businesses you have a little bit more leeway. Also, they're like, businesses I think have an incentive to want to receive AI phone calls. Especially if like, they're dealing with it. It's doing business. Right, like, it's more business. It's kind of like getting on a booking platform, right, you're exposed to more.[00:14:33] swyx: But, I think it's still very much like a gray area. Again, so. I think everybody should, you know, tread carefully, like, figure out what it is. I, I, I, the law is so recent, I didn't have enough time to, like, I'm also not a lawyer. Yeah, yeah, yeah, of course. Yeah.[00:14:49] swyx: Okay, cool fair enough. One other thing, this is kind of agentic.[00:14:52] swyx: Did you use a state machine at all? Did you use any framework? No. You just stick it in context and then just run it in a loop until it ends call?[00:15:01] swyx: Yeah, there isn't even a loop, like Okay. Because the API is just based on sessions. It's always just going to keep going. Every time you speak, it'll trigger a call.[00:15:11] swyx: And then after every function call was also invoked invoking like a generation. And so that is another difference here. It's like it's inherently almost like in a loop, be just by being in a session, right? No state machines needed. I'd say this is very similar to like, the notion of routines, where it's just like a list of steps.[00:15:29] swyx: And it, like, sticks to them softly, but usually pretty well. And the steps is the prompts? The steps, it's like the prompt, like the steps are in the prompt. Yeah, yeah, yeah. Right, it's like step one, do this, step one, step two, do that. What if I want to change the system prompt halfway through the conversation?[00:15:44] swyx: You can. Okay. You can. To be honest, I have not played without two too much. Yeah,[00:15:47] swyx: yeah.[00:15:48] swyx: But, I know you can.[00:15:49] swyx: Yeah, yeah. Yeah. Awesome. I noticed that you called it real time API, but not voice API. Mm hmm. So I assume that it's like real time API starting with voice. Right, I think that's what he said on the thing.[00:16:00] swyx: I can't imagine, like, what else is real[00:16:02] swyx: time? Well, I guess, to use ChatGPT's voice mode as an example, Like, we've demoed the video, right? Like, real time image, right? So, I'm not actually sure what timelines are, But I would expect, if I had to guess, That, like, that is probably the next thing that we're gonna be making.[00:16:17] swyx: You'd probably have to talk directly with the team building this. Sure. But, You can't promise their timelines. Yeah, yeah, yeah, right, exactly. But, like, given that this is the features that currently, Or that exists that we've demoed on Chachapiti. Yeah. There[00:16:29] swyx: will never be a[00:16:29] swyx: case where there's like a real time text API, right?[00:16:31] swyx: I don't Well, this is a real time text API. You can do text only on this. Oh. Yeah. I don't know why you would. But it's actually So text to text here doesn't quite make a lot of sense. I don't think you'll get a lot of latency gain. But, like, speech to text is really interesting. Because you can prevent You can prevent responses, like audio responses.[00:16:54] swyx: And force function calls. And so you can do stuff like UI control. That is like super super reliable. We had a lot of like, you know, un, like, we weren't sure how well this was gonna work because it's like, you have a voice answering. It's like a whole persona, right? Like, that's a little bit more, you know, risky.[00:17:10] swyx: But if you, like, cut out the audio outputs and make it so it always has to output a function, like you can end up with pretty pretty good, like, Pretty reliable, like, command like a command architecture. Yeah,[00:17:21] swyx: actually, that's the way I want to interact with a lot of these things as well. Like, one sided voice.[00:17:26] swyx: Yeah, you don't necessarily want to hear the[00:17:27] swyx: voice back. And like, sometimes it's like, yeah, I think having an output voice is great. But I feel like I don't always want to hear an output voice. I'd say usually I don't. But yeah, exactly, being able to speak to it is super sweet.[00:17:39] swyx: Cool. Do you want to comment on any of the other stuff that you announced?[00:17:41] swyx: From caching I noticed was like, I like the no code change part. I'm looking forward to the docs because I'm sure there's a lot of details on like, what you cache, how long you cache. Cause like, enthalpy caches were like 5 minutes. I was like, okay, but what if I don't make a call every 5 minutes?[00:17:56] swyx: Yeah,[00:17:56] swyx: to be super honest with you, I've been so caught up with the real time API and making the demo that I haven't read up on the other stuff. Launches too much. I mean, I'm aware of them, but I think I'm excited to see how all distillation works. That's something that we've been doing like, I don't know, I've been like doing it between our models for a while And I've seen really good results like I've done back in a day like from GPT 4 to GPT 3.[00:18:19] swyx: 5 And got like, like pretty much the same level of like function calling with like hundreds of functions So that was super super compelling So, I feel like easier distillation, I'm really excited for. I see. Is it a tool?[00:18:31] swyx: So, I saw evals. Yeah. Like, what is the distillation product? It wasn't super clear, to be honest.[00:18:36] swyx: I, I think I want to, I want to let that team, I want to let that team talk about it. Okay,[00:18:40] swyx: alright. Well, I appreciate you jumping on. Yeah, of course. Amazing demo. It was beautifully designed. I'm sure that was part of you and Roman, and[00:18:47] swyx: Yeah, I guess, shout out to like, the first people to like, creators of Wanderlust, originally, were like, Simon and Carolis, and then like, I took it and built the voice component and the voice calling components.[00:18:59] swyx: Yeah, so it's been a big team effort. And like the entire PI team for like Debugging everything as it's been going on. It's been, it's been so good working with them. Yeah, you're the first consumers on the DX[00:19:07] swyx: team. Yeah. Yeah, I mean, the classic role of what we do there. Yeah. Okay, yeah, anything else? Any other call to action?[00:19:13] swyx: No, enjoy Dev Day. Thank you. Yeah. That's it.[00:19:16] Olivier Godement, Head of Product, OpenAI[00:19:16] AI Charlie: The latent space crew then talked to Olivier Godmont, head of product for the OpenAI platform, who led the entire Dev Day keynote and introduced all the major new features and updates that we talked about today.[00:19:28] swyx: Okay, so we are here with Olivier Godmont. That's right.[00:19:32] swyx: I don't pronounce French. That's fine. It was perfect. And it was amazing to see your keynote today. What was the back story of, of preparing something like this? Preparing, like, Dev Day? It[00:19:43] Olivier Godement: essentially came from a couple of places. Number one, excellent reception from last year's Dev Day.[00:19:48] Olivier Godement: Developers, startup founders, researchers want to spend more time with OpenAI, and we want to spend more time with them as well. And so for us, like, it was a no brainer, frankly, to do it again, like, you know, like a nice conference. The second thing is going global. We've done a few events like in Paris and like a few other like, you know, non European, non American countries.[00:20:05] Olivier Godement: And so this year we're doing SF, Singapore, and London. To frankly just meet more developers.[00:20:10] swyx: Yeah, I'm very excited for the Singapore one.[00:20:12] Olivier Godement: Ah,[00:20:12] swyx: yeah. Will you be[00:20:13] Olivier Godement: there?[00:20:14] swyx: I don't know. I don't know if I got an invite. No. I can't just talk to you. Yeah, like, and then there was some speculation around October 1st.[00:20:22] Olivier Godement: Yeah. Is it because[00:20:23] swyx: 01, October 1st? It[00:20:25] Olivier Godement: has nothing to do. I discovered the tweet yesterday where like, people are so creative. No one, there was no connection to October 1st. But in hindsight, that would have been a pretty good meme by Tiana. Okay.[00:20:37] swyx: Yeah, and you know, I think like, OpenAI's outreach to developers is something that I felt the whole in 2022, when like, you know, like, people were trying to build a chat GPT, and like, there was no function calling, all that stuff that you talked about in the past.[00:20:51] swyx: And that's why I started my own conference as like like, here's our little developer conference thing. And, but to see this OpenAI Dev Day now, and like to see so many developer oriented products coming to OpenAI, I think it's really encouraging.[00:21:02] Olivier Godement: Yeah, totally. It's that's what I said, essentially, like, developers are basically the people who make the best connection between the technology and, you know, the future, essentially.[00:21:14] Olivier Godement: Like, you know, essentially see a capability, see a low level, like, technology, and are like, hey, I see how that application or that use case that can be enabled. And so, in the direction of enabling, like, AGI, like, all of humanity, it's a no brainer for us, like, frankly, to partner with Devs.[00:21:31] Alessio: And most importantly, you almost never had waitlists, which, compared to like other releases, people usually, usually have.[00:21:38] Alessio: What is the, you know, you had from caching, you had real time voice API, we, you know, Shawn did a long Twitter thread, so people know the releases. Yeah. What is the thing that was like sneakily the hardest to actually get ready for, for that day, or like, what was the kind of like, you know, last 24 hours, anything that you didn't know was gonna work?[00:21:56] Olivier Godement: Yeah. The old Fairly, like, I would say, involved, like, features to ship. So the team has been working for a month, all of them. The one which I would say is the newest for OpenAI is the real time API. For a couple of reasons. I mean, one, you know, it's a new modality. Second, like, it's the first time that we have an actual, like, WebSocket based API.[00:22:16] Olivier Godement: And so, I would say that's the one that required, like, the most work over the month. To get right from a developer perspective and to also make sure that our existing safety mitigation that worked well with like real time audio in and audio out.[00:22:30] swyx: Yeah, what design choices or what was like the sort of design choices that you want to highlight?[00:22:35] swyx: Like, you know, like I think for me, like, WebSockets, you just receive a bunch of events. It's two way. I obviously don't have a ton of experience. I think a lot of developers are going to have to embrace this real time programming. Like, what are you designing for, or like, what advice would you have for developers exploring this?[00:22:51] Olivier Godement: The core design hypothesis was essentially, how do we enable, like, human level latency? We did a bunch of tests, like, on average, like, human beings, like, you know, takes, like, something like 300 milliseconds to converse with each other. And so that was the design principle, essentially. Like, working backward from that, and, you know, making the technology work.[00:23:11] Olivier Godement: And so we evaluated a few options, and WebSockets was the one that we landed on. So that was, like, one design choice. A few other, like, big design choices that we had to make prompt caching. Prompt caching, the design, like, target was automated from the get go. Like, zero code change from the developer.[00:23:27] Olivier Godement: That way you don't have to learn, like, what is a prompt prefix, and, you know, how long does a cache work, like, we just do it as much as we can, essentially. So that was a big design choice as well. And then finally, on distillation, like, and evaluation. The big design choice was something I learned at Skype, like in my previous job, like a philosophy around, like, a pit of success.[00:23:47] Olivier Godement: Like, what is essentially the, the, the minimum number of steps for the majority of developers to do the right thing? Because when you do evals on fat tuning, there are many, many ways, like, to mess it up, frankly, like, you know, and have, like, a crappy model, like, evals that tell, like, a wrong story. And so our whole design was, okay, we actually care about, like, helping people who don't have, like, that much experience, like, evaluating a model, like, get, like, in a few minutes, like, to a good spot.[00:24:11] Olivier Godement: And so how do we essentially enable that bit of success, like, in the product flow?[00:24:15] swyx: Yeah, yeah, I'm a little bit scared to fine tune especially for vision, because I don't know what I don't know for stuff like vision, right? Like, for text, I can evaluate pretty easily. For vision let's say I'm like trying to, one of your examples was grab.[00:24:33] swyx: Which, very close to home, I'm from Singapore. I think your example was like, they identified stop signs better. Why is that hard? Why do I have to fine tune that? If I fine tune that, do I lose other things? You know, like, there's a lot of unknowns with Vision that I think developers have to figure out.[00:24:50] swyx: For[00:24:50] Olivier Godement: sure. Vision is going to open up, like, a new, I would say, evaluation space. Because you're right, like, it's harder, like, you know, to tell correct from incorrect, essentially, with images. What I can say is we've been alpha testing, like, the Vision fine tuning, like, for several weeks at that point. We are seeing, like, even higher performance uplift compared to text fine tuning.[00:25:10] Olivier Godement: So that's, there is something here, like, we've been pretty impressed, like, in a good way, frankly. But, you know, how well it works. But for sure, like, you know, I expect the developers who are moving from one modality to, like, text and images will have, like, more, you know Testing, evaluation, like, you know, to set in place, like, to make sure it works well.[00:25:25] Alessio: The model distillation and evals is definitely, like, the most interesting. Moving away from just being a model provider to being a platform provider. How should people think about being the source of truth? Like, do you want OpenAI to be, like, the system of record of all the prompting? Because people sometimes store it in, like, different data sources.[00:25:41] Alessio: And then, is that going to be the same as the models evolve? So you don't have to worry about, you know, refactoring the data, like, things like that, or like future model structures.[00:25:51] Olivier Godement: The vision is if you want to be a source of truth, you have to earn it, right? Like, we're not going to force people, like, to pass us data.[00:25:57] Olivier Godement: There is no value prop, like, you know, for us to store the data. The vision here is at the moment, like, most developers, like, use like a one size fits all model, like be off the shelf, like GP40 essentially. The vision we have is fast forward a couple of years. I think, like, most developers will essentially, like, have a.[00:26:15] Olivier Godement: An automated, continuous, fine tuned model. The more, like, you use the model, the more data you pass to the model provider, like, the model is automatically, like, fine tuned, evaluated against some eval sets, and essentially, like, you don't have to every month, when there is a new snapshot, like, you know, to go online and, you know, try a few new things.[00:26:34] Olivier Godement: That's a direction. We are pretty far away from it. But I think, like, that evaluation and decision product are essentially a first good step in that direction. It's like, hey, it's you. I set it by that direction, and you give us the evaluation data. We can actually log your completion data and start to do some automation on your behalf.[00:26:52] Alessio: And then you can do evals for free if you share data with OpenAI. How should people think about when it's worth it, when it's not? Sometimes people get overly protective of their data when it's actually not that useful. But how should developers think about when it's right to do it, when not, or[00:27:07] Olivier Godement: if you have any thoughts on it?[00:27:08] Olivier Godement: The default policy is still the same, like, you know, we don't train on, like, any API data unless you opt in. What we've seen from feedback is evaluation can be expensive. Like, if you run, like, O1 evals on, like, thousands of samples Like, your build will get increased, like, you know, pretty pretty significantly.[00:27:22] Olivier Godement: That's problem statement number one. Problem statement number two is, essentially, I want to get to a world where whenever OpenAI ships a new model snapshot, we have full confidence that there is no regression for the task that developers care about. And for that to be the case, essentially, we need to get evals.[00:27:39] Olivier Godement: And so that, essentially, is a sort of a two bugs one stone. It's like, we subsidize, basically, the evals. And we also use the evals when we ship new models to make sure that we keep going in the right direction. So, in my sense, it's a win win, but again, completely opt in. I expect that many developers will not want to share their data, and that's perfectly fine to me.[00:27:56] swyx: Yeah, I think free evals though, very, very good incentive. I mean, it's a fair trade. You get data, we get free evals. Exactly,[00:28:04] Olivier Godement: and we sanitize PII, everything. We have no interest in the actual sensitive data. We just want to have good evaluation on the real use cases.[00:28:13] swyx: Like, I always want to eval the eval. I don't know if that ever came up.[00:28:17] swyx: Like, sometimes the evals themselves are wrong, and there's no way for me to tell you.[00:28:22] Olivier Godement: Everyone who is starting with LLM, teaching with LLM, is like, Yeah, evaluation, easy, you know, I've done testing, like, all my life. And then you start to actually be able to eval, understand, like, all the corner cases, And you realize, wow, there's like a whole field in itself.[00:28:35] Olivier Godement: So, yeah, good evaluation is hard and so, yeah. Yeah, yeah.[00:28:38] swyx: But I think there's a, you know, I just talked to Brain Trust which I think is one of your partners. Mm-Hmm. . They also emphasize code based evals versus your sort of low code. What I see is like, I don't know, maybe there's some more that you didn't demo.[00:28:53] swyx: YC is kind of like a low code experience, right, for evals. Would you ever support like a more code based, like, would I run code on OpenAI's eval platform?[00:29:02] Olivier Godement: For sure. I mean, we meet developers where they are, you know. At the moment, the demand was more for like, you know, easy to get started, like eval. But, you know, if we need to expose like an evaluation API, for instance, for people like, you know, to pass, like, you know, their existing test data we'll do it.[00:29:15] Olivier Godement: So yeah, there is no, you know, philosophical, I would say, like, you know, misalignment on that. Yeah,[00:29:19] swyx: yeah, yeah. What I think this is becoming, by the way, and I don't, like it's basically, like, you're becoming AWS. Like, the AI cloud. And I don't know if, like, that's a conscious strategy, or it's, like, It doesn't even have to be a conscious strategy.[00:29:33] swyx: Like, you're going to offer storage. You're going to offer compute. You're going to offer networking. I don't know what networking looks like. Networking is maybe, like, Caching or like it's a CDN. It's a prompt CDN.[00:29:45] Alex Volkov: Yeah,[00:29:45] swyx: but it's the AI versions of everything, right? Do you like do you see the analogies or?[00:29:52] Olivier Godement: Whatever Whatever I took to developers. I feel like Good models are just half of the story to build a good app There's a third model you need to do Evaluation is the perfect example. Like, you know, you can have the best model in the world If you're in the dark, like, you know, it's really hard to gain the confidence and so Our philosophy is[00:30:11] Olivier Godement: The whole like software development stack is being basically reinvented, you know, with LLMs. There is no freaking way that open AI can build everything. Like there is just too much to build, frankly. And so my philosophy is, essentially, we'll focus on like the tools which are like the closest to the model itself.[00:30:28] Olivier Godement: So that's why you see us like, you know, investing quite a bit in like fine tuning, distillation, our evaluation, because we think that it actually makes sense to have like in one spot, Like, you know, all of that. Like, there is some sort of virtual circle, essentially, that you can set in place. But stuff like, you know, LLMOps, like tools which are, like, further away from the model, I don't know if you want to do, like, you know, super elaborate, like, prompt management, or, you know, like, tooling, like, I'm not sure, like, you know, OpenAI has, like, such a big edge, frankly, like, you know, to build this sort of tools.[00:30:56] Olivier Godement: So that's how we view it at the moment. But again, frankly, the philosophy is super simple. The strategy is super simple. It's meeting developers where they want us to be. And so, you know that's frankly, like, you know, day in, day out, like, you know, what I try to do.[00:31:08] Alessio: Cool. Thank you so much for the time.[00:31:10] Alessio: I'm sure you,[00:31:10] swyx: Yeah, I have more questions on, a couple questions on voice, and then also, like, your call to action, like, what you want feedback on, right? So, I think we should spend a bit more time on voice, because I feel like that's, like, the big splash thing. I talked well Well, I mean, I mean, just what is the future of real time for OpenAI?[00:31:28] swyx: Yeah. Because I think obviously video is next. You already have it in the, the ChatGPT desktop app. Do we just have a permanent, like, you know, like, are developers just going to be, like, sending sockets back and forth with OpenAI? Like how do we program for that? Like, what what is the future?[00:31:44] Olivier Godement: Yeah, that makes sense. I think with multimodality, like, real time is quickly becoming, like, you know, essentially the right experience, like, to build an application. Yeah. So my expectation is that we'll see like a non trivial, like a volume of applications like moving to a real time API. Like if you zoom out, like, audio is really simple, like, audio until basically now.[00:32:05] Olivier Godement: Audio on the web, in apps, was basically very much like a second class citizen. Like, you basically did like an audio chatbot for users who did not have a choice. You know, they were like struggling to read, or I don't know, they were like not super educated with technology. And so, frankly, it was like the crappy option, you know, compared to text.[00:32:25] Olivier Godement: But when you talk to people in the real world, the vast majority of people, like, prefer to talk and listen instead of typing and writing.[00:32:34] swyx: We speak before we write.[00:32:35] Olivier Godement: Exactly. I don't know. I mean, I'm sure it's the case for you in Singapore. For me, my friends in Europe, the number of, like, WhatsApp, like, voice notes they receive every day, I mean, just people, it makes sense, frankly, like, you know.[00:32:45] Olivier Godement: Chinese. Chinese, yeah.[00:32:46] swyx: Yeah,[00:32:47] Olivier Godement: all voice. You know, it's easier. There is more emotions. I mean, you know, you get the point across, like, pretty well. And so my personal ambition for, like, the real time API and, like, audio in general is to make, like, audio and, like, multimodality, like, truly a first class experience.[00:33:01] Olivier Godement: Like, you know, if you're, like, you know, the amazing, like, super bold, like, start up out of YC, you want to build, like, the next, like, billion, like, you know, user application to make it, like, truly your first and make it feel, like, you know, an actual good, like, you know, product experience. So that's essentially the ambition, and I think, like, yeah, it could be pretty big.[00:33:17] swyx: Yeah. I think one, one people, one issue that people have with the voice so far as, as released in advanced voice mode is the refusals.[00:33:24] Alex Volkov: Yeah.[00:33:24] swyx: You guys had a very inspiring model spec. I think Joanne worked on that. Where you said, like, yeah, we don't want to overly refuse all the time. In fact, like, even if, like, not safe for work, like, in some occasions, it's okay.[00:33:38] swyx: How, is there an API that we can say, not safe for work, okay?[00:33:41] Olivier Godement: I think we'll get there. I think we'll get there. The mobile spec, like, nailed it, like, you know. It nailed it! It's so good! Yeah, we are not in the business of, like, policing, you know, if you can say, like, vulgar words or whatever. You know, there are some use cases, like, you know, I'm writing, like, a Hollywood, like, script I want to say, like, will go on, and it's perfectly fine, you know?[00:33:59] Olivier Godement: And so I think the direction where we'll go here is that basically There will always be like, you know, a set of behavior that we will, you know, just like forbid, frankly, because they're illegal against our terms of services. But then there will be like, you know, some more like risky, like themes, which are completely legal, like, you know, vulgar words or, you know, not safe for work stuff.[00:34:17] Olivier Godement: Where basically we'll expose like a controllable, like safety, like knobs in the API to basically allow you to say, hey, that theme okay, that theme not okay. How sensitive do you want the threshold to be on safety refusals? I think that's the Dijkstra. So a[00:34:31] swyx: safety API.[00:34:32] Olivier Godement: Yeah, in a way, yeah.[00:34:33] swyx: Yeah, we've never had that.[00:34:34] Olivier Godement: Yeah. '[00:34:35] swyx: cause right now is you, it is whatever you decide. And then it's, that's it. That, that, that would be the main reason I don't use opening a voice is because of[00:34:42] Olivier Godement: it's over police. Over refuse over refusals. Yeah. Yeah, yeah. No, we gotta fix that. Yeah. Like singing,[00:34:47] Alessio: we're trying to do voice. I'm a singer.[00:34:49] swyx: And you, you locked off singing.[00:34:51] swyx: Yeah,[00:34:51] Alessio: yeah, yeah.[00:34:52] swyx: But I, I understand music gets you in trouble. Okay. Yeah. So then, and then just generally, like, what do you want to hear from developers? Right? We have, we have all developers watching you know, what feedback do you want? Any, anything specific as well, like from, especially from today anything that you are unsure about, that you are like, Our feedback could really help you decide.[00:35:09] swyx: For sure.[00:35:10] Olivier Godement: I think, essentially, it's becoming pretty clear after today that, you know, I would say the open end direction has become pretty clear, like, you know, after today. Investment in reasoning, investment in multimodality, Investment as well, like in, I would say, tool use, like function calling. To me, the biggest question I have is, you know, Where should we put the cursor next?[00:35:30] Olivier Godement: I think we need all three of them, frankly, like, you know, so we'll keep pushing.[00:35:33] swyx: Hire 10, 000 people, or actually, no need, build a bunch of bots.[00:35:37] Olivier Godement: Exactly, and so let's take O1 smart enough, like, for your problems? Like, you know, let's set aside for a second the existing models, like, for the apps that you would love to build, is O1 basically it in reasoning, or do we still have, like, you know, a step to do?[00:35:50] Olivier Godement: Preview is not enough, I[00:35:52] swyx: need the full one.[00:35:53] Olivier Godement: Yeah, so that's exactly that sort of feedback. Essentially what they would love to do is for developers I mean, there's a thing that Sam has been saying like over and over again, like, you know, it's easier said than done, but I think it's directionally correct. As a developer, as a founder, you basically want to build an app which is a bit too difficult for the model today, right?[00:36:12] Olivier Godement: Like, what you think is right, it's like, sort of working, sometimes not working. And that way, you know, that basically gives us like a goalpost, and be like, okay, that's what you need to enable with the next model release, like in a few months. And so I would say that Usually, like, that's the sort of feedback which is like the most useful that I can, like, directly, like, you know, incorporate.[00:36:33] swyx: Awesome. I think that's our time. Thank you so much, guys. Yeah, thank you so much.[00:36:38] AI Charlie: Thank you. We were particularly impressed that Olivier addressed the not safe for work moderation policy question head on, as that had only previously been picked up on in Reddit forums. This is an encouraging sign that we will return to in the closing candor with Sam Altman at the end of this episode.[00:36:57] Romain Huet, Head of DX, OpenAI[00:36:57] AI Charlie: Next, a chat with Roman Hewitt, friend of the pod, AI Engineer World's fair closing keynote speaker, and head of developer experience at OpenAI on his incredible live demos And advice to AI engineers on all the new modalities.[00:37:12] Alessio: Alright, we're live from OpenAI Dev Day. We're with Juan, who just did two great demos on, on stage.[00:37:17] Alessio: And he's been a friend of Latentspace, so thanks for taking some of the time.[00:37:20] Romain Huet: Of course, yeah, thank you for being here and spending the time with us today.[00:37:23] swyx: Yeah, I appreciate appreciate you guys putting this on. I, I know it's like extra work, but it really shows the developers that you're, Care and about reaching out.[00:37:31] Romain Huet: Yeah, of course, I think when you go back to the OpenAI mission, I think for us it's super important that we have the developers involved in everything we do. Making sure that you know, they have all of the tools they need to build successful apps. And we really believe that the developers are always going to invent the ideas, the prototypes, the fun factors of AI that we can't build ourselves.[00:37:49] Romain Huet: So it's really cool to have everyone here.[00:37:51] swyx: We had Michelle from you guys on. Yes, great episode. She very seriously said API is the path to AGI. Correct. And people in our YouTube comments were like, API is not AGI. I'm like, no, she's very serious. API is the path to AGI. Like, you're not going to build everything like the developers are, right?[00:38:08] swyx: Of[00:38:08] Romain Huet: course, yeah, that's the whole value of having a platform and an ecosystem of amazing builders who can, like, in turn, create all of these apps. I'm sure we talked about this before, but there's now more than 3 million developers building on OpenAI, so it's pretty exciting to see all of that energy into creating new things.[00:38:26] Alessio: I was going to say, you built two apps on stage today, an international space station tracker and then a drone. The hardest thing must have been opening Xcode and setting that up. Now, like, the models are so good that they can do everything else. Yes. You had two modes of interaction. You had kind of like a GPT app to get the plan with one, and then you had a cursor to do apply some of the changes.[00:38:47] Alessio: Correct. How should people think about the best way to consume the coding models, especially both for You know, brand new projects and then existing projects that you're trying to modify.[00:38:56] Romain Huet: Yeah. I mean, one of the things that's really cool about O1 Preview and O1 Mini being available in the API is that you can use it in your favorite tools like cursor like I did, right?[00:39:06] Romain Huet: And that's also what like Devin from Cognition can use in their own software engineering agents. In the case of Xcode, like, it's not quite deeply integrated in Xcode, so that's why I had like chat GPT side by side. But it's cool, right, because I could instruct O1 Preview to be, like, my coding partner and brainstorming partner for this app, but also consolidate all of the, the files and architect the app the way I wanted.[00:39:28] Romain Huet: So, all I had to do was just, like, port the code over to Xcode and zero shot the app build. I don't think I conveyed, by the way, how big a deal that is, but, like, you can now create an iPhone app from scratch, describing a lot of intricate details that you want, and your vision comes to life in, like, a minute.[00:39:47] Romain Huet: It's pretty outstanding.[00:39:48] swyx: I have to admit, I was a bit skeptical because if I open up SQL, I don't know anything about iOS programming. You know which file to paste it in. You probably set it up a little bit. So I'm like, I have to go home and test it. And I need the ChatGPT desktop app so that it can tell me where to click.[00:40:04] Romain Huet: Yeah, I mean like, Xcode and iOS development has become easier over the years since they introduced Swift and SwiftUI. I think back in the days of Objective C, or like, you know, the storyboard, it was a bit harder to get in for someone new. But now with Swift and SwiftUI, their dev tools are really exceptional.[00:40:23] Romain Huet: But now when you combine that with O1, as your brainstorming and coding partner, it's like your architect, effectively. That's the best way, I think, to describe O1. People ask me, like, can GPT 4 do some of that? And it certainly can. But I think it will just start spitting out code, right? And I think what's great about O1, is that it can, like, make up a plan.[00:40:42] Romain Huet: In this case, for instance, the iOS app had to fetch data from an API, it had to look at the docs, it had to look at, like, how do I parse this JSON, where do I store this thing, and kind of wire things up together. So that's where it really shines. Is mini or preview the better model that people should be using?[00:40:58] Romain Huet: Like, how? I think people should try both. We're obviously very excited about the upcoming O1 that we shared the evals for. But we noticed that O1 Mini is very, very good at everything math, coding, everything STEM. If you need for your kind of brainstorming or your kind of science part, you need some broader knowledge than reaching for O1 previews better.[00:41:20] Romain Huet: But yeah, I used O1 Mini for my second demo. And it worked perfectly. All I needed was very much like something rooted in code, architecting and wiring up like a front end, a backend, some UDP packets, some web sockets, something very specific. And it did that perfectly.[00:41:35] swyx: And then maybe just talking about voice and Wanderlust, the app that keeps on giving, what's the backstory behind like preparing for all of that?[00:41:44] Romain Huet: You know, it's funny because when last year for Dev Day, we were trying to think about what could be a great demo app to show like an assistive experience. I've always thought travel is a kind of a great use case because you have, like, pictures, you have locations, you have the need for translations, potentially.[00:42:01] Romain Huet: There's like so many use cases that are bounded to travel that I thought last year, let's use a travel app. And that's how Wanderlust came to be. But of course, a year ago, all we had was a text based assistant. And now we thought, well, if there's a voice modality, what if we just bring this app back as a wink.[00:42:19] Romain Huet: And what if we were interacting better with voice? And so with this new demo, what I showed was the ability to like, So, we wanted to have a complete conversation in real time with the app, but also the thing we wanted to highlight was the ability to call tools and functions, right? So, like in this case, we placed a phone call using the Twilio API, interfacing with our AI agents, but developers are so smart that they'll come up with so many great ideas that we could not think of ourselves, right?[00:42:48] Romain Huet: But what if you could have like a, you know, a 911 dispatcher? What if you could have like a customer service? Like center, that is much smarter than what we've been used to today. There's gonna be so many use cases for real time, it's awesome.[00:43:00] swyx: Yeah, and sometimes actually you, you, like this should kill phone trees.[00:43:04] swyx: Like there should not be like dial one[00:43:07] Romain Huet: of course para[00:43:08] swyx: espanol, you know? Yeah, exactly. Or whatever. I dunno.[00:43:12] Romain Huet: I mean, even you starting speaking Spanish would just do the thing, you know you don't even have to ask. So yeah, I'm excited for this future where we don't have to interact with those legacy systems.[00:43:22] swyx: Yeah. Yeah. Is there anything, so you are doing function calling in a streaming environment. So basically it's, it's web sockets. It's UDP, I think. It's basically not guaranteed to be exactly once delivery. Like, is there any coding challenges that you encountered when building this?[00:43:39] Romain Huet: Yeah, it's a bit more delicate to get into it.[00:43:41] Romain Huet: We also think that for now, what we, what we shipped is a, is a beta of this API. I think there's much more to build onto it. It does have the function calling and the tools. But we think that for instance, if you want to have something very robust, On your client side, maybe you want to have web RTC as a client, right?[00:43:58] Romain Huet: And, and as opposed to like directly working with the sockets at scale. So that's why we have partners like Life Kit and Agora if you want to, if you want to use them. And I'm sure we'll have many mores in the, in many more in the future. But yeah, we keep on iterating on that, and I'm sure the feedback of developers in the weeks to come is going to be super critical for us to get it right.[00:44:16] swyx: Yeah, I think LiveKit has been fairly public that they are used in, in the Chachapiti app. Like, is it, it's just all open source, and we just use it directly with OpenAI, or do we use LiveKit Cloud or something?[00:44:28] Romain Huet: So right now we, we released the API, we released some sample code also, and referenced clients for people to get started with our API.[00:44:35] Romain Huet: And we also partnered with LifeKit and Agora, so they also have their own, like ways to help you get started that plugs natively with the real time API. So depending on the use case, people can, can can decide what to use. If you're working on something that's completely client or if you're working on something on the server side, for the voice interaction, you may have different needs, so we want to support all of those.[00:44:55] Alessio: I know you gotta run. Is there anything that you want the AI engineering community to give feedback on specifically, like even down to like, you know, a specific API end point or like, what, what's like the thing that you want? Yeah. I[00:45:08] Romain Huet: mean, you know, if we take a step back, I think dev Day this year is all different from last year and, and in, in a few different ways.[00:45:15] Romain Huet: But one way is that we wanted to keep it intimate, even more intimate than last year. We wanted to make sure that the community is. Thank you very much for joining us on the Spotlight. That's why we have community talks and everything. And the takeaway here is like learning from the very best developers and AI engineers.[00:45:31] Romain Huet: And so, you know we want to learn from them. Most of what we shipped this morning, including things like prompt caching the ability to generate prompts quickly in the playground, or even things like vision fine tuning. These are all things that developers have been asking of us. And so, the takeaway I would, I would leave them with is to say like, Hey, the roadmap that we're working on is heavily influenced by them and their work.[00:45:53] Romain Huet: And so we love feedback From high feature requests, as you say, down to, like, very intricate details of an API endpoint, we love feedback, so yes that's, that's how we, that's how we build this API.[00:46:05] swyx: Yeah, I think the, the model distillation thing as well, it might be, like, the, the most boring, but, like, actually used a lot.[00:46:12] Romain Huet: True, yeah. And I think maybe the most unexpected, right, because I think if I, if I read Twitter correctly the past few days, a lot of people were expecting us. To shape the real time API for speech to speech. I don't think developers were expecting us to have more tools for distillation, and we really think that's gonna be a big deal, right?[00:46:30] Romain Huet: If you're building apps that have you know, you, you want high, like like low latency, low cost, but high performance, high quality on the use case distillation is gonna be amazing.[00:46:40] swyx: Yeah. I sat in the distillation session just now and they showed how they distilled from four oh to four mini and it was like only like a 2% hit in the performance and 50 next.[00:46:49] swyx: Yeah,[00:46:50] Romain Huet: I was there as well for the superhuman kind of use case inspired for an Ebola client. Yeah, this was really good. Cool man! so much for having me. Thanks again for being here today. It's always[00:47:00] AI Charlie: great to have you. As you might have picked up at the end of that chat, there were many sessions throughout the day focused on specific new capabilities.[00:47:08] Michelle Pokrass, Head of API at OpenAI ft. Simon Willison[00:47:08] AI Charlie: Like the new model distillation features combining EVOLs and fine tuning. For our next session, we are delighted to bring back two former guests of the pod, which is something listeners have been greatly enjoying in our second year of doing the Latent Space podcast. Michelle Pokras of the API team joined us recently to talk about structured outputs, and today gave an updated long form session at Dev Day, describing the implementation details of the new structured output mode.[00:47:39] AI Charlie: We also got her updated thoughts on the VoiceMode API we discussed in her episode, now that it is finally announced. She is joined by friend of the pod and super blogger, Simon Willison, who also came back as guest co host in our Dev Day. 2023 episode.[00:47:56] Alessio: Great, we're back live at Dev Day returning guest Michelle and then returning guest co host Fork.[00:48:03] Alessio: Fork, yeah, I don't know. I've lost count. I think it's been a few. Simon Willison is back. Yeah, we just wrapped, we just wrapped everything up. Congrats on, on getting everything everything live. Simon did a great, like, blog, so if you haven't caught up, I[00:48:17] Simon Willison: wrote my, I implemented it. Now, I'm starting my live blog while waiting for the first talk to start, using like GPT 4, I wrote me the Javascript, and I got that live just in time and then, yeah, I was live blogging the whole day.[00:48:28] swyx: Are you a cursor enjoyer?[00:48:29] Simon Willison: I haven't really gotten into cursor yet to be honest. I just haven't spent enough time for it to click, I think. I'm more a copy and paste things out of Cloud and chat GPT. Yeah. It's interesting.[00:48:39] swyx: Yeah. I've converted to cursor and 01 is so easy to just toggle on and off.[00:48:45] Alessio: What's your workflow?[00:48:46] Alessio: VS[00:48:48] Michelle Pokrass: Code co pilot, so Yep, same here. Team co pilot. Co pilot is actually the reason I joined OpenAI. It was, you know, before ChatGPT, this is the thing that really got me. So I'm still into it, but I keep meaning to try out Cursor, and I think now that things have calmed down, I'm gonna give it a real go.[00:49:03] swyx: Yeah, it's a big thing to change your tool of choice.[00:49:06] swyx: Yes,[00:49:06] Michelle Pokrass: yeah, I'm pretty dialed, so.[00:49:09] swyx: I mean, you know, if you want, you can just fork VS Code and make your own. That's the thing to dumb thing, right? We joked about doing a hackathon where the only thing you do is fork VS Code and bet me the best fork win.[00:49:20] Michelle Pokrass: Nice.[00:49:22] swyx: That's actually a really good idea. Yeah, what's up?[00:49:26] swyx: I mean, congrats on launching everything today. I know, like, we touched on it a little bit, but, like, everyone was kind of guessing that Voice API was coming, and, like, we talked about it in our episode. How do you feel going into the launch? Like, any design decisions that you want to highlight?[00:49:41] Michelle Pokrass: Yeah, super jazzed about it. The team has been working on it for a while. It's, like, a very different API for us. It's the first WebSocket API, so a lot of different design decisions to be made. It's, like, what kind of events do you send? When do you send an event? What are the event names? What do you send, like, on connection versus on future messages?[00:49:57] Michelle Pokrass: So there have been a lot of interesting decisions there. The team has also hacked together really cool projects as we've been testing it. One that I really liked is we had an internal hack a thon for the API team. And some folks built like a little hack that you could use to, like VIM with voice mode, so like, control vim, and you would tell them on like, nice, write a file and it would, you know, know all the vim commands and, and pipe those in.[00:50:18] Michelle Pokrass: So yeah, a lot of cool stuff we've been hacking on and really excited to see what people build with it.[00:50:23] Simon Willison: I've gotta call out a demo from today. I think it was Katja had a 3D visualization of the solar system, like WebGL solar system, you could talk to. That is one of the coolest conference demos I've ever seen.[00:50:33] Simon Willison: That was so convincing. I really want the code. I really want the code for that to get put out there. I'll talk[00:50:39] Michelle Pokrass: to the team. I think we can[00:50:40] Simon Willison: probably

Steven Tey is Founder & CEO of Dub.co, the open source link management infrastructure platform for modern marketing teams. Their open source project, also called dub, has almost 18K stars on GitHub and is used by teams at companies like Vercel, Raycast, and Perplexity. In this episode, we dig into starting Dub.co as a side project and how it ultimately turned into a company, their initial positioning as a Bitly alternative, their focus on great design and how that shows up in all parts of their product and marketing, how analytics unlocked much higher ACVs and value for customers, how getting a top Hackernews post helped drive early momentum, their vision to become an end-to-end attribution platform & more!

Ryan Worl, Co-founder and CTO at WarpStream, joins us to talk about the world of Kafka and data streaming and how WarpStream redesigned the idea of Kafka to run in modern cloud environments directly on top of object storage. Last year they posted a blog titled, "Kafka is dead, long live Kafka" that hit the top of Hacker News to put WarpStream on the map. We get the backstory on Kafka and why it's so widely used, who created it and for what purpose, and the behind the scenes on all things WarpStream.

Jim Lee (ClimateViewer.com) is back with us tonight, but instead of getting a geo-engineering update, we are keeping things focused primary on Hacker News. A week's worth of significant hacks and data breaches last week, and we are going to string them all together and see where the conversation takes us. In the second half we have highlight clips from the NPC Convention in Chicago. Watch the video rerun here: https://share-link.pilled.net/topic-detail/979427 Proudly Sponsored By: Blue Monster Prep: An Online Superstore for Emergency Preparedness Gear (Storable Food, Water, Filters, Radios, MEDICAL SUPPLIES, and so much more). Use code 'FRANKLY' for Free Shipping on every purchase you make @ https://bluemonsterprep.com/ SUPPORT Quite Frankly: Official Merch: https://tinyurl.com/f3kbkr4s Official Coffee: https://tinyurl.com/2p9m8ndb Sponsor QF Monthly Through: QFTV: https://www.quitefrankly.tv/sponsor SubscribeStar: https://www.subscribestar.com/quitefrankly Patreon: https://www.patreon.com/QuiteFrankly One-Time Tip: http://www.paypal.me/QuiteFranklyLive Sign up for the Free Mailing List: https://bit.ly/3frUdOj Send Crypto: BTC: 1EafWUDPHY6y6HQNBjZ4kLWzQJFnE5k9PK LTC: LRs6my7scMxpTD5j7i8WkgBgxpbjXABYXX ETH: 0x80cd26f708815003F11Bd99310a47069320641fC FULL Episodes On Demand: Spotify: https://spoti.fi/301gcES iTunes: http://apple.co/2dMURMq Amazon: https://amzn.to/3afgEXZ SoundCloud: http://bit.ly/2dTMD13 Google Play: https://bit.ly/2SMi1SF BitChute: https://bit.ly/2vNSMFq Rumble: https://bit.ly/31h2HUg Streaming Live On: QuiteFrankly.tv (Powered by Foxhole) DLive: https://bit.ly/2In9ipw Rokfin: https://bit.ly/3rjrh4q Twitch: https://bit.ly/2TGAeB6 YouTube: https://bit.ly/2exPzj4 Rumble: https://bit.ly/31h2HUg How Else to Find Us: Official WebSite: http://www.QuiteFrankly.tv Official Forum: https://bit.ly/3SToJFJ Official Telegram: https://t.me/quitefranklytv GUILDED Hangout: https://bit.ly/3SmpV4G Discord Hangout: https://discord.gg/4R6bkxqb Twitter: @PoliticalOrgy Gab: @QuiteFrankly Truth Social: @QuiteFrankly GETTR: @QuiteFrankly MINDS: @QuiteFrankly