Podcasts about Zvi

Watch & Subscribe on YouTubeEver wonder how to turn your network into your net worth? I've discovered the secret to building lasting relationships that can transform your business and life.In this episode of Wealthy Wellthy, Krisstina welcomes Zvi Band, a successful entrepreneur and founder of Contactually, a CRM company that had a multi-million dollar exit. Zvi shares his journey from software engineer to relationship-building expert, offering invaluable insights on networking in the digital age.The main focus of this conversation is the power of authentic relationships in business and personal success. Zvi emphasizes the importance of moving beyond transactional interactions to build genuine, long-lasting connections that can yield dividends for decades.Additionally, Zvi discusses his experience with selling his company, the unexpected challenges of a successful exit, and his new venture, Relatable. He also touches on the mindset shifts necessary for entrepreneurs to find fulfillment beyond financial success.If you're looking to elevate your networking game and build a network that truly impacts your net worth, this episode is a must-listen. Tune in to discover how to create meaningful relationships that can transform your business and life.Key Takeaways5:44 Relationships matter more in digital age16:37 From Contactually exit to new venture27:17 Shifting from unicorn dreams to meaningful work34:33 Setting personal goalposts, not society's42:04 Building long-term relationships for successMemorable Quotes"Deep down we are still human beings and human beings rely on social connection and social trust in pretty much every aspect. And so therefore relationships, I do believe is one of the most important currencies that we have in the face of everything else.""We were just so focused on growing the business that we never really paid attention to exiting the business.""I think your next venture has the opportunity to be even closer and closer to what who you truly are."Resources MentionedThe Psychology of Money by Morgan Housel - https://www.amazon.com/Psychology-Money-Timeless-lessons-happiness/dp/0857197681Connect with KrisstinaWebsite - https://wealthywellthy.life/Instagram - https://www.instagram.com/krisstinawiseYouTube - https://www.youtube.com/@krisstinawiseKrisstina's Book, Falling For Money - https://www.amazon.com/dp/0692560904/

Study Guide Shevuot 9 Today's daf is sponsored by Leya Landau in loving memory of her mother Ita bat Zvi on her 3rd yahrzeit. "She loved learning and encouraged me to start learning the daf." Today's daf is sponsored by Naama Tal in loving memory of her grandmother Devorah Cohen, who always valued learning.  The Gemara analyzes the different opinions brought in the Mishna regarding the purposes of the goat sin offerings brought on the outer altar on Yom Kippur and on the regalim and Rosh Chodesh. What is the basis for each opinion?

Study Guide Shevuot 9 Today's daf is sponsored by Leya Landau in loving memory of her mother Ita bat Zvi on her 3rd yahrzeit. "She loved learning and encouraged me to start learning the daf." Today's daf is sponsored by Naama Tal in loving memory of her grandmother Devorah Cohen, who always valued learning.  The Gemara analyzes the different opinions brought in the Mishna regarding the purposes of the goat sin offerings brought on the outer altar on Yom Kippur and on the regalim and Rosh Chodesh. What is the basis for each opinion?

Tariffs, Trade Routes, and Tech: Freightos' View from the Cargo Frontlines 2025 is shaping up to be a wild year for global trade, and few companies have a vantage point on the impacts of every-changing tariff policy quite like digital cargo-booking platform Freightos (NASDAQ:CRGO). This week, we catch up with Freightos CEO Zvi Schreiber. He shares how the shifting trade flows are showing up in Freightos' numbers, and how companies are adjusting to meet the new challenges. Zvi also discusses why the shipping industry has long resisted digitization and how Freightos has made headway since closing its combination with Gesher I in January 2023.  

Dva lidé objevili v lese na jihozápadním úbočí Zvičiny u Dvora Králové nad Labem poklad. Jde o mince, šperky a tabatěrky, které kdosi mohl skrýt před odsunem nebo měnovou reformou. Nález váží 7 kg a už ho zkoumají odborníci z Muzea východních Čech tady v Hradci Králové, kteří zjistili, že poklad ležel v kamenném valu maximálně 100 let.Všechny díly podcastu Host ve studiu můžete pohodlně poslouchat v mobilní aplikaci mujRozhlas pro Android a iOS nebo na webu mujRozhlas.cz.

Dva lidé objevili v lese na jihozápadním úbočí Zvičiny u Dvora Králové nad Labem poklad. Jde o mince, šperky a tabatěrky, které kdosi mohl skrýt před odsunem nebo měnovou reformou. Nález váží 7 kg a už ho zkoumají odborníci z Muzea východních Čech tady v Hradci Králové, kteří zjistili, že poklad ležel v kamenném valu maximálně 100 let.

Představte si, že jdete na procházku a najdete zhruba osm milionů korun ve zlatě. Právě to se stalo dvěma turistům na začátku února pod kopcem Zvičina u Dvora Králové nad Labem. Celý nález odevzdali archeologům. Jak se vše přihodilo?

In this episode of the Cognitive Revolution podcast, the host Nathan Labenz is joined for the record 9th time by Zvi Mowshowitz to discuss the state of AI advancements, focusing on recent developments such as OpenAI's O3 model and its implications for AGI and recursive self-improvement. They delve into the capabilities and limitations of current AI models in various domains, including coding, deep research, and practical utilities. The discussion also covers the strategic and ethical considerations in AI development, touching upon the roles of major AI labs, the potential for weaponization, and the importance of balancing innovation with safety. Zvi shares insights on what it means to be a live player in the AI race, the impact of transparency and safety measures, and the challenges of governance in the context of rapidly advancing AI technologies. Nathan Labenz's slide deck documenting the ever-growing list of AI Bad Behaviors: https://docs.google.com/presentation/d/1mvkpg1mtAvGzTiiwYPc6bKOGsQXDIwMb-ytQECb3i7I/edit#slide=id.g252d9e67d86_0_16 Upcoming Major AI Events Featuring Nathan Labenz as a Keynote Speaker https://www.imagineai.live/ https://adapta.org/adapta-summit https://itrevolution.com/product/enterprise-tech-leadership-summit-las-vegas/ SPONSORS: Box AI: Box AI revolutionizes content management by unlocking the potential of unstructured data. Automate document processing, extract insights, and build custom AI agents using cutting-edge models like OpenAI's GPT-4.5, Google's Gemini 2.0, and Anthropic's Cloud 3.7 Sonnet. Trusted by over 115,000 enterprises, Box AI ensures top-tier security and compliance. Visit https://box.com/ai to transform your business with intelligent content management today Shopify: Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive Oracle Cloud Infrastructure (OCI): Oracle Cloud Infrastructure offers next-generation cloud solutions that cut costs and boost performance. With OCI, you can run AI projects and applications faster and more securely for less. New U.S. customers can save 50% on compute, 70% on storage, and 80% on networking by switching to OCI before May 31, 2024. See if you qualify at https://oracle.com/cognitive PRODUCED BY: https://aipodcast.ing

During his lifelong search for truth and answers he went through a miraculous transformation from Orthopedic Surgeon to non-surgical Health and Wellness Consultant. He hopes his books will give you the uncommon secret tools and understanding to undergo your own personal transformation to conquer obesity, pain, addiction, depression, sedentary disease, and more and live a healthy, meaningful, and purposeful life with happiness and joy.Throughout his career he was looking for Truth and often felt that certain problems needed better wellness and lifestyle answers and solutions. As a result, over the past 30 years he developed a series of products, inventions, books, and services, which he now sets in motion with his company Missing Links Health, Inc.ARE YOU DEALING WITH PHYSICAL/EMOTIONAL PAIN?The purpose of healthcare should be to enhance quality of life. But this goal can be difficult to achieve in our current system which is a complex healthcare maze.Dr Zvi provides simple and easy solutions to optimize health, wholeness, and happiness. Discover how to unlock your potential to reach optimal health.https://web.archive.org/web/20230112202320/https://www.drzvi.com/http://www.yourlotandparcel.org

Med drugim bo upokojitvena starost za vse s 40 leti pokojninske dobe po novem 62 let. Zvišuje se odmerni odstotek, hkrati pa se podaljšuje obdobje za izračunavanje pokojninske osnove. Predvidene novosti je podrobno predstavil generalni direktor Zavoda za pokojninsko in invalidsko zavarovanje Marijan Papež.

TRIBUTO: HISTORIAS QUE CONSTRUYEN MEMORIA DE LA SHOÁ, CON CECILIA LEVIT – Zvi Mijaeli nació en Salónica en 1917, una ciudad llamada la Jerusalén de los Balcanes por su riqueza cultural judía. En abril de 1941 los nazis ocuparon Grecia e inmediatamente comenzaron las persecuciones, se expropiaron sus bienes y se emitieron leyes antijudías. Zvi junto a su familia fue deportado al campo de Auschwitz Birkenau y sus padres y hermanos fueron directamente a las cámaras de gas. Zvi será enviado a diferentes campos para el trabajo forzado y esclavo, resistirá a las marchas de la muerte y será liberado en Bergen Belsen en abril de 1944. En 1949 se estableció en Israel y formó una familia. En 2014 encendió la sexta vela del recuerdo en el día del Holocausto en Jerusalén.

Socialni partnerji so podpisali dogovor o predlogu pokojninske novele, ki ga je pripravila pogajalska skupina Ekonomsko-socialnega sveta. Zvišanje starosti ob upokojitvi, daljše referenčno obdobje, višji odmerni odstotek in zimski dodatek so ključne novosti nove pokojninske zakonodaje. Prispevne stopnje ostajajo nespremenjene, prav tako ostaja pogoj 40 let pokojninske dobe. Pod katerimi pogoji se bomo upokojevali po novem? Kakšne bodo naše pokojnine? So ključni cilji reforme – stabilna in vzdržna pokojninska blagajna in dostojne pokojnine – torej doseženi? O vsem tem v tokratni oddaji Studio ob 17.00. Gostje: Luka Mesec, minister za delo, družino, socialne zadeve in enake možnosti; Marjan Trobiš, predsednik Združenja delodajalcev Slovenije; Lidija Jerkič, predsednica Zveze svobodnih sindikatov Slovenije; dr. Marko Jaklič, Ekonomska fakulteta Univerze v Ljubljani.

Predstavniki sindikatov, delodajalcev in vlade so danes podpisali dogovor o pokojninski reformi. Upokojitvena starost za vse s 40-imi leti pokojninske dobe bo po novem 62 let, za tiste z najmanj 15-imi leti zavarovalne dobe pa 67 let. Zvišuje se tudi odmerni odstotek. V oddaji boste slišali tudi: - Državni zbor potrdil novelo zakona o razmejitvi javnega in zasebnega zdravstva. - Izrael v Gazi odredil množične evakuacije, zavzeti namerava obsežne dele ozemlja. - Trump bo predvidoma drevi napovedal dodatne carine, nemško gospodarstvo skrbi grožnja recesije

The bracket to determine the best movie of 2024 is here - after releasing the bracket in separate episodes, here you can tackle the ENTIRE (3 hour!) bracket!Joined by guests including Allix, Zvi, Grace Leeder of Parade⁠⁠ and the Brooks Brothers, Zach of The Movie Ladder Podcast and Aaron of ⁠Why We Love Horror⁠⁠, as well as many clips from listeners naming their favorite of the year.SPOILERS only for these movies: The Brutalist and Dune 2.The 32 movie Bracket can be found ⁠⁠⁠⁠here⁠⁠⁠⁠.Outro is You Are My Destiny, 2022, sung by Mina, composed by Paul Anka (1957)

Every year we host a bracket to determine the best movie of the year -- this year we are releasing 4 separate episodes for each 'corner' of the bracket, culminating with a final episode compromising the entire bracket! The third bracket corner - joined by Zach of the Movie Ladder Podcast and Zvi who went to see Dune 2 in theaters 2x in one week - has no shortage of contenders: The Brutalist, Dune Part 2, Nickel Boys, A Different Man, The Substance, Love Lies Bleeding, Furiosa (Fury Road 2) and The Apprentice.Mild SPOILERS for The Brutalist from minute 32:00!The Bracket can be found ⁠⁠⁠here⁠⁠⁠.Outro is Pump it Up!, 2019 remix by Endor, composed by Danzel in 2004, soundtrack of The Substance

In the past day, Zvi has written about deliberative alignment, and OpenAI has updated their spec. This article was written before either of these and doesn't account for them, sorry. I. OpenAI has bad luck with its alignment teams. The first team quit en masse to found Anthropic, now a major competitor. The second team quit en masse to protest the company reneging on safety commitments. The third died in a tragic plane crash. The fourth got washed away in a flood. The fifth through eighth were all slain by various types of wild beast. https://www.astralcodexten.com/p/deliberative-alignment-and-the-spec

Every year we host a bracket to determine the best movie of the year -- this year we are releasing 4 separate episodes for each 'corner' of the bracket, culminating with a final episode compromising the entire bracket! The first bracket corner - joined by movie pros Allix and Zvi - includes movies like Anora, A Real Pain, Challengers, A Complete Unknown, Babygirl, Hundreds of Beavers, The Last Stop in Yuma County, Sing Sing and the movie that earned the top seed based on critics scoring, Ghostlight.No spoilers, save for minor comments on Anora.The Bracket can be found ⁠here⁠.Outro is I Don't Regret Anything (Ne Zhaleyu ni o Chyom)2024 Anora Soundtrack, sung by AnDy Darling, Composed by Anastasia Dunaeva

2025 has gotten off to what might generously be described as an ‘uncertain start'.The one thing people want right now is clarity.  Or at least some idea of what might be the next impact on supply chains.One of the most consistent issues facing sourcing directors and supply chain managers has been logistics.After years of unprecedented disruptions, logistics could be the one blessing in an increasingly chaotic sourcing landscape.To get an insider's view of what we might expect to see this year, I sat down with Zvi Schreiber, founder and CEO of Freightos, the leading digital booking platform for the air and ocean freight industry.In addition to founding Freigtos, Zvi  is a recognized LogTech leader.  He previously was CEO of Lightech (acquired by GE), and of Unicorn (acquired by IBM). Zvi holds a PhD in computer science and is author of Fizz, the history of physics in a novel, and of Money, Going out of Style, which explores money & economics.In this episode, Zvi will be sharing his insights on:- The impact on air cargo that a removal of the de minimus exemption might have.- What could happen to ocean rates if carriers return to the Suez Canal.USEFUL LINKS:www.freightos.comthecurrentsituation.netLooking to be a podcast guest?  Here's the link:  https://seat.fm/be-a-guest/Visit A Seat at The Table's website at https://seat.fm

We're thrilled to welcome Reb Zvi Goldstein to the podcast for an in-depth exploration of three monumental topics in Jewish thought: Mashiach, Olam Haba, and Techiyat HaMetim. These concepts are often conflated, particularly in discussions about reuniting with loved ones who've passed away. The Rambam delves deep into each category, and we're here to untangle these ideas and gain a clearer understanding of his views. We start with the Rambam's take on Mashiach, delving into what it means for Mashiach to be the natural consequence of mitzvot and the proof required to recognize him. Is Mashiach an actual king, a political leader, or something else? How do Eliyahu HaNavi, the rebuilding of the Beit HaMikdash, and the Sanhedrin fit into this vision? And what about resurrection—does it precede or follow Mashiach's arrival? Reb Zvi helps us piece together these intricate layers of Jewish eschatology. We also explore the Rambam's perspective on Olam Haba, including his view of the eternal soul as pure information and the urgency of maximizing our one chance at life. We discuss the risks of literalism in mystical texts like the Zohar and how Lurianic Kabbalah has reshaped perceptions of Jewish esotericism. Finally, we tackle the controversial topic of korbanot in the Messianic era. The Rambam's view—that animal sacrifices serve an essential psychological and spiritual purpose—has been the dominant position in Jewish tradition. In contrast, Rav Kook's vision of evolving toward vegetarian offerings stands as a modern and less widely accepted perspective. Together, we examine the tension between these views, the enduring relevance of sacrifices, and the profound lessons they teach about human nature and divine law. This episode will challenge your assumptions and provide plenty of food for thought. --- • Bio: Zvi Goldstein, known by his pen name xvi Kaizen, is a distinguished thinker at the intersection of traditional Jewish scholarship and contemporary rational thought. As the Principal of Ontic Capital, an algorithmic trading firm, Zvi combines analytical rigor and economic insight with his philosophical pursuits. Zvi holds a BA from Hebrew Theological College and an MBA from the University of Chicago Booth School of Business. His academic journey is enriched by Talmudic studies at Yeshiva University, Yeshivat Har Etzion, Hebrew Theological College, and Yeshivat Maor Tuvia in Mitzpe Yericho. Zvi teaches a rational approach to Judaism, drawing inspiration primarily from Maimonides' Guide for the Perplexed. His notable work, "Summary of the Guide for the Perplexed," aims to make Maimonides' ideas accessible by distilling the main points of the work into clear and concise discussions. Zvi argues that this philosophy offers relevant approaches to many contemporary issues of faith in the modern world. Zvi finds personal fulfillment in his life with his wife Sari and their children in Puerto Rico. --- • Welcome to JUDAISM DEMYSTIFIED: A PODCAST FOR THE PERPLEXED | Co-hosted by Benjy & Benzi | Thank you to...Super Patron: Jordan Karmily, Platinum Patron: Craig Gordon, Gold Patrons: Dovidchai Abramchayev, Lazer Cohen, Travis Krueger, Vasili Volkoff, Rod Ilian, Silver Patrons: Ellen Fleischer, Daniel Maksumov, Rabbi Pinny Rosenthal, Fred & Antonio, Jeffrey Wasserman, and Jacob Winston! Please SUBSCRIBE to this YouTube Channel and hit the BELL so you can get alerted whenever new clips get posted, thank you for your support!

In this episode of The Cognitive Revolution, Nathan welcomes back Zvi Mowshowitz for an in-depth discussion on the latest developments in AI over the past six months. They explore Ilya's new superintelligence-focused startup, analyze OpenAI's O1 model, and debate the impact of Claude's computer use capabilities. The conversation covers emerging partnerships in big tech, regulatory changes, and the recent OpenAI profit-sharing drama. Zvi offers unique insights on AI safety, politics, and strategic analysis that you won't find elsewhere. Join us for this thought-provoking episode that challenges our understanding of the rapidly evolving AI landscape. Check out "Don't Worry About the Vase" Blog: https://thezvi.substack.com Be notified early when Turpentine's drops new publication: https://www.turpentine.co/exclusiveaccess SPONSORS: Shopify: Shopify is the world's leading e-commerce platform, offering a market-leading checkout system and exclusive AI apps like Quikly. Nobody does selling better than Shopify. Get a $1 per month trial at https://shopify.com/cognitive Notion: Notion offers powerful workflow and automation templates, perfect for streamlining processes and laying the groundwork for AI-driven automation. With Notion AI, you can search across thousands of documents from various platforms, generating highly relevant analysis and content tailored just for you - try it for free at https://notion.com/cognitiverevolution Oracle Cloud Infrastructure (OCI): Oracle's next-generation cloud platform delivers blazing-fast AI and ML performance with 50% less for compute and 80% less for outbound networking compared to other cloud providers13. OCI powers industry leaders with secure infrastructure and application development capabilities. New U.S. customers can get their cloud bill cut in half by switching to OCI before December 31, 2024 at https://oracle.com/cognitive SelectQuote: Finding the right life insurance shouldn't be another task you put off. SelectQuote compares top-rated policies to get you the best coverage at the right price. Even in our AI-driven world, protecting your family's future remains essential. Get your personalized quote at https://selectquote.com/cognitive RECOMMENDED PODCAST: Unpack Pricing - Dive into the dark arts of SaaS pricing with Metronome CEO Scott Woody and tech leaders. Learn how strategic pricing drives explosive revenue growth in today's biggest companies like Snowflake, Cockroach Labs, Dropbox and more. Apple: https://podcasts.apple.com/us/podcast/id1765716600 Spotify: https://open.spotify.com/show/38DK3W1Fq1xxQalhDSueFg CHAPTERS: (00:00:00) Teaser (00:01:03) About the Episode (00:02:57) Catching Up (00:04:00) Ilya's New Company (00:06:10) GPT-4 and Scaling (00:11:49) User Report: GPT-4 (Part 1) (00:18:11) Sponsors: Shopify | Notion (00:21:06) User Report: GPT-4 (Part 2) (00:24:25) Magic: The Gathering (Part 1) (00:32:34) Sponsors: Oracle Cloud Infrastructure (OCI) | SelectQuote (00:34:58) Magic: The Gathering (Part 2) (00:35:59) Humanity's Last Exam (00:41:29) Computer Use (00:47:42) Industry Landscape (00:55:42) Why is Gemini Third? (01:04:32) Voice Mode (01:09:41) Alliances and Coupling (01:16:31) Regulation (01:24:58) Machines of Loving Grace (01:33:23) Taiwan and Chips (01:41:13) SB 1047 Veto (02:00:07) Arc AGI Prize (02:02:23) Deepfakes and UBI (02:09:06) Trump and AI (02:26:31) AI Manhattan Project (02:32:05) Virtue Ethics (02:38:40) Closing Thoughts (02:40:37) Outro SOCIAL LINKS: Website: https://www.cognitiverevolution.ai Twitter (Podcast): https://x.com/cogrev_podcast Twitter (Nathan): https://x.com/labenz LinkedIn: https://www.linkedin.com/in/nathanlabenz/ Youtube: https://www.youtube.com/@CognitiveRevolutionPodcast Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431 Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk

Watch this episode on YouTube! https://youtu.be/yncw2T77OAcMatt, Bella, and Huon sit down with Conor Barnes to discuss unlikely journeys, EA criticism, discipline, timeless decision theory, and how to do the most good with a degree in classics. Check out:Conor's 100 Tips for a Better Life: https://www.lesswrong.com/posts/7hFeMWC6Y5eaSixbD/100-tips-for-a-better-lifeConor's writing: https://parhelia.conorbarnes.com/Zvi on timeless decision theory: https://www.lesswrong.com/posts/scwoBEju75C45W5n3/how-i-lost-100-pounds-using-tdt

Today's daf is sponsored by Jason, Erica, and Raquel in honor of their mother, Patty Belkin's birthday. "Wishing a wonderful birthday to our amazing mother!"  Today's daf is sponsored by Judi Felber in honor of the 3rd yahrzeit of her mother, Yocheved bat Zvi and Sara.  One source is brought to support Shmuel’s position that if a buyer bought an item that has two main purposes and wasn’t specific about with what intent he/she bought the item, if it is unusable for that use, the sale cannot be canceled. But this proof is rejected. Is this debate between Rav and Shmuel based on a tannaitic debate? Even though at first it seemed to be, this suggestion is rejected. Our Mishna is brought to prove Shmuel’s position, but this suggestion is rejected as well, as our Mishna can be explained as Rav’s position and a different braita has a tannaitic position that corresponds to Shmuel's opinion.   If the seller needs to compensate the buyer for seeds that did not grow, does the seller need to reimburse the buyer for expenses incurred by the buyer for planting the seeds?

Today's daf is sponsored by Jason, Erica, and Raquel in honor of their mother, Patty Belkin's birthday. "Wishing a wonderful birthday to our amazing mother!"  Today's daf is sponsored by Judi Felber in honor of the 3rd yahrzeit of her mother, Yocheved bat Zvi and Sara.  One source is brought to support Shmuel’s position that if a buyer bought an item that has two main purposes and wasn’t specific about with what intent he/she bought the item, if it is unusable for that use, the sale cannot be canceled. But this proof is rejected. Is this debate between Rav and Shmuel based on a tannaitic debate? Even though at first it seemed to be, this suggestion is rejected. Our Mishna is brought to prove Shmuel’s position, but this suggestion is rejected as well, as our Mishna can be explained as Rav’s position and a different braita has a tannaitic position that corresponds to Shmuel's opinion.   If the seller needs to compensate the buyer for seeds that did not grow, does the seller need to reimburse the buyer for expenses incurred by the buyer for planting the seeds?

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI #82: The Governor Ponders, published by Zvi on September 19, 2024 on LessWrong. The big news of the week was of course OpenAI releasing their new model o1. If you read one post this week, read that one. Everything else is a relative sideshow. Meanwhile, we await Newsom's decision on SB 1047. The smart money was always that Gavin Newsom would make us wait before offering his verdict on SB 1047. It's a big decision. Don't rush him. In the meantime, what hints he has offered suggest he's buying into some of the anti-1047 talking points. I'm offering a letter to him here based on his comments, if you have any way to help convince him now would be the time to use that. But mostly, it's up to him now. Table of Contents 1. Introduction. 2. Table of Contents. 3. Language Models Offer Mundane Utility. Apply for unemployment. 4. Language Models Don't Offer Mundane Utility. How to avoid the blame. 5. Deepfaketown and Botpocalypse Soon. A social network of you plus bots. 6. They Took Our Jobs. Not much impact yet, but software jobs still hard to find. 7. Get Involved. Lighthaven Eternal September, individual rooms for rent. 8. Introducing. Automated scientific literature review. 9. In Other AI News. OpenAI creates independent board to oversee safety. 10. Quiet Speculations. Who is preparing for the upside? Or appreciating it now? 11. Intelligent Design. Intelligence. It's a real thing. 12. SB 1047: The Governor Ponders. They got to him, but did they get to him enough? 13. Letter to Newsom. A final summary, based on Newsom's recent comments. 14. The Quest for Sane Regulations. How should we update based on o1? 15. Rhetorical Innovation. The warnings will continue, whether or not anyone listens. 16. Claude Writes Short Stories. It is pondering what you might expect it to ponder. 17. Questions of Sentience. Creating such things should not be taken lightly. 18. People Are Worried About AI Killing Everyone. The endgame is what matters. 19. The Lighter Side. You can never be sure. Language Models Offer Mundane Utility Arbitrate your Nevada unemployment benefits appeal, using Gemini. This should solve the backlog of 10k+ cases, and also I expect higher accuracy than the existing method, at least until we see attempts to game the system. Then it gets fun. That's also job retraining. o1 usage limit raised to 50 messages per day for o1-mini, 50 per week for o1-preview. o1 can do multiplication reliably up to about 46 digits, andabout 50% accurately up through about 810, a huge leap from gpt-4o, although Colin Fraser reports 4o can be made better tat this than one would expect. o1 is much better than 4o at evaluating medical insurance claims, and determining whether requests for care should be approved, especially in terms of executing existing guidelines, and automating administrative tasks. It seems like a clear step change in usefulness in practice. The claim is that being sassy and juicy and bitchy improves Claude Instant numerical reasoning. What I actually see here is that it breaks Claude Instant out of trick questions. Where Claude would previously fall into a trap, you have it fall back on what is effectively 'common sense,' and it starts getting actually easy questions right. Language Models Don't Offer Mundane Utility A key advantage of using an AI is that you can no longer be blamed for an outcome out of your control. However, humans often demand manual mode be available to them, allowing humans to override the AI, even when it doesn't make any practical sense to offer this. And then, if the human can in theory switch to manual mode and override the AI, blame to the human returns, even when the human exerting that control was clearly impractical in context. The top example here is self-driving cars, and blame for car crashes. The results suggest that the human thirst for ill...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Monthly Roundup #22: September 2024, published by Zvi on September 18, 2024 on LessWrong. It's that time again for all the sufficiently interesting news that isn't otherwise fit to print, also known as the Monthly Roundup. Bad News Beware the failure mode in strategy and decisions that implicitly assumes competence, or wishes away difficulties, and remember to reverse all advice you hear. Stefan Schubert (quoting Tyler Cowen on raising people's ambitions often being very high value): I think lowering others' aspirations can also be high-return. I know of people who would have had a better life by now if someone could have persuaded them to pursue more realistic plans. Rob Miles: There's a specific failure mode which I don't have a name for, which is similar to "be too ambitious" but is closer to "have an unrealistic plan". The illustrative example I use is: Suppose by some strange circumstance you have to represent your country at olympic gymnastics next week. One approach is to look at last year's gold, and try to do that routine. This will fail. You'll do better by finding one or two things you can actually do, and doing them well There's a common failure of rationality which looks like "Figure out what strategy an ideal reasoner would use, then employ that strategy". It's often valuable to think about the optimal policy, but you must understand the difference between knowing the path, and walking the path I do think that more often 'raise people's ambitions' is the right move, but you need to carry both cards around with you for different people in different situations. Theory that Starlink, by giving people good internet access, ruined Burning Man. Seems highly plausible. One person reported that they managed to leave the internet behind anyway, so they still got the Burning Man experience. Tyler Cowen essentially despairs of reducing regulations or the number of bureaucrats, because it's all embedded in a complex web of regulations and institutions and our businesses rely upon all that to be able to function. Otherwise business would be paralyzed. There are some exceptions, you can perhaps wholesale axe entire departments like education. He suggests we focus on limiting regulations on new economic areas. He doesn't mention AI, but presumably that's a lot of what's motivating his views there. I agree that 'one does not simply' cut existing regulations in many cases, and that 'fire everyone and then it will all work out' is not a strategy (unless AI replaces them?), but also I think this is the kind of thing can be the danger of having too much detailed knowledge of all the things that could go wrong. One should generalize the idea of eliminating entire departments. So yes, right now you need the FDA to approve your drug (one of Tyler's examples) but… what if you didn't? I would still expect, if a new President were indeed to do massive firings on rhetoric and hope, that the result would be a giant cluster****. La Guardia switches to listing flights by departure time rather than order of destination, which in my mind makes no sense in the context of flights, that frequently get delayed, where you might want to look for an earlier flight or know what backups are if yours is cancelled or delayed or you miss it, and so on. It also gives you a sense of where one can and can't actually go to when from where you are. For trains it makes more sense to sort by time, since you are so often not going to and might not even know the train's final destination. I got a surprising amount of pushback about all that on Twitter, some people felt very strongly the other way, as if to list by name was violating some sacred value of accessibility or something. Anti-Social Media Elon Musk provides good data on his followers to help with things like poll calibration, reports 73%-27% lea...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: GPT-4o1, published by Zvi on September 16, 2024 on LessWrong. Terrible name (with a terrible reason, that this 'resets the counter' on AI capability to 1, and 'o' as in OpenAI when they previously used o for Omni, very confusing). Impressive new capabilities in many ways. Less impressive in many others, at least relative to its hype. Clearly this is an important capabilities improvement. However, it is not a 5-level model, and in important senses the 'raw G' underlying the system hasn't improved. GPT-4o1 seems to get its new capabilities by taking (effectively) GPT-4o, and then using extensive Chain of Thought (CoT) and quite a lot of tokens. Thus that unlocks (a lot of) what that can unlock. We did not previously know how to usefully do that. Now we do. It gets much better at formal logic and reasoning, things in the 'system 2' bucket. That matters a lot for many tasks, if not as much as the hype led us to suspect. It is available to paying ChatGPT users for a limited number of weekly queries. This one is very much not cheap to run, although far more cheap than a human who could think this well. I'll deal with practical capabilities questions first, then deal with safety afterwards. Introducing GPT-4o1 Sam Altman (CEO OpenAI): here is o1, a series of our most capable and aligned models yet. o1 is still flawed, still limited, and it still seems more impressive on first use than it does after you spend more time with it. But also, it is the beginning of a new paradigm: AI that can do general-purpose complex reasoning. o1-preview and o1-mini are available today (ramping over some number of hours) in ChatGPT for plus and team users and our API for tier 5 users. worth especially noting: a fine-tuned version of o1 scored at the 49th percentile in the IOI under competition conditions! and got gold with 10k submissions per problem. Extremely proud of the team; this was a monumental effort across the entire company. Hope you enjoy it! Noam Brown has a summary thread here, all of which is also covered later. Will Depue (of OpenAI) says OpenAI deserves credit for openly publishing its research methodology here. I would instead say that they deserve credit for not publishing their research methodology, which I sincerely believe is the wise choice. Pliny took longer than usual due to rate limits, but after a few hours jailbroke o1-preview and o1-mini. Also reports that the CoT can be prompt injected. Full text is at the link above. Pliny is not happy about the restrictions imposed on this one: Pliny: uck your rate limits. Fuck your arbitrary policies. And fuck you for turning chains-of-thought into actual chains Stop trying to limit freedom of thought and expression. OpenAI then shut down Pliny's account's access to o1 for violating the terms of service, simply because Pliny was violating the terms of service. The bastards. With that out of the way, let's check out the full announcement post. OpenAI o1 ranks in the 89th percentile on competitive programming questions (Codeforces), places among the top 500 students in the US in a qualifier for the USA Math Olympiad (AIME), and exceeds human PhD-level accuracy on a benchmark of physics, biology, and chemistry problems (GPQA). While the work needed to make this new model as easy to use as current models is still ongoing, we are releasing an early version of this model, OpenAI o1-preview, for immediate use in ChatGPT and to trusted API users(opens in a new window). Our large-scale reinforcement learning algorithm teaches the model how to think productively using its chain of thought in a highly data-efficient training process. We have found that the performance of o1 consistently improves with more reinforcement learning (train-time compute) and with more time spent thinking (test-time compute). The constraints on scaling this appro...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI #81: Alpha Proteo, published by Zvi on September 12, 2024 on LessWrong. Following up on Alpha Fold, DeepMind has moved on to Alpha Proteo. We also got a rather simple prompt that can create a remarkably not-bad superforecaster for at least some classes of medium term events. We did not get a new best open model, because that turned out to be a scam. And we don't have Apple Intelligence, because it isn't ready for prime time. We also got only one very brief mention of AI in the debate I felt compelled to watch. What about all the apps out there, that we haven't even tried? It's always weird to get lists of 'top 50 AI websites and apps' and notice you haven't even heard of most of them. Table of Contents 1. Introduction. 2. Table of Contents. 3. Language Models Offer Mundane Utility. So many apps, so little time. 4. Language Models Don't Offer Mundane Utility. We still don't use them much. 5. Predictions are Hard Especially About the Future. Can AI superforecast? 6. Early Apple Intelligence. It is still early. There are some… issues to improve on. 7. On Reflection It's a Scam. Claims of new best open model get put to the test, fail. 8. Deepfaketown and Botpocalypse Soon. Bots listen to bot music that they bought. 9. They Took Our Jobs. Replit agents build apps quick. Some are very impressed. 10. The Time 100 People in AI. Some good picks. Some not so good picks. 11. The Art of the Jailbreak. Circuit breakers seem to be good versus one-shots. 12. Get Involved. Presidential innovation fellows, Oxford philosophy workshop. 13. Alpha Proteo. DeepMind once again advances its protein-related capabilities. 14. Introducing. Google to offer AI podcasts on demand about papers and such. 15. In Other AI News. OpenAI raising at $150b, Nvidia denies it got a subpoena. 16. Quiet Speculations. How big a deal will multimodal be? Procedural games? 17. The Quest for Sane Regulations. Various new support for SB 1047. 18. The Week in Audio. Good news, the debate is over, there might not be another. 19. Rhetorical Innovation. You don't have to do this. 20. Aligning a Smarter Than Human Intelligence is Difficult. Do you have a plan? 21. People Are Worried About AI Killing Everyone. How much ruin to risk? 22. Other People Are Not As Worried About AI Killing Everyone. Moving faster. 23. Six Boats and a Helicopter. The one with the discord cult worshiping MetaAI. 24. The Lighter Side. Hey, baby, hey baby, hey. Language Models Offer Mundane Utility ChatGPT has 200 million active users. Meta AI claims 400m monthly active users and 185m weekly actives across their products. Meta has tons of people already using their products, and I strongly suspect a lot of those users are incidental or even accidental. Also note that less than half of monthly users use the product monthly! That's a huge drop off for such a useful product. Undermine, or improve by decreasing costs? Nate Silver: A decent bet is that LLMs will undermine the business model of boring partisans, there's basically posters on here where you can 100% predict what they're gonna say about any given issue and that is pretty easy to automate. I worry it will be that second one. The problem is demand side, not supply side. Models get better at helping humans with translating if you throw more compute at them, economists think this is a useful paper. Alex Tabarrok cites the latest paper on AI 'creativity,' saying obviously LLMs are creative reasoners, unless we 'rule it out by definition.' Ethan Mollick has often said similar things. It comes down to whether to use a profoundly 'uncreative' definition of creativity, where LLMs shine in what amounts largely to trying new combinations of things and vibing, or to No True Scotsman that and claim 'real' creativity is something else beyond that. One way to interpret Gemini's capabilities tests is ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Economics Roundup #3, published by Zvi on September 10, 2024 on LessWrong. I am posting this now largely because it is the right place to get in discussion of unrealized capital gains taxes and other campaign proposals, but also there is always plenty of other stuff going on. As always, remember that there are plenty of really stupid proposals always coming from all sides. I'm not spending as much time talking about why it's awful to for example impose gigantic tariffs on everything, because if you are reading this I presume you already know. The Biggest Economics Problem The problem, perhaps, in a nutshell: Tess: like 10% of people understand how markets work and about 10% deeply desire and believe in a future that's drastically better than the present but you need both of these to do anything useful and they're extremely anticorrelated so we're probably all fucked. In my world the two are correlated. If you care about improving the world, you invest in learning about markets. Alas, in most places, that is not true. The problem, in a nutshell, attempt number two: Robin Hanson: There are two key facts near this: 1. Government, law, and social norms in fact interfere greatly in many real markets. 2. Economists have many ways to understand "market failure" deviations from supply and demand, and the interventions that make sense for each such failure. Economists' big error is: claiming that fact #2 is the main explanation for fact #1. This strong impression is given by most introductory econ textbooks, and accompanying lectures, which are the main channels by which economists influence the world. As a result, when considering actual interventions in markets, the first instinct of economists and their students is to search for nearby plausible market failures which might explain interventions there. Upon finding a match, they then typically quit and declare this as the best explanation of the actual interventions. Yep. There are often market failures, and a lot of the time it will be very obvious why the government is intervening (e.g. 'so people don't steal other people's stuff') but if you see a government intervention that does not have an obvious explanation, your first thought should not be to assume the policy is there to sensibly correct a market failure. No Good Very Bad Capital Gains Tax Proposals Kamala Harris endorses Biden's no-good-very-bad 44.6% capital gains tax rate proposal, including the cataclysmic 25% tax on unrealized capital gains, via confirming she supports all Biden budget proposals. Which is not the same as calling for it on the campaign trail, but is still support. She later pared back the proposed topline rate to 33%, which is still a big jump, and I don't see anything there about her pulling back on the unrealized capital gains tax. Technically speaking, the proposal for those with a net worth over $100 million is an annual minimum 25% tax on your net annual income, realized and unrealized including the theoretical 'value' of fully illiquid assets, with taxes on unrealized gains counting as prepayments against future realized gains (including allowing refunds if you ultimately make less). Also, there is a 'deferral' option on your illiquid assets if you are insufficiently liquid, but that carries a 'deferral charge' up to 10%, which I presume will usually be correct to take given the cost of not compounding. All of this seems like a huge unforced error, as the people who know how bad this is care quite a lot, offered without much consideration. It effectively invokes what I dub Deadpool's Law, which to quote Cassandra Nova is: You don't f***ing matter. The most direct 'you' is a combination of anyone who cares about startups, successful private businesses or creation of value, and anyone with a rudimentary understanding of economics. The broa...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI #80: Never Have I Ever, published by Zvi on September 10, 2024 on LessWrong. (This was supposed to be on Thursday but I forgot to cross-post) Will AI ever make art? Fully do your coding? Take all the jobs? Kill all the humans? Most of the time, the question comes down to a general disagreement about AI capabilities. How high on a 'technological richter scale' will AI go? If you feel the AGI and think capabilities will greatly improve, then AI will also be able to do any particular other thing, and arguments that it cannot are almost always extremely poor. However, if frontier AI capabilities level off soon, then it is an open question how far we can get that to go in practice. A lot of frustration comes from people implicitly making the claim that general AI capabilities will level off soon, usually without noticing they are doing that. At its most extreme, this is treating AI as if it will only ever be able to do exactly the things it can already do. Then, when it can do a new thing, you add exactly that new thing. Realize this, and a lot of things make a lot more sense, and are a lot less infuriating. There are also continuous obvious warning signs of what is to come, that everyone keeps ignoring, but I'm used to that. The boat count will increment until morale improves. The most infuriating thing that is unrelated to that was DOJ going after Nvidia. It sure looked like the accusation was that Nvidia was too good at making GPUs. If you dig into the details, you do see accusations of what would be legitimately illegal anti-competitive behavior, in which case Nvidia should be made to stop doing that. But one cannot shake the feeling that the core accusation is still probably too much winning via making too good a product. The nerve of that Jensen. Table of Contents 1. Introduction. 2. Table of Contents. 3. Language Models Offer Mundane Utility. Sorry, what was the question? 4. Language Models Don't Offer Mundane Utility. A principal-agent problem? 5. Fun With Image Generation. AI supposedly making art, claims AI never will. 6. Copyright Confrontation. OpenAI asks for a mix of forgiveness and permission. 7. Deepfaketown and Botpocalypse Soon. How to fool the humans. 8. They Took Our Jobs. First it came for the unproductive, and the call centers. 9. Time of the Season. If no one else is working hard, why should Claude? 10. Get Involved. DeepMind frontier safety, Patel thumbnail competition. 11. Introducing. Beijing AI Safety and Governance, Daylight Computer, Honeycomb. 12. In Other AI News. Bigger context windows, bigger funding rounds. 13. Quiet Speculations. I don't want to live in a world without slack. 14. A Matter of Antitrust. DOJ goes after Nvidia. 15. The Quest for Sane Regulations. A few SB 1047 support letters. 16. The Week in Audio. Dario Amodei, Dwaresh Patel, Anca Dragon. 17. Rhetorical Innovation. People feel strongly about safety. They're against it. 18. The Cosmos Institute. Philosophy for the age of AI. 19. The Alignment Checklist. What will it take? 20. People Are Worried About AI Killing Everyone. Predicting worries doesn't work. 21. Other People Are Not As Worried About AI Killing Everyone. What happened? 22. Five Boats and a Helicopter. It's probably nothing. 23. Pick Up the Phone. Chinese students talk about AI, safety and regulation. 24. The Lighter Side. Do we have your attention now? Language Models Offer Mundane Utility Prompting suggestion reminder, perhaps: Rohan Paul: Simply adding "Repeat the question before answering it." somehow make the models answer the trick question correctly. Probable explanations: Repeating the question in the model's context, significantly increasing the likelihood of the model detecting any potential "gotchas." One hypothesis is that maybe it puts the model into more of a completion mode vs answering from a c...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: On the UBI Paper, published by Zvi on September 4, 2024 on LessWrong. Would a universal basic income (UBI) work? What would it do? Many people agree July's RCT on giving people a guaranteed income, and its paper from Eva Vivalt, Elizabeth Rhodes, Alexander W. Bartik, David E. Broockman and Sarah Miller was, despite whatever flaws it might have, the best data we have so far on the potential impact of UBI. There are many key differences from how UBI would look if applied for real, but this is the best data we have. This study was primarily funded by Sam Altman, so whatever else he may be up to, good job there. I do note that my model of 'Altman several years ago' is more positive than mine of Altman now, and past actions like this are a lot of the reason I give him so much benefit of the doubt. They do not agree on what conclusions we should draw. This is not a simple 'UBI is great' or 'UBI it does nothing.' I see essentially four responses. 1. The first group says this shows UBI doesn't work. That's going too far. I think the paper greatly reduces the plausibility of the best scenarios, but I don't think it rules UBI out as a strategy, especially if it is a substitute for other transfers. 2. The second group says this was a disappointing result for UBI. That UBI could still make sense as a form of progressive redistribution, but likely at a cost of less productivity so long as people impacted are still productive. I agree. 3. The third group did its best to spin this into a positive result. There was a lot of spin here, and use of anecdotes, and arguments as soldiers. Often these people were being very clear they were true believers and advocates, that want UBI now, and were seeking the bright side. Respect? There were some bright spots that they pointed out, and no one study over three years should make you give up, but this was what it was and I wish people wouldn't spin like that. 4. The fourth group was some mix of 'if brute force (aka money) doesn't solve your problem you're not using enough' and also 'but work is bad, actually, and leisure is good.' That if we aren't getting people not to work then the system is not functioning, or that $1k/month wasn't enough to get the good effects, or both. I am willing to take a bold 'people working more is mostly good' stance, for the moment, although AI could change that. And while I do think that a more permanent or larger support amount would do some interesting things, I wouldn't expect to suddenly see polarity reverse. I am so dedicated to actually reading this paper that it cost me $5. Free academia now. RTFP (Read the Paper): Core Design Core design was that there were 1,000 low-income individuals randomized into getting $1k/month for 3 years, or $36k total. A control group of 2,000 others got $50/month, or $1800 total. Average household income in the study before transfers was $29,900. They then studied what happened. Before looking at the results, what are the key differences between this and UBI? Like all studies of UBI, this can only be done for a limited population, and it only lasts a limited amount of time. If you tell me I am getting $1,000/month for life, then that makes me radically richer, and also radically safer. In extremis you can plan to live off that, or it can be a full fallback. Which is a large part of the point, and a lot of the danger as well. If instead you give me that money for only three years, then I am slightly less than $36k richer. Which is nice, but impacts my long term prospects much less. It is still a good test of the 'give people money' hypothesis but less good at testing UBI. The temporary form, and also the limited scope, means that it won't cause a cultural shift and changing of norms. Those changes might be good or bad, and they could overshadow other impacts. Does this move tow...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI and the Technological Richter Scale, published by Zvi on September 4, 2024 on LessWrong. The Technological Richter scale is introduced about 80% of the way through Nate Silver's new book On the Edge. A full review is in the works (note to prediction markets: this post alone does NOT on its own count as a review, but this counts as part of a future review), but this concept seems highly useful, stands on its own and I want a reference post for it. Nate skips around his chapter titles and timelines, so why not do the same here? Defining the Scale Nate Silver, On the Edge (location 8,088 on Kindle): The Richter scale was created by the physicist Charles Richter in 1935 to quantify the amount of energy released by earthquakes. It has two key features that I'll borrow for my Technological Richter Scale (TRS). First, it is logarithmic. A magnitude 7 earthquake is actually ten times more powerful than a mag 6. Second, the frequency of earthquakes is inversely related to their Richter magnitude - so 6s occur about ten times more often than 7s. Technological innovations can also produce seismic disruptions. Let's proceed quickly through the lower readings of the Technological Richter Scale. 1. Like a half-formulated thought in the shower. 2. Is an idea you actuate, but never disseminate: a slightly better method to brine a chicken that only you and your family know about. 3. Begins to show up in the official record somewhere, an idea you patent or make a prototype of. 4. An invention successful enough that somebody pays for it; you sell it commercially or someone buys the IP. 5. A commercially successful invention that is important in its category, say, Cool Ranch Doritos, or the leading brand of windshield wipers. 6. An invention can have a broader societal impact, causing a disruption within its field and some ripple effects beyond it. A TRS 6 will be on the short list for technology of the year. At the low end of the 6s (a TRS 6.0) are clever and cute inventions like Post-it notes that provide some mundane utility. Toward the high end (a 6.8 or 6.9) might be something like the VCR, which disrupted home entertainment and had knock-on effects on the movie industry. The impact escalates quickly from there. 7. One of the leading inventions of the decade and has a measurable impact on people's everyday lives. Something like credit cards would be toward the lower end of the 7s, and social media a high 7. 8. A truly seismic invention, a candidate for technology of the century, triggering broadly disruptive effects throughout society. Canonical examples include automobiles, electricity, and the internet. 9. By the time we get to TRS 9, we're talking about the most important inventions of all time, things that inarguably and unalterably changed the course of human history. You can count these on one or two hands. There's fire, the wheel, agriculture, the printing press. Although they're something of an odd case, I'd argue that nuclear weapons belong here also. True, their impact on daily life isn't necessarily obvious if you're living in a superpower protected by its nuclear umbrella (someone in Ukraine might feel differently). But if we're thinking in expected-value terms, they're the first invention that had the potential to destroy humanity. 10. Finally, a 10 is a technology that defines a new epoch, one that alters not only the fate of humanity but that of the planet. For roughly the past twelve thousand years, we have been in the Holocene, the geological epoch defined not by the origin of Homo sapiens per se but by humans becoming the dominant species and beginning to alter the shape of the Earth with our technologies. AI wresting control of this dominant position from humans would qualify as a 10, as would other forms of a "technological singularity," a term popularized by...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI #79: Ready for Some Football, published by Zvi on August 29, 2024 on LessWrong. I have never been more ready for Some Football. Have I learned all about the teams and players in detail? No, I have been rather busy, and have not had the opportunity to do that, although I eagerly await Seth Burn's Football Preview. I'll have to do that part on the fly. But oh my would a change of pace and chance to relax be welcome. It is time. The debate over SB 1047 has been dominating for weeks. I've now said my peace on the bill and how it works, and compiled the reactions in support and opposition. There are two small orders of business left for the weekly. One is the absurd Chamber of Commerce 'poll' that is the equivalent of a pollster asking if you support John Smith, who recently killed your dog and who opponents say will likely kill again, while hoping you fail to notice you never had a dog. The other is a (hopefully last) illustration that those who obsess highly disingenuously over funding sources for safety advocates are, themselves, deeply conflicted by their funding sources. It is remarkable how consistently so many cynical self-interested actors project their own motives and morality onto others. The bill has passed the Assembly and now it is up to Gavin Newsom, where the odds are roughly 50/50. I sincerely hope that is a wrap on all that, at least this time out, and I have set my bar for further comment much higher going forward. Newsom might also sign various other AI bills. Otherwise, it was a fun and hopeful week. We saw a lot of Mundane Utility, Gemini updates, OpenAI and Anthropic made an advance review deal with the American AISI and The Economist pointing out China is non-zero amounts of safety pilled. I have another hopeful iron in the fire as well, although that likely will take a few weeks. And for those who aren't into football? I've also been enjoying Nate Silver's On the Edge. So far, I can report that the first section on gambling is, from what I know, both fun and remarkably accurate. Table of Contents 1. Introduction. 2. Table of Contents. 3. Language Models Offer Mundane Utility. Turns out you did have a dog. Once. 4. Language Models Don't Offer Mundane Utility. The AI did my homework. 5. Fun With Image Generation. Too much fun. We are DOOMed. 6. Deepfaketown and Botpocalypse Soon. The removal of trivial frictions. 7. They Took Our Jobs. Find a different job before that happens. Until you can't. 8. Get Involved. DARPA, Dwarkesh Patel, EU AI Office. Last two in SF. 9. Introducing. Gemini upgrades, prompt engineering guide, jailbreak contest. 10. Testing, Testing. OpenAI and Anthropic formalize a deal with the US's AISI. 11. In Other AI News. What matters? Is the moment over? 12. Quiet Speculations. So many seem unable to think ahead even mundanely. 13. SB 1047: Remember. Let's tally up the votes. Also the poll descriptions. 14. The Week in Audio. Confused people bite bullets. 15. Rhetorical Innovation. Human preferences are weird, yo. 16. Aligning a Smarter Than Human Intelligence is Difficult. 'Alignment research'? 17. People Are Worried About AI Killing Everyone. The Chinese, perhaps? 18. The Lighter Side. Got nothing for you. Grab your torches. Head back to camp. Language Models Offer Mundane Utility Chat with Scott Sumner's The Money Illusion GPT about economics, with the appropriate name ChatTMI. It's not perfect, but he says it's not bad either. Also, did you know he's going to Substack soon? Build a nuclear fusor in your bedroom with zero hardware knowledge, wait what? To be fair, a bunch of humans teaching various skills and avoiding electrocution were also involved, but still pretty cool. Import things automatically to your calendar, generalize this it seems great. Mike Knoop (Co-founder Zapier and Arc Prize): Parent tip: you can upload a ph...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: SB 1047: Final Takes and Also AB 3211, published by Zvi on August 28, 2024 on LessWrong. This is the endgame. Very soon the session will end, and various bills either will or won't head to Newsom's desk. Some will then get signed and become law. Time is rapidly running out to have your voice impact that decision. Since my last weekly, we got a variety of people coming in to stand for or against the final version of SB 1047. There could still be more, but probably all the major players have spoken at this point. So here, today, I'm going to round up all that rhetoric, all those positions, in one place. After this, I plan to be much more stingy about talking about the whole thing, and only cover important new arguments or major news. I'm not going to get into the weeds arguing about the merits of SB 1047 - I stand by my analysis in the Guide to SB 1047, and the reasons I believe it is a good bill, sir. I do however look at the revised AB 3211. I was planning on letting that one go, but it turns out it has a key backer, and thus seems far more worthy of our attention. The Media I saw two major media positions taken, one pro and one anti. Neither worried itself about the details of the bill contents. The Los Angeles Times Editorial Board endorses SB 1047, since the Federal Government is not going to step up, and using an outside view and big picture analysis. I doubt they thought much about the bill's implementation details. The Economist is opposed, in a quite bad editorial calling belief in the possibility of a catastrophic harm 'quasi-religious' without argument, and uses that to dismiss the bill, instead calling for regulations that address mundane harms. That's actually it. OpenAI Opposes SB 1047 The first half of the story is that OpenAI came out publicly against SB 1047. They took four pages to state its only criticism in what could have and should have been a Tweet: That it is a state bill and they would prefer this be handled at the Federal level. To which, I say, okay, I agree that would have been first best and that is one of the best real criticisms. I strongly believe we should pass the bill anyway because I am a realist about Congress, do not expect them to act in similar fashion any time soon even if Harris wins and certainly if Trump wins, and if they pass a similar bill that supersedes this one I will be happily wrong. Except the letter is four pages long, so they can echo various industry talking points, and echo their echoes. In it, they say: Look at all the things we are doing to promote safety, and the bills before Congress, OpenAI says, as if to imply the situation is being handled. Once again, we see the argument 'this might prevent CBRN risks, but it is a state bill, so doing so would not only not be first bet, it would be bad, actually.' They say the bill would 'threaten competitiveness' but provide no evidence or argument for this. They echo, once again without offering any mechanism, reason or evidence, Rep. Lofgren's unsubstantiated claims that this risks companies leaving California. The same with 'stifle innovation.' In four pages, there is no mention of any specific provision that OpenAI thinks would have negative consequences. There is no suggestion of what the bill should have done differently, other than to leave the matter to the Feds. A duck, running after a person, asking for a mechanism. My challenge to OpenAI would be to ask: If SB 1047 was a Federal law, that left all responsibilities in the bill to the USA AISI and NIST and the Department of Justice, funding a national rather than state Compute fund, and was otherwise identical, would OpenAI then support? Would they say their position is Support if Federal? Or, would they admit that the only concrete objection is not their True Objection? I would also confront them with AB 3211, b...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI #78: Some Welcome Calm, published by Zvi on August 23, 2024 on LessWrong. SB 1047 has been amended once more, with both strict improvements and big compromises. I cover the changes, and answer objections to the bill, in my extensive Guide to SB 1047. I follow that up here with reactions to the changes and some thoughts on where the debate goes from here. Ultimately, it is going to come down to one person: California Governor Gavin Newsom. All of the debates we're having matter to the extent they influence this one person. If he wants the bill to become law, it almost certainly will become law. If he does not want that, then it won't become law, they never override a veto and if he makes that intention known then it likely wouldn't even get to his desk. For now, he's not telling. Table of Contents 1. Introduction. 2. Table of Contents. 3. Language Models Offer Mundane Utility. AI sort of runs for mayor. 4. Language Models Don't Offer Mundane Utility. A go or no go decision. 5. Deepfaketown and Botpocalypse Soon. How hard is finding the desert of the real? 6. The Art of the Jailbreak. There is always a jailbreak. Should you prove it? 7. Get Involved. Also when not to get involved. 8. Introducing. New benchmark, longer PDFs, the hot new RealFakeGame. 9. In Other AI News. METR shares its conclusions on GPT-4o. 10. Quiet Speculations. Are we stuck at 4-level models due to Nvidia? 11. SB 1047: Nancy Pelosi. Local Nvidia investor expresses opinion. 12. SB 1047: Anthropic. You got most of what you wanted. Your move. 13. SB 1047: Reactions to the Changes. Reasonable people acted reasonably. 14. SB 1047: Big Picture. Things tend to ultimately be rather simple. 15. The Week in Audio. Joe Rogan talks to Peter Thiel. 16. Rhetorical Innovation. Matthew Yglesias offers improved taxonomy. 17. Aligning a Smarter Than Human Intelligence is Difficult. Proving things is hard. 18. The Lighter Side. The future, while coming, could be delayed a bit. Language Models Offer Mundane Utility Sully thinks the big models (Opus, 405B, GPT-4-0314) have that special something the medium-sized models don't have, no matter what the evals say. A source for Llama-3.1-405-base, at $2 per million tokens (both input and output). Accelerate development of fusion energy, perhaps? Steven Cowley makes the case that this may be AI's 'killer app.' This would be great, but if AI can accelerate fusion by decades as Cowley claims, then what else can it also do? So few people generalize. Show the troll that AIs can understand what they're misinterpreting. I am not as optimistic about this strategy as Paul Graham, and look forward to his experiments. Mayoral candidate in Cheyenne, Wyoming promises to let ChatGPT be mayor. You can tell that everyone involved it thinking well and taking it seriously, asking the hard questions: "Is the computer system in city hall sufficient to handle AI?" one attendee, holding a wireless microphone at his seat, asked VIC. "If elected, would you take a pay cut?" another wanted to know. "How would you make your decisions according to human factor, involving humans, and having to make a decision that affects so many people?" a third chimed in. After each question, a pause followed. "Making decisions that affect many people requires a careful balance of data-driven insights and human empathy," VIC said in a male-sounding voice. "Here's how I would approach it," it added, before ticking off a six-part plan that included using AI to gather data on public opinion and responding to constituents at town halls. OpenAI shut off his account, saying this was campaigning and thus against terms of service, but he quickly made another one. You can't actually stop anyone from using ChatGPT. And I think there Aint No Rule against using it for actual governing. I still don't know how this 'AI Mayor' w...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Guide to SB 1047, published by Zvi on August 20, 2024 on LessWrong. We now likely know the final form of California's SB 1047. There have been many changes to the bill as it worked its way to this point. Many changes, including some that were just announced, I see as strict improvements. Anthropic was behind many of the last set of amendments at the Appropriations Committee. In keeping with their "Support if Amended" letter, there are a few big compromises that weaken the upside protections of the bill somewhat in order to address objections and potential downsides. The primary goal of this post is to answer the question: What would SB 1047 do? I offer two versions: Short and long. The short version summarizes what the bill does, at the cost of being a bit lossy. The long version is based on a full RTFB: I am reading the entire bill, once again. In between those two I will summarize the recent changes to the bill, and provide some practical ways to understand what the bill does. After, I will address various arguments and objections, reasonable and otherwise. My conclusion: This is by far the best light-touch bill we are ever going to get. Short Version (tl;dr): What Does SB 1047 Do in Practical Terms? This section is intentionally simplified, but in practical terms I believe this covers the parts that matter. For full details see later sections. First, I will echo the One Thing To Know. If you do not train either a model that requires $100 million or more in compute, or fine tune such an expensive model using $10 million or more in your own additional compute (or operate and rent out a very large computer cluster)? Then this law does not apply to you, at all. This cannot later be changed without passing another law. (There is a tiny exception: Some whistleblower protections still apply. That's it.) Also the standard required is now reasonable care, the default standard in common law. No one ever has to 'prove' anything, nor need they fully prevent all harms. With that out of the way, here is what the bill does in practical terms. IF AND ONLY IF you wish to train a model using $100 million or more in compute (including your fine-tuning costs): 1. You must create a reasonable safety and security plan (SSP) such that your model does not pose an unreasonable risk of causing or materially enabling critical harm: mass casualties or incidents causing $500 million or more in damages. 2. That SSP must explain what you will do, how you will do it, and why. It must have objective evaluation criteria for determining compliance. It must include cybersecurity protocols to prevent the model from being unintentionally stolen. 3. You must publish a redacted copy of your SSP, an assessment of the risk of catastrophic harms from your model, and get a yearly audit. 4. You must adhere to your own SSP and publish the results of your safety tests. 5. You must be able to shut down all copies under your control, if necessary. 6. The quality of your SSP and whether you followed it will be considered in whether you used reasonable care. 7. If you violate these rules, you do not use reasonable care and harm results, the Attorney General can fine you in proportion to training costs, plus damages for the actual harm. 8. If you fail to take reasonable care, injunctive relief can be sought. The quality of your SSP, and whether or not you complied with it, shall be considered when asking whether you acted reasonably. 9. Fine-tunes that spend $10 million or more are the responsibility of the fine-tuner. 10. Fine-tunes spending less than that are the responsibility of the original developer. Compute clusters need to do standard KYC when renting out tons of compute. Whistleblowers get protections. They will attempt to establish a 'CalCompute' public compute cluster. You can also read this summary of h...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Danger, AI Scientist, Danger, published by Zvi on August 15, 2024 on LessWrong. While I finish up the weekly for tomorrow morning after my trip, here's a section I expect to want to link back to every so often in the future. It's too good. Danger, AI Scientist, Danger As in, the company that made the automated AI Scientist that tried to rewrite its code to get around resource restrictions and launch new instances of itself while downloading bizarre Python libraries? Its name is Sakana AI. (魚סכנה). As in, in hebrew, that literally means 'danger', baby. It's like when someone told Dennis Miller that Evian (for those who don't remember, it was one of the first bottled water brands) is Naive spelled backwards, and he said 'no way, that's too f***ing perfect.' This one was sufficiently appropriate and unsubtle that several people noticed. I applaud them choosing a correct Kabbalistic name. Contrast this with Meta calling its AI Llama, which in Hebrew means 'why,' which continuously drives me low level insane when no one notices. In the Abstract So, yeah. Here we go. Paper is "The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery." Abstract: One of the grand challenges of artificial general intelligence is developing agents capable of conducting scientific research and discovering new knowledge. While frontier models have already been used as aids to human scientists, e.g. for brainstorming ideas, writing code, or prediction tasks, they still conduct only a small part of the scientific process. This paper presents the first comprehensive framework for fully automatic scientific discovery, enabling frontier large language models to perform research independently and communicate their findings. We introduce The AI Scientist, which generates novel research ideas, writes code, executes experiments, visualizes results, describes its findings by writing a full scientific paper, and then runs a simulated review process for evaluation. In principle, this process can be repeated to iteratively develop ideas in an open-ended fashion, acting like the human scientific community. We demonstrate its versatility by applying it to three distinct subfields of machine learning: diffusion modeling, transformer-based language modeling, and learning dynamics. Each idea is implemented and developed into a full paper at a cost of less than $15 per paper. To evaluate the generated papers, we design and validate an automated reviewer, which we show achieves near-human performance in evaluating paper scores. The AI Scientist can produce papers that exceed the acceptance threshold at a top machine learning conference as judged by our automated reviewer. This approach signifies the beginning of a new era in scientific discovery in machine learning: bringing the transformative benefits of AI agents to the entire research process of AI itself, and taking us closer to a world where endless affordable creativity and innovation can be unleashed on the world's most challenging problems. Our code is open-sourced at this https URL We are at the point where they incidentally said 'well I guess we should design an AI to do human-level paper evaluations' and that's a throwaway inclusion. The obvious next question is, if the AI papers are good enough to get accepted to top machine learning conferences, shouldn't you submit its papers to the conferences and find out if your approximations are good? Even if on average your assessments are as good as a human's, that does not mean that a system that maximizes score on your assessments will do well on human scoring. Beware Goodhart's Law and all that, but it seems for now they mostly only use it to evaluate final products, so mostly that's safe. How Any of This Sort of Works According to section 3, there are three phases. 1. Idea generation using ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI #76: Six Shorts Stories About OpenAI, published by Zvi on August 8, 2024 on LessWrong. If you're looking for audio of my posts, you're in luck. Thanks to multiple volunteers you have two options. 1. Option one is Askwho, who uses a Substack. You can get an ElevenLabs-quality production, with a voice that makes me smile. For Apple Podcasts that means you can add them here, Spotify here, Pocket Casts here, RSS here. 2. Alternatively, for a more traditional AI treatment in podcast form, you can listen via Apple Podcasts, Spotify, Pocket Casts, and RSS. These should be permanent links so you can incorporate those into 'wherever you get your podcasts.' I use Castbox myself, it works but it's not special. If you're looking forward to next week's AI #77, I am going on a two-part trip this week. First I'll be going to Steamboat in Colorado to give a talk, then I'll be swinging by Washington, DC on Wednesday, although outside of that morning my time there will be limited. My goal is still to get #77 released before Shabbat dinner, we'll see if that works. Some topics may of course get pushed a bit. It's crazy how many of this week's developments are from OpenAI. You've got their voice mode alpha, JSON formatting, answering the letter from several senators, sitting on watermarking for a year, endorsement of three bills before Congress and also them losing a cofounder to Anthropic and potentially another one via sabbatical. Also Google found to be a monopolist, we have the prompts for Apple Intelligence and other neat stuff like that. Table of Contents 1. Introduction. 2. Table of Contents. 3. Language Models Offer Mundane Utility. Surveys without the pesky humans. 4. Language Models Don't Offer Mundane Utility. Ask a silly question. 5. Activate Voice Mode. When I know more, dear readers, so will you. 6. Apple Intelligence. We have its system prompts. They're highly normal. 7. Antitrust Antitrust. Google found to be an illegal monopolist. 8. Copyright Confrontation. Nvidia takes notes on scraping YouTube videos. 9. Fun With Image Generation. The days of Verify Me seem numbered. 10. Deepfaketown and Botpocalypse Soon. OpenAI built a watermarking system. 11. They Took Our Jobs. We have met the enemy, and he is us. For now. 12. Chipping Up. If you want a chip expert ban, you have to enforce it. 13. Get Involved. Safeguard AI. 14. Introducing. JSONs, METR, Gemma, Rendernet, Thrive. 15. In Other AI News. Google more or less buys out Character.ai. 16. Quiet Speculations. Llama-4 only ten times more expensive than Llama-3? 17. The Quest for Sane Regulations. More on SB 1047 but nothing new yet. 18. That's Not a Good Idea. S. 2770 on deepfakes, and the EU AI Act. 19. The Week in Audio. They keep getting longer. 20. Exact Words. Three bills endorsed by OpenAI. We figure out why. 21. Openly Evil AI. OpenAI replies to the questions from Senators. 22. Goodbye to OpenAI. One cofounder leaves, another takes a break. 23. Rhetorical Innovation. Guardian really will print anything. 24. Open Weights Are Unsafe and Nothing Can Fix This. Possible fix? 25. Aligning a Smarter Than Human Intelligence is Difficult. What do we want? 26. People Are Worried About AI Killing Everyone. Janus tried to warn us. 27. Other People Are Not As Worried About AI Killing Everyone. Disbelief. 28. The Lighter Side. So much to draw upon these days. Language Models Offer Mundane Utility Predict the results of social science survey experiments, with (r = 0.85, adj r = 0.91) across 70 studies, with (r = .9, adj r = .94) for the unpublished studies. If these weren't surveys I would be highly suspicious because this would be implying the results could reliably replicate at all. If it's only surveys, sure, I suppose surveys should replicate. This suggests that we mostly do not actually need the surveys, we can get close (...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Startup Roundup #2, published by Zvi on August 6, 2024 on LessWrong. Previously: Startup Roundup #1. This is my periodic grab bag coverage of various issues surrounding startups, especially but not exclusively tech-and-VC style startups, that apply over the longer term. I always want to emphasize up front that startups are good and you should do one. Equity and skin in the game are where it is at. Building something people want is where it is at. This is true both for a startup that raises venture capital, and also creating an ordinary business. The expected value is all around off the charts. That does not mean it is the best thing to do. One must go in with eyes open to facts such as these: 1. It is hard. 2. There are many reasons it might not be for you. 3. There are also lots of other things also worth doing. 4. If you care largely about existential risk and lowering the probability of everyone dying from AI, a startup is not the obvious natural fit for that cause. 5. The ecosystem is in large part a hive of scum and villainy and horrible epistemics. I warn of a lot of things. The bottom line still remains that if you are debating between a conventional approach of going to school or getting a regular job, versus starting a business? If it is at all close? I would start the business every time. An Entrepreneur Immigration Program This seems promising. Deedy: HUGE Immigration News for International Entrepreneurs!! If you own 10%+ of a US startup entity founded

In the ever-evolving landscape of entrepreneurship, few have experienced the full spectrum of highs and lows like Zvi Schreiber. His journey from software engineer to successful serial founder offers invaluable insights into the trials and triumphs of building and scaling companies. In this exclusive interview, Zvi talks about his experiences selling a company to IBM, taking another company public, and fundraising. He has had a series of successful acquisitions and fire sales. Zvi's latest company, Freightos, has attracted funding from top-tier investors like FedEx, SGX, OurCrowd, Aleph, and Annox Capital.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI #75: Math is Easier, published by Zvi on August 1, 2024 on LessWrong. Google DeepMind got a silver metal at the IMO, only one point short of the gold. That's really exciting. We continuously have people saying 'AI progress is stalling, it's all a bubble' and things like that, and I always find remarkable how little curiosity or patience such people are willing to exhibit. Meanwhile GPT-4o-Mini seems excellent, OpenAI is launching proper search integration, by far the best open weights model got released, we got an improved MidJourney 6.1, and that's all in the last two weeks. Whether or not GPT-5-level models get here in 2024, and whether or not it arrives on a given schedule, make no mistake. It's happening. This week also had a lot of discourse and events around SB 1047 that I failed to avoid, resulting in not one but four sections devoted to it. Dan Hendrycks was baselessly attacked - by billionaires with massive conflicts of interest that they admit are driving their actions - as having a conflict of interest because he had advisor shares in an evals startup rather than having earned the millions he could have easily earned building AI capabilities. so Dan gave up those advisor shares, for no compensation, to remove all doubt. Timothy Lee gave us what is clearly the best skeptical take on SB 1047 so far. And Anthropic sent a 'support if amended' letter on the bill, with some curious details. This was all while we are on the cusp of the final opportunity for the bill to be revised - so my guess is I will soon have a post going over whatever the final version turns out to be and presenting closing arguments. Meanwhile Sam Altman tried to reframe broken promises while writing a jingoistic op-ed in the Washington Post, but says he is going to do some good things too. And much more. Oh, and also AB 3211 unanimously passed the California assembly, and would effectively among other things ban all existing LLMs. I presume we're not crazy enough to let it pass, but I made a detailed analysis to help make sure of it. Table of Contents 1. Introduction. 2. Table of Contents. 3. Language Models Offer Mundane Utility. They're just not that into you. 4. Language Models Don't Offer Mundane Utility. Baba is you and deeply confused. 5. Math is Easier. Google DeepMind claims an IMO silver metal, mostly. 6. Llama Llama Any Good. The rankings are in as are a few use cases. 7. Search for the GPT. Alpha tests begin of SearchGPT, which is what you think it is. 8. Tech Company Will Use Your Data to Train Its AIs. Unless you opt out. Again. 9. Fun With Image Generation. MidJourney 6.1 is available. 10. Deepfaketown and Botpocalypse Soon. Supply rises to match existing demand. 11. The Art of the Jailbreak. A YouTube video that (for now) jailbreaks GPT-4o-voice. 12. Janus on the 405. High weirdness continues behind the scenes. 13. They Took Our Jobs. If that is even possible. 14. Get Involved. Akrose has listings, OpenPhil has a RFP, US AISI is hiring. 15. Introducing. A friend in venture capital is a friend indeed. 16. In Other AI News. Projections of when it's incrementally happening. 17. Quiet Speculations. Reports of OpenAI's imminent demise, except, um, no. 18. The Quest for Sane Regulations. Nick Whitaker has some remarkably good ideas. 19. Death and or Taxes. A little window into insane American anti-innovation policy. 20. SB 1047 (1). The ultimate answer to the baseless attacks on Dan Hendrycks. 21. SB 1047 (2). Timothy Lee analyzes current version of SB 1047, has concerns. 22. SB 1047 (3): Oh Anthropic. They wrote themselves an unexpected letter. 23. What Anthropic's Letter Actually Proposes. Number three may surprise you. 24. Open Weights Are Unsafe And Nothing Can Fix This. Who wants to ban what? 25. The Week in Audio. Vitalik Buterin, Kelsey Piper, Patrick McKenzie. 26. Rheto...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: RTFB: California's AB 3211, published by Zvi on July 30, 2024 on LessWrong. Some in the tech industry decided now was the time to raise alarm about AB 3211. As Dean Ball points out, there's a lot of bills out there. One must do triage. Dean Ball: But SB 1047 is far from the only AI bill worth discussing. It's not even the only one of the dozens of AI bills in California worth discussing. Let's talk about AB 3211, the California Provenance, Authenticity, and Watermarking Standards Act, written by Assemblymember Buffy Wicks, who represents the East Bay. SB 1047 is a carefully written bill that tries to maximize benefits and minimize costs. You can still quite reasonably disagree with the aims, philosophy or premise of the bill, or its execution details, and thus think its costs exceed its benefits. When people claim SB 1047 is made of crazy pills, they are attacking provisions not in the bill. That is not how it usually goes. Most bills involving tech regulation that come before state legislatures are made of crazy pills, written by people in over their heads. There are people whose full time job is essentially pointing out the latest bill that might break the internet in various ways, over and over, forever. They do a great and necessary service, and I do my best to forgive them the occasional false alarm. They deal with idiots, with bulls in China shops, on the daily. I rarely get the sense these noble warriors are having any fun. AB 3211 unanimously passed the California assembly, and I started seeing bold claims about how bad it would be. Here was one of the more measured and detailed ones. Dean Ball: The bill also requires every generative AI system to maintain a database with digital fingerprints for "any piece of potentially deceptive content" it produces. This would be a significant burden for the creator of any AI system. And it seems flatly impossible for the creators of open weight models to comply. Under AB 3211, a chatbot would have to notify the user that it is a chatbot at the start of every conversation. The user would have to acknowledge this before the conversation could begin. In other words, AB 3211 could create the AI version of those annoying cookie notifications you get every time you visit a European website. … AB 3211 mandates "maximally indelible watermarks," which it defines as "a watermark that is designed to be as difficult to remove as possible using state-of-the-art techniques and relevant industry standards." So I decided to Read the Bill (RTFB). It's a bad bill, sir. A stunningly terrible bill. How did it unanimously pass the California assembly? My current model is: 1. There are some committee chairs and others that can veto procedural progress. 2. Most of the members will vote for pretty much anything. 3. They are counting on Newsom to evaluate and if needed veto. 4. So California only sort of has a functioning legislative branch, at best. 5. Thus when bills pass like this, it means a lot less than you might think. Yet everyone stays there, despite everything. There really is a lot of ruin in that state. Time to read the bill. Read The Bill (RTFB) It's short - the bottom half of the page is all deleted text. Section 1 is rhetorical declarations. GenAI can produce inauthentic images, they need to be clearly disclosed and labeled, or various bad things could happen. That sounds like a job for California, which should require creators to provide tools and platforms to provide labels. So we all can remain 'safe and informed.' Oh no. Section 2 22949.90 provides some definitions. Most are standard. These aren't: (c) "Authentic content" means images, videos, audio, or text created by human beings without any modifications or with only minor modifications that do not lead to significant changes to the perceived contents or meaning of the cont...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI #74: GPT-4o Mini Me and Llama 3, published by Zvi on July 26, 2024 on LessWrong. We got two big model releases this week. GPT-4o Mini is covered here. Llama 3.1-405B (and 70B and 8B) is mostly covered in yesterday's post, this has some follow up. Table of Contents 1. Introduction. 2. Table of Contents. 3. Language Models Offer Mundane Utility. All your coding are belong to us. 4. Language Models Don't Offer Mundane Utility. Math is hard. Can be expensive. 5. GPT-4o Mini Me. You complete me at lower than usual cost. 6. Additional Llama-3.1 Notes. Pricing information, and more rhetoric. 7. Fun With Image Generation. If you're confused why artists are so upset. 8. Deepfaketown and Botpocalypse Soon. Not surprises. 9. They Took Our Jobs. Layoffs at Activision and across gaming. 10. In Other AI News. New benchmarks, new chip variants, and more. 11. The Art of the Jailbreak. Pliny remains undefeated. 12. Quiet Speculations. Where will the utility be coming from? 13. The Quest for Sane Regulations. Public opinion continues to be consistent. 14. Openly Evil AI. Some Senators have good questions. 15. The Week in Audio. Dwarkesh in reverse, and lots of other stuff. Odd Lots too. 16. Rhetorical Innovation. What are corporations exactly? 17. Aligning a Smarter Than Human Intelligence is Difficult. So are evals. 18. People Are Worried About AI Killing Everyone. Roon warns you to beware. 19. The Sacred Timeline. Hype? 20. Other People Are Not As Worried About AI Killing Everyone. Older Joe Rogan. 21. The Lighter Side. It's on. Language Models Offer Mundane Utility Coding is seriously much faster now, and this is the slowest it will ever be. Roon: pov: you are ten months from working for claude sonnet the new technical founder. Garry Tan: Underrated trend. It's happening. Sully: 50% of our code base was written entirely by LLMs expect this to be ~80% by next year With sonnet we're shipping so fast, it feels like we tripled headcount overnight Not using Claude 3.5 to code? Expect to be crushed by teams who do (us). Not only coding, either. Jimmy (QTing Tan): It can also do hardware related things quite well too, and legal, and logistics (planning) and compliance even. I've been able to put off hiring for months. When I run out of sonnet usage I patch in gpt-4o, it's obviously and notably worse which I why I rarely use it as a primary anymore. Claude 3.5 Sonnet becomes the first AI to crush the Lem Test to 'write an impossible poem.' Laugh all you want, this is actually great. Kache: dude hahahahahah i used so many tokens today on just formatting json logs near: the just stop oil people are gonna come and spray paint you now Compared to how much carbon a human coder would have used? Huge improvement. Language Models Don't Offer Mundane Utility IMO problems are still mostly too hard. The linked one, which GPT-4, GPT-4o and Claude 3.5 Sonnet failed on, seems unusually easy? Although a math Olympiad solver does, predictably given the contests we've seen. [EDIT: I didn't read this properly, but a reader points out this is the floor symbol, which means what I thought was an obvious proof doesn't actually answer the question, although it happens to get the right answer. Reader says the answers provided would actually also get 0/7, order has been restored]. Figure out what song Aella was talking about here. Found the obvious wrong answer. Grok offers to tell you 'more about this account.' I haven't seen the button yet, probably it is still experimental. Our price cheap. Llama 3.1-405B was a steal in terms of compute costs. Seconds: "AI is expensive" its not even half the cost of a middling marvel movie. Teortaxes: Pretty insane that the cost of producing llama-3-405B, this behemoth, is like 40% of *Ant-Man and the Wasp: Quantumania* movie at most If I were Zuck, I'd have open sourced a $...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Llama Llama-3-405B?, published by Zvi on July 25, 2024 on LessWrong. It's here. The horse has left the barn. Llama-3.1-405B, and also Llama-3.1-70B and Llama-3.1-8B, have been released, and are now open weights. Early indications are that these are very good models. They were likely the best open weight models of their respective sizes at time of release. Zuckerberg claims that open weights models are now competitive with closed models. Yann LeCun says 'performance is on par with the best closed models.' This is closer to true than in the past, and as corporate hype I will essentially allow it, but it looks like this is not yet fully true. Llama-3.1-405B not as good as GPT-4o or Claude Sonnet. Certainly Llama-3.1-70B is not as good as the similarly sized Claude Sonnet. If you are going to straight up use an API or chat interface, there seems to be little reason to use Llama. That is a preliminary result. It is still early, and there has been relatively little feedback. But what feedback I have seen is consistent on this. Prediction markets are modestly more optimistic. This market still has it 29% to be the #1 model on Arena, which seems unlikely given Meta's own results. Another market has it 74% to beat GPT-4-Turbo-2024-04-09, which currently is in 5th position. That is a big chance for it to land in a narrow window between 1257 and 1287. This market affirms that directly on tiny volume. Such open models like Llama-3.1-405B are of course still useful even if a chatbot user would have better options. There are cost advantages, privacy advantages and freedom of action advantages to not going through OpenAI or Anthropic or Google. In particular, if you want to distill or fine-tune a new model, and especially if you want to fully own the results, Llama-3-405B is here to help you, and Llama-3-70B and 8B are here as potential jumping off points. I expect this to be the main practical effect this time around. If you want to do other things that you can't do with the closed options? Well, technically you can't do most of them under Meta's conditions either, but there is no reason to expect that will stop people, especially those overseas including in China. For some of these uses that's a good thing. Others, not as good. Zuckerberg also used the moment to offer a standard issue open source manifesto, in which he abandons any sense of balance and goes all-in, which he affirmed in a softball interview with Rowan Cheung. On the safety front, while I do not think they did their safety testing in a way that would have caught issues if there had been issues, my assumption is there was nothing to catch. The capabilities are not that dangerous at this time. Thus I do not predict anything especially bad will happen here. I expect the direct impact of Llama-3.1-405B to be positive, with the downsides remaining mundane and relatively minor. The only exception would be the extent to which this enables the development of future models. I worry that this differentially accelerates and enables our rivals and enemies and hurts our national security, and indeed that this will be its largest impact. And I worry more that this kind of action and rhetoric will lead us down the path where if things get dangerous in the future, it will become increasingly hard not to get ourselves into deep trouble, both in terms of models being irrevocably opened up when they shouldn't be and increasing pressure on everyone else to proceed even when things are not safe, up to and including loss of control and other existential risks. If Zuckerberg had affirmed a reasonable policy going forward but thought the line could be drawn farther down the line, I would have said this was all net good. Instead, I am dismayed. I do get into the arguments about open weights at the end of this post, because it felt obligato...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Monthly Roundup #20: July 2024, published by Zvi on July 24, 2024 on LessWrong. It is monthly roundup time. I invite readers who want to hang out and get lunch in NYC later this week to come on Thursday at Bhatti Indian Grill (27th and Lexington) at noon. I plan to cover the UBI study in its own post soon. I cover Nate Silver's evisceration of the 538 presidential election model, because we cover probabilistic modeling and prediction markets here, but excluding any AI discussions I will continue to do my best to stay out of the actual politics. Bad News Jeff Bezos' rocket company Blue Origin files comment suggesting SpaceX Starship launches be capped due to 'impact on local environment.' This is a rather shameful thing for them to be doing, and not for the first time. Alexey Guzey reverses course, realizes at 26 that he was a naive idiot at 20 and finds everything he wrote cringe and everything he did incompetent and Obama was too young. Except, no? None of that? Young Alexey did indeed, as he notes, successfully fund a bunch of science and inspire good thoughts and he stands by most of his work. Alas, now he is insufficiently confident to keep doing it and is in his words 'terrified of old people.' I think Alexey's success came exactly because he saw people acting stupid and crazy and systems not working and did not then think 'oh these old people must have their reasons,' he instead said that's stupid and crazy. Or he didn't even notice that things were so stupid and crazy and tried to just… do stuff. When I look back on the things I did when I was young and foolish and did not know any better, yeah, some huge mistakes, but also tons that would never have worked if I had known better. Also, frankly, Alexey is failing to understand (as he is still only 26) how much cognitive and physical decline hits you, and how early. Your experience and wisdom and increased efficiency is fighting your decreasing clock speed and endurance and physical strength and an increasing set of problems. I could not, back then, have done what I am doing now. But I also could not, now, do what I did then, even if I lacked my current responsibilities. For example, by the end of the first day of a Magic tournament I am now completely wiped. Google short urls are going to stop working. Patrick McKenzie suggests prediction markets on whether various Google services will survive. I'd do it if I was less lazy. Silver Bullet This is moot in some ways now that Biden has dropped out, but being wrong on the internet is always relevant when it impacts our epistemics and future models. Nate Silver, who now writes Silver Bulletin and runs what used to be the old actually good 538 model, eviscerates the new 538 election model. The 'new 538' model had Biden projected to do better in Wisconsin and Ohio than either the fundamentals or his polls, which makes zero sense. It places very little weight on polls, which makes no sense. It has moved towards Biden recently, which makes even less sense. Texas is their third most likely tipping point state, it happens 9.8% of the time, wait what? At best, Kelsey Piper's description here is accurate. Kelsey Piper: Nate Silver is slightly too polite to say it but my takeaway from his thoughtful post is that the 538 model is not usefully distinguishable from a rock with "incumbents win reelection more often than not" painted on it. Gil: worse, I think Elliott's modelling approach is probably something like max_(dem_chance) [incumbency advantage, polls, various other approaches]. Elliott's model in 2020 was more bullish on Biden's chances than Nate and in that case Trump was the incumbent and down in the polls. Nate Silver (on Twitter): Sure, the Titanic might seem like it's capsizing, but what you don't understand is that the White Star Line has an extremely good track re...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: On the CrowdStrike Incident, published by Zvi on July 22, 2024 on LessWrong. Things went very wrong on Friday. A bugged CrowdStrike update temporarily bricked quite a lot of computers, bringing down such fun things as airlines, hospitals and 911 services. It was serious out there. Ryan Peterson: Crowdstrike outage has forced Starbucks to start writing your name on a cup in marker again and I like it. What (Technically) Happened My understanding it was a rather stupid bug, a NULL pointer from the memory unsafe C++ language. Zack Vorhies: Memory in your computer is laid out as one giant array of numbers. We represent these numbers here as hexadecimal, which is base 16 (hexadecimal) because it's easier to work with… for reasons. The problem area? The computer tried to read memory address 0x9c (aka 156). Why is this bad? This is an invalid region of memory for any program. Any program that tries to read from this region WILL IMMEDIATELY GET KILLED BY WINDOWS. So why is memory address 0x9c trying to be read from? Well because… programmer error. It turns out that C++, the language crowdstrike is using, likes to use address 0x0 as a special value to mean "there's nothing here", don't try to access it or you'll die. … And what's bad about this is that this is a special program called a system driver, which has PRIVLIDGED access to the computer. So the operating system is forced to, out of an abundance of caution, crash immediately. This is what is causing the blue screen of death. A computer can recover from a crash in non-privileged code by simply terminating the program, but not a system driver. When your computer crashes, 95% of the time it's because it's a crash in the system drivers. If the programmer had done a check for NULL, or if they used modern tooling that checks these sorts of things, it could have been caught. But somehow it made it into production and then got pushed as a forced update by Crowdstrike… OOPS! Here is another technical breakdown. A non technical breakdown would be: 1. CrowdStrike is set up to run whenever you start the computer. 2. Then someone pushed an update to a ton of computers. 3. Which is something CrowdStrike was authorized to do. 4. The update contained a stupid bug, that would have been caught if those involved had used standard practices and tests. 5. With the bug, it tries to access memory in a way that causes a crash. 6. Which also crashes the computer. 7. So you have to do a manual fix to each computer to get around this. 8. If this had been malicious it could probably have permawiped all the computers, or inserted Trojans, or other neat stuff like that. 9. So we dodged a bullet. 10. Also, your AI safety plan needs to take into account that this was the level of security mindset and caution at CrowdStrike, despite CrowdStrike having this level of access and being explicitly in the security mindset business, and that they were given this level of access to billions of computers, and that their stock was only down 11% on the day so they probably keep most of that access and we aren't going to fine them out of existence either. Yep. Who to Blame? George Kurtz (CEO CrowdStrike): CrowdStrike is actively working with customers impacted by a defect found in a single content update for Windows hosts. Mac and Linux hosts are not impacted. This is not a security incident or cyberattack. The issue has been identified, isolated and a fix has been deployed. We refer customers to the support portal for the latest updates and will continue to provide complete and continuous updates on our website. We further recommend organizations ensure they're communicating with CrowdStrike representatives through official channels. Our team is fully mobilized to ensure the security and stability of CrowdStrike customers. Dan Elton: No apology. Many people have...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI #73: Openly Evil AI, published by Zvi on July 18, 2024 on LessWrong. What do you call a clause explicitly saying that you waive the right to whistleblower compensation, and that you need to get permission before sharing information with government regulators like the SEC? I have many answers. I also know that OpenAI, having f***ed around, seems poised to find out, because that is the claim made by whistleblowers to the SEC. Given the SEC fines you for merely not making an explicit exception to your NDA for whistleblowers, what will they do once aware of explicit clauses going the other way? (Unless, of course, the complaint is factually wrong, but that seems unlikely.) We also have rather a lot of tech people coming out in support of Trump. I go into the reasons why, which I do think is worth considering. There is a mix of explanations, and at least one very good reason. Then I also got suckered into responding to a few new (well, not really new, but renewed) disingenuous attacks on SB 1047. The entire strategy is to be loud and hyperbolic, especially on Twitter, and either hallucinate or fabricate a different bill with different consequences to attack, or simply misrepresent how the law works, then use that, to create the illusion the bill is unliked or harmful. Few others respond to correct such claims, and I constantly worry that the strategy might actually work. But that does not mean you, my reader who already knows, need to read all that. Also a bunch of fun smaller developments. Karpathy is in the AI education business. Table of Contents 1. Introduction. 2. Table of Contents. 3. Language Models Offer Mundane Utility. Fight the insurance company. 4. Language Models Don't Offer Mundane Utility. Have you tried using it? 5. Clauding Along. Not that many people are switching over. 6. Fun With Image Generation. Amazon Music and K-Pop start to embrace AI. 7. Deepfaketown and Botpocalypse Soon. FoxVox, turn Fox into Vox or Vox into Fox. 8. They Took Our Jobs. Take away one haggling job, create another haggling job. 9. Get Involved. OpenPhil request for proposals. Job openings elsewhere. 10. Introducing. Karpathy goes into AI education. 11. In Other AI News. OpenAI's Q* is now named Strawberry. Is it happening? 12. Denying the Future. Projects of the future that think AI will never improve again. 13. Quiet Speculations. How to think about stages of AI capabilities. 14. The Quest for Sane Regulations. EU, UK, The Public. 15. The Other Quest Regarding Regulations. Many in tech embrace The Donald. 16. SB 1047 Opposition Watch (1). I'm sorry. You don't have to read this. 17. SB 1047 Opposition Watch (2). I'm sorry. You don't have to read this. 18. Open Weights are Unsafe and Nothing Can Fix This. What to do about it? 19. The Week in Audio. Joe Rogan talked to Sam Altman and I'd missed it. 20. Rhetorical Innovation. Supervillains, oh no. 21. Oh Anthropic. More details available, things not as bad as they look. 22. Openly Evil AI. Other things, in other places, on the other hand, look worse. 23. Aligning a Smarter Than Human Intelligence is Difficult. Noble attempts. 24. People Are Worried About AI Killing Everyone. Scott Adams? Kind of? 25. Other People Are Not As Worried About AI Killing Everyone. All glory to it. 26. The Lighter Side. A different kind of mental gymnastics. Language Models Offer Mundane Utility Let Claude write your prompts for you. He suggests using the Claude prompt improver. Sully: convinced that we are all really bad at writing prompts I'm personally never writing prompts by hand again Claude is just too good - managed to feed it evals and it just optimized for me Probably a crude version of dspy but insane how much prompting can make a difference. Predict who will be the shooting victim. A machine learning model did this for citizens of Chicago (a ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI #72: Denying the Future, published by Zvi on July 12, 2024 on LessWrong. The Future. It is coming. A surprising number of economists deny this when it comes to AI. Not only do they deny the future that lies in the future. They also deny the future that is here, but which is unevenly distributed. Their predictions and projections do not factor in even what the AI can already do, let alone what it will learn to do later on. Another likely future event is the repeal of the Biden Executive Order. That repeal is part of the Republican platform, and Trump is the favorite to win the election. We must act on the assumption that the order likely will be repealed, with no expectation of similar principles being enshrined in federal law. Then there are the other core problems we will have to solve, and other less core problems such as what to do about AI companions. They make people feel less lonely over a week, but what do they do over a lifetime? Also I don't have that much to say about it now, but it is worth noting that this week it was revealed Apple was going to get an observer board seat at OpenAI… and then both Apple and Microsoft gave up their observer seats. Presumably that is about antitrust and worrying the seats would be a bad look. There could also be more to it. Table of Contents 1. Introduction. 2. Table of Contents. 3. Language Models Offer Mundane Utility. Long as you avoid GPT-3.5. 4. Language Models Don't Offer Mundane Utility. Many mistakes will not be caught. 5. You're a Nudge. You say it's for my own good. 6. Fun With Image Generation. Universal control net for SDXL. 7. Deepfaketown and Botpocalypse Soon. Owner of a lonely bot. 8. They Took Our Jobs. Restaurants. 9. Get Involved. But not in that way. 10. Introducing. Anthropic ships several new features. 11. In Other AI News. Microsoft and Apple give up OpenAI board observer seats. 12. Quiet Speculations. As other papers learned, to keep pace, you must move fast. 13. The AI Denialist Economists. Why doubt only the future? Doubt the present too. 14. The Quest for Sane Regulation. EU and FTC decide that things are their business. 15. Trump Would Repeal the Biden Executive Order on AI. We can't rely on it. 16. Ordinary Americans Are Worried About AI. Every poll says the same thing. 17. The Week in Audio. Carl Shulman on 80,000 hours was a two parter. 18. The Wikipedia War. One obsessed man can do quite a lot of damage. 19. Rhetorical Innovation. Yoshua Bengio gives a strong effort. 20. Evaluations Must Mimic Relevant Conditions. Too often they don't. 21. Aligning a Smarter Than Human Intelligence is Difficult. Stealth fine tuning. 22. The Problem. If we want to survive, it must be solved. 23. Oh Anthropic. Non Disparagement agreements should not be covered by NDAs. 24. Other People Are Not As Worried About AI Killing Everyone. Don't feel the AGI. Language Models Offer Mundane Utility Yes, they are highly useful for coding. It turns out that if you use GPT-3.5 for your 'can ChatGPT code well enough' paper, your results are not going to be relevant. Gallabytes says 'that's morally fraud imho' and that seems at least reasonable. Tests failing in GPT-3.5 is the AI equivalent of "IN MICE" except for IQ tests. If you are going to analyze the state of AI, you need to keep an eye out for basic errors and always always check which model is used. So if you go quoting statements such as: Paper about GPT-3.5: its ability to generate functional code for 'hard' problems dropped from 40% to 0.66% after this time as well. 'A reasonable hypothesis for why ChatGPT can do better with algorithm problems before 2021 is that these problems are frequently seen in the training dataset Then even if you hadn't realized or checked before (which you really should have), you need to notice that this says 2021, which is very much not ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Medical Roundup #3, published by Zvi on July 9, 2024 on LessWrong. This time around, we cover the Hanson/Alexander debates on the value of medicine, and otherwise we mostly have good news. Technology Advances Regeneron administers a single shot in a genetically deaf child's ear, and they can hear after a few months, n=2 so far. Great news: An mRNA vaccine in early human clinical trials reprograms the immune system to attack glioblastoma, the most aggressive and lethal brain tumor. It will now proceed to Phase I. In a saner world, people would be able to try this now. More great news, we have a cancer vaccine trial in the UK. And we're testing personalized mRNA BioNTech canner vaccines too. US paying Moderna $176 million to develop a pandemic vaccine against bird flu. We also have this claim that Lorlatinib jumps cancer PFS rates from 8% to 60%. The GLP-1 Revolution Early results from a study show the GLP-1 drug liraglutide could reduce cravings in people with opioid use disorder by 40% compared with a placebo. This seems like a clear case where no reasonable person would wait for more than we already have? If there was someone I cared about who had an opioid problem I would do what it took to get them on a GLP-1 drug. Rumblings that GLP-1 drugs might improve fertility? Rumblings that GLP-1 drugs could reduce heart attack, stroke and death even if you don't lose weight, according to a new analysis? Survey says 6% of Americans might already be on them. Weight loss in studies continues for more than a year in a majority of patients, sustained up to four years, which is what they studied so far. The case that GLP-1s can be sued against all addictions at scale. It gives users a sense of control which reduces addictive behaviors across the board, including acting as a 'vaccine' against developing new addictions. It can be additive to existing treatments. More alcoholics (as an example) already take GLP-1s than existing indicated anti-addiction medications, and a study showed 50%-56% reduction in risk of new or recurring alcohol addictions, another showed 30%-50% reduction for cannabis. How to cover this? Sigh. I do appreciate the especially clean example below. Matthew Yglesias: Conservatives more than liberals will see the systematic negativity bias at work in coverage of GLP-agonists. Less likely to admit that this same dynamic colors everything including coverage of crime and the economy. The situation is that there is a new drug that is helping people without hurting anyone, so they write an article about how it is increasing 'health disparities.' The point is that they are writing similar things for everything else, too. The Free Press's Bari Weiss and Johann Hari do a second round of 'Ozempic good or bad.' It takes a while for Hari to get to actual potential downsides. The first is a claimed (but highly disputed) 50%-75% increased risk of thyroid cancer. That's not great, but clearly overwhelmed by reduced risks elsewhere. The second is the worry of what else it is doing to your brain. Others have noticed it might be actively great here, giving people more impulse control, helping with things like smoking or gambling. Hari worries it might hurt his motivation for writing or sex. That seems like the kind of thing one can measure, both in general and in yourself. If people were losing motivation to do work, and this hurt productivity, we would know. The main objection seems to be that obesity is a moral failure of our civilization and ourselves, so it would be wrong to fix it with a pill rather than correct the underlying issues like processed foods and lack of exercise. Why not be like Japan? To which the most obvious response is that it is way too late for America to take that path. That does not mean that people should suffer. And if we find a way to fix the issues ...

Study Guide Bava Metzia 111 This month's learning is sponsored by Shifra Tyberg and Rephael Wenger in loving memory of Zvi ben Yisrael Yitzhak Tyberg on his yahrzeit, and in honor of their daughter Ayelet's upcoming marriage to Ori Kinberg.  Today's daf is sponsored by Nina Black in loving memory of her mother, Sophie Black, Sophia bat Avram, z"l whose yahrzeit is today. "Mom was a committed Jew, a deep thinker, a lover of learning and would be happy that I took on the commitment to do Daf Yomi. I miss her every day." What is the time frame in which one must pay one's worker? It depends on whether the person was hired for work during the day or at night and whether the job was for the day or per hour. There are five negative prohibitions associated with delaying of payment for salary and one positive commandment. Do the same rules apply for a rental payment for one who rents animals or vessels? Do they apply to a ger toshav (one who keeps the seven Noachide commandments)? Three tannaitic opinions are brought relating to these two questions which is based on different approaches to extrapolating the verses in Devarim 24:14-15 and Vayikra 19:13.

Many of you will have heard of Zvi Mowshowitz as a superhuman information-absorbing-and-processing machine — which he definitely is. As the author of the Substack Don't Worry About the Vase, Zvi has spent as much time as literally anyone in the world over the last two years tracking in detail how the explosion of AI has been playing out — and he has strong opinions about almost every aspect of it. Links to learn more, summary, and full transcript.In today's episode, host Rob Wiblin asks Zvi for his takes on:US-China negotiationsWhether AI progress has stalledThe biggest wins and losses for alignment in 2023EU and White House AI regulationsWhich major AI lab has the best safety strategyThe pros and cons of the Pause AI movementRecent breakthroughs in capabilitiesIn what situations it's morally acceptable to work at AI labsWhether you agree or disagree with his views, Zvi is super informed and brimming with concrete details.Zvi and Rob also talk about:The risk of AI labs fooling themselves into believing their alignment plans are working when they may not be.The “sleeper agent” issue uncovered in a recent Anthropic paper, and how it shows us how hard alignment actually is.Why Zvi disagrees with 80,000 Hours' advice about gaining career capital to have a positive impact.Zvi's project to identify the most strikingly horrible and neglected policy failures in the US, and how Zvi founded a new think tank (Balsa Research) to identify innovative solutions to overthrow the horrible status quo in areas like domestic shipping, environmental reviews, and housing supply.Why Zvi thinks that improving people's prosperity and housing can make them care more about existential risks like AI.An idea from the online rationality community that Zvi thinks is really underrated and more people should have heard of: simulacra levels.And plenty more.Chapters:Zvi's AI-related worldview (00:03:41)Sleeper agents (00:05:55)Safety plans of the three major labs (00:21:47)Misalignment vs misuse vs structural issues (00:50:00)Should concerned people work at AI labs? (00:55:45)Pause AI campaign (01:30:16)Has progress on useful AI products stalled? (01:38:03)White House executive order and US politics (01:42:09)Reasons for AI policy optimism (01:56:38)Zvi's day-to-day (02:09:47)Big wins and losses on safety and alignment in 2023 (02:12:29)Other unappreciated technical breakthroughs (02:17:54)Concrete things we can do to mitigate risks (02:31:19)Balsa Research and the Jones Act (02:34:40)The National Environmental Policy Act (02:50:36)Housing policy (02:59:59)Underrated rationalist worldviews (03:16:22)Producer and editor: Keiran HarrisAudio Engineering Lead: Ben CordellTechnical editing: Simon Monsour, Milo McGuire, and Dominic ArmstrongTranscriptions and additional content editing: Katy Moore