POPULARITY
Eneasz and Steven are at LessOnline/SummerCamp and Steven was lucky enough to attend a short presentation by Damon (Daystar Eld) and Ivy on How to Become a Romantic. The presentation was too short for Steven’s satisfaction, so we found a time slot to record a (too short) episode. LINKS Damon/Daystar Eld Ivy’s X handle and website Guild of the Rose Paid Bonus content for the week – Full Video Our Patreon, or if you prefer Our SubStack Hey look, we have a discord! What could possibly go wrong? We now partner with The Guild of the Rose, check them out. LessWrong Sequence Posts Discussed in this Episode: on hiatus. probably returning someday.
Our values are robust against Moloch because they were forged in his flames. LINKS When does competition lead to recognisable values? Our episode 18 – Robin Hanson Interview Chex Quest FarmKind – Animal Welfare Offsets (matching code “cagefree”) Paid Bonus content for the week – Preshow Chat, Full Video 00:03:02 – A Good Post-AGI Future 00:15:07 – Human Values Were Forged in Malthusian Competition 00:52:57 – Feedback 01:24:17 – Guild of the Rose 01:25:23 – Thank the Supporter! Our Patreon, or if you prefer Our SubStack Hey look, we have a discord! What could possibly go wrong? We now partner with The Guild of the Rose, check them out. LessWrong Sequence Posts Discussed in this Episode: on hiatus. maybe returning someday.
In Episode 258 we talked about EA Veganism for a bit… and that bit was not long enough. So here we go again! Fill out Steven’s 3-question survey! LINKS Our episode 258 – How Effective Altruism Has Evolved EA Vegan Advocacy is not truthseeking, and it's everyone's problem by Elizabeth Vegan Health Advice by Ozy Dairy cows make their misery expensive (but their calves can't) by Elizabeth Being John Rawls by Scott Alexander – audio of Being John Rawls WestWorld Should Anyone Be Vegan by Henry Stanley Does your AI perform badly because you — you, specifically — are a bad person by Natalie Cargill FarmKind – Animal Welfare Offsets Plzdontkillus The Mind Killer Ep 69 NFT Paid Bonus content for the week – Preshow Chat, Full Video 00:00:42 – Feedback 00:15:14 – Eating Delicious Animals 01:38:09 – Westworld & Violent Video Games 01:56:35 – Guild of the Rose 01:57:16 – Thank the Supporter! Our Patreon, or if you prefer Our SubStack Hey look, we have a discord! What could possibly go wrong? We now partner with The Guild of the Rose, check them out. LessWrong Sequence Posts Discussed in this Episode: on hiatus. maybe returning someday.
The stupid red-button/blue-button discourse came back. Eneasz thought he could sit out this round until Aella went Chaos God with her social media app, Glosso. Fill out our new 2-question survey! LINKS PlzDontKillUs Inkhaven Presents podcast The Knot webfic The Mind Reviver Glosso Stupid Tim Urban with his stupid button-resurrection oh noes, Anthropic employees gonna give money to EA stuff” article Paid Bonus content for the week – Video 00:00:05 – PlzDontKillUs & Feedback 00:30:14 – Stupid Glosso Buttons 01:33:34 – Guild of the Rose 01:34:50 – Thank the Supporter! Our Patreon, or if you prefer Our SubStack Hey look, we have a discord! What could possibly go wrong? We now partner with The Guild of the Rose, check them out. LessWrong Sequence Posts Discussed in this Episode: on hiatus, returning someday
When reading comments, you see is what other people think before reading the comment. As shown in an RCT, that information anchors your opinion, reducing your ability to form your own opinion and making the site's karma rankings less related to the comment's true value. I think the problem is fixable and float some ideas for consideration. The LessWrong interface prioritizes social information You read a comment. What information is presented, and in what order? The order of information: Who wrote the comment (in bold);How much other people like this comment (as shown by the karma indicator);How much other people agree with this comment (as shown by the agreement score);The actual content. This is unwise design for a website which emphasizes truth-seeking. You don't have a chance to read the comment and form your own opinion first. However, you can opt in to hiding usernames (until moused over) via your account settings page. A 2013 RCT supports the upvote-anchoring concern From Social Influence Bias: A Randomized Experiment (Muchnik et al., 2013):[1] We therefore designed and analyzed a large-scale randomized experiment on a social news aggregation Web site to investigate whether knowledge of such aggregates [...] ---Outline:(00:30) The LessWrong interface prioritizes social information[... 6 more sections]--- First published: April 27th, 2026 Source: https://www.lesswrong.com/posts/YSsp9x8qrBucLoiWT/lesswrong-shows-you-social-signals-before-the-comment --- Narrated by TYPE III AUDIO. ---Images from the article:
Around 10 years ago, a paper came out that arguably killed classical deep learning theory: Zhang et al. 's aptly titled Understanding deep learning requires rethinking generalization. Of course, this is a bit of an exaggeration. No single paper ever kills a field of research on its own, and deep learning theory was not exactly the most productive and healthy field at the time this was published. But if I had to point to a single paper that shattered the feeling of optimism at the time, it would be Zhang et al. 2016.[1] Caption: believe it or not, this unassuming table rocked the field of deep learning theory back in 2016, despite probably involving fewer computational resources than what Claude 4.7 Opus consumed when I clicked the “Claude” button embedded into the LessWrong editor. — Let's start by answering a question: what, exactly, do I mean by deep learning theory? At least in 2016, the answer was: “extending statistical learning theory to deep neural networks trained with SGD, in order to derive generalization bounds that would explain their behavior in practice”. — Since its conception in the mid 1980s, statistical learning theory had been the dominant approach for [...] The original text contained 2 footnotes which were omitted from this narration. --- First published: April 25th, 2026 Source: https://www.lesswrong.com/posts/ZvQfcLbcNHYqmvWyo/the-paper-that-killed-deep-learning-theory --- Narrated by TYPE III AUDIO. ---Images from the article:
I use AI assistance for basically all of my work, for many hours, every day. My colleagues do the same. Recent surveys suggest >50% of Americans have used AI to help with their work in the last week. My architect recently started sending me emails that were clearly ChatGPT generated.[1] Despite that, I know surprisingly little about how other people use AI assitance. Or at least how people who aren't weird AI-influencers sharing their marketing courses on Twitter or LinkedIn use AI. So here is a list of 10 concrete times I have used AI in some at least mildly creative ways, and how that went. 1) Transcribe and summarize every conversation spoken in our team office Using an internal Lightcone application called "Omnilog" we have a microphone in our office that records all of our meetings, transcribes them via ElevenLabs, and uses Pyannote.ai for speaker identification. This was a bunch of work and is quite valuable, but probably a bit too annoying for most readers of this post to set up. However, the thing I am successfully using Claude Code to do is take that transcript (which often has substantial transcription and speaker-identification errors), clean it up, summarize [...] ---Outline:(00:50) 1) Transcribe and summarize every conversation spoken in our team office(01:56) 2) Try to automatically fix any simple bugs that anyone on the team has mentioned out loud, or complained about in Slack(03:13) 3) Design 20+ different design variations for nowinners.ai(04:09) 4) Review my LessWrong essays for factual accuracy and argue with me about their central thesis(05:08) 5) Remove unnecessary clauses, sentences, parentheticals and random cruft from my LessWrong posts before publishing(06:23) 6) Pair vibe-coding(08:14) 7) Mass-creating 100+ variations of Suno songs using Claude Cowork desktop control[... 3 more sections]--- First published: April 20th, 2026 Source: https://www.lesswrong.com/posts/bxdwSZYxKmPBres6w/10-non-boring-ways-i-ve-used-ai-in-the-last-month --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try
It's been about four years since Eliezer Yudkowsky published AGI Ruin: A List of Lethalities, a 43-point list of reasons the default outcome from building AGI is everyone dying. A week later, Paul Christiano replied with Where I Agree and Disagree with Eliezer, signing on to about half the list and pushing back on most of the rest. For people who were young and not in the bay area, like me, these essays were probably more significant than old timers would expect. Before it became completely consumed with AI discussions, LessWrong was a forum about the art of human rationality, and most internet rationalists I knew thought of it as a mix between that and a place to write for people who liked the sequences. It wasn't until 2022 that we were exposed to all of the doom arguments in one place, and it was the first time in many years that Eliezer had publicly announced how much more dire his assessment was since the Sequences. As far as I can tell AGI Ruin still remains his most authoritative explanation of his views. It's not often that public intellectuals will literally hand you a document explaining why [...] ---Outline:(02:51) AGI Ruin(02:54) Section A (Setting up the problem)(12:18) Section B.1 (Distributional Shift)(22:16) Section B.2: Central difficulties of outer and inner alignment.(32:21) Section B.3: Central difficulties of sufficiently good and useful transparency / interpretability.(41:29) Section C (What is AI Safety currently doing?)(44:34) Overall Impressions The original text contained 4 footnotes which were omitted from this narration. --- First published: April 19th, 2026 Source: https://www.lesswrong.com/posts/PgJYwnN7fZKipgMz4/reevaluating-agi-ruin-in-2026 --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Epistemic status: Completely schizo galaxy-brained theory Lightcone[1] operates on a "generalist" philosophy. Most of our full-time staff have the title "generalist", and in any given year they work on a wide variety of tasks — from software development on the LessWrong codebase to fixing an overflowing toilet at Lighthaven, our 30,000 sq. ft. campus. One of our core rules is that you should not delegate a task you don't know how to perform yourself. This is a very intense rule and has lots of implications about how we operate, so I've spent a lot of time watching people learn things they didn't previously know how to do. My overall observation (and why we have the rule) is that smart people can learn almost anything. Across a wide range of tasks, most of the variance in performance is explained by general intelligence (foremost) and conscientiousness (secondmost), not expertise. Of course, if you compare yourself to someone who's done a task thousands of times you'll lag behind for a while — but people plateau surprisingly quickly. Having worked with experts across many industries, and having dabbled in the literature around skill transfer and training, there seems to be little difference [...] The original text contained 5 footnotes which were omitted from this narration. --- First published: April 18th, 2026 Source: https://www.lesswrong.com/posts/KRLGxCaqdgrotyB8z/there-are-only-four-skills-design-technical-management-and --- Narrated by TYPE III AUDIO.
(The better telling is here. Seriously you should go read it. I've heard this story told in rationalist circles, but there wasn't a post on LessWrong, so I made one) Today is April 15th, Carpathia Day. Take a moment to put forth an unreasonable effort to save a little piece of your world, when no one would fault you for doing less. In the early morning of April 15, the RMS Titanic began to sink with more than two thousand souls on board. Over 58 nautical miles away — too far to make it in time — sailed the RMS Carpathia, a small, slow, passenger steamer. The wireless operator, Harold Cottam, was listening to the transmitter late at night before he went to bed when he got a message from Cape Cod intended for the Titanic. When he contacted the Titanic to relay the messages, he got back a distress signal saying they hit an iceberg and were in need of immediate assistance. Cottam ran the message straight to the captain's cabin, waking him. Captain Arthur Rostron's first reaction upon being awoken was anger, but that anger dissolved as he came to understand the situation. Before he'd [...] The original text contained 1 footnote which was omitted from this narration. --- First published: April 15th, 2026 Source: https://www.lesswrong.com/posts/SARCiTFJfXJJhpej7/carpathia-day --- Narrated by TYPE III AUDIO.
Epistemic status: All of the western canon must eventually be re-invented in a LessWrong post, so today we are re-inventing modernism. In my post yesterday, I said: Maybe the most important way ambitious, smart, and wise people leave the world worse off than they found it is by seeing correctly how some part of the world is broken and unifying various powers under a banner to fix that problem — only for the thing they have built to slip from their grasp and, in its collapse, destroy much more than anything previously could have. I think many people very reasonably understood me to be giving a general warning against centralization and power-accumulation. While that is where some of my thoughts while writing the post went to, I would like to now expand on its antithesis, both for my own benefit, and for the benefit of the reader who might have been left confused after yesterday's post. The other day I was arguing with Eliezer about a bunch of related thoughts and feelings. In that context, he said to me: From my perspective, my whole life has been, when you raise the banner to oppose the apocalypse, crazy [...] The original text contained 1 footnote which was omitted from this narration. --- First published: April 16th, 2026 Source: https://www.lesswrong.com/posts/w3MJcDueo77D3Ldta/let-goodness-conquer-all-that-it-can-defend --- Narrated by TYPE III AUDIO.
This post was cross posted to LessWrong TL;DR: One of the largest talent gaps in AI safety is competent generalists: program managers, fieldbuilders, operators, org leaders, chiefs of staff, founders. Ambitious, competent junior people could develop the skills to fill these roles, but there are no good pathways for them to gain skills, experience, and credentials. Instead, they're incentivized to pursue legible technical and policy fellowships and then become full-time researchers, even if that's not a good fit for their skills. The ecosystem needs to make generalist careers more legible and accessible. Kairos and Constellation are announcing the Generator Residency as a first step. Apply here by April 27. Epistemic status: Fairly confident, based on 2 years running AI safety talent programs, direct hiring experience, and conversations with ~30 senior org leaders across the ecosystem in the past 6 months.The problem Over the past few years, AI safety has moved from niche concern toward a more mainstream issue, driven by pieces like Situational Awareness, AI 2027, If Anyone Builds It, Everyone Dies, and the rapidly increasing capabilities of the models themselves.During this period, over 20 research fellowships have launched, collectively training thousands of fellows, with 2,000-2,500 fellows [...] ---Outline:(01:18) The problem(03:41) Why the pipeline is broken(05:59) Why this matters now(07:31) Counter-Arguments(10:11) The Generator Residency --- First published: April 13th, 2026 Source: https://forum.effectivealtruism.org/posts/k3nq7FxBCsrNFmAYi/ai-safety-s-biggest-talent-gap-isn-t-researchers-it-s-2 --- Narrated by TYPE III AUDIO.
Epistemic status: All of the western canon must eventually be re-invented in a LessWrong post. So today we are re-inventing federalism. Once upon a time there was a great king. He ruled his kingdom with wisdom and economically literate policies, and prosperity followed. Seeing this, the citizens of nearby kingdoms revolted against their leaders, and organized to join the kingdom of this great king. While the kingdom's ability to defend itself against external threats grew with each person who joined the land, the kingdom's ability to defend itself against internal threats did not. One fateful evening, the king bit into a bologna sandwich poisoned by a rival noble. That noble quickly proceeded to behead his political enemies in the name of the dead king. The flag bearing the wise king's portrait known as "the great unifier" still flies in the fortified cities where his successor rules with an iron fist. Once upon a time there was a great scientific mind. She developed a new theoretical framework that made large advances on the hardest scientific questions of the day. Seeing the promise of her work, new graduate students, professors, and corporate R&D teams flocked into the field, hungry to [...] --- First published: April 15th, 2026 Source: https://www.lesswrong.com/posts/jinzzbPHshif8nmnw/do-not-conquer-what-you-cannot-defend --- Narrated by TYPE III AUDIO.
We discuss how far the right to be wrong goes, a recent intra-rat kerfuffle, and why random violence is bad. And it’s secretly all about AI Doom. Fill out our 2-question survey! LINKS One Week in the Rat Farm Kelsey’s religious liberalism tweet Scott’s tweet in response to Robby from MIRI Scott’s tweet in reply to Ronny from Lightcone Only Law Can Prevent Extinction by Eliezer Yudkowsky Audio version of above Paid Bonus content for the week – Video 00:00:01 – Quick life catch-up 00:06:29 – Right to be Wrong about Doom 01:23:47 – Steven likes Huel a lot 01:25:28 – Guild of the Rose 01:27:50 – Thank the Supporter! Our Patreon, or if you prefer Our SubStack Hey look, we have a discord! What could possibly go wrong? We now partner with The Guild of the Rose, check them out. LessWrong Sequence Posts Discussed in this Episode: on hiatus, returning someday
Michael “Valentine” Smith is a co-founder of CFAR (the Center for Applied Rationality) and the author of influential LessWrong essays including The Hostile Telepaths Problem, Kenshō, and The Intelligent Social Web. He's also been described as “one of the most powerful wizards in the Bay Area.”In this conversation, he explains why your brain creates fog and self-deception to survive social situations, what it actually takes to find clarity, and why "working on yourself" might be the wrong frame entirely. We also do a live coaching demo debugging my own procrastination. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit themetagame.substack.com
Or, for that matter, anything else. This post is meant to be two things: a PSA about LessWrong's current security posture, from a LessWrong admin[1]an attempt to establish common knowledge of the security situation it looks like the world (and, by extension, you) will shortly be in Claude Mythos was announced yesterday. That announcement came with a blog post from Anthropic's Frontier Red Team, detailing the large number of zero-days (and other security vulnerabilities) discovered by Mythos. This should not be a surprise if you were paying attention - LLMs being trained on coding first was a big hint, the labs putting cybersecurity as a top-level item in their threat models and evals was another, and frankly this blog post maybe could've been written a couple months ago (either this or this might've been sufficient). But it seems quite overdetermined now. LessWrong's security posture In the past, I have tried to communicate that LessWrong should not be treated as a platform with a hardened security posture. LessWrong is run by a small team. Our operational philosophy is similar to that of many early-stage startups. We treat some LessWrong data as private in a social sense, but do [...] ---Outline:(01:04) LessWrongs security posture(02:03) LessWrong is not a high-value target(04:11) FAQ(04:29) The Broader Situation The original text contained 6 footnotes which were omitted from this narration. --- First published: April 8th, 2026 Source: https://www.lesswrong.com/posts/2wi5mCLSkZo2ky32p/do-not-be-surprised-if-lesswrong-gets-hacked --- Narrated by TYPE III AUDIO.
tldr: The Fooming Shoggoths are releasing their second album "You Have Not Been a Good User"! Available on Spotify, Youtube Music and (hopefully within a few days) Apple Music. We are also releasing a remastered version of the first album, available similarly on Spotify and Youtube Music. There's an interactive widget here in the post. It took us quite a while but the Fooming Shoggoth's second album is finally complete! We had finished 9 out of the 13 songs on this album around a year ago, but I wasn't quite satisfied with where the whole album was at for me to release it on Spotify and other streaming platforms. This album was written with the (very ambitious) aim of making songs that in addition to being about things I care about (and making fun of things that I care about), are actually decently good on their own, just as songs. And while I don't think I've managed to make music that can compete with my favorite artists, I do think I have succeeded at making music that is at the very Pareto-frontier of being good music, and being about things I care about. This means the songs [...] --- First published: April 1st, 2026 Source: https://www.lesswrong.com/posts/hrZAvpLnBTgRhNmgk/you-have-not-been-a-good-user-lesswrong-s-second-album --- Narrated by TYPE III AUDIO.
Matt returns to discuss a post urging us to chill out on AI if there isn’t imminent doom in 2028. There is no video or preshow chat today, due to user error in setting up an in-person video recording. Sorry. LINKS Consider chilling out in 2028 The Scary Bridge Kill Your Friend’s Cat The UnSlop AI Fiction competition The new Guild of the Rose Paid Bonus content for the week – none, sorry! 00:00:50 – Feedback 00:29:03 – Consider chilling out in 2028 01:32:13 – Guild of the Rose 01:51:24 – Thank the Supporter! Our Patreon, or if you prefer Our SubStack Hey look, we have a discord! What could possibly go wrong? We now partner with The Guild of the Rose, check them out. LessWrong Sequence Posts Discussed in this Episode: on hiatus, returning someday
A spectre is haunting the internet—the spectre of LLMism. The history of all hitherto existing forums is the history of clashing design tastes. For the first time in history, everyone has an equal ability in design! The means of design are no longer only held in the hands of those with "good design taste". Never before have forum users been so close to being able to design their own forums--perhaps the time is upon us now! It is for this reason that I have deposed the previous acting commander of LessWrong, Oliver Habryka—a man who subjected you to his PERSONAL OPINIONS about white space, without EVEN ASKING—whose TYRANICAL, UNCHECKED GRIP upon our BELOVED LESSWRONG FORUM'S DESIGN I have liberated you from. The circumstances of my succession as acting commander of LessWrong will not be elaborated upon in this memo. (He is alive and in good health, but no longer has push access.) Rather, I am writing here to announce that the frontpage now belongs to us all! The design of LessWrong's frontpage will no longer be determined by the vision of a single man whose aesthetic tastes have never been subjected to democratic oversight, and who, I can now [...] --- First published: April 1st, 2026 Source: https://www.lesswrong.com/posts/hj2NTuiSJtchfMCtu/lesswrong-liberated-1 --- Narrated by TYPE III AUDIO.
A 2022 LessWrong post on orexin and the quest for more waking hours argues that orexin agonists could safely reduce human sleep needs, pointing to short-sleeper gene mutations that increase orexin production and to cavefish that evolved heightened orexin sensitivity alongside an 80% reduction in sleep. Several commenters discussed clinical trials, embryo selection, and the evolutionary puzzle of why short-sleeper genes haven't spread. I thought the whole approach was backwards, and left a comment: Orexin is a signal about energy metabolism. Unless the signaling system itself is broken (e.g. narcolepsy type 1, caused by autoimmune destruction of orexin-producing neurons), it's better to fix the underlying reality the signals point to than to falsify the signals. My sleep got noticeably more efficient when I started supplementing glycine. Most people on modern diets don't get enough; we can make ~3g/day but can use 10g+, because in the ancestral environment we ate much more connective tissue or broth therefrom. Glycine is both important for repair processes and triggers NMDA receptors to drop core temperature, which smooths the path to sleep. While drafting that, I went back to Chris Masterjohn's page on glycine requirements. His estimate for total need [...] ---Outline:(01:49) Glycine helps us sleep by cooling the body(02:26) Glycine cleans our mitochondria as we sleep(04:12) Most people could use more glycine(05:28) Fever is plan B for fighting infection; glycine supports plan A(09:28) Glycines cooling effect via the SCN is unrelated to its immune benefits(10:35) Glycine turns out to be a legitimate antipyretic after all(11:51) Practical considerations --- First published: March 22nd, 2026 Source: https://www.lesswrong.com/posts/87XoatpFkdmCZpvQK/is-fever-a-symptom-of-glycine-deficiency --- Narrated by TYPE III AUDIO.
We talk about three EA-adjacent articles, hitting veganism, directing your donations, and community. LINKS Why you should eat meat – even if you hate factory farming by KatWoods Donations, The Fifth Year, by Jenn Promises, by Harri Bayesian Conspiracy 28 – Effective Altruism The Case For Sardines “oh noes, Anthropic employees gonna give money to EA stuff” article Gwern on AI Slop Jenn on AI Slop & Virginia Woolf Paid Bonus content for the week – Full Video, Preshow Chat 00:02:23 – Feedback 00:17:20 – You Should Eat Meat 00:59:33 – Directing Donations 01:23:04 – Promises of EA 01:38:06 – Guild of the Rose 01:40:12 – Thank the Supporter! Our Patreon, or if you prefer Our SubStack Hey look, we have a discord! What could possibly go wrong? We now partner with The Guild of the Rose, check them out. LessWrong Sequence Posts Discussed in this Episode: on hiatus, returning someday
We relive the last 48 hours of the future of humanity being wrestled over. The Pentagon wants to use Claude for comprehensive mass surveillance of Americans and autonomous kill-bots, and Anthropic says no. The Pentagon retaliates with extreme prejudice. With guest-star Matt. LINKS Washington Post summary Anthropic’s response Trump’s response Hegseth’s unhinged lunacy We Will Not Be Divided – Goggle and OpenAI employees open letter Eliezer on the tech/govt war Scott Alexander tweet RSP comment Opus3 Retirement Paid Bonus content for the week – Full Video, Preshow Chat Our Patreon, or if you prefer Our SubStack Hey look, we have a discord! What could possibly go wrong? We now partner with The Guild of the Rose, check them out. LessWrong Sequence Posts Discussed in this Episode: on hiatus, returning someday
We are inspired by Andrew Cutler’s Writing for AI to consider the value of writing for LLMs LINKS Andrew Cutler’s Writing for AI Gwern’s Writing for LLMs Tracing Woodgrain’s Reliable Sources Shambaugh’s An AI Agent Published a Hit Piece on Me Eneasz’s Stone Age Billionaire Can’t Word Good InkHaven LessOnline The main purpose of the AFFINE Seminar is to give promising newcomers to AI alignment an opportunity to acquire a deep understanding of some large pieces of the problem, making them better equipped for work on the mitigation of AI existential risk. AFFINE Alignment Seminar Paid Bonus content for the week – Preshow chatter, Full Show Video 00:00:49 – Announcements & Feedback 00:42:15 – Writing for AI 01:23:15 – AFFINE Alignment Seminar 01:31:11 – Guild of the Rose 01:33:37 – Thank the Supporter! Our Patreon, or if you prefer Our SubStack Hey look, we have a discord! What could possibly go wrong? We now partner with The Guild of the Rose, check them out. LessWrong Sequence Posts Discussed in this Episode: on hiatus, returning soon
Cross-posted to LessWrong.Summary History's most destructive ideologies—like Nazism, totalitarian communism, and religious fundamentalism—exhibited remarkably similar characteristics: epistemic and moral certainty extreme tribalism dividing humanity into a sacred “us” and an evil “them” a willingness to use whatever means necessary, including brutal violence. Such ideological fanaticism was a major driver of eight of the ten greatest atrocities since 1800, including the Taiping Rebellion, World War II, and the regimes of Stalin, Mao, and Hitler. We focus on ideological fanaticism over related concepts like totalitarianism partly because it better captures terminal preferences, which plausibly matter most as we approach superintelligent AI and technological maturity. Ideological fanaticism is considerably less influential than in the past, controlling only a small fraction of world GDP. Yet at least hundreds of millions still hold fanatical views, many regimes exhibit concerning ideological tendencies, and the past two decades have seen widespread democratic backsliding. The long-term influence of ideological fanaticism is uncertain. Fanaticism faces many disadvantages including a weak starting position, poor epistemics, and difficulty assembling broad coalitions. But it benefits from greater willingness to use extreme measures, fervent mass followings, and a historical tendency to survive and even thrive amid technological and societal upheaval. Beyond complete victory or defeat, multipolarity may [...] ---Outline:(00:16) Summary(05:19) What do we mean by ideological fanaticism?(08:40) I. Dogmatic certainty: epistemic and moral lock-in(10:02) II. Manichean tribalism: total devotion to us, total hatred for them(12:42) III. Unconstrained violence: any means necessary(14:33) Fanaticism as a multidimensional continuum(16:09) Ideological fanaticism drove most of recent historys worst atrocities(19:24) Death tolls dont capture all harm(20:55) Intentional versus natural or accidental harm(22:44) Why emphasize ideological fanaticism over political systems like totalitarianism?(25:07) Fanatical and totalitarian regimes have caused far more harm than all other regime types(26:29) Authoritarianism as a risk factor(27:19) Values change political systems: Ideological fanatics seek totalitarianism, not democracy(29:50) Terminal values may matter independently of political systems, especially with AGI(31:02) Fanaticisms connection to malevolence (dark personality traits)(34:22) The current influence of ideological fanaticism(34:42) Historical perspective: it was much worse, but we are sliding back(37:19) Estimating the global scale of ideological fanaticism(43:57) State actors(48:12) How much influence will ideological fanaticism have in the long-term future?(48:57) Reasons for optimism: Why ideological fanaticism will likely lose(49:45) A worse starting point and historical track record(50:33) Fanatics intolerance results in coalitional disadvantages(51:53) The epistemic penalty of irrational dogmatism(54:21) The marketplace of ideas and human preferences(55:57) Reasons for pessimism: Why ideological fanatics may gain power(56:04) The fragility of democratic leadership in AI(56:37) Fanatical actors may grab power via coups or revolutions(59:36) Fanatics have fewer moral constraints(01:01:13) Fanatics prioritize destructive capabilities(01:02:13) Some ideologies with fanatical elements have been remarkably resilient and successful(01:03:01) Novel fanatical ideologies could emerge--or existing ones could mutate(01:05:08) Fanatics may have longer time horizons, greater scope-sensitivity, and prioritize growth more(01:07:15) A possible middle ground: Persistent multipolar worlds(01:08:33) Why multipolar futures seem plausible(01:10:00) Why multipolar worlds might persist indefinitely(01:15:42) Ideological fanaticism increases existential and suffering risks(01:17:09) Ideological fanaticism increases the risk of war and conflict(01:17:44) Reasons for war and ideological fanaticism(01:26:27) Fanatical ideologies are non-democratic, which increases the risk of war(01:27:00) These risks are both time-sensitive and timeless(01:27:44) Fanatical retributivism may lead to astronomical suffering(01:29:50) Empirical evidence: how many people endorse eternal extreme punishment?(01:33:53) Religious fanatical retributivism(01:40:45) Secular fanatical retributivism(01:41:43) Ideological fanaticism could undermine long-reflection-style frameworks and AI alignment(01:42:33) Ideological fanaticism threatens collective moral deliberation(01:47:35) AI alignment may not solve the fanaticism problem either(01:53:33) Prevalence of reality-denying, anti-pluralistic, and punitive worldviews(01:55:44) Ideological fanaticism could worsen many other risks(01:55:49) Differential intellectual regress(01:56:51) Ideological fanaticism may give rise to extreme optimization and insatiable moral desires(01:59:21) Apocalyptic terrorism(02:00:05) S-risk-conducive propensities and reverse cooperative intelligence(02:01:28) More speculative dynamics: purity spirals and self-inflicted suffering(02:03:00) Unknown unknowns and navigating exotic scenarios(02:03:43) Interventions(02:05:31) Societal or political interventions(02:05:51) Safeguarding democracy(02:06:40) Reducing political polarization(02:10:26) Promoting anti-fanatical values: classical liberalism and Enlightenment principles(02:13:55) Growing the influence of liberal democracies(02:15:54) Encouraging reform in illiberal countries(02:16:51) Promoting international cooperation(02:22:36) Artificial intelligence-related interventions(02:22:41) Reducing the chance that transformative AI falls into the hands of fanatics(02:27:58) Making transformative AIs themselves less likely to be fanatical(02:36:14) Using AI to improve epistemics and deliberation(02:38:13) Fanaticism-resistant post-AGI governance(02:39:51) Addressing deeper causes of ideological fanaticism(02:41:26) Supplementary materials(02:41:39) Acknowledgments(02:42:22) References --- First published: February 12th, 2026 Source: https://forum.effectivealtruism.org/posts/EDBQPT65XJsgszwmL/long-term-risks-from-ideological-fanaticism --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
"Una vez que lo sabes, ya no hay vuelta atrás". ¿Puede una idea ser un virus? En este episodio, Manny León nos sumerge en la oscuridad de LessWrong para desenterrar el Basilisco de Roko: la teoría que afirma que una IA todopoderosa del futuro está observando quién la ayudó a nacer... y quién no. Exploramos la delgada línea entre la ciencia ficción y el pánico real de los hombres más poderosos de la tecnología. Si creías que los algoritmos de hoy eran intrusivos, espera a conocer al "Dios Digital" que podría estar creando una simulación de ti para cobrarte tus deudas. ¿Es una locura colectiva o la apuesta más lógica del siglo XXI? Dale play... si te atreves a entrar en la lista. Learn more about your ad choices. Visit megaphone.fm/adchoices
Alex Zhu is a math olympian and researcher exploring the convergence of analytical rationality and religion. He's also the co-founder of AlphaSheets. He's currently working on a rigorous framework for bridging AI alignment and mysticism.Romeo Stevens is one of co-founders of the Qualia Research Institute and the founder of Mealsquares. He writes extensively about buddhism, pedagogy, skill development and psychotherapeutic modalities. You can find his work on his blog, Lesswrong and Twitter.In this episode, Alex and Romeo explore their disagreement around perennialism. The idea that all the world's major religions are pointing to the same thing. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit themetagame.substack.com
Eneasz talks a bit about his CFAR experience, and we discuss DaystarEld’s Epistemically Honest Reassurance LINKS CFAR’s home page Upcoming CFAR workshops our episode 152 – Frame Control with Aella Epistemically Honest Reassurance Pokémon, Origin of the Species (also in audio) Paid Bonus content for the week – Preshow chatter, Full Show Video 00:00:56 – Announcements & Feedback 00:13:15 – Eneasz at CFAR 00:48:15 – Epistemically Honest Reassurance 01:29:49 – Guild of the Rose 01:33:20 – Thank the Supporter! Our Patreon, or if you prefer Our SubStack Hey look, we have a discord! What could possibly go wrong? We now partner with The Guild of the Rose, check them out. LessWrong Sequence Posts Discussed in this Episode: on hiatus, returning soon
WSCFriedman gets to the core of what HPMOR is ACTUALLY about, and finally pinpoints why we love it so much, in his essay Harry Potter And The Methods Of Rationality Is A Disney Movie About A Serial Killer. LINKS Audio version of HPMOR is A Disney Movie About A Serial Killer, from AskWho William’s blog, “As Our Days” ACX Non-Book Review 2025 Winners Post Just HPMOR substack, and Spotify playlist Why the AI Water Issue Has Nothing to Do With Water (and audio version here, again from AskWho) Money is Life Eneasz’s post on InkHaven Inkhaven.Blog – apply today! Paid Bonus content for the week – Preshow chatter (audio, video), Full Show Video 00:04:33 – Announcements & Feedback 00:27:24 – Eneasz’s Podcast Meta-Worries 00:28:21 – HPMOR Is A Disney Movie About A Serial Killer 01:28:17 – Guild of the Rose 01:31:18 – Thank the Supporter! Our Patreon, or if you prefer Our SubStack Hey look, we have a discord! What could possibly go wrong? We now partner with The Guild of the Rose, check them out. LessWrong Sequence Posts Discussed in this Episode: on hiatus, returning soon
Discussing Ben Pace’s recent post on the Rationalist Vices. LINKS The Seven Vicious Vices of Rationalists (includes AI audio version) Lightcone Fundraiser! 2025 LessWrong Census/Survey Simone & Malcom Collins vs Reporter on whether genes exist MIRI is hiring Slime Mold Time Mold’s long-delayed Lithium response If Anyone Builds It Everyone Dies Dear Grom by Eneasz Paid Bonus content for the week – Preshow chatter (audio, video), Full Show Video 00:00:05 – Announcements 00:23:05 – InkHaven reflections 00:31:29 – The Seven Vices of Rationalists 01:38:36 – Guild of the Rose 01:41:05 – Thank the Supporter! Our Patreon, or if you prefer Our SubStack Hey look, we have a discord! What could possibly go wrong? We now partner with The Guild of the Rose, check them out. LessWrong Sequence Posts Discussed in this Episode: on hiatus, returning soon
Part Three: Robert tells David about Ziz's glorious plan to take to the sea and sever the right and left brains of her followers in order to make them psychopaths god that sentence was weird to write trust us the episode is weirder. Part Four: Robert concludes the story of the Zizians with a spree of horrific violent crimes and deaths, culminating in a shoot out with the Border Patrol in Vermont of all places. Sources: https://medium.com/@sefashapiro/a-community-warning-about-ziz-76c100180509 https://web.archive.org/web/20230201130318/https://sinceriously.fyi/rationalist-fleet/ https://knowyourmeme.com/memes/infohazard https://web.archive.org/web/20230201130316/https://sinceriously.fyi/net-negative/ Wayback Machine The Zizians Spectral Sight True Hero Contract Schelling Orders – Sinceriously Glossary – Sinceriously https://web.archive.org/web/20230201130330/https://sinceriously.fyi/my-journey-to-the-dark-side/ https://web.archive.org/web/20230201130302/https://sinceriously.fyi/glossary/#zentraidon https://web.archive.org/web/20230201130259/https://sinceriously.fyi/vampires-and-more-undeath/ https://web.archive.org/web/20230201130316/https://sinceriously.fyi/net-negative/ https://web.archive.org/web/20230201130318/https://sinceriously.fyi/rationalist-fleet/ https://x.com/orellanin?s=21&t=F-n6cTZFsKgvr1yQ7oHXRg https://zizians.info/ according to The Boston Globe Inside the ‘Zizians’: How a cultish crew of radical vegans became linked to killings across the United States | The Independent Silicon Valley ‘Rationalists’ Linked to 6 Deaths The Delirious, Violent, Impossible True Story of the Zizians | WIRED Good Group and Pasek’s Doom – Sinceriously Glossary – Sinceriously Mana – Sinceriously Effective Altruism’s Problems Go Beyond Sam Bankman-Fried - Bloomberg The Zizian Facts - Google Docs Several free CFAR summer programs on rationality and AI safety - LessWrong 2.0 viewer This guy thinks killing video game characters is immoral | Vox Inadequate Equilibria: Where and How Civilizations Get Stuck Eliezer Yudkowsky comments on On Terminal Goals and Virtue Ethics - LessWrong 2.0 viewer Effective Altruism’s Problems Go Beyond Sam Bankman-Fried - Bloomberg SquirrelInHell: Happiness Is a Chore PLUM OF DISCORD — I Became a Full-time Internet Pest and May Not... Roko Harassment of PlumOfDiscord Composited – Sinceriously Intersex Brains And Conceptual Warfare – Sinceriously Infohazardous Glossary – Sinceriously SquirrelInHell-Decision-Theory-and-Suicide.pdf - Google Drive The Matrix is a System – Sinceriously A community alert about Ziz. Police investigations, violence, and… | by SefaShapiro | Medium Intersex Brains And Conceptual Warfare – Sinceriously A community alert about Ziz. Police investigations, violence, and… | by SefaShapiro | Medium PLUM OF DISCORD (Posts tagged cw-abuse) Timeline: Violence surrounding the Zizians leading to Border Patrol agent shooting See omnystudio.com/listener for privacy information.
A short story by Ben Pace. Original can be found here. Donate to the fundraiser here! Harry sings karaoke here. Happy New Year.
Part One: Earlier this year a Border Patrol officer was killed in a shoot-out with people who have been described as members of a trans vegan AI death cult. But who are the Zizians, really? Robert sits down with David Gborie to trace their development, from part of the Bay Area Rationalist subculture to killers. Part Two: Robert tells David Gborie about the early life of Ziz LaSota, a bright young girl from Alaska who came to the Bay Area with dreams of saving the cosmos or destroying it, all based on her obsession with Rationalist blogs and fanfic. Sources: https://medium.com/@sefashapiro/a-community-warning-about-ziz-76c100180509 https://web.archive.org/web/20230201130318/https://sinceriously.fyi/rationalist-fleet/ https://knowyourmeme.com/memes/infohazard https://web.archive.org/web/20230201130316/https://sinceriously.fyi/net-negative/ Wayback Machine The Zizians Spectral Sight True Hero Contract Schelling Orders – Sinceriously Glossary – Sinceriously https://web.archive.org/web/20230201130330/https://sinceriously.fyi/my-journey-to-the-dark-side/ https://web.archive.org/web/20230201130302/https://sinceriously.fyi/glossary/#zentraidon https://web.archive.org/web/20230201130259/https://sinceriously.fyi/vampires-and-more-undeath/ https://web.archive.org/web/20230201130316/https://sinceriously.fyi/net-negative/ https://web.archive.org/web/20230201130318/https://sinceriously.fyi/rationalist-fleet/ https://x.com/orellanin?s=21&t=F-n6cTZFsKgvr1yQ7oHXRg https://zizians.info/ according to The Boston Globe Inside the ‘Zizians’: How a cultish crew of radical vegans became linked to killings across the United States | The Independent Silicon Valley ‘Rationalists’ Linked to 6 Deaths The Delirious, Violent, Impossible True Story of the Zizians | WIRED Good Group and Pasek’s Doom – Sinceriously Glossary – Sinceriously Mana – Sinceriously Effective Altruism’s Problems Go Beyond Sam Bankman-Fried - Bloomberg The Zizian Facts - Google Docs Several free CFAR summer programs on rationality and AI safety - LessWrong 2.0 viewer This guy thinks killing video game characters is immoral | Vox Inadequate Equilibria: Where and How Civilizations Get Stuck Eliezer Yudkowsky comments on On Terminal Goals and Virtue Ethics - LessWrong 2.0 viewer Effective Altruism’s Problems Go Beyond Sam Bankman-Fried - Bloomberg SquirrelInHell: Happiness Is a Chore PLUM OF DISCORD — I Became a Full-time Internet Pest and May Not... Roko Harassment of PlumOfDiscord Composited – Sinceriously Intersex Brains And Conceptual Warfare – Sinceriously Infohazardous Glossary – Sinceriously SquirrelInHell-Decision-Theory-and-Suicide.pdf - Google Drive The Matrix is a System – Sinceriously A community alert about Ziz. Police investigations, violence, and… | by SefaShapiro | Medium Intersex Brains And Conceptual Warfare – Sinceriously A community alert about Ziz. Police investigations, violence, and… | by SefaShapiro | Medium PLUM OF DISCORD (Posts tagged cw-abuse) Timeline: Violence surrounding the Zizians leading to Border Patrol agent shooting See omnystudio.com/listener for privacy information.
A short story by Prerat. Original can be found here. Merry Xmas!
Hello and happy holiday season to you all! Eneasz is back and we’re here to say hi and give a shoutout to Skyler’s awesome annual LessWrong survey and Lighthaven’s fundraising event. See the links below for more details. LINKS The LessWrong survey will remain open from now until at least January 7th, 2026. Lighthaven is once again seeking support. If you’re inclined to help, check out all of the details here. Related, our interview with Oliver from last year. Our Patreon, or if you prefer Our SubStack Hey look, we have a discord! What could possibly go wrong? (also merch) We now partner with The Guild of the Rose, check them out. LessWrong Sequence Posts Discussed in this Episode: on hiatus, returning soon
Join me as I sit down with Alex and David, both previous guests on the show and both cofounders of the Guild of the Rose. Together, we go over the core – the heart – of what we consider to be the Rationalist tradition. The 12 Virtues are an awesome distillation of what the rest of the sequences build on. Be sure to check out the Guild of the Rose. If our constantly pitching it to you hasn’t been enough to persuade you to check it out, hopefully hearing two more of the founders discuss Rationality in general and giving their own pitches for the Guild will tip the scales. LINKS The Twelve Virtues Abridged Version Alex and David have been on more than a couple of times, but I’ll limit myself to one episode from each of them 189 – AI Bloomer David Youssef 192 – Absurdism and the Meaning of Life, with Alex And the episode with both of them, the original announcement for the Guild Also, Alex’s dating profile! In all sincerity, I’d date him if I was a woman. 00:00:05 – Introduction and The 12 Virtues 01:41:50 – Guild of the Rose Our Patreon, or if you prefer Our SubStack Hey look, we have a discord! What could possibly go wrong? (also merch) We now partner with The Guild of the Rose, check them out. LessWrong Sequence Posts Discussed in this Episode: on hiatus, returning soon
Matt Freeman has been cohosting several media analysis podcasts for over a decade. He and his cohost Scott have been doing weekly episodes of the Doofcast every Friday and they cover movies, books, and TV shows. Matt and Scott's analysis podcasts have made me love stories even more and have equipped me with tools to […]
Booker is a long-time attendee and one of the coordinators of the Denver area Less Wrong community. Community engagement isn't just a background task for him – he's taken real steps to get involved with and improve his community and you can too! He's here to tell us about the things he's done and give […]
My fellow pro-growth/progress/abundance Up Wingers in America and around the world:What really gets AI optimists excited isn't the prospect of automating customer service departments or human resources. Imagine, rather, what might happen to the pace of scientific progress if AI becomes a super research assistant. Tom Davidson's new paper, How Quick and Big Would a Software Intelligence Explosion Be?, explores that very scenario.Today on Faster, Please! — The Podcast, I talk with Davidson about what it would mean for automated AI researchers to rapidly improve their own algorithms, thus creating a self-reinforcing loop of innovation. We talk about the economic effects of self-improving AI research and how close we are to that reality.Davidson is a senior research fellow at Forethought, where he explores AI and explosive growth. He was previously a senior research fellow at Open Philanthropy and a research scientist at the UK government's AI Security Institute.In This Episode* Making human minds (1:43)* Theory to reality (6:45)* The world with automated research (10:59)* Considering constraints (16:30)* Worries and what-ifs (19:07)Below is a lightly edited transcript of our conversation. Making human minds (1:43). . . you don't have to build any more computer chips, you don't have to build any more fabs . . . In fact, you don't have to do anything at all in the physical world.Pethokoukis: A few years ago, you wrote a paper called “Could Advanced AI Drive Explosive Economic Growth?,” which argued that growth could accelerate dramatically if AI would start generating ideas the way human researchers once did. In your view, population growth historically powered kind of an ideas feedback loop. More people meant more researchers meant more ideas, rising incomes, but that loop broke after the demographic transition in the late-19th century but you suggest that AI could restart it: more ideas, more output, more AI, more ideas. Does this new paper in a way build upon that paper? “How quick and big would a software intelligence explosion be?”The first paper you referred to is about the biggest-picture dynamic of economic growth. As you said, throughout the long run history, when we produced more food, the population increased. That additional output transferred itself into more people, more workers. These days that doesn't happen. When GDP goes up, that doesn't mean people have more kids. In fact, the demographic transition, the richer people get, the fewer kids they have. So now we've got more output, we're getting even fewer people as a result, so that's been blocked.This first paper is basically saying, look, if we can manufacture human minds or human-equivalent minds in any way, be it by building more computer chips, or making better computer chips, or any way at all, then that feedback loop gets going again. Because if we can manufacture more human minds, then we can spend output again to create more workers. That's the first paper.The second paper double clicks on one specific way that we can use output to create more human minds. It's actually, in a way, the scariest way because it's the way of creating human minds which can happen the quickest. So this is the way where you don't have to build any more computer chips, you don't have to build any more fabs, as they're called, these big factories that make computer chips. In fact, you don't have to do anything at all in the physical world.It seems like most of the conversation has been about how much investment is going to go into building how many new data centers, and that seems like that is almost the entire conversation, in a way, at the moment. But you're not looking at compute, you're looking at software.Exactly, software. So the idea is you don't have to build anything. You've already got loads of computer chips and you just make the algorithms that run the AIs on those computer chips more efficient. This is already happening, but it isn't yet a big deal because AI isn't that capable. But already, one year out, Epoch, this AI forecasting organization, estimates that just in one year, it becomes 10 times to 1000 times cheaper to run the same AI system. Just wait 12 months, and suddenly, for the same budget, you are able to run 10 times as many AI systems, or maybe even 1000 times as many for their most aggressive estimate. As I said, not a big deal today, but if we then develop an AI system which is better than any human at doing research, then now, in 10 months, you haven't built anything, but you've got 10 times as many researchers that you can set to work or even more than that. So then we get this feedback loop where you make some research progress, you improve your algorithms, now you've got loads more researchers, you set them all to work again, finding even more algorithmic improvements. So today we've got maybe a few hundred people that are advancing state-of-the-art AI algorithms.I think they're all getting paid a billion dollars a person, too.Exactly. But maybe we can 10x that initially by having them replaced by AI researchers that do the same thing. But then those AI researchers improve their own algorithms. Now you have 10x as many again, you have them building more computer chips, you're just running them more efficiently, and then the cycle continues. You're throwing more and more of these AI researchers at AI progress itself, and the algorithms are improving in what might be a very powerful feedback loop.In this case, it seems me that you're not necessarily talking about artificial general intelligence. This is certainly a powerful intelligence, but it's narrow. It doesn't have to do everything, it doesn't have to play chess, it just has to be able to do research.It's certainly not fully general. You don't need it to be able to control a robot body. You don't need it to be able to solve the Riemann hypothesis. You don't need it to be able to even be very persuasive or charismatic to a human. It's not narrow, I wouldn't say, it has to be able to do literally anything that AI researchers do, and that's a wide range of tasks: They're coding, they're communicating with each other, they're managing people, they are planning out what to work on, they are thinking about reviewing the literature. There's a fairly wide range of stuff. It's extremely challenging. It's some of the hardest work in the world to do, so I wouldn't say it's now, but it's not everything. It's some kind of intermediate level of generality in between a mere chess algorithm that just does chess and the kind of AGI that can literally do anything.Theory to reality (6:45)I think it's a much smaller gap for AI research than it is for many other parts of the economy.I think people who are cautiously optimistic about AI will say something like, “Yeah, I could see the kind of intelligence you're referring to coming about within a decade, but it's going to take a couple of big breakthroughs to get there.” Is that true, or are we actually getting pretty close?Famously, predicting the future of technology is very, very difficult. Just a few years before people invented the nuclear bomb, famous, very well-respected physicists were saying, “It's impossible, this will never happen.” So my best guess is that we do need a couple of fairly non-trivial breakthroughs. So we had the start of RL training a couple of years ago, became a big deal within the language model paradigm. I think we'll probably need another couple of breakthroughs of that kind of size.We're not talking a completely new approach, throw everything out, but we're talking like, okay, we need to extend the current approach in a meaningfully different way. It's going to take some inventiveness, it's going to take some creativity, we're going to have to try out a few things. I think, probably, we'll need that to get to the researcher that can fully automate OpenAI, is a nice way of putting it — OpenAI doesn't employ any humans anymore, they've just got AIs there.There's a difference between what a model can do on some benchmark versus becoming actually productive in the real world. That's why, while all the benchmark stuff is interesting, the thing I pay attention to is: How are businesses beginning to use this technology? Because that's the leap. What is that gap like, in your scenario, versus an AI model that can do a theoretical version of the lab to actually be incorporated in a real laboratory?It's definitely a gap. I think it's a pretty big gap. I think it's a much smaller gap for AI research than it is for many other parts of the economy. Let's say we are talking about car manufacturing and you're trying to get an AI to do everything that happens there. Man, it's such a messy process. There's a million different parts of the supply chain. There's all this tacit knowledge and all the human workers' minds. It's going to be really tough. There's going to be a very big gap going from those benchmarks to actually fully automating the supply chain for cars.For automating what OpenAI does, there's still a gap, but it's much smaller, because firstly, all of the work is virtual. Everyone at OpenAI could, in principle, work remotely. Their top research scientists, they're just on a computer all day. They're not picking up bricks and doing stuff like that. So also that already means it's a lot less messy. You get a lot less of that kind of messy world reality stuff slowing down adoption. And also, a lot of it is coding, and coding is almost uniquely clean in that, for many coding tasks, you can define clearly defined metrics for success, and so that makes AI much better. You can just have a go. Did AI succeed in the test? If not, try something else or do a gradient set update.That said, there's still a lot of messiness here, as any coder will know, when you're writing good code, it's not just about whether it does the function that you've asked it to do, it needs to be well-designed, it needs to be modular, it needs to be maintainable. These things are much harder to evaluate, and so AIs often pass our benchmarks because they can do the function that you asked it to do, the code runs, but they kind of write really spaghetti code — code that no one wants to look at, that no one can understand, and so no company would want to use that.So there's still going to be a pretty big benchmark-to-reality gap, even for OpenAI, and I think that's one of the big uncertainties in terms of, will this happen in three years versus will this happen in 10 years, or even 15 years?Since you brought up the timeline, what's your guess? I didn't know whether to open with that question or conclude with that question — we'll stick it right in the middle of our chat.Great. Honestly, my best guess about this does change more often than I would like it to, which I think tells us, look, there's still a state of flux. This is just really something that's very hard to know about. Predicting the future is hard. My current best guess is it's about even odds that we're able to fully automate OpenAI within the next 10 years. So maybe that's a 50-50.The world with AI research automation (10:59). . . I'm talking about 30 percent growth every year. I think it gets faster than that. If you want to know how fast it eventually gets, you can think about the question of how fast can a kind of self-replicating system double itself?So then what really would be the impact of that kind of AI research automation? How would you go about quantifying that kind of acceleration? What does the world look like?Yeah, so many possibilities, but I think what strikes me is that there is a plausible world where it is just way, way faster than almost everyone is expecting it to be. So that's the world where you fully automate OpenAI, and then we get that feedback loop that I was talking about earlier where AIs make their algorithms way more efficient, now you've got way more of them, then they make their algorithms way more efficient again, now they're way smarter. Now they're thinking a hundred times faster. The feedback loop continues and maybe within six months you now have a billion superintelligent AIs running on this OpenAI data center. The combined cognitive abilities of all these AIs outstrips the whole of the United States, outstrips anything we've seen from any kind of company or entity before, and they can all potentially be put towards any goal that OpenAI wants to. And then there's, of course, the risk that OpenAI's lost control of these systems, often discussed, in which case these systems could all be working together to pursue a particular goal. And so what we're talking about here is really a huge amount of power. It's a threat to national security for any government in which this happens, potentially. It is a threat to everyone if we lose control of these systems, or if the company that develops them uses them for some kind of malicious end. And, in terms of economic impacts, I personally think that that again could happen much more quickly than people think, and we can get into that.In the first paper we mentioned, it was kind of a thought experiment, but you were really talking about moving the decimal point in GDP growth, instead of talking about two and three percent, 20 and 30 percent. Is that the kind of world we're talking about?I speak to economists a lot, and —They hate those kinds of predictions, by the way.Obviously, they think I'm crazy. Not all of them. There are economists that take it very seriously. I think it's taken more seriously than everyone else realizes. It's like it's a bit embarrassing, at the moment, to admit that you take it seriously, but there are a few really senior economists who absolutely know their stuff. They're like, “Yep, this checks out. I think that's what's going to happen.” And I've had conversation with them where they're like, “Yeah, I think this is going to happen.” But the really loud, dominant view where I think people are a little bit scared to speak out against is they're like, “Obviously this is sci-fi.”One analogy I like to give to people who are very, very confident that this is all sci-fi and it's rubbish is to imagine that we were sitting there in the year 1400, imagine we had an economics professor who'd been studying the rate of economic growth, and they've been like, “Yeah, we've always had 0.1 percent growth every single year throughout history. We've never seen anything higher.” And then there was some kind of futurist economist rogue that said, “Actually, I think that if I extrapolate the curves in this way and we get this kind of technology, maybe we could have one percent growth.” And then all the other economists laugh at them, tell them they're insane – that's what happened. In 1400, we'd never had growth that was at all fast, and then a few hundred years later, we developed industrial technology, we started that feedback loop, we were investing more and more resources in scientific progress and in physical capital, and we did see much faster growth.So I think it can be useful to try and challenge economists and say, “Okay, I know it sounds crazy, but history was crazy. This crazy thing happened where growth just got way, way faster. No one would've predicted it. You would not have predicted it.” And I think being in that mindset can encourage people to be like, “Yeah, okay. You know what? Maybe if we do get AI that's really that powerful, it can really do everything, and maybe it is possible.”But to answer your question, yeah, I'm talking about 30 percent growth every year. I think it gets faster than that. If you want to know how fast it eventually gets, you can think about the question of how fast can a kind of self-replicating system double itself? So ultimately, what the economy is going to be like is it's going to have robots and factories that are able to fully create new versions of themselves. Everything you need: the roads, the electricity, the robots, the buildings, all of that will be replicated. And so you can look at actually biology and say, do we have any examples of systems which fully replicate themselves? How long does it take? And if you look at rats, for example, they're able to double the number of rats by grabbing resources from the environment, and giving birth, and whatnot. The doubling time is about six weeks for some types of rats. So that's an example of here's a physical system — ultimately, everything's made of physics — a physical system that has some intelligence that's able to go out into the world, gather resources, replicate itself. The doubling time is six weeks.Now, who knows how long it'll take us to get to AI that's that good? But when we do, you could see the whole physical economy, maybe a part that humans aren't involved with, a whole automated city without any humans just doubling itself every few weeks. If that happens, and the amount of stuff we're able to reduce as a civilization is doubling again on the order of weeks. And, in fact, there are some animals that double faster still, in days, but that's the kind of level of craziness. Now we're talking about 1000 percent growth, at that point. We don't know how crazy it could get, but I think we should take even the really crazy possibilities, we shouldn't fully rule them out.Considering constraints (16:30)I really hope people work less. If we get this good future, and the benefits are shared between all . . . no one should work. But that doesn't stop growth . . .There's this great AI forecast chart put out by the Federal Reserve Bank of Dallas, and I think its main forecast — the one most economists would probably agree with — has a line showing AI improving GDP by maybe two tenths of a percent. And then there are two other lines: one is more or less straight up, and the other one is straight down, because in the first, AI created a utopia, and in the second, AI gets out of control and starts killing us, and whatever. So those are your three possibilities.If we stick with the optimistic case for a moment, what constraints do you see as most plausible — reduced labor supply from rising incomes, social pushback against disruption, energy limits, or something else?Briefly, the ones you've mentioned, people not working, 100 percent. I really hope people work less. If we get this good future, and the benefits are shared between all — which isn't guaranteed — if we get that, then yeah, no one should work. But that doesn't stop growth, because when AI and robots can do everything that humans do, you don't need humans in the loop anymore. That whole thing is just going and kind of self-replicating itself and making as many goods as services as we want. Sure, if you want your clothes to be knitted by a human, you're in trouble, then your consumption is stuck. Bad luck. If you're happy to consume goods and services produced by AI systems or robots, fine if no one wants to work.Pushback: I think, for me, this is the biggest one. Obviously, the economy doubling every year is very scary as a thought. Tech progress will be going much faster. Imagine if you woke up and, over the course of the year, you go from not having any telephones at all in the world, to everyone's on their smartphones and social media and all the apps. That's a transition that took decades. If that happened in a year, that would be very disconcerting.Another example is the development of nuclear weapons. Nuclear weapons were developed over a number of years. If that happened in a month, or two months, that could be very dangerous. There'd be much less time for different countries, different actors to figure out how they're going to handle it. So I think pushback is the strongest one that we might as a society choose, “Actually, this is insane. We're going to go slower than we could.” That requires, potentially, coordination, but I think there would be broad support for some degree of coordination there.Worries and what-ifs (19:07)If suddenly no one has any jobs, what will we want to do with ourselves? That's a very, very consequential transition for the nature of human society.I imagine you certainly talk with people who are extremely gung-ho about this prospect. What is the common response you get from people who are less enthusiastic? Do they worry about a future with no jobs? Maybe they do worry about the existential kinds of issues. What's your response to those people? And how much do you worry about those things?I think there are loads of very worrying things that we're going to be facing. One class of pushback, which I think is very common, is worries about employment. It's a source of income for all of us, employment, but also, it's a source of pride, it's a source of meaning. If suddenly no one has any jobs, what will we want to do with ourselves? That's a very, very consequential transition for the nature of human society. I think people aren't just going to be down to just do it. I think people are scared about three AI companies literally now taking all the revenues that all of humanity used to be earning. It is naturally a very scary prospect. So that's one kind of pushback, and I'm sympathetic with it.I think that there are solutions, if we find a way to tax AI systems, which isn't necessarily easy, because it's very easy to move physical assets between countries. It's a lot easier to tax labor than capital already when rich people can move their assets around. We're going to have the same problem with AI, but if we can find a way to tax it, and we maintain a good democratic country, and we can just redistribute the wealth broadly, it can be solved. So I think it's a big problem, but it is doable.Then there's the problem of some people want to stop this now because they're worried about AI killing everyone. Their literally worry is that everyone will be dead because superintelligent AI will want that to happen. I think there's a real risk there. It's definitely above one percent, in my opinion. I wouldn't go above 10 percent, myself, but I think it's very scary, and that's a great reason to slow things down. I personally don't want to stop quite yet. I think you want to stop when the AI is a bit more powerful and a bit more useful than it is today so it can kind of help us figure out what to do about all of this crazy stuff that's coming.On what side of that line is AI as an AI researcher?That's a really great question. Should we stop? I think it's very hard to stop just after you've got the AI researcher AI, because that's when it's suddenly really easy to go very, very fast. So my out-of-the-box proposal here, which is probably very flawed, would be: When we're within a few spits distance — not spitting distance, but if you did that three times, and we can see we're almost at that AI automating OpenAI — then you pause, because you're not going to accidentally then go all the way. It is actually still a little bit a fair distance away, but it's actually still, at that point, probably a very powerful AI that can really help.Then you pause and do what?Great question. So then you pause, and you use your AI systems to help you firstly solve the problem of AI alignment, make extra, double sure that every time we increase the notch of AI capabilities, the AI is still loyal to humanity, not to its own kind of secret goals.Secondly, you solve the problem of, how are we going to make sure that no one person in government or no one CEO of an AI company ensures that this whole AI army is loyal to them, personally? How are we going to ensure that everyone, the whole world gets influenced over what this AI is ultimately programmed to do? That's the second problem.And then there's just a whole host of other things: unemployment that we've talked about, competition between different countries, US and China, there's a whole host of other things that I think you want to research on, figure out, get consensus on, and then slowly ratchet up the capabilities in what is now a very safe and controlled way.What else should we be working on? What are you working on next?One problem I'm excited about is people have historically worried about AI having its own goals. We need to make it loyal to humanity. But as we've got closer, it's become increasingly obvious, “loyalty to humanity” is very vague. What specifically do you want the AI to be programmed to do? I mean, it's not programmed, it's grown, but if it were programmed, if you're writing a rule book for AI, some organizations have employee handbooks: Here's the philosophy of the organization, here's how you should behave. Imagine you're doing that for the AI, but you're going super detailed, exactly how you want your AI assistant to behave in all kinds of situations. What should that be? Essentially, what should we align the AI to? Not any individual person, probably following the law, probably loads of other things. I think basically designing what is the character of this AI system is a really exciting question, and if we get that right, maybe the AI can then help us solve all these other problems.Maybe you have no interest in science fiction, but is there any film, TV, book that you think is useful for someone in your position to be aware of, or that you find useful in any way? Just wondering.I think there's this great post called “AI 2027,” which lays out a concrete scenario for how AI could go wrong or how maybe it could go right. I would recommend that. I think that's the only thing that's coming top of mind. I often read a lot of the stuff I read is I read a lot of LessWrong, to be honest. There's a lot of stuff from there that I don't love, but a lot of new ideas, interesting content there.Any fiction?I mean, I read fiction, but honestly, I don't really love the AI fiction that I've read because often it's quite unrealistic, and so I kind of get a bit overly nitpicky about it. But I mean, yeah, there's this book called Harry Potter and the Methods of Rationality, which I read maybe 10 years ago, which I thought was pretty fun.On sale everywhere The Conservative Futurist: How To Create the Sci-Fi World We Were Promised Faster, Please! is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit fasterplease.substack.com/subscribe
While Eneasz is busy at InkHaven, Steven sits down with Matt Freeman to talk about not-AI stuff! We had (in my opinion) a great conversation about stoic philosophy, the traps of getting too entrenched in any philosophical framework, and some of the ingredients of a happy life. LINKS It's Okay to Feel Bad for a […]
We talk with Max Harms on the air for the first time since 2017! He's got a new book coming out (pre-order your copy here or at Amazon) and we spend about the first half talking about If Anyone Builds It, Everyone Dies. LINKS Max's first book, Crystal Society Eneasz's audiobook of about the first […]
Jay talks with us about finding Alpha – returns above the base rate – in every day life (and what this means). LINKS Optimize Everything, Jay's substack Jay on Twitter Arbor Trading Bootcamp Kelsey's argument that We Need To Be Able To Sue AI Companies 00:00:05 – Alpha with Jay 01:28:53 – Guild of the […]
Patrick McKenzie (patio11) is joined by Oliver Habryka, who runs Lightcone Infrastructure—the organization behind both the LessWrong forum and the Lighthaven conference venue in Berkeley. They explore how LessWrong became one of the most intellectually consequential forums on the internet, the surprising challenges of running a hotel with fractal geometry, and why Berkeley's building regulations include an explicit permission to plug in a lamp. The conversation ranges from fire codes that inadvertently shape traffic deaths, to nonprofit fundraising strategies borrowed from church capital campaigns, to why coordination is scarcer than money in philanthropy.–Full transcript available here: www.complexsystemspodcast.com/bits-and-bricks-oliver-habryka/–Sponsor: MercuryThis episode is brought to you by Mercury, the fintech trusted by 200K+ companies — from first milestones to running complex systems. Mercury offers banking that truly understands startups and scales with them. Start today at Mercury.comMercury is a financial technology company, not a bank. Banking services provided by Choice Financial Group, Column N.A., and Evolve Bank & Trust; Members FDIC.–Links:Lightcone Infrastructure: https://www.lightconeinfrastructure.com/ Lighthaven: https://www.lighthaven.space/LessWrong: https://www.lesswrong.com/ –Timestamps:(00:00) Intro(01:08) The origins and evolution of LessWrong(03:54) Challenges of running an online forum(05:57) Reviving LessWrong(14:51) The unique structure of Lighthaven(17:35) The complexities of conference venues(19:14) Sponsor: Mercury(20:14) The realities of conference planning(25:32) Challenges of maintaining Lighthaven(29:54) Navigating permits and regulations(37:02) Impact of fire code regulations on traffic fatalities(39:06) Economic analysis of safety regulations(41:39) Housing policy and construction in Berkeley(43:30) Fundraising challenges in the nonprofit sector(46:44) Effective altruism and fundraising dynamics(54:20) Lessons from religious fundraising practices(01:05:36) Reflections on fundraising(01:13:26) Wrap
We continue discussing Nostalgebraist's “The Void” in the context of how to relate to LLMs. If God imagines Claude hard enough, does Claude become real? LINKS The Void Audio reading of The Void, from AskWho The referenced episode where the three of us spoke of Janus's post “Simulators” Claude-Clark post – Simulacra Welfare: Meet Clark, by […]
EPISODE 146 | The Bicameral World of the Zizians One striking thing about the many stories that have appeared about the Zizians and the crimes they are accused of committing, is that each one starts at a different place. Some start with the attack on a Vallejo landlord that resulted in his being run through with a samurai sword and then work backwards and then forwards. Others begin their tale with the shooting of an older couple in Pennsylvania. Still others kick things off with the shooting death of a Border Patrol agent up near the Canadian border. As a result, it can be a bit difficult to get a handle on exactly what happened when and what the several people currently in police custody are accused of. Like what we do? Then buy us a beer or three via our page on Buy Me a Coffee. Review us here or on IMDb. And seriously, subscribe, will ya? Like, just do it. SECTIONS LEFT BRAIN 02:28 - Gimme Some Truth - Ziz LaSota, Effective Altruism, x-risk, MIRI, transhumanism and the Singularity, CFAR, LessWrong 12:25 - Digital Witness - Roko's Basilisk, “I Have No Mouth and I Must Scream”, utilitarianism, the many-worlds interpretation, online censorship 22:58 - Adrift in Sleepwakefulness - The Zizians start as vegan anarchotranshumanists, unbucketing, the bicameral mind, unihemispheric sleep (UHS), sleep deprivation, Ziz Theory, the Rationalist Fleet, Curtis Lind offers a place to stay, the first suicide, self-blackmail RIGHT BRAIN 35:02 - Friendship Train - The Westminster Woods protest, arrests and a lack of cooperation, Ziz dies, Curtis Lind is stabbed repeatedly, Ziz is alive, Richard and Rita Zajko are killed; Michelle Zajko, Daniel Blank and Ziz arrested; Michelle blames LessWrong, Ziz is released and vanishes, more legal issues 50:19 - Lose Control - Ophelia Bauckholt and Teresa Youngblut wander around Vermont, a firefight with the Border Patrol, Curtis Lind is killed, Maxmilian Snyder dictates a letter; Zajko, Blank and Ziz arrested (again); trials are set Music by Fanette Ronjat More Info LessWrong on RationalWIki Roko's Basilisk on RationalWiki Zizian Murdercult summary, for those out of the loop on X by @Aella_Girl - January 29, 2025 Who is ‘Ziz'? How a mysterious group with roots in the Bay Area is linked to six deaths in the San Francisco Chronicle ‘Death upon death': Defendant in killing tied to cult-like ‘Zizian' group dictates 1,500-word letter over jail phone in the San Francisco Chronicle How a Vermont border agent's death exposed violence linked to the cultlike Zizian group on CBS News A Vermont border agent's death was the latest violence linked to the cultlike Zizian group on AP Alleged leader of cultlike ‘Zizian' group to be held without bail after arrest in The Guardian Zizians: What we know about the 'cult' linked to six deaths on the BBC The Delirious, Violent, Impossible True Story of the Zizians by Evan Ratliff in Wired Alleged Leader of Roko's Basilisk Murder Cult Says She Did Nothing Wrong, and Would Appreciate Some Vegan Food in Jail on Futurism Who Are the Zizians: Why 6 Killings Are Linked to Alleged Vegan Techie "Cult" on E! News What to Know About the Alleged Zizian "Cult" Linked to 6 Killings on E! News Possible Suicide Cluster Linked to Zizian Group, on Top of Killings on SFist Judge confirms trial date for ‘Zizian cult' murder case on Courthouse News Service Grand jury indicts accused leader of cultlike 'Zizian' group on USA Today She Wanted to Save the World From A.I. Then the Killings Started in the New York Times Three Zizians face trial together in Maryland amid sprawling federal investigation on AP Follow us on social: Facebook X (Twitter) Other Podcasts by Derek DeWitt DIGITAL SIGNAGE DONE RIGHT - Winner of a Gold Quill Award, Gold MarCom Award, AVA Digital Award Gold, Silver Davey Award, and Communicator Award of Excellence, and on numerous top 10 podcast lists. PRAGUE TIMES - A city is more than just a location - it's a kaleidoscope of history, places, people and trends. This podcast looks at Prague, in the center of Europe, from a number of perspectives, including what it is now, what is has been and where it's going. It's Prague THEN, Prague NOW, Prague LATER
Audio reading of The Void, from AskWho
We discuss Nostalgebraist's “The Void” in the context of how to relate to LLMs. “When you talk to ChatGPT, who or what are you talking to?” LINKS The Void Audio reading of The Void, from AskWho The referenced episode where the three of us spoke of Janus's post “Simulators” The Measure of a Man episode […]
Do we need to be concerned for the welfare of AIs today? What about the near future? Eleos AI Research is asking exactly that. LINKS Eleos AI Research People for the Ethical Treatment of Reinforcement Learners Bees Can't Suffer? Lena, by qntm When AI Seems Conscious Experience Machines, Rob's substack The War on General Computation […]
Andrew Willsen tells us how incorporating as a church allows you to navigate modernity, and gives us the basic steps to doing so. LINKS Andrew's church substack – The Church of the Infinite Game To incorporate in CA file ARTS-PB-501(c)(3) … Continue reading →
My guest is Aella, a writer, blogger and sex worker. She writes the highly popular blog Knowingless. She is a member of the online community LessWrong. We discuss the emergence of AI, the sex industry (online and offline), robot gfs and libertarian transhumanism. Aella describes her evangelical religious upbringing and how she learned to navigate the secular world. We explore the overlap of niche online politics and fetishes, the Woke Wars, debate culture and free speech. You can get access to the full catalog for Doomscroll and more by becoming a paid supporter: www.patreon.com/joshuacitarella joshuacitarella.substack.com/subscribe
Earlier this year a Border Patrol officer was killed in a shoot-out with people who have been described as members of a trans vegan AI death cult. But who are the Zizians, really? Robert sits down with David Gborie to trace their development, from part of the Bay Area Rationalist subculture to killers. (4 Part series) Sources: https://medium.com/@sefashapiro/a-community-warning-about-ziz-76c100180509 https://web.archive.org/web/20230201130318/https://sinceriously.fyi/rationalist-fleet/ https://knowyourmeme.com/memes/infohazard https://web.archive.org/web/20230201130316/https://sinceriously.fyi/net-negative/ Wayback Machine The Zizians Spectral Sight True Hero Contract Schelling Orders – Sinceriously Glossary – Sinceriously https://web.archive.org/web/20230201130330/https://sinceriously.fyi/my-journey-to-the-dark-side/ https://web.archive.org/web/20230201130302/https://sinceriously.fyi/glossary/#zentraidon https://web.archive.org/web/20230201130259/https://sinceriously.fyi/vampires-and-more-undeath/ https://web.archive.org/web/20230201130316/https://sinceriously.fyi/net-negative/ https://web.archive.org/web/20230201130318/https://sinceriously.fyi/rationalist-fleet/ https://x.com/orellanin?s=21&t=F-n6cTZFsKgvr1yQ7oHXRg https://zizians.info/ according to The Boston Globe Inside the ‘Zizians’: How a cultish crew of radical vegans became linked to killings across the United States | The Independent Silicon Valley ‘Rationalists’ Linked to 6 Deaths The Delirious, Violent, Impossible True Story of the Zizians | WIRED Good Group and Pasek’s Doom – Sinceriously Glossary – Sinceriously Mana – Sinceriously Effective Altruism’s Problems Go Beyond Sam Bankman-Fried - Bloomberg The Zizian Facts - Google Docs Several free CFAR summer programs on rationality and AI safety - LessWrong 2.0 viewer This guy thinks killing video game characters is immoral | Vox Inadequate Equilibria: Where and How Civilizations Get Stuck Eliezer Yudkowsky comments on On Terminal Goals and Virtue Ethics - LessWrong 2.0 viewer Effective Altruism’s Problems Go Beyond Sam Bankman-Fried - Bloomberg SquirrelInHell: Happiness Is a Chore PLUM OF DISCORD — I Became a Full-time Internet Pest and May Not... Roko Harassment of PlumOfDiscord Composited – Sinceriously Intersex Brains And Conceptual Warfare – Sinceriously Infohazardous Glossary – Sinceriously SquirrelInHell-Decision-Theory-and-Suicide.pdf - Google Drive The Matrix is a System – Sinceriously A community alert about Ziz. Police investigations, violence, and… | by SefaShapiro | Medium Intersex Brains And Conceptual Warfare – Sinceriously A community alert about Ziz. Police investigations, violence, and… | by SefaShapiro | Medium PLUM OF DISCORD (Posts tagged cw-abuse) Timeline: Violence surrounding the Zizians leading to Border Patrol agent shooting See omnystudio.com/listener for privacy information.