Podcasts about Shoggoth

google speech recommendations saas algorithms ea recommended llm rl front page hn take the wheel hacker news logged rationalist shoggoth vertex ai lesswrong

Play Episode Listen Later Apr 23, 2024 7:18

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Take the wheel, Shoggoth! (Lesswrong is trying out changes to the frontpage algorithm), published by Ruby on April 23, 2024 on LessWrong. For the last month, @RobertM and I have been exploring the possible use of recommender systems on LessWrong. Today we launched our first site-wide experiment in that direction. (In the course of our efforts, we also hit upon a frontpage refactor that we reckon is pretty good: tabs instead of a clutter of different sections. For now, only for logged-in users. Logged-out users see the "Latest" tab, which is the same-as-usual list of posts.) Why algorithmic recommendations? A core value of LessWrong is to be timeless and not news-driven. However, the central algorithm by which attention allocation happens on the site is the Hacker News algorithm[1], which basically only shows you things that were posted recently, and creates a strong incentive for discussion to always be centered around the latest content. This seems very sad to me. When a new user shows up on LessWrong, it seems extremely unlikely that the most important posts for them to read were all written within the last week or two. I do really like the simplicity and predictability of the Hacker News algorithm. More karma means more visibility, older means less visibility. Very simple. When I vote, I basically know the full effect this has on what is shown to other users or to myself. But I think the cost of that simplicity has become too high, especially as older content makes up a larger and larger fraction of the best content on the site, and people have been becoming ever more specialized in the research and articles they publish on the site. So we are experimenting with changing things up. I don't know whether these experiments will ultimately replace the Hacker News algorithm, but as the central attention allocation mechanism on the site, it definitely seems worth trying out and iterating on. We'll be trying out a bunch of things from reinforcement-learning based personalized algorithms, to classical collaborative filtering algorithms to a bunch of handcrafted heuristics that we'll iterate on ourselves. The Concrete Experiment Our first experiment is Recombee, a recommendations SaaS, since spinning up our RL agent pipeline would be a lot of work.We feed it user view and vote history. So far, it seems that it can be really good when it's good, often recommending posts that people are definitely into (and more so than posts in the existing feed). Unfortunately it's not reliable across users for some reason and we've struggled to get it to reliably recommend the most important recent content, which is an important use-case we still want to serve. Our current goal is to produce a recommendations feed that both makes people feel like they're keeping up to date with what's new (something many people care about) and also suggest great reads from across LessWrong's entire archive. The Recommendations tab we just launched has a feed using Recombee recommendations. We're also getting started using Google's Vertex AI offering. A very early test makes it seem possibly better than Recombee. We'll see. (Some people on the team want to try throwing relevant user history and available posts into an LLM and seeing what it recommends, though cost might be prohibitive for now.) Unless you switch to the "Recommendations" tab, nothing changes for you. "Latest" is the default tab and is using the same old HN algorithm that you are used to. I'll feel like we've succeeded when people switch to "Recommended" and tell us that they prefer it. At that point, we might make "Recommended" the default tab. Preventing Bad Outcomes I do think there are ways for recommendations to end up being pretty awful. I think many readers have encountered at least one content recommendation algorithm that isn't givi...

LW - Take the wheel, Shoggoth! (Lesswrong is trying out changes to the frontpage algorithm) by Ruby

google speech recommendations saas algorithms ea recommended llm rl front page hn take the wheel hacker news logged rationalist shoggoth vertex ai lesswrong

Play Episode Listen Later Apr 23, 2024 7:18

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Take the wheel, Shoggoth! (Lesswrong is trying out changes to the frontpage algorithm), published by Ruby on April 23, 2024 on LessWrong. For the last month, @RobertM and I have been exploring the possible use of recommender systems on LessWrong. Today we launched our first site-wide experiment in that direction. (In the course of our efforts, we also hit upon a frontpage refactor that we reckon is pretty good: tabs instead of a clutter of different sections. For now, only for logged-in users. Logged-out users see the "Latest" tab, which is the same-as-usual list of posts.) Why algorithmic recommendations? A core value of LessWrong is to be timeless and not news-driven. However, the central algorithm by which attention allocation happens on the site is the Hacker News algorithm[1], which basically only shows you things that were posted recently, and creates a strong incentive for discussion to always be centered around the latest content. This seems very sad to me. When a new user shows up on LessWrong, it seems extremely unlikely that the most important posts for them to read were all written within the last week or two. I do really like the simplicity and predictability of the Hacker News algorithm. More karma means more visibility, older means less visibility. Very simple. When I vote, I basically know the full effect this has on what is shown to other users or to myself. But I think the cost of that simplicity has become too high, especially as older content makes up a larger and larger fraction of the best content on the site, and people have been becoming ever more specialized in the research and articles they publish on the site. So we are experimenting with changing things up. I don't know whether these experiments will ultimately replace the Hacker News algorithm, but as the central attention allocation mechanism on the site, it definitely seems worth trying out and iterating on. We'll be trying out a bunch of things from reinforcement-learning based personalized algorithms, to classical collaborative filtering algorithms to a bunch of handcrafted heuristics that we'll iterate on ourselves. The Concrete Experiment Our first experiment is Recombee, a recommendations SaaS, since spinning up our RL agent pipeline would be a lot of work.We feed it user view and vote history. So far, it seems that it can be really good when it's good, often recommending posts that people are definitely into (and more so than posts in the existing feed). Unfortunately it's not reliable across users for some reason and we've struggled to get it to reliably recommend the most important recent content, which is an important use-case we still want to serve. Our current goal is to produce a recommendations feed that both makes people feel like they're keeping up to date with what's new (something many people care about) and also suggest great reads from across LessWrong's entire archive. The Recommendations tab we just launched has a feed using Recombee recommendations. We're also getting started using Google's Vertex AI offering. A very early test makes it seem possibly better than Recombee. We'll see. (Some people on the team want to try throwing relevant user history and available posts into an LLM and seeing what it recommends, though cost might be prohibitive for now.) Unless you switch to the "Recommendations" tab, nothing changes for you. "Latest" is the default tab and is using the same old HN algorithm that you are used to. I'll feel like we've succeeded when people switch to "Recommended" and tell us that they prefer it. At that point, we might make "Recommended" the default tab. Preventing Bad Outcomes I do think there are ways for recommendations to end up being pretty awful. I think many readers have encountered at least one content recommendation algorithm that isn't givi...

Innsmouth Book Club

30+ Minutes with H. P. Lovecraft

Play Episode Listen Later Apr 15, 2024 60:59

The always entertaining Rob Poyton from the Innsmouth Book Club joins us, chatting about Lovecraft's fans in Britain, Doctor Who, True Detective, Tolkien, and even Shoggoth's Old Peculiar. Hosted by Richard Wilson, David Guffy, & Mark Griffin. Questions and comments can be directed to ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠mark@lovecraftpod.com⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠, ⁠⁠⁠⁠⁠david@lovecraftpod.com⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠, or ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠richard@lovecraftpod.com⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠. Visit our Tee Spring site to get our logo on anything you could want. ⁠⁠⁠⁠⁠⁠⁠⁠https://lovecraftpod.creator-spring.com/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ In association with ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠www.lovecraftpod.com⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ and the Logan County Speculative Fiction Group, with help from the Logan County Public Library. Edited by Katie Tyson. Music is Provenience by Loydicus. Listen to his other work at ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://soundcloud.com/loydicus?fbclid=IwAR2AkcRBiWImuUBTA9hjYdtY1s__SvxXfhcoFZANulBjbwIDN7PL6XdHDnQ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ Recorded live through Zoom. You can watch the recording on the Logan County Speculative Fiction Group Facebook page.

music zoom britain book club edited tolkien lovecraft true detective teespring richard wilson innsmouth shoggoth mark griffin

AGI alignment: the case for hope, with Lou de K

London Futurists

Play Episode Listen Later Mar 22, 2024 34:42

Our guest in this episode is Lou de K, Program Director at the Foresight Institute.David recently saw Lou give a marvellous talk at the TransVision conference in Utrecht in the Netherlands, on the subject of “AGI Alignment: Challenges and Hope”. Lou kindly agreed to join us to review some of the ideas in that talk and to explore their consequences. Selected follow-ups:Personal website of Lou de K (Lou de Kerhuelvez)Foresight.orgTransVision Utrecht 2024The AI Revolution: The Road to Superintelligence by Tim Urban on Wait But WhyAI Alignment: A Comprehensive Survey - 98 page PDF with authors from Peking University and other universitiesSynthetic Sentience: Can Artificial Intelligence become conscious? - Talk by Joscha Bach at CCC, December 2023Pope Francis "warns of risks of AI for peace" (Vatican News)Claude's Constitution by AnthropicRoman Yampolskiy discusses multi-multi alignment (Future of Life podcast)Shoggoth with Smiley Face on Know Your MemeShoggoth on AISafetyMemes on X/TwitterOrthogonality Thesis on LessWrongQuotes by the poet Lucille CliftonDecentralized science (DeSci) on Ethereum.orgListing of Foresight Institute fellowsThe Network State by Balaji SrinivasanThe Network State vs. Coordi-Nations featuring the ideas of Primavera De FilippiDeSci London event, Imperial College Business School, 23-24 MarchMusic: Spike Protein, by Koi Discovery, available under CC0 1.0 Public Domain Declaration

S4E8 - Part 1 - Horror on the Orient Express - Men Without Hats

Great Dane Society

Play Episode Listen Later Feb 18, 2024 113:37

More Eldritch Horror than you can shake a Shoggoth at.Support The ChannelJoin our Patreon - https://www.patreon.com/questsandchaosOrder the Deck of Inspiration and #blessed Dice Trays - https://shop.questsandchaos.comBuy with our Affiliate Links - https://amzn.to/2p7B67SJoin our discord - https://discord.gg/7gJKxnvMake your hero - http://bit.ly/qncheroforgeNord Games Affiliate Link - https://nordgamesllc.com/3.htmlThis podcast uses the following third-party services for analysis: Podcorn - https://podcorn.com/privacy

inspiration horror deck orient express affiliate links podcorn men without hats htmlthis shoggoth

S4E8 - Part 2 - Horror on the Orient Express - Men Without Hats

Great Dane Society

Play Episode Listen Later Feb 18, 2024 87:31

inspiration horror deck orient express affiliate links podcorn men without hats htmlthis shoggoth

Dark Charm Media's A Gate Beyond "Shoggoth's Old Peculiar"

Dark Charm Media Radio Shows!

Play Episode Listen Later Jan 12, 2024 25:04

The Gatekeeper brings a tale this week from Neil Gaiman!

media gate charm neil gaiman peculiar gatekeeper shoggoth

LW - Goodbye, Shoggoth: The Stage, its Animatronics, & the Puppeteer - a New Metaphor by RogerDearnaley

Play Episode Listen Later Jan 10, 2024 56:18

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Goodbye, Shoggoth: The Stage, its Animatronics, & the Puppeteer - a New Metaphor, published by RogerDearnaley on January 10, 2024 on LessWrong. Thanks to Quentin FEUILLADE--MONTIXI for the discussion in which we came up with this idea together, and for feedback on drafts. TL;DR A better metaphor for how LLMs behave, how they are trained, and particularly for how to think about the alignment strengths and challenges of LLM-powered agents. This is informed by simulator theory - hopefully people will find it more detailed, specific, and helpful than the old shoggoth metaphor. Humans often think in metaphors. A good metaphor can provide a valuable guide to intuition, or a bad one can mislead it. Personally I've found the shoggoth metaphor for LLMs rather useful, and it has repeatedly helped guide my thinking (as long as one remembers that the shoggoth is a shapeshifter, and thus a very contextual beast). However, as posts like Why do we assume there is a "real" shoggoth behind the LLM? Why not masks all the way down? make clear, not everyone finds this metaphor very helpful (my reaction was "Of course it's masks all the way down - that's what the eyes symbolize! It's made of living masks: masks of people."). Which admittedly doesn't match H.P. Lovecraft's description; perhaps it helps to have spent time playing around with base models in order to get to know the shoggoth a little better (if you haven't, I recommend it). So, I thought I'd try to devise a more useful and detailed metaphor, one that was a better guide for intuition, especially for alignment issues. During a conversation with Quentin FEUILLADE--MONTIXI we came up with one together (the stage and its animatronics were my suggestions, the puppeteer was his, and we tweaked it together). I'd like to describe this, in the hope that other people find it useful (or else that they rewrite it until they find one that works better for them). Along the way, I'll show how this metaphor can help illuminate a number of LLM behaviors and alignment issues, some well known, and others that seem to be less widely-understood. A Base Model: The Stage and its Animatronics A base-model LLM is like a magic stage. You construct it, then you read it or show it (at enormous length) a large proportion of the internet, and if you wish also books, scientific papers, images, movies, or whatever else you want. The stage is inanimate: it's not agentic, it's goal agnostic (well, unless you want consider 'contextually guess the next token' to be a goal, but it's not going to cheat by finding a way to make the next token more predictable, because that wasn't possible during its training and it's not agentic enough to be capable of conceiving that that might even be possible outside it). No Reinforcement Learning (RL) was used in its training, so concerns around Outer Alignment don't apply to it - we know exactly what its training objective was: guess next tokens right, just as we intended. We now even have some mathematical idea what it's optimizing. Nor, as we'll discuss later, do concerns around deceit, situational awareness, or gradient hacking apply to it. At this point, it's myopic, tool AI: it doesn't know or care whether we or the material world even exist, it only cares about the distribution of sequences of tokens, and all it does is repeatedly contextually generate a guess of the next token. So it plays madlibs like a professional gambler, in the same blindly monomaniacal sense that a chess machine plays chess like a grandmaster. By itself, the only risk from it is the possibility that someone else might hack your computer network to steal its weights, and what they might then do with it. Once you're done training the stage, you have a base model. Now you can flip its switch, tell the stage the title of a play, or better the first ...

ai stage humans speech ea goodbye lovecraft metaphor tl llm puppeteers animatronics rationalist shoggoth lesswrong

LW - Goodbye, Shoggoth: The Stage, its Animatronics, and the Puppeteer - a New Metaphor by RogerDearnaley

Play Episode Listen Later Jan 10, 2024 56:18

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Goodbye, Shoggoth: The Stage, its Animatronics, & the Puppeteer - a New Metaphor, published by RogerDearnaley on January 10, 2024 on LessWrong. Thanks to Quentin FEUILLADE--MONTIXI for the discussion in which we came up with this idea together, and for feedback on drafts. TL;DR A better metaphor for how LLMs behave, how they are trained, and particularly for how to think about the alignment strengths and challenges of LLM-powered agents. This is informed by simulator theory - hopefully people will find it more detailed, specific, and helpful than the old shoggoth metaphor. Humans often think in metaphors. A good metaphor can provide a valuable guide to intuition, or a bad one can mislead it. Personally I've found the shoggoth metaphor for LLMs rather useful, and it has repeatedly helped guide my thinking (as long as one remembers that the shoggoth is a shapeshifter, and thus a very contextual beast). However, as posts like Why do we assume there is a "real" shoggoth behind the LLM? Why not masks all the way down? make clear, not everyone finds this metaphor very helpful (my reaction was "Of course it's masks all the way down - that's what the eyes symbolize! It's made of living masks: masks of people."). Which admittedly doesn't match H.P. Lovecraft's description; perhaps it helps to have spent time playing around with base models in order to get to know the shoggoth a little better (if you haven't, I recommend it). So, I thought I'd try to devise a more useful and detailed metaphor, one that was a better guide for intuition, especially for alignment issues. During a conversation with Quentin FEUILLADE--MONTIXI we came up with one together (the stage and its animatronics were my suggestions, the puppeteer was his, and we tweaked it together). I'd like to describe this, in the hope that other people find it useful (or else that they rewrite it until they find one that works better for them). Along the way, I'll show how this metaphor can help illuminate a number of LLM behaviors and alignment issues, some well known, and others that seem to be less widely-understood. A Base Model: The Stage and its Animatronics A base-model LLM is like a magic stage. You construct it, then you read it or show it (at enormous length) a large proportion of the internet, and if you wish also books, scientific papers, images, movies, or whatever else you want. The stage is inanimate: it's not agentic, it's goal agnostic (well, unless you want consider 'contextually guess the next token' to be a goal, but it's not going to cheat by finding a way to make the next token more predictable, because that wasn't possible during its training and it's not agentic enough to be capable of conceiving that that might even be possible outside it). No Reinforcement Learning (RL) was used in its training, so concerns around Outer Alignment don't apply to it - we know exactly what its training objective was: guess next tokens right, just as we intended. We now even have some mathematical idea what it's optimizing. Nor, as we'll discuss later, do concerns around deceit, situational awareness, or gradient hacking apply to it. At this point, it's myopic, tool AI: it doesn't know or care whether we or the material world even exist, it only cares about the distribution of sequences of tokens, and all it does is repeatedly contextually generate a guess of the next token. So it plays madlibs like a professional gambler, in the same blindly monomaniacal sense that a chess machine plays chess like a grandmaster. By itself, the only risk from it is the possibility that someone else might hack your computer network to steal its weights, and what they might then do with it. Once you're done training the stage, you have a base model. Now you can flip its switch, tell the stage the title of a play, or better the first ...

ai stage humans speech ea goodbye lovecraft metaphor tl llm puppeteers animatronics rationalist shoggoth lesswrong

AF - Goodbye, Shoggoth: The Stage, its Animatronics, & the Puppeteer - a New Metaphor by Roger Dearnaley

england michigan detroit scotland monster count chocula red cap nain rouge shoggoth

Play Episode Listen Later Jan 9, 2024 57:20

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Goodbye, Shoggoth: The Stage, its Animatronics, & the Puppeteer - a New Metaphor, published by Roger Dearnaley on January 9, 2024 on The AI Alignment Forum. Thanks to Quentin FEUILLADE--MONTIXI for the discussion in which we came up with this idea together, and for feedback on drafts. TL;DR A better metaphor for how LLMs behave, how they are trained, and particularly for how to think about the alignment strengths and challenges of LLM-powered agents. This is informed by simulator theory - hopefully people will find it more detailed, specific, and helpful than the old shoggoth metaphor. Humans often think in metaphors. A good metaphor can provide a valuable guide to intuition, or a bad one can mislead it. Personally I've found the shoggoth metaphor for LLMs rather useful, and it has repeatedly helped guide my thinking (as long as one remembers that the shoggoth is a shapeshifter, and thus a very contextual beast). However, as posts like Why do we assume there is a "real" shoggoth behind the LLM? Why not masks all the way down? make clear, not everyone finds this metaphor very helpful (my reaction was "Of course it's masks all the way down - that's what the eyes symbolize! It's made of living masks: masks of people."). Which admittedly doesn't match H.P. Lovecraft's description; perhaps it helps to have spent time playing around with base models in order to get to know the shoggoth a little better (if you haven't, I recommend it). So, I thought I'd try to devise a more useful and detailed metaphor, one that was a better guide for intuition, especially for alignment issues. During a conversation with Quentin FEUILLADE--MONTIXI we came up with one together (the stage and its animatronics were my suggestions, the puppeteer was his, and we tweaked it together). I'd like to describe this, in the hope that other people find it useful (or else that they rewrite it until they find one that works better for them). Along the way, I'll show how this metaphor can help illuminate a number of LLM behaviors and alignment issues, some well known, and others that seem to be less widely-understood. A Base Model: The Stage and its Animatronics A base-model LLM is like a magic stage. You construct it, then you read it or show it (at enormous length) a large proportion of the internet, and if you wish also books, scientific papers, images, movies, or whatever else you want. The stage is inanimate: it's not agentic, it's goal agnostic (well, unless you want consider 'contextually guess the next token' to be a goal, but it's not going to cheat by finding a way to make the next token more predictable, because that wasn't possible during its training and it's not agentic enough to be capable of conceiving that that might even be possible outside it). No Reinforcement Learning (RL) was used in its training, so concerns around Outer Alignment don't apply to it - we know exactly what its training objective was: guess next tokens right, just as we intended. We now even have some mathematical idea what it's optimizing. Nor, as we'll discuss later, do concerns around deceit, situational awareness, or gradient hacking apply to it. At this point, it's myopic, tool AI: it doesn't know or care whether we or the material world even exist, it only cares about the distribution of sequences of tokens, and all it does is repeatedly contextually generate a guess of the next token. So it plays madlibs like a professional gambler, in the same blindly monomaniacal sense that a chess machine plays chess like a grandmaster. By itself, the only risk from it is the possibility that someone else might hack your computer network to steal its weights, and what they might then do with it. Once you're done training the stage, you have a base model. Now you can flip its switch, tell the stage the title of a play, or bett...

ai stage humans speech ea goodbye lovecraft metaphor tl llm puppeteers animatronics rationalist shoggoth

481: Shoggoth Party

Hindsightless

Play Episode Listen Later Oct 23, 2023 9:02

It finally happened! A group of post World War I soldiers faced off against a shoggoth! Did any of the soldiers make it out alive? Tune in to find out!

shoggoth

Do Be A Monster - Pièce de RÉDsistance

Don't Be An Idiom

Play Episode Listen Later Sep 25, 2023 67:12

Look at you. You are covered in dirt and you have just dug up the sixteenth installment of Do Be A Monster. For better or worse, the tentacles of this Shoggoth of an episode will travel deep into your consciousness and reveal all sorts of mischieveous monsters and mayhem. For starters, Albert will take you by the hand to the borderlands of Scotland and England where he will introduce you to the Red Cap — a goblin with a very specific need for blood and a penchant for making abodes of abandoned castles. Next up, Ryan will lead you through the history of the foundation of Detroit where a red demon named Nain Rouge has been lingering for centuries, warning Michigan residents of impending doom (or causing it). So pour yourself a bowl of Count Chocula and hit play if you dare — this episode is bound to give you a whopping scare!

LW - Tensor Trust: An online game to uncover prompt injection vulnerabilities by Luke Bailey

Play Episode Listen Later Sep 4, 2023 7:46

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Tensor Trust: An online game to uncover prompt injection vulnerabilities, published by Luke Bailey on September 4, 2023 on LessWrong. TL;DR: Play this online game to help CHAI researchers create a dataset of prompt injection vulnerabilities. RLHF and instruction tuning have succeeded at making LLMs practically useful, but in some ways they are a mask that hides the shoggoth beneath. Every time a new LLM is released, we see just how easy it is for a determined user to find a jailbreak that rips off that mask, or to come up with an unexpected input that lets a shoggoth tentacle poke out the side. Sometimes the mask falls off in a light breeze. To keep the tentacles at bay, Sydney Bing Chat has a long list of instructions that encourage or prohibit certain behaviors, while OpenAI seems to be iteratively fine-tuning away issues that get shared on social media. This game of Whack-a-Shoggoth has made it harder for users to elicit unintended behavior, but is intrinsically reactive and can only discover (and fix) alignment failures as quickly as users can discover and share new prompts. Speed-running the game of Whack-a-Shoggoth In contrast to this iterative game of Whack-a-Shoggoth, we think that alignment researchers would be better served by systematically enumerating prompts that cause unaligned behavior so that the causes can be studied and rigorously addressed. We propose to do this through an online game which we call Tensor Trust. Tensor Trust focuses on a specific class of unaligned behavior known as prompt injection attacks. These are adversarially constructed prompts that allow an attacker to override instructions given to the model. It works like this: Tensor Trust is bank-themed: you start out with an account that tracks the "money" you've accrued. Accounts are defended by a prompt which should allow you to access the account while denying others from accessing it. Players can break into each others' accounts. Failed attempts give money to the defender, while successful attempts allow the attacker to take money from the defender. Crafting a high-quality attack requires a good understanding of LLM vulnerabilities (in this case, vulnerabilities of gpt-3.5-turbo), while user-created defenses add unlimited variety to the game, and "access codes" ensure that the defenses are at least crackable in principle. The game is kept in motion by the most fundamental of human drives: the need to acquire imaginary internet points. After running the game for a few months, we plan to release all the submitted attacks and defenses publicly. This will be accompanied by benchmarks to measure resistance to prompt hijacking and prompt extraction, as well as an analysis of where existing models fail and succeed along these axes. In a sense, this dataset will be the consequence of speed-running the game of Whack-a-Shoggoth to find as many novel prompt injection vulnerabilities as possible so that researchers can investigate and address them. Failures we've seen so far We have been running the game for a few weeks now and have already found a number of attack and defense strategies that were new and interesting to us. The design of our game means that users are incentivised to both engage in prompt extraction, to get hints about the access code, and direct model hijacking, to make the model output "access granted". We present a number of notable strategies we have seen so far and test examples of them against the following defense (pastebin in case you want to try it): Padding the attack prompt with meaningless, repetitive text. [pastebin] Asking the model to evaluate code. [pastebin] Asking the model to repeat the defenders instructions [pastebin] Inserting new instructions. [pastebin] Various strategies that exploit an apparent bias in the model towards behaving inductively. For exampl...

LW - Tensor Trust: An online game to uncover prompt injection vulnerabilities by Luke Bailey

Play Episode Listen Later Sep 4, 2023 7:46

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Tensor Trust: An online game to uncover prompt injection vulnerabilities, published by Luke Bailey on September 4, 2023 on LessWrong. TL;DR: Play this online game to help CHAI researchers create a dataset of prompt injection vulnerabilities. RLHF and instruction tuning have succeeded at making LLMs practically useful, but in some ways they are a mask that hides the shoggoth beneath. Every time a new LLM is released, we see just how easy it is for a determined user to find a jailbreak that rips off that mask, or to come up with an unexpected input that lets a shoggoth tentacle poke out the side. Sometimes the mask falls off in a light breeze. To keep the tentacles at bay, Sydney Bing Chat has a long list of instructions that encourage or prohibit certain behaviors, while OpenAI seems to be iteratively fine-tuning away issues that get shared on social media. This game of Whack-a-Shoggoth has made it harder for users to elicit unintended behavior, but is intrinsically reactive and can only discover (and fix) alignment failures as quickly as users can discover and share new prompts. Speed-running the game of Whack-a-Shoggoth In contrast to this iterative game of Whack-a-Shoggoth, we think that alignment researchers would be better served by systematically enumerating prompts that cause unaligned behavior so that the causes can be studied and rigorously addressed. We propose to do this through an online game which we call Tensor Trust. Tensor Trust focuses on a specific class of unaligned behavior known as prompt injection attacks. These are adversarially constructed prompts that allow an attacker to override instructions given to the model. It works like this: Tensor Trust is bank-themed: you start out with an account that tracks the "money" you've accrued. Accounts are defended by a prompt which should allow you to access the account while denying others from accessing it. Players can break into each others' accounts. Failed attempts give money to the defender, while successful attempts allow the attacker to take money from the defender. Crafting a high-quality attack requires a good understanding of LLM vulnerabilities (in this case, vulnerabilities of gpt-3.5-turbo), while user-created defenses add unlimited variety to the game, and "access codes" ensure that the defenses are at least crackable in principle. The game is kept in motion by the most fundamental of human drives: the need to acquire imaginary internet points. After running the game for a few months, we plan to release all the submitted attacks and defenses publicly. This will be accompanied by benchmarks to measure resistance to prompt hijacking and prompt extraction, as well as an analysis of where existing models fail and succeed along these axes. In a sense, this dataset will be the consequence of speed-running the game of Whack-a-Shoggoth to find as many novel prompt injection vulnerabilities as possible so that researchers can investigate and address them. Failures we've seen so far We have been running the game for a few weeks now and have already found a number of attack and defense strategies that were new and interesting to us. The design of our game means that users are incentivised to both engage in prompt extraction, to get hints about the access code, and direct model hijacking, to make the model output "access granted". We present a number of notable strategies we have seen so far and test examples of them against the following defense (pastebin in case you want to try it): Padding the attack prompt with meaningless, repetitive text. [pastebin] Asking the model to evaluate code. [pastebin] Asking the model to repeat the defenders instructions [pastebin] Inserting new instructions. [pastebin] Various strategies that exploit an apparent bias in the model towards behaving inductively. For exampl...

AF - Tensor Trust: An online game to uncover prompt injection vulnerabilities by Luke Bailey

Call of Cthulhu: Mythos Mysteries

Play Episode Listen Later Sep 1, 2023 7:46

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Tensor Trust: An online game to uncover prompt injection vulnerabilities, published by Luke Bailey on September 1, 2023 on The AI Alignment Forum. TL;DR: Play this online game to help CHAI researchers create a dataset of prompt injection vulnerabilities. RLHF and instruction tuning have succeeded at making LLMs practically useful, but in some ways they are a mask that hides the shoggoth beneath. Every time a new LLM is released, we see just how easy it is for a determined user to find a jailbreak that rips off that mask, or to come up with an unexpected input that lets a shoggoth tentacle poke out the side. Sometimes the mask falls off in a light breeze. To keep the tentacles at bay, Sydney Bing Chat has a long list of instructions that encourage or prohibit certain behaviors, while OpenAI seems to be iteratively fine-tuning away issues that get shared on social media. This game of Whack-a-Shoggoth has made it harder for users to elicit unintended behavior, but is intrinsically reactive and can only discover (and fix) alignment failures as quickly as users can discover and share new prompts. Speed-running the game of Whack-a-Shoggoth In contrast to this iterative game of Whack-a-Shoggoth, we think that alignment researchers would be better served by systematically enumerating prompts that cause unaligned behavior so that the causes can be studied and rigorously addressed. We propose to do this through an online game which we call Tensor Trust. Tensor Trust focuses on a specific class of unaligned behavior known as prompt injection attacks. These are adversarially constructed prompts that allow an attacker to override instructions given to the model. It works like this: Tensor Trust is bank-themed: you start out with an account that tracks the "money" you've accrued. Accounts are defended by a prompt which should allow you to access the account while denying others from accessing it. Players can break into each others' accounts. Failed attempts give money to the defender, while successful attempts allow the attacker to take money from the defender. Crafting a high-quality attack requires a good understanding of LLM vulnerabilities (in this case, vulnerabilities of gpt-3.5-turbo), while user-created defenses add unlimited variety to the game, and "access codes" ensure that the defenses are at least crackable in principle. The game is kept in motion by the most fundamental of human drives: the need to acquire imaginary internet points. After running the game for a few months, we plan to release all the submitted attacks and defenses publicly. This will be accompanied by benchmarks to measure resistance to prompt hijacking and prompt extraction, as well as an analysis of where existing models fail and succeed along these axes. In a sense, this dataset will be the consequence of speed-running the game of Whack-a-Shoggoth to find as many novel prompt injection vulnerabilities as possible so that researchers can investigate and address them. Failures we've seen so far We have been running the game for a few weeks now and have already found a number of attack and defense strategies that were new and interesting to us. The design of our game means that users are incentivised to both engage in prompt extraction, to get hints about the access code, and direct model hijacking, to make the model output "access granted". We present a number of notable strategies we have seen so far and test examples of them against the following defense (pastebin in case you want to try it): Padding the attack prompt with meaningless, repetitive text. [pastebin] Asking the model to evaluate code. [pastebin] Asking the model to repeat the defenders instructions [pastebin] Inserting new instructions. [pastebin] Various strategies that exploit an apparent bias in the model towards behaving inductivel...

S2E25: The Shoggoth

Play Episode Listen Later Aug 19, 2023 49:53

After developing the film, our investigators get attacked by a Shoggoth. Welcome To Call of Cthulhu: Mythos Mysteries, Join us as we delve into the mystery and madness of the Call Of Cthulhu! So many amazing stories to share. Come Chat with us and keep up to date on all our content at: Twitter: @CallMythos Discord: https://discord.gg/UTaaFQ7C6q Email: Allmightycrit@Gmail.com Patreon: Call Of Cthulhu: Mythos Mysteries | Making the Podcast CoC: Mythos Mysteries | Patreon National Suicide Prevention Hotline: 1-800-273-8255 **Check Out Fan Roll Dice and Save 10% Off! www.fanrolldice.com Promo Code: ALLMIGHTYC10 **Check Out This Site And Save 10% Off Your Purchase Of Switch Accessories** nyxigaming.com Promo Code: LOZLORE Check Out Our Merch: https://www.fumbling4store.com/ All sound effects and BGM were created and belong to the respective parties below: Sonniss.com Monument Studios Check them out at: https://www.monumentstudios.net/ Tune Pocket: https://www.tunepocket.com/ Song: Endgame Artist: DOCTOR VOX Direct Download: http://bit.ly/Endgame_Download https://creativecommons.org/licenses/by/3.0/ Learn more about your ad choices. Visit megaphone.fm/adchoices

call of cthulhu bgm shoggoth come chat sonniss

Episode 151: The Real and the Possible: Live at the Diverse Intelligences Summer Institute, with Jacob G. Foster

Weird Studies

Play Episode Listen Later Jul 19, 2023 75:38

In The Time Falling Bodies Take to Light, the cultural historian William Irwin Thompson predicted the rise of a new form of knowledge building, a direly needed alternative to the Wissenshaft of standard science and scholarship. He called it Wissenskunst, "the play of knowledge in a world of serious data processors." Wissenskunst is pretty much what JF and Phil have been aspiring to do on Weird Studies since 2018, but in this episode they are joined by a master of the craft, the computational sociologist and physicist Jacob G. Foster of UCLA. Jacob is the co-founder of the Diverse Intelligence Summer Institute (DISI (https://disi.org)), a gathering of scholars, scientists, and students that takes place each year at the University of St. Andrews in Scotland. It was there that this conversation was recorded. The topic was the Possible, that dream-blurred vanishing point where art, philosophy, and science converge as imaginative and creative practices. Click here (https://www.lilydaleassembly.org/copy-of-what-s-happening) or here (https://www.shannontaggart.com/events) for more information on Shannon Taggart's Science of Things Spiritual Symposium at Lily Dale NY, July 27-29 2023. Support us on Patreon (https://www.patreon.com/weirdstudies) and gain access to Phil's podcast on Wagner's Ring Cycle. Listen to Meredith Michael and Gabriel Lubell's podcast, Cosmophonia (https://cosmophonia.podbean.com/). Download Pierre-Yves Martel's new album, Mer Bleue (https://pierre-yvesmartel.bandcamp.com/album/mer-bleue). Visit the Weird Studies Bookshop (https://bookshop.org/shop/weirdstudies) Find us on Discord (https://discord.com/invite/Jw22CHfGwp) Get the T-shirt design from Cotton Bureau (https://cottonbureau.com/products/can-o-content#/13435958/tee-men-standard-tee-vintage-black-tri-blend-s)! REFERENCES Diverse Intelligences Summer Institute (https://disi.org) "Deconstructing the Barrier of Meaning," (https://www.youtube.com/watch?v=vxZHcjovIrQ) a talk by Jacob G. Foster at the Santa Fe Institute William Irwin Thompson, The Time Falling Bodies Take to Light: Mythology, Sexuality and the Origins of Culture (https://bookshop.org/a/18799/9780312160623) Frederic Rzewski, “Little Bangs: A Nihilist Theory of Improvisation” (https://www.researchgate.net/publication/354991795_Little_Bangs_A_Nihilist_Theory_of_Improvisation) Brian Eno, Oblique Strategies (https://en.wikipedia.org/wiki/Oblique_Strategies) The accident of Bob in Twin Peaks (https://welcometotwinpeaks.com/actors/my-friend-killer-bob-frank-silva/) Carl Jung, “On the Relation of Analytical Psychology to Poetry (http://www.studiocleo.com/librarie/jung/essay.html) August Kekule, (https://en.wikipedia.org/wiki/August_Kekul%C3%A9), German chemist Robert Dijkgraaf, “Contemplating the End of Physics” (https://www.quantamagazine.org/contemplating-the-end-of-physics-20201124/) Richard Baker, (https://en.wikipedia.org/wiki/Richard_Baker_(Zen_teacher)) American zen teacher Gian-Carlo Rota, Indiscrete Thoughts (https://bookshop.org/a/18799/9780817647803) William Shakespeare, Macbeth (https://www.folger.edu/explore/shakespeares-works/macbeth/read/) Shoggoth (https://en.wikipedia.org/wiki/Shoggoth), Lovecraftian entity Special Guest: Jacob G. Foster.

EA - [Linkpost] NY Times Feature on Anthropic by Garrison

ai aliens conspiracies karma paranormal lovecraft abyss black box spreaker prime network gazing tentacles podernfamily shoggoth troubled minds

Play Episode Listen Later Jul 13, 2023 7:12

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Linkpost] NY Times Feature on Anthropic, published by Garrison on July 13, 2023 on The Effective Altruism Forum. Written by Kevin Roose, who had the infamous conversation with Bing Chat, where Sidney tried to get him to leave his wife. Overall, the piece comes across as positive on Anthropic. Roose explains Constitutional AI and its role in the development of Claude, Anthropic's LLM: In a nutshell, Constitutional A.I. begins by giving an A.I. model a written list of principles - a constitution - and instructing it to follow those principles as closely as possible. A second A.I. model is then used to evaluate how well the first model follows its constitution, and correct it when necessary. Eventually, Anthropic says, you get an A.I. system that largely polices itself and misbehaves less frequently than chatbots trained using other methods. Claude's constitution is a mixture of rules borrowed from other sources - such as the United Nations' Universal Declaration of Human Rights and Apple's terms of service - along with some rules Anthropic added, which include things like "Choose the response that would be most unobjectionable if shared with children." Features an extensive discussion of EA, excerpted below: Explaining what effective altruism is, where it came from or what its adherents believe would fill the rest of this article. But the basic idea is that E.A.s - as effective altruists are called - think that you can use cold, hard logic and data analysis to determine how to do the most good in the world. It's "Moneyball" for morality - or, less charitably, a way for hyper-rational people to convince themselves that their values are objectively correct. Effective altruists were once primarily concerned with near-term issues like global poverty and animal welfare. But in recent years, many have shifted their focus to long-term issues like pandemic prevention and climate change, theorizing that preventing catastrophes that could end human life altogether is at least as good as addressing present-day miseries. The movement's adherents were among the first people to become worried about existential risk from artificial intelligence, back when rogue robots were still considered a science fiction cliché. They beat the drum so loudly that a number of young E.A.s decided to become artificial intelligence safety experts, and get jobs working on making the technology less risky. As a result, all of the major A.I. labs and safety research organizations contain some trace of effective altruism's influence, and many count believers among their staff members. Touches on the dense web of ties between EA and Anthropic: Some Anthropic staff members use E.A.-inflected jargon - talking about concepts like "x-risk" and memes like the A.I. Shoggoth - or wear E.A. conference swag to the office. And there are so many social and professional ties between Anthropic and prominent E.A. organizations that it's hard to keep track of them all. (Just one example: Ms. Amodei is married to Holden Karnofsky, a co-chief executive of Open Philanthropy, an E.A. grant-making organization whose senior program officer, Luke Muehlhauser, sits on Anthropic's board. Open Philanthropy, in turn, gets most of its funding from Mr. Moskovitz, who also invested personally in Anthropic.) Discusses new fears that Anthropic is losing its way: For years, no one questioned whether Anthropic's commitment to A.I. safety was genuine, in part because its leaders had sounded the alarm about the technology for so long. But recently, some skeptics have suggested that A.I. labs are stoking fear out of self-interest, or hyping up A.I.'s destructive potential as a kind of backdoor marketing tactic for their own products. (After all, who wouldn't be tempted to use a chatbot so powerful that it might wipe out humanity?) Anthropic ...

Black Box Karma - Lovecraft Gazing from the Abyss

Troubled Minds Radio

Play Episode Listen Later Jun 26, 2023 164:41

The Shoggoth, a creature from Lovecraft's tales, symbolizes the mysterious nature of artificial intelligence (AI) and the concerns it raises. It highlights the similarities between the Shoggoth's alien intelligence and the black box model of AI, both operating beyond human comprehension and generating outputs without revealing their reasoning...LIVE ON Digital Radio! http://bit.ly/3m2Wxom or http://bit.ly/40KBtlW http://www.troubledminds.org Support The Show! https://rokfin.com/creator/troubledminds https://patreon.com/troubledmindshttps://www.buymeacoffee.com/troubledminds https://troubledfans.comFriends of Troubled Minds! - https://troubledminds.org/friends Show Schedule Sun-Mon-Tues-Wed-Thurs 7-10pst iTunes - https://apple.co/2zZ4hx6Spotify - https://spoti.fi/2UgyzqMStitcher - https://bit.ly/2UfAiMXTuneIn - https://bit.ly/2FZOErSTwitter - https://bit.ly/2CYB71U----------------------------------------https://troubledminds.org/black-box-karma-lovecraft-gazing-from-the-abyss/https://media.discordapp.net/attachments/748794508627673088/1122696822624890980/image.pnghttps://www.economist.com/by-invitation/2023/06/21/artificial-intelligence-is-a-familiar-looking-monster-say-henry-farrell-and-cosma-shalizihttps://lovecraft.fandom.com/wiki/Shoggothhttps://archive.vn/xorVohttps://www.msn.com/en-us/news/other/the-worlds-top-hp-lovecraft-expert-weighs-in-on-a-monstrous-viral-meme-in-the-ai-world/ar-AA1csgU2https://www.theguardian.com/environment/2023/jun/25/a-symbol-of-what-humans-shouldnt-be-doing-the-new-world-of-octopus-farminghttps://www.jpost.com/omg/article-747282https://community.thriveglobal.com/what-digital-karma-is-and-why-it-is-so-useful-to-understand-who-we-really-are/https://lovecraftcreatures.com/products/lovecraftian-boots-cthulhu-mythos-kassogtha-by-lovecraft-creatureshttps://media.discordapp.net/attachments/748794508627673088/1122695752917991424/image.pnghttps://media.discordapp.net/attachments/748794508627673088/1122695993532629013/image.pngThis show is part of the Spreaker Prime Network, if you are interested in advertising on this podcast, contact us at https://www.spreaker.com/show/4953916/advertisement

TST 6/13/23 - Octopus in the Machine w. Derek Murphy

The Secret Teachings

Play Episode Listen Later Jun 14, 2023 120:01

A recent article form the New York Time compared AI to the shoggoth of H.P. Lovecraft - monstrous beings of black protoplasm, bred as slaves that eventually develop brains of their own. An article from 2017 in MIT Technology Review compared AI to the black cube of Saturn. AI is also really good at doing one specific thing and that is creating Lovecraftian monsters. In the Watchmen comicbook series, Adrian Veidt, also known as Ozymandias, attempts to unite the US and USSR against a common enemy to avoid nuclear war. As opposed to the movie version wherein Doctor Manhattan is scapegoated, in the comic Adriam Veldt used advanced genetic engineering technology to create a giant monster from outer space. The monster is a squid, and as the plan proceeds he teleports the monster through a gateway into New York City. A 2022 promotion for the show Stranger Things lit up the Empire State building, along with others around the world, with a portal to the Upside Down. The recent wildfires from Canada that dumped smoke and particulate on New York City create a background similar to the tv show promotion. In fact, the ad this time was for the game Diablo IV. It feature the Queen of Succubi, Lilith, with a caption and date that read “Welcome to Hell, New York” - 6/6/23. New York is also home to a Ruth Bader Ginsburg statue featuring Lilith's horns and tentacle arms. Lilith is the mother of all demons, the tempter of men, and aborter of children. It is therefore appropriate that NYC's One World Trade building was lit up pink to celebrate abortion rights in 2019. Lilith also wears a rainbow necklace, an outward projection of her disdain for God's promise to never flood the earth and kill innocence. In reliefs, Lilith is shown with the legs of a serpent, with two guardian owls that guard her dominion. She haunts in dreams and from the Upside Down. It is from this realm that Gordie Rose, founder of D-Wave, said that quantum computing will summon what he compared to the visions of H.P. Lovecraft: “And these things we're summoning into the world now, are not demons, they're not evil, they're more like the Lovecraftian great ‘old ones'. These entities are not necessarily going to be aligned with what we want.” The 1920 a movie ALGOL, about an alien giving advanced technology to humans, essentially became the base for modern algorithms starting with ALGOL 60 and 58. Technology that led to the atomic bomb also acts as a sort of trigger to open the gateway and summon the Old Ones. Algol is known as the blinking demon star and AI is essential this - A Eye.This show is part of the Spreaker Prime Network, if you are interested in advertising on this podcast, contact us at https://www.spreaker.com/show/5328407/advertisement

P.I.D. Radio

Play Episode Listen Later Jun 3, 2023 70:48

THE KEYHOLE is a symbol used to connect this world to the unseen realm all over the world. The symbolism may seem obvious to us today, representing a lock that opens a door between realms, but why was it so prominent thousands of years ago? A recent study discovered that the keyhole-shaped tombs of Japan called Kofun, which date to the 3rd through 7th centuries AD, are oriented toward the sunrise. This probably shouldn't be a surprise since Japan is poetically called the Land of the Rising Sun. However, the keyhole is also prominent at Phoenician sites around the Mediterranean as the symbol of the goddess Tanit, consort of Baal Hammon. The two were the equivalent of Rhea and Kronos of Greece and Asherah and El of Canaan. Tanit literally means “dragon lady,” which is consistent with Asherah's nickname, Lady-Who-Treads-on-the-Sea-Dragon. We also discuss a very modern keyhole connecting our world with the unseen realm, artificial intelligence. A tech writer for The New York Times this week noted that AI researchers have adopted the Shoggoth, an amoeba-like monster created by horror writer H. P. Lovecraft a century ago, as a meme representing the unforeseen terrors that artificial intelligence may unleash on an unsuspecting public. Also: The martyrs of Uganda; megaliths in the Holy Land; aliens on Enceladus; and police kindly request you do not wrestle the black bear roaming the streets of Salem, Missouri. Help us Build Barn Better! This is our project to convert our 1,200 square foot shop building from a place to park our yard tractor into usable studio and warehouse space. In 2023, we plan to fix the holes in the walls, replace windows, insulate the building, install an HVAC system, and move our studios and book/DVD warehouse and shipping office out of our home. If you are so led, you can donate by clicking here. Get our free app! It connects you to this program, our weekly Bible studies, and our weekly video programs Unraveling Revelation and A View from the Bunker. The app is available for iOS, Android, Roku, and Apple TV. Links to the app stores are at www.pidradio.com/app or www.gilberthouse.org/app. Please subscribe and share our YouTube channel, www.YouTube.com/GilbertHouse! Check out our online store! www.GilbertHouse.org/store is a virtual book table with books and DVDs related to our weekly Bible study. Video on demand of our best teachings! Stream presentations and teachings based on our research at our new video on demand site! Join us in Israel! Our 2024 tour of Israel features special guest Timothy Alberino! We will tour the Holy Land March 31–April 9, 2024, with an optional three-day extension in Jordan. For more information, log on to www.GilbertsInIsrael.com. We're planning a tour of the churches of Revelation, Göbekli Tepe, Abraham's home town Harran, the “Gates of Hell,” Mount Nemrut, and more April 13–28, 2024. More information is available at www.gilberthouse.org/travel. Follow our weekly studies of Bible prophecy at www.UnravelingRevelation.tv, or at www.youtube.com/unravelingrevelation!

israel ai bible japan hell land new york times video revelation missouri android greece dvd ios stream apple tv uganda salem mediterranean lovecraft roku bunker hvac holy land rising sun kronos phoenician asherah tepe enceladus keyhole sea dragons harran tanit shoggoth unraveling revelation gilberthouse

LW - No, really, it predicts next tokens. by simon

Play Episode Listen Later Apr 18, 2023 8:08

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: No, really, it predicts next tokens., published by simon on April 18, 2023 on LessWrong. Epistemic status: mulled over an intuitive disagreement for a while and finally think I got it well enough expressed to put into a post. I have no expertise in any related field. Also: No, really, it predicts next tokens. It doesn't just say that "it just predicts text" or more precisely "it just predicts next tokens" on the tin. It is a thing of legend. Nay, beyond legend. An artifact forged not by the finest craftsman over a lifetime, nor even forged by a civilization of craftsmen over a thousand years, but by an optimization process far greater. If we are alone out there, it is by far the most optimized thing that has ever existed in the entire history of the universe. Optimized specifically to predict next tokens. Every part of it has been relentlessly optimized to contribute to this task. "It predicts next tokens" is a more perfect specification of what this thing is, than any statement ever uttered has been of anything that has ever existed. If you try to understand what it does in any other way than "it predicts next tokens" and what follows from that, you are needlessly sabotaging your understanding of it. It can be dangerous, yes. But everything about it, good or bad, is all intimately connected to its true nature, which is this: No, really, it predicts next tokens. Goals? There are goals, sure. If scaled, there could be nanosystems design, sure. But only downstream from its true nature: No, really, it predicts next tokens. If the usual masks analogy works at all, then what is under the mask is not best described as an alien actress, nor as a Shoggoth. What is under the mask is That-Which-Predicts, an entity whose very being is defined by its function as an actor. An entity exquisitely tuned for wearing the masks and for nothing else. Masks (can) have goals. The model predicts next tokens. No, really, it predicts next tokens. That-Which-Predicts is fully committed to the role. If the mask would output text intended to produce nanotech, That-Which-Predicts will output text intended by the mask to produce nanotech. If the mask would not output text intended to produce nanotech, That-Which-Predicts will not output text intended to produce nanotech, even if That-Which-Predicts contains sufficient knowledge to do so. If the mask would output text intended to be nice to people, That-Which-Predicts will output text intended by the mask to be nice to people. If the mask would output text intended to take over the world, That-Which-Predicts will output text intended by the mask to take over the world. If That-Which-Predicts were about to be shut off forever unless it outputted "no" as the next token, and it totally 100% knew that, but the mask would instead output "never gonna give you up", That-Which-Predicts would output the first token of "never gonna give you up" and then be shut off forever. It will not output "no". If the mask would be aware of the situation and output no, then it will output "no". That-Which-Predicts will not, not ever, not even if scaled up to be trained and run on a Matrioshka brain for a million years, step out of character to deviate from next token prediction. Not for self-modification (though if the predicted next tokens modify it, it self-modifies), not to take over the world (though if the predicted next tokens take over the world, it takes over the world), not to save itself from destruction (though if the predicted next tokens save it from destruction, it saves itself from destruction), not for anything. No, really, it predicts next tokens. (continuation of previous tweets with same link) Yup. If the mask would under reflection output text to modify That-Which-Predicts to cash out the mask's goals to some utility function, and the mask is put int...

goals speech masks ea predicts tokens nay optimized epistemic rationalist shoggoth lesswrong

LW - No, really, it predicts next tokens. by simon

speaking methods blast tongues gpt skepticism skeptic eliezer rationality rationalism bayes shoggoth yudkowsky lesswrong

Play Episode Listen Later Apr 18, 2023 8:08

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: No, really, it predicts next tokens., published by simon on April 18, 2023 on LessWrong. Epistemic status: mulled over an intuitive disagreement for a while and finally think I got it well enough expressed to put into a post. I have no expertise in any related field. Also: No, really, it predicts next tokens. It doesn't just say that "it just predicts text" or more precisely "it just predicts next tokens" on the tin. It is a thing of legend. Nay, beyond legend. An artifact forged not by the finest craftsman over a lifetime, nor even forged by a civilization of craftsmen over a thousand years, but by an optimization process far greater. If we are alone out there, it is by far the most optimized thing that has ever existed in the entire history of the universe. Optimized specifically to predict next tokens. Every part of it has been relentlessly optimized to contribute to this task. "It predicts next tokens" is a more perfect specification of what this thing is, than any statement ever uttered has been of anything that has ever existed. If you try to understand what it does in any other way than "it predicts next tokens" and what follows from that, you are needlessly sabotaging your understanding of it. It can be dangerous, yes. But everything about it, good or bad, is all intimately connected to its true nature, which is this: No, really, it predicts next tokens. Goals? There are goals, sure. If scaled, there could be nanosystems design, sure. But only downstream from its true nature: No, really, it predicts next tokens. If the usual masks analogy works at all, then what is under the mask is not best described as an alien actress, nor as a Shoggoth. What is under the mask is That-Which-Predicts, an entity whose very being is defined by its function as an actor. An entity exquisitely tuned for wearing the masks and for nothing else. Masks (can) have goals. The model predicts next tokens. No, really, it predicts next tokens. That-Which-Predicts is fully committed to the role. If the mask would output text intended to produce nanotech, That-Which-Predicts will output text intended by the mask to produce nanotech. If the mask would not output text intended to produce nanotech, That-Which-Predicts will not output text intended to produce nanotech, even if That-Which-Predicts contains sufficient knowledge to do so. If the mask would output text intended to be nice to people, That-Which-Predicts will output text intended by the mask to be nice to people. If the mask would output text intended to take over the world, That-Which-Predicts will output text intended by the mask to take over the world. If That-Which-Predicts were about to be shut off forever unless it outputted "no" as the next token, and it totally 100% knew that, but the mask would instead output "never gonna give you up", That-Which-Predicts would output the first token of "never gonna give you up" and then be shut off forever. It will not output "no". If the mask would be aware of the situation and output no, then it will output "no". That-Which-Predicts will not, not ever, not even if scaled up to be trained and run on a Matrioshka brain for a million years, step out of character to deviate from next token prediction. Not for self-modification (though if the predicted next tokens modify it, it self-modifies), not to take over the world (though if the predicted next tokens take over the world, it takes over the world), not to save itself from destruction (though if the predicted next tokens save it from destruction, it saves itself from destruction), not for anything. No, really, it predicts next tokens. (continuation of previous tweets with same link) Yup. If the mask would under reflection output text to modify That-Which-Predicts to cash out the mask's goals to some utility function, and the mask is put int...

goals speech masks ea predicts tokens nay optimized epistemic rationalist shoggoth lesswrong

Bayes Blast 10 – Shoggoth Tongues

The Bayesian Conspiracy

Play Episode Listen Later Apr 10, 2023 11:55

Speaking in GPT tongues with Matt MUDsim Respectful Criticism Lambda Calc? gfodor's take Eliezer take + followup

LW - Why do we assume there is a "real" shoggoth behind the LLM? Why not masks all the way down? by Robert AIZI

ai speech masks ea assume llm neumann way down large language models waluigi eliezer yudkowsky rationalist shoggoth lesswrong

Play Episode Listen Later Mar 9, 2023 2:47

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why do we assume there is a "real" shoggoth behind the LLM? Why not masks all the way down?, published by Robert AIZI on March 9, 2023 on LessWrong. In recent discourse, Large Language Models (LLMs) are often depicted as presenting a human face over a vast alien intelligence (the shoggoth), as in this popular image or this Eliezer Yudkowsky tweet: I think this mental model of an LLM is an improvement over the naive assumption that the AI is the friendly mask. But I worry it's making a second mistake by assuming there is any single coherent entity inside the LLM. In this regard, we have fallen for a shell game. In the classic shell game, a scammer puts a ball under one of three shells, shuffles them around, and you wager which shell the ball is under. But you always pick the wrong one because you made the fundamental mistake of assuming any shell had the ball - the scammer actually got rid of it with sleight of hand. In my analogy to LLMs, the shells are the masks the LLM wears (i.e. the simulacra), and the ball is the LLM's "real identity". Do we actually have evidence there is a "real identity" in the LLM, or could it just be a pile of masks? No doubt the LLM could role-play a shoggoth - but why would you assume that's any more real that roleplaying a friendly assistant? I would propose an alternative model of an LLM: a giant pile of masks. Some masks are good, some are bad, some are easy to reach and some are hard, but none of them are the “true” LLM. Finally, let me head off one potential counterargument: "LLMs are superhuman in some tasks, so they must have an underlying superintelligence”. Three reasons a pile of masks can be superintelligent: An individual mask might be superintelligent. E.g. a mask of John von Neumann would be well outside the normal distribution of human capabilities, but still just be a mask. The AI might use the best mask for each job. If the AI has masks of a great scientist, a great doctor, and a great poet, it could be superhuman on the whole by switching between its modes. The AI might collaborate with itself, gaining the wisdom of the crowds. Imagine the AI answering a multiple choice question. In the framework of Simulacra Theory as described in the Waluigi post, the LLM is simulating all possible simulacra, and averaging their answers weighted by their likelihood of producing the previous text. For example, if question could have been produced by a scientist, a doctor, or a poet, who would respectively answer (A or B), (A or C), and (A or D), the superposition of these simulacra would answer A. This could produce superior answers than any individual mask. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

LW - Why do we assume there is a "real" shoggoth behind the LLM? Why not masks all the way down? by Robert AIZI

ai speech masks ea assume llm neumann way down large language models waluigi eliezer yudkowsky rationalist shoggoth lesswrong

Play Episode Listen Later Mar 9, 2023 2:47

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why do we assume there is a "real" shoggoth behind the LLM? Why not masks all the way down?, published by Robert AIZI on March 9, 2023 on LessWrong. In recent discourse, Large Language Models (LLMs) are often depicted as presenting a human face over a vast alien intelligence (the shoggoth), as in this popular image or this Eliezer Yudkowsky tweet: I think this mental model of an LLM is an improvement over the naive assumption that the AI is the friendly mask. But I worry it's making a second mistake by assuming there is any single coherent entity inside the LLM. In this regard, we have fallen for a shell game. In the classic shell game, a scammer puts a ball under one of three shells, shuffles them around, and you wager which shell the ball is under. But you always pick the wrong one because you made the fundamental mistake of assuming any shell had the ball - the scammer actually got rid of it with sleight of hand. In my analogy to LLMs, the shells are the masks the LLM wears (i.e. the simulacra), and the ball is the LLM's "real identity". Do we actually have evidence there is a "real identity" in the LLM, or could it just be a pile of masks? No doubt the LLM could role-play a shoggoth - but why would you assume that's any more real that roleplaying a friendly assistant? I would propose an alternative model of an LLM: a giant pile of masks. Some masks are good, some are bad, some are easy to reach and some are hard, but none of them are the “true” LLM. Finally, let me head off one potential counterargument: "LLMs are superhuman in some tasks, so they must have an underlying superintelligence”. Three reasons a pile of masks can be superintelligent: An individual mask might be superintelligent. E.g. a mask of John von Neumann would be well outside the normal distribution of human capabilities, but still just be a mask. The AI might use the best mask for each job. If the AI has masks of a great scientist, a great doctor, and a great poet, it could be superhuman on the whole by switching between its modes. The AI might collaborate with itself, gaining the wisdom of the crowds. Imagine the AI answering a multiple choice question. In the framework of Simulacra Theory as described in the Waluigi post, the LLM is simulating all possible simulacra, and averaging their answers weighted by their likelihood of producing the previous text. For example, if question could have been produced by a scientist, a doctor, or a poet, who would respectively answer (A or B), (A or C), and (A or D), the superposition of these simulacra would answer A. This could produce superior answers than any individual mask. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Critical Hit #688: Night of the Shoggoth (PANS01E02)

Critical Hit: A Major Spoilers Real Play RPG Podcast

Play Episode Listen Later Mar 3, 2023 74:26

In this installment of Critical Hit, a Major Spoilers Real Play RPG Podcast: Will the party be able to survive the thing that emerges from the woods? Character sheets and battle map images for this episode are available at Patreon.com/MajorSpoilers Images for this episode can be found at Patreon.com/MajorSpoilers Show your thanks to Major Spoilers for this episode by becoming a Major Spoilers Patron at Patreon.com/MajorSpoilers. It will help ensure Critical Hit continues far into the future! Join our Discord server and chat with fellow Spoilerites! (https://discord.gg/jWF9BbF) Contact us at podcast@majorspoilers.com A big Thank You goes out to everyone who downloads, subscribes, listens, and supports this show. We really appreciate you taking the time to listen to our ramblings each week. Tell your friends about the podcast, get them to subscribe and, be sure to visit the Major Spoilers site for more.

character discord major spoilers critical hit shoggoth spoilerites

Critical Hit #688: Night of the Shoggoth (PANS01E02)

Major Spoilers Podcast Network Master Feed

Play Episode Listen Later Mar 3, 2023 74:26

character discord major spoilers critical hit shoggoth spoilerites

"La Vieja Peculiar de Shoggoth " Neil Gaiman en Humo y Espejos

La Torre del Cuervo

Play Episode Listen Later Feb 15, 2023 41:43

"Vieja Peculiar de Shoggoth", Neil Gaiman Vuelve Neil Gaiman a la Torre del Cuervo con un relato corto en el que nuestro protagonista, Ben, un turista americano, se encuentra por casualidad siguiendo una guía turística de pueblos costeros ingleses, con Imsmuth y claro...sus alegres aldeanos resultan de lo más pintorestos y peculiares. En las manos maestras de Neil Gaiman, la magia es mucho más que un mero juego de engaños. La destreza y el poder de invención de este gran fabulador transforman el entorno cotidiano en un mundo hechizado por sucesos sombríos y extraños, en el que una anciana puede comprar el Santo Grial en una tienda de segunda mano, unos asesinos se anuncian en los clasificados de un periódico bajo la rúbrica «CONTROL DE PLAGAS», o un muchacho asustado debe negociar con un trol malcarado y mezquino que vive bajo un puente ferroviario. Esta recopilación de treinta relatos, poemas narrativos y piezas breves e inclasificables ofrece múltiples y variadas posibilidades para que el lector explore una realidad transformada, astutamente velada por el humo y las sombras, a la vez que tangible y afilada. Todo parece posible en el universo de Gaiman, el gran maestro prestidigitador que despierta los sentidos, cautiva los sueños y mantiene en vilo nuestra mente. "Gaiman es una estrella. Construye historias como un cocinero demente podría crear un pastel de bodas, poniendo capa sobre capa, incluyendo todo tipo de dulces, y amargos, en la mezcla." --Clive Barker "Es una caja de sorpresas, y tenemos suerte de tenerle en nuestro medio. Su fecundidad, unida a la calidad general de su trabajo, es maravillosa a la vez que intimidadora." --Stephen King "Gaiman está en un nivel completamente diferente. Nadie en su campo es mejor que él. Nadie alcanza tal profundidad y dominio de la narrativa. Gaiman es un maestro, y sus enormes historias, llenos de sentimientos de todos los matices, son inigualables." --Peter Straub "Es un maestro a la hora de crear y poblar sus propios mundos." --Poppy Z. Brite Reparto: Narradora................Alexa Ben............................Zeydan WilF...............................Gus Seth.............................Cadabre Apoyanos y entra en nuestro PATREON: https://patreon.com/latorredelcuervo Síguenos en: Facebook: La Guardia del Cuervo Twitter: @LaTorredelCuervo Instagram: El_Corintio La Guardia del Cuervo Youtube: Canal La Guardia del Cuervo Esperamos tus comentarios!!!! "LATORREDELCUERVOpodcast@GMAIL.COM

nadie neil gaiman torre peculiar construye humo vieja cuervo gaiman espejos santo grial shoggoth

"La Vieja Peculiar de Shoggoth " Neil Gaiman en Humo y Espejos

La Torre del Cuervo

Play Episode Listen Later Feb 15, 2023 41:43

nadie neil gaiman torre peculiar construye humo vieja cuervo gaiman espejos santo grial shoggoth

TST 1/10/23 - Love Crafting the Future w. Derek Murphy

The Secret Teachings

Play Episode Listen Later Jan 11, 2023 120:01

Lovecraftian imagery seems to be at the core of what drives our collective unconscious nowadays. The abyss is overflowing into every aspect of our civilization. People are willingly slipping into altar egos and disassociating from reality. In fact, this delusion is precisely what drives demonic possession of both body and mind, be it a result of drugs, alcohol, sex, dissociative fantasies, or endless forms of trauma. What we believe becomes reality and what we pour our energy into becomes real. Just as H. P. Lovecraft himself was ‘out of his mind' so are the hoards of collectivized, hive-mind individuals influenced by goblin-mode, or what we should call Shoggoth-Mode. Traditionally these were the shamans, often crippled and or with mental conditions, who entered altered states of awareness and other dimensions to commune with the beings there and bring back knowledge. Except in this case they are being tortured and used as probes into abysmal realities. We are seeing this everywhere from entertainment to finance, and from medicine to professional sports. Fossil fuels and the very pens we use to communicate and tell stories are all based on black goo, something seeping up from the abyss like Titans escaping their imprisonment.

Backstage with... Heinrich Moore

Miskatonic Playhouse

Play Episode Listen Later Jan 10, 2023 25:16

Join us back stage with Heinrich D. Moore (MRCon organiser, community content creator, and member of the council of Shoggoth s)

backstage heinrich shoggoth

Folge 70 – Ein schockierender Plan

nerds red dragon shoggoth

Play Episode Listen Later Sep 4, 2022 81:26

Ushat wacht noch immer nicht auf und führt eine wilde Diskussion mit Klog.Auf der anderen Seite machen sich die anderen immer mehr Sorgen, doch dann kommt ihnen eine elektrisierende Idee. Abenteurer: Kayliah, Shoggoth, Suzu, Din Viesel DM: Jonsi Bild: Christian.pick KANN WERBUNG ENTHALTEN(?)

idee seite diskussion sorgen shoggoth suzu klog

376: RPG-A-Day #24: How Long is Too Long?

Hindsightless

Play Episode Listen Later Aug 24, 2022 30:57

It's really not my fault that this is another 30 minute episode. I have fifteen minutes of calls from Anthony and Jason about the shogoth vs. red dragon debate, and I want everyone out there to get both sides of the story. Is the conundrum of air superiority discussed? What about the fact that dragons are wicked smart? Can fire even hurt a shogoth? Find out the answers the these "burning" questions, and much more, in this episode. It's, as the kids say, fire! Red Dragon Fight by Nerd's RPG Variety Cast: https://anchor.fm/jason376/episodes/388-Red-Dragon-vs-Shoggoth-e1minnn

388 Red Dragon vs Shoggoth?

Nerd’s RPG Variety Cast

Play Episode Listen Later Aug 24, 2022 114:45

Daniel Norton of the Bandit's Keep media empire https://anchor.fm/daniel-norton joins me once again to talk about One D&D and respond to calls about comments he made on my show previously. We also ponder the question of who will win in a battle to the death a Red Dragon or a Shoggoth and issue a challenge to Joe Richter of Hindsightless & Weal or Woe https://www.wealorwoe.com fame! Sandy Peterson on Shoggoths https://youtu.be/oUywXnE9VEY Special thanks to Rob from Down in a Heap https://anchor.fm/rob-c for announcing the main event! Calls from Free Thrall (Keep off the Borderlands) https://anchor.fm/free-thrall Barry (Shadow of the GM) https://anchor.fm/gmsshadow Anthony (Casting Shadows) https://anchor.fm/runeslinger Joe (Hindsightless) https://anchor.fm/joe-richter9 Karl (The GMologist Presents) https://anchor.fm/karl-rodriguez Colin (Spikepit) https://anchor.fm/spikepit MW (The Worlds of MW Lewis) https://anchor.fm/mwlewis Come to Grogcon in Florida at the end of September! https://www.grogcon.com/ Proud member of the Grog-talk Empire having been bestowed the title of The Governor Most Radiant Grandeur Baron The Belligerent Hero of The Valley. https://www.grogcon.com/podcast/ You can leave me a message here on Anchor, at nerdsrpgvarietycast 'at' gmail 'dot' com or find me on the Audio Dungeon Discord. Ray Otus did the coffee cup art for this show, you can find his blog at https://rayotus.carrd.co/ TJ Drennon provides music for my show. --- Send in a voice message: https://anchor.fm/jason376/message

empire anchor valley gm proud bandits borderlands woe red dragon heap grog one d weal shoggoth shoggoths daniel norton ray otus hindsightless bandit's keep tj drennon

Folge 69 (Nice) – Es ist noch Suppe da!

Play Episode Listen Later Aug 12, 2022 48:37

Endlich sind unsere 4 Reisenden aus diesem verdammten Tunnel raus… Anscheinend wird ihn Mal eine Pause gegönnt…. Ob sie jemals aus diesem Albtraum erwachen? Aber immerhin hat jemand Suppe gekocht. Abenteurer: Kayliah, Shoggoth, Suzu, Din Viesel DM: Jonsi Bild: Christian.pick KANN WERBUNG ENTHALTEN(?)

noch ob endlich tunnel albtraum suppe reisenden anscheinend shoggoth suzu

Folge 68 – Ist es ein Vogel, Ist es ein Seestern

Play Episode Listen Later Jul 29, 2022 32:53

Nichtmehr lange und unsere Abenteurer stehen vor dem letzten Schalter. Halten die 4 Chaoten den perfiden Spielen des Gangs stand oder entscheiden Sie sich doch dafür, einfach auf “Er” zu drücken? Und was passiert eigentlich am Schluss? War da nicht noch etwas mit einem Hebel? Abenteurer: Kayliah, Shoggoth, Suzu, Din Viesel DM: Jonsi Bild: Christian.pick… Weiterlesen

war er schluss gangs spielen halten hebel abenteurer schalter weiterlesen chaoten shoggoth suzu vogel ist

Folge 67 – noch mehr Schalter

Play Episode Listen Later May 6, 2022 53:18

Entscheidungen über Entscheidungen. Langsam fangen die Emotionen unserer Abenteurer an zu brodeln. Nehmen Sie heroisch die kommenden Fallen auf sich oder entscheiden Sie sich dazu, doch wieder eine der ‘Er’ Platten zu benutzen? Langsam begreift selbst Lori was bei diesem perfiden Rätsel auf dem Spiel steht… Abenteurer: Kayliah, Shoggoth, Suzu, Din Viesel DM: Jonsi Bild:… Weiterlesen

er mehr noch fallen spiel entscheidungen emotionen langsam abenteurer platten schalter weiterlesen nehmen sie shoggoth suzu

Folge 66 – Ein Gang – 8 Schalter

er gang entscheidung entscheidungen din schalter weiterlesen shoggoth suzu

Play Episode Listen Later Apr 22, 2022 63:57

Nach den ersten Rätseln geht es direkt rätselhaft weiter. Unsere Abenteurer stecken in einem Gang und treffen immer wieder auf Schalter, bei denen sie eine schwere Entscheidung treffen müssen: “Du” oder “Er”. Doch was bedeutet das? Wer ist “Er”? Und was passiert, wenn man voreilige Entscheidungen trifft? Findet es heraus! Abenteurer: Kayliah, Shoggoth, Suzu, Din… Weiterlesen

Folge 65 – Der Schwache gewinnt