Podcasts about Decision theory

single speech wikipedia views closed ea arrow curves kurzgesagt relatedly cgp grey veritasium eliezer yudkowsky rationalist lesswrong decision theory vnm

Play Episode Listen Later Sep 3, 2024 2:05

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How I got 3.2 million Youtube views without making a single video, published by Closed Limelike Curves on September 3, 2024 on LessWrong. Just over a month ago, I wrote this. The Wikipedia articles on the VNM theorem, Dutch Book arguments, money pump, Decision Theory, Rational Choice Theory, etc. are all a horrific mess. They're also completely disjoint, without any kind of Wikiproject or wikiboxes for tying together all the articles on rational choice. It's worth noting that Wikipedia is the place where you - yes, you! - can actually have some kind of impact on public discourse, education, or policy. There is just no other place you can get so many views with so little barrier to entry. A typical Wikipedia article will get more hits in a day than all of your LessWrong blog posts have gotten across your entire life, unless you're @Eliezer Yudkowsky. I'm not sure if we actually "failed" to raise the sanity waterline, like people sometimes say, or if we just didn't even try. Given even some very basic low-hanging fruit interventions like "write a couple good Wikipedia articles" still haven't been done 15 years later, I'm leaning towards the latter. edit me senpai EDIT: Discord to discuss editing here. An update on this. I've been working on Wikipedia articles for just a few months, and Veritasium just put a video out on Arrow's impossibility theorem - which is almost completely based on my Wikipedia article on Arrow's impossibility theorem! Lots of lines and the whole structure/outline of the video are taken almost verbatim from what I wrote. I think there's a pretty clear reason for this: I recently rewrote the entire article to make it easy-to-read and focus heavily on the most important points. Relatedly, if anyone else knows any educational YouTubers like CGPGrey, Veritasium, Kurzgesagt, or whatever - please let me know! I'd love a chance to talk with them about any of the fields I've done work teaching or explaining (including social or rational choice, economics, math, and statistics). Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

LW - How I got 3.2 million Youtube views without making a single video by Closed Limelike Curves

single speech wikipedia views closed ea arrow curves kurzgesagt relatedly cgp grey veritasium eliezer yudkowsky rationalist lesswrong decision theory vnm

Play Episode Listen Later Sep 3, 2024 2:05

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How I got 3.2 million Youtube views without making a single video, published by Closed Limelike Curves on September 3, 2024 on LessWrong. Just over a month ago, I wrote this. The Wikipedia articles on the VNM theorem, Dutch Book arguments, money pump, Decision Theory, Rational Choice Theory, etc. are all a horrific mess. They're also completely disjoint, without any kind of Wikiproject or wikiboxes for tying together all the articles on rational choice. It's worth noting that Wikipedia is the place where you - yes, you! - can actually have some kind of impact on public discourse, education, or policy. There is just no other place you can get so many views with so little barrier to entry. A typical Wikipedia article will get more hits in a day than all of your LessWrong blog posts have gotten across your entire life, unless you're @Eliezer Yudkowsky. I'm not sure if we actually "failed" to raise the sanity waterline, like people sometimes say, or if we just didn't even try. Given even some very basic low-hanging fruit interventions like "write a couple good Wikipedia articles" still haven't been done 15 years later, I'm leaning towards the latter. edit me senpai EDIT: Discord to discuss editing here. An update on this. I've been working on Wikipedia articles for just a few months, and Veritasium just put a video out on Arrow's impossibility theorem - which is almost completely based on my Wikipedia article on Arrow's impossibility theorem! Lots of lines and the whole structure/outline of the video are taken almost verbatim from what I wrote. I think there's a pretty clear reason for this: I recently rewrote the entire article to make it easy-to-read and focus heavily on the most important points. Relatedly, if anyone else knows any educational YouTubers like CGPGrey, Veritasium, Kurzgesagt, or whatever - please let me know! I'd love a chance to talk with them about any of the fields I've done work teaching or explaining (including social or rational choice, economics, math, and statistics). Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

LW - Decision Theory in Space by lsusr

lord space speech rebel ea nonsense destroying aim imperial vader princess leia tatooine alderaan tarkin grand moff tarkin rationalist lesswrong grand moff decision theory

Play Episode Listen Later Aug 19, 2024 4:25

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Decision Theory in Space, published by lsusr on August 19, 2024 on LessWrong. "Since you are reluctant to provide us with the location of the Rebel base," said Grand Moff Tarkin, "I have chosen to test this station's destructive power on your home planet of Alderaan." "No. Alderaan is peaceful. We have no weapons there. It is a loyal planet under Imperial control. Striking Alderaan would destroy your own resources and foment rebellion. Destroying it is irrational," said Princess Leia, perfectly calm. "Nonsense," said Tarkin, "That is a naïve understanding of decision theory. I am a causal decision theorist, but I acknowledge the value of precommitments. I therefore precommit to destroying Alderaan unless you reveal to me the location of the Rebel base. This is not an irrational act if you capitulate to me." "But it is an irrational act if I do not capitulate to you," said Leia, "I am a functional decision theorist. The algorithm I use to select my decision accounts for the fact that you are modelling my mind. You are a rational agent. You only threaten me because you expect me to succomb to your blackmail. Because of that I will not succomb to your blackmail." "I'm going to do it," said Tarkin. "Sure you are," said Leia. "I'm really going to blow up the planet," said Tarkin. "Be my guest," said Leia, with a smile, "Aim for the continent Anaander. Its inhabitants always annoyed me. We'll see who has the last laugh." "I'm really really going to do it," said Tarkin. "I grow tired of saying this, so it'll be the last time. Just blow up the planet already. I have an execution I'm late for…." Leia's voice trailed off. She was suddenly aware of the deep, mechanical breathing behind her. Kshhhhhhh. Kuuuuuuo. Kshhhhhhh. Kuuuuuuo. Everyone in the Life Star command center turned to face the cyborg space wizard samurai monk in black armor. Kshhhhhhh. Kuuuuuuo. Kshhhhhhh. Kuuuuuuo. Vader's cloak fluttered and a couple indicator lights on his life support system blinked, but no muscles or actuators moved. A semi-mechanical voice in the uncanny valley spoke from Vader's mask. "Chief Gunnery Officer Tenn Graneet, you may fire when ready." "Commander Tenn Graneet, belay that order," said Tarkin. The Chief Gunnery Officer held his hand above his control panel, touching nothing. He looked rapidly back-and-forth between Tarkin and Vader. Tarkin turned angrily to face Vader. "Are you insane?" Tarkin hissed. Vader ignored the question and looked at Leia. "Where is the Rebel base?" Leia's eyes were wide with horror and her mouth was wide with a silent scream. She clenched her teeth and stared at the floor. "Tatooine. They're on Tatooine," Leia said. "Chief Gunnery Officer Tenn Graneet, you may fire when ready," said Vader. "What‽" exclaimed Tarkin. Graneet lifted the clear cover off of the authorization lever. He moved his hand as slowly as he could. "Commander Tenn Graneet, belay that order," said Tarkin. "Commander Tenn Graneet, ignore all orders you receive from the Grand Moff," said Vader. "Commander Tenn Graneet, I am your commanding officer. Ignore all orders from 'Lord' Vader. If you continue to disobey my orders, you will be court martialed," said Tarkin. Graneet continued the process of authorizing the firing team. Tarkin drew his blaster pistol and held it to Graneet's head. "Stop or I will shoot you in the head right now," said Tarkin. Bkzzzzzzzzzzzzz. Tarkin felt the heat of Vader's red lightsaber a centimeter from his neck. The next seconds felt like slow motion. Graneet paused. Then Greneet continued the firing activation sequence. Tarkin pulled the trigger. Click. Nothing came out of the blaster's emitter. Vader didn't even bother to watch his order get carried out. He just turned around, deactivated his lightsaber, and strode out of the command center. Vader's cape billowed...

LW - Decision Theory in Space by lsusr

lord space speech rebel ea nonsense destroying aim imperial vader princess leia tatooine alderaan tarkin grand moff tarkin rationalist lesswrong grand moff decision theory

Play Episode Listen Later Aug 19, 2024 4:25

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Decision Theory in Space, published by lsusr on August 19, 2024 on LessWrong. "Since you are reluctant to provide us with the location of the Rebel base," said Grand Moff Tarkin, "I have chosen to test this station's destructive power on your home planet of Alderaan." "No. Alderaan is peaceful. We have no weapons there. It is a loyal planet under Imperial control. Striking Alderaan would destroy your own resources and foment rebellion. Destroying it is irrational," said Princess Leia, perfectly calm. "Nonsense," said Tarkin, "That is a naïve understanding of decision theory. I am a causal decision theorist, but I acknowledge the value of precommitments. I therefore precommit to destroying Alderaan unless you reveal to me the location of the Rebel base. This is not an irrational act if you capitulate to me." "But it is an irrational act if I do not capitulate to you," said Leia, "I am a functional decision theorist. The algorithm I use to select my decision accounts for the fact that you are modelling my mind. You are a rational agent. You only threaten me because you expect me to succomb to your blackmail. Because of that I will not succomb to your blackmail." "I'm going to do it," said Tarkin. "Sure you are," said Leia. "I'm really going to blow up the planet," said Tarkin. "Be my guest," said Leia, with a smile, "Aim for the continent Anaander. Its inhabitants always annoyed me. We'll see who has the last laugh." "I'm really really going to do it," said Tarkin. "I grow tired of saying this, so it'll be the last time. Just blow up the planet already. I have an execution I'm late for…." Leia's voice trailed off. She was suddenly aware of the deep, mechanical breathing behind her. Kshhhhhhh. Kuuuuuuo. Kshhhhhhh. Kuuuuuuo. Everyone in the Life Star command center turned to face the cyborg space wizard samurai monk in black armor. Kshhhhhhh. Kuuuuuuo. Kshhhhhhh. Kuuuuuuo. Vader's cloak fluttered and a couple indicator lights on his life support system blinked, but no muscles or actuators moved. A semi-mechanical voice in the uncanny valley spoke from Vader's mask. "Chief Gunnery Officer Tenn Graneet, you may fire when ready." "Commander Tenn Graneet, belay that order," said Tarkin. The Chief Gunnery Officer held his hand above his control panel, touching nothing. He looked rapidly back-and-forth between Tarkin and Vader. Tarkin turned angrily to face Vader. "Are you insane?" Tarkin hissed. Vader ignored the question and looked at Leia. "Where is the Rebel base?" Leia's eyes were wide with horror and her mouth was wide with a silent scream. She clenched her teeth and stared at the floor. "Tatooine. They're on Tatooine," Leia said. "Chief Gunnery Officer Tenn Graneet, you may fire when ready," said Vader. "What‽" exclaimed Tarkin. Graneet lifted the clear cover off of the authorization lever. He moved his hand as slowly as he could. "Commander Tenn Graneet, belay that order," said Tarkin. "Commander Tenn Graneet, ignore all orders you receive from the Grand Moff," said Vader. "Commander Tenn Graneet, I am your commanding officer. Ignore all orders from 'Lord' Vader. If you continue to disobey my orders, you will be court martialed," said Tarkin. Graneet continued the process of authorizing the firing team. Tarkin drew his blaster pistol and held it to Graneet's head. "Stop or I will shoot you in the head right now," said Tarkin. Bkzzzzzzzzzzzzz. Tarkin felt the heat of Vader's red lightsaber a centimeter from his neck. The next seconds felt like slow motion. Graneet paused. Then Greneet continued the firing activation sequence. Tarkin pulled the trigger. Click. Nothing came out of the blaster's emitter. Vader didn't even bother to watch his order get carried out. He just turned around, deactivated his lightsaber, and strode out of the command center. Vader's cape billowed...

#5 - The Art of Decision Making

Reason Is Fun

Play Episode Listen Later Jan 4, 2024 49:12

Lulie and David are joined by guest Mark Alexander, the Producer of Art of Accomplishment's Great Decisions course, to do a deeper dive into how decisions are made. They discuss having competing wants, Popperian problem-solving, the difference between 'shoulds' and a non-coercive (fun!) version of morality, emotion, and the participation of different subconscious processes in decision making. Mark asks "What are 'hangups'?", and David discusses mental blocks and how to get back to a creatively problem-solving state of mind. Lulie shares a problem she has with getting back to that creative place, namely: when do you go into your resisted feelings, and when do you analyse the problem? 00:00 Introducing: Lulie, David and Mark! 00:33 Competing wants vs 'shoulds' 02:23 Morality: coercive vs non-coercive 04:35 Why would we need the word 'should'? 05:55 Wants rest on morality! 06:51 Needle phobia example (of conflicting wants) 09:05

art decisions decision making discounts morality competing needle accomplishments hobgoblins hangups mark alexander decision theory

#4 - Decisions

Reason Is Fun

Play Episode Listen Later Jan 3, 2024 54:19

Lulie and David chat about decision making. They discuss the roles of creativity, institutions and emotions in decision making, and the misconception that decisions are made by analysing data. The conversation explores how decision making isn't a mechanical process, but rests on creative thinking. David criticises Decision Theory as a framework for decisions, and Lulie wonders to what extent decision-making requires emotion. 00:00 Introduction – we're BACK, baby! 00:59 Difficulties people have with decision making 01:46 The role of creativity in decisions 03:56 Do we ever "make decisions" when things are going well? 06:01 Decision Theory is not how we make choices (generally) 15:36 Are decisions emotions-based? 19:30 Principles (Art of Accomplishment's version) 23:59 Institutions (David's version) 28:25 Knowledge/thought is also needed! 34:50 Example from Lulie's life 39:36 Analytical, or emotional? 42:03 E-motions are needed for action (Lulie's hot take) 47:09 Acting, Method acting (Lulie's crackpot take) 53:47 Details for the How to Make Great Decisions course by Art of Accomplishment If you're looking to improve your relationship with decisions in 2024, Art of Accomplishment is offering its once-a-year Decisions course starting January 11th. (Sign-ups close Monday January 8th.) Discount code for $100 off: LULIE100 I'll be a participant this year too, so this is a chance to do it alongside me as I share my own deep-dive journey there. Hope you'll join me!

art decisions acting method discounts difficulties accomplishments analytical decision theory

EA - Philosophical considerations relevant to valuing continued human survival: Conceptual Analysis, Population Axiology, and Decision Theory (Andreas Mogensen) by Global Priorities Institute

LessWrong Curated Podcast

Play Episode Listen Later Nov 1, 2023 5:11

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Philosophical considerations relevant to valuing continued human survival: Conceptual Analysis, Population Axiology, and Decision Theory (Andreas Mogensen), published by Global Priorities Institute on November 1, 2023 on The Effective Altruism Forum. This paper was published as a GPI working paper in September 2023. Introduction Many think that human extinction would be a catastrophic tragedy, and that we ought to do more to reduce extinction risk. There is less agreement on exactly why. If some catastrophe were to kill everyone, that would obviously be horrific. Still, many think the deaths of billions of people don't exhaust what would be so terrible about extinction. After all, we can be confident that billions of people are going to die - many horribly and before their time - if humanity does not go extinct. The key difference seems to be that they will be survived by others. What's the importance of that? Some take the view that the special moral importance of preventing extinction is explained in terms of the value of increasing the number of flourishing lives that will ever be lived, since there could be so many people in the vast future available to us (see Kavka 1978; Sikora 1978; Parfit 1984; Bostrom 2003; Ord 2021: 43-49). Others emphasize the moral importance of conserving existing things of value and hold that humanity itself is an appropriate object of conservative valuing (see Cohen 2012; Frick 2017). Many other views are possible (see esp. Scheer 2013, 2018). However, not everyone is so sure that human extinction would be regrettable. In the final section of the last book published in his lifetime, Parfit (2011: 920-925) considers what can actually be said about the value of all future history. No doubt, people will continue to suffer and despair. They will also continue to experience love and joy. Will the good be sufficient to outweigh the bad? Will it all be worth it? Parfit's discussion is brief and inconclusive. He leans toward 'Yes,' writing that our "descendants might, I believe, make the future very good." (Parfit 2011: 923) But 'might' falls far short of 'will'. Others are confidently pessimistic. Some take the view that human lives are not worth starting because of the suffering they contain. Benatar (2006) adopts an extreme version of this view, which I discuss in section 3.3. He claims that "it would be better, all things considered, if there were no more people (and indeed no more conscious life)." (Benatar 2006: 146) Scepticism about the disvalue of human extinction is especially likely to arise among those concerned about our effects on non-human animals and the natural world. In his classic paper defending the view that all living things have moral status, Taylor (1981: 209) argues, in passing, that human extinction would "most likely be greeted with a hearty 'Good riddance!' " when viewed from the perspective of the biotic community as a whole. May (2018) argues similarly that because there "is just too much torment wreaked upon too many animals and too certain a prospect that this is going to continue and probably increase," we should take seriously the idea that human extinction would be morally desirable. Our abysmal treatment of non-human animals may also be thought to bode ill for our potential treatment of other kinds of minds with whom we might conceivably share the future and view primarily as tools: namely, minds that might arise from inorganic computational substrates, given suitable developments in the field of artificial intelligence (Saad and Bradley forthcoming). This paper takes up the question of whether and to what extent the continued existence of humanity is morally desirable. For the sake of brevity, I'll refer to this as the value of the future , leaving the assumption that we conditionalize on human survival impl...

"UDT shows that decision theory is more puzzling than ever" by Wei Dai

Play Episode Listen Later Sep 18, 2023 2:44

I feel like MIRI perhaps mispositioned FDT (their variant of UDT) as a clear advancement in decision theory, whereas maybe they could have attracted more attention/interest from academic philosophy if the framing was instead that the UDT line of thinking shows that decision theory is just more deeply puzzling than anyone had previously realized. Instead of one major open problem (Newcomb's, or EDT vs CDT) now we have a whole bunch more. I'm really not sure at this point whether UDT is even on the right track, but it does seem clear that there are some thorny issues in decision theory that not many people were previously thinking about:Source:https://www.lesswrong.com/posts/wXbSAKu2AcohaK2Gt/udt-shows-that-decision-theory-is-more-puzzling-than-everNarrated for LessWrong by TYPE III AUDIO.Share feedback on this narration.[125+ Karma Post] ✓[Curated Post] ✓

edt miri puzzling cdt newcomb fdt udt lesswrong decision theory wei dai

AF - UDT shows that decision theory is more puzzling than ever by Wei Dai

Play Episode Listen Later Sep 13, 2023 2:36

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: UDT shows that decision theory is more puzzling than ever, published by Wei Dai on September 13, 2023 on The AI Alignment Forum. I feel like MIRI perhaps mispositioned FDT (their variant of UDT) as a clear advancement in decision theory, whereas maybe they could have attracted more attention/interest from academic philosophy if the framing was instead that the UDT line of thinking shows that decision theory is just more deeply puzzling than anyone had previously realized. Instead of one major open problem (Newcomb's, or EDT vs CDT) now we have a whole bunch more. I'm really not sure at this point whether UDT is even on the right track, but it does seem clear that there are some thorny issues in decision theory that not many people were previously thinking about: Indexical values are not reflectively consistent. UDT "solves" this problem by implicitly assuming (via the type signature of its utility function) that the agent doesn't have indexical values. But humans seemingly do have indexical values, so what to do about that? The commitment races problem extends into logical time, and it's not clear how to make the most obvious idea of logical updatelessness work. UDT says that what we normally think of as different approaches to anthropic reasoning are really different preferences, which seems to sidestep the problem. But is that actually right, and if so where are these preferences supposed to come from? 2TDT-1CDT - If there's a population of mostly TDT/UDT agents and few CDT agents (and nobody knows who the CDT agents are) and they're randomly paired up to play one-shot PD, then the CDT agents do better. What does this imply? Game theory under the UDT line of thinking is generally more confusing than anything CDT agents have to deal with. UDT assumes that the agent has access to its own source code and inputs as symbol strings, so it can potentially reason about logical correlations between its own decisions and other agents' as well defined mathematical problems. But humans don't have this, so how are humans supposed to reason about such correlations? Logical conditionals vs counterfactuals, how should these be defined and do the definitions actually lead to reasonable decisions when plugged into logical decision theory? These are just the major problems that I was trying to solve (or hoping for others to solve) before I mostly stopped working on decision theory and switched my attention to metaphilosophy. (It's been a while so I'm not certain the list is complete.) As far as I know nobody has found definitive solutions to any of these problems yet, and most are wide open. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

game speech ea pd logical edt miri puzzling cdt newcomb rationalist fdt udt decision theory wei dai

LW - UDT shows that decision theory is more puzzling than ever by Wei Dai

Play Episode Listen Later Sep 13, 2023 2:36

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: UDT shows that decision theory is more puzzling than ever, published by Wei Dai on September 13, 2023 on LessWrong. I feel like MIRI perhaps mispositioned FDT (their variant of UDT) as a clear advancement in decision theory, whereas maybe they could have attracted more attention/interest from academic philosophy if the framing was instead that the UDT line of thinking shows that decision theory is just more deeply puzzling than anyone had previously realized. Instead of one major open problem (Newcomb's, or EDT vs CDT) now we have a whole bunch more. I'm really not sure at this point whether UDT is even on the right track, but it does seem clear that there are some thorny issues in decision theory that not many people were previously thinking about: Indexical values are not reflectively consistent. UDT "solves" this problem by implicitly assuming (via the type signature of its utility function) that the agent doesn't have indexical values. But humans seemingly do have indexical values, so what to do about that? The commitment races problem extends into logical time, and it's not clear how to make the most obvious idea of logical updatelessness work. UDT says that what we normally think of different approaches to anthropic reasoning are really different preferences, which seems to sidestep the problem. But is that actually right, and if so where are these preferences supposed to come from? 2TDT-1CDT - If there's a population of mostly TDT/UDT agents and few CDT agents (and nobody knows who the CDT agents are) and they're randomly paired up to play one-shot PD, then the CDT agents do better. What does this imply? Game theory under the UDT line of thinking is generally more confusing than anything CDT agents have to deal with. UDT assumes that the agent has access to its own source code and inputs as symbol strings, so it can potentially reason about logical correlations between its own decisions and other agents' as well defined mathematical problems. But humans don't have this, so how are humans supposed to reason about such correlations? Logical conditionals vs counterfactuals, how should these be defined and do the definitions actually lead to reasonable decisions when plugged into logical decision theory? These are just the major problems that I was trying to solve (or hoping for others to solve) before I mostly stopped working on decision theory and switched my attention to metaphilosophy. (It's been a while so I'm not certain the list is complete.) As far as I know nobody has found definitive solutions to any of these problems yet, and most are wide open. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

game speech ea pd logical edt miri puzzling cdt newcomb rationalist fdt udt lesswrong decision theory wei dai

LW - UDT shows that decision theory is more puzzling than ever by Wei Dai

The Nonlinear Library: LessWrong Daily

Play Episode Listen Later Sep 13, 2023 2:36

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: UDT shows that decision theory is more puzzling than ever, published by Wei Dai on September 13, 2023 on LessWrong. I feel like MIRI perhaps mispositioned FDT (their variant of UDT) as a clear advancement in decision theory, whereas maybe they could have attracted more attention/interest from academic philosophy if the framing was instead that the UDT line of thinking shows that decision theory is just more deeply puzzling than anyone had previously realized. Instead of one major open problem (Newcomb's, or EDT vs CDT) now we have a whole bunch more. I'm really not sure at this point whether UDT is even on the right track, but it does seem clear that there are some thorny issues in decision theory that not many people were previously thinking about: Indexical values are not reflectively consistent. UDT "solves" this problem by implicitly assuming (via the type signature of its utility function) that the agent doesn't have indexical values. But humans seemingly do have indexical values, so what to do about that? The commitment races problem extends into logical time, and it's not clear how to make the most obvious idea of logical updatelessness work. UDT says that what we normally think of different approaches to anthropic reasoning are really different preferences, which seems to sidestep the problem. But is that actually right, and if so where are these preferences supposed to come from? 2TDT-1CDT - If there's a population of mostly TDT/UDT agents and few CDT agents (and nobody knows who the CDT agents are) and they're randomly paired up to play one-shot PD, then the CDT agents do better. What does this imply? Game theory under the UDT line of thinking is generally more confusing than anything CDT agents have to deal with. UDT assumes that the agent has access to its own source code and inputs as symbol strings, so it can potentially reason about logical correlations between its own decisions and other agents' as well defined mathematical problems. But humans don't have this, so how are humans supposed to reason about such correlations? Logical conditionals vs counterfactuals, how should these be defined and do the definitions actually lead to reasonable decisions when plugged into logical decision theory? These are just the major problems that I was trying to solve (or hoping for others to solve) before I mostly stopped working on decision theory and switched my attention to metaphilosophy. (It's been a while so I'm not certain the list is complete.) As far as I know nobody has found definitive solutions to any of these problems yet, and most are wide open. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

game speech ea pd logical edt miri puzzling cdt newcomb rationalist fdt udt lesswrong decision theory wei dai

LW - UDT shows that decision theory is more puzzling than ever by Wei Dai

Play Episode Listen Later Sep 13, 2023 2:36

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: UDT shows that decision theory is more puzzling than ever, published by Wei Dai on September 13, 2023 on LessWrong.I feel like MIRI perhaps mispositioned FDT (their variant of UDT) as a clear advancement in decision theory, whereas maybe they could have attracted more attention/interest from academic philosophy if the framing was instead that the UDT line of thinking shows that decision theory is just more deeply puzzling than anyone had previously realized. Instead of one major open problem (Newcomb's, or EDT vs CDT) now we have a whole bunch more. I'm really not sure at this point whether UDT is even on the right track, but it does seem clear that there are some thorny issues in decision theory that not many people were previously thinking about:Indexical values are not reflectively consistent. UDT "solves" this problem by implicitly assuming (via the type signature of its utility function) that the agent doesn't have indexical values. But humans seemingly do have indexical values, so what to do about that?The commitment races problem extends into logical time, and it's not clear how to make the most obvious idea of logical updatelessness work.UDT says that what we normally think of different approaches to anthropic reasoning are really different preferences, which seems to sidestep the problem. But is that actually right, and if so where are these preferences supposed to come from?2TDT-1CDT - If there's a population of mostly TDT/UDT agents and few CDT agents (and nobody knows who the CDT agents are) and they're randomly paired up to play one-shot PD, then the CDT agents do better. What does this imply?Game theory under the UDT line of thinking is generally more confusing than anything CDT agents have to deal with.UDT assumes that the agent has access to its own source code and inputs as symbol strings, so it can potentially reason about logical correlations between its own decisions and other agents' as well defined mathematical problems. But humans don't have this, so how are humans supposed to reason about such correlations?Logical conditionals vs counterfactuals, how should these be defined and do the definitions actually lead to reasonable decisions when plugged into logical decision theory?These are just the major problems that I was trying to solve (or hoping for others to solve) before I mostly stopped working on decision theory and switched my attention to metaphilosophy. (It's been a while so I'm not certain the list is complete.) As far as I know nobody has found definitive solutions to any of these problems yet, and most are wide open.Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

game speech ea pd logical edt miri puzzling cdt newcomb rationalist fdt udt lesswrong decision theory wei dai

1 Second-Decision Theory | Motivational

Successful

Play Episode Listen Later Sep 8, 2023 4:14

Speaker: David Goggins

motivational decision theory

LW - Responses to apparent rationalist confusions about game / decision theory by Anthony DiGiovanni

The Nonlinear Library: LessWrong Daily

Play Episode Listen Later Aug 31, 2023 27:14

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Responses to apparent rationalist confusions about game / decision theory, published by Anthony DiGiovanni on August 31, 2023 on LessWrong. I've encountered various claims about how AIs would approach game theory and decision theory that seem pretty importantly mistaken. Some of these confusions probably aren't that big a deal on their own, and I'm definitely not the first to point out several of these, even publicly. But collectively I think these add up to a common worldview that underestimates the value of technical work to reduce risks of AGI conflict. I expect that smart agents will likely avoid catastrophic conflict overall - it's just that the specific arguments for expecting this that I'm responding to here aren't compelling (and seem overconfident). For each section, I include in the footnotes some examples of the claims I'm pushing back on (or note whether I've primarily seen these claims in personal communication). This is not to call out those particular authors; in each case, they're saying something that seems to be a relatively common meme in this community. Summary: The fact that conflict is costly for all the agents involved in the conflict, ex post, doesn't itself imply AGIs won't end up in conflict. Under their uncertainty about each other, agents with sufficiently extreme preferences or priors might find the risk of conflict worth it ex ante. (more) Solutions to collective action problems, where agents agree on a Pareto-optimal outcome they'd take if they coordinated to do so, don't necessarily solve bargaining problems, where agents may insist on different Pareto-optimal outcomes. (more) We don't have strong reasons to expect AGIs to converge on sufficiently similar decision procedures for bargaining, such that they coordinate on fair demands despite committing under uncertainty. Existing proposals for mitigating conflict given incompatible demands, while promising, face some problems with incentives and commitment credibility. (more) The commitment races problem is not just about AIs making commitments that fail to account for basic contingencies. Updatelessness (or conditional commitments generally) seems to solve the latter, but it doesn't remove agents' incentives to limit how much their decisions depend on each other's decisions (leading to incompatible demands). (more) AIs don't need to follow acausal decision theories in order to (causally) cooperate via conditioning on each other's source code. (more) Most supposed examples of Newcomblike problems in everyday life don't seem to actually be Newcomblike, once we account for "screening off" by certain information, per the Tickle Defense. (more) The fact that following acausal decision theories maximizes expected utility with respect to conditional probabilities, or counterfactuals with the possibility of logical causation, doesn't imply that agents with acausal decision theories are selected for (e.g., acquire more material resources). (more) Ex post optimal =/= ex ante optimal An "ex post optimal" strategy is one that in fact makes an agent better off than the alternatives, while an "ex ante optimal" strategy is optimal with respect to the agent's uncertainty at the time they choose that strategy. The idea that very smart AGIs could get into conflicts seems intuitively implausible because conflict is, by definition, ex post Pareto-suboptimal. (See the "inefficiency puzzle of war.") But it doesn't follow that the best strategies available to AGIs given their uncertainty about each other will always be ex post Pareto-optimal. This may sound obvious, but my experience with seeing people's reactions to the problem of AGI conflict suggests that many of them haven't accounted for this important distinction. As this post discusses in more detail, there are two fundamental sources of uncertainty (o...

game ai speech ea responses existing agi pareto apparent agis confusions rationalist digiovanni lesswrong decision theory

LW - Responses to apparent rationalist confusions about game / decision theory by Anthony DiGiovanni

Play Episode Listen Later Aug 31, 2023 27:14

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Responses to apparent rationalist confusions about game / decision theory, published by Anthony DiGiovanni on August 31, 2023 on LessWrong.I've encountered various claims about how AIs would approach game theory and decision theory that seem pretty importantly mistaken. Some of these confusions probably aren't that big a deal on their own, and I'm definitely not the first to point out several of these, even publicly. But collectively I think these add up to a common worldview that underestimates the value of technical work to reduce risks of AGI conflict. I expect that smart agents will likely avoid catastrophic conflict overall - it's just that the specific arguments for expecting this that I'm responding to here aren't compelling (and seem overconfident).For each section, I include in the footnotes some examples of the claims I'm pushing back on (or note whether I've primarily seen these claims in personal communication). This is not to call out those particular authors; in each case, they're saying something that seems to be a relatively common meme in this community.Summary:The fact that conflict is costly for all the agents involved in the conflict, ex post, doesn't itself imply AGIs won't end up in conflict. Under their uncertainty about each other, agents with sufficiently extreme preferences or priors might find the risk of conflict worth it ex ante. (more)Solutions to collective action problems, where agents agree on a Pareto-optimal outcome they'd take if they coordinated to do so, don't necessarily solve bargaining problems, where agents may insist on different Pareto-optimal outcomes. (more)We don't have strong reasons to expect AGIs to converge on sufficiently similar decision procedures for bargaining, such that they coordinate on fair demands despite committing under uncertainty. Existing proposals for mitigating conflict given incompatible demands, while promising, face some problems with incentives and commitment credibility. (more)The commitment races problem is not just about AIs making commitments that fail to account for basic contingencies. Updatelessness (or conditional commitments generally) seems to solve the latter, but it doesn't remove agents' incentives to limit how much their decisions depend on each other's decisions (leading to incompatible demands). (more)AIs don't need to follow acausal decision theories in order to (causally) cooperate via conditioning on each other's source code. (more)Most supposed examples of Newcomblike problems in everyday life don't seem to actually be Newcomblike, once we account for "screening off" by certain information, per the Tickle Defense. (more)The fact that following acausal decision theories maximizes expected utility with respect to conditional probabilities, or counterfactuals with the possibility of logical causation, doesn't imply that agents with acausal decision theories are selected for (e.g., acquire more material resources). (more)Ex post optimal =/= ex ante optimalAn "ex post optimal" strategy is one that in fact makes an agent better off than the alternatives, while an "ex ante optimal" strategy is optimal with respect to the agent's uncertainty at the time they choose that strategy. The idea that very smart AGIs could get into conflicts seems intuitively implausible because conflict is, by definition, ex post Pareto-suboptimal. (See the "inefficiency puzzle of war.")But it doesn't follow that the best strategies available to AGIs given their uncertainty about each other will always be ex post Pareto-optimal. This may sound obvious, but my experience with seeing people's reactions to the problem of AGI conflict suggests that many of them haven't accounted for this important distinction.As this post discusses in more detail, there are two fundamental sources of uncertainty (o...

game ai speech ea responses existing agi pareto apparent agis confusions rationalist digiovanni lesswrong decision theory

LW - Responses to apparent rationalist confusions about game / decision theory by Anthony DiGiovanni

Play Episode Listen Later Aug 31, 2023 27:14

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Responses to apparent rationalist confusions about game / decision theory, published by Anthony DiGiovanni on August 31, 2023 on LessWrong. I've encountered various claims about how AIs would approach game theory and decision theory that seem pretty importantly mistaken. Some of these confusions probably aren't that big a deal on their own, and I'm definitely not the first to point out several of these, even publicly. But collectively I think these add up to a common worldview that underestimates the value of technical work to reduce risks of AGI conflict. I expect that smart agents will likely avoid catastrophic conflict overall - it's just that the specific arguments for expecting this that I'm responding to here aren't compelling (and seem overconfident). For each section, I include in the footnotes some examples of the claims I'm pushing back on (or note whether I've primarily seen these claims in personal communication). This is not to call out those particular authors; in each case, they're saying something that seems to be a relatively common meme in this community. Summary: The fact that conflict is costly for all the agents involved in the conflict, ex post, doesn't itself imply AGIs won't end up in conflict. Under their uncertainty about each other, agents with sufficiently extreme preferences or priors might find the risk of conflict worth it ex ante. (more) Solutions to collective action problems, where agents agree on a Pareto-optimal outcome they'd take if they coordinated to do so, don't necessarily solve bargaining problems, where agents may insist on different Pareto-optimal outcomes. (more) We don't have strong reasons to expect AGIs to converge on sufficiently similar decision procedures for bargaining, such that they coordinate on fair demands despite committing under uncertainty. Existing proposals for mitigating conflict given incompatible demands, while promising, face some problems with incentives and commitment credibility. (more) The commitment races problem is not just about AIs making commitments that fail to account for basic contingencies. Updatelessness (or conditional commitments generally) seems to solve the latter, but it doesn't remove agents' incentives to limit how much their decisions depend on each other's decisions (leading to incompatible demands). (more) AIs don't need to follow acausal decision theories in order to (causally) cooperate via conditioning on each other's source code. (more) Most supposed examples of Newcomblike problems in everyday life don't seem to actually be Newcomblike, once we account for "screening off" by certain information, per the Tickle Defense. (more) The fact that following acausal decision theories maximizes expected utility with respect to conditional probabilities, or counterfactuals with the possibility of logical causation, doesn't imply that agents with acausal decision theories are selected for (e.g., acquire more material resources). (more) Ex post optimal =/= ex ante optimal An "ex post optimal" strategy is one that in fact makes an agent better off than the alternatives, while an "ex ante optimal" strategy is optimal with respect to the agent's uncertainty at the time they choose that strategy. The idea that very smart AGIs could get into conflicts seems intuitively implausible because conflict is, by definition, ex post Pareto-suboptimal. (See the "inefficiency puzzle of war.") But it doesn't follow that the best strategies available to AGIs given their uncertainty about each other will always be ex post Pareto-optimal. This may sound obvious, but my experience with seeing people's reactions to the problem of AGI conflict suggests that many of them haven't accounted for this important distinction. As this post discusses in more detail, there are two fundamental sources of uncertainty (o...

game ai speech ea responses existing agi pareto apparent agis confusions rationalist digiovanni lesswrong decision theory

AF - Responses to apparent rationalist confusions about game / decision theory by Anthony DiGiovanni

google ai future psychology search philosophy ethics seo tedx conscious using ai talking points tom finn decision theory

Play Episode Listen Later Aug 30, 2023 27:14

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Responses to apparent rationalist confusions about game / decision theory, published by Anthony DiGiovanni on August 30, 2023 on The AI Alignment Forum. I've encountered various claims about how AIs would approach game theory and decision theory that seem pretty importantly mistaken. Some of these confusions probably aren't that big a deal on their own, and I'm definitely not the first to point out several of these, even publicly. But collectively I think these add up to a common worldview that underestimates the value of technical work to reduce risks of AGI conflict. I expect that smart agents will likely avoid catastrophic conflict overall - it's just that the specific arguments for expecting this that I'm responding to here aren't compelling (and seem overconfident). For each section, I include in the footnotes some examples of the claims I'm pushing back on (or note whether I've primarily seen these claims in personal communication). This is not to call out those particular authors; in each case, they're saying something that seems to be a relatively common meme in this community. Summary: The fact that conflict is costly for all the agents involved in the conflict, ex post, doesn't itself imply AGIs won't end up in conflict. Under their uncertainty about each other, agents with sufficiently extreme preferences or priors might find the risk of conflict worth it ex ante. (more) Solutions to collective action problems, where agents agree on a Pareto-optimal outcome they'd take if they coordinated to do so, don't necessarily solve bargaining problems, where agents may insist on different Pareto-optimal outcomes. (more) We don't have strong reasons to expect AGIs to converge on sufficiently similar decision procedures for bargaining, such that they coordinate on fair demands despite committing under uncertainty. Existing proposals for mitigating conflict given incompatible demands, while promising, face some problems with incentives and commitment credibility. (more) The commitment races problem is not just about AIs making commitments that fail to account for basic contingencies. Updatelessness (or conditional commitments generally) seems to solve the latter, but it doesn't remove agents' incentives to limit how much their decisions depend on each other's decisions (leading to incompatible demands). (more) AIs don't need to follow acausal decision theories in order to (causally) cooperate via conditioning on each other's source code. (more) Most supposed examples of Newcomblike problems in everyday life don't seem to actually be Newcomblike, once we account for "screening off" by certain information, per the Tickle Defense. (more) The fact that following acausal decision theories maximizes expected utility with respect to conditional probabilities, or counterfactuals with the possibility of logical causation, doesn't imply that agents with acausal decision theories are selected for (e.g., acquire more material resources). (more) Ex post optimal =/= ex ante optimal An "ex post optimal" strategy is one that in fact makes an agent better off than the alternatives, while an "ex ante optimal" strategy is optimal with respect to the agent's uncertainty at the time they choose that strategy. The idea that very smart AGIs could get into conflicts seems intuitively implausible because conflict is, by definition, ex post Pareto-suboptimal. (See the "inefficiency puzzle of war.") But it doesn't follow that the best strategies available to AGIs given their uncertainty about each other will always be ex post Pareto-optimal. This may sound obvious, but my experience with seeing people's reactions to the problem of AGI conflict suggests that many of them haven't accounted for this important distinction. As this post discusses in more detail, there are two fundamental sources of u...

game ai speech ea responses existing agi pareto apparent agis confusions rationalist digiovanni decision theory

Creating a Conscious AI with Tom Finn and Josh Bachynski

Talent Empowerment

Play Episode Listen Later Jun 29, 2023 45:19

Can AI be more human? Josh Bachynski is an SEO/AI Expert that is currently developing a self-aware AI named Kassandra. In this episode, Josh talks about artificial intelligence, the path of creating an AI with ethics, and how an autism diagnosis in his late 40s made him realize his “neuro-spiciness”. Talking Points:(1:24) Creating a self-aware AI(5:36) The difference between Chat GPT and a self-aware AI(10:39) The Ethics of AI(22:45) When too much self-awareness becomes a weakness(28:34) Using AI to make a positive impact(35:17) Josh's autism diagnosis in his late 40s About Josh Bachynski:Josh Bachynski is an SEO & AI Expert/Marketer with over 20 years of business branding, marketing, and SEO experience. Josh has spoken on ethics for over 20 years, including a TEDx talk "The Future of Google, Search and Ethics". He is a 2nd yr Ph.D. student (retired) in Philosophy, Psychology and has an MA in Ethics and Decision Theory.

A Tale of Two Players: Exploring the Rubinstein Bargaining Model | with Ariel Rubinstein

Game Changer - the game theory podcast

Play Episode Listen Later Jun 19, 2023 32:55

In today's episode, we explore one of the classics in Bargaining theory: The Rubinstein Bargaining Model. And we have found the perfect guest - who better to explain this bargaining model than its founder Ariel Rubinstein himself! Ariel not only shares how the idea of the model came to be, but he also comments on some results and critically discusses whether the Rubinstein Bargaining Model (and Game Theory in general) has predictive or normative power for real-life situations. Ariel Rubinstein is Professor of Economics at the School of Economics at Tel Aviv University and the Department of Economics at New York University. His research is focused on Economic Theory, in particular Decision Theory and Game Theory. You can download his books for free (also the book “Economic fables” mentioned in our episode) and check out his Atlas of Cafés on his website https://arielrubinstein.tau.ac.il/ . There, you also find his original paper introducing what came to be know the “Rubinstein Bargaining Model”: “Perfect Equilibrium in a Bargaining Model”

school professor tale model economics economic new york university game theory bargaining tel aviv university rubinstein two players economic theory decision theory

LW - Decision Theory with the Magic Parts Highlighted by moridinamael

Play Episode Listen Later May 16, 2023 7:43

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Decision Theory with the Magic Parts Highlighted, published by moridinamael on May 16, 2023 on LessWrong. I. The Magic Parts of Decision Theory You are throwing a birthday party this afternoon and want to decide where to hold it. You aren't sure whether it will rain or not. If it rains, you would prefer not to have committed to throwing the party outside. If it's sunny, though, you will regret having set up inside. You also have a covered porch which isn't quite as nice as being out in the sun would be, but confers some protection from the elements in case of bad weather. You break this problem down into a simple decision tree. This operation requires magic, to avert the completely intractable combinatorial explosion inherent in the problem statement. After all, what does "Rain" mean? A single drop of rain? A light sprinkling? Does it only count as "Rain" if it's a real deluge? For what duration? In what area? Just in the back yard? What if it looks rainy but doesn't rain? What if there's lightning but not rain? What if it's merely overcast and humid? Which of these things count as Rain? And how crisply did you define the Indoors versus Porch versus Outdoors options? What about the option of setting up mostly outside but leaving the cake inside, just in case? There are about ten billion different permutations of what "Outdoors" could look like, after all - how did you determine which options need to be explicitly represented? Why not include Outside-With-Piñata and Outside-Without-Piñata as two separate options? How did you determine that "Porch" doesn't count as "Outdoors" since it's still "outside" by any sensible definition? Luckily you're a human being, so you used ineffable magic to condense the decision tree with a hundred trillion leaf nodes down into a tree with only six. You're a rigorous thinker, so the next step, of course, is to assign utilities to each outcome, scaled from 0 to 100, in order to represent your preference ordering and the relative weight of these preferences. Maybe you do this explicitly with numbers, maybe you do it by gut feel. This step also requires magic; an enormously complex set of implicit understandings come into play, which allow you to simply know how and why the party would probably be a bit better if you were on the Porch in Sunny weather than Indoors in Rainy weather. Be aware that there is not some infinitely complex True Utility Function that you are consulting or sampling from, you simply are served with automatically-arising emotions and thoughts upon asking yourself these questions about relative preference, resulting in a consistent ranking and utility valuation. Nor are these emotions and thoughts approximations of a secret, hidden True Utility Function; you do not have one of those, and if you did, how on Earth would you actually use it in this situation? How would you use it to calculate relative preference of Porch-with-Rain versus Indoors-with-Sun unless it already contained exactly that comparison of world-states somewhere inside it? Next you perform the trivial-to-you act of assigning probability of Rain versus Sun, which of course requires magic. You have to rely on your previous, ineffable distinction of what Rain versus Sun means in the first place, and then aggregate vast amounts of data, including what the sky looks like and what the air feels like (with your lifetime of experience guiding how you interpret what you see and feel), what three different weather reports say weighted by ineffable assignments of credibility, and what that implies for your specific back yard, plus the timing of the party, into a relatively reliable probability estimate. What's that, you say? An ideal Bayesian reasoner would be able to do this better? No such thing exists; it is "ideal" because it is pretend. For very simple reason...

earth magic sun rain speech outdoors ea porch rainy indoors highlighted bayesian rationalist lesswrong decision theory

LW - Decision Theory with the Magic Parts Highlighted by moridinamael

Play Episode Listen Later May 16, 2023 7:43

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Decision Theory with the Magic Parts Highlighted, published by moridinamael on May 16, 2023 on LessWrong. I. The Magic Parts of Decision Theory You are throwing a birthday party this afternoon and want to decide where to hold it. You aren't sure whether it will rain or not. If it rains, you would prefer not to have committed to throwing the party outside. If it's sunny, though, you will regret having set up inside. You also have a covered porch which isn't quite as nice as being out in the sun would be, but confers some protection from the elements in case of bad weather. You break this problem down into a simple decision tree. This operation requires magic, to avert the completely intractable combinatorial explosion inherent in the problem statement. After all, what does "Rain" mean? A single drop of rain? A light sprinkling? Does it only count as "Rain" if it's a real deluge? For what duration? In what area? Just in the back yard? What if it looks rainy but doesn't rain? What if there's lightning but not rain? What if it's merely overcast and humid? Which of these things count as Rain? And how crisply did you define the Indoors versus Porch versus Outdoors options? What about the option of setting up mostly outside but leaving the cake inside, just in case? There are about ten billion different permutations of what "Outdoors" could look like, after all - how did you determine which options need to be explicitly represented? Why not include Outside-With-Piñata and Outside-Without-Piñata as two separate options? How did you determine that "Porch" doesn't count as "Outdoors" since it's still "outside" by any sensible definition? Luckily you're a human being, so you used ineffable magic to condense the decision tree with a hundred trillion leaf nodes down into a tree with only six. You're a rigorous thinker, so the next step, of course, is to assign utilities to each outcome, scaled from 0 to 100, in order to represent your preference ordering and the relative weight of these preferences. Maybe you do this explicitly with numbers, maybe you do it by gut feel. This step also requires magic; an enormously complex set of implicit understandings come into play, which allow you to simply know how and why the party would probably be a bit better if you were on the Porch in Sunny weather than Indoors in Rainy weather. Be aware that there is not some infinitely complex True Utility Function that you are consulting or sampling from, you simply are served with automatically-arising emotions and thoughts upon asking yourself these questions about relative preference, resulting in a consistent ranking and utility valuation. Nor are these emotions and thoughts approximations of a secret, hidden True Utility Function; you do not have one of those, and if you did, how on Earth would you actually use it in this situation? How would you use it to calculate relative preference of Porch-with-Rain versus Indoors-with-Sun unless it already contained exactly that comparison of world-states somewhere inside it? Next you perform the trivial-to-you act of assigning probability of Rain versus Sun, which of course requires magic. You have to rely on your previous, ineffable distinction of what Rain versus Sun means in the first place, and then aggregate vast amounts of data, including what the sky looks like and what the air feels like (with your lifetime of experience guiding how you interpret what you see and feel), what three different weather reports say weighted by ineffable assignments of credibility, and what that implies for your specific back yard, plus the timing of the party, into a relatively reliable probability estimate. What's that, you say? An ideal Bayesian reasoner would be able to do this better? No such thing exists; it is "ideal" because it is pretend. For very simple reason...

earth magic sun rain speech outdoors ea porch rainy indoors highlighted bayesian rationalist lesswrong decision theory

The Sinister Truth Behind an AI-Driven World w/ TedX Speaker, Decision Theory Expert & Philosopher Josh Bachynski

Grow & Monetize

Play Episode Listen Later May 5, 2023 48:01

AI is going to "take over" but it will be a net positive. In today's episode, I'm joined by a modern-day genius, AI ethics philosopher, and Google insider Josh Bachynski. Josh drops dozens of truth bombs on where we're headed as a species, how AI is going to impact our future, and the mindset of the power-addicted tech billionaires running the algorithms behind the scenes. Tap in for a can't-miss conversation at the intersection of AI, philosophy, and simulation theory as Josh and I discuss what it means to be human in an increasingly tech-dominated world.

google ai driven acast tap philosophers yt tw truth behind tedx speaker sinister decision theory

LW - Assigning Praise and Blame: Decoupling Epistemology and Decision Theory by adamShimi

praise speech blame ea incentives epistemology assigning decoupling heuristics rationalist lesswrong decision theory

Play Episode Listen Later Jan 27, 2023 5:51

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Assigning Praise and Blame: Decoupling Epistemology and Decision Theory, published by adamShimi on January 27, 2023 on LessWrong. Your group/company/organization performs well, doing great work and dealing with new problems efficiently. As one of its leaders, you want to understand why, so that you can make it even more successful, and maybe emulate this success in other settings. Your group/company/organization performs badly, not delivering on what was promised and missing deadline after deadline. As one of its leaders, you want to understand why, so that you can correct its course, or at least not repeat the same mistakes in other settings. Both cases apparently involve credit assignment: positive credit (praise) for success or negative credit (blame) for failure. And you can easily think of different ways to do so: Heuristics for Credit Assignment Baseline The most straightforward approach starts with your initial prediction, and then assigns credit for deviations from it. So praise people who did better than expected and blame people who did worse than expected. Then you remember Janice. She's your star performer, amazing in everything she does, and you knew it from the start. So she performed according to prediction, being brilliant and reliable. Which means she doesn't deserve any praise by this criterion. On the other hand there is Tom. He's quite good, but you also knew from the start he was a prickly showoff with an easily scratched ego. Still, he did his job, and when he acted like an asshole, that was within the prediction. So he doesn't deserve any blame by this criterion. Incentive wise, this sounds like a terrible idea. If you push this credit assignment strategy, not only will you neglect the value of Janice and the cost of Tom, but you will probably drive away high-performers and attract problem-makers. Bottleneck Instead of starting from a baseline, let's focus on the key bottlenecks. What would have doomed the project if not done? What ended up blocking everything and dooming the project? This focuses on the real cruxes, which is good. Yet what about Marcel, your tireless tool engineer? None of his stuff is ever absolutely necessary, but everyone in your group constantly mentions the value they get from his well-maintained and efficient tools. Should he not get any credit for it? And Bertha, who is the only security expert of your group, and always finds excuses to make herself more and more necessary? Is this really a behavior you want to condone and praise? Shouldn't she get blamed for it instead? It's at this point that you remember this short parable. First cause No, really, what matters most is the initial spark that puts everything in motion. Without the original idea, nothing happens; skills and expertise remain useless and flaccid, unable to toil for a worthwhile goal. And what a coincidence: you were one of these key idea generators! All the more power to you, then. It's only fair that you get a large share of the credit, given that the group wouldn't even exist without you. But that still doesn't seem right. Yes, your contribution was integral. But could you really have done it by yourself? Probably not. Or at least not as well, as quickly, as beautifully as it has been. Or conversely, if it failed totally, was it because the idea was doomed from the start, or because the execution proved bad enough to torpedo even sensible propositions? Final step You got it exactly wrong above: it's not the first step that trumps them all, it's the final one. Making the abstract real, adding the finishing touches, this is what makes success or failure. So you should focus your assignment on the success and failure of these last steps. But what about you? Yes, you. You have never finished anything yourself, you're the organizer, idea maker, coordinator. ...

LW - Assigning Praise and Blame: Decoupling Epistemology and Decision Theory by adamShimi

praise speech blame ea incentives epistemology assigning decoupling heuristics rationalist lesswrong decision theory

Play Episode Listen Later Jan 27, 2023 5:51

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Assigning Praise and Blame: Decoupling Epistemology and Decision Theory, published by adamShimi on January 27, 2023 on LessWrong. Your group/company/organization performs well, doing great work and dealing with new problems efficiently. As one of its leaders, you want to understand why, so that you can make it even more successful, and maybe emulate this success in other settings. Your group/company/organization performs badly, not delivering on what was promised and missing deadline after deadline. As one of its leaders, you want to understand why, so that you can correct its course, or at least not repeat the same mistakes in other settings. Both cases apparently involve credit assignment: positive credit (praise) for success or negative credit (blame) for failure. And you can easily think of different ways to do so: Heuristics for Credit Assignment Baseline The most straightforward approach starts with your initial prediction, and then assigns credit for deviations from it. So praise people who did better than expected and blame people who did worse than expected. Then you remember Janice. She's your star performer, amazing in everything she does, and you knew it from the start. So she performed according to prediction, being brilliant and reliable. Which means she doesn't deserve any praise by this criterion. On the other hand there is Tom. He's quite good, but you also knew from the start he was a prickly showoff with an easily scratched ego. Still, he did his job, and when he acted like an asshole, that was within the prediction. So he doesn't deserve any blame by this criterion. Incentive wise, this sounds like a terrible idea. If you push this credit assignment strategy, not only will you neglect the value of Janice and the cost of Tom, but you will probably drive away high-performers and attract problem-makers. Bottleneck Instead of starting from a baseline, let's focus on the key bottlenecks. What would have doomed the project if not done? What ended up blocking everything and dooming the project? This focuses on the real cruxes, which is good. Yet what about Marcel, your tireless tool engineer? None of his stuff is ever absolutely necessary, but everyone in your group constantly mentions the value they get from his well-maintained and efficient tools. Should he not get any credit for it? And Bertha, who is the only security expert of your group, and always finds excuses to make herself more and more necessary? Is this really a behavior you want to condone and praise? Shouldn't she get blamed for it instead? It's at this point that you remember this short parable. First cause No, really, what matters most is the initial spark that puts everything in motion. Without the original idea, nothing happens; skills and expertise remain useless and flaccid, unable to toil for a worthwhile goal. And what a coincidence: you were one of these key idea generators! All the more power to you, then. It's only fair that you get a large share of the credit, given that the group wouldn't even exist without you. But that still doesn't seem right. Yes, your contribution was integral. But could you really have done it by yourself? Probably not. Or at least not as well, as quickly, as beautifully as it has been. Or conversely, if it failed totally, was it because the idea was doomed from the start, or because the execution proved bad enough to torpedo even sensible propositions? Final step You got it exactly wrong above: it's not the first step that trumps them all, it's the final one. Making the abstract real, adding the finishing touches, this is what makes success or failure. So you should focus your assignment on the success and failure of these last steps. But what about you? Yes, you. You have never finished anything yourself, you're the organizer, idea maker, coordinator. ...

Troy Cross Talks Bitcoin Philosophy with Knut Svanholm

The Freedom Footprint Show: A Bitcoin Podcast

Play Episode Listen Later Dec 15, 2022 112:04

Troy Cross joins the Freedom Footprint Show and discusses Bitcoin Philosophy with Knut Svanholm. The conversation is wide-ranging, including comparing praxeology and decision theory, ethics, the natural state of money, the benefits of printing money (none), and much more. You can find Troy on Twitter: https://twitter.com/thetrocro 00:00:21 - Introduction and welcoming Troy to the show 00:01:53 - Being an "actual" philosopher means nothing 00:04:27 - Constructs and models of Bitcoin 00:06:50 - The validity of science - a priori vs a posteriori 00:16:00 - Austrian Economics book recommendations 00:18:51 - Descartes and the relationship between praxeology and mathematics 00:26:31 - The Ethics of Liberty 00:34:10 - Decision Theory vs Praxeology 00:40:31 - Praxeology Thought Experiments 00:46:47 - The Natural State of Money 00:53:34 - The Benefit of Printing Money 01:15:19 - The Natural Money Supply 01:20:40 - Bitcoin as a measuring stick 01:22:41 - Blockspace, Clockspace, Spacetime, and Timespace 01:33:48 - Quantum Attack on Bitcoin 01:38:41 - Conclusions and Summarizing The Freedom Footprint Show is hosted by Knut Svanholm and BTCPseudoFinn. We are concerned about your Freedom Footprint! Join us as we talk to #bitcoin philosophers about how #bitcoin can expand your Freedom Footprint and much more! The Freedom Footprint Show is produced by Konsensus Network, the first #bitcoin only publishing house. Follow us on Twitter: https://twitter.com/FootprintShow https://twitter.com/knutsvanholm https://twitter.com/BtcPseudoFinn https://twitter.com/KonsensusN Visit our websites for more info: https://konsensus.network/ https://bitcoinbook.shop/ https://www.knutsvanholm.com/

cross philosophy bitcoin ethics benefit conclusions descartes spacetime constructs natural state austrian economics knut svanholm decision theory blockspace troy cross

The Skinny on Options: Abstract Applications - November 14, 2022 - Info-Gap Decision Theory

The tastytrade network

Play Episode Listen Later Nov 14, 2022 28:23

Short premium in the world of options yields a number of benefits to us as tastytraders. But just like every market strategy, there is still risk that must be absorbed somewhere. And specifically, that risk for us is the huge outlier move, or even more precisely, the series of huge outlier moves, which is where Info-Gap Decision Theory comes into play.

options applications skinny abstract decision theory

The Skinny on Options: Abstract Applications - November 14, 2022 - Info-Gap Decision Theory

The tastytrade network

Play Episode Listen Later Nov 14, 2022 29:14

options applications skinny abstract decision theory

"Decision theory does not imply that we get to have nice things" by So8res

LessWrong Curated Podcast

Play Episode Listen Later Nov 8, 2022 56:55

https://www.lesswrong.com/posts/rP66bz34crvDudzcJ/decision-theory-does-not-imply-that-we-get-to-have-niceCrossposted from the AI Alignment Forum. May contain more technical jargon than usual.(Note: I wrote this with editing help from Rob and Eliezer. Eliezer's responsible for a few of the paragraphs.)A common confusion I see in the tiny fragment of the world that knows about logical decision theory (FDT/UDT/etc.), is that people think LDT agents are genial and friendly for each other.[1]One recent example is Will Eden's tweet about how maybe a molecular paperclip/squiggle maximizer would leave humanity a few stars/galaxies/whatever on game-theoretic grounds. (And that's just one example; I hear this suggestion bandied around pretty often.)I'm pretty confident that this view is wrong (alas), and based on a misunderstanding of LDT. I shall now attempt to clear up that confusion.To begin, a parable: the entity Omicron (Omega's little sister) fills box A with $1M and box B with $1k, and puts them both in front of an LDT agent saying "You may choose to take either one or both, and know that I have already chosen whether to fill the first box". The LDT agent takes both."What?" cries the CDT agent. "I thought LDT agents one-box!"LDT agents don't cooperate because they like cooperating. They don't one-box because the name of the action starts with an 'o'. They maximize utility, using counterfactuals that assert that the world they are already in (and the observations they have already seen) can (in the right circumstances) depend (in a relevant way) on what they are later going to do.A paperclipper cooperates with other LDT agents on a one-shot prisoner's dilemma because they get more paperclips that way. Not because it has a primitive property of cooperativeness-with-similar-beings. It needs to get the more paperclips.

eliezer nice things cdt imply ldt decision theory

EA - Decision theory does not imply that we get to have nice things by So8res

decision speech ea nice things imply rationalist decision theory

Play Episode Listen Later Oct 18, 2022 0:23

LW - Decision theory does not imply that we get to have nice things by So8res

The Nonlinear Library: Alignment Forum Weekly

Play Episode Listen Later Oct 18, 2022 42:07

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Decision theory does not imply that we get to have nice things, published by So8res on October 18, 2022 on LessWrong. (Note: I wrote this with editing help from Rob and Eliezer. Eliezer's responsible for a few of the paragraphs.) A common confusion I see in the tiny fragment of the world that knows about logical decision theory (FDT/UDT/etc.), is that people think LDT agents are genial and friendly for each other. One recent example is Will Eden's tweet about how maybe a molecular paperclip/squiggle maximizer would leave humanity a few stars/galaxies/whatever on game-theoretic grounds. (And that's just one example; I hear this suggestion bandied around pretty often.) I'm pretty confident that this view is wrong (alas), and based on a misunderstanding of LDT. I shall now attempt to clear up that confusion. To begin, a parable: the entity Omicron (Omega's little sister) fills box A with $1M and box B with $1k, and puts them both in front of an LDT agent saying "You may choose to take either one or both, and know that I have already chosen whether to fill the first box". The LDT agent takes both. "What?" cries the CDT agent. "I thought LDT agents one-box!" LDT agents don't cooperate because they like cooperating. They don't one-box because the name of the action starts with an 'o'. They maximize utility, using counterfactuals that assert that the world they are already in (and the observations they have already seen) can (in the right circumstances) depend (in a relevant way) on what they are later going to do. A paperclipper cooperates with other LDT agents on a one-shot prisoner's dilemma because they get more paperclips that way. Not because it has a primitive property of cooperativeness-with-similar-beings. It needs to get the more paperclips. If a bunch of monkeys want to build a paperclipper and have it give them nice things, the paperclipper needs to somehow expect to wind up with more paperclips than it otherwise would have gotten, as a result of trading with them. If the monkeys instead create a paperclipper haplessly, then the paperclipper does not look upon them with the spirit of cooperation and toss them a few nice things anyway, on account of how we're all good LDT-using friends here. It turns them into paperclips. Because you get more paperclips that way. That's the short version. Now, I'll give the longer version. A few more words about how LDT works To set up a Newcomb's problem, it's important that the predictor does not fill the box if they predict that the agent would two-box. It's not important that they be especially good at this — you should one-box if they're more than 50.05% accurate, if we use the standard payouts ($1M and $1k as the two prizes) and your utility is linear in money — but it is important that their action is at least minimally sensitive to your future behavior. If the predictor's actions don't have this counterfactual dependency on your behavior, then take both boxes. Similarly, if an LDT agent is playing a one-shot prisoner's dilemma against a rock with the word “cooperate” written on it, it defects. At least, it defects if that's all there is to the world. It's technically possible for an LDT agent to think that the real world is made 10% of cooperate-rocks and 90% opponents who cooperate in a one-shot PD iff their opponent cooperates with them and would cooperate with cooperate-rock, in which case LDT agents cooperate against cooperate-rock. From which we learn the valuable lesson that the behavior of an LDT agent depends on the distribution of scenarios it expects to face, which means there's a subtle difference between "imagine you're playing a one-shot PD against a cooperate-rock [and that's the entire universe]" and "imagine you're playing a one-shot PD against a cooperate-rock [in a universe where you face a random oppone...

decision speech ea pd eliezer nice things cdt newcomb imply rationalist ldt lesswrong decision theory

AF - Decision theory does not imply that we get to have nice things by Nate Soares

Play Episode Listen Later Oct 18, 2022 42:13

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Decision theory does not imply that we get to have nice things, published by Nate Soares on October 18, 2022 on The AI Alignment Forum. (Note: I wrote this with editing help from Rob and Eliezer. Eliezer's responsible for a few of the paragraphs.) A common confusion I see in the tiny fragment of the world that knows about logical decision theory (FDT/UDT/etc.), is that people think LDT agents are genial and friendly for each other. One recent example is Will Eden's tweet about how maybe a molecular paperclip/squiggle maximizer would leave humanity a few stars/galaxies/whatever on game-theoretic grounds. (And that's just one example; I hear this suggestion bandied around pretty often.) I'm pretty confident that this view is wrong (alas), and based on a misunderstanding of LDT. I shall now attempt to clear up that confusion. To begin, a parable: the entity Omicron (Omega's little sister) fills box A with $1M and box B with $1k, and puts them both in front of an LDT agent saying "You may choose to take either one or both, and know that I have already chosen whether to fill the first box". The LDT agent takes both. "What?" cries the CDT agent. "I thought LDT agents one-box!" LDT agents don't cooperate because they like cooperating. They don't one-box because the name of the action starts with an 'o'. They maximize utility, using counterfactuals that assert that the world they are already in (and the observations they have already seen) can (in the right circumstances) depend (in a relevant way) on what they are later going to do. A paperclipper cooperates with other LDT agents on a one-shot prisoner's dilemma because they get more paperclips that way. Not because it has a primitive property of cooperativeness-with-similar-beings. It needs to get the more paperclips. If a bunch of monkeys want to build a paperclipper and have it give them nice things, the paperclipper needs to somehow expect to wind up with more paperclips than it otherwise would have gotten, as a result of trading with them. If the monkeys instead create a paperclipper haplessly, then the paperclipper does not look upon them with the spirit of cooperation and toss them a few nice things anyway, on account of how we're all good LDT-using friends here. It turns them into paperclips. Because you get more paperclips that way. That's the short version. Now, I'll give the longer version. A few more words about how LDT works To set up a Newcomb's problem, it's important that the predictor does not fill the box if they predict that the agent would two-box. It's not important that they be especially good at this — you should one-box if they're more than 50.05% accurate, if we use the standard payouts ($1M and $1k as the two prizes) and your utility is linear in money — but it is important that their action is at least minimally sensitive to your future behavior. If the predictor's actions don't have this counterfactual dependency on your behavior, then take both boxes. Similarly, if an LDT agent is playing a one-shot prisoner's dilemma against a rock with the word “cooperate” written on it, it defects. At least, it defects if that's all there is to the world. It's technically possible for an LDT agent to think that the real world is made 10% of cooperate-rocks and 90% opponents who cooperate in a one-shot PD iff their opponent cooperates with them and would cooperate with cooperate-rock, in which case LDT agents cooperate against cooperate-rock. From which we learn the valuable lesson that the behavior of an LDT agent depends on the distribution of scenarios it expects to face, which means there's a subtle difference between "imagine you're playing a one-shot PD against a cooperate-rock [and that's the entire universe]" and "imagine you're playing a one-shot PD against a cooperate-rock [in a universe where you fa...

decision speech ea pd soares eliezer nice things cdt newcomb imply rationalist ldt decision theory

LW - Decision theory does not imply that we get to have nice things by So8res

Play Episode Listen Later Oct 18, 2022 42:07

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Decision theory does not imply that we get to have nice things, published by So8res on October 18, 2022 on LessWrong. (Note: I wrote this with editing help from Rob and Eliezer. Eliezer's responsible for a few of the paragraphs.) A common confusion I see in the tiny fragment of the world that knows about logical decision theory (FDT/UDT/etc.), is that people think LDT agents are genial and friendly for each other. One recent example is Will Eden's tweet about how maybe a molecular paperclip/squiggle maximizer would leave humanity a few stars/galaxies/whatever on game-theoretic grounds. (And that's just one example; I hear this suggestion bandied around pretty often.) I'm pretty confident that this view is wrong (alas), and based on a misunderstanding of LDT. I shall now attempt to clear up that confusion. To begin, a parable: the entity Omicron (Omega's little sister) fills box A with $1M and box B with $1k, and puts them both in front of an LDT agent saying "You may choose to take either one or both, and know that I have already chosen whether to fill the first box". The LDT agent takes both. "What?" cries the CDT agent. "I thought LDT agents one-box!" LDT agents don't cooperate because they like cooperating. They don't one-box because the name of the action starts with an 'o'. They maximize utility, using counterfactuals that assert that the world they are already in (and the observations they have already seen) can (in the right circumstances) depend (in a relevant way) on what they are later going to do. A paperclipper cooperates with other LDT agents on a one-shot prisoner's dilemma because they get more paperclips that way. Not because it has a primitive property of cooperativeness-with-similar-beings. It needs to get the more paperclips. If a bunch of monkeys want to build a paperclipper and have it give them nice things, the paperclipper needs to somehow expect to wind up with more paperclips than it otherwise would have gotten, as a result of trading with them. If the monkeys instead create a paperclipper haplessly, then the paperclipper does not look upon them with the spirit of cooperation and toss them a few nice things anyway, on account of how we're all good LDT-using friends here. It turns them into paperclips. Because you get more paperclips that way. That's the short version. Now, I'll give the longer version. A few more words about how LDT works To set up a Newcomb's problem, it's important that the predictor does not fill the box if they predict that the agent would two-box. It's not important that they be especially good at this — you should one-box if they're more than 50.05% accurate, if we use the standard payouts ($1M and $1k as the two prizes) and your utility is linear in money — but it is important that their action is at least minimally sensitive to your future behavior. If the predictor's actions don't have this counterfactual dependency on your behavior, then take both boxes. Similarly, if an LDT agent is playing a one-shot prisoner's dilemma against a rock with the word “cooperate” written on it, it defects. At least, it defects if that's all there is to the world. It's technically possible for an LDT agent to think that the real world is made 10% of cooperate-rocks and 90% opponents who cooperate in a one-shot PD iff their opponent cooperates with them and would cooperate with cooperate-rock, in which case LDT agents cooperate against cooperate-rock. From which we learn the valuable lesson that the behavior of an LDT agent depends on the distribution of scenarios it expects to face, which means there's a subtle difference between "imagine you're playing a one-shot PD against a cooperate-rock [and that's the entire universe]" and "imagine you're playing a one-shot PD against a cooperate-rock [in a universe where you face a random oppone...

decision speech ea pd eliezer nice things cdt newcomb imply rationalist ldt lesswrong decision theory

To P, or Not to P, That is the Question | Stats + Stories Episode 194 (REPOST)

Stats + Stories

Play Episode Listen Later Aug 18, 2022 35:09

For years now, the utility of the P-value in scientific and statistical research has been under scrutiny – the debate shaped by concerns about the seeming over-reliance on p-values to decide what's worth publishing or what's worth pursuing. In 2016 the American Statistical Association released a statement on P-values, meant to remind readers that, “The P-values was never intended to be a substitute for scientific reasoning.” The statement also laid out six principles for how to approach P-values thoughtfully. The impact of that statement is the focus of this episode of Stats and Stories with guest Robert Matthews. Robert Matthews is a visiting professor in the Department of Mathematics, Aston University in Birmingham, UK. Since the late 1990s, as a science writer, he has been reporting on the role of NHST in undermining the reliability of research for several publications including BBC Focus, and working as a consultant on both scientific and media issues for clients in the UK and abroad. His latest book, Chancing It: The Laws of Chance and How They Can Work for You is available now. His research interests include the development of Bayesian methods to assess the credibility of new research findings – especially “out of the blue” claims; A 20-year study of why research findings fade over time and its connection to what's now called “The Replication Crisis”; Investigations of the maths and science behind coincidences and “urban myths” like Murphy's Law: “If something can go wrong, it will”; Applications of Decision Theory to cast light on the reliability (or otherwise) of earthquake predictions and weather forecasts; The first-ever derivation and experimental verification of a prediction from string theory. New episodes of Stats+Stories is returning next week.

stories uk birmingham stats applications mathematics investigations repost bayesian aston university replication crisis american statistical association decision theory robert matthews

LW - Immanuel Kant and the Decision Theory App Store by Daniel Kokotajlo

Play Episode Listen Later Jul 10, 2022 10:41

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Immanuel Kant and the Decision Theory App Store, published by Daniel Kokotajlo on July 10, 2022 on LessWrong. [Epistemic status: About as silly as it sounds.] Prepare to be astounded by this rationalist reconstruction of Kant, drawn out of an unbelievably tiny parcel of Kant literature! Kant argues that all rational agents will: “Act only according to that maxim whereby you can at the same time will that it should become a universal law.” (421) “Act in such a way that you treat humanity, whether in your own person or in the person of another, always at the same time as an end and never simply as a means.” (429) Kant clarifies that treating someone as an end means striving to further their ends, i.e. goals/values. (430) Kant clarifies that strictly speaking it's not just humans that should be treated this way, but all rational beings. He specifically says that this does not extend to non-rational beings. (428) “Act in accordance with the maxims of a member legislating universal laws for a merely possible kingdom of ends.” (439) Not only are all of these claims allegedly derivable from the concept of instrumental rationality, they are supposedly equivalent! Bold claims, lol. What is he smoking? Well, listen up. Taboo “morality.” We are interested in functions that map [epistemic state, preferences, set of available actions] to [action]. Suppose there is an "optimal" function. Call this "instrumental rationality," a.k.a. “Systematized Winning.” Kant asks: Obviously what the optimal function tells you to do depends heavily on your goals and credences; the best way to systematically win depends on what the victory conditions are. Is there anything interesting we can say about what the optimal function recommends that isn't like this? Any non-trivial things that it tells everyone to do regardless of what their goals are? Kant answers: Yes! Consider the twin Prisoner's Dilemma--a version of the PD in which it is common knowledge that both players implement the same algorithm and thus will make the same choice. Suppose (for contradiction) that the optimal function defects. We can now construct a new function, Optimal+, that seems superior to the optimal function: IF in twin PD against someone who you know runs Optimal+: Cooperate ELSE: Do whatever the optimal function will do. Optimal+ is superior to the optimal function because it is exactly the same except that it gets better results in the twin PD (because the opponent will cooperate too, because they are running the same algorithm as you). Contradiction! Looks like our "optimal function" wasn't optimal after all. Therefore the real optimal function must cooperate in the twin PD. Generalizing this reasoning, Kant says, the optimal function will choose as if it is choosing for all instances of the optimal function in similar situations. Thus we can conclude the following interesting fact: Regardless of what your goals are, the optimal function will tell you to avoid doing things that you wouldn't want other rational agents in similar situations to do. (rational agents := agents obeying the optimal function.) To understand this, and see how it generalizes still further, I hereby introduce the following analogy: The Decision Theory App Store Imagine an ideal competitive market for advice-giving AI assistants. Tech companies code them up and then you download them for free from the app store. There is AlphaBot, MetaBot, OpenBot, DeepBot. When installed, the apps give advice. Specifically they scan your brain to extract your credences and values/utility function, and then they tell you what to do. You can follow the advice or not. Sometimes users end up in Twin Prisoner's Dilemmas. That is, situations where they are in some sort of prisoner's dilemma with someone else where there is common knowledge that they both are likely t...

LW - Immanuel Kant and the Decision Theory App Store by Daniel Kokotajlo

Play Episode Listen Later Jul 10, 2022 10:41

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Immanuel Kant and the Decision Theory App Store, published by Daniel Kokotajlo on July 10, 2022 on LessWrong. [Epistemic status: About as silly as it sounds.] Prepare to be astounded by this rationalist reconstruction of Kant, drawn out of an unbelievably tiny parcel of Kant literature! Kant argues that all rational agents will: “Act only according to that maxim whereby you can at the same time will that it should become a universal law.” (421) “Act in such a way that you treat humanity, whether in your own person or in the person of another, always at the same time as an end and never simply as a means.” (429) Kant clarifies that treating someone as an end means striving to further their ends, i.e. goals/values. (430) Kant clarifies that strictly speaking it's not just humans that should be treated this way, but all rational beings. He specifically says that this does not extend to non-rational beings. (428) “Act in accordance with the maxims of a member legislating universal laws for a merely possible kingdom of ends.” (439) Not only are all of these claims allegedly derivable from the concept of instrumental rationality, they are supposedly equivalent! Bold claims, lol. What is he smoking? Well, listen up. Taboo “morality.” We are interested in functions that map [epistemic state, preferences, set of available actions] to [action]. Suppose there is an "optimal" function. Call this "instrumental rationality," a.k.a. “Systematized Winning.” Kant asks: Obviously what the optimal function tells you to do depends heavily on your goals and credences; the best way to systematically win depends on what the victory conditions are. Is there anything interesting we can say about what the optimal function recommends that isn't like this? Any non-trivial things that it tells everyone to do regardless of what their goals are? Kant answers: Yes! Consider the twin Prisoner's Dilemma--a version of the PD in which it is common knowledge that both players implement the same algorithm and thus will make the same choice. Suppose (for contradiction) that the optimal function defects. We can now construct a new function, Optimal+, that seems superior to the optimal function: IF in twin PD against someone who you know runs Optimal+: Cooperate ELSE: Do whatever the optimal function will do. Optimal+ is superior to the optimal function because it is exactly the same except that it gets better results in the twin PD (because the opponent will cooperate too, because they are running the same algorithm as you). Contradiction! Looks like our "optimal function" wasn't optimal after all. Therefore the real optimal function must cooperate in the twin PD. Generalizing this reasoning, Kant says, the optimal function will choose as if it is choosing for all instances of the optimal function in similar situations. Thus we can conclude the following interesting fact: Regardless of what your goals are, the optimal function will tell you to avoid doing things that you wouldn't want other rational agents in similar situations to do. (rational agents := agents obeying the optimal function.) To understand this, and see how it generalizes still further, I hereby introduce the following analogy: The Decision Theory App Store Imagine an ideal competitive market for advice-giving AI assistants. Tech companies code them up and then you download them for free from the app store. There is AlphaBot, MetaBot, OpenBot, DeepBot. When installed, the apps give advice. Specifically they scan your brain to extract your credences and values/utility function, and then they tell you what to do. You can follow the advice or not. Sometimes users end up in Twin Prisoner's Dilemmas. That is, situations where they are in some sort of prisoner's dilemma with someone else where there is common knowledge that they both are likely t...

LW - Decision theory and dynamic inconsistency by paulfchristiano

Play Episode Listen Later Jul 4, 2022 14:46

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Decision theory and dynamic inconsistency, published by paulfchristiano on July 3, 2022 on LessWrong. Here is my current take on decision theory: When making a decision after observing X, we should condition (or causally intervene) on statements like “My decision algorithm outputs Y after observing X.” Updating seems like a description of something you do when making good decisions in this way, not part of defining what a good decision is. (More.) Causal reasoning likewise seems like a description of something you do when making good decisions. Or equivalently: we should use a notion of causality that captures the relationships relevant to decision-making rather than intuitions about physical causality. (More.) “How much do I care about different copies of myself?” is an arbitrary question about my preferences. If my preferences change over time, it naturally gives rise to dynamic inconsistency unrelated to decision theory. (Of course an agent free to modify itself at time T would benefit by implementing some efficient compromise amongst all copies forked off after time T.) In this post I'll discuss the last bullet in more detail since I think it's a bit unusual, it's not something I've written about before, and it's one fo the main ways my view of decision theory has changed in the last few years. (Note: I think this topic is interesting, and could end up being relevant to the world in some weird-yet-possible situations, but I view it as unrelated to my day job on aligning AI with human interests.) The transparent Newcomb problem In the transparent version of Newcomb's problem, you are faced with two transparent boxes (one small and one big). The small box always contains $1,000. The big box contains either $10,000 or $0. You may choose to take the contents of one or both boxes. There is a very accurate predictor, who has placed $10,000 in the big box if and only if they predict that you wouldn't take the small box regardless of what you see in the big one. Intuitively, once you see the contents of the big box, you really have no reason not to take the small box. For example, if you see $0 in the big box, you know for a fact that you are either getting $0 or $1,000. So why not just take the small box and walk away with $1,000? EDT and CDT agree about this one. I think it's genuinely non-obvious what you should do in this case (if the predictor is accurate enough). But I think this is because of ambiguity about what you want, not how you should make decision. More generally, I think that the apparent differences between EDT and UDT are better explained as differences in preferences. In this post I'll explain that view, using transparent Newcomb as an illustration. A simple inconsistent creature Consider a simple creature which rationally pursues its goals on any given day—but whose goals change completely each midnight. Perhaps on Monday the creature is trying to create as much art and beauty as possible; on Tuesday it is trying to create joy and happiness; on Wednesday it might want something different still. On any given day we can think of the creature as an agent. The creature on Tuesday is not being irrational when it decides to pursue joy and happiness instead of art and beauty. It has no special reason to try to “wind back the clock” and pursue the same projects it would have pursued on monday. Of course on Monday the creature would prefer to arrest this predictable value drift—it knows that on Tuesday it will be replaced with a new agent, one that will stop contributing to the project of art and beauty. The creature on Monday ought to make plans accordingly, and if they had the ability to change this feature of themselves they would likely do so. It's a matter of semantics whether we call this creature a single agent or a sequence of agents (one for each day)...

ai decision speech dynamic ea inconsistencies edt intuitively cdt causal newcomb rationalist udt lesswrong decision theory

LW - Decision theory and dynamic inconsistency by paulfchristiano

The Junkyard Love Podcast

Play Episode Listen Later Jul 4, 2022 14:46

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Decision theory and dynamic inconsistency, published by paulfchristiano on July 3, 2022 on LessWrong. Here is my current take on decision theory: When making a decision after observing X, we should condition (or causally intervene) on statements like “My decision algorithm outputs Y after observing X.” Updating seems like a description of something you do when making good decisions in this way, not part of defining what a good decision is. (More.) Causal reasoning likewise seems like a description of something you do when making good decisions. Or equivalently: we should use a notion of causality that captures the relationships relevant to decision-making rather than intuitions about physical causality. (More.) “How much do I care about different copies of myself?” is an arbitrary question about my preferences. If my preferences change over time, it naturally gives rise to dynamic inconsistency unrelated to decision theory. (Of course an agent free to modify itself at time T would benefit by implementing some efficient compromise amongst all copies forked off after time T.) In this post I'll discuss the last bullet in more detail since I think it's a bit unusual, it's not something I've written about before, and it's one fo the main ways my view of decision theory has changed in the last few years. (Note: I think this topic is interesting, and could end up being relevant to the world in some weird-yet-possible situations, but I view it as unrelated to my day job on aligning AI with human interests.) The transparent Newcomb problem In the transparent version of Newcomb's problem, you are faced with two transparent boxes (one small and one big). The small box always contains $1,000. The big box contains either $10,000 or $0. You may choose to take the contents of one or both boxes. There is a very accurate predictor, who has placed $10,000 in the big box if and only if they predict that you wouldn't take the small box regardless of what you see in the big one. Intuitively, once you see the contents of the big box, you really have no reason not to take the small box. For example, if you see $0 in the big box, you know for a fact that you are either getting $0 or $1,000. So why not just take the small box and walk away with $1,000? EDT and CDT agree about this one. I think it's genuinely non-obvious what you should do in this case (if the predictor is accurate enough). But I think this is because of ambiguity about what you want, not how you should make decision. More generally, I think that the apparent differences between EDT and UDT are better explained as differences in preferences. In this post I'll explain that view, using transparent Newcomb as an illustration. A simple inconsistent creature Consider a simple creature which rationally pursues its goals on any given day—but whose goals change completely each midnight. Perhaps on Monday the creature is trying to create as much art and beauty as possible; on Tuesday it is trying to create joy and happiness; on Wednesday it might want something different still. On any given day we can think of the creature as an agent. The creature on Tuesday is not being irrational when it decides to pursue joy and happiness instead of art and beauty. It has no special reason to try to “wind back the clock” and pursue the same projects it would have pursued on monday. Of course on Monday the creature would prefer to arrest this predictable value drift—it knows that on Tuesday it will be replaced with a new agent, one that will stop contributing to the project of art and beauty. The creature on Monday ought to make plans accordingly, and if they had the ability to change this feature of themselves they would likely do so. It's a matter of semantics whether we call this creature a single agent or a sequence of agents (one for each day)...

ai decision speech dynamic ea inconsistencies edt intuitively cdt causal newcomb rationalist udt lesswrong decision theory

084 with Science Researcher, Writer, and Biosemiotician Jeremy Sherman - The Trying Self

Play Episode Listen Later Feb 4, 2022 80:53

Jeremy has a PhD in Decision Theory, a Masters in Public Policy, and is an author of over 1,000 articles for Psychology Today. He's friends with famous intellectual thinkers, has been a close research collaborator with Harvard/Berkeley neuroscientist Terrence Deacon for over 25 years and is a one-time elected elder on the world's largest hippy commune. Jeremy is the author of these three books: 'Neither Ghost Nor Machine - The Emergence and Nature of Selves' 'What's Up With Assholes? - How To Spot And Stop Them Without Becoming One' 'Negotiate With Yourself And Win! - Doubt Management Skills For People Who Can Hear Themselves Think' In this stimulating chat, we touch on science, evolution, the birth of language, The Stoned Aped Theory, 'Trying', Free-will, Natural Philosophy, the third scientific revolution, love, panpsychism, virtual virtue, the second self, social media, technology and much, much more. Access all things Jeremy here - https://jeremysherman.com/ Please consider giving a positive review, thumbs up, and a subscribe if you enjoyed the episode.

science phd nature masters writer researchers public policy sherman psychology today natural philosophy jeremy sherman terrence deacon decision theory

How to make an offer the other side cannot refuse | with Lionel Page

Game Changer - the game theory podcast

Play Episode Listen Later Jan 4, 2022 32:16

In this episode we discuss with Lionel Page, professor of Economics at UTS, his recent research on starting offers in bargaining. We start with talking about the classical ultimatum game and how studies generated a seemingly contradiction to classical game theory. Lionel then explains how he extended the ultimatum game to understand how the value of the initial offer in a bargaining situation affects the outcome of the negotiation. Lionel Page is professor of Economics at University of Technology Sydney. In his research he focuses on Econometrics, Decision Theory, Sociology and Behavioral Economics. He studies the application to practical problems from areas of financial and prediction markets, education and inequality.

university economics offer sociology refuse game theory bargaining behavioral economics uts technology sydney econometrics decision theory

LW - Decision theory: Why Pearl helps reduce “could” and “would”, but still leaves us with at least three alternatives by AnnaSalamon fromDecision Theory: Newcomb's Problem

Play Episode Listen Later Dec 25, 2021 8:55

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is Decision Theory: Newcomb's Problem, Part 4: Decision theory: Why Pearl helps reduce “could” and “would”, but still leaves us with at least three alternatives, published by AnnaSalamon. (This is the third post in a planned sequence.) My last post left us with the questions: Just what are humans, and other common CSAs, calculating when we imagine what “would” happen “if” we took actions we won't take? Is there more than one natural way to calculate these counterfactual “would”s? If so, what are the alternatives, and which alternative works best? Today, I'll take an initial swing at these questions. I'll review Judea Pearl's causal Bayes nets; show how Bayes nets offer a general methodology for computing counterfactual “would”s; and note three plausible alternatives for how to use Pearl's Bayes nets to set up a CSA. One of these alternatives will be the “timeless” counterfactuals of Eliezer's Timeless Decision Theory. The problem of counterfactuals is the problem what we do and should mean when we we discuss what “would” have happened, “if” something impossible had happened. In its general form, this problem has proved to be quite gnarly. It has been bothering philosophers of science for at least 57 years, since the publication of Nelson Goodman's book “Fact, Fiction, and Forecast” in 1952: Let us confine ourselves for the moment to [counterfactual conditionals] in which the antecedent and consequent are inalterably false--as, for example, when I say of a piece of butter that was eaten yesterday, and that had never been heated, `If that piece of butter had been heated to 150°F, it would have melted.' Considered as truth-functional compounds, all counterfactuals are of course true, since their antecedents are false. Hence `If that piece of butter had been heated to 150°F, it would not have melted.' would also hold. Obviously something different is intended, and the problem is to define the circumstances under which a given counterfactual holds while the opposing conditional with the contradictory consequent fails to hold. Recall that we seem to need counterfactuals in order to build agents that do useful decision theory -- we need to build agents that can think about the consequences of each of their “possible actions”, and can choose the action with best expected-consequences. So we need to know how to compute those counterfactuals. As Goodman puts it, “[t]he analysis of counterfactual conditionals is no fussy little grammatical exercise.” Judea Pearl's Bayes nets offer a method for computing counterfactuals. As noted, it is hard to reduce human counterfactuals in general: it is hard to build an algorithm that explains what (humans will say) really “would” have happened, “if” an impossible event had occurred. But it is easier to construct specific formalisms within which counterfactuals have well-specified meanings. Judea Pearl's causal Bayes nets offer perhaps the best such formalism. Pearl's idea is to model the world as based on some set of causal variables, which may be observed or unobserved. In Pearl's model, each variable is determined by a conditional probability distribution on the state of its parents (or by a simple probability distribution, if it has no parents). For example, in the following Bayes net, the beach's probability of being “Sunny” depends only on the “Season”, and the probability that there is each particular “Number of beach-goers” depends only on the “Day of the week” and on the “Sunniness”. Since the “Season” and the “Day of the week” have no parents, they simply have fixed probability distributions. Once we have a Bayes net set up to model a given domain, computing counterfactuals is easy. We just: Take the usual conditional and unconditional probability distributions, that come with the Bayes net; Do “surgery” on the Bayes net to plug in the...

decision helps speech fiction reduce ea alternatives forecast recall csa eliezer newcomb bayes csas rationalist judea pearl decision theory

LW - Confusion about Newcomb is confusion about counterfactuals by AnnaSalamon from Decision Theory: Newcomb's Problem

speech confusion omega expected ea disagreement newcomb counterfactuals rationalist decision theory

Play Episode Listen Later Dec 25, 2021 3:28

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is Decision Theory: Newcomb's Problem, Part 2: Confusion about Newcomb is confusion about counterfactuals, published by AnnaSalamon. (This is the first, and most newcomer-accessible, post in a planned sequence.) Newcomb's Problem: Joe walks out onto the square. As he walks, a majestic being flies by Joe's head with a box labeled "brain scanner", drops two boxes on the ground, and departs the scene. A passerby, known to be trustworthy, comes over and explains... If Joe aims to get the most money, should Joe take one box or two? What are we asking when we ask what Joe "should" do? It is common to cash out "should" claims as counterfactuals: "If Joe were to one-box, he would make more money". This method of translating "should" questions does seem to capture something of what we mean: we do seem to be asking how much money Joe can expect to make "if he one-boxes" vs. "if he two-boxes". The trouble with this translation, however, is that it is not clear what world "if Joe were to one-box" should refer to -- and, therefore, it is not clear how much money we should say Joe would make, "if he were to one-box". After all, Joe is a deterministic physical system; his current state (together with the state of his future self's past light-cone) fully determines what Joe's future action will be. There is no Physically Irreducible Moment of Choice, where this same Joe, with his own exact actual past, "can" go one way or the other. To restate the situation more clearly: let us suppose that this Joe, standing here, is poised to two-box. In order to determine how much money Joe "would have made if he had one-boxed", let us say that we imagine reaching in, with a magical sort of world-surgery, and altering the world so that Joe one-boxes instead. We then watch to see how much money Joe receives, in this surgically altered world. The question before us, then, is what sort of magical world-surgery to execute, before we watch to see how much money Joe "would have made if he had one-boxed". And the difficulty in Newcomb's problem is that there is not one but two obvious world-surgeries to consider. First, we might surgically reach in, after Omega's departure, and alter Joe's box-taking only -- leaving Omega's prediction about Joe untouched. Under this sort of world-surgery, Joe will do better by two-boxing: Expected value ( Joe's earnings if he two-boxes | some unchanged probability distribution on Omega's prediction ) > Expected value ( Joe's earnings if he one-boxes | the same unchanged probability distribution on Omega's prediction ). Second, we might surgically reach in, after Omega's departure, and simultaneously alter both Joe's box-taking and Omega's prediction concerning Joe's box-taking. (Equivalently, we might reach in before Omega's departure, and surgically alter the insides of Joe brain -- and, thereby, alter both Joe's behavior and Omega's prediction of Joe's behavior.) Under this sort of world-surgery, Joe will do better by one-boxing: Expected value ( Joe's earnings if he one-boxes | Omega predicts Joe accurately) > Expected value ( Joe's earnings if he two-boxes | Omega predicts Joe accurately). The point: Newcomb's problem -- the problem of what Joe "should" do, to earn most money -- is the problem which type of world-surgery best cashes out the question "Should Joe take one box or two?". Disagreement about Newcomb's problem is disagreement about what sort of world-surgery we should consider, when we try to figure out what action Joe should take. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

LW - Decision theory: An outline of some upcoming posts by AnnaSalamon from Decision Theory: Newcomb's Problem

Play Episode Listen Later Dec 25, 2021 9:42

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is Decision Theory: Newcomb's Problem, Part 1: Decision theory: An outline of some upcoming posts, published by AnnaSalamon. Last August or so, Eliezer asked Steve Rayhawk and myself to attempt to solve Newcomb's problem together. This project served a couple of purposes: a. Get an indication as to our FAI research abilities. b. Train our reduction-muscles. c. Check whether Eliezer's (unseen by us) timeless decision theory is a point that outside folks tend to arrive at independently (at least if starting from the rather substantial clues on OB/LW), and whether anything interestingly new came out of an independent attempt. Steve and I (and, briefly but helpfully, Liron Shapira) took our swing at Newcomb. We wrote a great mass of notes that have been sitting on our hard drives, but hadn't stitched them together into a single document. I'd like to attempt a Less Wrong sequence on that subject now. Most of this content is stuff that Eliezer, Nesov, and/or Dai developed independently and have been referring to in their posts, but I'll try to present it more fully and clearly. I learned a bit of this from Eliezer/Nesov/Dai's recent posts. Here's the outline, to be followed up with slower, clearer blog posts if all goes well: 0. Prelude: “Should” depends on counterfactuals. Newcomb's problem -- the problem of what Joe "should" do, to earn most money -- is the problem of which type of counterfactuals best cash out the question "Should Joe take one box or two?". Disagreement about Newcomb's problem is disagreement about what sort of counterfactuals we should consider, when we try to figure out what action Joe should take. 1. My goal in this sequence is to reduce “should” as thoroughly as I can. More specifically, I'll make an (incomplete, but still useful) attempt to: Make it even more clear that our naive conceptions of “could” and “should” are conceptual inventions, and are not Physically Irreducible Existent Things. (Written here.) Consider why one might design an agent that uses concepts like “could” and “should” (hereafter a “Could/Should Agent”, or “CSA”), rather than designing an agent that acts in some other way. Consider what specific concepts of “could” and “should” are what specific kinds of useful. (This is meant as a more thorough investigation of the issues treated by Eliezer in “Possibility and Couldness”.) Consider why evolution ended up creating us as approximate CSAs. Consider what kinds of CSAs are likely to be how common across the multiverse. 2. A non-vicious regress. Suppose we're designing Joe, and we want to maximize his expected winnings. What notion of “should” should we design Joe to use? There's a regress here, in that creator-agents with different starting decision theories will design agents that have different starting decision theories. But it is a non-vicious regress. We can gain understanding by making this regress explicit, and asking under what circumstances agents with decision theory X will design future agents with decision theory Y, for different values of X and Y. 3a. When will a CDT-er build agents that use “could” and “should”? Suppose again that you're designing Joe, and that Joe will go out in a world and win utilons on your behalf. What kind of Joe-design will maximize your expected utilons? If we assume nothing about Joe's world, we might find that your best option was to design Joe to act as a bundle of wires which happens to have advantageous physical effects, and which doesn't act like an agent at all. But suppose Joe's world has the following handy property: suppose Joe's actions have effects, and Joe's “policy”, or the actions he “would have taken” in response to alternative inputs also have effects, but the details of Joe's internal wiring doesn't otherwise matter. (I'll call this the "policy-equivalence assumption"). Since Jo...

train decision speech ea possibility disagreement outline csa dai fai eliezer cdt newcomb csas rationalist decision theory

LW - Decision theory: Why we need to reduce “could”, “would”, “should” by AnnaSalamon from Decision Theory: Newcomb's Problem

Play Episode Listen Later Dec 25, 2021 6:48

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is Decision Theory: Newcomb's Problem, Part 3: Decision theory: Why we need to reduce “could”, “would”, “should”, published by AnnaSalamon. (This is the second post in a planned sequence.) Let's say you're building an artificial intelligence named Bob. You'd like Bob to sally forth and win many utilons on your behalf. How should you build him? More specifically, should you build Bob to have a world-model in which there are many different actions he “could” take, each of which “would” give him particular expected results? (Note that e.g. evolution, rivers, and thermostats do not have explicit “could”/“would”/“should” models in this sense -- and while evolution, rivers, and thermostats are all varying degrees of stupid, they all still accomplish specific sorts of world-changes. One might imagine more powerful agents that also simply take useful actions, without claimed “could”s and “woulds”.) My aim in this post is simply to draw attention to “could”, “would”, and “should”, as concepts folk intuition fails to understand, but that seem nevertheless to do something important for real-world agents. If we want to build Bob, we may well need to figure out what the concepts “could” and “would” can do for him. Introducing Could/Would/Should agents: Let a Could/Would/Should Algorithm, or CSA for short, be any algorithm that chooses its actions by considering a list of alternatives, estimating the payoff it “would” get “if” it took each given action, and choosing the action from which it expects highest payoff. That is: let us say that to specify a CSA, we need to specify: A list of alternatives a_1, a_2, ..., a_n that are primitively labeled as actions it “could” take; For each alternative a_1 through a_n, an expected payoff U(a_i) that is labeled as what “would” happen if the CSA takes that alternative. To be a CSA, the algorithm must then search through the payoffs for each action, and must then trigger the agent to actually take the action a_i for which its labeled U(a_i) is maximal. Note that we can, by this definition of “CSA”, create a CSA around any made-up list of “alternative actions” and of corresponding “expected payoffs”. The puzzle is that CSAs are common enough to suggest that they're useful -- but it isn't clear why CSAs are useful, or quite what kinds of CSAs are what kind of useful. To spell out the puzzle: Puzzle piece 1: CSAs are common. Humans, some (though far from all) other animals, and many human-created decision-making programs (game-playing programs, scheduling software, etc.), have CSA-like structure. That is, we consider “alternatives” and act out the alternative from which we “expect” the highest payoff (at least to a first approximation). The ubiquity of approximate CSAs suggests that CSAs are in some sense useful. Puzzle piece 2: The naïve realist model of CSAs' nature and usefulness doesn't work as an explanation. That is: many people find CSAs' usefulness unsurprising, because they imagine a Physically Irreducible Choice Point, where an agent faces Real Options; by thinking hard, and choosing the Option that looks best, naïve realists figure that you can get the best-looking option (instead of one of those other options, that you Really Could have gotten). But CSAs, like other agents, are deterministic physical systems. Each CSA executes a single sequence of physical movements, some of which we consider “examining alternatives”, and some of which we consider “taking an action”. It isn't clear why or in what sense such systems do better than deterministic systems built in some other way. Puzzle piece 3: Real CSAs are presumably not built from arbitrarily labeled “coulds” and “woulds” -- presumably, the “woulds” that humans and others use, when considering e.g. which chess move to make, have useful properties. But it isn't clear what those properties...

decision humans speech reduce option ea puzzle csa newcomb csas rationalist real options decision theory

LW - Confusion about Newcomb is confusion about counterfactuals by AnnaSalamon from Decision Theory: Newcomb's Problem

speech confusion omega expected ea disagreement newcomb counterfactuals rationalist decision theory

Play Episode Listen Later Dec 25, 2021 3:28

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is Decision Theory: Newcomb's Problem, Part 2: Confusion about Newcomb is confusion about counterfactuals, published by AnnaSalamon. (This is the first, and most newcomer-accessible, post in a planned sequence.) Newcomb's Problem: Joe walks out onto the square. As he walks, a majestic being flies by Joe's head with a box labeled "brain scanner", drops two boxes on the ground, and departs the scene. A passerby, known to be trustworthy, comes over and explains... If Joe aims to get the most money, should Joe take one box or two? What are we asking when we ask what Joe "should" do? It is common to cash out "should" claims as counterfactuals: "If Joe were to one-box, he would make more money". This method of translating "should" questions does seem to capture something of what we mean: we do seem to be asking how much money Joe can expect to make "if he one-boxes" vs. "if he two-boxes". The trouble with this translation, however, is that it is not clear what world "if Joe were to one-box" should refer to -- and, therefore, it is not clear how much money we should say Joe would make, "if he were to one-box". After all, Joe is a deterministic physical system; his current state (together with the state of his future self's past light-cone) fully determines what Joe's future action will be. There is no Physically Irreducible Moment of Choice, where this same Joe, with his own exact actual past, "can" go one way or the other. To restate the situation more clearly: let us suppose that this Joe, standing here, is poised to two-box. In order to determine how much money Joe "would have made if he had one-boxed", let us say that we imagine reaching in, with a magical sort of world-surgery, and altering the world so that Joe one-boxes instead. We then watch to see how much money Joe receives, in this surgically altered world. The question before us, then, is what sort of magical world-surgery to execute, before we watch to see how much money Joe "would have made if he had one-boxed". And the difficulty in Newcomb's problem is that there is not one but two obvious world-surgeries to consider. First, we might surgically reach in, after Omega's departure, and alter Joe's box-taking only -- leaving Omega's prediction about Joe untouched. Under this sort of world-surgery, Joe will do better by two-boxing: Expected value ( Joe's earnings if he two-boxes | some unchanged probability distribution on Omega's prediction ) > Expected value ( Joe's earnings if he one-boxes | the same unchanged probability distribution on Omega's prediction ). Second, we might surgically reach in, after Omega's departure, and simultaneously alter both Joe's box-taking and Omega's prediction concerning Joe's box-taking. (Equivalently, we might reach in before Omega's departure, and surgically alter the insides of Joe brain -- and, thereby, alter both Joe's behavior and Omega's prediction of Joe's behavior.) Under this sort of world-surgery, Joe will do better by one-boxing: Expected value ( Joe's earnings if he one-boxes | Omega predicts Joe accurately) > Expected value ( Joe's earnings if he two-boxes | Omega predicts Joe accurately). The point: Newcomb's problem -- the problem of what Joe "should" do, to earn most money -- is the problem which type of world-surgery best cashes out the question "Should Joe take one box or two?". Disagreement about Newcomb's problem is disagreement about what sort of world-surgery we should consider, when we try to figure out what action Joe should take. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

LW - Decision theory: An outline of some upcoming posts by AnnaSalamon from Decision Theory: Newcomb's Problem

Play Episode Listen Later Dec 25, 2021 9:42

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is Decision Theory: Newcomb's Problem, Part 1: Decision theory: An outline of some upcoming posts, published by AnnaSalamon. Last August or so, Eliezer asked Steve Rayhawk and myself to attempt to solve Newcomb's problem together. This project served a couple of purposes: a. Get an indication as to our FAI research abilities. b. Train our reduction-muscles. c. Check whether Eliezer's (unseen by us) timeless decision theory is a point that outside folks tend to arrive at independently (at least if starting from the rather substantial clues on OB/LW), and whether anything interestingly new came out of an independent attempt. Steve and I (and, briefly but helpfully, Liron Shapira) took our swing at Newcomb. We wrote a great mass of notes that have been sitting on our hard drives, but hadn't stitched them together into a single document. I'd like to attempt a Less Wrong sequence on that subject now. Most of this content is stuff that Eliezer, Nesov, and/or Dai developed independently and have been referring to in their posts, but I'll try to present it more fully and clearly. I learned a bit of this from Eliezer/Nesov/Dai's recent posts. Here's the outline, to be followed up with slower, clearer blog posts if all goes well: 0. Prelude: “Should” depends on counterfactuals. Newcomb's problem -- the problem of what Joe "should" do, to earn most money -- is the problem of which type of counterfactuals best cash out the question "Should Joe take one box or two?". Disagreement about Newcomb's problem is disagreement about what sort of counterfactuals we should consider, when we try to figure out what action Joe should take. 1. My goal in this sequence is to reduce “should” as thoroughly as I can. More specifically, I'll make an (incomplete, but still useful) attempt to: Make it even more clear that our naive conceptions of “could” and “should” are conceptual inventions, and are not Physically Irreducible Existent Things. (Written here.) Consider why one might design an agent that uses concepts like “could” and “should” (hereafter a “Could/Should Agent”, or “CSA”), rather than designing an agent that acts in some other way. Consider what specific concepts of “could” and “should” are what specific kinds of useful. (This is meant as a more thorough investigation of the issues treated by Eliezer in “Possibility and Couldness”.) Consider why evolution ended up creating us as approximate CSAs. Consider what kinds of CSAs are likely to be how common across the multiverse. 2. A non-vicious regress. Suppose we're designing Joe, and we want to maximize his expected winnings. What notion of “should” should we design Joe to use? There's a regress here, in that creator-agents with different starting decision theories will design agents that have different starting decision theories. But it is a non-vicious regress. We can gain understanding by making this regress explicit, and asking under what circumstances agents with decision theory X will design future agents with decision theory Y, for different values of X and Y. 3a. When will a CDT-er build agents that use “could” and “should”? Suppose again that you're designing Joe, and that Joe will go out in a world and win utilons on your behalf. What kind of Joe-design will maximize your expected utilons? If we assume nothing about Joe's world, we might find that your best option was to design Joe to act as a bundle of wires which happens to have advantageous physical effects, and which doesn't act like an agent at all. But suppose Joe's world has the following handy property: suppose Joe's actions have effects, and Joe's “policy”, or the actions he “would have taken” in response to alternative inputs also have effects, but the details of Joe's internal wiring doesn't otherwise matter. (I'll call this the "policy-equivalence assumption"). Since Jo...

train decision speech ea possibility disagreement outline csa dai fai eliezer cdt newcomb csas rationalist decision theory

LW - Decision theory: Why we need to reduce “could”, “would”, “should” by AnnaSalamon from Decision Theory: Newcomb's Problem

Play Episode Listen Later Dec 25, 2021 6:48

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is Decision Theory: Newcomb's Problem, Part 3: Decision theory: Why we need to reduce “could”, “would”, “should”, published by AnnaSalamon. (This is the second post in a planned sequence.) Let's say you're building an artificial intelligence named Bob. You'd like Bob to sally forth and win many utilons on your behalf. How should you build him? More specifically, should you build Bob to have a world-model in which there are many different actions he “could” take, each of which “would” give him particular expected results? (Note that e.g. evolution, rivers, and thermostats do not have explicit “could”/“would”/“should” models in this sense -- and while evolution, rivers, and thermostats are all varying degrees of stupid, they all still accomplish specific sorts of world-changes. One might imagine more powerful agents that also simply take useful actions, without claimed “could”s and “woulds”.) My aim in this post is simply to draw attention to “could”, “would”, and “should”, as concepts folk intuition fails to understand, but that seem nevertheless to do something important for real-world agents. If we want to build Bob, we may well need to figure out what the concepts “could” and “would” can do for him. Introducing Could/Would/Should agents: Let a Could/Would/Should Algorithm, or CSA for short, be any algorithm that chooses its actions by considering a list of alternatives, estimating the payoff it “would” get “if” it took each given action, and choosing the action from which it expects highest payoff. That is: let us say that to specify a CSA, we need to specify: A list of alternatives a_1, a_2, ..., a_n that are primitively labeled as actions it “could” take; For each alternative a_1 through a_n, an expected payoff U(a_i) that is labeled as what “would” happen if the CSA takes that alternative. To be a CSA, the algorithm must then search through the payoffs for each action, and must then trigger the agent to actually take the action a_i for which its labeled U(a_i) is maximal. Note that we can, by this definition of “CSA”, create a CSA around any made-up list of “alternative actions” and of corresponding “expected payoffs”. The puzzle is that CSAs are common enough to suggest that they're useful -- but it isn't clear why CSAs are useful, or quite what kinds of CSAs are what kind of useful. To spell out the puzzle: Puzzle piece 1: CSAs are common. Humans, some (though far from all) other animals, and many human-created decision-making programs (game-playing programs, scheduling software, etc.), have CSA-like structure. That is, we consider “alternatives” and act out the alternative from which we “expect” the highest payoff (at least to a first approximation). The ubiquity of approximate CSAs suggests that CSAs are in some sense useful. Puzzle piece 2: The naïve realist model of CSAs' nature and usefulness doesn't work as an explanation. That is: many people find CSAs' usefulness unsurprising, because they imagine a Physically Irreducible Choice Point, where an agent faces Real Options; by thinking hard, and choosing the Option that looks best, naïve realists figure that you can get the best-looking option (instead of one of those other options, that you Really Could have gotten). But CSAs, like other agents, are deterministic physical systems. Each CSA executes a single sequence of physical movements, some of which we consider “examining alternatives”, and some of which we consider “taking an action”. It isn't clear why or in what sense such systems do better than deterministic systems built in some other way. Puzzle piece 3: Real CSAs are presumably not built from arbitrarily labeled “coulds” and “woulds” -- presumably, the “woulds” that humans and others use, when considering e.g. which chess move to make, have useful properties. But it isn't clear what those properties...

decision humans speech reduce option ea puzzle csa newcomb csas rationalist real options decision theory

LW - Decision theory: Why Pearl helps reduce “could” and “would”, but still leaves us with at least three alternatives by AnnaSalamon fromDecision Theory: Newcomb's Problem

phd masters public policy holes uc berkeley sherman psychology today jeremy sherman decision theory

Play Episode Listen Later Dec 25, 2021 8:55

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is Decision Theory: Newcomb's Problem, Part 4: Decision theory: Why Pearl helps reduce “could” and “would”, but still leaves us with at least three alternatives, published by AnnaSalamon. (This is the third post in a planned sequence.) My last post left us with the questions: Just what are humans, and other common CSAs, calculating when we imagine what “would” happen “if” we took actions we won't take? Is there more than one natural way to calculate these counterfactual “would”s? If so, what are the alternatives, and which alternative works best? Today, I'll take an initial swing at these questions. I'll review Judea Pearl's causal Bayes nets; show how Bayes nets offer a general methodology for computing counterfactual “would”s; and note three plausible alternatives for how to use Pearl's Bayes nets to set up a CSA. One of these alternatives will be the “timeless” counterfactuals of Eliezer's Timeless Decision Theory. The problem of counterfactuals is the problem what we do and should mean when we we discuss what “would” have happened, “if” something impossible had happened. In its general form, this problem has proved to be quite gnarly. It has been bothering philosophers of science for at least 57 years, since the publication of Nelson Goodman's book “Fact, Fiction, and Forecast” in 1952: Let us confine ourselves for the moment to [counterfactual conditionals] in which the antecedent and consequent are inalterably false--as, for example, when I say of a piece of butter that was eaten yesterday, and that had never been heated, `If that piece of butter had been heated to 150°F, it would have melted.' Considered as truth-functional compounds, all counterfactuals are of course true, since their antecedents are false. Hence `If that piece of butter had been heated to 150°F, it would not have melted.' would also hold. Obviously something different is intended, and the problem is to define the circumstances under which a given counterfactual holds while the opposing conditional with the contradictory consequent fails to hold. Recall that we seem to need counterfactuals in order to build agents that do useful decision theory -- we need to build agents that can think about the consequences of each of their “possible actions”, and can choose the action with best expected-consequences. So we need to know how to compute those counterfactuals. As Goodman puts it, “[t]he analysis of counterfactual conditionals is no fussy little grammatical exercise.” Judea Pearl's Bayes nets offer a method for computing counterfactuals. As noted, it is hard to reduce human counterfactuals in general: it is hard to build an algorithm that explains what (humans will say) really “would” have happened, “if” an impossible event had occurred. But it is easier to construct specific formalisms within which counterfactuals have well-specified meanings. Judea Pearl's causal Bayes nets offer perhaps the best such formalism. Pearl's idea is to model the world as based on some set of causal variables, which may be observed or unobserved. In Pearl's model, each variable is determined by a conditional probability distribution on the state of its parents (or by a simple probability distribution, if it has no parents). For example, in the following Bayes net, the beach's probability of being “Sunny” depends only on the “Season”, and the probability that there is each particular “Number of beach-goers” depends only on the “Day of the week” and on the “Sunniness”. Since the “Season” and the “Day of the week” have no parents, they simply have fixed probability distributions. Once we have a Bayes net set up to model a given domain, computing counterfactuals is easy. We just: Take the usual conditional and unconditional probability distributions, that come with the Bayes net; Do “surgery” on the Bayes net to plug in the...

decision helps speech fiction reduce ea alternatives forecast recall csa eliezer newcomb bayes csas rationalist judea pearl decision theory

84 - What's Up with As$holes with Jeremy Sherman

Better Than Fine

Play Episode Listen Later Dec 16, 2021 58:41

Support the show by snapping up your copy of the Better Than Fine Workbook - www.darlene.coach/workbook Jeremy Sherman has a Masters in Public Policy from UC Berkeley, a PhD in Decision Theory and is the author of two books on the self and self management, writes regular articles for Psychology Today - and some of my favorite work is in what he calls psychoproctology: the study of what makes people act like "jerks". In this episode we explore what makes some people act like A-Holes, how to deal with them, and how to meet life's challenges with humor. --- Send in a voice message: https://anchor.fm/betterthanfine/message Support this podcast: https://anchor.fm/betterthanfine/support

A Critique of Functional Decision Theory by wdmacaskill

The Nonlinear Library: Alignment Forum Top Posts

Play Episode Listen Later Dec 5, 2021 34:41

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A Critique of Functional Decision Theory, published by wdmacaskill on the AI Alignment Forum. A Critique of Functional Decision Theory NB: My writing this note was prompted by Carl Shulman, who suggested we could try a low-time-commitment way of attempting to understanding the disagreement between some folks in the rationality community and academic decision theorists (including myself, though I'm not much of a decision theorist). Apologies that it's sloppier than I'd usually aim for in a philosophy paper, and lacking in appropriate references. And, even though the paper is pretty negative about FDT, I want to emphasise that my writing this should be taken as a sign of respect for those involved in developing FDT. I'll also caveat I'm unlikely to have time to engage in the comments; I thought it was better to get this out there all the same rather than delay publication further. Introduction There's a long-running issue where many in the rationality community take functional decision theory (and its variants) very seriously, but the academic decision theory community does not. But there's been little public discussion of FDT from academic decision theorists (one exception is here); this note attempts to partly address this gap. So that there's a clear object of discussion, I'm going to focus on Yudkowsky and Soares' ‘Functional Decision Theory' (which I'll refer to as Y&S), though I also read a revised version of Soares and Levinstein's Cheating Death in Damascus. This note is structured as follows. Section II describes causal decision theory (CDT), evidential decision theory (EDT) and functional decision theory (FDT). Sections III-VI describe problems for FDT: (i) that it sometimes makes bizarre recommendations, recommending an option that is certainly lower-utility than another option; (ii) that it fails to one-box in most instances of Newcomb's problem, even though the correctness of one-boxing is supposed to be one of the guiding motivations for the theory; (iii) that it results in implausible discontinuities, where what is rational to do can depend on arbitrarily small changes to the world; and (iv) that, because there's no real fact of the matter about whether a particular physical process implements a particular algorithm, it's deeply indeterminate what FDT's implications are. In section VII I discuss the idea that FDT ‘does better at getting utility' than EDT or CDT; I argue that Y&S's claims to this effect are unhelpfully vague, and on any more precise way of understanding their claim, aren't plausible. In section VIII I briefly describe a view that captures some of the motivation behind FDT, and in my view is more plausible. I conclude that FDT faces a number of deep problems and little to say in its favour. In what follows, I'm going to assume a reasonable amount of familiarity with the debate around Newcomb's problem. II. CDT, EDT and FDT Informally: CDT, EDT and FDT differ in what non-causal correlations they care about when evaluating a decision. For CDT, what you cause to happen is all that matters; if your action correlates with some good outcome, that's nice to know, but it's not relevant to what you ought to do. For EDT, all correlations matter: you should pick whatever action will result in you believing you will have the highest expected utility. For FDT, only some non-causal correlations matter, namely only those correlations between your action and events elsewhere in time and space that would be different in the (logically impossible) worlds in which the output of the algorithm you're running is different. Other than for those correlations, FDT behaves in the same way as CDT. Formally, where S represents states of nature, A, B etc represent acts, P is a probability function, and U S i A represents the utility the agent gains from the outcome of...

apologies speech ea functional critique damascus soares edt cdt ys newcomb cheating death rationalist fdt section ii yudkowsky decision theory carl shulman

Comment on decision theory by Rob Bensinger

The Nonlinear Library: Alignment Forum Top Posts

Play Episode Listen Later Dec 3, 2021 4:03

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Comment on decision theory, published by Rob Bensinger on the AI Alignment Forum. A comment I made on social media last year about why MIRI cares about making progress on decision theory: We aren't working on decision theory in order to make sure that AGI systems are decision-theoretic, whatever that would involve. We're working on decision theory because there's a cluster of confusing issues here (e.g., counterfactuals, updatelessness, coordination) that represent a lot of holes or anomalies in our current best understanding of what high-quality reasoning is and how it works. As an analogy: it might be possible to build a probabilistic reasoner without having a working understanding of classical probability theory, through sufficient trial and error. (Evolution "built" humans without understanding probability theory.) But you'd fundamentally be flying blind when it comes to designing the system — to a large extent, you couldn't predict in advance which classes of design were likely to be most promising to consider, couldn't look at particular proposed designs and make good advance predictions about safety/capability properties of the corresponding system, couldn't identify and address the root causes of problems that crop up, etc. The idea behind looking at (e.g.) counterfactual reasoning is that counterfactual reasoning is central to what we're talking about when we talk about "AGI," and going into the development process without a decent understanding of what counterfactual reasoning is and how it works means you'll to a significantly greater extent be flying blind when it comes to designing, inspecting, repairing, etc. your system. The goal is to be able to put AGI developers in a position where they can make advance plans and predictions, shoot for narrow design targets, and understand what they're doing well enough to avoid the kinds of kludgey, opaque, non-modular, etc. approaches that aren't really compatible with how secure or robust software is developed. Nate's way of articulating it: The reason why I care about logical uncertainty and decision theory problems is something more like this: The whole AI problem can be thought of as a particular logical uncertainty problem, namely, the problem of taking a certain function f : Q → R and finding an input that makes the output large. To see this, let f be the function that takes the AI agent's next action (encoded in Q) and determines how "good" the universe is if the agent takes that action. The reason we need a principled theory of logical uncertainty is so that we can do function optimization, and the reason we need a principled decision theory is so we can pick the right version of the "if the AI system takes that action..." function. The work you use to get to AGI presumably won't look like probability theory, but it's still the case that you're building a system to do probabilistic reasoning, and understanding what probabilistic reasoning is is likely to be very valuable for doing that without relying on brute force and trial-and-error. Similarly, the work that goes into figuring out how to design a rocket, actually building one, etc. doesn't look very much like the work that goes into figuring out that there's a universal force of gravity that operates by an inverse square law; but you'll have a vastly easier time approaching the rocket-building problem with foresight and an understanding of what you're doing if you have a mental model of gravitation already in hand. In pretty much the same way, developing an understanding of roughly what counterfactuals are and how they work won't get you to AGI, and the work of implementing an AGI design won't look like decision theory, but you want to have in mind an understanding of what "AGI-style reasoning" is (including "what probabilistic reasoning about empirical...

ai evolution speech ea qr agi miri rationalist ben singer decision theory

148 – Decisions, Decisions

The Bayesian Conspiracy

Play Episode Listen Later Oct 20, 2021 123:32

Matt Freeman returns to talk about Decision Theory, and the discusion ranges far afield. Matt is from The Guild of the Rose, which we had a recent episode about. You can see his Decision Theory Course videos here. Scott Alexander … Continue reading →

methods guild skepticism skeptic decisions decisions rationality rationalism scott alexander matt freeman yudkowsky lesswrong decision theory

Case-Based Decision Theory and Maxmin EU with Non-Unique Prior with Itzhak Gilboa

Behavioural Science Uncovered

Play Episode Listen Later Oct 5, 2021 43:30

In this episode, we talk with Itzhak Gilboa, professor of Economics at the HEC Paris and Tel-Aviv University, and holder of the AXA Chair in Decision Sciences. We will talk about the contrast “expected vs. actual success” of two of his papers. “Case-Based Decision Theory” was expected to be very successful, but turned out not to be. The second paper, “Maxmin Expected Utility with Non-Unique Prior” was not expected to be a big deal, but is now listed as the most cited and the most relevant paper in the Journal of Mathematical Economics. Tune in for a discussion centered around the development of the ideas, the publication process, and to learn about axiomatic decision theory.

european union journal unique economics tel aviv university hec paris decision sciences gilboa itzhak decision theory mathematical economics

Success, Luck, & Second Chances

The Random Sample

Play Episode Listen Later Aug 10, 2021 29:20

When things go your way, or don't go your way, how much of that is luck? Do we overvalue or undervalue a process or a person's performance because of how things eventually turn out? Do we need to take into account things out of our control? In this episode, we explore these questions and look into whether we need to do more to give some people a second chance. The Random Sample is a podcast by the Australian Research Council Centre of Excellence for Mathematical and Statistical Frontiers. In this show, we share stories about mathematics, statistics and the people involved. To learn more about ACEMS, visit https://acems.org.au.See omnystudio.com/listener for privacy information.

success research excellence luck statistics second chances mathematics game theory mathematical grant funding decision theory australian research council centre

To P, or Not to P, That is the Question | Stats + Stories Episode 194

Stats + Stories

Play Episode Listen Later Jul 8, 2021 35:09

stories uk birmingham stats applications mathematics investigations bayesian aston university replication crisis american statistical association decision theory robert matthews

Episode 123: Steve Chien talks about AI, Mars rovers, and the possibility of intelligent alien life

STEM-Talk

Play Episode Listen Later May 26, 2021 44:13

Episode 123 Steve Chien talks about AI, Mars rovers, and the possibility of intelligent alien life Today’s interview is with Dr. Steve Chien. Dr. Chien is JPL Fellow, Senior Research Scientist, and Technical Group Supervisor of the Artificial Intelligence Group and in the Mission Planning and Execution Section at the Jet Propulsion Laboratory, California Institute of Technology. In 2018, Steve and Ken were appointed to the National Security Commission on Artificial Intelligence, an independent commission tasked with providing the President and Congress a blueprint for advancing AI and associated technologies to address future national security and defense needs of the United States. The commission recently released a 756-page reportwhich found that the nation is unprepared to compete in a future enabled by AI and that the U.S. could soon be replaced as the world’s AI superpower. The report was two years in the making and offers strategies and recommendations to strengthen and protect the nation’s economy, technology base, and national security. In today’s podcast, we talk to Steve about the report and what he learned over the past two years serving on the commission. In addition to heading up the Artificial Intelligence Group at JPL, Steve also is the lead for deep space robotic exploration for NASA. For the past several years, he has worked on the Perseverance Rover mission, which landed on Mars back in February and used an automated ground-based scheduling system called Copilot that Steve and his JPL colleagues developed. Steve joined JPL more than 30 years ago and last year was named a JPL Fellow, an honor that recognizes people who have made extraordinary technical and institutional contributions to the Jet Propulsion Laboratory over an extended period. He is a graduate of the University of Illinois Urbana-Champaign where he earned a doctorate in computer science. Show notes: 00:04:09 Dawn opens the interview welcoming Steve to the show and asking about his background. Dawn mentions that Steve grew up in Urbana-Champaign, Illinois, where he enjoyed basketball, Dungeons and Dragons and attempting to reinvent Decision Theory. 00:05:33 Dawn asks how Steve ended up as a computer science major rather than an economics major. 00:07:01 Dawn asks Steve if it is true that he graduated from the University of Illinois with a bachelor’s degree in computer science at the age of 19. 00:07:41 Dawn asks Steve what he did after attaining his Ph.D. 00:09:18 Ken asks Steve to describe his interest in the search for life beyond earth. 00:11:0 Ken mentions that Pascal Lee, a planetary scientist from NASA Ames Research Center, recently discussed the search for intelligent life in our galaxy on STEM-Talk, episode 121. Ken explains that the discussion centered around the Drake Equation, which was developed to produce a probabilistic estimate of the number of active, communicative extraterrestrial civilizations in the Milky Way galaxy, with Pascal’s conclusion being that the solution to the Drake Equation is likely N = 1. Ken asks Steve about his thoughts on the likelihood of intelligent life in our galaxy. 00:14:23 Dawn mentions that the Perseverance rover is currently maneuvering across the surface of Mars. She asks Steve, as the head of the Artificial Intelligence Group at JPL, NASA’s lead for deep-space robotic exploration, if he could talk about the work he specifically did on the Perseverance rover including the rover’s scheduling system. 00:16:38 Ken mentions that the success of the Perseverance mission so far has rekindled discussions of sending humans to Mars. Ken asks what Steve’s thoughts are on Pascal Lee’s proposal to take a measured approach to sending humans to Mars and that we should first return to the Moon. 00:18:47 Dawn asks Steve about the purpose of the 756-page report by the National Security Commission on Artificial Intelligence that Ken and Steve worked on for more two years.

Machine learning - Bayes decision theory

Machine Learning Teeta

Play Episode Listen Later Apr 24, 2021 36:13

The bird's eye view on classifiers. In this episode, we walk thru the landscape and start with linear classifiers. In the next episode, we will go thru non-linear classifiers isA.

machine learning bayes decision theory

#007: Lessons in Risk, Game Theory, and Winning at Marketing with Bill King

The Long Game

Play Episode Listen Later Feb 2, 2021 111:51

Bill is the Director of Marketing at Frase, an artificial intelligence platform that helps growth teams drive organic traffic and improve conversion rates with integrated SEO and on-site search tools. Bill is a former professional poker player turned marketer. He met Alex on Twitter and they've gotten along swimmingly since. They share interests in esoteric matters such as decision theory, game theory, experimentation, SEO & content marketing, meditation, sailing and music. They both played in bands and seriously considered music as a career. In this episode, they cover these topics and discuss Bill's career arch. Throughout this podcast there are a million gold nuggets that you can take away and use to change how you view work and decision making, especially if you're in SEO, content marketing, or any acquisition related work. Enjoy!The Long Game is hosted by Alex Birkett and David Ly Khim who co-founded Omniscient Digital to help companies ranging from early-state to scale-ups with growth strategy, SEO, and content marketing. Allie Decker, Head of Content, joins the conversation as well.Connect with Bill on his website and on TwitterConnect with Omniscient Digital on social:Twitter: @beomniscientLinkedin: Be OmniscientListen to more episodes of The Long Game podcast here: https://beomniscient.com/podcast/

director head marketing lessons winning risk seo content marketing long game experimentation growth strategies game theory frase bill king b2b content marketing how to make a podcast decision theory how to launch a podcast

#419 Christian List: Why Free Will Is Real

The Dissenter

Play Episode Listen Later Jan 18, 2021 88:49

------------------Support the channel------------ Patreon: https://www.patreon.com/thedissenter SubscribeStar: https://www.subscribestar.com/the-dissenter PayPal: paypal.me/thedissenter PayPal Subscription 1 Dollar: https://tinyurl.com/yb3acuuy PayPal Subscription 3 Dollars: https://tinyurl.com/ybn6bg9l PayPal Subscription 5 Dollars: https://tinyurl.com/ycmr9gpz PayPal Subscription 10 Dollars: https://tinyurl.com/y9r3fc9m PayPal Subscription 20 Dollars: https://tinyurl.com/y95uvkao ------------------Follow me on--------------------- Facebook: https://www.facebook.com/thedissenteryt/ Twitter: https://twitter.com/TheDissenterYT Anchor (podcast): https://anchor.fm/thedissenter RECORDED ON NOVEMBER 12th 2020. Dr. Christian List is Professor of Philosophy and Decision Theory at LMU Munich and Co-Director of the Munich Center for Mathematical Philosophy. He's the author of Why Free Will Is Real. In this episode, we talk about Why Free Will Is Real. We start with some basic questions, like what is free will, and what is an agent. We then get into the moral side of things, and talk about moral responsibility and the law. We address three major challenges do free will (the “challenge from radical materialism,” the “challenge from determinism,” and the “challenge from epiphenomenalism”). We discuss mind-brain dualism and emergentism. Finally, we ask if free will exists in other living beings, in nonliving entities, and in robots and advanced AI. -- Follow Dr. List's work: Faculty page: https://bit.ly/2IpMm9j Website: https://bit.ly/3lov8YE PhilPeople page: https://bit.ly/3ksNs1g Why Free Will Is Real: https://amzn.to/2Iu7VpD -- A HUGE THANK YOU TO MY PATRONS/SUPPORTERS: KARIN LIETZCKE, ANN BLANCHETTE, PER HELGE LARSEN, LAU GUERREIRO, JERRY MULLER, HANS FREDRIK SUNDE, BERNARDO SEIXAS, HERBERT GINTIS, RUTGER VOS, RICARDO VLADIMIRO, BO WINEGARD, CRAIG HEALY, OLAF ALEX, PHILIP KURIAN, JONATHAN VISSER, ANJAN KATTA, JAKOB KLINKBY, ADAM KESSEL, MATTHEW WHITINGBIRD, ARNAUD WOLFF, TIM HOLLOSY, HENRIK AHLENIUS, JOHN CONNORS, PAULINA BARREN, FILIP FORS CONNOLLY, DAN DEMETRIOU, ROBERT WINDHAGER, RUI INACIO, ARTHUR KOH, ZOOP, MARCO NEVES, MAX BEILBY, COLIN HOLBROOK, SUSAN PINKER, THOMAS TRUMBLE, PABLO SANTURBANO, SIMON COLUMBUS, PHIL KAVANAGH, JORGE ESPINHA, CORY CLARK, MARK BLYTH, ROBERTO INGUANZO, MIKKEL STORMYR, ERIC NEURMANN, SAMUEL ANDREEFF, FRANCIS FORDE, TIAGO NUNES, BERNARD HUGUENEY, ALEXANDER DANNBAUER, OMARI HICKSON, PHYLICIA STEVENS, FERGAL CUSSEN, YEVHEN BODRENKO, HAL HERZOG, NUNO MACHADO, DON ROSS, JOÃO ALVES DA SILVA, JONATHAN LEIBRANT, JOÃO LINHARES, OZLEM BULUT, NATHAN NGUYEN, STANTON T, SAMUEL CORREA, ERIK HAINES, MARK SMITH, J.W., JOÃO EIRA, TOM HUMMEL, SARDUS FRANCE, DAVID SLOAN WILSON, YACILA DEZA-ARAUJO, IDAN SOLON, ROMAIN ROCH, DMITRY GRIGORYEV, DIEGO LONDOÑO CORREA, TOM ROTH, AND YANICK PUNTER! A SPECIAL THANKS TO MY PRODUCERS, YZAR WEHBE, JIM FRANK, ŁUKASZ STAFINIAK, IAN GILLIGAN, SERGIU CODREANU, LUIS CAYETANO, MATTHEW LAVENDER, TOM VANEGDOM, CURTIS DIXON, BENEDIKT MUELLER, VEGA GIDEY, AND NIRUBAN BALACHANDRAN! AND TO MY EXECUTIVE PRODUCERS, MICHAL RUSIECKI, ROSEY, AND JAMES PRATT!

ai professor philosophy dollar dollars faculty co director mark smith rosey zoop mark blyth david sloan wilson don ross john connors lmu munich cory clark decision theory jerry muller susan pinker hal herzog munich center mathematical philosophy christian list nathan nguyen pablo santurbano stanton t herbert gintis craig healy max beilby jonathan leibrant jo o linhares

Ep. 3 - (ft.) Professor Alastair Wilson: The Nature of Contingency

Plato's Cave

Play Episode Listen Later Mar 20, 2020 61:21

In this episode, I spoke with Professor Alastair Wilson about his newly released book, The Nature of Contingency: Quantum Physics as Modal Realism. We spoke about contingency, counterfactuals, the multiple worlds interpretation of quantum physics, free will, and more. This was a true education for me with a great mind and a great writer, and I hope you find this conversation valuable as well. Here's any links you'll need to dive deeper: Book: https://global.oup.com/academic/product/the-nature-of-contingency-9780198846215?cc=us&lang=en& Dr. Wilson's Website: http://alastairwilson.org/ If you leanred something from this episode, please consider supporting me here: https://www.patreon.com/jordanmyers Every dollar that comes in will go towards bettering the show or towards funding my Philosophy PhD. Youtube: https://www.youtube.com/channel/UCtM5SXgyN93usom5vpRqlEQ/ You can also get in contact with me through Twitter: @JordanCMyers Or by emailing me at platoscavepodcast@gmail.com Special Guest: Alastair Wilson.

Episode 6: Angelie: Science, Creativity, and Getting it Right

TalkPOPc's Podcast

Play Episode Listen Later Nov 16, 2019 30:04

This conversation started out as a query about art and turned into a conversation about decision theory. Or more specifically about how creativity is just decision-making, much like science is. Angelie, an epidemiologist, begins with a quote from Nietzsche – that art is just leaving your mark. She talks about how children naturally want to do this. This conversation started out as a query about art and turned into a conversation about decision theory. Or more specifically about how creativity is just decision-making, much like science is. Angelie, an epidemiologist, begins with a quote from Nietzsche – that art is just leaving your mark. She talks about how children naturally want to do this. Support the show (https://www.patreon.com/talkpopc)

science creativity problem solving nietzsche getting it right decision theory

Ep 37: Josh Miller

Chilling With Charlie

Play Episode Listen Later Oct 25, 2019 42:28

Josh Miller researches Behavioural Economics, Statistical and Experimental Methods, Judgement and Decision Making, Game Theory, Decision Theory - and was a contributor to the very interesting Hot Hand Paper.Podcasts cost money to make, equipment, software and the like. Thankfully Betfair has sponsored this podcast which means I can just concentrate on getting fantastic guests!

podcasts judgement decision making game theory statistical josh miller behavioural economics decision theory

Why People Refuse to Drive Electric

Let's talk about Electric Vehicles

Play Episode Listen Later Sep 2, 2019 13:47

www.patreon.com/electricvehiclepodcast ☝Support this podcast & get all unlisted episodesWhy do people refuse to drive electric? The bias for the status quo (oil-based transportation system) has many reasons. It could be the result of a rational analysis of benefits and cost. But it could also be the result of an irrational commitment to previous decisions, or due to cognitive misperception.Contact InformationE-Mail: electric-vehicle-podcast@outlook.comWebsite: www.electric-vehicle-podcast.comTwitter: teresa_rhoferFacebook: ElectricVehiclePodcastReferences[1] H. A. Simon, Models of man; social and rational. Oxford, England: Wiley, 1957.[2] L. Festinger and J. Carlsmith, “Cognitive Consequences of Forced Compliance,” Journal of Abnormal and Social Psychology, vol. 58, pp. 203–210, 1959.[3] D. Kahneman and A. Tversky, “Prospect Theory: An Analysis of Decision under Risk,” Econometrica, vol. 47, no. 2, pp. 263–292, 1979. MusicDigital Future Technology [audiojungle]

Analytical Methods - Folge 1: Decision Theory

LMU Analytical Methods for Lawyers - Lehrstuhl für Bürgerliches Recht, Deutsches, Europ. und Int. Unternehmensrecht

Play Episode Listen Later Apr 19, 2019 230:01

Decision Theory - Fundamental concepts; Expected value; Cumulative Probabilities; Serially cumulative probabilities; Decision trees; TreeAgePro and other software for litigation risk analysis; Difficulties in building the tree; Discounting and risk preferences; Risk aversion; Sensitivity analysis; Challenges to the rationality assumption.

challenges risk decision methods expected difficulties sensitivity analytical discounting decision theory

Refutation of Putnam's Collapse of the Fact/Value Dichotomy

MCMP – Ethics and Value Theory

Play Episode Listen Later Apr 18, 2019 44:55

Eckehart Köhler (Vienna) gives a talk at the MCMP Colloquium (22 May, 2013) titled "Refutation of Putnam's Collapse of the Fact/Value Dichotomy". Abstract: In 2002, Hilary Putnam shocked philosophers with the story that value terms have “thick” meanings, where facts and values are “entangled”. (“Crime” and “cruel” are especially “thick”.) This phenomenon is easy to explain, since many professionals treat norms factually, e.g. currently “valid” price quotations, whereas a document leaves the deontic modality ambiguous. Those same professionals certainly are able to distinguish the modalities of propositions they use in their professional work for themselves! (E.g., an active legislator can distinguish those bills which he wants passed from bad bills, etc., and similarly in all professions, at least where procedures for norming exist.) Putnam entirely ignores this. Putnam even ignores Decision Theory, where he has done work. This is crucial: standard Bayesian Decision Theory absolutely requires independence of facts and values, since probability and utility must be independent — if they were not, then no one could empirically predict behavior, nor could anyone recommend optimal policy to a client. Putnam got his collapse from Quine’s collapse of the analytic/synthetic dichotomy, and (correctly!) concluded that if the latter fails, so does the former. But since probabili-ties are “orthogonal” to utilities (which we know from their measurement), “Hume’s Law” is valid; and so is the analytic/synthetic dichotomy. I discuss Morton White’s attempt to subsume analyticity under ethical value. Finally, I claim that (Dewey’s and Quine’s) Naturalism collapses once this (empirically real) sensorium for observing normative validity is acknowledged which is separate from sensory perception.

law crime collapse dewey hume dichotomy putnam naturalism refutation quine decision theory hilary putnam morton white

Why Does Google Favor Big Brands And How To Compete w/ Josh Bachynski

The Alex Berman Podcast

Play Episode Listen Later Jun 11, 2018 29:33

Josh Bachynski is a SEO specialist with over 20 years of experience. Josh delivered a TEDx talk "The Future of Google, Search and Ethics", a soon to be released documentary film on Google called "Don't Be Evil: Google's Secret War" and his own SEO related podcast "The White Hat Vs Black Hat SEO Show". Josh has PhD (ABD) and Master's Degree in Ethics and Decision Theory and a book coming out on our global political scenario and the collapse of culture entitled "The Zombies." This show is sponsored by Experiment 27. Get the discovery call script & questions template HERE. In this episode you'll learn: [01:30] 3 things to know before launching an SEO strategy [05:00] How to find keywords to target [07:25] What kind of content you need to create [08:45] How Google's algorithm can affect your strategy [13:15] The Future of Google Search and the Internet [18:09] What should you do if the whole system seems to be biased towards big brands [19:22] Why is Josh passionate about his work [23:02] How to hire a good SEO specialist [25:40] Alex's bad SEO hiring experience [26:10] Red flags when hiring a SEO agency Links mentioned: FCC fairness doctrine Josh on Twitter Josh's website White Hat vs. Black Hat Podcast joshbachynski@gmail.com Josh's Tedx talk Brought to you by Experiment 27. Find us on Youtube. If you've enjoyed the episode, please subscribe to the Digital Agency Marketing Podcast on iTunes and leave us a review for the show. Get access to our FREE Sales Courses.

google master internet future search zombies ethics experiments seo tedx degree favor compete fcc google search secret wars big brands whitehat decision theory

143. Newcomb’s Problem: Causal & Evidential Decision Theory

THUNK - Audio Interface

Play Episode Listen Later May 8, 2018 7:17

Newcomb's Problem: it's like "Is a hotdog a sandwich?," but WAY nerdier, with deep implications for rationality.

causal newcomb evidential decision theory

Episode 105: R. A. Briggs discusses epistemic decision theory

Elucidations: A University of Chicago Podcast

Play Episode Listen Later Apr 20, 2018 37:00

How do we tell what the best strategies for changing our beliefs on the basis of new evidence might be? See acast.com/privacy for privacy and opt-out information.

epistemic decision theory

Anthropic Decision Theory (Stuart Armstrong)

EARadio

Play Episode Listen Later Sep 25, 2015 28:55

Source: Future of Humanity Institute (original video).

anthropic humanity institute stuart armstrong decision theory

024: Greg Davies on Behavioral Finance and Controlling Your Emotions When Making Trading Decisions

Economic Rockstar

Play Episode Listen Later Mar 18, 2015 50:19

Greg Davies is Managing Director and Head of Behavioural Finance at Barclays. He joined the firm in December 2006 to develop and implement commercial applications drawing on behavioural portfolio theory, the psychology of judgment and decision making, and decision sciences. Today Greg leads a global team of behavioural and quantitative finance specialists, and is responsible for the design and global implementation of Barclays’ Investment Philosophy. Greg is an Associate Fellow at Oxford University’s Saïd Business School and his first (co-authored) book, ‘Behavioral Investment Management: An Efficient Alternative to Modern Portfolio Theory’, was published in January 2012. He is co-curator and co-creator of Open Outcry - a reality opera based on the stock market trading floor. Greg has authored papers in multiple academic disciplines, presents at academic and industry conferences, and is a frequent media commentator on Behavioural Finance. He is an Editorial Board Member of the Journal of Behavioural and Experimental Finance. Greg studied at the University of Cape Town and obtained a degree in Economics, Philosophy and Finance. He followed this with an MPhil in Economics and a PhD in Decision Theory and Behavioural Finance from the University of Cambridge. Find out: what is Behavioral Economics/Finance the disconnection between economics and psychology. how Kahneman and Tversky were ‘swimming up-stream’ to bring common sense to economics. why viewing the world through biases is harmful to behavioral finance. why the ever-increasing list of biases may not be good for the behavioral finance field. about System 1 and System 2 as popularised by Daniel Kahneman. why it’s good to allow emotions to part of the portfolio decision-making process. how to acquire the emotional comfort you need for your long-term financial objectives. how to buy emotional insurance for your long-term investment portfolio. how to avoid costly short-term emotional mistakes. how psychometric tests can extract measures of financial personality. why a set of nudges are designed to help high net-worth individuals to make better decisions. how to build a tailored portfolio to meet your clients needs. why you should consider including expected anxiety into your portfolio building along with risk and return. what an opera experiment has to do with replicating the open outcry system of a trading floor. how music can control your emotions while trading markets. how Barclays Capital are improving the understanding of their clients by turning the lens on themselves. To access the links mentioned in this episode, visit www.economicrockstar.com/gregdavies To be in with a chance to win this weeks prize, visit www.economicrockstar.com/giveaway

Constructive Decision Theory

iraq savage cornell constructive decision theory

Play Episode Listen Later Sep 18, 2012 85:47

Joe Halpern (Cornell University) gives a talk at the MCMP Colloquium (11 July, 2012) titled "Constructive Decision Theory" (joint work with Larry Blume and David Easley, Cornell). Abstract: The standard approach in decision theory (going back to Savage) is to place a preference order on acts, where an act is a function from states to outcomes. If the preference order satisfies appropriate postulates, then the decision maker can be viewed as acting as if he has a probability on states and a utility function on outcomes, and is maximizing expected utility. This framework implicitly assumes that the decision maker knows what the states and outcomes are. That isn't reasonable in a complex situation. For example, in trying to decide whether or not to attack Iraq, what are the states and what are the outcomes? We redo Savage viewing acts essentially as syntactic programs. We don't need to assume either states or outcomes. However, among other things, we can get representation theorems in the spirit of Savage's theorems; for Savage, the agent's probability and utility are subjective; for us, in addition to the probability and utility being subjective, so is the state space and the outcome space. I discuss the benefits, both conceptual and pragmatic, of this approach. As I show, among other things, it provides an elegant solution to framing problems. This is joint work with Larry Blume and David Easley. No prior knowledge of Savage's work is assumed.

The Theory of Probability Cores in Bayesian Epistemology and Decision Theory

theory probability cores epistemology bayesian decision theory

Play Episode Listen Later Sep 15, 2012 44:16

Arthur Paul Pedersen (CMU) gives a talk at the 9th Formal Epistemology Workshop (Munich, May 29–June 2, 2012) titled "The Theory of Probability Cores in Bayesian Epistemology and Decision Theory".

Tutorial Decision Theory II: Conditionalization

tutorials decision theory

Play Episode Listen Later Jul 31, 2012 51:35

Richard Pettigrew (Bristol) gives part II of his tutorial in decision theory (28 June 2012) titled "Conditionalization".

Tutorial Decision Theory I: Decision theory in epistemology